Systems and methods for real-time visual guidance during surgical procedures

Abstract

Systems and methods for real-time visual guidance during surgical procedures are provided. The systems and methods may involve the use of electromagnetic sensors that are provided on and / or within patient body part, surgical instruments, surgical implants, and / or any other types of objects that are desired to be tracked during a surgical procedure. The electromagnetic sensors may be provided within a generated electromagnetic field such that geometric and spatial information about the patient body part, surgical instruments, surgical implants, etc. may be tracked during the procedure. This data may be co-registered with a 3D model of the patient's body part such that the 3D model may be updated in real-time during the procedure. The 3D model may be augmented with other visual information to guide a surgeon during the procedure.

Description

SYSTEMS AND METHODS FOR REAL-TIME VISUAL GUIDANCE DURING SURGICAL PROCEDURESCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to and benefit of U.S. provisional patent application nos. 63 / 661,671 filed June 19, 2024, and U.S. provisional patent application 63 / 696,102 filed September 18, 2024, both of which are herein incorporated by reference.BACKGROUND

[0002] In a conventional bunion correction procedure, there are several steps that need to be performed accurately for the procedure to be correctly performed. For example, an osteotomy should be performed in the correct region of a bone fragment so that bone fragment translation may be appropriately performed. Additionally, a wire jig may be used to place guide wires within the foot prior to screw implantation, and the alignment of the wire jig relative to the foot ensures that the guide wires are correctly aligned within the foot. Oftentimes, performing these actions takes multiple attempts by the surgeon, which increases the time of the surgical procedure. In some cases, even after multiple attempts, these steps may still be performed with some degree of error, resulting in complications in the procedure and an undesired result. Further, radiographs are typically used to capture imaging during the procedure, and these radiographs expose the patient to a degree of radiation. Lengthening the procedure with these multiple attempts also increases the amount of radiation that the patient is exposed to during the procedure. Similar challenges are also applicable to other types of foot and ankle surgeries, as well as other ty pes of surgeries in general.BRIEF DESCRIPTION OF THE DRAWINGS

[0003] The detailed description is set forth with reference to the accompanying drawings. The drawings are provided for purposes of illustration only and merely depict example embodiments of the disclosure. The drawings are provided to facilitate understanding of the disclosure and shall not be deemed to limit the breadth, scope, or applicability' of the disclosure. In the drawings, the left-most digit(s) of a reference numeral may identity' the drawing in w hich the reference numeral first appears. The use of the same reference numerals indicates similar, but not necessarily the same or identical components.However, different reference numerals may be used to identify similar components as well. Various embodiments may utilize elements or components other than those illustrated in the drawings, and some elements and / or components may not be present in various embodiments. The use of singular terminology to describe a component or element may depend on the context, encompass a plural number of such components or elements and vice versa.

[0004] FIGS. 1 A-1B depict an exemplary use case for real-time visual guidance during surgical procedures, in accordance with one or more example embodiments of the disclosure.

[0005] FIG. 2 depicts a process for generating and modifying a real-time three- dimensional visual model of a patient’s body part to provide visual guidance during a surgical procedure, in accordance with one or more example embodiments of the disclosure.

[0006] FIG. 3 depicts an exemplary surgical procedure to which the real-time visual guidance may be applied, in accordance with one or more example embodiments of the disclosure.

[0007] FIG. 4 depicts another exemplary surgical procedure to which the real-time visual guidance may be applied, in accordance with one or more example embodiments of the disclosure.

[0008] FIG. 5 depicts a visualization of surgical instrument tracking, in accordance with one or more example embodiments of the disclosure.

[0009] FIG. 6 depicts another visualization of surgical instrument tracking, in accordance with one or more example embodiments of the disclosure.

[0010] FIG. 7 depicts another visualization of surgical instrument tracking, in accordance with one or more example embodiments of the disclosure.

[0011] FIGS. 8-10 depict alignment visualization during a surgical procedure, in accordance with one or more example embodiments of the disclosure.

[0012] FIG. 11 depicts a process for generating a three-dimensional model, in accordance with one or more example embodiments of the disclosure.

[0013] FIG. 12 depicts an example of data co-registration with fiducial markers and a three-dimensional model, in accordance with one or more example embodiments of the disclosure.

[0014] FIG. 13A depicts a heel cup, in accordance with one or more example embodiments of the disclosure.

[0015] FIG. 13B depicts exemplary fiducial markers, in accordance with one or more example embodiments of the disclosure.

[0016] FIG. 14A depicts an exemplary system including a guide wire with an embedded sensor, in accordance with one or more example embodiments of the disclosure.

[0017] FIG. 14B depicts a close-up view of the slip ring of FIG. 14 A. in accordance with one or more example embodiments of the disclosure.

[0018] FIG. 15 A depicts a close-up view of an exemplary' guide wire with an embedded sensor, in accordance with one or more example embodiments of the disclosure.

[0019] FIG. 15B depicts a guide wire connection, in accordance with one or more example embodiments of the disclosure.

[0020] FIGS. 15C-15E depict close-up views of another exemplary guide wire with an embedded sensor, in accordance with one or more example embodiments of the disclosure.

[0021] FIG. 16A depicts an exemplary guide wire and an anti-rotation sensor encasement, in accordance with one or more example embodiments of the disclosure.

[0022] FIG. 16B depicts a close-up view of the encasement of FIG. 16A, in accordance with one or more example embodiments of the disclosure.

[0023] FIG. 16C depicts a close-up view of the anti-rotation arms of the encasement of FIGS. 16A-16B, in accordance with one or more example embodiments of the disclosure.

[0024] FIG. 16D depicts the anti -rotation sensor encasement of FIG. 16A attached to a drill, in accordance with one or more example embodiments of the disclosure.

[0025] FIG. 16E depicts another exemplary' guide yvire and an anti-rotation sensor encasement, in accordance with one or more example embodiments of the disclosure.

[0026] FIG. 16F depicts a close-up view of the encasement of FIG. 16E, in accordance with one or more example embodiments of the disclosure.

[0027] FIGS. 17A-17C depict exemplary' usage of the surgical instruments as described herein, in accordance with one or more example embodiments of the disclosure

[0028] FIGS. 18A-18D depict exemplary' two-dimensional (2D) and three-dimensional (3D) visualizations, in accordance yvith one or more example embodiments of the disclosure.

[0029] FIG. 19A depicts another exemplary' system including various components with embedded sensors, in accordance with one or more example embodiments of the disclosure.

[0030] FIG. 19B depicts close-up views of the various components with embedded sensors of FIG. 18A, in accordance with one or more example embodiments of the disclosure.

[0031] FIG. 19C depicts a close-up view of an attachment mechanism for a sensor, in accordance with one or more example embodiments of the disclosure.

[0032] FIG. 19D depicts additional mechanisms for housing a sensor, in accordance with one or more example embodiments of the disclosure.

[0033] FIG. 20 depicts a method for real-time visual guidance during surgical procedures, in accordance with one or more example embodiments of the disclosure.

[0034] FIG. 21 depicts an exemplary system for real-time visual guidance during surgical procedures, in accordance with one or more example embodiments of the disclosure.

[0035] FIG. 22 depicts an example computing system, in accordance with one or more example embodiments of the disclosure.DETAILED DESCRIPTION

[0036] This disclosure relates to, among other things, systems and methods for real-time visual guidance during surgical procedures. Particularly, the system leverages electromagnetic tracking (e.g., tracking the positions of electromagnetic sensors within a generated electromagnetic field) in combination with other imaging modalities to provide real-time three-dimensional (3D) visualizations during surgical procedures. This allows for minimally invasive guidance during such procedures. The system is also particularly advantageous because it allows for more effective tracking accuracy (for example, millimeter level tracking) without necessitating the use of a larger profile sensor (described in further detail below).

[0037] While conventional approaches exist for providing visual information during surgical procedures, these conventional approaches typically involve the use of radiographs. However, radiographs do not provide real-time information (even if the radiographs are performed in short time intervals, this information is still not real-time information). Instead, a surgeon performs a task, a radiograph is captured, and then the surgeon determines from the radiograph if the task needs to be attempted again (resulting in an inefficient process).

[0038] Additionally, each radiograph may expose the patient, surgeons, and / or any other individuals involved in a procedure to some ionizing radiation. Accordingly, more frequent updates to the visual information necessitate more frequent radiographs, which results in the patient and / or any other individuals being exposed to increasing amounts of radiation (leading to a trade-off between imaging frequency and radiation exposure). In contrast with this conventional imaging approach, the approach described herein provides actual real-time 3D visual information without exposing any participants in the procedure to the same amount of potentially harmful radiation (non-ionizing radiation may be used).

[0039] One prominent advantage of this improved system for real-time visual guidance is for tasks performed in foot and ankle orthopedic procedures. Specifically, the system allows for idealized implant placement without requiring additional steps and / or instrumentation. Furthermore, large tracking devices or optical tracking systems are not required. This is a significant benefit in such procedures, given that feet and ankles are much smaller relative to other body parts, which results in technical challenges and increased exposure to bones that may otherwise result in surgical complications. It should be noted that while reference is made herein to foot and ankle procedures, this is not intended to be limiting, and the systems and methods may also be applicable to any other type of surgical procedure as well. Accordingly, any reference to a foot and / or ankle procedure may be replaced by any other type of procedure.

[0040] In embodiments, the generation of the 3D visual guidance may involve a two- step process. In the first step, initial imaging is performed on the foot (as an example) of the patient. For example, one or more computer tomography (CT) scans or x-ray images of the patient’s foot may be captured. Specifically, bi-planar imaging may be performed, meaning that images at multiple different angles of the patient's foot and / or ankle may be taken (however, bi-planar imaging may not necessarily be required depending on the imaging device being used). The result of this initial imaging step may be the generation of a 3D model of the patient’s foot. This initial step may be performed prior to the initiation of the surgical procedure itself (or may occur as a first step of the surgical procedure).

[0041] In the second step (which occurs during the surgical procedure itself), a localized electromagnetic field (or multiple electromagnetic fields) is generated within a ‘’navigation volume” (for example, the region of the patient that is the focus of the procedure). Electromagnetic sensors may be provided at various locations within the navigation volume to produce real-time tracking data during the procedure. This tracking data may includegeometrical and spatial information as well as other types of relevant information that may be used by the surgeon during the procedure. For example, during bunion correction surgery, electromagnetic sensors may be provided on or proximate to the bones within the foot of the patient (such as on the skin of the foot, on the bones themselves, etc.) such that movement of the bones during the procedure may be tracked and visualized in real-time. The electromagnetic sensors may also be used to track the soft tissue of the patient (e.g., ligaments, tendons, etc.). Data relating to these types of tissues may be beneficial in procedures, such as total ankle procedures. Any other parts of the patient’s body may also be tracked using the sensors (depending on the type of procedure).

[0042] The electromagnetic sensors may also be used to track other types of objects within the navigation volume. For example, a sensor or sensors may be provided on and / or within surgical instruments that are used during the surgical procedure, such as a burr, guide wire jig. etc. These sensor(s) may be used to track the positioning of the surgical instruments within the navigation volume to allow- the surgeon to accurately align the instruments to perform tasks during the procedure. As another example, a sensor or sensors may also be provided on surgical implants that are provided within the patient’s body (e.g., guide wires, screws, etc.) to ensure that the surgical implants are provided in the correct locations. The sensor(s) may also be provided to track any other types of objects involved in the procedure.

[0043] One or more fiducial markers may also be provided on and / or in the foot and / or ankle of the patient during this initial imaging step. A fiducial marker may be an object that is provided at a fixed location and serves as a fixed frame of reference for the electromagnetic tracking that is performed during the second step (during the surgical procedure itself). That is, the location of a fiducial marker is captured during the initial imaging stage and the location may then be compared to detected locations of electromagnetic sensors during the surgical procedure to aid in tracking the location of the sensors within the generated electromagnetic field.

[0044] In some embodiments, a fiducial marker may be an electromagnetic sensor that is provided at the fixed location during the initial imaging and remains in the fixed location or is provided back in the same location during the procedure. That is, one (or multiple) of the sensors may serve as a reference sensor, and the remaining sensors may serve as tracking sensors. In some embodiments, the tracking sensors themselves may serve a dual purpose as the fiducial markers (for example, the sensors may include a radio opaque portion or may be generally radio opaque). The fiducial marker does not necessarily need to be a sensor,however, and may include any other type of object, such as a sticker. The fiducial marker also does not necessarily even need to be an object (for example, the fiducial marker may include a marking). Furthermore, a fiducial marker may be provided on another object (for example, an object on which a sensor is provided, surgical equipment, and / or any other type of object present within the navigation volume of the surgical procedure), which is illustrated in further detail in at least FIGS. 19A-19B. Even if the fiducial marker is initially a non-sensor object during the initial imaging stage, an electromagnetic sensor may be placed at the location of the fiducial marker during the procedure itself. Additionally, the fiducial marker does not necessarily need to be an actual object during the initial imaging step and may instead be a virtual element that is provided as an overlay in a particular spot in the generated 3D model. After the model is generated, an actual object or marker may be placed at the location of the digital overlay during the surgical procedure. Furthermore, more than one fiducial marker may be used as well.

[0045] To allow for tracking of the electromagnetic sensors to be performed without significant distortion to the electromagnetic field even while ferromagnetic and / or paramagnetic instruments (such as high-speed burrs or other ty pes of surgical instruments) are being used during the procedure, a modular electromagnetic field generator with reconfigurable magnetic field generating antennas may be used (such as described in U.S. Published Application No. 2022 / 0037085 titled ‘'Modular reconfigurable magnetic navigation system and method,” which is incorporated herein in its entirety by reference). Conventionally, electromagnetic tracking systems are heavily influenced by objects with magnetic properties, such as surgical instruments, surgical implants being inserted into the patient’s body, etc. This results in incorrect information being displayed when such objects are within the electromagnetic field. This is sub-optimal as objects with magnetic properties (such as metallic objects) are commonplace in surgical procedures, thereby limiting the usefulness of the electromagnetic tracking information captured during the procedure using conventional approaches.

[0046] To reduce the impact of objects with magnetic properties on electromagnetic tracking, a reconfigurable antenna array including individual antenna elements may be used, with each antenna element creating its own magnetic field with accurate, environmentally controlled electrical currents. Multiple of these antenna elements may be used to generate the electromagnetic fields in which the electromagnetic sensors may be localized. This provides modularity in the tracking system in that the navigation volume may bereconfigured on demand and in real-time. Further, using sensors at fixed locations within the navigation volume and using magnetic field-generating antennas as sensors allows an additional level of electromagnetic interference detection and mitigation.

[0047] The antenna coils used for the generation of the electromagnetic field may be formed as printed circuits or attached to different layers that are coupled together. These antennas may also, in some instances, simultaneously serve as sensors to detect the electromagnetic field. Multi-layer printed circuit boards (PCBs) (which may be the antenna "elements") can be used with specifically chosen shapes of traces on individual layers to increase the magnetic field strength and / or shape the magnetic field.

[0048] To perform interference detection, for a system including a given number of antenna elements, each antenna may monitor the electromagnetic fields from the remaining antenna elements. For example, in an array including three antennas, the center antenna element may generate the electromagnetic field, which may be sensed by the other two antenna elements. Initially, an antenna element may detect an electromagnetic field of a first value. When a disturbance is introduced into the navigation volume (as a non-limiting example, a metallic surgical instrument is introduced into the navigation volume), the antenna element detects a change in the electromagnetic field, indicating that interference has been detected.

[0049] While the antennas and antenna elements may be primarily used to generate the electromagnetic fields used for tracking, the antennas and antenna elements may also simultaneously be used to monitor electromagnetic fields produced by other antennas and antenna elements. Antenna pods will primarily be used to generate magnetic fields within the navigation volume, but they can also be used to simultaneously monitor fields from the other antenna pods in the system. The detected signals may be digitized and demodulated by circuitry into a pickup proportional to the amplitude of the sensed waveform. This pickup can be measured for normal conditions where no interference is present.

[0050] Due to the interference, a signal sensed by a first antenna as a response to the field generated by a second antenna is different, as the field has been distorted. The signal sensed by a third antenna as a response to the field generated by a second antenna is unchanged. In this example, the field sensed by the first antenna has a larger amplitude than before and therefore produces a different pickup than the pickup produced in normal operation. This difference in pickups may be used as an additional interference detection method as well as a warning mechanism for users. Also, in combination with the knowntheoretical magnetic field pattern that would be produced by an antenna element, the magnetic field map may be adjusted to aid in compensating for these disturbance effects.

[0051] During the procedure, as data is captured from the electromagnetic sensors within the generated electromagnetic field, the data may be co-registered with the 3D model generated during the initial imaging to modify the appearance of the 3D model to match the actual appearance of the foot of the patient (or any other body part that is the subject of the surgical procedure). For example, a translation of a bone fragment may be tracked based on the movement of an electromagnetic sensor associated with that bone fragment. The same bone fragment in the 3D model may then also be translated to provide a visual indication of the translation.

[0052] Additionally, data captured from sensors associated with any surgical instruments, surgical implants, etc. may also be co-registered with the 3D model based on the location of the surgical instruments, surgical implants, etc. within the navigation volume relative to the foot. The 3D model (as well as any other visual guidance information) may be presented to the surgeon via a user interface of a display device, such as a computer monitor, tablet, or television that is provided within the operating room. Accordingly, the surgeon may view the display device to determine the positioning of the surgical instruments, surgical implants, etc. within the foot based on their presented locations relative to the 3D model.

[0053] In addition to presenting visual information corresponding to actual objects that are present during the procedure (e.g., the patient’s foot, the surgical instruments and implants, etc.), other types of visual information may also be generated and presented. This information may provide further guidance to the surgeon while the surgeon is performing tasks during the procedure. As one example (shown in further detail in FIG. 5), the surgeon may be using a burr to perform a cut within a region of bone of the patient. The user interface may allow the surgeon to accurately track the positioning of the burr within the patient’s body through the virtual representation of the burr. Other visual information may also be generated and overlaid over the patient’s bone on the user interface to provide guidance to the surgeon. For example, a colored box may be provided over the bone to indicate the region of the bone that needs to be cut using the bun. Another color may be used to represent the region of the bone that has already been cut by the burr.

[0054] As another example (shown in further detail in at least FIGS. 8-10), the visual information may also include virtual lines indicative of a desired alignment of guide wireswithin the foot of the patient and other virtual lines indicative of the projected actual alignment of the guide wires (for example, based on the current orientation of a wire jig). This visual information may allow the surgeon to accurately align the guide wires before placing them within the patient’s foot so that the placement is performed accurately at the first attempt. These are merely two non-limiting examples of types of visual information that may be presented to guide the surgeon, and any other types of information may also be presented.

[0055] In some instances, accurate geometric, spatial, and other types of information may be produced regardless of the location of the sensor or sensors. As an example, it may be desired to track the position of the tip of a burr that is used during a procedure. Even if a sensor is provided at an offset from the tip of the burr (such as in the handle), the location of the tip may still be accurately tracked. This may be accomplished by using a predetermined offset value indicating a distance between the location of the offset sensor and the location of the tip of the burr. If the amount of offset from the desired location is known, then the offset may be applied to the data received from the electromagnetic sensor to track the location of the tip even if the electromagnetic sensor is not provided at the tip.

[0056] In addition to providing this visual geometric and spatial information, more specific guidance may also be presented to the surgeon via the user interface. Continuing the scenario where the surgeon is using the burr to cut a region of the patient’s bone, text may also be presented on the user interface to provide further guidance. For example, the text may state “tilt the burr by five degrees,” “proceed to cut 5 mm into the bone,” or any other type of instruction that may be used to guide the surgeon while performing the cut. This guidance may also be provided via any other suitable mechanism, such as auditory instructions provided through a speaker of a device within the operating room.

[0057] These real-time 3D visualizations allow for surgical procedures to be performed quicker, in a predictable manner, while significantly reducing the amount of radiation exposure for the patient. For example, a bunion correction may be performed in under 15 minutes, within millimeter accuracy, and while exposing the patient to 80% less radiation. The approach allows for the surgery to be performed without requiring multiple trial-and- error attempts to accurately manipulate the bones, instrumentation, implants, etc. within the small, confined space that is the focus of the operation (e.g., foot, etc.). In a conventional procedure, performing accurate actions may be difficult w ithin such a confined space and the surgeon may not be able to perform the procedure without performing the same actionsmultiple times (or taking an extended period to ensure that the instruments are accurately aligned to allow the procedure to be performed correctly). Even so, the results of the actions may not be perfectly accurate as it may be difficult for the surgeon to accurately perform the actions within the small space at all. Therefore, the systems and methods described herein improve the accuracy of the procedure as well as the amount of time it takes to complete the procedure.

[0058] This disclosure also specifically relates to surgical instruments including embedded sensor(s). An embedded sensor may capture real-time data useful during a surgical procedure, such as a foot and / or ankle procedure (or any other type of surgical procedure). In some embodiments, a sensor may advantageously be provided at and / or within the “tip’' of the surgical instrument and is configured to allow the sensor to rotate with the surgical instrument (or allow the surgical instrument to rotate relative to the sensor), while still allowing the sensor to transmit data along a wired connection during the rotation of the surgical instrument. While reference is made throughout to wired connections, it should be noted that any of these wired connections may also be replaced with wireless connections. That is, any of these sensors may be wireless sensors that may transmit data using any suitable wireless communication protocol (including any short-range and / or long- range wireless communication protocols, such as Bluetooth, Wi-Fi, etc.) and therefore do not require hardwired connections.

[0059] This configuration described herein is particularly advantageous because it allows for a high degree of accuracy in measurements (such as millimeter-level accuracy or an even greater level of accuracy) while still using a smaller profile sensor (for example, the height, width, and length of the sensor may all be less than 100 mm, however, other sizes are also possible). Typically, there is a tradeoff between sensor size and accuracy (that is, a smaller sensor allows for a smaller form factor but may sacrifice accuracy, whereas a larger sensor may be more accurate but takes up additional space in the system). This high accuracy is possible without requiring a larger sensor package based on the implementation of a specialized antenna array with the sensor, as is described in further detail below.

[0060] Particularly, in some embodiments, the surgical instrument may be a guide wire. A guide wire may be a thin wire (that may be flexible or rigid / stiff) used to guide the placement of larger surgical instruments or implants. The guide wire may serve as a pathway or guide for the accurate insertion of devices such as catheters, stents, or screws. The guide wire may more specifically be a k-wire, however, other t pes of guide wires may also beapplicable. While reference is made to a single guide wire, this is not intended to be limiting and multiple of such guide wires may be used simultaneously during a surgical procedure. While reference is made herein to a sensor provided on and / or within a guide wire, this is merely for illustrative purposes and the sensor may also be provided on any other surgical equipment.

[0061] In one or more embodiments, the guide wire may be hollow and thus include an internal cavity in which the sensor may be provided (close-up views of a sensor that is positioned inside a guide wire are illustrated in FIGS. 15 A, 15C-15E, etc.). The guide wire may include a proximal end and a distal end (the ‘'tip”). The distal end may be the end of the guide wire that first comes into contact with the patient during the surgical procedure. That is, the proximal end is the end of the guide wire that is more “proximal” to the surgeon during the surgical procedure (relative to the distal end). The proximal end may be located between the computing system that processes signals from the sensor and the distal end. In some instances, a material (for example, a resin, epoxy, metal, and / or any other type of material) may be provided that serves as a barrier between the sensor and biological fluid / tissue so as to not elicit an adverse host response and / or protect the sensor from damage during insertion.

[0062] The guide wire with the sensor at its tip is advantageous for several reasons. First, it may be more desirable for real-time information about the current location of the tip of the guide wire to be captured as opposed to other portions of the guide wire (however, data corresponding to other portions of the guide wire may still be valuable). Precise positioning of the guide wire proximate to the patient’s body and within the patient’s body (for example, through the foot of the patient if the surgical procedure is a bunion correction) is important because the guide wire serves as a reference for other elements that are inserted into the patient (such as screws, for example). Given that the progression of the guide wire through the patient’s body begins with the tip of the guide wire, it is important that this portion of the guide wire in particular is tracked (although other portions of the guide wire may also be tracked simultaneously).

[0063] Second, the guide wire may potentially be bent or experience other types of temporary deformation during use. However, the guide wire is less likely to bend and interfere with the sensor at the tip of the guide wire than in the middle (or another portion) of the guide wire. Therefore, providing the sensor at the tip of the guide wire mitigates the likelihood that the sensor is impacted by these temporary deformations of the guide wire.Additionally, if the sensor is positioned at a distance from the tip of the guide wire, then tracking the position of the tip may require calibration of the sensor to account for the distance between the actual location of the sensor and the actual location of the tip. Providing the sensor directly at and / or within the tip removes this calibration requirement. Furthermore, positioning the sensor at the tip may reduce the amount of electromagnetic interference from other metallic components. However, there may be other advantages to providing the sensor at a distance from the tip of the guide wire, as is explained further with respect to alternative embodiments described herein (for example, FIGS. 1 A6-16F).

[0064] Given that some types of guide wires (e.g., a k-wire or other type of guide wire) may be used to drill into a bone of the patient during the surgical procedure (for example, a guide wire may be drilled into a patient’s foot during a bunion correction), the guide wire may necessarily rotate at a certain number of rotations per minute (RPM) during the drilling process. Accordingly, in some embodiments, the sensor may also rotate along with the rotation of the guide wire. The sensor may include a wired connection with a computing system (or more generally, a “processor”).

[0065] In some embodiments, to allow the wired connection to rotate along with the sensor without twisting and breaking or otherwise becoming unusable, or otherwise impacting the operation of the guide wire or other devices connected to the guide wire, a mechanism may be provided that allow s the wired connection. The mechanism allow s the wired connection to rotate while still transmitting signals between the sensor and the computing system that processes the signals.

[0066] One example of such a mechanism is a slip ring. A slip ring is an electromechanical device that allow s the transmission of power and electrical signals from a stationary to a rotating structure. A slip ring typically includes two main parts: the rotor (the rotating component) and the stator (the stationary component. The rotor may include conductive rings, while the stator may include brushes or contact points that maintain electrical contact with the rings. The rotor may have one or more concentric conductive rings mounted on it. These rings may be made of conductive materials such as copper or gold, which provide effective electrical conductivity’ and wear resistance. The stator may have brushes or contact points that press against the conductive rings on the rotor. As the rotor rotates, the brushes maintain continuous contact with the conductive rings. This allows electrical signals and power to be transmitted from the stationary stator to the rotating rotor without interruption.

[0067] A slip ring is merely one example of such a mechanism and any other suitable mechanism that allows two wired connections to transmit data while one connection rotates relative to the other may also be used.

[0068] While in some embodiments the sensor may rotate with the wired connection, in other embodiments, the sensor and the wired connection extending from the sensor to the computing system may remain fixed and the guide wire may rotate relative to the sensor and the wired connection. In yet further embodiments, the sensor may include an outer shell and an inner portion, and the outer shell may be configured to rotate with the guide wire relative to the inner portion. The wired connection may be provided between the inner portion and the computing system. That is, the wired connection remains stationary in such embodiments.

[0069] As mentioned above, the data captured by the embedded sensor may be in combination with other imaging modalities to provide real-time three-dimensional (3D) visualizations during surgical procedures (although reference is made throughout to 3D visualizations, 2D visualizations, or a combination of 2D and 3D visualizations, are also possible and any reference to 3D visualizations is not intended to be limiting). Non-limiting examples of such visualizations are shown in at least FIGS. 18A-18D.

[0070] Although reference is made herein to a single sensor provided within the guide wire, this is not intended to be limiting. That is, although it is important that one sensor is provided at the "tip " of the guide wire, there may also be one or more additional sensors provided at various other positions within the guide wire as well. Additionally, although reference is made specifically to a guide wire herein, such sensors may also be provided in other types of surgical instruments as well (or even other types of elements used during a surgical procedure, such as surgical implants, etc.).

[0071] Turning to the figures, FIGS. 1 A-1B depicts an exemplary use case 100 for realtime visual guidance during surgical procedures. Particularly, the use case 100 shows a scene in which a surgeon 102 is performing a bunion correction surgery on a foot 106 of a patient 104 and is viewing real-time visualizations of the procedure via monitor 114 during the procedure.

[0072] To allow' for the real-time visual guidance to be provided to the surgeon 102, one or more sensors 108 (which may be electromagnetic sensors) are disposed at various locations on and / or within the foot 106 of the patient 104. For example, the one or more sensors 108 may be provided at various intervals along (for example, on, within, orproximate to) the bones within the foot 106 that are being adjusted during the bunion correction surgery (the bunion surgery' involves adjusting the positioning of the bones to remove the bunion from the foot 106).

[0073] In one or more embodiments, some or all of the one or more sensors 108 may’ be provided on and / or within other objects, and those objects may be affixed at certain locations on and / or within the bones of the foot 106. Given that the one or more sensors 108 are disposed at known locations with respect to portions of the bones, the movement of the bones may be tracked by tracking the movement of the one or more sensors 108. For example, FIG. IB shows sensors 108 embedded within a first object 140, a second object 142, a third object 144, and a fourth object 146 (the first object 140, second object 142, and third object 144 are also generally referred to as “posts” herein given their shape). These objects are described in further detail with respect to FIGS. 19A-19C (FIG. 19A-19C shows posts 1904, 1906. 1908. and 1912).

[0074] Although FIG. IB only shows one sensor 108 embedded in each of the objects, this is merely for illustrative purposes and any other number of sensors 108 may also be provided on and / or within any of the objects. Additionally, the sensors 108 may be provided at any other positions on and / or within each of the objects and may be any shape and / or size.

[0075] The objects may be temporarily positioned at desired locations relative to the bones of the patient to track the position and movement of the bones during the surgical procedure. The objects may be configured in any number of different ways to allow the objects to be temporarily fixed in desired positions. As one example, the objects may include threaded or otherwise pointed ends that allow the objects to be embedded at least partially within the bones of the patient. As another example, the ends of the objects may be temporarily affixed to a surface of the bones using an adhesive or other ty pe of material such that the objects do not need to be embedded into the bones. As yet another example, the objects themselves may not be embedded directly into the bones but rather may be configured to receive a fastener or fasteners, and the fastener or fasteners may be embedded into the bones to secure the objects to the bones.

[0076] As mentioned in FIGS. 19A-19C, the specific objects shown in FIG. IB are merely exemplary’ types of objects and other types of objects are also possible for holding the one or more sensors 108. For example, objects of any other sizes and / or shapes may instead be used. Additionally, the specific arrangement of the objects shown in FIG. IB is also exemplary and the objects may be arranged on the bones at any other positions,depending on the surgical procedure, among other factors. Furthermore, in one or more embodiments, the sensors 108 do not necessarily need to be disposed on and / or within separate objects but may rather be disposed on and / or within the bones directly. As one nonlimiting example, a sensor 108 may be directly affixed to a bone using an adhesive or other type of material, or may be secured to the bone using any other suitable mechanism.

[0077] It is also critical that the surgeon 102 can accurately track the positioning of any surgical instruments that are used to perform the procedure. That is, in addition to tracking the movement of the bones, the sensors may also be used to track the positioning of the surgical instruments relative to the bones. Accordingly, the use case 100 shows another sensor 109 is provided on or within a burr 118 (a surgical instrument used to perform bone cuts). FIG. IB also shows that a sensor 111 may be provided at a tip of a guide wire 148 that is used during the surgical procedure. Although not shown in FIGS. 1A-1B, further sensors may also be provided on and / or within any implants that are provided within the foot 106 as part of the bunion correction, such as guide wires, screws used to keep certain bones in a desired arrangement, etc. This allows the positioning of these implants to be tracked in a similar manner. FIGS. 14A-17C describe the use of one or more sensors on and / or within the surgical equipment in greater detail.

[0078] Any of the data captured by any of the sensors may be transmitted to a computing system 112 for processing (and ultimately for presentation to the surgeon 102 via the monitor 114, which may be in communication with the computing system 112). For example, the computing system 112 may be a desktop computer or any other type of device capable of processing data. FIGS. 1 A-1B showthe one or more electromagnetic sensors 108 and the sensor 109 in communication with the computing system 112 (any of the other sensors, including sensors 111 may also be in communication with the computing system 112) via one or more wires, however, the sensors may also be in wireless communication with the computing system 112 as well.

[0079] The sensors may be any number of different types of electromagnetic sensors. The sensors may be appropriately sized such that they can be provided on and / or within the foot 106 of the patient 104, surgical instruments, surgical implants, etc. Additionally, the sensors may have multiple degrees of freedom, so only one sensor may be required to obtain data in the x, y, and z axes, as well as pitch and yaw information.

[0080] As aforementioned, the sensors do not necessarily need to be provided at the precise locations that are desired to be tracked. For example, if it is desired to track thelocation of a tip of the burr 1 18 (and / or any other type of surgical equipment), this information may still be obtained even if the sensor 109 is provided in a handle of the bun- 118 instead of the tip. If the amount of offset from the desired location is known, then the offset may be applied to the data received from the sensor 109 to obtain the desired location information even if the sensor 109 is not provided in the tip of the burr 118.

[0081] To allow for any of the sensors to be used to track geometrical and spatial information about the foot 106, surgical instruments, surgical implants, etc., an electromagnetic field generator 130 may also be provided to generate one or more electromagnetic fields in the navigation volume 132. In this example, the navigation volume 132 is a region including the foot 106 of the patient. As aforementioned, a reconfigurable antenna array including individual antenna elements may be used, with each antenna element creating its own magnetic field with accurate environmentally controlled electrical currents. Multiple of these antenna elements may be used to create the electromagnetic fields in which the electromagnetic sensors may be localized. The antenna coils used for the generation of the electromagnetic field may be formed as printed circuits or attached to different layers that are coupled together. These antennas may also, in some instances, simultaneously sene as sensors to detect the electromagnetic field. Multi-layer printed circuit boards (PCBs) (which may be the antenna “elements’’) can be used with specifically chosen shapes of traces on individual layers in order to increase the magnetic field strength and / or shape the magnetic field.

[0082] This is merely one example of an electromagnetic field generator that may be used and any other type of electromagnetic field generator may also be used.

[0083] Any of the data captured by the sensors may be co-registered with a 3D model of the foot 106 generated in the first step (the initial imaging) of the two-step process described above. For example, a CT scan or an x-ray of the foot 106 may previously have been captured to produce a 3D model of the foot 106. The data captured during the surgical procedure using the sensors may then be co-registered with the 3D model to provide the real-time 3D visual information to the surgeon 102.

[0084] As one example of information that may be presented to the surgeon 102, the monitor 114 is shown as displaying real-time visualizations of the progress of a cut that is being performed on a bone in the foot 106 using the burr 118. A visual element 120 representing the position of the burr 118 is show n on the monitor 114 such that the surgeon 102 can accurately track the movement of the bun 118 within the foot 106 in real-time. Asecond visual element 122 indicates regions of the bone that have already been cut by the burr 118. A third visual element 114 (a boxed region show n within the bone) indicates the total region of the bone that needs to be cut by the burr 118. In this manner, the surgeon 102 may perform an accurate cut of the bone using the burr 118 by tracking the progress of the cut in real-time via the monitor 114.

[0085] FIG. 2 illustrates an exemplary process for generating and modifying a real-time 3D visual model of a body part of a patient to provide visual guidance during a surgical procedure on the body part. That is, the two-step process for generating the real-time 3D model is illustrated. As aforementioned, the two-step process may begin with a first step 202 in which initial imaging is performed by an imaging device (such as imaging device 1608 or any other imaging device described herein or otherwise). FIG. 2 shows a CT scan being performed on the foot of a patient. More specifically, a CT scan may be performed such that images of the foot are captured at multiple angles. By performing the scan, a 3D model of the patient’s foot may be generated using the images at the different angles. In some instances, depending on the type of imaging device, bi-planar imaging may be used. While the example shown in FIG. 2 illustrates the use of a CT scanning device, any other type of imaging device may be used to generate the 3D model of the foot as well (e.g., an X-ray machine, etc.). One or more fiducial markers (described in further detail elsewhere herein) may also be provided on and / or within the foot of the patient during the first step 202. For example, an object serving as the fiducial marker may be provided on the foot of the patient before the CT scans are performed. The first step may be performed prior to the surgical procedure or intra-procedure, in some instances.

[0086] Once the initiating imaging is performed, the second step 204 (which occurs during the surgical procedure itself) may involve performing electromagnetic tracking such that the 3D model 220 generated in the first step 202 may be updated in real-time to provide visual guidance to the surgeon during the surgical procedure. In other words, the first step 202 generates the 3D model of the patient’s foot, and the second step 204 adjusts the 3D model 220 during the procedure as the patient’s foot is adjusted, surgical instruments and surgical implants are introduced into the navigation volume, etc.

[0087] This electromagnetic tracking may be performed by generating a localized electromagnetic field (or multiple of such electromagnetic fields) within the navigation volume of the procedure using an electromagnetic field generator (e.g., electromagnetic field generator 1612 or any other electromagnetic field generator described herein orotherwise). In the example shown in FIG. 2, the electromagnetic field may be generated at the location of the patient’s foot during the bunion correction procedure. Additionally, electromagnetic sensors (e.g., electromagnetic sensors 108, 109, 206, 1102, 1104, 1106, 1408, 1508. 1538, 1608, 1644. 1930, 2112, and / or any other sensors described herein or otherwise) may be disposed at various locations within the navigation volume to produce real-time tracking data during the procedure.

[0088] This tracking data may include geometrical and spatial information as well as other types of relevant information that may be used by the surgeon during the procedure. For example, during bunion correction surgery, electromagnetic sensors (such as electromagnetic sensors 206, 208, 210, etc.) may be provided on or proximate to the bones (e.g., on the skin of the foot, on the bones themselves, etc.) on and / or within the foot of the patient such that movement of the bones during the procedure may be tracked and visualized in real-time via the 3D model 220 of the patient’s foot.

[0089] FIG. 2 also shows that, in addition to the 3D model 220 being adjusted based on the data captured from the electromagnetic sensors, the 3D model 220 may also be augmented with other visual elements based on data captured from sensors associated with surgical instruments and implants that are used during the surgical procedure. For example, the figure shows a virtual bounding box 222, a virtual dotted line 224, and a virtual line 226 that are provided as visual elements to guide the surgeon through a portion of the bunion correction procedure (shown in additional detail in FIGS. 8-9). For example, the virtual bounding box 222 provides an indication of a location to which a bone fragment should be translated during the procedure, the virtual dotted line 224 provides an indication of the desired trajectory of a guide wire to be inserted into the patient’s foot, and the virtual line 226 provides an indication of the current actual trajectory of the guide wire. This additional visual guidance allows the surgeon to more accurately perform the procedure while eliminating trial-and-error or other less efficient techniques.

[0090] The visual elements shown in FIG. 2 are merely exemplary and the 3D model 220 may also be enhanced with any other types of visual guidance depending on the actions required during the current stage of the procedure. As another non-limiting example. FIG. 5 (described in further detail below) shows that electromagnetic sensors may also be placed on and / or within surgical instruments such that the movement of the surgical instruments within the foot may be accurately tracked in real-time as the surgical instruments are used during the procedure. The 3D model 220 may also be augmented with other visual elementsto indicate to the surgeon how the surgical instruments should be used during the procedure (for example highlighting regions of bone that need to be cut using a burr, etc.).

[0091] While reference is mere herein to a 3D model of a body part of a patient, this is not necessarily intended to be limiting. Generally, the model may be a multi-dimensional model (2D, 3D, etc.) and other visual information may be superimposed on the multidimensional model. As another example, a 2D model (image) of the body part may be captured and 3D data may be superimposed on the 2D model. One of ordinary skill in the art would appreciate that other combinations are also possible.

[0092] FIG. 3 illustrates an exemplary surgical procedure 300 (specifically, the procedure 300 is a bunion correction procedure) that may be enhanced using the real-time visual guidance system as described herein. Conventionally, the bunion correction may include a preparation stage 302, including landmarking and incisions, a correction stage 304, in which a wire jig is used to translate a bone fragment within the foot, a k-wire placement stage 306, and an implantation stage 308, in which screws are implanted into the foot to maintain the correction.

[0093] FIG. 4 illustrates an improved exemplary surgical procedure 400 (which may be the same bunion correction procedure that is improved using the real-time visual guidance system described herein. Particularly, the use of the real-time visual guidance system eliminates the need for the wire jig (or at least eliminates the need to use the wire jig in the same manner that the wire jig would be used in a conventional procedure) to be used to perform mechanical alignment. Rather, the real-time visual guidance may be used to enhance the efficiency and accuracy of the process. Although FIGS. 3-4 illustrate a bunion correction, the systems and methods described herein may also be applicable to any other type of surgical procedure.

[0094] FIG. 5 depicts one example of a type of real-time visual guidance that may be presented to a surgeon while performing a procedure. Particularly, FIG. 5 illustrates guidance used to aid a surgeon in performing an osteotomy of a bone 501 of a patient (similar to the exemplary visual guidance shown in the use case 100 of FIG. 1).

[0095] In a first image 500, a virtual bounding box 502 is provided over the 3D model of the bone 501 . The virtual bounding box 502 provides a visual indication to a surgeon of the region of the bone 501 that needs to be cut during a portion of the procedure. Also shown in the first image 500 is a virtual burr 504, which is a visual representation indicating thegeometry and the positioning of the actual burr relative to the bone 501. That is, the first image 500 shows the surgeon initiating the cut of the bone 501 using the burr.

[0096] The second image 510 illustrates visual information that may be presented as the surgeon progresses through the bone cut. In addition to the virtual bounding box 502 and the virtual burr 504, virtual box 512 is shown indicating the region of the bone 501 that has already been cut during the procedure. This allows the surgeon to accurately track the regions of the bone 501 that have already been cut and the regions of the bone 501 that still need to be cut.

[0097] Finally, the third image 520 illustrates that all of the region of the bone 501 encompassed by the virtual bounding box 502 has been cut (indicated by the virtual box 512 completely filling the virtual bounding box 502). In some instances, further visual information may be presented to indicate to the surgeon that the bone cut has been completed. For example, while the cut is being performed, the virtual box 512 may be presented in one color, and when the cut has been completed, the color of the virtual box 512 may change to another color (such as green, for example). The indication may also be provided in any other suitable form. For example, text may be presented indicating the cut has been completed or an auditory alert may be generated indicating that the cut has been completed.

[0098] FIG. 5 also includes a fourth image 530, fifth image 540, and sixth image 550, illustrating that the 3D models allow for different angles of the bone 501 to be presented to the surgeon during the procedure. The fourth image 530 provides another angle of the first image 500, the fifth image 540 provides another angle of the second image 510, and the sixth image 550 provides another angle of the third image 520. Different angles of the bone 501 may be desirable at different points as the cut is being performed and the visual guidance may switch between these different angles. For example, the fourth image 530, fifth image 540, and sixth image 550 show a perspective of an axis passing lengthwise through the burr (for example, the tip of the virtual burr 504 is show n in these images). This angle may be advantageous, for example, when the surgeon is initially inserting the burr into the bone 501 such that the tip of the bun may be accurately aligned with the portion of the bone 501 at which the initial cut needs to be performed. However, once the burr begins to cut into the bone 501, the alternative angles shown in the first image 500, second image 510, and third image 520 may provide more useful information by visualizing the depth of the cut as the burr progresses deeper into the bone 501.

[0099] In some instances, the user interface may automatically transition between the different angles based on different types of triggers. For example, the computing system (for example, computing system 112, computing system 2104, computing system 2200. etc.) performing data processing may determine that the cut has begun and may then automatically transition between displaying one angle instead of another angle via the user interface. As another example, the surgeon (or another user) may provide a command to transition between the different angles (for example, pressing a button, providing a voice command, etc.). The user interface may also transition between angles based on any other number of conditions. Additionally, the user interface may also present multiple angles simultaneously such that a transition is not necessary between angles. Instead, all beneficial angles may be presented via the user interface at all times.

[0100] FIGS. 6-7 depict another example of a type of real-time visual guidance that may be presented to a surgeon while performing a procedure. Particularly, FIGS. 6-7 illustrate guidance associated with the positioning of a wire jig used to insert wire guides into the foot 600 of a patient during bunion correction.

[0101] In embodiments, the wire jig may be a Pecaplasty® system produced by Novastep® (however, any other type of wire jig may also be used). This wire jig delivers reproducible outcomes in percutaneous bunion correction. The instrument helps navigate the challenges of the procedure by controlling translation and providing accurate placement of the guide wires. Peca implants may be used to fasten and stabilize the correction during the healing process.

[0102] However, for the bone translation and guide wire placement to be performed correctly, the wire jig may need to be accurately aligned with certain portions of the foot. Accordingly, the user interface may present a virtual wire jig including geometrical and spatial information about the wire jig to aid the surgeon in the alignment of the wire jig relative to the bone. As shown in FIG. 6, a visual indication may be provided when the wire jig is correctly aligned with the bone. For example, the first virtual wire jig 602 is show n in a first color (such as a red color), indicating that the wire jig is not yet properly aligned. In contrast, the second virtual wire jig 602 is shown in a second color (such as a green color), indicating that the wire jig is now properly aligned. Any other types of indications may also be provided, such as text indications, auditory alerts, etc.

[0103] Furthermore, other types of guidance may be provided to the surgeon as the surgeon attempts to align the wire jig with the bone. For example, text may be presented onthe user interface indicating that the surgeon should rotate the wire jig by seven degrees in the clockwise direction. This is merely one non-limiting example of a type of guidance and any other types of guidance may also be provided. Additionally, the guidance may also be provided in any other form.

[0104] FIG. 7 illustrates that the visual information may also be presented to the surgeon at any other angle, similar to the images shown in FIG. 5. As mentioned with respect to FIG. 5, the user interface may automatically transition between the different angles or may present images of the different angles simultaneously. Additionally, while reference is made to the automatic transition between angles, the surgeon (or another user) may also manually cause the transitions as well.

[0105] The real-time visualizations that are presented to the user may not necessarily be fixed images or 3D models. That is. a user may interact with the visualizations to rotate, zoom, enlarge, or otherwise adjust the visualizations. For example, a user may manually rotate the visualizations to view a different angle. As another example, a user may zoom in on a specific region of the visualization to view more granular information. These adjustments may be performed in any suitable manner, such as a touch of a touchscreen on which the user interface is presented, an interaction with an input device, such as a keyboard, an auditory command, etc.

[0106] FIGS. 8-9 provide yet another exemplary type of visualization that may be presented during a surgical procedure. Particularly, FIG. 8 shows a first 3D model 800 of a patient’s foot during bunion correction. The first 3D model 800 shows virtual lines representing the predicted trajectories of guide wires that are to be inserted into the foot 801 of a patient during bunion correction. In the first 3D model 800, a virtual bounding box 802 is shown that represents the location to which a particular bone of the foot should be translated during the procedure. A virtual dotted line 803 provides an indication of the desired trajectory of the guide wire to be inserted into the foot of the patient. Likewise, a virtual line 804 provides an indication of the actual trajectory of the guide wire if adjustments are not performed. This allows the surgeon to make adjustments until the virtual dotted line 803 aligns with the virtual line 804.

[0107] Given that the surgeon may focus on particular portions of the patient’s foot and that portions of the patient’s foot may be adjusted during the procedure, the 3D model 800 (and / or any other 3D model described herein) may also be “segmented” or separated into distinct components of the overall 3D model. For example, if the surgeon is performing atask on one particular bone or combination of bones, a segment of the 3D model may be presented including only that bone or those bones. Segmentation may also be performed in any other suitable manner.

[0108] FIG. 8 also shows a second 3D model 810 (which may be the same as 3D model 800 with additional visual elements overlaid on the 3D model) of a patient’s foot during bunion correction after the guide wires have been inserted into the foot. Accordingly, virtual lines 805 and 805 are show n representing the locations of the guide wires within the foot of the patient. As aforementioned, one or more sensors may be provided in surgical implants (such as the guide wires) to allow geometrical and spatial information about the guide wires to be obtained such that the virtual lines 805 and 805 may be presented to track the location of the guide wires within the foot.

[0109] FIG. 9 shows further visual representations of the patient’s foot during the bunion correction procedure. Particularly, the first 3D model 900 show's that virtual line 804 is perfectly aligned with the virtual dotted line 803. This provides an indication to the surgeon that the guide wire is properly aligned and will be inserted into the proper location with the patient’s foot. The second 3D model 910, similar to the second 3D model 810 of FIG. 8, shows the virtual lines 805 and 805 are shown representing the locations of the guide wires within the foot of the patient. Additionally, the second 3D model 910 also shows the translation of a portion of the bone into the virtual bounding box 802 (this portion of the bone is translated towards the other bones of the foot as a part of the bunion correction).

[0110] FIG. 10 shows yet another exemplary type of visualization that may be presented during a surgical procedure. Particularly, FIG. 10 shows portions of a 3D model that present a different angle of the visualizations presented in FIGS. 8-9 used to assist the surgeon in properly aligning the guide wire within the patient’s foot 1001. In FIG. 10, the first 3D model 1000 shows an example in which the actual trajectory of the guide wire is not properly aligned with the ideal trajectory’ (indicated by the misalignment of the virtual line 1002 and the virtual dotted line 1004). In contrast, the second 3D model 1010 show's an example in which the actual trajectory of the guide wire is properly aligned with the ideal trajectory (indicated by the alignment of the virtual line 1002 and the virtual dotted line 1004).[OHl] FIG. 10 also show's an exemplary visual guidance element 1020 that may further supplement the virtual dotted line 1004 and the virtual line 1002 in assisting the surgeon in properly aligning the guide wire in the patient's foot. For example, the visual guidance element 1020 may be provided in the shape of a target and a dot 1022 corresponding to thelocation of the virtual line 1002 may be provided on the target. The distance between the dot 1022 and the center of the target provides an indication of the relative alignment of the virtual line 1002 relative to the virtual dotted line 1004. The dot being 1022 being located exactly in the center of the target provides an indication to the surgeon that the guide wire will be properly aligned within the patient’s foot. Although FIG. 10 provides the example of a target and a dot, these are merely non-limiting examples, and any other types of visual elements may be used to indicate alignment.

[0112] FIG. 11 provides an illustration of 3D model of a patient’s foot 1100 that may be generated using one or more electromagnetic sensors as described herein. The figure shows that three electromagnetic sensors (for example, first electromagnetic sensor 1102, second electromagnetic sensor 1104, and third electromagnetic sensor 1106 are provided at various locations on the patient’s foot 1100.

[0113] These locations may, in some instances, be pre-determined locations corresponding to portions of a patient’s foot that are desired to be tracked during a particular surgical procedure (however, the sensors may also be provided at any other location). Although the figure shows three electromagnetic sensors as being provided, any other number of sensors may also be provided. The electromagnetic sensors may be provided on the foot and / or within the foot as well.

[0114] As aforementioned, the data captured from the electromagnetic sensors within the electromagnetic field generated by the electromagnetic field generator (not shown in the figure) may be co-registered with a 3D model 11 10 of the patient’s foot that was generated using images of the patient’s foot 1 10 that were captured using any type of imaging device. As data is obtained from the electromagnetic sensors, the 3D model 1110 may be adjusted to reflect changes in the spatial positioning of the electromagnetic sensors over time. For example, one electromagnetic sensor may be provided on a specific bone in the foot 1100 and movement of that bone (either intentionally or unintentionally during the surgical procedure) may be captured and reflected by an update to the 3D model 1110.

[0115] FIG. 12 depicts an example of data co-registration with fiducial markers and a 3D model.

[0116] FIG. 13 A shows a heel cup that may be used to perform the 3D reconstruction during the initial imaging stage. The heel cup is a structure with known physical dimensions, angles, etc. that is configured to receive the heel of a patient (for example, the patient may place their foot in the heel cup during initial imaging, the surgical procedure, etc. The heelcup may be desirable because it provides a mechanism by which the angle of the foot may be fixed during imaging and 3D model construction. The heel of the patient may be placed in the heel cup with prescribed angles such that the angle of the foot is known. In this manner, the heel cap may also be used as the fiducial marker by ensuring that the foot is in the same position during initial imaging and during the surgical procedure itself. While the heel cup is provided as one example, any other type of mechanism in any other shape and / or size may also be used.

[0117] FIG. 13B shows exemplary fiducial markers 1300 that may be used as a frame of reference while performing tracking using the electromagnetic sensors. The fiducial marker 1300 may be an object that is provided at a fixed location and serves as a fixed frame of reference for the electromagnetic tracking that is performed during the second step (during the surgical procedure itself). That is, the location of the fiducial marker 1300 is captured during the initial imaging stage and the location may then be compared to detected locations of electromagnetic sensors during the surgical procedure to aid in tracking the location of the sensors within the generated electromagnetic field. That is, one of the electromagnetic sensors (or any other type of marker) used during the procedure may be used as a “reference sensor’ that allows for co-registration through calibration / serves as a reference point.

[0118] In some instances, the fiducial marker 1300 may be an electromagnetic sensor that is provided at the fixed location during the initial imaging and remains in the fixed location or is provided back in the same location during the procedure. The fiducial marker 1300 does not necessarily need to be a sensor, however, and may include any other ty pe of object, such as a sticker. The fiducial marker 1300 also does not necessarily even need to be an object (for example, the fiducial marker 1300 may include a marking). Even if the fiducial marker 1300 is initially a non-sensor object during the initial imaging stage, an electromagnetic sensor may be placed at the location of the fiducial marker 1300 during the procedure itself. Furthermore, more than one fiducial marker 1300 may be used as well.

[0119] FIG. 14A depicts an exemplary system 1400 including a guide wire 1402 with an embedded sensor 1408. As previously indicated, the guide wire 1402 may include a distal end 1404 (the “tip” as described herein) and a proximal end 1406. The distal end 1404 of the guide wire 1402 is the portion of the wire that would come into contact with the patient first during the surgical procedure. The sensor 1408 may be embedded at the distal end 1404 of the guide wire 1402. For example, the guide wire 1402 may be hollow and the sensor1408 may be provided in the cavity internal to the guide wire 1402. However, the sensor 1408 may also be embedded within the guide wire 1402 in any other suitable manner. The sensor 1408, in some embodiments, may also be provided on the exterior surface of the guide wire 1402 as well.

[0120] The system 1400 may also include any other elements of any other system described herein. For example, computing system 112, electromagnetic field generator 130, any of the elements described with respect to the system 2100, etc.). That is, the illustration shown in FIG. 14A is merely intended to show additional components that may be included in a system (such as the posts that may be inserted into the bones) and is not intended to be an illustration of a comprehensive system including all of the elements that may be present during the surgical procedure. This is also be applicable to any other system described herein.

[0121] A first wired connection 1410 may be connected between the sensor 1408 embedded in the guide wire 1420 and a slip ring 1412. A second wired connection 1410 may be connected between the slip ring 1412 and a computing system 1414. That is, the first wired connection 1410 may be routed from the sensor 1408, through the internal cavity of the guide wire 1402, and to the second wired connection 1413 via the slip ring 1412 such that the sensor 1408 may transmit data to the computing system 1414 for processing. For example, the sensor 1408 may be a position sensor that may transmit position information to the computing system 1414 such that the computing system 1414 may be used to track the current position and movement of the guide wire 1402 during a surgical procedure. The computing system 1414 may be any system with processing capabilities, such as the computing system 112, computing system 2104, computing system 2200, etc.

[0122] The first wired connection 1410 and the second wired connection 1413 may be any type of wire that may be used to transmit signals, such as a conventional copper wire. However, other types of mediums may be used for signal transmission. For example, the first wired connection 1410 and / or second wired connection 1413 may instead be a fiber optic cable or any other type of medium that may be used to transmit signals from the sensor 1408 to the computing system 1414. Additionally, in some embodiments, the sensor 1408 (or any other sensor described herein) may transmit data over a wireless connection using any suitable wireless communication protocol.

[0123] FIG. 14B shows a close-up view of the slip ring 1412. The slip ring 1412 allows the first wired connection 1410 to rotate along with the rotation of the sensor 1408 whilestill being in electrical communication with the second wired connection 1412 (to transmit signals from the sensor 1408 to the computing system 1414 in real-time). The slip ring 212 includes a stator 1422 and a rotor 1420. The rotor 1420 may include conductive rings, while the stator 1422 may include brushes or contact points that maintain electrical contact with the rings. As the rotor 1420 rotates, the brushes maintain continuous contact with the conductive rings. This allows electrical signals to be transmitted from the rotating rotor 1420 (connected to the first wired connection 1410) to the stationary stator 1422 (connected to the second wired connection 1413) without interruption.

[0124] While FIG. 14A shows the usage of the slip ring 1412, any other suitable mechanism that allows for rotation of two components relative to one another while still allowing for electrical signals to be transmitted between the two components during the rotation may also be used.

[0125] FIG. 15A depicts a close-up view of an exemplary guide wire 1500 (which may be the same as, or similar to, guide wire 1402 shown in FIGS. 14A-14B or any other guide wire described herein or otherwise) with an embedded sensor 1508. In FIG. 15, the guide wire 1500 is partially transparent such that components provided within the guide wire 1500 are visible. For example, an inner tube 1503 may be provided within the guide wire 1500. The inner tube 1503 may provide additional structural rigidity to the guide wire 1500, however, in some embodiments, the inner tube 1503 may not be used. The inner tube 1503 may include a cavity 1504. The sensor 1508 and the wire 1512 that is connected to the sensor 1508 may be provided within the cavity 1504. As described above, the sensor 1508 is provided at the distal end 1502 (the “tip”) of the guide wire 1500.

[0126] FIG. 15B show s an exemplary' wired connection interface 1520 that may be used. The wired connection interface 1520 may include any number of different types of interfaces that may be used to connect a portion of a wired connection to the guide wire 1500 (such that the guide wire 1500 may be easily removed from the remainder of the wired connection). For example, the guide wire 1500 may include a first conductive element that is in contact with the sensor in the guide wire 1500 that may come into contact with a second conductive element in the wired connection 1522 when the wired connection 1522 is connected to the guide wire 1500 via the wired connection interface 1520. This is merely one exemplary manner by which the wired connection 1522 may' be connected to the sensor and is not intended to be limiting. The wired connection interface 1520 may be facilitated using any suitable type of connector, heat shrink tubing, etc.

[0127] FIGS. 15C-15E depict close-up views of another exemplary guide wire 1530. Particularly, the guide wire 1530 differs from the guide wire 1500 in that an outer tube 1534 of the guide wire 1530 rotates and the sensor 1538 is provided around an inner tube 1536 that remains stationary (in this embodiment, the sensor 1538 is hollow). The wired connection 1540 connected to the sensor 1538 would also remain stationary during the rotation of the guide wire 1530 in this embodiment as well. Thus, in the embodiment shown in FIGS. 15C-15E, the rotation of the guide wire 1530 may be performed without requiring a slip ring or other suitable mechanism for the wired connection from the sensor 1538.

[0128] FIGS. 1 A-16F show additional embodiments for receiving real-time information about the position of surgical instruments during a surgical procedure. FIGS. 16A-16D show one additional exemplary embodiment and FIGS. 16E-16F show another exemplary’ embodiment.

[0129] Beginning with the embodiment shown in FIGS. 16A-16D, FIG. 16A depicts an exemplary' guide wire 1604 and an anti-rotation apparatus 1600. In this embodiment, the anti-rotation apparatus 1600 also serves as an encasement for the sensor 1608 (shown in FIG. 16B). FIG. 16B depicts a close-up view of the anti-rotation apparatus 1600 of FIG. 16A. In the embodiment shown in FIGS. 16A-16B, the sensor 1608 may be provided within the anti-rotation apparatus 1600 at a specific distance and depth w ithin the encasement 1600. The wired connection 1606 is shown as being connected to the sensor 1608. The antirotation apparatus 1600 may sit on a positive stop 1610 on the guide wire 1604 at a specific distance from the tip of the guide wire 1604. The positive stop 1610 may be used to determine the position of the tip of the guide wire 1604 via the handle through programmed dimensional offsets.

[0130] Given that in this embodiment the sensor 1608 is provided within the antirotation apparatus 1600 rather than at the tip 1605 of the guide wire 1604, the sensor 1608 may’ be calibrated such that the data produced by the sensor can be used to calculate the position of the tip 1605 of the guide wire 1604 as the guide w ire 1604 is used during a surgical procedure. As a simplified example, if the distance from the sensor 1608 to the tip 1605 of the guide wire 1604 is 5”, then this distance may be added to the position data obtained from the sensor 1608 to determine the position information for the tip 1605 of the guide wire 1604.

[0131] FIG. 16C depicts a close-up view of the anti-rotation apparatus 1600 and the anti-rotation arms 1602 of the anti-rotation apparatus 1600 of FIGS. 16A-16B. FIG. 16Ddepicts the anti-rotation apparatus 1600 of FIG. 16A attached to a drill 1620. FIG. 16C shows that the anti-rotation apparatus 1600 may include a sensor hole 1612 through which the sensor 1608 and / or wired connection 1606 may be provided. FIG. 16C also shows that the anti-rotation apparatus 1600 may include a guide wire hole 1614 through which the guide wire 1604 may be provided. FIG. 16D shows that the anti -rotation apparatus 1600 sits up against a guide wire collet within the drill 1620. When the anti -rotation apparatus 1600 is attached to the drill 1620, the anti-rotation arms 1602 may be positioned on either side of the trigger 1603 for the drill 1620 and hold the guide wire 1604 within the collet so that the guide wire 1604 rotates with the drill 1620, but the anti-rotation apparatus 1600 remains in place by the anti-rotation arms 1602.

[0132] Turning to FIGS. 16E-16F, another exemplary' anti-rotation apparatus 1630 is shown. In this embodiment, the anti-rotation apparatus 1630 may also use anti -rotation arms 1632 (which may be the same as, or similar to, anti-rotation arms 1602 shown in FIGS. 16A- 16D). In contrast with the embodiment shown in FIGS. 16A-16D, the embodiment shown in FIGS. 16E-16F includes a separate sensor encasement 1632 and a non-insertable guide wire 1634 connecting the anti-rotation apparatus 1630 and the sensor encasement 1632 and preventing rotation of the anti-rotation apparatus 1630 and the sensor encasement 1632 with respect to each other. The location of the anti-rotation apparatus 1630 may be adjusted. The anti-rotation apparatus 1630 may also hold the wared connection 1636 from the sensor in the sensor encasement 1632 to prevent tangling of the wired connection 1636. In some instances, the non-insertion guide wire 1634 can be bent out of the way or cut to not interfere with the drill 1620 (not shown in the figure).

[0133] FIG. 16F depicts a close-up view of the sensor encasement 1632 of FIG. 16E. The sensor 1644 may be provided at a specific distance and depth within the sensor encasement 1632. The sensor encasement 1632 sits on a positive stop 1640 on the guide wire 1638 at a specific distance from the tip 1646 of the guide wire 1638. Similar to the embodiment shown in FIGS. 16A-16D, the sensor 1644 in the embodiment shown in FIGS. 16D-16E may be calibrated to its position to calculate the location of the tip 1646 of the guide wire 1638. This design allows the sensor 1644 to be closer to the tip 1646 of the guide wire 1638 without embedding the sensor 1644 within the guide wire 1638. The location of the positive stop 1640 may be such that the guide wire 1638 may be inserted up to the positive stop and therefore up to the sensor encasement 1632.

[0134] 17A-17C depict exemplary usage of the guide wires as described herein (for example, guide wire 1402, guide wire 1500, guide wire 1604, guide wire 1638, guide wire 1702, etc.) in a surgical procedure. In the examples shown in FIGS. 17A-17C, the surgical procedure is a foot and ankle procedure (specifically, a bunion correction procedure), however, the guide wires may also be used in other surgical procedures involving other patient body parts as well.

[0135] Specifically. FIGS. 17A-17B show a first guide wire 1702 that is provided through the the foot 1700 of the patient. FIGS. 17A-17B also show that a first wired connection 1704 is provided between the first guide wire 1702 and the computing system 1708. The wired connection 1704 may be used to transmit signals from the sensor provided in the first guide wire 1702 to the computing system 1708 for processing. Although a single guide wire 1702 is shown as being used in the exemplary procedure shown in FIGS. 17A- 17B, multiple guide wires may also be provided in the foot 1700 of the patient in some instances as well.

[0136] In an exemplary' procedure, an image of the foot 1700 may be captured in multiple views to generate a 3D model of the foot 1700. For example, an image may be captured of the foot 1700 in the view shown in FIG. 17A and another image may be captured of the foot 1700 in the view shown in FIG. 17B. The views shown in FIGS. 17A-17B are merely exemplary and any other number of images of other views of the foot 1700 may also be captured. The images may be captured, for example, by an X-ray machine, however, the images may also be captured by any other type of device. The computing system 1708 may then locate the guide wires within the foot 1700 by using a combination of the data received from the sensor(s) and the generated 3D model.

[0137] FIG. 17C shows a second guide wire 1710 that is also inserted into the foot 1700 of the patient. The second guide wire 1710 also includes a wired connection 1712 that may allow a sensor embedded in the tip of the second guide wire 1710 to transmit signals to the computing system 1708 (or another computing system not shown in the figure). In an exemplary' procedure, once the second guide wire 1710 is inserted into the foot, a bone segment of the foot 1700 (at the bunion) may be cut. shifted laterally, and the second guide wire 1710 may be inserted into the shifted bone segment. Screws used to hold the shifted bone in place may then be inserted into the foot 1700 at the location of the first guide wire 1702 and the second guide wire 1710.

[0138] FIG. 19A depicts another exemplary system 1900 including various components with embedded sensors. Specifically, the system 1900 includes various posts (for example, post 1904, post 1906. post 1910, etc.) that are inserted into different locations on the foot 1902 of a patient for tracking movement of the bones of the foot 1902 during a surgical procedure. The system 1900 may also include any other elements of any other system described herein. For example, computing system 112, electromagnetic field generator 130, any of the elements described with respect to the system 2100, etc.). That is, the illustration shown in FIG. 19A is merely intended to show additional components that may be included in a system (such as the posts that may be inserted into the bones) and is not intended to be an illustration of a comprehensive system including all of the elements that may be present during the surgical procedure.

[0139] In one or more embodiments, the posts may be rigid structures that are inserted into one or more different bones of the foot 1902 (or any other bone, depending on the particular surgical procedure) and remain fixed such that the posts can be used for tracking purposes. As shown in further detail in FIG. 19B, the posts may be various shapes and / or sizes (the shapes and / or sizes or the posts 1904-1908 shown in FIG. 19B are also merely exemplary and other shapes and / or sizes are also possible).

[0140] In some instances, the posts may include a first portion that is removably or permanently affixed to a second portion. For example, FIG. 19B shows that the post 1904 includes a first portion 1920 and a second portion 1922. The first portion 1920 may be configured to be inserted into a bone of the patient. The second portion 1922 may be configured to receive one or more sensors and / or any other elements used for tracking purposes during the surgical procedure. FIG. 19C depicts an exemplary' mechanism that allows a sensor 1930 to be inserted into a post 1904. Specifically, FIG. 19C shows the post 1904 (which may be any other post shown herein or otherwise) including a hollow 1932 region that is sized and shaped to receive the sensor 1930. The post 1904 further includes a cutout region 1933 that is sized and shaped to receive a corresponding engagement portion 1935 of a housing 1934 to which the sensor 1930 is attached. To affix the sensor 1930 to the post 1904. the sensor 1930 and housing 1934 are inserted into the cutout region 1933 of the post 1904. When fully inserted, the sensor 1930 resides within the hollow portion 1932 of the post 1904. The housing 1934 also includes a hollow interior 1936 such that a wired connection 1938 may be routed from the sensor 1930, through the housing 1934, and to a controller or other type of processing element that receives data from the sensor 1930 (inembodiment in which a wired sensor is used). However, the sensor 1930 may also be a wireless sensor, in which case the wired connection 1938 would not be required. This is merely one mechanism by which a sensor may be removably or permanently affixed within or on a post and any other suitable mechanism may also be used. Additionally, while FIGS. 19A-19B show separate first and second portions, this is not intended to be limiting and the posts may instead be a single structure or may be made from any other number of individual structures.

[0141] In one or more embodiments, some or all of the posts may also include fiducial markers. Examples of different fiducial markers, as well as exemplary arrangements of such fiducial markers are also shown in FIG. 19B. For example, the post 1906 is shown as including a removable pin 1907 that serves as a fiducial marker. The removable pin 1907 is beneficial because it can be inserted into an object (such as the post 1906 or any other post or object describe herein) when it is desired to use a fiducial marker, and can be removed when not in use. This reduces the amount of visual clutter present in the navigation volume to provide additional visibili ty for the surgeon. As another example, the post 1908 is shown as including two fiducial markers (fiducial marker 1909 and fiducial marker 1910) arranged at different heights on the post 1908. As yet another example, the post 1921 (provided in a different shape) includes six different fiducial markers (fiducial markers 1914-1919) arranged at various locations on the post 1912. This configuration provides a larger format for accuracy in total ankle procedures. These are merely a few examples of different fiducial marker configurations, and a post may include any other number of fiducial markers of any shape and / or size (or combinations of shapes and / or sizes) that are provided at any arrangement.

[0142] FIG. 19D depicts additional mechanisms for housing a sensor. Specifically, FIG. 19D shows a front view 1941 and a rear view 1942 of a first magnetic clip 1940. The first magnetic clip 1940 is formed as a combination of a first portion 1943 and a second portion 1944. The second portion 1944 is configured to be affixed to a surgical instrument (or other ty pe of object), such as the drill 1950 shown in FIG. 19A. The first portion 1943 is then removably affixed to the second portion 1944 via the magnets. In some instances, the second portion 1944 may not necessarily be magnetic, but may instead affix to the surgical instrument (or other object) using any other suitable mechanism (e g., adhesive, etc.). This is because it may be undesirable for objects provided within or proximate to a navigationvolume during a surgical procedure to be magnetized, which may influence any electromagnetic fields within the navigation volume (and / or cause other complications).

[0143] FIG. 19D also shows a front view 1945 and a rear view 1946 of a second clip 1947 that uses a key hole 1948 to selectively lock a first portion 1949 and a second portion 1951 in a similar manner that the first portion 1943 and the second portion 1944 are combined in the first magnetic clip 1940 (using the key hole 1948 instead of the magnets). FIG. 19D also shows a wire sleeve 1952 with and integrated sensor.

[0144] FIG. 20 depicts an example method 2000 for real-time visual guidance during surgical procedures. Some or all of the blocks of the process flows or methods in this disclosure may be performed in a distributed manner across any number of devices or systems (for example, computing system 112, computing system 1414, computing system 1708, user device 2102, computing system 2104. imaging device 2108. electromagnetic tracking system 2110, etc.). The operations of the method 2000 may be optional and may be performed in a different order.

[0145] At block 2002 of the method 2000, computer-executable instructions stored on a memory of a system or device may be executed to dispose a first electromagnetic sensor (for example, electromagnetic sensors 108, 109, 206, 1102, 1104, 1106, 1408, 1508, 1538, 1608, 1644, 1930, 2112, and / or any other sensors described herein or otherwise) at least one of: on or within a bone of the patient. In some embodiments, a sensor or sensors may be disposed on and / or within another object and that object may itself be disposed on and / or at least partially within a bone of the patient. For example, at least FIGS. 19A-19B show various types of ’‘posts” (such as posts 1904, 1906, 1908, 1912, etc.) that are partially inserted into a bone of a patient, and the posts may include one or more sensors.

[0146] At block 2004 of the method 2000. computer-executable instructions stored on a memory of a system or device may be executed to dispose a second electromagnetic sensor (for example, electromagnetic sensors 108, 109, 206, 1102, 1104, 1106, 1408, 1508, 1538, 1608, 1644, 1930, 2112, and / or any other sensors described herein or otherwise) at least one of: on or within a surgical instrument (for example, guide wires 1402, 1500, 1604, 1634, 1638, 1702. 1710, and / or any other guide wires described herein, as well as any other type of surgical instrument described herein or otherwise).

[0147] At block 2006 of the method 2000, computer-executable instructions stored on a memory of a system or device may be executed to generate, by an electromagnetic fieldgenerator (or example, EMF generator 21 10, etc.), an electromagnetic field within a navigation volume during the surgical procedure.

[0148] At block 2008 of the method 2000, computer-executable instructions stored on a memory of a system or device may be executed to receive, by a computing system, data from the first electromagnetic sensor and the second electromagnetic sensor. For example, the data may be position data that provides an indication of the various sensors within the navigation volume at any given time (such as in real-time). Given that the sensors may be attached to (or otherwise associated with) various elements involved in the surgical procedure (such as the bones of the patient, the surgical instruments used by the surgeon, any surgical implant or implants being inserted into the patient, etc.), this positional data may allow for real-time tracking of these various elements to be performed during the procedure.

[0149] At block 2010 of the method 2000, computer-executable instructions stored on a memory of a system or device may be executed to cause to display, by the computing system, in real-time, and based on the data, visual information. For example, a 2D or 3D model of the body part of the patient may be generated and visual representations of any of the elements involved in the surgery may be presented relative to the model based on their determined positional data. The visual information may also include other types of information, such as augmented visual elements that are also presented on the model to guide the surgeon during the surgical procedure. Non-limiting examples of ty pes of visual information that may be presented as shown in at least FIGS. 5-11 and 18A-18D. Any other types of visual information may be presented in any suitable form.

[0150] In one or more embodiments, the computer-executable instructions may further be executed to receive, from an imaging device (for example, imaging device 1608), and prior to a surgical procedure, a three-dimensional model (for example, three-dimensional model 220, three-dimensional models shown in FIGS. 4-12 or any other three-dimensional model described herein or otherwise) of a body part of a patient. In one specific embodiment, the three-dimensional model may be a model of the bone structure of the foot of a patient. This three-dimensional model may be used to assist a user (such as a surgeon) in performing a surgical procedure on the foot or ankle, such as a bunion correction. However, any other model of any other body part (e.g., the foot or ankle) may also be received depending on the particular surgical procedure that is being performed. For example, the imaging device may be a CT scanner or an x-ray machine that is used to capture images to generate the three-dimensional model. Tn some instances, depending on the type of imaging device, bi-planar imaging may be used.

[0151] In one or more embodiments, the computer-executable instructions may further be executed to generate, by an electromagnetic field generator (for example, electromagnetic field generator 1612 or any other electromagnetic field generator described herein or otherwise) and during the surgical procedure, an electromagnetic field at a navigation volume of the surgical procedure. The navigation volume may refer to an area of focus during the surgical procedure. For example, if a bunion correction is being performed, the navigation volume may include the foot of the patient. The navigation volume may also include at least some area around the foot of the patient as well. In some instances, the navigation volume may more specifically refer to the precise location that the surgeon is currently working within. For example, the navigation volume for one part of the surgery may be a specific bone of the foot and the navigation volume for another part of the surgery may be another bone.

[0152] In one or more embodiments, the computer-executable instructions may further executed to generate, in real-time during the surgical procedure, a modified three- dimensional model of the body part of the patient based on a combination of the three- dimensional model and the first data. That is, the positioning of the electromagnetic sensors within the generated electromagnetic field may be tracked based on data from the sensors and the data may be co-registered with the three-dimensional model to adjust the appearance of the model to match the actual appearance of the body part. For example, the surgeon may translate a bone fragment within the foot from one location to another. The data from the sensors may capture this translation and a similar translation may then be show n in the three- dimensional model.

[0153] In one or more embodiments, the computer-executable instructions may further executed to cause to display, by the one or more processors, in real-time during the surgical procedure, and via a display device (for example, display device described with respect to FIG. 16 (which may be the same as a user device 1602 or may be associated with a user device 1602, etc.), the modified three-dimensions model of the body part that is augmented with one or more visual elements based on the second data. The visual elements may include any number of different types of visual guidance described herein or otherwise (for example, the 3D model of the patient’s body part that is modified based on data from theelectromagnetic sensors, any visual elements that are presented with the 3D model to guide the surgeon during the surgical procedure, and / or any other visual elements).

[0154] FIG. 21 is an example system 2100 for real-time visual guidance during surgical procedures. In one or more embodiments, the system may include, one or more user devices 2102 (which may be associated with one or more users 2101 ), one or more computing systems 2104, one or more imaging device(s) 2106, an electromagnetic tracking system 2108, one or more tracking object(s) 2114, one or more surgical implant(s) 2118, one or more surgical instrument(s) 2122, and / or one or more databases 2130, etc. However, these components of the system 2100 are merely exemplary and are not intended to be limiting in any way. For simplicity, reference may be made hereinafter to, user device 2102, computing system 2104, imaging device 2106, electromagnetic tracking system 2108, tracking object 2114, surgical implant 2118, surgical instrument 2122, database 2130, etc., however, this is not intended to be limiting and may still refer to any number of such elements (this applies to any other elements described with respect tot the system 2100 or otherwise.

[0155] The user device 2102 may be any type of device, such as an e-reader, personal assistant device, speaker, gaming console, smartphone, desktop computer, laptop computer, tablet, smart television (for example, a television with Internet connectivity, the capability to install applications, etc.), computer monitor, and / or any other type of device that is configured to receive data from the EM tracking system 2110 and present information to the user 2101. One non-limiting example of a user device 2102 may be a desktop computer that includes the monitor 114 shown in the use case 100 of FIG. 1. Thus, the user device 2102 may provide real-time visual guidance to the user 2101 during a surgical procedure.

[0156] The user device 2102 may also be associated with a display device. As used herein, a display device may be any type of device that presents information to a user. For example, if the user device 2102 is a desktop computer, the display device may be a monitor that presents information received from the desktop computer. In some instances, the user device 2102 may itself be a display device. For example, the user device 2102 may be a television or tablet that includes a built-in screen that serves as the display.

[0157] The user device 2102 may also be a virtual reality and / or augmented reality device, such as a headset, glasses, a smartphone or any other type of device configured for virtual reality and / or augmented reality. For example, during a surgical procedure, a surgeon may wear the virtual reality and / or augmented reality device and the device may present anyof the real-time visual guidance to the surgeon through virtual reality and / or augmented reality.

[0158] The computing system 2104 may be a local or remote system that may also be used to perform some or all of the processing associated with providing real-time visual guidance to the user 2101. For example, the computing system 2104 may be a local or remote server (the computing system 2104 may be located in the operating room, within the hospital or other building in which the operating room is located, or may be located in a separate building) that may also receive any of the data from the EM tracking system 2110, the images from the imaging device 2108, and / or any other types of data. The computing system 2104 may process the data and generate the visual information that is then presented to the user 2101 via the user device 2102. However, some or all of this processing may also be performed locally at the user device 2102 as well (and the computing system 2104 may not necessarily be required in the system 2100.

[0159] The imaging device 2106 may be any type of device that is used to capture images of a body part of a patient that is to undergo a surgical procedure. Continuing the example bunion correction procedure, initial imaging is performed on the foot of the patient. For example, the imaging device 2106 may be a computer tomography (CT) device or an x- ray imaging device that may be used to obtain CT scans or x-ray images of the patient’s foot. The imaging device 2106 may also be any other device that is capable of capturing images or other representations of the patient's foot. The imaging may be bi-planar imaging, meaning that images at multiple different angles of the patient’s foot and / or ankle may be taken to produce the 3D image. However, bi-planar imaging may not always be required depending on the ty pe of imaging device.

[0160] The electromagnetic tracking system 2108 may include an electromagnetic field generator 2110 (which may be the same as, or similar to. electromagnetic field generator 130 of FIGS. 1A-1B or any other electromagnetic field generator described herein or otherwise) and one or more electromagnetic sensors 2112. As aforementioned, the electromagnetic field generator 2110 may be used to generate one or more electromagnetic fields within a navigation volume during a surgical procedure. A reconfigurable antenna array including individual antenna elements may be used, with each antenna element creating its own magnetic field with accurate environmentally controlled electrical currents. Multiple of these antenna elements may be used to create the electromagnetic fields in which the electromagnetic sensors may be localized. The antenna coils used for the generation ofthe electromagnetic field may be formed as printed circuits or attached to different layers that are coupled together. These antennas may also, in some instances, simultaneously serve as sensors to detect the electromagnetic field. Multi-layer printed circuit boards (PCBs) (which may be the antenna "‘elements”) can be used with specifically chosen shapes of traces on individual layers in order to increase the magnetic field strength and / or shape the magnetic field.

[0161] This is merely one example of an electromagnetic field generator that may be used and any other type of electromagnetic field generator may also be used.

[0162] The one or more electromagnetic sensors 2112 may be tracked within the one or more electromagnetic fields generated by the electromagnetic field generator 2110 to provide geometric and spatial information for various objects involved in a surgical procedure. The one or more electromagnetic sensors 2112 may be disposed at various locations within the navigation volume to produce real-time tracking data during the procedure. The one or more electromagnetic sensors 2112 may be placed on or within the patient’s body part that is the subject of the surgical procedure. For example, during bunion correction surgery, at least some of the one or more electromagnetic sensors 2112 may be provided on or proximate to the bones within the foot of the patient such that movement of the bones during the procedure may be tracked and visualized in real-time. The one or more electromagnetic sensors 2112 may be provided proximate to the bones in any suitable manner. For example, the one or more electromagnetic sensors 2112 may be provided on the skin of the patient, on the bones themselves, and / or at any other suitable location.

[0163] The one or more electromagnetic sensors 2112 may also be used to track objects within the navigation volume other than parts of the patient’s body. For example, a sensor or sensors may be provided on and / or within surgical instruments that are used during the surgical procedure, such as a burr, guide wire jig, etc. These sensors may be used to accurately track the positioning of the instrumentation within the navigation volume such that the surgeon can accurately align the instruments in the proper locations to perform actions during the surgery.

[0164] As another example, a sensor or sensors may also be provided on surgical implants that are provided within the patient’s body (e.g., guide wires, screws, etc ). The one or more electromagnetic sensors 2112 may also be provided to track any other types of objects as well. A few- non-limiting examples of surgical instruments and surgical implants (and / or other types of like devices) may be found in the following patents, which are allincorporated herein by reference: European Patent No. titled “2254492” titled “Intramedullary' medical device and methods of manufacture,” US Patent No. 7985222 titled “Osteosynthetic implants and methods of use and manufacture,” US Patent No. 8118952 titled “Osteosynthetic implants and methods of use and manufacture,” US Patent No. 8491583 titled “Intramedullary medical device and methods of use and manufacture,” US Patent No. 8551106 titled “Method and apparatus for installation of intramedullary medical device,” US Design Patent No. 611145 titled “Intramedullary medical device,” US Design Patent No. 611225 titled “Intramedullary’ medical device,” US Design Patent No. 625818 titled “Intramedullary medical device,” and US Design Patent No. 657873 titled “Intramedullary^ medical device.”

[0165] The electromagnetic sensors may be any size and / or shape (or different sensors may be sizes and / or shapes depending on where the sensors are being provided). For example, the sensors may be centimeters or millimeters in diameter (or length, width, and / or height depending on the shape of the sensor), as well as any’ other sizes (including smaller than or larger than centimeters or millimeters). The sensors may also include any number of degrees of freedom. Additionally, the sensors that are used may not necessarily all be the same size and / or shape and different types of sensors may be used, even within the same surgical procedure. For example, one sensor may need to be sized and shaped to fit on or within a first type of surgical instrument and another sensor may need to be sized and shaped to fit on or within a second type of surgical instrument that may be a different size and shape compared to the first surgical instrument.

[0166] In embodiments, the electromagnetic sensors may also include a mechanism used to attach the sensors to any object (e.g., patient, instruments, implants, etc.). For example, the sensors may be attached via permanent or temporary bonding (e.g., an adhesive, etc.), quick connections, or other mechanisms for mechanical or other types of attachment and / or detachment.

[0167] Additionally, one or more embodiments, some or all of the sensors 2112 may be disposed on and / or within other objects provided within the navigation volume during the surgical procedure. For example, one or more sensors 2112 may be disposed on and / or within tracking object(s) 2114 (such as posts 1904, 1906, 1907, and 1912 shown in FIGS. 19A-19B, as well as any other types of objects). As an example (described with respect to FIGS. 18A-18B), a tracking object 2114 can be inserted into a bone of the patient during the surgical procedure. The tracking object 2114 may remain fixed at the location, therebyresulting in a fixed position of the sensor 21 12 disposed on and / or within the tracking object 2114. Any number of such tracking objects 2114 can be provided at various locations (for example, inserted into different bones or portions of a bone of the patient).

[0168] As another example, one or more sensors 2112 may be disposed on and / or within any surgical instruments 2122 used during the surgical procedure (to track the position of the surgical instruments 2122 in real-time during the procedure). As one example specifically described with respect to FIGS. 14A-17C, a sensor 2112 may be disposed on and / or within a guide wire, however, one or more sensors 2112 may also be disposed on and / or within any other types of surgical instruments 2122. As yet another example, one or more sensors 2112 may be disposed on and / or within any surgical implants 2118 inserted into the patient during the surgical procedure (to track the position of the surgical instruments 2118 in real-time during the procedure). In some embodiments, sensors 2112 may be disposed on and / or within any combination of these or other objects involved in the surgical procedure to capture real-time position information (and other types of information) about such objects during the procedure.

[0169] The database 2130 may store any of the data that is used as described herein. For example, the database 2130 may store any of the images and / or other types of virtual representations that are captured by the imaging device 2108. The database 2130 may also store some or all of the data captured by the EM tracking system 2108. The database 2130 may also store any other data that may be relevant to a surgical procedure, such as information about various aspects of the surgical procedure, the equipment required, etc.

[0170] In one or more embodiments, any of the elements of the system 2100 (for example, one or more user devices 2102, one or more computing systems 2104, imaging device(s) 2106, EM tracking system 2108, tracking object 2114, surgical implant 2118, surgical instrument 2122. one or more databases 2130, and / or any other element described with respect to FIG. 21 or otherwise) may be configured to communicate via a communications network 2150. The communications network 2150 may include, but not limited to, any one of a combination of different ty pes of suitable communications networks such as. for example, broadcasting networks, cable networks, public networks (e.g., the Internet), private networks, wireless networks, cellular networks, or any other suitable private and / or public networks. Further, the communications network 2150 may have any suitable communication range associated therewith and may include, for example, global networks (e.g., the Internet), metropolitan area networks (MANs), wide area networks(WANs), local area networks (LANs), or personal area networks (PANs). In addition, communications network 2150 may include any type of medium over which network traffic may be carried including, but not limited to, coaxial cable, twisted-pair wire, optical fiber, a hybrid fiber coaxial (HFC) medium, micro wave terrestrial transceivers, radio frequency communication mediums, white space communication mediums, ultra-high frequency communication mediums, satellite communication mediums, or any combination thereof.

[0171] Finally, any of the elements (for example, one or more user devices 2102, one or more computing systems 2104, imaging device(s) 2106, EM tracking system 2108, tracking object 2114, surgical implant 2118, surgical instrument 2122, one or more databases 2130, and / or any other element described with respect to FIG. 21 or otherwise) of the system 2100 may include any of the elements of the computing system 2200 as well (such as the processor 2202, memory 2204, etc.).

[0172] FIG. 22 is a schematic block diagram of an illustrative computing system 2200 in accordance with one or more example embodiments of the disclosure. The computing system 2200 may include any suitable computing system capable of receiving and / or generating data including, but not limited to, a user device such as a smartphone, tablet, e- reader, wearable device, or the like; a desktop computer; a laptop computer; or the like. The computing system 2200 may correspond to an illustrative device configuration for the devices of FIGS. 1-21 (for example, computing system 112, computing system 1414, computing system 1708, user device 2102. computing system 2104, imaging device 2106, electromagnetic tracking system 2108, etc.).

[0173] The computing system 2200 may be configured to communicate via one or more networks with one or more servers, search engines, user devices, or the like. In some embodiments, a single remote server or single group of remote servers may be configured to perform more than one t pe of content rating and / or machine learning functionality.

[0174] Example network(s) may include, but are not limited to, any one or more different types of communications networks such as, for example, cable networks, public networks (e.g., the Internet), private networks (e.g., frame-relay networks), wireless networks, cellular networks, telephone networks (e.g., a public switched telephone network), or any other suitable private or public packet-switched or circuit-switched networks. Further, such network(s) may have any suitable communication range associated therewith and may include, for example, global networks (e.g., the Internet), metropolitan area networks (MANs), wide area networks (WANs), local area networks (LANs), orpersonal area networks (PANs). In addition, such network(s) may include communication links and associated networking devices (e.g., link-layer switches, routers, etc.) for transmitting network traffic over any suitable ty pe of medium including, but not limited to, coaxial cable, twisted-pair wire (e.g., twisted-pair copper wire), optical fiber, a hybrid fibercoaxial (HFC) medium, a microwave medium, a radio frequency communication medium, a satellite communication medium, or any combination thereof.

[0175] In an illustrative configuration, the computing system 2200 may include one or more processors (processor(s)) 2202, one or more memory’ devices 2204 (generically referred to herein as memory 2204), one or more input / output (I / O) interface(s) 2206, one or more network interface(s) 2208, one or more sensors or sensor interface(s) 2210, one or more transceivers 2212, one or more optional speakers 2214, one or more optional microphones 2216, and data storage 2220. The computing system 2200 may further include one or more buses 2218 that functionally couple various components of the computing system 2200. The computing system 2200 may further include one or more antenna(e) 2234 that may include, without limitation, a cellular antenna for transmitting or receiving signals to / from a cellular network infrastructure, an antenna for transmitting or receiving Wi-Fi signals to / from an access point (AP), a Global Navigation Satellite System (GNSS) antenna for receiving GNSS signals from a GNSS satellite, a Bluetooth antenna for transmitting or receiving Bluetooth signals, a Near Field Communication (NFC) antenna for transmitting or receiving NFC signals, and so forth. These various components will be described in more detail hereinafter.

[0176] The bus(es) 2218 may include at least one of a system bus, a memory bus, an address bus, or a message bus, and may permit exchange of information (e.g., data (including computer-executable code), signaling, etc.) between various components of the computing system 2200. The bus(es) 2218 may include, without limitation, a memory bus or a memory controller, a peripheral bus, an accelerated graphics port, and so forth. The bus(es) 2218 may be associated with any suitable bus architecture including, without limitation, an Industry’ Standard Architecture (ISA), a Micro Channel Architecture (MCA), an Enhanced ISA (EISA), a Video Electronics Standards Association (VESA) architecture, an Accelerated Graphics Port (AGP) architecture, a Peripheral Component Interconnects (PCI) architecture, a PCI-Express architecture, a Personal Computer Memory Card International Association (PCMCIA) architecture, a Universal Serial Bus (USB) architecture, and so forth.

[0177] The memory 2204 of the computing system 2200 may include volatile memory (memory that maintains its state when supplied with power) such as random access memory7(RAM) and / or non-volatile memory (memory that maintains its state even when not supplied with power) such as read-only memory (ROM), flash memory, ferroelectric RAM (FRAM), and so forth. Persistent data storage, as that term is used herein, may include nonvolatile memory. In certain example embodiments, volatile memory may enable faster read / write access than non-volatile memory. However, in certain other example embodiments, certain types of non-volatile memory7(e.g., FRAM) may enable faster read / write access than certain types of volatile memory'.

[0178] In various implementations, the memory72204 may include multiple different types of memory such as various types of static random access memory7(SRAM), various types of dynamic random access memory (DRAM), various types of unalterable ROM, and / or writeable variants of ROM such as electrically erasable programmable read-only memory7(EEPROM), flash memory, and so forth. The memory 2204 may include main memory as well as various forms of cache memory such as instruction cache(s), data cache(s), translation lookaside buffer(s) (TLBs), and so forth. Further, cache memory such as a data cache may be a multi-level cache organized as a hierarchy of one or more cache levels (LI, L2, etc.).

[0179] The data storage 2220 may include removable storage and / or non-removable storage including, but not limited to, magnetic storage, optical disk storage, and / or tape storage. The data storage 2220 may provide non-volatile storage of computer-executable instructions and other data. The memory 2204 and the data storage 2220, removable and / or non-removable, are examples of computer-readable storage media (CRSM) as that term is used herein.

[0180] The data storage 2220 may store computer-executable code, instructions, or the like that may be loadable into the memory 2204 and executable by the processor(s) 2202 to cause the processor(s) 2202 to perform or initiate various operations. The data storage 2220 may additionally store data that may be copied to memory 2204 for use by the processor(s) 2202 during the execution of the computer-executable instructions. Moreover, output data generated as a result of execution of the computer-executable instructions by the processor(s) 2202 may be stored initially in memory 2204, and may ultimately be copied to data storage 2220 for non-volatile storage.

[0181] More specifically, the data storage 2220 may store one or more operating systems (O / S) 2222; one or more database management systems (DBMS) 2224; and one or more program module(s), applications, engines, computer-executable code, scripts, or the like such as. for example, one or more module(s) 2226. Any of the components depicted as being stored in data storage 2220 may include any combination of software, firmware, and / or hardware. The software and / or firmware may include computer-executable code, instructions, or the like that may be loaded into the memory 2204 for execution by one or more of the processor(s) 2202. Any of the components depicted as being stored in data storage 2220 may support functionality described in reference to correspondingly named components earlier in this disclosure.

[0182] The data storage 2220 may further store various types of data utilized by components of the computing system 2200. Any data storage in the data storage 2220 may be loaded into the memory 2204 for use by the processor(s) 2202 in executing computerexecutable code. In addition, any data depicted as being stored in the data storage 2220 may potentially be stored in one or more datastore(s) and may be accessed via the DBMS 2224 and loaded in the memory 2204 for use by the processor(s) 2202 in executing computerexecutable code. The datastore(s) may include, but are not limited to. databases (e.g.. relational, object-oriented, etc ), file systems, flat files, distributed datastores in which data is stored on more than one node of a computer network, peer-to-peer network datastores, or the like. In FIG. 22, the datastore(s) may include pre-generated content data segments and / or summaries, data arrays including starting and stopping points and pointers to the content data segments and / or summaries, previously-generated recaps, etc.

[0183] The processor(s) 2202 may be configured to access the memory' 2204 and execute computer-executable instructions loaded therein. For example, the processor(s) 2202 may be configured to execute computer-executable instructions of the various program module(s), applications, engines, or the like of the computing system 2200 to cause or facilitate various operations to be performed in accordance with one or more embodiments of the disclosure. The processor(s) 2202 may include any suitable processing unit capable of accepting data as input, processing the input data in accordance with stored computerexecutable instructions, and generating output data. The processor(s) 2202 may include any type of suitable processing unit including, but not limited to, a central processing unit, a microprocessor, a Reduced Instruction Set Computer (RISC) microprocessor, a Complex Instruction Set Computer (CISC) microprocessor, a microcontroller, an ApplicationSpecific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), a System- on-a-Chip (SoC), a digital signal processor (DSP), and so forth. Further, the processor(s) 2202 may have any suitable microarchitecture design that includes any number of constituent components such as, for example, registers, multiplexers, arithmetic logic units, cache controllers for controlling read / write operations to cache memory, branch predictors, or the like. The microarchitecture design of the processor(s) 2202 may be capable of supporting any of a variety of instruction sets.

[0184] Referring now to functionality supported by the various program module(s) depicted in FIG. 22, the module(s) 2226 may include computer-executable instructions, code, or the like that responsive to execution by one or more of the processor(s) 2202 may perform functions including, but not limited to, providing real-time visual guidance during a surgical procedure.

[0185] Referring now to other illustrative components depicted as being stored in the data storage 2220, the O / S 2222 may be loaded from the data storage 2220 into the memory 2204 and may provide an interface between other application software executing on the computing system 2200 and hardware resources of the computing system 2200. More specifically, the O / S 2222 may include a set of computer-executable instructions for managing hardware resources of the computing system 2200 and for providing common services to other application programs (e.g., managing memory' allocation among various application programs). In certain example embodiments, the O / S 2222 may control execution of the other program module(s) to dynamically enhance characters for content rendering. The O / S 2222 may include any operating system now known or which may be developed in the future including, but not limited to, any server operating system, any mainframe operating system, or any other proprietary' or non-proprietary operating system.

[0186] The DBMS 2224 may be loaded into the memory 2204 and may support functionality for accessing, retrieving, storing, and / or manipulating data stored in the memory' 2204 and / or data stored in the data storage 2220. The DBMS 2224 may use any of a variety of database models (e.g.. relational model, object model, etc.) and may support any of a variety of query languages. The DBMS 2224 may access data represented in one or more data schemas and stored in any suitable data repository including, but not limited to, databases (e.g., relational, object-oriented, etc.), file systems, flat files, distributed datastores in which data is stored on more than one node of a computer network, peer-to-peer network datastores, or the like. In those example embodiments in which the computing system 2200is a user device, the DBMS 2224 may be any suitable light-weight DBMS optimized for performance on a user device.

[0187] Referring now to other illustrative components of the computing system 2200, the input / output (I / O) interface(s) 2206 may facilitate the receipt of input information by the computing system 2200 from one or more I / O devices as well as the output of information from the computing system 2200 to the one or more I / O devices. The I / O devices may include any of a variety of components such as a display or display screen having a touch surface or touchscreen; an audio output device for producing sound, such as a speaker; an audio capture device, such as a microphone; an image and / or video capture device, such as a camera; a haptic unit; and so forth. Any of these components may be integrated into the computing system 2200 or may be separate. The I / O devices may further include, for example, any number of peripheral devices such as data storage devices, printing devices, and so forth.

[0188] The I / O interface(s) 2206 may also include an interface for an external peripheral device connection such as universal serial bus (USB), FireWire, Thunderbolt, Ethernet port or other connection protocol that may connect to one or more networks. The I / O interface(s) 2206 may also include a connection to one or more of the antenna(e) 2234 to connect to one or more networks via a wireless local area network (WLAN) (such as Wi-Fi) radio, Bluetooth, ZigBee, and / or a wireless network radio, such as a radio capable of communication with a wireless communication network such as a Long Term Evolution (LTE) network. WiMAX network, 3G network, ZigBee network, etc.

[0189] The computing system 2200 may further include one or more network interface(s) 2208 via which the computing system 2200 may communicate with any of a variety' of other systems, platforms, networks, devices, and so forth. The network interface(s) 2208 may enable communication, for example, with one or more wireless routers, one or more host servers, one or more web servers, and the like via one or more of networks.

[0190] The antenna(e) 2234 may include any suitable type of antenna depending, for example, on the communications protocols used to transmit or receive signals via the antenna(e) 2234. Non-limiting examples of suitable antennas may include directional antennas, non-directional antennas, dipole antennas, folded dipole antennas, patch antennas, multiple-input multiple-output (MIMO) antennas, or the like. The antenna(e) 2234 may becommunicatively coupled to one or more transceivers 2212 or radio components to which or from which signals may be transmitted or received.

[0191] As previously described, the antenna(e) 2234 may include a cellular antenna configured to transmit or receive signals in accordance with established standards and protocols, such as Global System for Mobile Communications (GSM), 3G standards (e.g., Universal Mobile Telecommunications System (UMTS), Wideband Code Division Multiple Access (W-CDMA), CDMA2000, etc.), 4G standards (e.g., Long-Term Evolution (LTE), WiMax. etc.), direct satellite communications, or the like.

[0192] The antenna(e) 2234 may additionally, or alternatively, include a Wi-Fi antenna configured to transmit or receive signals in accordance with established standards and protocols, such as the IEEE 802.11 family of standards, including via 2.4 GHz channels (e.g., 802.11b, 802.11g, 802.1 In), 5 GHz channels (e.g., 802.1 In, 802.1 lac), or 60 GHz channels (e.g., 802. Had). In alternative example embodiments, the antenna(e) 2234 may be configured to transmit or receive radio frequency signals within any suitable frequency range forming part of the unlicensed portion of the radio spectrum.

[0193] The antenna(e) 2234 may additionally, or alternatively, include a GNSS antenna configured to receive GNSS signals from three or more GNSS satellites carrying timeposition information to triangulate a position therefrom. Such a GNSS antenna may be configured to receive GNSS signals from any current or planned GNSS such as, for example, the Global Positioning System (GPS), the GLONASS System, the Compass Navigation System, the Galileo System, or the Indian Regional Navigational System.

[0194] The transceiver(s) 2212 may include any suitable radio component(s) for - in cooperation with the antenna(e) 2234 - transmitting or receiving radio frequency (RF) signals in the bandwidth and / or channels corresponding to the communications protocols utilized by the computing system 2200 to communicate with other devices. The transceiver(s) 2212 may include hardware, software, and / or firmware for modulating, transmitting, or receiving - potentially in cooperation with any of antenna(e) 2234 - communications signals according to any of the communications protocols discussed above including, but not limited to, one or more Wi-Fi and / or Wi-Fi direct protocols, as standardized by the IEEE 802. 11 standards, one or more non-Wi-Fi protocols, or one or more cellular communications protocols or standards. The transceiver(s) 2212 may further include hardware, firmware, or software for receiving GNSS signals. The transceiver(s) 2212 may include any known receiver and baseband suitable for communicating via thecommunications protocols utilized by the computing system 2200. The transceiver(s) 2212 may further include a low noise amplifier (LNA), additional signal amplifiers, an analog- to-digital (A / D) converter, one or more buffers, a digital baseband, or the like.

[0195] The sensor(s) / sensor interface(s) 2210 may include or may be capable of interfacing with any suitable type of sensing device such as, for example, inertial sensors, force sensors, thermal sensors, and so forth. Example types of inertial sensors may include accelerometers (e g., MEMS-based accelerometers), gy roscopes, and so forth.

[0196] The optional speaker(s) 2214 may be any device configured to generate audible sound. The optional microphone(s) 2216 may be any device configured to receive analog sound input or voice data.

[0197] It should be appreciated that the program module(s), applications, computerexecutable instructions, code, or the like depicted in FIG. 22 as being stored in the data storage 2220 are merely illustrative and not exhaustive and that processing described as being supported by any particular module may alternatively be distributed across multiple module(s) or performed by a different module. In addition, various program module(s), script(s), plug-in(s), Application Programming Interface(s) (API(s)), or any other suitable computer-executable code hosted locally on the computing system 2200, and / or hosted on other computing system(s) accessible via one or more networks, may be provided to support functionality' provided by the program module(s), applications, or computer-executable code depicted in FIG. 22 and / or additional or alternate functionality. Further, functionality may be modularized differently such that processing described as being supported collectively by the collection of program module(s) depicted in FIG. 22 may be performed by a fewer or greater number of module(s), or functionality' described as being supported by any particular module may be supported, at least in part, by another module. In addition, program module(s) that support the functionality described herein may form part of one or more applications executable across any number of systems or devices in accordance with any suitable computing model such as, for example, a client-server model, a peer-to-peer model, and so forth. In addition, any of the functionality' described as being supported by any of the program module(s) depicted in FIG. 22 may be implemented, at least partially, in hardware and / or firmware across any number of devices.

[0198] It should further be appreciated that the computing system 2200 may include alternate and / or additional hardware, software, or firmware components beyond those described or depicted without departing from the scope of the disclosure. More particularly,it should be appreciated that software, firmware, or hardware components depicted as forming part of the computing system 2200 are merely illustrative and that some components may not be present or additional components may be provided in various embodiments. While various illustrative program module(s) have been depicted and described as software module(s) stored in data storage 2220, it should be appreciated that functionality described as being supported by the program module(s) may be enabled by any combination of hardware, software, and / or firmware. It should further be appreciated that each of the above-mentioned module(s) may, in various embodiments, represent a logical partitioning of supported functionality. This logical partitioning is depicted for ease of explanation of the functionality and may not be representative of the structure of software, hardware, and / or firmware for implementing the functionality. Accordingly, it should be appreciated that functionality described as being provided by a particular module may, in various embodiments, be provided at least in part by one or more other module(s). Further, one or more depicted module(s) may not be present in certain embodiments, while in other embodiments, additional module(s) not depicted may be present and may support at least a portion of the described functionality and / or additional functionality. Moreover, while certain module(s) may be depicted and described as sub-module(s) of another module, in certain embodiments, such module(s) may be provided as independent module(s) or as sub- module(s) of other module(s).

[0199] Program module(s), applications, or the like disclosed herein may include one or more software components including, for example, software objects, methods, data structures, or the like. Each such software component may include computer-executable instructions that, responsive to execution, cause at least a portion of the functionality described herein (e.g., one or more operations of the illustrative methods described herein) to be performed.

[0200] A software component may be coded in any of a variety of programming languages. An illustrative programming language may be a lower-level programming language such as an assembly language associated with a particular hardware architecture and / or operating system platform. A software component comprising assembly language instructions may require conversion into executable machine code by an assembler prior to execution by the hardware architecture and / or platform.

[0201] Another example programming language may be a higher-level programming language that may be portable across multiple architectures. A software componentcomprising higher-level programming language instructions may require conversion to an intermediate representation by an interpreter or a compiler prior to execution.

[0202] Other examples of programming languages include, but are not limited to, a macro language, a shell or command language, a job control language, a script language, a database query or search language, or a report writing language. In one or more example embodiments, a software component comprising instructions in one of the foregoing examples of programming languages may be executed directly by an operating system or other software component without having to be first transformed into another form.

[0203] A software component may be stored as a file or other data storage construct. Software components of a similar type or functionally related may be stored together such as, for example, in a particular directory, folder, or library. Software components may be static (e.g., pre-established or fixed) or dynamic (e.g., created or modified at the time of execution).

[0204] Software components may invoke or be invoked by other software components through any of a wide variety' of mechanisms. Invoked or invoking software components may comprise other custom-developed application software, operating system functionality (e.g., device drivers, data storage (e.g., file management) routines, other common routines and services, etc.), or third-party software components (e.g., middleware, encry ption, or other security software, database management software, file transfer or other network communication software, mathematical or statistical software, image processing software, and format translation software).

[0205] Software components associated with a particular solution or system may reside and be executed on a single platform or may be distributed across multiple platforms. The multiple platforms may be associated with more than one hardware vendor, underlying chip technology, or operating system. Furthermore, software components associated with a particular solution or system may be initially written in one or more programming languages, but may invoke software components written in another programming language.

[0206] Computer-executable program instructions may be loaded onto a specialpurpose computer or other particular machine, a processor, or other programmable data processing apparatus to produce a particular machine, such that execution of the instructions on the computer, processor, or other programmable data processing apparatus causes one or more functions or operations specified in the flow diagrams to be performed. These computer program instructions may also be stored in a computer-readable storage medium(CRSM) that upon execution may direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement one or more functions or operations specified in the flow diagrams. The computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational elements or steps to be performed on the computer or other programmable apparatus to produce a computer- implemented process.

[0207] Additional types of CRSM that may be present in any of the devices described herein may include, but are not limited to, programmable random access memory (PRAM), SRAM, DRAM, RAM, ROM, electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the information and which can be accessed. Combinations of any of the above are also included within the scope of CRSM. Alternatively, computer-readable communication media (CRCM) may include computer- readable instructions, program module(s), or other data transmitted within a data signal, such as a carrier wave, or other transmission. However, as used herein, CRSM does not include CRCM.

[0208] Although embodiments have been described in language specific to structural features and / or methodological acts, it is to be understood that the disclosure is not necessarily limited to the specific features or acts described. Rather, the specific features and acts are disclosed as illustrative forms of implementing the embodiments. Conditional language, such as, among others, “can ’ “could,’' “might,” or “may.” unless specifically stated otherwise, or otherwise understood within the context as used, is generally intended to convey that certain embodiments could include, while other embodiments do not include, certain features, elements, and / or steps. Thus, such conditional language is not generally intended to imply that features, elements, and / or steps are in any way required for one or more embodiments or that one or more embodiments necessarily include logic for deciding, with or without user input or prompting, whether these features, elements, and / or steps are included or are to be performed in any particular embodiment.

[0209] Example Embodiments

[0210] Embodiment 1. A system for providing real-time guidance during a surgical procedure performed on a patient, the system comprising: an electromagnetic field generator configured to generate an electromagnetic field within a navigation volume during the surgical procedure; a first electromagnetic sensor configured to be disposed at least one of: on or within a bone of the patient; a second electromagnetic sensor configured to be disposed at least one of: on or within a surgical instrument; and a computing system in electrical communication with the one or more electromagnetic sensors, the computing system comprising a processor configured to: receive data from the first electromagnetic sensor and the second electromagnetic sensor; and cause to display, in real-time and based on the data, visual information.

[0211] Embodiment 2. The system of Embodiment 1, further comprising a third electromagnetic sensor configured to be disposed at least one of: on or within a surgical implant.

[0212] Embodiment 3. The system of Embodiment 2, wherein the second electromagnetic sensor has a height, length, and width of less than 100mm, and wherein the second electromagnetic sensor is configured to produce the data with positional accuracy below than ,9mm.

[0213] Embodiment 4. The system of any of Embodiments 1-3, wherein the first electromagnetic sensor is disposed at least one of: on or within an object that is partially inserted into the bone of the patient.

[0214] Embodiment 5. The system of any of Embodiments 1-4, wherein the processor is further configured to: receive, from an imaging device, a three-dimensional model of the bone of the patient; and generate, in real-time during the surgical procedure, a modified three-dimensional model of the bone of the patient based on a combination of the three- dimensional model and the data, wherein causing cause to display the visual information further comprises causing to display the visual information the modified three-dimensional

[0215] Embodiment 6. The system of any of Embodiments 1-5, wherein the visual information indicates a position of the bone and a position of the surgical instrument in realtime.

[0216] Embodiment 7. The system of any of Embodiments 106, wherein the one or more visual elements include at least one of: an indication of a region of bone to be cut using the surgical instrument, a first line indicative of a projected placement of the surgical implant within the patient and a second line indicative of a desired placement of the surgical implantwithin the patient, or a region to which a bone is to be translated during the surgical procedure.

[0217] Embodiment 8. The system for any of Embodiment 1-7, wherein the surgical instrument is a guide wire comprising a distal end and a proximal end, wherein a sensor is disposed at the distal end of the guide wire.

[0218] Embodiment 9. The system of any of Embodiments 1-8, wherein the guide w ire comprises an internal cavity, and wherein the sensor is disposed within the internal cavity at the distal end.

[0219] Embodiment 10. The system of any of Embodiments 1-9, further comprising: a first wired connection configured to be connected to the sensor; and a second wired connection configured to be connected to a processor, wherein the first wired connection is configured to rotate relative to the second wired connection and is also configured to transmit signals to the second wired connection during rotation.

[0220] Embodiment 11. The system of any of Embodiments 1-10, further comprising: a slip ring, wherein the first wired connection is configured to transmit signals to the second wired connection during rotation via the slip ring.

[0221] Embodiment 12. The system of any of Embodiments 1-11, wherein the sensor comprises an outer portion and an inner portion, wherein the outer portion is configured to rotate with the guide wire relative to the inner portion.

[0222] Embodiment 13. The system of Embodiment 8, wherein the sensor is a wireless sensor.

[0223] Embodiment 14. The system of any of Embodiments 1-13, further comprising: one or more fiducial markers disposed within the navigation volume.

[0224] Embodiment 15. A method for providing real-time guidance during a surgical procedure performed on a patient, the method comprising: disposing a first electromagnetic sensor at least one of: on or within a bone of the patient; disposing a second electromagnetic sensor at least one of: on or within a surgical instrument; generating, by an electromagnetic field generator, an electromagnetic field within a navigation volume during the surgical procedure; receiving, by a computing system, data from the first electromagnetic sensor and the second electromagnetic sensor; and causing to display, by the computing system, in realtime, and based on the data, visual information.

[0225] Embodiment 16. The method of Embodiment 15, further comprising: receiving, from an imaging device, a three-dimensional model of the bone of the patient; and generating, in real-time during the surgical procedure, a modified three-dimensional model of the bone of the patient based on a combination of the three-dimensional model and the data, wherein causing cause to display the visual information further comprises causing to display the visual information the modified three-dimensional model of the bone that is augmented with one or more visual elements based on the data.

[0226] Embodiment 17. The method of Embodiment 16, further comprising: disposing a third electromagnetic sensor at least one of: on or within a surgical implant.

[0227] Embodiment 18. The method of any of Embodiments 15-17, wherein the second electromagnetic sensor has a height, length, and width of less than 100mm, and wherein the second electromagnetic sensor is configured to produce the data with positional accuracy below than ,9mm.

[0228] Embodiment 19. The method of any of Embodiments 15-18, wherein the surgical instrument is a guide wire comprising a distal end and a proximal end, wherein a sensor is disposed at the distal end of the guide wire.

[0229] Embodiment 20. The method of any of Embodiments 15-19, further comprising: disposing one or more fiducial markers within the navigation volume.

[0230] Embodiment 21. A system for providing real-time visual guidance during a surgical procedure, the system comprising: a guide wire comprising an internal cavity, a distal end, and a proximal end; and a sensor disposed within the internal cavity about the distal end.

Claims

CLAIMSTHAT WHICH IS CLAIMED IS :

1. A system for providing real-time guidance during a surgical procedure performed on a patient, the system comprising: an electromagnetic field generator configured to generate an electromagnetic field within a navigation volume during the surgical procedure; a first electromagnetic sensor configured to be disposed at least one of: on or within a bone of the patient; a second electromagnetic sensor configured to be disposed at least one of: on or within a surgical instrument; and a computing system in electrical communication with the one or more electromagnetic sensors, the computing system comprising a processor configured to: receive data from the first electromagnetic sensor and the second electromagnetic sensor; and cause to display, in real-time and based on the data, visual information.

2. The system of claim 1, further comprising: a third electromagnetic sensor configured to be disposed at least one of: on or within a surgical implant.

3. The system of claim 2, wherein the second electromagnetic sensor has a height, length, and width of less than 100mm, and wherein the second electromagnetic sensor is configured to produce the data with positional accuracy below than .9mm.

4. The system of any of claims 1-3, wherein the first electromagnetic sensor is disposed at least one of: on or within an object that is partially inserted into the bone of the patient.

5. The system of any of claims 1-4, wherein the processor is further configured to: receive, from an imaging device, a multi-dimensional model of the bone of the patient; and superimpose, in real-time during the surgical procedure, three-dimensional data on the multi-dimensional model of the bone of the patient.

6. The system of any of claims 1-5, wherein the visual information indicates a position of the bone and a position of the surgical instrument in real-time.

7. The system of any of claims 1-6, wherein the one or more visual elements include at least one of: an indication of a region of bone to be cut using the surgical instrument, a first line indicative of a projected placement of the surgical implant within the patient and a second line indicative of a desired placement of the surgical implant within the patient, or a region to which a bone is to be translated during the surgical procedure.

8. The system of any of claims 1-7, wherein the surgical instrument is a guide wire comprising a distal end and a proximal end, wherein a sensor is disposed at the distal end of the guide wire.

9. The system of any of claims 1-8, wherein the guide wire comprises an internal cavity, and wherein the sensor is disposed within the internal cavity at the distal end.

10. The system of any of claims 1-9, further comprising: a first wired connection configured to be connected to the sensor; and a second wired connection configured to be connected to a processor, wherein the first wired connection is configured to rotate relative to the second wired connection and is also configured to transmit signals to the second wired connection during rotation.

11. The system of any of claims 1-10, further comprising: a slip ring, wherein the first wired connection is configured to transmit signals to the second wired connection during rotation via the slip ring.

12. The system of any of claims 1-11, wherein the sensor comprises an outer portion and an inner portion, wherein the outer portion is configured to rotate with the guide wire relative to the inner portion.

13. The system of claim 8, wherein the sensor is a wireless sensor.

14. The system of any of claims 1 -13, further comprising: one or more fiducial markers disposed within the navigation volume.

15. A method for providing real-time guidance during a surgical procedure performed on a patient, the method comprising: disposing a first electromagnetic sensor at least one of: on or within a bone of the patient; disposing a second electromagnetic sensor at least one of: on or within a surgical instrument; generating, by an electromagnetic field generator, an electromagnetic field within a navigation volume during the surgical procedure; receiving, by a computing system, data from the first electromagnetic sensor and the second electromagnetic sensor; and causing to display, by the computing system, in real-time, and based on the data, visual information.

16. The method of claim 15, further comprising: receiving, from an imaging device, a multi-dimensional model of the bone of the patient; and superimposing, in real-time during the surgical procedure, three-dimensional data on the multi-dimensional model of the bone of the patient.

17. The method of claim 16, further comprising: disposing a third electromagnetic sensor at least one of: on or within a surgical implant.

18. The method of any of claims 15-17, wherein the second electromagnetic sensor has a height, length, and width of less than 100mm, and wherein the second electromagnetic sensor is configured to produce the data with positional accuracy below than ,9mm.

19. The method of any of claims 1 -18, wherein the surgical instrument is a guide wire comprising a distal end and a proximal end, wherein a sensor is disposed at the distal end of the guide wire.

20. The method of any of claims 15-19, further comprising: disposing one or more fiducial markers within the navigation volume.

21. A system for providing real-time visual guidance during a surgical procedure, the system comprising: a guide wire comprising an internal cavity, a distal end, and a proximal end; and a sensor disposed within the internal cavity about the distal end.

Images