Easy-to-manufacture, autoclavable LEDs for optical tracking

Abstract

An optical tracking system is provided. The optical tracking system includes an autoclavable fiducial marker assembly comprising an opaque housing, a light source, a window panel configured to refract light from the light source passing therethrough, and a metallized coating that forms an airtight seal at the junction of the window panel and the opaque housing. The fiducial marker assembly is configured to shield a peripheral edge of the window panel from the light. The system also includes a tracking device comprising at least two optical sensors configured to detect positions of light rays emitted by the light source. The system also includes a processor configured to receive positions of the light rays from the optical sensors, shift the position of each light ray based on a calculated refractive deviation, and triangulate the positioning of the light source based on the shifted position of each light ray.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of priority to U.S. Provisional Application No. 63 / 049,319, filed on July 8, 2020, entitled “Easy to Manufacture Autoclavable LED for Optical Tracking,” the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present disclosure generally relates to methods, systems, and devices for optical tracking. The disclosed technology can be applied, for example, to shoulder, hip, and knee replacement surgeries, as well as other surgical procedures such as arthroscopic surgery, spinal surgery, maxillofacial surgery, rotator cuff surgery, and ligament repair and replacement surgery. More specifically, the present disclosure relates to systems and devices that include autoclavable LED markers configured to facilitate optical tracking with high positioning accuracy. Background Art

[0004] Optical tracking is commonly used in computer-assisted surgery (CAS) applications to help doctors locate and navigate to target tissue and treat the target tissue. Fiducial markers can be attached to an object of interest and used as optical references to be tracked by the system. In some applications, the fiducial markers include light-emitting diodes (LEDs) that emit light to be tracked by the system's optical tracking device, such as a camera. The system determines the location of the object of interest based on continuous monitoring of the emitted light.

[0005] Fiducial markers, which are typically packaged individually and coupled to a housing or support, can be designed for long-term and repeated use in a surgical environment. However, to be used in this manner, the fiducial markers must be designed to withstand autoclave cycles to meet the sterilization requirements of the operating room environment. While designs for sealed, autoclavable LEDs have been proposed (see, for example, DE 10 2015 103 331 B4 and DE 20 2010 000 518 U1), the encapsulation of the light source often results in undesirable optical effects. In general, designing autoclavable LEDs for optical tracking without sacrificing metrological accuracy has been difficult.

[0006] Now refer to Figure 8, shows a two-dimensional illustration of a sealed light source and the resulting positioning error. Fiducial marker 800 may include a housing 805, an exit window 810, and a light source 815, and may be used in conjunction with a stereoscopic optical tracking device 820 including an optical sensor 825 to triangulate the position of light source 815. However, when light source 815 emits light 830, the light may undergo refraction 830A upon passing through exit window 810 before exiting light 830B reaches optical sensor 825. Consequently, light 830 does not form a continuous straight line, and the amount of refraction 830A also varies based on the optical properties of exit window 810 and the tilt angle of fiducial marker 800 relative to each optical sensor 825. When the system assumes light 830 is a straight line 835, the calculated position 840 of light source 815 can deviate from the true position by a significant margin. Consequently, large metrological errors in the calculation may result in unacceptable positioning inaccuracies.

[0007] Current solutions include fiducial markers designed to mitigate refraction. For example, U.S. Patent No. 7,147,352 proposes a light source located at the spherical center of a domed exit window, such that light rays are substantially normal to the exit window in all directions. However, such designs require construction with very high levels of accuracy that exceed the mechanical tolerances of standard manufacturing techniques. Producing fiducial markers with the accuracy required to reduce positioning errors requires complex manufacturing and assembly techniques, resulting in very high manufacturing costs.

[0008] Therefore, it would be advantageous to have an autoclavable LED that facilitates simple manufacturing techniques and reduces positioning inaccuracies. Additionally, it would be advantageous to have a system that is configured to reduce metrology errors associated with light refraction. Summary of the Invention

[0009] An optical tracking system is provided. The optical tracking system includes an autoclavable fiducial marker assembly comprising an opaque housing defining an interior cavity, a light emitting semiconductor die disposed in the interior cavity and in electrical communication with an anode and a cathode, a window panel coupled to the opaque housing to enclose the interior cavity between the window panel and the opaque housing, the window panel configured to refract a plurality of light rays emitted by the light emitting semiconductor die, and a metallized coating forming an airtight seal at a junction of the window panel and the opaque housing, wherein the fiducial marker assembly is configured to shield a peripheral edge of the window panel from the plurality of light rays; a tracking device comprising at least two optical sensors, each configured to detect a position of a light ray from the plurality of light rays; a processor; and a non-transitory computer-readable medium storing instructions that, when executed, cause the processor to: receive a position of each light ray from each optical sensor, shift the position of each light ray based on a calculated refractive deviation, and triangulate the position of the light ray based on the shifted position of each light ray.

[0010] According to some embodiments, the calculated refractive deviation is based on a known refractive index of the window panel, a known thickness of the window panel, and an orientation of the fiducial marker assembly relative to each optical sensor. According to additional embodiments, the tracking device is configured to detect image information related to the orientation of the fiducial marker. According to additional embodiments, the optical tracking system further includes an accelerometer configured to detect and transmit orientation information related to the orientation of the fiducial marker.

[0011] According to some embodiments, the metallized coating forms a ring extending radially inward from the junction to cover a portion of the window panel, wherein the ring is configured to shield a peripheral edge of the window panel from the plurality of light rays.

[0012] According to some embodiments, the metallization coating includes solder.

[0013] According to some embodiments, the anode and the cathode extend through the opaque housing.

[0014] According to some embodiments, the opaque housing defines a recess configured to receive the window panel.

[0015] According to some embodiments, the window panel defines a recess configured to secure the light emitting diode within the sign support.

[0016] According to some embodiments, the fiducial marker assembly further comprises a rod that supports the light-emitting semiconductor die adjacent the window panel. According to additional embodiments, the rod comprises a heat sink configured to remove heat from the light-emitting semiconductor die. According to additional embodiments, the diameter of the light-emitting semiconductor die is greater than or equal to the diameter of the rod. According to additional embodiments, the coefficient of thermal expansion of each of the opaque housing, the window panel, and the rod is substantially equal. According to additional embodiments, the opaque housing comprises a nickel-cobalt-iron alloy; the window panel comprises aluminum oxide; and the rod comprises a nickel-cobalt-iron alloy.

[0017] According to some embodiments, the window panel includes a light absorbing layer secured to an inner face of the window panel facing the metallized coating.

[0018] An autoclavable fiducial marker assembly is also provided. The fiducial marker assembly includes an opaque housing defining an interior cavity; a light-emitting semiconductor die disposed in the interior cavity and in electrical communication with an anode and a cathode; a window panel bonded to the opaque housing to enclose the interior cavity between the window panel and the opaque housing, the window panel being configured to refract a plurality of light rays emitted by the light-emitting semiconductor die; and a metallized coating forming a hermetic seal at the interface between the window panel and the opaque housing, wherein the fiducial marker assembly is configured to shield a peripheral edge of the window panel from the plurality of light rays.

[0019] According to some embodiments, the metallized coating forms a ring extending radially inward from the junction to cover a portion of the window panel, wherein the ring is configured to shield a peripheral edge of the window panel from the plurality of light rays.

[0020] According to some embodiments, the metallization coating includes solder.

[0021] According to some embodiments, the anode and the cathode extend through the opaque housing.

[0022] According to some embodiments, the opaque housing defines a recess configured to receive the window panel.

[0023] According to some embodiments, the window panel defines a recess configured to secure the light emitting diode within the sign support.

[0024] According to some embodiments, the fiducial marker assembly further comprises a rod that supports the light-emitting semiconductor die adjacent the window panel. According to additional embodiments, the rod comprises a heat sink configured to remove heat from the light-emitting semiconductor die. According to additional embodiments, the diameter of the light-emitting semiconductor die is greater than or equal to the diameter of the rod. According to additional embodiments, the rod comprises a nickel-cobalt-iron alloy. According to additional embodiments, the coefficient of thermal expansion of each of the opaque housing, the window panel, and the rod is substantially equal.

[0025] According to some embodiments, the window panel includes a light absorbing layer secured to an inner face of the window panel facing the metallized coating.

[0026] According to some embodiments, the window panel comprises aluminum oxide.

[0027] According to some embodiments, the opaque casing comprises a nickel-cobalt-iron alloy.

[0028] A method for tracking an object is also provided. The method includes coupling a fiducial marker assembly to the object, wherein the fiducial marker assembly includes an opaque housing defining an interior cavity, a light-emitting semiconductor die disposed in the interior cavity and electrically connected to an anode and a cathode, a window panel bonded to the opaque housing to enclose the interior cavity between the window panel and the opaque housing, the window panel configured to refract a plurality of light rays emitted by the light-emitting semiconductor die, and a metallized coating forming an airtight seal at a junction of the window panel and the opaque housing, wherein the fiducial marker assembly is configured to shield a peripheral edge of the window panel from the object. the plurality of light rays being affected; each of two or more optical sensors detecting a position of a light ray among the plurality of light rays; a processor receiving the detected position of the light ray from each of the two or more optical sensors; the processor adjusting the detected position of each light ray from each of the two or more optical sensors based on the refractive deviation to obtain an adjusted position of each light ray; triangulating the position of the light emitting semiconductor die from the adjusted position of each light ray by the computing device; and calculating the position of the object by the computing device based on a known spatial relationship between the object and the light emitting semiconductor die. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments of the present disclosure and, together with the written description, serve to explain the principles, characteristics, and features of the present invention.

[0030] Figure 1 An operating room including an exemplary computer-assisted surgery system (CASS) is shown, according to an embodiment.

[0031] Figure 2 An example of an electromagnetic sensor device according to some embodiments is shown.

[0032] Figure 3A An alternative example of an electromagnetic sensor device having three perpendicular coils is shown in accordance with some embodiments.

[0033] Figure 3B An alternative example of an electromagnetic sensor device having two non-parallel fixed coils is shown in accordance with some embodiments.

[0034] Figure 3C An alternative example of an electromagnetic sensor device having two non-parallel separate coils is shown in accordance with some embodiments.

[0035] Figure 4 An example of an electromagnetic sensor device and a patient's bone is shown, according to some embodiments.

[0036] Figure 5A Illustrative control instructions provided by the surgical computer to other components of the CASS are shown, according to an embodiment.

[0037] Figure 5B Illustrative control instructions provided to a surgical computer by components of a CASS are shown, according to an embodiment.

[0038] Figure 5C An illustrative implementation is shown in which a surgical computer is connected to a surgical data server via a network according to an embodiment.

[0039] Figure 6 A surgical patient care system and illustrative data sources are shown, according to an embodiment.

[0040] Figure 7A An exemplary flow chart for determining a preoperative surgical plan according to an embodiment is shown.

[0041] Figure 7B An exemplary flow chart for determining a care period, including pre-operative, intra-operative, and post-operative actions, according to an embodiment is shown.

[0042] Figure 7C An illustrative graphical user interface is shown including an image depicting implant placement according to an embodiment.

[0043] Figure 8 An illustrative sealed light source and typical positioning errors are shown, according to an embodiment.

[0044] Figures 9A-9B Illustrative fiducial markers for tracking an object during a surgical procedure are shown in accordance with an embodiment.

[0045] Figure 10 Alternative fiducial markers for tracking a subject during a surgical procedure are shown in accordance with an embodiment.

[0046] Figures 11A-11B Alternative fiducial markers for tracking a subject during a surgical procedure are shown in accordance with an embodiment.

[0047] Figure 12 A block diagram of an illustrative system for tracking an object is shown in accordance with an embodiment.

[0048] Figure 13 An illustrative calculation of the positioning of a light source with refraction correction is shown, according to an embodiment.

[0049] Figures 14A-14C An exemplary evaluation of positioning error according to an embodiment is shown.

[0050] Figure 15 A flow chart of an illustrative method of tracking an object with fiducial markers is shown in accordance with an embodiment.

[0051] Figure 16 A block diagram of an exemplary data processing system is shown in which embodiments are implemented. DETAILED DESCRIPTION

[0052] The present disclosure is not limited to the particular systems, devices, and methods described, as these may vary. The terminology used in the description is for the purpose of describing particular versions or embodiments only and is not intended to limit the scope.

[0053] As used in this document, the singular forms "a," "an," and "the" include plural references unless the context clearly dictates otherwise. Unless otherwise defined, all technical and scientific terms used herein have the same meanings as commonly understood by one of ordinary skill in the art. Nothing in this disclosure should be construed as an admission that the embodiments described in this disclosure are not entitled to antedate the present disclosure by virtue of prior invention. As used in this document, the term "including" means "including but not limited to."

[0054] definition

[0055] For the purposes of this disclosure, the term "implant" is used to refer to a prosthetic device or structure that is manufactured to replace or enhance a biological structure. For example, in a total hip replacement procedure, a prosthetic acetabular cup (implant) is used to replace or enhance a patient's worn or damaged acetabulum. Although the term "implant" is generally considered to mean an artificial structure (as opposed to a transplant), for the purposes of this specification, an implant may include biological tissue or material that is transplanted to replace or enhance a biological structure.

[0056] For the purposes of this disclosure, the term "real time" is used to refer to computations or operations that are performed immediately upon an event occurring or input being received by an operating system. However, the use of the term "real time" is not intended to exclude operations that incur some delay between input and response, as long as the delay is an unintended consequence of the performance characteristics of the machine.

[0057] Although much of this disclosure refers to surgeons or other medical professionals by specific job titles or roles, nothing in this disclosure is intended to be limited to a specific job title or function. A surgeon or medical professional can include any doctor, nurse, medical professional, or technician. Any of these terms or job titles can be used interchangeably with users of the system disclosed herein unless expressly specified otherwise. For example, in some embodiments, reference to a surgeon can also apply to a technician or nurse.

[0058] The systems, methods, and apparatus disclosed herein are particularly well suited for use with surgical navigation systems (e.g., NAVIO is a registered trademark of BLUE BELTTECHNOLOGIES, Inc. of Pittsburgh, Pennsylvania, a subsidiary of SMITH & NEPHEW, Inc. of Memphis, Tennessee.

[0059] CASS Ecosystem Overview

[0060] Figure 1 An illustration of an example computer-assisted surgery system (CASS) 100 is provided in accordance with some embodiments. As described in further detail in the following sections, CASS uses computers, robotics, and imaging technology to assist surgeons in performing orthopedic surgical procedures, such as total knee arthroplasty (TKA) or total hip arthroplasty (THA). For example, a surgical navigation system can assist surgeons in locating a patient's anatomy, guiding surgical instruments, and implanting medical devices with high precision. Surgical navigation systems, such as CASS 100, often employ various forms of computing technology to perform a wide variety of standard and minimally invasive surgical procedures and techniques. Furthermore, these systems allow surgeons to more accurately plan, track, and navigate the placement of instruments and implants relative to the patient's body and to perform preoperative and intraoperative body imaging.

[0061] The effector platform 105 positions the surgical tools relative to the patient during surgery. The exact components of the effector platform 105 will vary depending on the embodiment being employed. For example, for knee surgery, the effector platform 105 may include an end effector 105B that holds the surgical tools or instruments during their use. The end effector 105B may be a handheld device or instrument (e.g., handpiece or cutting guide or clamp), or alternatively, the end effector 105B may include a device or instrument held or positioned by the robotic arm 105A. Figure 1 One robotic arm 105A is shown in the figure, but in some embodiments, there may be multiple devices. For example, there may be one robotic arm 105A on each side of the operating table T, or there may be two devices on one side of the operating table T. The robotic arm 105A can be mounted directly to the operating table T, located on a floor platform (not shown) next to the operating table T, mounted on a floor pole, or mounted on the wall or ceiling of the operating room. The floor platform can be fixed or movable. In a specific embodiment, the robotic arm 105A is mounted on a floor pole located between the patient's legs or feet. In some embodiments, the end effector 105B may include a suture holder or stapler to assist in closing the wound. In addition, in the case of two robotic arms 105A, the surgical computer 150 can drive the robotic arms 105A to work together to suture the wound when closed. Alternatively, the surgical computer 150 can drive one or more robotic arms 105A to suture the wound when closed.

[0062] The actuator platform 105 may include a limb positioner 105C for positioning a patient's limb during surgery. An example of a limb positioner 105C is the SMITH AND NEPHEW SPIDER2 system. The limb positioner 105C may be manually operated by the surgeon, or alternatively, may be operated based on instructions received from a surgical computer 150 (described below) to change the position of the limb. Although in Figure 1 One limb positioner 105C is shown in the figures, but in some embodiments there may be multiple devices. As an example, there may be one limb positioner 105C on each side of the operating table T, or there may be two devices on one side of the operating table T. The limb positioner 105C may be mounted directly to the operating table T, located on a floor platform (not shown) next to the operating table T, mounted on a pole, or mounted on a wall or ceiling of the operating room. In some embodiments, the limb positioner 105C may be used in unconventional ways, such as a retractor or a specific bone holder. As an example, the limb positioner 105C may include an ankle boot, a soft tissue clamp, a bone clamp, or a soft tissue retractor spoon, such as a hooked, curved, or angled blade. In some embodiments, the limb positioner 105C may include a suture holder to assist in closing a wound.

[0063] The actuator platform 105 may include tools such as a screwdriver, a light or laser to indicate an axis or plane, a level, a pin driver, a pin puller, a flatness checker, an indicator, fingers, or some combination thereof.

[0064] Resection device 110 ( Figure 1The resection device 110 is a device that is not shown in the figure and is used to perform bone or tissue resection using, for example, mechanical, ultrasonic, or laser techniques. Examples of the resection device 110 include drilling devices, deburring devices, oscillating sawing devices, vibrating impact devices, reamer, ultrasonic bone cutting devices, radiofrequency ablation devices, reciprocating motion devices (such as files or broaches), and laser ablation systems. In some embodiments, the resection device 110 is held and operated by the surgeon during surgery. In other embodiments, the actuator platform 105 can be used to hold the resection device 110 during use.

[0065] The effector platform 105 may also include a cutting guide or clamp 105D for guiding a saw or drill used to remove tissue during surgery. Such a cutting guide 105D may be integrally formed as part of the effector platform 105 or robotic arm 105A, or the cutting guide may be a separate structure that can be cooperatively and / or removably attached to the effector platform 105 or robotic arm 105A. The effector platform 105 or robotic arm 105A may be controlled by the CASS 100 to position the cutting guide or clamp 105D adjacent to the patient's anatomy according to a surgical plan developed preoperatively or intraoperatively, so that the cutting guide or clamp will produce precise bone cuts according to the surgical plan.

[0066] The tracking system 115 uses one or more sensors to collect real-time positional data for locating the patient's anatomy and surgical instruments. For example, for a TKA procedure, the tracking system can provide the position and orientation of the end effector 105B during the procedure. In addition to positional data, data from the tracking system 115 can also be used to infer the velocity / acceleration of the anatomy / instrument, which can be used for tool control. In some embodiments, the tracking system 115 can use a tracker array attached to the end effector 105B to determine the position and orientation of the end effector 105B. The position of the end effector 105B can be inferred based on the position and orientation of the tracking system 115 and the known relationship in three-dimensional space between the tracking system 115 and the end effector 105B. Various types of tracking systems can be used in various embodiments of the present invention, including but not limited to infrared (IR) tracking systems, electromagnetic (EM) tracking systems, video or image-based tracking systems, and ultrasound registration and tracking systems. Using the data provided by the tracking system 115, the surgical computer 150 can detect objects and prevent collisions. For example, the surgical computer 150 can prevent the robotic arm 105A and / or end effector 105B from colliding with soft tissue.

[0067] Any suitable tracking system can be used to track surgical objects and patient anatomy in the operating room. For example, a combination of infrared and visible light cameras can be used in an array. Various lighting sources (e.g., infrared LED light sources) can illuminate the scene, allowing for three-dimensional imaging. In some embodiments, this can include stereoscopic, three-view, four-view, and other imaging. In addition to the camera array fixed to the cart in some embodiments, additional cameras can be placed throughout the operating room. For example, a handheld tool or headgear worn by the operator / surgeon can include imaging capabilities that transmit images back to a central processor to correlate those images with images acquired by the camera array. This can provide more robust images for environments modeled using multiple perspectives. In addition, some imaging devices can have appropriate resolution or viewing angles on the scene to pick up information stored in quick response (QR) codes or barcodes. This helps identify specific objects that have not been manually registered with the system. In some embodiments, the camera can be mounted on the robotic arm 105A.

[0068] As discussed herein, although most tracking and / or navigation technologies utilize image-based tracking systems (e.g., IR tracking systems, video or image-based tracking systems, etc.), electromagnetic (EM)-based tracking systems are becoming increasingly common for a variety of reasons. For example, implantation of standard optical trackers requires tissue excision (e.g., down to the cortex) and subsequent drilling and driving of cortical pins. Additionally, because optical trackers require direct line of sight with the tracking system, placement of such trackers may need to be away from the surgical site to ensure that they do not restrict the movement of the surgeon or medical professional.

[0069] Typically, an EM-based tracking device includes one or more coils and a reference field generator. The one or more coils can be powered (e.g., via a wired or wireless power source). Once powered, the coils generate an electromagnetic field that can be detected and measured (e.g., by a reference field generator or an attached device) in a manner that allows the position and orientation of the one or more coils to be determined. As will be understood by one of ordinary skill in the art, for example, Figure 2 The single coil shown in FIG is limited to detecting five (5) total degrees of freedom (DOF). For example, the sensor 200 is capable of tracking / determining movement in the X, Y, or Z directions, as well as rotation about the Y axis 202 or the Z axis 201. However, due to the electromagnetic properties of the coil, it is not possible to accurately track rotational motion about the X axis.

[0070] Therefore, in most electromagnetic tracking applications, such as Figure 3AThe three-coil system shown in is used to achieve tracking in all six degrees of freedom (i.e., forward / backward 310, up / down 320, left / right 330, roll 340, pitch 350, and yaw 360) that can move a rigid body in three-dimensional space. However, the inclusion of two additional coils and their positioning at a 90° offset angle may require the tracking device to be much larger. Alternatively, as known to those skilled in the art, fewer than three complete coils may be used to track all 6DOF. In some EM-based tracking devices, the two coils may be fixed to each other, for example Figure 3B Since the two coils 301B, 302B are rigidly fixed to each other, are not completely parallel, and have known positions relative to each other, a sixth degree of freedom 303B can be determined using this arrangement.

[0071] Although the use of two fixed coils (e.g., 301B, 302B) allows for the use of EM-based tracking in 6DOF, the diameter of the sensor device is much larger than a single coil due to the additional coils. Therefore, practical applications of using an EM-based tracking system in a surgical setting may require tissue removal and drilling of a portion of the patient's bone to allow for insertion of the EM tracker. Alternatively, in some embodiments, a single coil or 5DOF EM tracking device can be implanted / inserted into the patient's bone using only pins (e.g., without the need for drilling or resection of a large amount of bone).

[0072] Therefore, as described herein, a solution is needed that can limit the use of EM tracking systems to devices that are small enough to be inserted / embedded using a small diameter needle or pin (i.e., without the need to make a new incision or large diameter opening in the bone). Therefore, in some embodiments, a second 5DOF sensor that is not attached to the first sensor and therefore has a small diameter can be used to track all 6DOF. Now referring to Figure 3C In some embodiments, two 5DOF EM sensors (eg, 301C and 302C) may be inserted into a patient (eg, a patient's bone) at different locations and with different angular orientations (eg, angle 303C is non-zero).

[0073] Now refer to Figure 4, shows an example embodiment of inserting a first 5DOF EM sensor 401 and a second 5DOF EM sensor 402 into a patient's bone 403 using a standard hollow needle 405 typical of most ORs. In another embodiment, the first sensor 401 and the second sensor 402 may have an angular offset "α" 404. In some embodiments, the offset angle "α" 404 may need to be greater than a predetermined value (e.g., a minimum angle of 0.50°, 0.75°, etc.). In some embodiments, this minimum value may be determined by CASS during surgical planning and provided to the surgeon or medical professional. In some embodiments, the minimum value may be based on one or more factors, such as the orientation accuracy of the tracking system, the distance between the first EM sensor and the second EM sensor, the location of the field generator, the location of the field detector, the type of EM sensor, the quality of the EM sensor, the patient's anatomy, and the like.

[0074] Thus, as discussed herein, in some embodiments, a pin / needle (e.g., a cannula mounting needle, etc.) can be used to insert one or more EM sensors. Typically, the pin / needle will be a disposable component, while the sensor itself can be reusable. However, it should be understood that this is only one possible system, and various other systems can be used in which the pin / needle and / or EM sensor are independently disposable or reusable. In another embodiment, the EM sensor can be secured to the mounting pin / pin (e.g., using a Luer lock fitting, etc.), which can allow for rapid assembly and disassembly. In additional embodiments, the EM sensor can utilize an alternative sleeve and / or anchoring system that allows for minimally invasive placement of the sensor.

[0075] In another embodiment, the above-described system can enable a multi-sensor navigation system that can detect and correct for field distortions that plague electromagnetic tracking systems. It should be understood that field distortions can be caused by the movement of any ferromagnetic material within the reference field. Therefore, as known to those skilled in the art, a typical operating room (OR) has a large number of devices that can cause interference (e.g., operating tables, LCD displays, lighting equipment, imaging systems, surgical instruments, etc.). Furthermore, field distortions are notoriously difficult to detect. Using multiple EM sensors enables the system to accurately detect field distortions and / or alert the user that the current position measurement may be inaccurate. Because the sensors are securely fixed to the bony anatomy (e.g., via pins / needles), relative measurements of the sensor positions (X, Y, Z) can be used to detect field distortions. As a non-limiting example, in some embodiments, after the EM sensors are fixed to the bone, the relative distance between the two sensors is known and should remain constant. Therefore, any change in this distance can indicate the presence of field distortions.

[0076] In some embodiments, the surgeon can manually register specific objects with the system before or during surgery. For example, by interacting with the user interface, the surgeon can identify the starting position of a tool or bone structure. By tracking fiducial markers associated with the tool or bone structure, or by using other conventional image tracking methods, the processor can track the tool or bone as it moves through the environment in the three-dimensional model.

[0077] In some embodiments, certain markers, such as fiducial markers that identify individuals, important tools, or bones in the operating room, may include passive or active identification that can be picked up by a camera or camera array associated with the tracking system. For example, an infrared LED may flash a pattern that conveys a unique identification to the source of the pattern, thereby providing a dynamic identification marker. Similarly, one-dimensional or two-dimensional optical codes (barcodes, QR codes, etc.) may be affixed to objects in the operating room to provide passive identification that can occur based on image analysis. If these codes are placed asymmetrically on the object, they can also be used to determine the orientation of the object by comparing the position of the identification with the extent of the object in the image. For example, a QR code may be placed in the corner of a tool tray, allowing the orientation and identification of the tray to be tracked. Other tracking methods will be described throughout the text. For example, in some embodiments, surgeons and other personnel may wear augmented reality headsets to provide additional camera angles and tracking capabilities.

[0078] In addition to optical tracking, certain features of an object can be tracked by aligning the physical properties of the object and associating them with objects that can be tracked (e.g., fiducial markers fixed to a tool or bone). For example, a surgeon can perform a manual alignment process whereby the tracked tool and the tracked bone can be manipulated relative to each other. By striking the tip of the tool against the surface of the bone, a three-dimensional surface can be mapped for the bone, the three-dimensional surface being associated with the position and orientation of a reference frame relative to the fiducial marker. By optically tracking the position and orientation (pose) of the fiducial marker associated with the bone, a model of the surface can be tracked in the environment by extrapolation.

[0079] The registration process of registering the CASS 100 to the patient's relevant anatomical structures can also involve the use of anatomical landmarks, such as landmarks on bone or cartilage. For example, the CASS 100 can include a 3D model of the relevant bone or joint, and the surgeon can use a probe connected to the CASS to collect data about the position of bone landmarks on the patient's actual bone during surgery. Bone landmarks can include, for example, the medial and lateral malleolus, the ends of the proximal femur and distal tibia, and the center of the hip joint. The CASS 100 can compare and register the position data of the bone landmarks collected by the surgeon using the probe with the position data of the same landmarks in the 3D model. Alternatively, the CASS 100 can construct a 3D model of the bone or joint without preoperative image data by using the position data of the bone landmarks and bone surfaces collected by the surgeon using the CASS probe or other means. The registration process can also include determining the various axes of the joint. For example, for TKA, the surgeon can use the CASS 100 to determine the anatomical and mechanical axes of the femur and tibia. The surgeon and CASS 100 can identify the center of the hip joint by moving the patient's leg in a spiral direction (ie, circumduction) so that CASS can determine the location of the hip joint center.

[0080] Organization Navigation System 120( Figure 1 (not shown) provides the surgeon with intraoperative real-time visualization of the patient's bone, cartilage, muscle, nerve and / or vascular tissue surrounding the surgical area. Examples of systems that can be used for tissue navigation include fluorescence imaging systems and ultrasound systems.

[0081] The display 125 provides a graphical user interface (GUI) that displays images collected by the tissue navigation system 120 and other information related to the surgery. For example, in one embodiment, the display 125 overlays image information collected from various modalities (e.g., CT, MRI, X-ray, fluorescence, ultrasound, etc.) collected before or during surgery to provide the surgeon with various views of the patient's anatomy and real-time status. The display 125 may include, for example, one or more computer monitors. As an alternative to or in addition to the display 125, one or more of the surgical staff may wear an augmented reality (AR) head mounted device (HMD). For example, in Figure 1 In FIG, surgeon 111 wears an AR HMD 155 that can, for example, overlay preoperative image data on the patient or provide surgical planning suggestions. Various exemplary uses of the AR HMD 155 in surgical procedures are described in detail in the following sections.

[0082] The surgical computer 150 provides control instructions to the various components of the CASS 100, collects data from those components, and performs general processing for the various data required during surgery. In some embodiments, the surgical computer 150 is a general-purpose computer. In other embodiments, the surgical computer 150 may be a parallel computing platform that uses multiple central processing units (CPUs) or graphics processing units (GPUs) to perform processing. In some embodiments, the surgical computer 150 is connected to a remote server via one or more computer networks (e.g., the Internet). The remote server can be used, for example, for data storage or for executing computationally intensive processing tasks.

[0083] The surgical computer 150 can be connected to the other components of the CASS 100 using various technologies known in the art. Moreover, the computer can be connected to the surgical computer 150 using a variety of technologies. For example, the end effector 105B can be connected to the surgical computer 150 via a wired (i.e., serial) connection. The tracking system 115, the tissue navigation system 120, and the display 125 can similarly be connected to the surgical computer 150 using wired connections. Alternatively, the tracking system 115, the tissue navigation system 120, and the display 125 can be connected to the surgical computer 150 using wireless technologies, such as, but not limited to, Wi-Fi, Bluetooth, near field communication (NFC), or ZigBee.

[0084] Power impact and acetabular reamer device

[0085] The above about Figure 1 Part of the flexibility of the described CASS design is that additional or alternative devices can be added to CASS 100 as needed to support specific surgical procedures. For example, in the case of hip surgery, CASS 100 can include a powered impact device. The impact device is designed to repeatedly apply an impact force that the surgeon can use to perform activities such as implant alignment. For example, in total hip arthroplasty (THA), the surgeon typically uses an impact device to insert a prosthetic acetabular cup into the acetabulum of the implanted host. Although the impact device can be manual in nature (for example, operated by the surgeon striking the impactor with a hammer), powered impact devices are typically easier and faster to use in a surgical environment. The powered impact device can, for example, be powered using a battery attached to the device. Various attachments can be connected to the powered impact device to allow the impact force to be directed in various ways as needed during surgery. Similarly, in the case of hip surgery, CASS 100 can include a powered, robotically controlled end effector to expand the acetabulum to accommodate the acetabular cup implant.

[0086] In robotic-assisted THA, the patient's anatomy can be registered to the CASS 100 using CT or other image data, identification of anatomical landmarks, a tracker array attached to the patient's bones, and one or more cameras. The tracker array can be mounted on the iliac crest using a clamp and / or bone pins and can be installed externally through the skin or internally (posterolaterally or anterolaterally) through an incision made for THA. For THA, the CASS 100 can utilize one or more femoral cortical screws inserted into the proximal femur as checkpoints to aid in the registration process. The CASS 100 can also use one or more checkpoint screws inserted into the pelvis as additional checkpoints to aid in the registration process. The femoral tracker array can be fixed or mounted in the femoral cortical screws. The CASS 100 can employ the following steps, wherein verification is performed using a probe that the surgeon precisely places on key areas of the proximal femur and pelvis identified by the surgeon on the display 125. The tracker can be located on the robotic arm 105A or end effector 105B to align the arm and / or end effector to the CASS 100. The verification step may also utilize proximal and distal femoral checkpoints. CASS 100 may utilize color cues or other cues to inform the surgeon that the registration process between the bone and the robotic arm 105A or end effector 105B has been verified to a certain degree of accuracy (e.g., within 1 mm).

[0087] For THA, the CASS 100 can include a broach tracking option using a femoral array to allow the surgeon to intraoperatively capture the broach position and orientation and calculate the patient's hip length and offset. Based on the information provided about the patient's hip joint and the planned implant position and orientation after broach tracking is completed, the surgeon can modify or adjust the surgical plan.

[0088] For robotic-assisted THA, CASS 100 may include one or more powered reamers connected or attached to a robotic arm 105A or end effector 105B that prepare the pelvic bone to receive the acetabular implant according to the surgical plan. The robotic arm 105A and / or end effector 105B may notify the surgeon and / or control the power of the reamers to ensure that the acetabulum is resected (reamed) according to the surgical plan. For example, if the surgeon attempts to resect bone outside the boundaries of the bone to be resected according to the surgical plan, CASS 100 may cut off the power to the reamer or instruct the surgeon to cut off the power to the reamer. CASS 100 may provide the surgeon with the option to turn off or disengage the robotic control of the reamer. Compared to the surgical plan using different colors, the display 125 may show the progress of the bone being resected (reamed). The surgeon may view the display of the bone being resected (reamed) to guide the reamer to complete the reaming according to the surgical plan. CASS 100 may provide the surgeon with visual or auditory prompts to warn the surgeon that a resection that does not conform to the surgical plan is being performed.

[0089] After reaming, CASS 100 can use a manual or powered impactor attached to or connected to the robotic arm 105A or end effector 105B to impact the trial implant and the final implant into the acetabulum. The robotic arm 105A and / or the end effector 105B can be used to guide the impactor to impact the trial implant and the final implant into the acetabulum according to the surgical plan. CASS 100 can display the position and orientation of the trial implant and the final implant relative to the bone to inform the surgeon how to compare the orientation and position of the trial implant and the final implant with the surgical plan, and the display 125 can display the position and orientation of the implant as the surgeon manipulates the leg and hip. If the surgeon is not satisfied with the initial implant position and orientation, CASS 100 can provide the surgeon with the option of replanning and redoing the reaming and implant impaction by preparing a new surgical plan.

[0090] Preoperatively, CASS 100 can develop a proposed surgical plan based on a three-dimensional model of the hip joint and other patient-specific information (such as the mechanical and anatomical axes of the leg bones, the epicondylar axis, the femoral neck axis, the dimensions of the femur and hip (e.g., length), the midline axis of the hip joint, the ASIS axis of the hip joint, and the location of anatomical landmarks such as the lesser trochanter landmark, the distal landmark, and the center of rotation of the hip joint). The surgical plan developed by CASS can provide recommended optimal implant size and implant position and orientation based on the three-dimensional model of the hip joint and other patient-specific information. The surgical plan developed by CASS can include recommended details regarding offset values, inclination and anteversion values, center of rotation, cup size, midline value, superior-inferior fit, and femoral stem size and length.

[0091] For THA, the surgical plan generated by CASS can be viewed preoperatively and intraoperatively, and the surgeon can modify the surgical plan generated by CASS preoperatively or intraoperatively. The surgical plan generated by CASS can display the planned hip resection and superimpose the planned implant on the hip joint based on the planned resection. CASS 100 can provide the surgeon with a choice of different surgical processes, which will be displayed to the surgeon based on the surgeon's preferences. For example, the surgeon can choose from different workflows based on the number and type of anatomical landmarks examined and acquired and / or the position and number of tracker arrays used in the registration process.

[0092] According to some embodiments, the powered impact device used with CASS 100 can be operated with a variety of different settings. In some embodiments, the surgeon adjusts the settings via a manual switch or other physical mechanism on the powered impact device. In other embodiments, a digital interface can be used that allows settings to be entered, for example, via a touch screen on the powered impact device. Such a digital interface can allow the available settings to vary based on, for example, the type of attachment connected to the power attachment device. In some embodiments, settings can be changed by communicating with a robot or other computer system within CASS 100, rather than adjusting the settings on the powered impact device itself. Such a connection can be established using, for example, a Bluetooth or Wi-Fi networking module on the powered impact device. In another embodiment, the impact device and end piece can include features that allow the impact device to know what end piece (cup impactor, broach handle, etc.) is attached without the surgeon having to take any action, and adjust the settings accordingly. This can be achieved, for example, through a QR code, barcode, RFID tag, or other method.

[0093] Examples of settings that can be used include cup impact settings (e.g., unidirectional, specified frequency range, specified force and / or energy range); broach impact settings (e.g., bidirectional / oscillating within a specified frequency range, specified force and / or energy range); femoral head impact settings (e.g., unidirectional / single blow at a specified force or energy); and dry impact settings (e.g., unidirectional at a specified frequency with a specified force or energy). Additionally, in some embodiments, the powered impact device includes settings related to acetabular liner impact (e.g., unidirectional / single blow at a specified force or energy). There may be multiple settings for each type of liner (e.g., polymeric, ceramic, oxinium, or other material). Additionally, the powered impact device can provide settings for different bone qualities based on preoperative testing / imaging / knowledge and / or the surgeon's intraoperative assessment. In some embodiments, the powered impact device can have dual functionality. For example, the powered impact device can provide not only reciprocating motion to provide impact force, but also reciprocating motion for the broach or rasp.

[0094] In some embodiments, the powered impact device includes a feedback sensor that collects data during use of the device and sends the data to a computing device, such as a controller within the device or a surgical computer 150. The computing device can then record the data for later analysis and use. Examples of data that can be collected include, but are not limited to, acoustic waves, a predetermined resonant frequency of each instrument, reaction forces or rebound energy from the patient's bone, the position of the device relative to an imaged (e.g., fluorescence, CT, ultrasound, MRI, etc.) image of the registered bone anatomy, and / or external strain gauges on the bone.

[0095] Once the data is collected, the computing device can execute one or more algorithms in real time or near real time to assist the surgeon in performing the surgical procedure. For example, in some embodiments, the computing device uses the collected data to derive information such as the correct final broach size (femoral); when the stem is fully seated (femoral side); or when the cup is seated for the THA (depth and / or orientation). Once this information is known, it can be displayed for the surgeon to review, or it can be used to activate haptic or other feedback mechanisms to guide the surgical procedure.

[0096] In addition, the data derived from the aforementioned algorithm can be used to drive the operation of the device. For example, during the insertion of a prosthetic acetabular cup with a powered impact device, the device can automatically extend the impact head (e.g., end effector) to move the implant to the appropriate position, or shut off the power to the device once the implant is fully seated. In one embodiment, the derived information can be used to automatically adjust the setting of the bone quality, where the powered impact device should use less power to reduce femoral / acetabulum / pelvic fractures or damage to surrounding tissues.

[0097] Robotic Arm

[0098] In some embodiments, the CASS 100 includes a robotic arm 105A that serves as an interface for stabilizing and holding various instruments used during the surgical procedure. For example, in the case of hip surgery, these instruments may include, but are not limited to, retractors, sagittal or reciprocating saws, reamer handles, cup impactors, broach handles, and stem inserters. The robotic arm 105A may have multiple degrees of freedom (similar to a Spider device) and the ability to lock into place (e.g., by pressing a button, voice activation, the surgeon removing their hand from the robotic arm, or other methods).

[0099] In some embodiments, movement of the robotic arm 105A can be accomplished using a control panel built into the robotic arm system. For example, a display screen can include one or more input sources, such as physical buttons or a user interface with one or more icons, to direct the movement of the robotic arm 105A. A surgeon or other healthcare professional can engage with one or more input sources to position the robotic arm 105A during a surgical procedure.

[0100] The tools or end effectors 105B attached to or integrated into the robotic arm 105A may include, but are not limited to, deburring devices, scalpels, cutting devices, retractors, joint tensioning devices, and the like. In embodiments utilizing an end effector 105B, the end effector may be positioned at the end of the robotic arm 105A, allowing any motor control operations to be performed within the robotic arm system. In embodiments utilizing a tool, the tool may be secured to the distal end of the robotic arm 105A, but the motor control operations may be located within the tool itself.

[0101] The robotic arm 105A can be internally motorized to stabilize the robotic arm, thereby preventing it from falling and striking the patient, the operating table, surgical staff, etc., and allowing the surgeon to move the robotic arm without having to fully support its weight. As the surgeon moves the robotic arm 105A, the robotic arm can provide some resistance to prevent the robotic arm from moving too quickly or activating too many degrees of freedom at once. The position and locking state of the robotic arm 105A can be tracked, for example, by a controller or surgical computer 150.

[0102] In some embodiments, the robotic arm 105A can be moved manually (e.g., by a surgeon) or with internal motors to its ideal position and orientation for the task being performed. In some embodiments, the robotic arm 105A can be capable of operating in a "free" mode, allowing the surgeon to position the arm in a desired position without constraints. In free mode, the position and orientation of the robotic arm 105A can still be tracked, as described above. In one embodiment, certain degrees of freedom can be selectively released upon input from a user (e.g., a surgeon) during a designated portion of a surgical plan tracked by the surgical computer 150. Designs in which the robotic arm 105A is powered internally by hydraulics or a motor, or provides resistance to external manual movement by similar means, can be described as powered robotic arms, while arms that are manually manipulated without powered feedback but can be locked in position manually or automatically can be described as passive robotic arms.

[0103] The robotic arm 105A or end effector 105B may include a trigger or other device to control the power of the saw or drill. The surgeon's engagement of the trigger or other device can cause the robotic arm 105A or end effector 105B to transition from a motorized alignment mode to a mode in which the saw or drill is engaged and powered. In addition, the CASS 100 may include a foot pedal (not shown) that, when activated, causes the system to perform certain functions. For example, the surgeon can activate the foot pedal to instruct the CASS 100 to place the robotic arm 105A or end effector 105B in an automated mode, which positions the robotic arm or end effector 105B in an appropriate position relative to the patient's anatomy in order to perform the necessary resection. The CASS 100 can also place the robotic arm 105A or end effector 105B in a collaborative mode, which allows the surgeon to manually manipulate the robotic arm or end effector and position it in a specific location. The collaborative mode can be configured to allow the surgeon to move the robotic arm 105A or end effector 105B medially or laterally while restricting movement in other directions. As discussed, the robotic arm 105A or end effector 105B may include a cutting device (saw, drill, and sharpener) or a cutting guide or clamp 105D to guide the cutting device. In other embodiments, the movement of the robotic arm 105A or robotically controlled end effector 105B may be completely controlled by the CASS 100 without any assistance or input from the surgeon or other medical professional, or with minimal assistance or input. In still other embodiments, the surgeon or other medical professional may remotely control the movement of the robotic arm 105A or robotically controlled end effector 105B using a control mechanism separate from the robotic arm or robotically controlled end effector device, such as using a joystick or an interactive monitor or display control device.

[0104] The following example describes the use of a robotic device in the context of hip surgery; however, it should be understood that the robotic arm may have other applications in surgical procedures involving the knee, shoulder, etc. One example of the use of a robotic arm in the context of forming an anterior cruciate ligament (ACL) graft tunnel is described in WIPO Publication No. WO2020 / 047051, filed on August 28, 2019, entitled “Robotic Assisted Ligament Graft Placement and Tensioning,” the entire contents of which are incorporated herein by reference.

[0105] The robotic arm 105A can be used to hold a retractor. For example, in one embodiment, the surgeon can move the robotic arm 105A to a desired position. At this point, the robotic arm 105A can lock into place. In some embodiments, the robotic arm 105A is provided with data regarding the patient's position so that if the patient moves, the robotic arm can adjust the retractor position accordingly. In some embodiments, multiple robotic arms can be used, thereby allowing for holding multiple retractors or performing more than one action simultaneously (e.g., retractor holding and reaming).

[0106] The robotic arm 105A can also be used to help stabilize the surgeon's hand when making the femoral neck incision. In this application, the control of the robotic arm 105A can impose certain restrictions to prevent soft tissue damage from occurring. For example, in one embodiment, the surgical computer 150 tracks the position of the robotic arm 105A as it operates. If the tracked position approaches an area where tissue damage is predicted, a command can be sent to the robotic arm 105A to stop it. Alternatively, where the robotic arm 105A is automatically controlled by the surgical computer 150, the surgical computer can ensure that no instructions are provided to the robotic arm that would cause it to enter an area where soft tissue damage might occur. The surgical computer 150 can impose certain restrictions on the surgeon to prevent the surgeon from reaming too deep into the medial wall of the acetabulum or reaming at an incorrect angle or orientation.

[0107] In some embodiments, the robotic arm 105A can be used to hold the cup impactor at a desired angle or orientation during cup impaction. When the final position has been reached, the robotic arm 105A can prevent any further positioning to prevent damage to the pelvis.

[0108] The surgeon can use the robotic arm 105A to position the broach handle in the desired position and allow the surgeon to impact the broach into the femoral canal in the desired orientation. In some embodiments, once the surgical computer 150 receives feedback that the broach is fully seated, the robotic arm 105A can constrain the handle to prevent further advancement of the broach.

[0109] The robotic arm 105A can also be used for resurfacing applications. For example, the robotic arm 105A can stabilize the surgeon while using traditional instruments and provide certain constraints or restrictions to allow for proper placement of implant components (e.g., guidewire placement, chamfer cutters, sleeve cutters, planar cutters, etc.). In the case of using only a sharpening knife, the robotic arm 105A can stabilize the surgeon's handpiece and can impose constraints on the handpiece to prevent the surgeon from violating the surgical plan and removing undesired bone.

[0110] The robotic arm 105A can be a passive arm. As an example, the robotic arm 105A can be a CIRQ robotic arm available from Brainlab AG. CIRQ is a registered trademark of Brainlab AG, Olof-Palme-Str. 981829 Munich, Germany. In a specific embodiment, the robotic arm 105A is an intelligent gripping arm, as disclosed in U.S. patent application Ser. No. 15 / 525,585 to Krinninger et al., U.S. patent application Ser. No. 15 / 561,042 to Nowatschin et al., U.S. Patent No. 15 / 561,048 to Nowatschin et al., and U.S. Patent No. 10,342,636 to Nowatschin et al., each of which is incorporated herein by reference in its entirety.

[0111] Generation and collection of surgical procedure data

[0112] The various services provided by a healthcare professional to treat a clinical condition are collectively referred to as a "care episode." For a particular surgical procedure, the care episode may include three phases: preoperative, intraoperative, and postoperative. During each phase, data is collected or generated that can be used to analyze the care episode in order to understand the various characteristics of the procedure and identify patterns that can be used, for example, in training models to make decisions with minimal human intervention. The data collected during a care episode may be stored as a complete data set at the surgical computer 150 or surgical data server 180. Thus, for each care episode, there is one data set that includes all of the data collectively collected about the patient preoperatively, all of the data collected or stored by CASS 100 intraoperatively, and any postoperative data provided by the patient or by the healthcare professional monitoring the patient.

[0113] As explained in further detail, the data collected during the care session can be used to enhance the execution of the surgical procedure or provide a holistic understanding of the surgical procedure and patient outcomes. For example, in some embodiments, the data collected during the care session can be used to generate a surgical plan. In one embodiment, as data is collected during surgery, a high-level preoperative plan is refined during surgery. In this way, the surgical plan can be viewed as dynamically changing in real time or near real time as new data is collected by the components of CASS 100. In other embodiments, preoperative images or other input data can be used to preoperatively develop a robust plan that is easily executed during surgery. In this case, the data collected by CASS 100 during surgery can be used to make recommendations to ensure that the surgeon remains within the preoperative surgical plan. For example, if the surgeon is unsure how to achieve certain prescribed cuts or implant alignments, the surgeon can query the surgical computer 150 for recommendations. In yet other embodiments, preoperative and intraoperative planning protocols can be combined so that the refined preoperative plan can be dynamically modified as needed or desired during the surgical procedure. In some embodiments, a biomechanically based model of the patient's anatomy contributes simulation data to be considered by the CASS 100 in formulating pre-operative, intra-operative, and post-operative / rehabilitation procedures to optimize the patient's implant performance outcomes.

[0114] In addition to changing the surgical procedure itself, the data collected during the nursing session can also be used as input for other surgical assistance procedures. For example, in some embodiments, the nursing session data can be used to design an implant. Example data-driven techniques for designing, sizing, and fitting implants are described in U.S. patent application Ser. No. 13 / 814,531, filed on August 15, 2011, entitled “Systems and Methods for Optimizing Parameters for Orthopaedic Procedures”; U.S. patent application Ser. No. 14 / 232,958, filed on July 20, 2012, entitled “Systems and Methods for Optimizing Fit of an Implant to Anatomy”; and U.S. patent application Ser. No. 12 / 234,444, filed on September 19, 2008, entitled “Operatively Tuning Implants for Increased Performance,” the entire contents of each of which are hereby incorporated by reference into this patent application.

[0115] In addition, the data may be used for educational, training or research purposes. For example, using Figure 5C Using the web-based solution described in

[15] , other physicians or students can remotely view the surgery in an interface that allows them to selectively view data collected from various components of the CASS 100. Following the procedure, a similar interface can be used to "replay" the surgery for training or other educational purposes, or to identify the source of any problems or complications during the procedure.

[0116] The data obtained during the preoperative phase typically includes all information collected or generated before the operation. Thus, for example, information about the patient can be obtained from a patient entry form or electronic medical record (EMR). Examples of patient information that can be collected include, but are not limited to, patient demographics, diagnosis, medical history, medical records, vital signs, medical history information, allergies, and laboratory test results. Preoperative data can also include images related to the anatomical region of interest. These images can be obtained, for example, using magnetic resonance imaging (MRI), computed tomography (CT), X-rays, ultrasound, or any other method known in the art. Preoperative data can also include quality of life data obtained from the patient. For example, in one embodiment, preoperative patients use a mobile application ("app") to answer a questionnaire about their current quality of life. In some embodiments, the preoperative data used by CASS 100 includes demographic, anthropometric, cultural, or other specific characteristics about the patient, which can be consistent with activity level and specific patient activities to customize the surgical plan for the patient. For example, people of certain cultures or demographics may prefer to use a toilet with a squat toilet every day.

[0117] Figure 5A and 5B Examples of data that can be acquired during the intraoperative phase of a care episode are provided. These examples are based on the above references Figure 1 Various components of the CASS 100 are described; however, it should be understood that other types of data may be used based on the type of equipment used during surgery and its use.

[0118] Figure 5A 1 shows some examples of control instructions provided by the surgical computer 150 to other components of the CASS 100 according to some embodiments. Note that Figure 5A The example assumes that the components of the effector platform 105 are all directly controlled by the surgical computer 150. In embodiments where the components are manually controlled by the surgeon 111, instructions may be provided on the display 125 or AR HMD 155 to instruct the surgeon 111 how to move the components.

[0119] The various components included in the effector platform 105 are controlled by the surgical computer 150, which provides positional instructions that indicate where the components are to move within a coordinate system. In some embodiments, the surgical computer 150 provides instructions to the effector platform 105 that define how components of the effector platform 105 should react when they deviate from the surgical plan. These commands are Figure 5A

[0046] These are referred to as "haptic" commands in

[0047] For example, the end effector 105B can provide force to resist motion outside of the planned resection area. Other commands that can be used by the effector platform 105 include vibration and audio cues.

[0120] In some embodiments, the end effector 105B of the robotic arm 105A is operably coupled to a cutting guide 105D. In response to an anatomical model of the surgical scenario, the robotic arm 105A can move the end effector 105B and the cutting guide 105D to the appropriate position to match the location of the femoral or tibial cut to be performed according to the surgical plan. This can reduce the possibility of error, thereby allowing a vision system and a processor utilizing the vision system to implement the surgical plan to place the cutting guide 105D in a precise position and orientation relative to the tibia or femur to align the cutting groove of the cutting guide with the cut to be performed according to the surgical plan. The surgeon can then use any suitable tool, such as a vibrating or rotating saw or drill, to perform the cut (or drilling) with perfect placement and orientation, as the tool is mechanically constrained by the features of the cutting guide 105D. In some embodiments, the cutting guide 105D can include one or more pin holes that the surgeon uses to drill and tighten or pin the cutting guide into place before using the cutting guide to perform the resection of the patient's tissue. This can free up the robotic arm 105A or ensure that the cutting guide 105D is completely fixed and does not move relative to the bone to be resected. For example, this procedure can be used to make a first distal incision of the femur during a total knee replacement. In some embodiments, where the joint replacement is a hip replacement, the cutting guide 105D can be fixed to the femoral head or acetabulum for the corresponding hip replacement resection. It should be understood that any joint replacement that utilizes a precision incision can use the robotic arm 105A and / or cutting guide 105D in this manner.

[0121] The ablation device 110 is provided with a variety of commands to perform bone or tissue manipulation. As with the actuator platform 105, position information can be provided to the ablation device 110 to specify where it should be positioned when performing ablation. Other commands provided to the ablation device 110 may depend on the type of ablation device. For example, for mechanical or ultrasonic ablation tools, the commands may specify the speed and frequency of the tool. For radiofrequency ablation (RFA) and other laser ablation tools, these commands may specify the intensity and pulse duration.

[0122] Some components of the CASS 100 need not be directly controlled by the surgical computer 150; rather, the surgical computer 150 need only activate the component, which then executes software locally to specify the manner in which data is collected and provided to the surgical computer 150. Figure 5A In the example of , there are two components that operate in this manner: tracking system 115 and tissue navigation system 120 .

[0123] The surgical computer 150 provides the display 125 with any visualizations required by the surgeon 111 during surgery. The surgical computer 150 can provide instructions for displaying images, a GUI, and the like to the monitor using techniques known in the art. The display 125 can include various components of the surgical planning workflow. For example, during the registration process, the display 125 can show the 3D bone model constructed preoperatively and illustrate the position of the probe as the surgeon uses it to collect the locations of anatomical landmarks on the patient. The display 125 can also include information regarding the target surgical area. For example, in conjunction with a TKA, the display 125 can show the mechanical and anatomical axes of the femur and tibia. The display 125 can also show the varus and valgus angles of the knee joint based on the surgical plan, and the CASS 100 can display how these angles will be affected if a desired modification to the surgical plan is made. Thus, the display 125 is an interactive interface that can dynamically update and display how changes to the surgical plan will affect the procedure and the final position and orientation of the implant installed in the bone.

[0124] As the workflow proceeds to preparation for bone cutting or resection, the display 125 can show the planned or recommended bone cuts before any cuts are performed. The surgeon 111 can manipulate the image display to provide different anatomical perspectives of the target area and can have the option of changing or revising the planned bone cuts based on the patient's intraoperative assessment. The display 125 can show how the selected implant will be installed on the bone if the planned bone cuts are performed. If the surgeon 111 chooses to change the previously planned bone cuts, the display 125 can show how the revised bone cuts will change the position and orientation of the implant when installed on the bone.

[0125] The display 125 can provide the surgeon 111 with various data and information regarding the patient, the planned surgical procedure, and the implant. Various patient-specific information can be displayed, including real-time data regarding the patient's health, such as heart rate and blood pressure. The display 125 can also include information regarding the anatomy of the surgical target area (including the location of landmarks), the current state of the anatomy (e.g., whether any resections have been performed, the depth and angle of planned and executed bone cuts), and the future state of the anatomy as the surgical plan progresses. The display 125 can also provide or illustrate additional information regarding the surgical target area. For TKA, the display 125 can provide information regarding the gap between the femur and tibia (e.g., gap balance) and how this gap will change if the planned surgical plan is implemented. For TKA, the display 125 can provide additional relevant information regarding the knee joint, such as data regarding joint tension (e.g., ligament laxity) and information regarding the joint's rotation and alignment. The display 125 can also illustrate how the planned implant positioning and location will affect the patient when the knee is flexed. Display 125 can illustrate how the use of different implants or the use of the same implant of different sizes will affect the surgical plan, and preview how such implants will be positioned on the bone. CASS 100 can provide such information for each planned osteotomy in TKA or THA. In TKA, CASS 100 can provide robot control for one or more planned osteotomies. For example, CASS 100 can only provide robot control for the initial distal femoral cutting, and surgeon 111 can use conventional means (such as 4-in-1 cutting guide or fixture 105D) to manually perform other cuttings (front, back and chamfer cutting).

[0126] Display 125 can use different colors to inform the surgeon of the status of the surgical plan. For example, unresected bone can be displayed in a first color, resected bone can be displayed in a second color, and planned resection can be displayed in a third color. Implants can be superimposed on the bones in display 125, and the implant colors can change or correspond to different types or sizes of implants.

[0127] The information and options shown on the display 125 can vary depending on the type of surgical procedure being performed. In addition, the surgeon 111 can request or select a specific surgical flow display that matches or is consistent with his or her surgical plan preferences. For example, for a surgeon 111 who typically performs a tibial cut before a femoral cut in a TKA, the display 125 and the associated workflow can be adapted to take this preference into account. The surgeon 111 can also pre-select to include or delete certain steps from the standard surgical workflow display. For example, if the surgeon 111 uses resection measurements to finalize an implant plan, but does not analyze ligament gap balance when finalizing the implant plan, the surgical flow display can be organized into modules, and the surgeon can select the modules to be displayed and the order in which the modules are provided based on the surgeon's preferences or the circumstances of a particular operation. For example, a module involving ligament and gap balance can include ligament / gap balance before and after resection, and the surgeon 111 can select which modules to include in its default surgical plan workflow based on whether such ligament and gap balance is performed before or after (or before and after) performing a bone resection.

[0128] For more specialized display devices, such as AR HMDs, the surgical computer 150 can provide images, text, etc. using data formats supported by the device. For example, if the display 125 is a Microsoft HoloLens TM or Magic Leap One TM If the surgeon 111 has a holographic device, the surgical computer 150 can use the HoloLens application programming interface (API) to send commands that specify the location and content of the hologram displayed in the surgeon's 111 field of view.

[0129] In some embodiments, one or more surgical planning models may be incorporated into CASS 100 and used in the development of a surgical plan provided to surgeon 111. The term "surgical planning model" refers to software that simulates the biomechanical properties of anatomical structures in various situations to determine the best way to perform cuts and other surgical activities. For example, for a knee replacement surgery, a surgical planning model may measure parameters of functional activities, such as deep knee flexion, gait, etc., and select cutting locations on the knee to optimize implant placement. An example of a surgical planning model is the LIFEMOD® from SMITH AND NEPHEW. TM In some embodiments, the surgical computer 150 includes a computing architecture (e.g., a GPU-based parallel processing environment) that allows for the complete execution of the surgical planning model during surgery. In other embodiments, the surgical computer 150 may be connected via a network to a remote computer that allows for such execution, such as a surgical data server 180 (see Figure 5CAs an alternative to a full implementation of the surgical planning model, in some embodiments, a set of transfer functions is derived that simplifies the mathematical operations obtained by the model into one or more prediction equations. Then, instead of performing a full simulation during surgery, the prediction equations are used. More details on the use of transfer functions are described in WIPO Publication No. 2020 / 037308, entitled “Patient Specific Surgical Method and System,” filed on August 19, 2019, the entire contents of which are incorporated herein by reference.

[0130] Figure 5B Examples of some types of data that can be provided to the surgical computer 150 from the various components of CASS 100 are shown. In some embodiments, the components can stream data to the surgical computer 150 in real time or near real time during surgery. In other embodiments, the components can queue the data and send it to the surgical computer 150 at a set interval (e.g., every second). The data can be transmitted using any format known in the art. Thus, in some embodiments, all components transmit data to the surgical computer 150 in a common format. In other embodiments, each component can use a different data format, and the surgical computer 150 is configured with one or more software applications capable of converting the data.

[0131] Typically, the surgical computer 150 can be used as a central point for collecting CASS data. The exact content of the data will depend on the source. For example, each component of the actuator platform 105 provides a measured position to the surgical computer 150. Thus, by comparing the measured position with the position initially specified by the surgical computer 150 (see Figure 5B ) for comparison, the surgical computer can identify deviations that occur during surgery.

[0132] The resection device 110 can send various types of data to the surgical computer 150 depending on the type of device used. Exemplary data types that can be sent include measured torque, audio signatures, and measured displacement values. Similarly, the tracking technology 115 can provide different types of data depending on the tracking method used. Exemplary tracking data types include position values ​​of the tracked item (e.g., anatomical structure, tool, etc.), ultrasound images, and surface or landmark collection points or axes. When the system is operating, the tissue navigation system 120 provides anatomical position, shape, etc. to the surgical computer 150.

[0133] While the display 125 is typically used to output data for presentation to a user, it can also provide data to the surgical computer 150. For example, for embodiments in which a monitor is used as part of the display 125, the surgeon 111 can interact with the GUI to provide input, which is sent to the surgical computer 150 for further processing. For AR applications, the measured position and displacement of the HMD can be sent to the surgical computer 150 so that it can update the presented view as needed.

[0134] During the postoperative phase of the care period, various types of data can be collected to quantify the overall improvement or deterioration of the patient's condition due to the surgery. The data can take the form of self-reported information reported by the patient through a questionnaire, for example. For example, in the case of a knee replacement surgery, the Oxford knee score questionnaire can be used to measure functional status, and the EQ5D-5L questionnaire can be used to measure postoperative quality of life. Other examples in the case of hip replacement surgery can include the Oxford hip score, the Harris hip score, and the WOMAC (Western Ontario and McMaster University Osteoarthritis Index). Such questionnaires can be managed, for example, by medical professionals directly in a clinical setting, or using a mobile application that allows patients to answer questions directly. In some embodiments, one or more wearable devices that collect data related to the surgery can be provided to the patient. For example, after a knee surgery, a knee brace can be provided to the patient, and the knee brace includes sensors for monitoring knee position, flexibility, etc. This information can be collected and transmitted to the patient's mobile device for the surgeon to view to evaluate the results of the surgery and solve any problems. In some embodiments, one or more cameras can capture and record the movement of the patient's body parts during designated activities after the surgery. This motion capture can be compared to a biomechanical model to better understand the function of the patient's joint and to better predict rehabilitation progress and determine any modifications that may be needed.

[0135] The postoperative phase of the care period can continue throughout the patient's lifespan. For example, in some embodiments, the surgical computer 150 or other components comprising the CASS 100 can continue to receive and collect data related to the surgical procedure even after the surgery has been performed. This data can include, for example, images, answers to questions, "normal" patient data (e.g., blood type, blood pressure, condition, medications, etc.), biometric data (e.g., gait, etc.), and objective and subjective data related to specific issues (e.g., knee or hip pain). This data can be explicitly provided to the surgical computer 150 or other CASS component by the patient or the patient's physician. Alternatively or additionally, the surgical computer 150 or other CASS component can monitor the patient's EMR and retrieve relevant information when available. This longitudinal view of the patient's recovery allows the surgical computer 150 or other CASS component to provide a more objective analysis of patient outcomes to measure and track the success of a given procedure. For example, by performing regression analysis on various data items collected during the care period, the conditions experienced by the patient long after the surgical procedure can be linked to the surgery. This analysis can be further enhanced by analyzing groups of patients who have had similar procedures and / or similar anatomy.

[0136] In some embodiments, data is collected at a central location to provide easier analysis and use. In some cases, data can be collected manually from various CASS components. For example, a portable storage device (e.g., a USB stick) can be attached to the surgical computer 150 to retrieve data collected during surgery. The data can then be transferred to a centralized storage device, such as via a desktop computer. Alternatively, in some embodiments, the surgical computer 150 is directly connected to the centralized storage device via the network 175, such as Figure 5C As shown in .

[0137] Figure 5C A "cloud-based" embodiment is shown, in which the surgical computer 150 is connected to a surgical data server 180 via a network 175. The network 175 can be, for example, a private intranet or the Internet. In addition to data from the surgical computer 150, other sources can also transmit relevant data to the surgical data server 180. Figure 5CThe example of FIG1 shows three additional data sources: patient 160, healthcare professional 165, and EMR database 170. Thus, patient 160 can send pre-operative and post-operative data to surgical data server 180, for example, using a mobile application. Healthcare professionals 165 include the surgeon and his or her staff, as well as any other professionals working with patient 160 (e.g., personal physician, rehabilitation specialist, etc.). It should also be noted that EMR database 170 can be used for both pre-operative and post-operative data. For example, assuming that patient 160 has given sufficient permission, surgical data server 180 can collect the patient's pre-operative EMR. The surgical data server 180 can then continue to monitor the EMR for any updates after the surgery.

[0138] At the surgical data server 180, a care session database 185 is used to store various data collected during a patient's care session. The care session database 185 can be implemented using any technology known in the art. For example, in some embodiments, a SQL-based database can be used, in which all various data items are structured in a manner that allows them to be easily incorporated into two SQL sets of rows and columns. However, in other embodiments, a No-SQL database can be employed to accommodate unstructured data while providing the ability to quickly process and respond to queries. As understood in the art, the term "No-SQL" is used to define a class of databases that are unrelated in their design. Various types of No-SQL databases can generally be grouped according to their underlying data models. These groups can include databases that use a column-based data model (e.g., Cassandra), a document-based data model (e.g., MongoDB), a key-value-based data model (e.g., Redis), and / or a graph-based data model (e.g., Allego). Any type of No-SQL database can be used to implement the various embodiments described herein, and in some embodiments, different types of databases can support the care session database 185.

[0139] Data may be transferred between the various data sources and the surgical data server 180 using any data format and transmission technology known in the art. Figure 5C The architecture shown in allows for the transmission of data from data sources to the surgical data server 180, and the retrieval of data by the data sources from the surgical data server 180. For example, as explained in detail below, in some embodiments, the surgical computer 150 can use data from past surgeries, machine learning models, etc. to help guide the surgical procedure.

[0140] In some embodiments, the surgical computer 150 or surgical data server 180 may perform a de-identification process to ensure that the data stored in the care session database 185 meets Health Insurance Portability and Accountability Act (HIPAA) standards or other requirements imposed by law. HIPAA provides a list of certain identifiers that must be removed from the data during de-identification. The aforementioned de-identification process can scan for these identifiers in the data transmitted to the care session database 185 for storage. For example, in one embodiment, the surgical computer 150 performs a de-identification process immediately prior to initially transmitting a particular data item or group of data items to the surgical data server 180. In some embodiments, a unique identifier is assigned to the data from a particular care session to facilitate re-identification of the data when necessary.

[0141] although Figure 5A –5C discusses data collection in the context of a single care session, but it will be appreciated that the general concepts can be extended to data collection across multiple care sessions. For example, surgical data can be collected throughout the care session each time a procedure is performed using the CASS 100 and stored at the surgical computer 150 or surgical data server 180. As explained in further detail below, a robust database of care session data allows for the generation of optimized values, measurements, distances or other parameters, and other recommendations related to the surgical procedure. In some embodiments, various data sets are indexed in a database or other storage medium in a manner that allows for rapid retrieval of relevant information during the surgical procedure. For example, in one embodiment, a patient-centric set of indexes can be used so that data for a particular patient or a group of patients similar to a particular patient can be easily extracted. This concept can be similarly applied to surgeons, implant characteristics, CASS component types, and the like.

[0142] Further details of managing episode of care data are described in U.S. patent application Ser. No. 62 / 783,858, filed on Dec. 21, 2018, entitled “Methods and Systems for Providing an Episode of Care,” which is incorporated herein by reference in its entirety.

[0143] Open and closed digital ecosystems

[0144] In some embodiments, CASS 100 is designed to function as a standalone or "closed" digital ecosystem. Each component of CASS 100 is specifically designed for use within the closed ecosystem, and data is generally inaccessible to devices outside the digital ecosystem. For example, in some embodiments, each component includes software or firmware that implements proprietary protocols for activities such as communication, storage, and security. The closed digital ecosystem concept may be ideal for companies that want to control all components of CASS 100 to ensure that certain compatibility, safety, and reliability standards are met. For example, CASS 100 can be designed so that new components cannot be used with CASS unless they are certified by the company.

[0145] In other embodiments, CASS 100 is designed to function as an "open" digital ecosystem. In these embodiments, components can be produced by a variety of different companies according to standards for activities such as communication, storage, and security. Thus, by using these standards, any company is free to build independent, compliant components of the CASS platform. Data can be transferred between components using publicly available application programming interfaces (APIs) and open, shareable data formats.

[0146] To illustrate one type of recommendation that can be performed using CASS 100, a technique for optimizing surgical parameters is disclosed below. The term "optimization" herein refers to selecting optimal parameters based on certain specified criteria. In an extreme case, optimization can refer to selecting optimal parameters based on data from the entire care episode (including any preoperative data, the state of CASS data at a given point in time, and postoperative goals). Furthermore, optimization can be performed using historical data, such as data generated during past surgeries involving, for example, the same surgeon, past patients with similar physical characteristics to the current patient, and the like.

[0147] The optimized parameters can depend on the part of the patient's anatomical structure to be operated on. For example, for knee surgery, the surgical parameters can include the positioning information of the femoral and tibial components, including but not limited to rotational alignment (for example, varus / valgus rotation, external rotation, flexion rotation of the femoral component, the posterior tilt angle of the tibial component), resection depth (for example varus knee, valgus knee), and the type, size and position of the implant. The positioning information can also include the surgical parameters for the combined implant, such as overall limb alignment, combined tibiofemoral hyperextension and combined tibiofemoral resection. Other examples of parameters that CASS 100 can optimize for a given TKA femoral implant include the following:

[0148]

[0149] Other examples of parameters that the CASS 100 can optimize for a given TKA tibial implant include the following:

[0150]

[0151]

[0152] For hip surgery, surgical parameters may include femoral neck resection location and angle, cup inclination, cup anteversion, cup depth, femoral stem design, femoral stem size, fit of the femoral stem within the canal, femoral offset, leg length, and femoral pattern of the implant.

[0153] Shoulder parameters may include, but are not limited to, humeral resection depth / angle, humeral shaft type, humeral offset, glenoid type and inclination, and reverse shoulder parameters such as humeral resection depth / angle, humeral shaft type, glenoid inclination / type, glenosphere orientation, glenosphere offset, and offset direction.

[0154] Various conventional techniques exist for optimizing surgical parameters. However, these techniques are often computationally intensive, and therefore, the parameters typically need to be determined preoperatively. Consequently, the surgeon's ability to modify the optimization parameters based on potential issues that may arise during surgery is limited. Furthermore, conventional optimization techniques often operate in a "black box" manner, with little or no explanation regarding the recommended parameter values. Therefore, if the surgeon decides to deviate from the recommended parameter values, they often do so without fully understanding the impact of that deviation on the rest of the surgical process or the impact on the patient's postoperative quality of life.

[0155] Surgical patient care system

[0156] The general concept of optimization can be extended to the entire care continuum using a surgical patient care system 620 that uses surgical data and other data from the patient 605 and healthcare professional 630 to optimize outcomes and patient satisfaction, such as Figure 6 As shown in .

[0157] Conventionally, preoperative diagnosis, preoperative surgical planning, intraoperative execution of the established plan, and postoperative management of total joint arthroplasty are based on personal experience, published literature, and the surgeon's training knowledge base (ultimately, the tribal knowledge of the individual surgeon and their peer "network" and journal publications), as well as their instinctive ability to accurately discern intraoperative "balance" and manually execute plane resections using guidance and visual cues. This existing knowledge base and execution approach are limited in optimizing outcomes for patients requiring care. For example, limitations exist in accurately diagnosing patients for appropriate, minimally invasive established care; aligning dynamic patient, medical economic, and surgeon preferences with patients' desired outcomes; executing surgical plans to ensure proper bone alignment and balance; and receiving data from disconnected sources with diverse deviations that are difficult to reconcile into a holistic patient framework. Therefore, data-driven tools that more accurately simulate anatomical responses and guide surgical planning could improve upon existing approaches.

[0158] The surgical patient care system 620 is designed to utilize patient-specific data, surgeon data, institutional data, and historical outcome data to develop algorithms that suggest or recommend the best overall treatment plan for the patient throughout the care period (pre-operative, intra-operative, and post-operative) based on the desired clinical outcomes. For example, in one embodiment, the surgical patient care system 620 tracks adherence to the suggested or recommended plan and adjusts the plan based on patient / care provider performance. Once the surgical treatment plan is completed, the surgical patient care system 620 records the collected data in a historical database. This database is available for future patients to access and develop future treatment plans. In addition to utilizing statistical and mathematical models, simulation tools (e.g., ) simulates outcomes, alignment, kinematics, etc. based on a preliminary or proposed surgical plan, and reconfigures the preliminary or proposed plan to achieve a desired or optimal outcome based on the patient's profile or surgeon's preferences. The surgical patient care system 620 ensures that each patient is receiving personalized surgical and rehabilitation care, thereby improving the chances of a successful clinical outcome and reducing the financial burden on the facility associated with near-term revisions.

[0159] In some embodiments, the surgical patient care system 620 employs data collection and management methods to provide a detailed surgical case plan with distinct steps that are monitored and / or executed using the CASS 100. The user's performance is calculated as each step is completed and used to suggest changes to subsequent steps in the case plan. The case plan is generated based on a series of input data stored in a local or cloud storage database. The input data can be related to the patient currently being treated or historical data from patients who have received similar treatments.

[0160] The patient 605 provides input such as current patient data 610 and historical patient data 615 to the surgical patient care system 620. Various methods generally known in the art can be used to collect such input from the patient 605. For example, in some embodiments, the patient 605 fills out a paper or digital survey that the surgical patient care system 620 parses to extract patient data. In other embodiments, the surgical patient care system 620 can extract patient data from existing information sources such as electronic medical records (EMRs), health history files, and payer / provider history files. In still other embodiments, the surgical patient care system 620 can provide an application program interface (API) that allows external data sources to push data to the surgical patient care system. For example, the patient 605 can have a mobile phone, wearable device, or other mobile device that collects data (e.g., heart rate, pain or discomfort level, movement or activity level, or patient-submitted responses to the patient's compliance with any number of preoperative planning standards or conditions) and provides the data to the surgical patient care system 620. Similarly, patient 605 may have a digital application on their mobile or wearable device that can collect data and transmit it to the surgical patient care system 620.

[0161] Current patient data 610 may include, but is not limited to: activity level, past conditions, comorbidities, pre-rehabilitation performance, health and fitness level, pre-operative expectation level (related to hospital, surgery, and rehabilitation), metropolitan statistical area (MSA) driver score, genetic background, previous injuries (sports, trauma, etc.), previous joint replacements, previous trauma surgeries, previous sports medicine surgeries, treatment of contralateral joints or limbs, gait or biomechanical information (dorsal and ankle tissue), pain or discomfort level, care infrastructure information (payer coverage type, home healthcare infrastructure level, etc.), and an indication of the expected ideal outcome of the surgery.

[0162] Historical patient data 615 may include, but is not limited to: activity level, past conditions, comorbidities, pre-rehabilitation performance, health and fitness level, pre-operative expected level (related to hospital, surgery, and rehabilitation), MSA driver score, genetic background, previous injuries (sports, trauma, etc.), previous joint replacements, previous trauma surgeries, previous sports medicine surgeries, treatment of contralateral joints or limbs, gait or biomechanical information (dorsal and ankle tissues), pain or discomfort level, care infrastructure information (payer coverage type, level of home healthcare infrastructure, etc.), expected ideal outcome of the surgery, actual outcome of the surgery (patient reported outcomes [PROs], implant survival, pain level, activity level, etc.), size of implant used, position / orientation / alignment of implant used, soft tissue balance achieved, etc.

[0163] The healthcare professional 630 performing the surgery or treatment can provide various types of data 625 to the surgical patient care system 620. This healthcare professional data 625 can include, for example, a description of known or preferred surgical techniques (e.g., cruciate retention (CR) versus posterior stabilization (PS), size increase versus size reduction, with tourniquet versus without tourniquet, femoral stem style, preferred options for THA, etc.), the healthcare professional's 630 level of training (e.g., years in practice, position trained, location of training, technique modeled), previous success levels including historical data (outcomes, patient satisfaction), and expected ideal outcomes with respect to range of motion, recovery days, and device longevity. The healthcare professional data 625 can be obtained, for example, through a paper or digital survey provided to the healthcare professional 630, via healthcare professional input into a mobile application, or by extracting relevant data from an EMR. Additionally, the CASS 100 can provide data such as profile data (e.g., a patient-specific knee instrument profile) or a historical record describing the use of the CASS during surgery.

[0164] Information about the facility where the surgery or treatment is to be performed may be included in the input data. This data may include, but is not limited to, the following: ambulatory surgical center (ASC) versus hospital, facility trauma level, comprehensive joint replacement (CJR) or bundle candidacy, MSA driver score, community versus metropolitan, academic versus non-academic, postoperative network access (skilled nursing facility [SNF] only, home health, etc.), availability of medical professionals, availability of implants, and availability of surgical equipment.

[0165] These facility inputs may be obtained, for example, but not limited to, through surveys (paper / digital), surgical planning tools (e.g., apps, websites, electronic medical records [EMR], etc.), hospital information databases (on the Internet), etc. Input data related to the economics of healthcare may also be obtained, including but not limited to the patient's socioeconomic profile, the expected level of reimbursement the patient will receive, and whether the treatment is patient-specific.

[0166] These healthcare economic inputs can be obtained, for example and without limitation, through surveys (paper / digital), direct payer information, socioeconomic status databases (postal codes available on the internet), etc. Finally, data derived from simulations of the procedure are obtained. Simulation inputs include implant size, location, and orientation. Custom or commercially available anatomical modeling software programs (e.g., It should be noted that the above data inputs may not be available for every patient and the available data will be used to generate the treatment plan.

[0167] Prior to the procedure, patient data 610, 615 and healthcare professional data 625 may be acquired and stored in a cloud-based or online database (e.g., Figure 5C 100). Information related to the procedure is provided to the computing system via wireless data transmission or manually using portable media storage. The computing system is configured to generate a case plan for CASS 100. The generation of the case plan will be described below. It should be noted that the system can access historical data of previously treated patients, including implant sizes, positions and orientations automatically generated by a computer-assisted patient-specific knee instrument (PSKI) selection system or CASS 100 itself. To this end, a surgical sales representative or case engineer uploads case log data to a historical database using an online portal. In some embodiments, the data transmission to the online database is wireless and automated.

[0168] Historical data sets from online databases are used as input to machine learning models (e.g., recurrent neural networks (RNNs) or other forms of artificial neural networks). As is generally understood in the art, artificial neural networks function similarly to biological neural networks and consist of a series of nodes and connections. The machine learning model is trained to predict one or more values ​​based on the input data. For the following sections, it is assumed that the machine learning model is trained to generate prediction equations. These prediction equations can be optimized to determine the optimal size, position, and orientation of the implant to achieve the best results or satisfaction.

[0169] Once the procedure is complete, all patient data and available outcome data, including the implant size, position, and orientation determined by the CASS100, are collected and stored in a historical database. Any subsequent calculation of the objective equation by the RNN will include data from previous patients in this way, allowing for continuous improvement of the system.

[0170] In addition to or as an alternative to determining implant positioning, in some embodiments, prediction equations and associated optimization can be used to generate resection planes for use with the PSKI system. When used with the PSKI system, the calculation and optimization of the prediction equations are completed before surgery. The patient's anatomical structure is estimated using medical image data (X-ray, CT, MRI). Global optimization of the prediction equations can provide the ideal size and position of the implant components. The Boolean intersection of the implant components and the patient's anatomical structure is defined as the resection volume. PSKI can be generated to remove the optimized resection envelope. In this embodiment, the surgeon cannot change the surgical plan intraoperatively.

[0171] The surgeon may choose to change the surgical case plan at any time before or during surgery. If the surgeon chooses to deviate from the surgical case plan, the sizes, positions, and / or orientations of the components that were changed are locked, and the global optimization (using the techniques previously described) is refreshed to find the new ideal positions for the other components based on the new sizes, positions, and / or orientations of the components, and the corresponding resections that need to be performed to achieve the new optimized sizes, positions, and / or orientations of the components. For example, if the surgeon determines that the size, position, and / or orientation of the femoral implant in a TKA needs to be updated or modified intraoperatively, the position of the femoral implant will be locked relative to the anatomy, and a new optimal position of the tibia will be calculated (via global optimization) by taking into account the surgeon's changes to the femoral implant size, position, and / or orientation. Additionally, if the surgical system used to implement the case plan is robotically assisted (e.g., using If a surgical procedure involves a 3D image sensor (e.g., a 3D image sensor or MAKO Rio), bone removal and bone morphology during surgery can be monitored in real time. If the resection performed during the procedure deviates from the surgical plan, the processor can optimize the subsequent placement of additional components taking into account the actual resection that has been performed.

[0172] Figure 7AThe surgical patient care system 620 is shown as an example of how it can be adapted to perform a case plan matching service. In this example, data related to a current patient 610 is obtained and compared to all or part of a historical database of patient data and associated outcomes 615. For example, a surgeon may choose to compare the current patient's plan with a subset of the historical database. The data in the historical database can be filtered to include, for example, only datasets with favorable outcomes, datasets corresponding to historical surgeries on patients with the same or similar profile to the current patient, datasets corresponding to specific surgeons, datasets corresponding to specific elements of the surgical plan (e.g., surgeries that only preserve specific ligaments), or any other criteria selected by the surgeon or medical professional. For example, if the current patient's data matches or correlates with data from a previous patient who experienced favorable outcomes, the previous patient's case plan can be accessed and adapted or adopted for the current patient. The predictive equation can be used in conjunction with an intraoperative algorithm that identifies or determines actions relevant to the case plan. Based on relevant information from the historical database and / or preselected information, the intraoperative algorithm determines a series of recommended actions for the surgeon to perform. Each execution of the algorithm generates the next action in the case plan. If the surgeon performs that action, the outcome is evaluated. The results of the actions performed by the surgeon are used to refine and update the inputs to the intraoperative algorithm that is used to generate the next step in the case plan. Once the case plan has been fully executed, all data related to the case plan (including any deviations by the surgeon from performing the suggested actions) are stored in a database of historical data. In some embodiments, the system uses preoperative, intraoperative, or postoperative modules in a segmented manner rather than across the entire continuum of care. In other words, the caregiver can prescribe any permutation or combination of treatment modules, including the use of a single module. These concepts are discussed in detail in the next section. Figure 7B and can be applied to any type of surgery using CASS 100.

[0173] The surgical process is displayed

[0174] As mentioned above about Figure 1 and Figures 5A-5CAs described above, the various components of CASS 100 generate detailed data records during surgery. CASS 100 can track and record the surgeon's various movements and activities during each step of the procedure, comparing actual activities with preoperative or intraoperative surgical plans. In some embodiments, software tools can be employed to process this data into a format that effectively allows for "replaying" the surgery. For example, in one embodiment, one or more GUIs can be used that display all information presented on display 125 during surgery. This can be supplemented with graphs and images showing data collected by various tools. For example, a GUI that provides a visual representation of the knee during tissue resection can display the measured torque and displacement of the resection device adjacent to the visual representation to better understand any deviations from the planned resection area. The ability to view a replay of the surgical plan or toggle between the actual surgery and different stages of the planned surgery can benefit the surgeon and / or surgical staff, allowing them to identify any deficiencies or challenging stages of the procedure so that they can be modified in future surgeries. Similarly, in an academic setting, the aforementioned GUIs can be used as teaching tools for training future surgeons and / or surgical staff. Additionally, because the dataset effectively records many elements of a surgeon's activities, it can also be used as evidence of the correct or incorrect performance of a specific surgical procedure for other reasons, such as legal or compliance reasons.

[0175] Over time, as more and more surgical data is collected, a rich database may be acquired that describes surgical procedures performed by different surgeons on different patients for various types of anatomical structures (knee, shoulder, hip, etc.). Moreover, information such as implant type and size, patient demographics, etc. can be further used to enhance the overall dataset. Once the dataset has been established, it can be used to train a machine learning model (e.g., an RNN) to predict how the surgery will proceed based on the current state of the CASS 100.

[0176] Training of the machine learning model can be performed as follows. During surgery, the overall state of CASS 100 can be sampled over multiple time periods. The machine learning model can then be trained to transform the current state of a first time period into a future state for a different time period. By analyzing the entire state of CASS 100 rather than individual data items, any causal effects of interactions between different components of CASS 100 can be captured. In some embodiments, multiple machine learning models can be used rather than a single model. In some embodiments, the machine learning model can be trained not only with the state of CASS 100, but also with patient data (e.g., obtained from an EMR) and the identity of the surgical staff. This allows the model to make predictions with greater specificity. Moreover, if desired, it allows the surgeon to selectively make predictions based solely on their own surgical experience.

[0177] In some embodiments, the predictions or recommendations made by the aforementioned machine learning models can be integrated directly into the surgical process. For example, in some embodiments, the surgical computer 150 can execute the machine learning model in the background to make predictions or recommendations for upcoming actions or surgical conditions. Therefore, multiple states can be predicted or recommended for each period. For example, the surgical computer 150 can predict or recommend the state of the next 5 minutes in 30-second increments. Using this information, the surgeon can utilize a "process display" view of the surgery to allow visualization of future states. For example, Figure 7C A series of images that can be displayed to the surgeon are shown, which illustrate the implant placement interface. The surgeon can, for example, traverse these images by entering a specific time in the display 125 of the CASS 100 or instructing the system to advance or rewind the display in specific time increments using tactile, verbal, or other instructions. In one embodiment, the process display can be presented in the upper portion of the surgeon's field of view in the AR HMD. In some embodiments, the process display can be updated in real time. For example, as the surgeon moves the resection tool around the planned resection area, the process display can be updated so that the surgeon can see how his or her actions affect other factors of the surgery.

[0178] In some embodiments, rather than simply using the current state of the CASS 100 as input to the machine learning model, the input to the model may include the planned future state. For example, a surgeon may indicate that he or she is planning to perform a specific bone resection of the knee joint. This indication may be manually entered into the surgical computer 150, or the surgeon may provide the indication verbally. The surgical computer 150 may then generate a film showing the expected effect of the incision on the surgery. Such a film may show, at specific time increments, how the surgery would be affected if the intended course of action were to be performed, including, for example, changes in the patient's anatomy, changes in the position and orientation of the implant, and changes in the surgical procedure and instrumentation. The surgeon or medical professional may call up or request this type of film at any time during the surgery to preview how the intended course of action would affect the surgical plan if the intended action were to be performed.

[0179] It should be further noted that using a fully trained machine learning model and robotic CASS, various elements of the surgery can be automated, requiring minimal involvement by the surgeon, for example by providing approval for individual steps of the procedure. For example, over time, robotic control using an arm or other means can be gradually integrated into the surgical workflow, with the surgeon's manual interaction with the robotic operation gradually becoming less and less. In this case, the machine learning model can learn which robotic commands are required to achieve certain states planned by CASS. Ultimately, the machine learning model can be used to generate a film or similar view or display that predicts and allows for a preview of the entire surgery from an initial state. For example, an initial state can be defined that includes patient information, surgical plan, implant characteristics, and surgeon preferences. Based on this information, the surgeon can preview the entire surgery to confirm that the CASS-recommended plan meets the surgeon's expectations and / or requirements. Furthermore, since the output of the machine learning model is the state of CASS 100 itself, commands can be derived to control CASS components to achieve each predicted state. Thus, in extreme cases, an entire surgery can be automated based solely on initial state information.

[0180] Using a point probe to obtain high resolution in critical areas during hip surgery

[0181] The use of a point probe is described in U.S. patent application Ser. No. 14 / 955,742, entitled “Systems and Methods for Planning and Performing Image Free Implant Revision Surgery,” the entire contents of which are incorporated herein by reference. In short, an optically tracked point probe can be used to map the actual surface of a target bone where a new implant is needed. Mapping is performed after a defective or worn implant has been removed, and after any diseased or otherwise unwanted bone has been removed. By brushing or scraping the entire remaining bone with the tip of the point probe, multiple points can be collected on the bone surface. This is called tracking or “mapping” the bone. The collected points are used to create a three-dimensional model or surface map of the bone surface in a computer planning system. The created 3D model of the remaining bone is then used as the basis for planning the surgery and the necessary implant size. Alternative techniques for determining 3D models using X-rays are described in U.S. patent application Ser. No. 16 / 387,151, filed on April 17, 2019, and entitled “Three-Dimensional Selective Bone Matching,” and U.S. patent application Ser. No. 16 / 789,930, filed on February 13, 2020, and entitled “Three-Dimensional Selective Bone Matching,” the entire contents of each of which are incorporated herein by reference.

[0182] For hip applications, point probe mapping can be used to obtain high-resolution data of key areas such as the acetabular rim and acetabular fossa. This allows the surgeon to obtain a detailed view before starting reaming. For example, in one embodiment, a point probe can be used to identify the bottom (fossa) of the acetabulum. As is well known in the art, in hip surgery, it is important to ensure that the bottom of the acetabulum is not damaged during reaming to avoid damaging the medial wall. If the medial wall is inadvertently damaged, the operation will require an additional bone grafting step. With this in mind, information from the point probe can be used to provide operational guidance for the acetabular reamer during the surgical procedure. For example, the acetabular reamer can be configured to provide tactile feedback to the surgeon when the surgeon reaches the bottom or otherwise deviates from the surgical plan. Alternatively, the CASS100 can automatically stop the reamer when the bottom is reached or when the reamer is within a threshold distance.

[0183] As an added safeguard, the thickness of the area between the acetabulum and the medial wall can be estimated. For example, once the acetabular rim and acetabular socket have been mapped and registered to the preoperative 3D model, the thickness can be easily estimated by comparing the position of the acetabular surface with that of the medial wall. Using this knowledge, the CASS100 can provide an alert or other response if any surgical activity is predicted to protrude through the acetabular wall during reaming.

[0184] The point probe can also be used to collect high-resolution data of common reference points used when orienting the 3D model to the patient. For example, for pelvic plane landmarks like the ASIS and pubic symphysis, the surgeon can use the point probe to map the bones to represent the true pelvic plane. With a more complete view of these landmarks, the registration software will have more information to orient the 3D model.

[0185] Point probes can also be used to collect high-resolution data describing proximal femoral reference points that can be used to improve the accuracy of implant placement. For example, the relationship between the tip of the greater trochanter (GT) and the center of the femoral head is often used as a reference point for aligning femoral components during hip replacement surgery. Alignment is highly dependent on the correct position of the GT; therefore, in some embodiments, a point probe is used to map the GT to provide a high-resolution view of this area. Similarly, in some embodiments, a high-resolution view of the lesser trochanter (LT) may be useful. For example, during hip replacement surgery, the Dorr classification helps select a stem that will maximize the ability to achieve a press fit during surgery, thereby preventing micro-motion of the femoral component after surgery and ensuring optimal bone ingrowth. As understood in the art, the Dorr classification measures the ratio between the tube width at the LT and the tube width 10 cm below the LT. The accuracy of the classification is highly dependent on the correct position of the relevant anatomical structures. Therefore, it may be advantageous to map the LT to provide a high-resolution view of this area.

[0186] In some embodiments, a point probe is used to map the femoral neck to provide high-resolution data, allowing the surgeon to better understand where to make the neck incision. The navigation system can then guide the surgeon as he or she performs the neck incision. For example, as understood in the art, the femoral neck angle is measured by placing a line below the center of the femoral stem and a second line below the center of the femoral neck. Therefore, a high-resolution view of the femoral neck (and possibly the femoral stem) will provide a more accurate calculation of the femoral neck angle.

[0187] High-resolution femoral head and neck data can also be used to navigate resurfacing procedures, where software / hardware helps the surgeon prepare the proximal femur and place the femoral component. As is generally understood in the art, during hip resurfacing, the femoral head and neck are not removed; rather, the head is trimmed and covered with a smooth metal covering. In this case, it would be advantageous for the surgeon to map the femur and cap so that an accurate assessment of their respective geometries can be understood and used to guide the trimming and placement of the femoral component.

[0188] Register preoperative data to patient anatomy using a point probe

[0189] As described above, in some embodiments, a 3D model is developed during the preoperative phase based on 2D or 3D images of the anatomical region of interest. In such embodiments, registration between the 3D model and the surgical site is performed prior to the surgical procedure. The registered 3D model can be used to track and measure the patient's anatomy and surgical tools during the procedure.

[0190] During the surgical procedure, landmarks are acquired to facilitate registration of the preoperative 3D model to the patient's anatomy. For knee surgery, these points may include the center of the femoral head, distal femoral axis, medial and lateral epicondyles, medial and lateral malleolus, proximal tibial mechanical axis, and tibial A / P orientation. For hip surgery, these points may include the anterior superior iliac spine (ASIS), pubic symphysis, points along the acetabular rim and within the hemisphere, greater trochanter (GT), and lesser trochanter (LT).

[0191] During revision surgery, the surgeon may draw certain areas containing anatomical defects in order to better visualize and navigate implant insertion. These defects can be identified based on analysis of preoperative images. For example, in one embodiment, each preoperative image is compared to a library of images showing "healthy" anatomical structures (i.e., without defects). Any significant deviation between the patient image and the healthy image can be marked as a potential defect. Then, during surgery, the surgeon can be warned of the possible defect by a visual alarm on the display 125 of the CASS 100. The surgeon can then draw the area to provide more detailed information about the potential defect to the surgical computer 150.

[0192] In some embodiments, the surgeon can use non-contact methods to align incisions within the bone anatomy. For example, in one embodiment, laser scanning is used for alignment. A laser bar is projected onto the anatomical region of interest, and changes in the height of the region are detected as changes in the line. Other non-contact optical methods, such as white light interferometry or ultrasound, can also be used alternatively for surface height measurement or to align anatomical structures. For example, where there is soft tissue between the alignment point and the bone being aligned (e.g., the ASIS, pubic symphysis in hip surgery), ultrasound technology may be beneficial, thereby providing a more precise definition of the anatomical plane.

[0193] Fiducial marker assembly for optical tracking

[0194] As discussed herein, surgical procedures often utilize fiducial markers comprising light emitting diodes (LEDs) to track the location of an object of interest, such as a patient's bone or a surgical instrument. The LEDs emit light to be tracked by an optical tracking device, and a computing device can determine the location of the object of interest based on continuous monitoring of the emitted light. It would be advantageous to have fiducial markers configured for long-term use, however, standard LED housings are not suitable in this regard due to the use of autoclaves in typical sterilization procedures between uses. Autoclaves involve temperature cycles that impose large mechanical stresses on the housing and can cause the housing to crack. Furthermore, the steam cycles involved in autoclaves can cause electrical failures, particularly if the housing's seal is compromised. Due to the temperature and steam cycles, the transparency of the outlet window is also reduced, thereby impacting metering performance.

[0195] Now refer to Figures 9A-9B , shows an illustrative fiducial marker for tracking an object during a surgical procedure in accordance with an embodiment. The fiducial marker 900 can be a light emitting diode that is designed and constructed to be autoclaved according to standard medical sterilization procedures. The fiducial marker 900 can include an opaque housing 905, a window panel 910, a metallized coating 915, and a light emitting semiconductor die 920. The light emitting semiconductor die 920 can be connected (e.g., via bonding wires 935) to an anode 925 and a cathode 930 that extend through a portion of the opaque housing 905. In some embodiments, the fiducial marker 900 also includes a support rod 940 for supporting the light emitting semiconductor die 920.

[0196] Figure 9A A side view of a fiducial marker 900 is shown. As depicted, the opaque housing 905 may include a base 905A and a peripheral wall 905B defining an interior cavity 905C. In some embodiments, the opaque housing 905 is substantially cylindrical. Figure 9BAs shown in the top view of the reference marker 900 in FIG. 1 , the peripheral wall 905B may form a hollow circular cross-section. In additional embodiments, the opaque housing 905 may be formed into another shape. In some embodiments, the base 905A and the peripheral wall 905B are formed as a single unit. However, in additional embodiments, the base 905A and the peripheral wall 905B are formed as separate components. For example, the base 905A and the peripheral wall 905B may be formed from the same material or different materials and may be joined to form a fluid-tight seal capable of withstanding the conditions associated with autoclaving. The base 905A and the peripheral wall 905B may be joined by any technique known to those of ordinary skill in the art.

[0197] In some embodiments, the opaque shell 905 or its various components include a material that is configured to seal to the window panel 910. For example, where the window panel 910 is formed of glass, the opaque shell 905 can be formed of Kovar, a nickel-cobalt-iron alloy available from CRS Holdings, Inc. of Delaware, USA. Kovar is configured to have thermal expansion properties similar to those of glass so as to allow a tight mechanical bond to be formed between the Kovar material and the glass, thereby avoiding stressing or cracking the bond. The Kovar material of the opaque shell 905 can also be customized or optimized to match the thermal expansion properties of the material of the window panel 910. For example, the material and / or construction of the opaque shell 905 can be configured to optimize the coefficient of thermal expansion of the opaque shell. In some embodiments, the opaque shell 905 or its various components may include additional or alternative materials having adjustable thermal expansion properties and / or capable of forming a tight mechanical bond with the window panel 910.

[0198] In some embodiments, the anode 925 and cathode 930 extend through the base 905A. Figures 9A-9B As shown in , the anode 925 and cathode 930 may each include a cylindrical rod extending through the base 905A. In some embodiments, the anode 925 is oriented parallel to the cathode 930. However, additional shapes and configurations of the anode 925 and cathode 930 are contemplated herein. In some embodiments, the anode 925 and / or cathode 930 may alternatively be arranged on other portions of the opaque housing, such as the peripheral wall 905B. The anode 925 and cathode 930 may be formed as separate components from the base opaque housing 905. For example, the anode 925 and cathode 930 may be formed of a different material than the opaque housing 905 and may be joined to form a fluid-tight seal capable of withstanding the conditions associated with autoclaving. The anode 925 and cathode 930 may be joined to the opaque housing 905 by any technique known to those of ordinary skill in the art.

[0199] Reference again Figures 9A-9B, the window panel 910 is joined to the opaque housing 905 to enclose the interior cavity 905C. For example, the window panel 910 can be joined to the end of the peripheral wall 905B opposite the base 905A. However, alternative configurations are within the scope of the present disclosure. Figure 9B As shown in FIG, window panel 910 can be transparent to transmit light therethrough. In some embodiments, window panel 910 is completely or substantially transparent to minimize scattering and / or reflection of light as it passes through the window panel. However, in some embodiments, window panel 910 can be configured to be translucent or translucent. In some embodiments, window panel 910 is a flat panel to simplify the optical effects on the emitted light.

[0200] In some embodiments, the window panel 910 is formed of a variety of materials such as glass. In some embodiments, the window panel 910 includes aluminum oxide. In some embodiments, the window panel 910 includes aluminum (I) oxide (Al2O3), aluminum (II) oxide (AlO), and / or aluminum (III) oxide (Al2O3). The aluminum oxide used in the window panel 910 can be provided in a variety of forms. For example, the aluminum oxide can include alumina and / or corundum (e.g., sapphire, ruby, etc.). Additional or alternative materials can be used to form the window panel 910 to improve various properties of the window panel 910, as known to anyone of ordinary skill in the art.

[0201] In some embodiments, the material and / or construction of window panel 910 is configured to optimize the thermal expansion characteristics of window panel 910. For example, the material and / or construction of window panel 910 can be configured to optimize the coefficient of thermal expansion of window panel 910. In some embodiments, window panel 910 is configured to have thermal expansion characteristics similar to those of opaque housing 905 in order to minimize the effects of autoclaving on the construction of fiducial marker 900. When the thermal expansion characteristics of the components of fiducial marker 900 are similar, the effects of the autoclave process on the structure of fiducial marker 900 (e.g., stress, cracking, etc.) can be reduced or minimized. In some embodiments, window panel 910 is configured to have thermal expansion characteristics similar to one or more individual components of the opaque housing (e.g., perimeter wall 905B to which window panel 910 is coupled).

[0202] The window panel 910 can have a thickness configured to reduce refraction and / or scattering of light passing through the window panel 910. In some embodiments, the window panel 910 has a thickness of 20 mm, 15 mm, 10 mm, 9 mm, 8 mm, 7 mm, 6 mm, 5 mm, 4 mm, 3 mm, 2 mm, 1 mm, less than 1 mm, or various values ​​or ranges therebetween. The reduced thickness of the window panel 910 can reduce refraction and / or scattering of light passing through the window panel 910, thereby simplifying the optical correction to be performed by the system, as discussed further herein.

[0203] Reference again Figure 9A , the metallized coating 915 is provided at the interface between the opaque housing 905 and the window panel 910 to fuse the opaque housing 905 and the window panel 910 and form a fluid-tight seal (i.e., an airtight seal) therebetween. In some embodiments, the metallized coating 915 is a material that blocks and / or absorbs light. In some embodiments, the metallized coating 915 covers the entire interface between the opaque housing 905 and the window panel 910 so that the opaque housing 905 and the window panel 910 do not directly contact each other. For example, Figure 9A As shown in the cross-section in FIG, the metallized coating covers the entire shown surface of the interface. In some embodiments, the metallized coating 915 covers only a portion of the interface between the opaque housing 905 and the window panel 910, such as a radially inward portion of the interface.

[0204] Furthermore, the metallized coating 915 can extend around the entire perimeter of the window panel 910 to completely cover the junction and form a ring around the perimeter of the fiducial marker 900. Thus, the metallized coating 915 completely seals the opaque housing 905 and the window panel 910 in a fluid-tight manner (i.e., hermetically seals the interior cavity 905C from the external environment).

[0205] In some embodiments, the metallized coating 915 includes a solder material, that is, a fusible metal alloy that is configured to melt and applied to each component to produce a permanent bond. Soldering material can be applied as the metallized coating 915 using a soldering iron, a soldering gun, or another tool for heating and applying soldering material. In some embodiments, the metallized coating 915 includes a gold-tin solder. However, any soldering material known to those of ordinary skill in the art can be used, including but not limited to soft solder, hard solder, glass solder, tin-based solder (e.g., gold-tin solder, tin-lead solder, and tin-silver-copper solder), lead-based solder, indium-based solder, and lead-free solder. In some embodiments, the metallized coating 915 may include multiple layers of different materials and / or different types of solder.

[0206] In some embodiments, fiducial marker 900 further comprises a light-absorbing coating or layer facing window panel 910. In some embodiments, the light-absorbing coating can be affixed to the interior face of window panel 910, i.e., between window panel 910 and metallized coating 915. Thus, light internally reflected within the window can be absorbed by the light-absorbing coating to attenuate internal reflections and prevent additional reflections from metallized coating 915, thereby reducing the amount of reflections detectable by a system as further discussed herein. In some embodiments, the light-absorbing coating is black in order to absorb light. However, materials of additional or alternative colors can be used to absorb light. In some embodiments, the light-absorbing coating can be applied to the interior face of window panel 910 prior to applying metallized coating 915. However, the light-absorbing coating can be applied to a variety of additional or alternative surfaces in a variety of ways.

[0207] like Figures 9A-9B As shown in , the light emitting semiconductor die 920 can be positioned within the inner cavity 905C. Figure 9B , the light emitting semiconductor die can be centrally located relative to the perimeter wall 905B. The light emitting semiconductor die 920 can be connected to the anode 925 and cathode 930, for example, by bonding wires 935. However, alternative means for connecting the light emitting semiconductor die 920 to the anode 925 and cathode 930 are contemplated herein.

[0208] In some embodiments, the light emitting semiconductor die 920 is positioned adjacent to the window panel 910 to transmit light through the window panel 910 and reduce reflections of light from the light emitting semiconductor die 920 within the fiducial marker 900. For example, as the distance of the light emitting semiconductor die 920 from the window panel 910 increases, the amount of light emitted from the light emitting semiconductor die 920 on the base 905A and / or the peripheral wall 905B increases accordingly and may be reflected from these surfaces in various directions, including through the window panel 910. Additionally, as the distance of the light emitting semiconductor die 920 from the window panel 910 increases, the amount of light emitted from the light emitting semiconductor die 920 that reaches the peripheral edge of the window panel 910 (e.g., as the distance of the light emitting semiconductor die 920 from the window panel 910 increases). Figure 9A As can be seen in the figure, the amount of light that reaches the left and right outer sides or edges of the window panel 910 increases accordingly and can be reflected back into the housing and / or undergo multiple reflections within the window. Thus, reducing the distance of the light emitting semiconductor die 920 from the window panel 910 reduces internal light reflections, thereby simplifying the optical correction to be performed by the system as further discussed herein.

[0209] In some embodiments, the support rods 940 support the light emitting semiconductor die 920 at a position adjacent to the window panel 910. In some embodiments, the height of the support rods 940 is configured to minimize the distance between the light emitting semiconductor die 920 and the window panel 910. However, the height of the support rods 940 can vary. In some embodiments, the support rods 940 have a diameter that matches the diameter of the light emitting semiconductor die 920. In some embodiments, the support rods 940 have a diameter that is slightly larger than the diameter of the light emitting semiconductor die 920. In some embodiments, the support rods 940 have a diameter that is substantially equal to the diameter of the light emitting semiconductor die 920. In some embodiments, the support rods 940 have a diameter that is smaller than the diameter of the light emitting semiconductor die 920. By reducing the diameter of the support rods 940, the amount of light reaching and / or reflected from the support rods (e.g., light emitted from the light emitting semiconductor die 920) can be further reduced, thereby simplifying the optical correction to be performed by the system as further discussed herein.

[0210] The support rod 940 can be formed from a variety of materials. In some embodiments, the support rod 940 is formed from any material described with respect to the opaque housing 905. For example, the support rod 940 can be formed from the same material as the base 905A and / or the peripheral wall 905B. In some embodiments, the support rod 940 can be formed from a thermally conductive material. Therefore, the support rod 940 can also serve as a heat sink to dissipate the heat accumulated at the light-emitting semiconductor die 920. In some embodiments, the support rod 940 can be formed from a material configured to serve as one of the anode and cathode. Therefore, the support rod 940 can replace one of the anode 925 and cathode 930, or be configured as an extension of one of the anode 925 and cathode 930. In embodiments where the support rod 940 is configured as one of the anode 925 and cathode 930 or an extension thereof, the connection method of the anode 925 and cathode 930 can be changed accordingly. In some embodiments, when the support rod 940 serves as the anode 925, the light emitting semiconductor die 920 can be directly connected to the support rod 940 without using the bonding wire 935, and can be connected to the cathode 930 via the bonding wire 935. In some embodiments, when the support rod 940 serves as the anode 925, the bonding wire 935 can still be used to connect the light emitting semiconductor die 920 to the support rod 940. For example, the bonding wire 935 can connect the light emitting semiconductor die 920 (at the top of the support rod 940) to the portion of the support rod 940 near the base 905A.

[0211] Now refer to Figure 10 , shows a partial side view of an alternative fiducial marker for tracking an object during a surgical procedure, according to an embodiment. The fiducial marker 1000 may be a light emitting diode designed and constructed to be autoclaved according to standard medical sterilization procedures. Similar to Figures 9A-9B900, the fiducial marker 1000 may include an opaque housing 1005 (e.g., a base 1005A, a perimeter wall 1005B, and an inner cavity 1005C), a window panel 1010, a metallized coating 1015, a light emitting semiconductor die 1020, an anode 1025 and a cathode 1030 connected to the light emitting semiconductor die 1020 via bonding wires 1035, and a support rod 1040 for supporting the light emitting semiconductor die 1020. In some embodiments, the fiducial marker 1000 further includes a light absorbing coating or layer facing the window panel 1010 as described herein, for example, a black material fixed to the inner face of the window panel 1010. The fiducial marker 1000 may also include other components other than those distinguished herein as described with respect to FIG. Figures 9A-9B 900 . The perimeter wall 1005B may include a recess 1045 on a radially inwardly facing surface of the perimeter wall 1005B that defines the interface between the opaque housing 1005 and the window panel 1010. The recess 1045 may be configured to receive a portion of the window panel 1010 therein to form a stable interface between the opaque housing 1005 and the window panel 1010. As shown, the metallized coating 1015 at the interface between the opaque housing 1005 and the window panel 1010 may cover the entire interface (i.e., the entire surface of the recess 1045), including both horizontal and vertical portions of the interface. The recess 1045 provides a mating joint between the opaque housing 1005 and the window panel 1010 and additionally increases the surface area of ​​the interface that can be fused by the metallized coating 1015, thereby providing a more stable joint. However, in some embodiments, the metallized coating 1015 may only cover a portion of the junction, such as a horizontal portion of the junction.

[0212] Now refer to Figures 11A-11B , showing a partial side view of an alternative embodiment of a fiducial marker for tracking a subject during a surgical procedure, according to an embodiment ( Figure 11A ) and top view ( Figure 11B ). The fiducial marker 1100 may be a light emitting diode designed and constructed to be autoclaved according to standard medical sterilization procedures. Similar to Figure 101000, fiducial marker 1100 may include an opaque housing 1105 (e.g., a base (not shown), a perimeter wall 1105B, and an inner cavity 1105C), a window panel 1110, a metallized coating 1115, a light emitting semiconductor die 1120, an anode 1125 and a cathode 1130 connected to the light emitting semiconductor die 1120 via bonding wires 1135, support rods 1140 for supporting the light emitting semiconductor die 1120, and a recess 1145 in the perimeter wall 1105B for receiving the window panel 1110. In some embodiments, fiducial marker 1100 further includes a light absorbing coating or layer facing the window panel 1110 as described herein, for example, a black material secured to the inner face of the window panel 1110. Fiducial marker 1100 may also include features and configurations as depicted and described with respect to fiducial marker 900 and / or fiducial marker 1000, except as distinguished herein. The window panel 1110 may include a notch 1150 on an upper peripheral surface of the window panel 1110. The notch 1150 may be configured to cooperate with a portion of a marker support 1155 to secure the fiducial marker 1100 thereto. The cooperation of the notch 1150 with the marker support 1155 clamps the fiducial marker into the marker support 1155 with high mechanical stability. The marker support 1155 can be part of any device that is intended to be tracked by an optical tracking system. In some embodiments, the marker support 1155 can be part of a surgical instrument. In some embodiments, the marker support 1155 can be a tracking support or tracking array for attaching to a portion of a patient's anatomy (e.g., a bone) or other object of interest. In some embodiments, the marker support 1155 can be a tracking support or tracking array for attaching to a location of interest (e.g., a reference point).

[0213] In some embodiments, the metallized coating 1115 may include a material that blocks and / or absorbs light and may extend beyond the intersection of the opaque housing 1105 and the window panel 1110. Figures 11A-11B As shown in FIG, the metallized coating 1115 forms a ring around the perimeter of the window panel 1110. The ring further extends radially inward beyond the junction to cover the lower surface of the window panel 1110, thereby preventing light from being transmitted through a portion of the window panel 1110. In some embodiments, the window panel 1110 and the metallized coating 1115 are configured to block light from the light emitting semiconductor die 1120 from reaching the peripheral edge of the window panel 1110 (e.g., as shown in FIG). Figure 11A1150 and the indicia support 1155). As seen in FIG. 115 , the left and right outer sides or edges of the window panel 1110 are blocked. In other words, light that would otherwise reach the peripheral edge of the window panel 1110 is instead blocked and / or absorbed by the metallized coating 1115, thereby reducing internal light reflections. Furthermore, the window panel 1110 and the metallized coating 1115 are configured to block light from the light-emitting semiconductor die 1120 from reaching the clamping member (i.e., the recess 1150 and the indicia support 1155), thereby reducing light reflections from the clamping member. Light can be blocked from reaching the peripheral edge of the window panel 1110, the recess 1150, and the indicia support 1155 by reducing the inner diameter of the ring formed by the metallized coating 1115, thereby increasing the coverage of the metallized coating 1115 on the window panel 1110.

[0214] like Figure 11B As shown in FIG, the widened ring of metallized coating 1115 can cover a substantial portion of the lower surface of the window panel. The window panel 1110 can also maintain a sufficient uncovered portion (i.e., defined by the inner diameter of the ring, as shown in FIG. Figure 11B 10 ) to facilitate detection of light from the light emitting semiconductor die 1120 from a sufficiently wide array of angles required in typical surgical procedures (e.g., as a marker support 1155). In some embodiments, the inner diameter of the ring is 0.05 cm, 0.1 cm, 0.2 cm, 0.3 cm, 0.4 cm, 0.5 cm, 0.6 cm, 0.7 cm, 0.8 cm, 0.9 cm, 1 cm, 2 cm, 3 cm, 4 cm, 5 cm, greater than 5 cm, or various values ​​or ranges therebetween. In some embodiments, the metallized coating 1115 covers a percentage of the lower surface of the window panel 1110, such as 5%, 10%, 20%, 30%, 40%, 50%, 60%, 70%, 80%, 90%, greater than 90%, or various values ​​or ranges therebetween.

[0215] In some embodiments, the fiducial marker 1100 is configured to prevent light from reaching the peripheral edge of the window panel 1110 and / or the clamping member based on the configuration of various parameters. For example, the inner diameter of the ring of the metallized coating 1115, the thickness of the window panel 1110, the diameter of the window panel 1110, and the distance of the light emitting semiconductor die 1120 from the window panel 1110 can all have an impact on the direction and reach of light from the light emitting semiconductor die 1120. Therefore, these parameters can be configured together to prevent light from reaching the peripheral edge of the window panel 1110 and / or the clamping member based on the configuration of the various parameters.

[0216] Embodiments of the disclosed fiducial markers offer significant advantages over conventional fiducial markers. For example, the optical effects on light from the light-emitting semiconductor die are greatly simplified, facilitating calculations as further described herein. Furthermore, the window panel and other components can be manufactured using standard techniques with acceptable tolerances, resulting in a simple design. For example, a flat window panel can be manufactured using standard techniques, and slight variations in the thickness of the window panel and / or metallized coating have a negligible effect on refraction and other optical effects. Therefore, these factors do not need to be considered or accounted for during subsequent calculations.

[0217] System for optically tracking an object having a fiducial marker assembly

[0218] Now refer to Figure 12 , shows a block diagram of an illustrative system for tracking an object according to an embodiment. In some embodiments, system 1200 is a surgical system or a robotic surgical system. Figure 12 As shown in FIG, system 1200 may include a computing device 1205, an optical tracking device 1210, and one or more fiducial markers 1215. The fiducial marker 1215 may be any of the embodiments depicted or described herein (e.g., fiducial marker 900 of FIG. 9, Figure 10 11 ). In some embodiments, the fiducial marker 1215 is coupled to a marker support 1220, such as a surgical instrument, a tracking support, or a tracking array.

[0219] In some embodiments, the computing device 1205 is a processor, a computing device of a CASS (e.g., 150), or other type of computing or data processing system as described herein. The computing device 1205 can be in electronic communication with the optical tracking device 1210 to receive signals from the optical tracking device. In some embodiments, the electronic communication between the computing device 1205 and the optical tracking device 1210 can be wired. In additional embodiments, the electronic communication between the computing device 1205 and the optical tracking device 1210 can be performed via a wireless transmission system.

[0220] In some embodiments, the optical tracking device 1210 includes two or more optical sensors 1225 ( Figure 13). In some embodiments, the optical tracking device 1210 includes more than two optical sensors 1225. In some embodiments, the optical sensors 1225 are cameras, although other CMOS or CCD sensors may be used. In some embodiments, the cameras are configured to detect light within a specific wavelength range. For example, the cameras may be configured to detect light in the near-IR spectrum, the visible spectrum, or a subset of the visible spectrum (e.g., wavelengths associated with a specific color or colors). The optical sensors 1225 of the optical tracking device 1210 may be configured to detect light emitted from the fiducial marker 1215.

[0221] In some embodiments, for example Figure 13 As shown in FIG, the reference marker 1215 includes a housing 1230, a window panel 1235, and a light source 1240. When the light source 1240 emits light, the light passes through the window panel 1235 and leaves the reference marker 1215. The optical sensors 1225 each sense the position of the light emitted from the light source 1240. Figure 8 As shown and discussed, light rays from light source 1240 may be refracted as they pass through window panel 1235 such that the light rays do not form continuous straight lines.

[0222] Now refer to Figure 13 , shows an illustrative calculation of the positioning of the light source with refraction correction according to an embodiment. The perceived position of each ray 1245 (in Figure 13 ) are detected by corresponding optical sensors 1225 and transmitted via electronic communication to computing device 1205. Computing device 1205 calculates and applies the refractive deviation 1250 of the position of each ray 1245, thereby generating adjusted ray positions 1255 (in Figure 13 1235) to correct for refraction. A refractive error 1250 is calculated based on the refractive index of window panel 1235, the thickness of window panel 1235, and the angle of light ray 1245 with window panel 1235. The angle of light ray 1245 with the window panel varies based on the orientation of fiducial marker 1215 relative to each optical sensor 1225, and thus, refractive error 1250 may be different for each optical sensor 1225. Additional optical properties of window panel 1235 may be included in refractive error 1250, as known to those of ordinary skill in the art.

[0223] In some embodiments, the orientation of the fiducial marker 1215 relative to each optical sensor 1225 can be determined by the computing device 1205. For example, the optical tracking device 1210 can collect image information related to the fiducial marker 1215 and transmit this information to the computing device 1205. The computing device 1205 can determine the orientation of the fiducial marker 1215 based on the image information and the known geometry of the fiducial marker 1215. In the case of a marker support, the optical tracking device 1210 can collect image information related to the marker support 1220 and transmit this information to the computing device 1205. The computing device 1205 can determine the orientation of the fiducial marker 1215 based on the image information, the known geometry of the marker support 1220, and the known orientation of the fiducial marker 1215 relative to the marker support 1220. In some embodiments, an orientation sensor such as an accelerometer can be coupled to the fiducial marker 1215 and / or the marker support 1220, and the accelerometer can relay the orientation information to the computing device 1205 to calculate the orientation of the fiducial marker. Additional types of orientation sensors known to those of ordinary skill in the art may additionally or alternatively be used in a similar manner.In some embodiments, additional systems and methods may be used to determine the orientation of the fiducial marker 1215, as will be apparent to those of ordinary skill in the art.

[0224] Based on the adjusted ray positions 1255, the true position of the light source 1240 can be calculated by triangulation. Without correcting for refraction, the position of the light source 1240 can include significant metrological errors, resulting in Figure 13 The perceptual positioning 1260 is shown in for comparison. Figure 13 Refraction deviations and corrections in two dimensions are shown, but refraction occurs in three dimensions and therefore significant metrology errors can exist in both the transverse and longitudinal directions.The system 1200 and techniques described herein can correct for refraction in both the transverse and longitudinal directions.

[0225] In some embodiments, the positioning of the light source 1240 is calculated iteratively. For example, the computing device 1205 can determine the orientation of the fiducial marker 1215 based on image information, the known geometry of the marker support 1220, and / or the known orientation of the fiducial marker 1215 relative to the marker support 1220. The computing device 1205 can calculate the positioning of the light source 1240 based in part on the determined orientation of the fiducial marker 1215 using the techniques described herein. The calculated position of the light source 1240 can be used to calculate a new value for the orientation of the fiducial marker 1215. This new value can be used for a second iterative calculation of the positioning of the light source 1240, which can provide a more accurate result. This process can be repeated for any number of iterations. In some embodiments, the process is repeated until a predetermined accuracy measure is achieved. For example, the process can be repeated until the change in the positioning of the light source 1240 determined in consecutive iterations is below a predetermined threshold.

[0226] Based on the positioning of light source 1240, the position of the object of interest connected thereto can be calculated using the known spatial relationship between light source 1240 and the object of interest. For example, light source 1240 can be at a known distance from the object of interest based on the size and orientation of fiducial marker 1215, the size and orientation of marker support 1220, and / or the position of the object of interest coupled thereto. In some embodiments, system 1200 is configured to calculate the position of the object of interest based on an algorithm that includes the techniques described herein. In some embodiments, system 1200 is configured to calculate the position of the object of interest in real time. In some embodiments, system 1200 is configured to continuously calculate and monitor the position of the object of interest over a period of time, such as the duration of a surgical procedure.

[0227] In some embodiments, the system 1200 is configured to calculate and monitor the positioning of an object of interest based on a plurality of fiducial markers 1215 (e.g., two, three, four, or more fiducial markers 1215) attached thereto. In some embodiments, the plurality of fiducial markers 1215 may be supported by a single marker support 1220. In some embodiments, each fiducial marker 1215 may be uniquely oriented relative to the object of interest. When calculating the positioning of the object of interest in any given example, the system 1200 may select a fiducial marker 1215 for calculation based on its tilt angle relative to the optical sensor 1225. Tilt angles may increase positioning errors in a manner known to the computing device 1205 (e.g., where tilt angles are particularly high), so appropriate fiducial markers 1215 may be selected for calculation to reduce positioning errors. In some embodiments, calculations may be performed based on a plurality of fiducial markers 1215, and the calculated positioning from each fiducial marker 1215 may be averaged to generate a more accurate overall calculation of the positioning of the object of interest.

[0228] Now refer to Figures 14A-14C, showing an exemplary evaluation of positioning error according to an embodiment. Figure 14A shows the positioning error in the lateral direction, Figure 14B shows the positioning error in the longitudinal direction, and Figure 14C shows the total positioning error. Figures 14A-14C In each of the figures, the solid line shows the positioning error as a function of the average tilt angle without correction for refraction, and the dashed line shows the positioning error as a function of the average tilt angle with correction for refraction using the techniques described herein. Without correction for refraction, the positioning error quickly rises to over 0.1 mm for tilt angles greater than 20°. After correction for refraction, a significant reduction in positioning error can be achieved up to very large tilt angles.

[0229] In some embodiments, the system 1200 also includes one or more displays in wired or wireless electronic communication with the computing device 1205. The one or more displays can display the location of the object of interest and / or related information to the user. In embodiments, the one or more displays may include digital displays. Any collected or calculated data described herein can be displayed to the user in real time on the one or more displays. In addition, as will be apparent to one of ordinary skill in the art, alternative or additional information can be provided to the user within the scope of this disclosure. In some embodiments, the display can be an external screen or monitor, a tablet computer, a display of a CASS, and / or an augmented reality headset worn by the user.

[0230] It should be noted that the metrological accuracy of the techniques described herein can be further improved by using the fiducial markers described herein. Fiducial markers 900, 1000, and 1100 provide a sufficiently simple design to facilitate calculation of the marker positions with acceptable accuracy. Additionally, fiducial markers 900, 1000, and 1100 include features that reduce refraction and other optical effects to simplify calculations and minimize the magnitude of corrections.

[0231] Method for optically tracking an object having a fiducial marker assembly

[0232] Now refer to Figure 15 , a flowchart 1500 of an illustrative method of tracking an object with a fiducial marker is shown in accordance with an embodiment. Figure 15As shown in FIG, a fiducial marker (e.g., fiducial marker 900, fiducial marker 1000, or fiducial marker 1100) including a light source is coupled 1505 to an object. In some embodiments, a marker support may be used to couple 1505 the fiducial marker to the object. The position of a light ray emitted by the light source is detected 1510 by each of a plurality of optical sensors of an optical tracking device, and the positions of the light rays are received 1515 from the optical tracking device by a computing device. The position of each light ray is adjusted 1520 by the computing device based on refractive deviation to obtain an adjusted position of each light ray. The position of the light source is triangulated 1525 based on the adjusted position of each light ray, and the position of the object is calculated 1530 based on the position of the light source.

[0233] In some embodiments, the method further includes registering a fiducial marker with a computing device. For example, when an object is placed in a known position, light from a light source can be detected so that the computing device can determine a spatial relationship between the light source and the object. In some embodiments, the spatial relationship can be determined based on input from a user. In some embodiments, the spatial relationship can be determined based on known parameters. For example, the light source can be at a known distance from the object based on the size and orientation of the fiducial marker, the size and orientation of the marker support, and / or the location of the light source coupled to the object of interest. As will be apparent to one of ordinary skill in the art, additional registration or calibration procedures can be implemented.

[0234] In some embodiments, method 1500 includes attaching a plurality of fiducial markers to an object. In some embodiments, each fiducial marker can be uniquely oriented relative to the object. In some embodiments, method 1500 also includes selecting one of the plurality of fiducial markers for calculation based on an angle of inclination relative to an optical sensor. The angle of inclination can increase positioning error in a manner known to the computing device, and therefore, an appropriate fiducial marker can be selected to reduce positioning error. In some embodiments, selecting the fiducial marker includes selecting the fiducial marker with the lowest angle of inclination. In some embodiments, the fiducial marker is selected based on a known relationship between angle of inclination and positioning error.

[0235] In some embodiments, method 1500 includes calculating the position of the object as described based on each of two or more fiducial markers in the plurality of fiducial markers. In some embodiments, the position of the object calculated from each fiducial marker can be averaged to generate a more accurate overall calculation of the position of the object.

[0236] In some embodiments, method 1500 further includes displaying the location of the object and / or related information on a display. In embodiments, one or more displays may include a digital display. Any collected or calculated data described herein may be displayed to the user in real time on one or more displays. In addition, as will be apparent to one of ordinary skill in the art, alternative or additional information may be provided to the user within the scope of this disclosure. In some embodiments, the display may be an external screen or monitor, a tablet computer, a display of a CASS, and / or an augmented reality headset worn by the user.

[0237] The devices, systems, and methods described herein are not intended to be limited in aspect to the specific embodiments described, which are intended only as illustrations of various features. Many modifications and variations may be made to the devices, systems, and methods without departing from the spirit and scope as would be apparent to one skilled in the art.

[0238] Additionally, the embodiments depicted and described herein are intended to be exemplary, and not to limit the scope of the subject matter herein. It is contemplated that features or configurations described with respect to one disclosed embodiment may be applied to additional disclosed embodiments within the scope of the present disclosure.

[0239] In some embodiments, the systems and methods described herein can be used to simultaneously calculate and monitor the positioning of multiple objects of interest. In some embodiments, each object of interest includes one or more fiducial markers coupled thereto. In some embodiments, the fiducial markers for each object of interest are distinguished by a recognizable pattern or arrangement of the fiducial markers. In some embodiments, the fiducial markers for each object of interest are distinguished by the wavelength of light emitted from the fiducial markers. For example, the fiducial markers for each object of interest can emit light of different colors or different wavelength ranges. When these distinguishing features are known to the computing device (e.g., through user input, registration, or calibration procedures), each object of interest can be tracked simultaneously and individually in real time.

[0240] In some embodiments, one or more components of the present disclosure may be used independently. For example, any of the fiducial markers described herein may be implemented without a system that corrects for refraction. In some embodiments, the window panel may be thin enough so that refraction is negligible, and positioning may be calculated with acceptable accuracy without refraction correction. Furthermore, in some embodiments, the methods of tracking objects as described herein may be used with other fiducial markers besides those described.

[0241] The methods, systems, and devices described herein can be used in various medical procedures where tracking one or more objects can facilitate planning and / or performing the surgical procedure. By way of non-limiting example, the methods, systems, and devices can be used in orthopedic procedures including arthroplasty, arthroscopic procedures, spinal procedures, maxillofacial procedures, rotator cuff procedures, ligament repair and replacement procedures, reconstructive surgery, ear, nose, and throat procedures, neurological procedures, and other types of medical procedures that will be apparent to one of ordinary skill in the art.

[0242] Figure 16 A block diagram of an illustrative data processing system 1600 in which embodiments are implemented is shown. Data processing system 1600 is an example of a computer, such as a server or client, in which computer usable code or instructions implementing the processes of exemplary embodiments of the present invention are located. In some embodiments, data processing system 1600 may be a server computing device. For example, data processing system 1600 may be implemented in a server or another similar computing device operably connected to surgical system 100 as described above. Data processing system 1600 may be configured to transmit and receive information related to a patient and / or a surgical plan associated with surgical system 100, for example.

[0243] In the depicted example, data processing system 1600 may employ a hub architecture including north bridge and memory controller hub (NB / MCH) 1601 and south bridge and input / output (I / O) controller hub (SB / ICH) 1602. Processing unit 1603, main memory 1604, and graphics processor 1605 may be connected to NB / MCH 1601. Graphics processor 1605 may be connected to NB / MCH 1601 through, for example, an accelerated graphics port (AGP).

[0244] In the depicted example, a network adapter 1606 is connected to the SB / ICH 1602. An audio adapter 1607, a keyboard and mouse adapter 1608, a modem 1609, a read-only memory (ROM) 1610, a hard disk drive (HDD) 1611, an optical drive (e.g., a CD or DVD) 1612, a universal serial bus (USB) port and other communication ports 1613, and PCI / PCIe devices 1614 can be connected to the SB / ICH 1602 via a bus system 1616. The PCI / PCIe devices 1614 can include Ethernet adapters, add-in cards, and PC cards for laptop computers. The ROM 1610 can be, for example, a flash memory basic input / output system (BIOS). The HDD 1611 and the optical drive 1612 can use an integrated drive electronics (IDE) or serial advanced technology attachment (SATA) interface. A super I / O (SIO) device 1615 can be connected to the SB / ICH 1602.

[0245] An operating system may run on processing unit 1603. The operating system may coordinate and provide control of various components within data processing system 1600. As a client, the operating system may be a commercially available operating system. For example, Java TM An object-oriented programming system such as a programming system can run in conjunction with the operating system and provide calls to the operating system from object-oriented programs or application programs executing on data processing system 1600. As a server, data processing system 1600 can be a server running Advanced Interactive Executive operating system or Linux operating system. eServer TM Data processing system 1600 may be a symmetric multiprocessor (SMP) system, which may include multiple processors in processing unit 1603. Alternatively, a single processor system may be employed.

[0246] Instructions for the operating system, object-oriented programming system, and application programs or programs are located on a storage device, such as HDD 1611, and are loaded into main memory 1604 for execution by processing unit 1603. The processes of the embodiments described herein may be performed by processing unit 1603 using computer-usable program code, which may be located in a memory, such as main memory 1604, ROM 1610, or in one or more peripheral devices.

[0247] The bus system 1616 may be composed of one or more buses. The bus system 1616 may be implemented using any type of communication structure or architecture that can provide data transmission between different components or devices attached to the structure or architecture. A communication unit such as a modem 1609 or a network adapter 1606 may include one or more devices that can be used to transmit and receive data.

[0248] Those skilled in the art will understand that Figure 16 The hardware depicted in the figures may vary depending on the implementation. Other internal hardware or peripheral devices, such as flash memory, equivalent non-volatile memory, or optical disk drives, may be used in addition to or in place of the hardware shown. Furthermore, data processing system 1600 may take the form of any of a number of different data processing systems, including but not limited to a client computing device, a server computing device, a tablet computer, a laptop computer, a telephone or other communication device, a personal digital assistant, and the like. Essentially, data processing system 1600 may be any known or later developed data processing system without architectural limitations.

[0249] Although various exemplary embodiments incorporating the principles of the present teachings have been disclosed, the present teachings are not limited to the disclosed embodiments. Rather, this application is intended to cover any variations, uses, or modifications of the present teachings, and to apply their general principles. Furthermore, this application is intended to cover departures from the present disclosure that fall within known or customary practices in the art to which these teachings pertain.

[0250] In the above detailed description, reference is made to the accompanying drawings which form a part thereof. In the accompanying drawings, similar symbols generally identify similar parts unless the context otherwise dictates. The illustrative embodiments described in this disclosure are not meant to be restrictive. Other embodiments may be used, and other changes may be made without departing from the spirit or scope of the subject matter presented herein. It is readily understood that the various features of the present disclosure (as generally described herein and illustrated in the accompanying drawings) may be arranged, substituted, combined, separated, and designed into a variety of different configurations, all of which are expressly contemplated herein.

[0251] The present disclosure is not limited to the specific embodiment aspects described in this application, which are intended to serve as illustrations of various features. Without departing from the spirit and scope that are apparent to those skilled in the art, many modifications and variations may be made. According to the foregoing description, functionally equivalent methods and devices (except those listed herein) within the scope of the present disclosure will be apparent to those skilled in the art. It should be understood that the present disclosure is not limited to specific methods, reagents, compounds, compositions or biological systems, which can certainly vary. It should also be understood that the terms used herein are only used to describe the purpose of specific embodiments, and are not intended to be restrictive.

[0252] With respect to the use of substantially any plural and / or singular terms herein, those skilled in the art may translate from the plural to the singular and / or from the singular to the plural, as appropriate to the context and / or application. For clarity, the various singular / plural permutations may be expressly set forth herein.

[0253] Those skilled in the art will understand that, in general, the terms used herein are generally intended to be "open" terms (e.g., the term "including" should be interpreted as "including but not limited to," the term "having" should be interpreted as "having at least," the term "comprising" should be interpreted as "including but not limited to," etc.). Although various compositions, methods, and apparatus are described as "comprising" various components or steps (interpreted to mean "including but not limited to"), the compositions, methods, and apparatus may also "consist essentially of" or "consist of" the various components and steps, and such terms should be interpreted as defining substantially closed groups of components.

[0254] In addition, even if a specific number is explicitly recited, those skilled in the art will recognize that such recitation should be interpreted as meaning at least the recited number (e.g., reciting "two items" without other modifiers means at least two items or two or more items). Furthermore, in those instances where phraseology similar to "at least one of A, B, and C, etc." is used, generally, such construction is intended so that those skilled in the art will understand the meaning of the phrase (e.g., "a system having at least one of A, B, and C" would include, but is not limited to, systems having only A, only B, only C, A and B together, A and C together, B and C together, and / or A, B, and C together, etc.). In those instances where phraseology similar to "at least one of A, B, or C, etc." is used, generally, such construction is intended so that those skilled in the art will understand the meaning of the phrase (e.g., "a system having at least one of A, B, or C" would include, but is not limited to, systems having only A, only B, only C, A and B together, A and C together, B and C together, and / or A, B, and C together, etc.). Those skilled in the art will also understand that, whether in the specification, sample examples, or figures, almost any transition word and / or phrase presenting two or more alternative terms should be understood to include the possibility of one, either, or both terms. For example, the phrase "A or B" will be understood to include the possibility of "A" or "B" or "A and B."

[0255] In addition, where features of the disclosure are described in terms of Markush groups, those skilled in the art will recognize that the disclosure is also described in terms of any individual member or subgroup of members of the Markush group.

[0256] It will be understood by those skilled in the art that, for any and all purposes, for example, with respect to providing a written description, all ranges disclosed herein also encompass any possible subranges and all possible subranges and combinations thereof. Any listed range can be easily considered to fully describe and achieve decomposition into at least equal half, one-third, one-quarter, one-fifth, one-tenth, etc., the same range. As a non-limiting example, each range discussed herein can be easily decomposed into the lower third, middle third, and upper third, etc. It will also be understood by those skilled in the art that all languages ​​such as "up to," "at least," etc. include narrated numbers and refer to the ranges that can be subsequently decomposed into subranges as described above. Finally, it will be understood by those skilled in the art that a range includes each individual member. Therefore, for example, a group having 1-3 cells refers to a group having 1, 2, or 3 cells. Similarly, a group having 1-5 cells refers to a group having 1, 2, 3, 4, or 5 cells, and so on.

[0257] As used herein, the term "about" refers to a variation in a numerical amount that may occur, for example, through measurement or processing procedures in the real world, through inadvertent errors in these procedures, through differences in the manufacture, source or purity of compositions or reagents, and the like. Typically, the term "about" as used herein refers to a value or range of values ​​that is greater than or less than 1 / 10 (e.g., ±10%) of the value being described. The term "about" also refers to variations that a person skilled in the art can understand as equivalent, as long as such variations do not contain known values ​​practiced in the prior art. Each value or range of values ​​following the term "about" is also intended to encompass embodiments of the absolute value or range of values. Whether or not modified by the term "about," the quantitative values ​​cited in this disclosure include equivalents to the cited values, for example, numerical variations of such values ​​that may occur, but those skilled in the art will recognize equivalents.

[0258] The various features and functions disclosed above and their alternatives can be combined into many other different systems or applications. Those skilled in the art may subsequently make various currently unforeseen or unexpected alternatives, modifications, changes or improvements, each of which is also intended to be covered by the disclosed embodiments.

Claims

1. An optical tracking system comprising: An autoclavable fiducial marker assembly, the autoclavable fiducial marker assembly comprising: an opaque shell defining an inner cavity, a light emitting semiconductor die disposed in the cavity and in electrical communication with the anode and the cathode, a window panel bonded to the opaque housing to enclose the interior cavity between the window panel and the opaque housing, the window panel being configured to refract a plurality of light rays emitted by the light emitting semiconductor die, and a metallized coating that forms an airtight seal at the junction of the window panel and the opaque housing, wherein the fiducial marker assembly is configured to shield a peripheral edge of the window panel from the plurality of light rays; a tracking device comprising at least two optical sensors, each optical sensor configured to detect a position of a light ray of the plurality of light rays; processor; and a non-transitory computer-readable medium storing instructions that, when executed, cause the processor to: The position where each light is received from each optical sensor, shifting the position of each ray based on the calculated refractive deviation, and The positioning of the light emitting semiconductor die is triangulated based on the shifted position of each light ray. 2 . The optical tracking system of claim 1 , wherein the calculated refractive deviation is based on a known refractive index of the window panel, a known thickness of the window panel, and an orientation of the fiducial marker assembly relative to each optical sensor.

3. The optical tracking system of claim 2, wherein the tracking device is configured to detect image information related to the orientation of the fiducial marker assembly.

4. The optical tracking system of claim 2, further comprising an accelerometer configured to detect and transmit orientation information related to the orientation of the fiducial marker assembly.

5. The optical tracking system of claim 1 , wherein the metallized coating forms a ring extending radially inward from the junction to cover a portion of the window panel, wherein the ring is configured to shield a peripheral edge of the window panel from the plurality of light rays. The optical tracking system of claim 1 , wherein the metallized coating comprises solder.

7. The optical tracking system of claim 1, wherein the anode and the cathode extend through the opaque housing.

8. The optical tracking system of claim 1, wherein the opaque housing defines a recess configured to receive the window panel.

9. The optical tracking system of claim 1, wherein the window panel defines a recess configured to secure the fiducial marker assembly within a marker support.

10. The optical tracking system of claim 1, wherein the fiducial marker assembly further comprises a rod supporting the light emitting semiconductor die adjacent the window panel.

11. The optical tracking system of claim 10, wherein the rod comprises a heat sink configured to remove heat from the light emitting semiconductor die.

12. The optical tracking system of claim 10, wherein a diameter of the light emitting semiconductor die is greater than or equal to a diameter of the rod.

13. The optical tracking system of claim 10, wherein the coefficient of thermal expansion of each of the opaque housing, the window panel, and the rod is substantially equal.

14. The optical tracking system of claim 10, wherein: The opaque shell comprises a nickel-cobalt-iron alloy; The window panel comprises aluminum oxide; and The rod comprises a nickel-cobalt-iron alloy.

15. The optical tracking system of claim 1, wherein the window panel includes a light absorbing layer affixed to an inner surface of the window panel facing the metallized coating.

Images