Automatic detection of tracking array motion during navigated surgery

Abstract

1. A system and method for detecting movement of a first navigation array relative to a bone during computer-assisted surgery, comprising: monitoring a location of a landmark on the bone using a first navigation array, the landmark being at a first end of the bone and the first navigation array being adjacent to a second end of the bone; determining that the location of the landmark has moved a distance that is greater than a threshold; and indicating suspicious activity when the distance is greater than the threshold.

Description

[Technical Field]

[0001] (CROSS-REFERENCE TO RELATED APPLICATIONS) This international patent application claims the benefit of U.S. Non-provisional Patent Application No. 18 / 153,855, filed January 12, 2023, the disclosure of which is incorporated herein by reference in its entirety.

[0002] FIELD OF THE INVENTION Various exemplary embodiments disclosed herein relate to automatic detection of tracking array motion during navigated surgery based on analysis of landmark motion. [Background technology]

[0003] A computer-assisted surgical system may include a robotic arm, a controller, and a navigation system. Robotic or robot-assisted surgery has many associated advantages, particularly with respect to precise placement of surgical tools and / or implants. For example, in multi-step surgeries such as, by way of non-limiting example, drill, tap, and screw techniques, the conceptual ability of a robotic surgical system to track the position and / or orientation of a first tool to achieve a desired trajectory and then return precisely to the same position and / or orientation as a second tool in the desired trajectory is particularly advantageous. Summary of the Invention [Problem to be solved by the invention]

[0004] Whatever the principle of a computer-assisted surgery system, the position of the tool relative to the bone is entirely dependent on the navigation arrays rigidly attached to each element. Any event that affects the rigid attachment of the elements to their respective navigation arrays introduces errors, thereby reducing the accuracy of the system. Therefore, there is a need for systems, devices, and methods that improve computer-assisted surgery systems, for example, by automatically detecting when the tracking array moves relative to the bone. [Means for solving the problem]

[0005] A summary of various exemplary embodiments is presented below. In the following summary, some simplifications and omissions may be made in order to highlight and introduce some aspects of various exemplary embodiments and are not intended to limit the scope of the present invention. Detailed descriptions of exemplary embodiments adequate to enable those skilled in the art to make and use the concepts of the present invention follow in the following sections.

[0006] Various embodiments relate to a method of detecting movement of a first navigation array relative to a bone during computer-assisted surgery, including monitoring a location of a landmark on the bone using the first navigation array, the landmark being at a first end of the bone and the first navigation array being adjacent to a second end of the bone; determining that the location of the landmark has moved a distance that is greater than a threshold; and indicating suspicious activity when the distance is greater than the threshold.

[0007] Various embodiments are described that further include monitoring a location of a second navigation array, analyzing the similarity of the movement of the second navigation array and the landmark, and determining that the suspicious activity is camera movement when the movement of the second navigation array and the landmark is similar.

[0008] Various embodiments are described in which analyzing the similarity of the motion between the second navigation array and the landmark includes monitoring the frame-to-frame motion of the landmark and the second navigation array and comparing the frame-to-frame motion of the landmark to the frame-to-frame motion of the second navigation array.

[0009] Various embodiments are described that further include calculating bone motion information and analyzing landmark metrics based on the bone motion information to determine whether the bone motion information indicates that the suspected activity is movement of the array.

[0010] Various embodiments are described that further include analyzing landmark metrics based on the bone movement information to determine whether the bone movement information indicates that the suspicious activity is bone movement.

[0011] Various embodiments are described in which the computer-assisted surgery is knee surgery and the landmark is the hip joint center.

[0012] Various embodiments are described in which the landmarks are at locations that move less than a threshold during computer-assisted surgery.

[0013] Various embodiments are described that further include monitoring a location of a second navigation array, and using a machine learning model to analyze the similarity of the movement of the second navigation array and a landmark, and determining that the suspicious activity is camera movement when the movement of the second navigation array and the landmark is similar.

[0014] Various embodiments are described that further include calculating bone motion information and using a machine learning model to analyze landmark metrics based on the bone motion information to determine whether the bone motion information indicates that the suspected activity is array motion or bone motion.

[0015] Further various embodiments relate to a method of detecting movement of a first navigation array relative to a femur during computer-assisted knee surgery, including monitoring a location of a hip joint center of the femur using the first navigation array, the first navigation array being adjacent to the knee; determining that the location of the hip joint center has moved a distance that is greater than a threshold; and indicating suspicious activity when the distance is greater than the threshold.

[0016] Various embodiments are described that further include monitoring a location of a second navigation array, analyzing the similarity of a movement between the second navigation array and the hip joint center, and determining that the suspicious activity is camera movement when the movement between the second navigation array and the hip joint center is similar.

[0017] Various embodiments are described in which analyzing the similarity of the movement between the second navigation array and the hip joint center includes monitoring the frame-to-frame movement of the hip joint center and the second navigation array, and comparing the frame-to-frame movement of the hip joint center with the frame-to-frame movement of the second navigation array.

[0018] Various embodiments are described that further include calculating leg movement information and analyzing a hip center metric based on the leg movement information to determine whether the leg movement information indicates that the suspect activity is array movement.

[0019] Various embodiments are described that further include analyzing a hip center metric based on the leg movement information to determine whether the leg movement information indicates that the suspect activity is leg movement.

[0020] Various embodiments are described that further include monitoring a location of a second navigation array, using a machine learning model to analyze the similarity of a movement between the second navigation array and the hip joint center, and determining that the suspicious activity is camera movement when the movement between the second navigation array and the hip joint center is similar.

[0021] Various embodiments are described that further include calculating leg movement information and analyzing landmark metrics based on the leg movement information using a machine learning model to determine whether the leg movement information indicates that the suspicious activity is array movement or leg movement. [Brief explanation of the drawings]

[0022] For a better understanding of various exemplary embodiments, reference is made to the accompanying drawings in the following list. [Figure 1] 1 shows an overall view of a computer-assisted surgery system. [Figure 2] 1 shows a patient's leg undergoing knee surgery using a navigation array. [Figure 3] Shows the perceived movement of the hip joint center when the femoral navigation array is rotated 5°. [Figure 4A] 1 shows the movement of several anatomical landmarks during surgical steps without any significant movement events. [Figure 4B] 1 shows the movement of several anatomical landmarks during surgical steps without any significant movement events. [Figure 4C] 1 shows the movement of several anatomical landmarks during surgical steps without any significant movement events. [Figure 5A] 10 shows plots of landmark movement during surgical steps involving large leg movements. [Figure 5B] 10 shows plots of landmark movement during surgical steps involving large leg movements. [Figure 5C] 10 shows plots of landmark movement during surgical steps involving large leg movements. [Figure 6A] 10 depicts a plot of landmark movement during a surgical step with both camera and leg movement events. [Figure 6B] 10 depicts a plot of landmark movement during a surgical step with both camera and leg movement events. [Figure 6C] 10 depicts a plot of landmark movement during a surgical step with both camera and leg movement events. [Figure 7A] 10A-B represent plots of landmark movement during a surgical step in which the navigation array moves relative to the bone. [Figure 7B] 10A-B represent plots of landmark movement during a surgical step in which the navigation array moves relative to the bone. [Figure 7C] 10A-B represent plots of landmark movement during a surgical step in which the navigation array moves relative to the bone. [Figure 8] 1 shows a flow diagram of a high level method for detecting movement of a navigation array relative to a bone. [Figure 9] 10 illustrates another method for detecting the movement of the navigation array relative to the bone. [Figure 10] 1 illustrates an exemplary hardware diagram for implementing a motion detector for a navigation array.

[0023] For ease of understanding, the same reference numbers are used to indicate components having substantially the same or similar structure and / or substantially the same or similar function. DETAILED DESCRIPTION OF THE INVENTION

[0024] The present description and drawings exemplify the principles of the present invention. Thus, it should be understood that those skilled in the art can devise various configurations that, although not explicitly described or shown herein, embody the principles of the present invention and are encompassed within the scope of the present invention. Furthermore, all examples shown herein are expressly intended primarily for educational purposes to aid the reader in understanding the principles of the present invention and the concepts contributed by the inventor(s) to advance the present technology, and should not be construed as being limited to such specifically shown examples and conditions. Furthermore, the term "or," as used herein, refers to a non-exclusive logical or (i.e., and / or) unless otherwise indicated (e.g., "or otherwise" or "or alternatively"). Furthermore, the various embodiments described herein are not necessarily mutually exclusive, as some embodiments can be combined with one or more other embodiments to form new embodiments.

[0025] FIG. 1 shows an overall view of a computer-assisted surgery system 100. A surgical robot base 102 supports a robotic arm 104. The base 102 is depicted as a mobile base, although fixed bases are also contemplated. The robotic arm 104 includes multiple arm segments connected by rotatable or otherwise articulating joints, which can be moved or locked in position by actuation of the joints. The robotic arm 104 can move in all six degrees of freedom during a surgical procedure. The robotic arm 104 may be configured to change increments (e.g., in each of the six degrees of freedom) to ensure the precision needed during surgery. The robotic arm 104 may actively move about the joints to position the arm in a desired location relative to the patient (not depicted), or the robotic arm may be set and locked in a position. For example, a tool can be used by a user with some degree of robotic assistance, e.g., by placing a guide tube in a precise location and orientation, and the user then uses the guide tube to guide the tool. In other situations, a cutting tool may be attached to the robotic arm 104, which uses the attached cutting tool to perform a surgical step, such as cutting or drilling.

[0026] A control unit or controller 106 controls the robotic arm 104 and associated navigation system. The controller 106 typically includes a power supply, an AC / DC converter, motion controllers for powering the motors of the actuation units in each joint, fuses, real-time interface circuitry, and other components conventionally included in a robotic surgical system. An external device 108 may communicate with the controller 106. The external device 108 may be a display, computing device, remote server, etc. configured to allow a surgeon or other user to directly input data into the controller 106. Such data may include patient information and / or surgical procedure information. The external device 108 may display information from the controller 106, such as alerts. Communication between the external device 108 and the controller 106 may be wireless or wired.

[0027] System 100 may also include a navigation system that includes tracking unit 110 so that the relative attitudes or three-dimensional positions and orientations of fiducials attached to multiple navigation system navigation arrays (e.g., navigation array 112, navigation array 114, and optional navigation array 116 (and / or other navigation arrays)) can be tracked in real time and shared with controller 106 for planning or control. Tracking unit 110 may measure relative motion between any and all components coupled to the navigation arrays in a known manner. Tracking can be performed in a number of ways, for example, using stereoscopic optical detectors 118, ultrasonic detectors, radio frequency (RF) location detectors, sensors configured to receive position information from an inertial measurement unit, etc. Real-time tracking means high frequencies, in some embodiments greater than 20 Hz, and in some embodiments, low latency, in the range of 100 to 500 Hz, in some embodiments, less than 5 milliseconds. Regardless of how collected, the position and orientation data may be transferred between components (e.g., to the controller 106) via any suitable connection, e.g., wired or wirelessly using a low latency transfer protocol. The real-time controller 106 may perform real-time control algorithms at moderately high frequencies with additional low latency to coordinate the movement of the system 100. The tracking unit may also include a camera or use a stereoscopic optical detector 116, for example, to detect characteristics of an end effector attached to the robotic arm 104.

[0028] Fiducials (not depicted) of the navigation system may be attached to the navigation array (e.g., navigation array 112, navigation array 114, and / or any navigation array 116 (and / or other navigation arrays)), for example, via multiple mounting points 112a (e.g., on navigation array 112). The fiducials may be positioned at predetermined positions and orientations relative to one another. The fiducials may be aligned to lie in a plane of known orientation (e.g., a vertical plane) to enable establishment of a Cartesian reference frame. The fiducials may be positioned within the field of view of the navigation system and identified in images captured by the navigation system. The fiducials may be single-use reflective navigation markers. Exemplary fiducials include infrared reflectors, light-emitting diodes (LEDs), radio frequency (RF) emitters, spherical reflective markers, flashing LEDs, augmented reality markers, etc. The navigation array may be or include an inertial measurement unit (IMU), an accelerometer, a gyroscope, a magnetometer, other sensors, or a combination thereof. The sensors may transmit position and / or orientation information to a navigation system, for example, a processing unit of the navigation system, for example, the controller 106 .

[0029] A navigation array 114 may be mounted on the robotic arm 104 to determine the position of the robotic arm or a distal portion thereof (either indicative of the end effector position or referenced by difference from the position of the navigation array 112). The structure and operation of the navigation array 114 may vary depending on the type of navigation system used. In some embodiments, the navigation array 114 may include one or more spherical or other fiducials for use with an optical navigation system, e.g., a robotic navigation system. The navigation system may facilitate registering and tracking the position and / or orientation of the navigation array 114 and, therefore, the end effector 120, and its relative distance to other objects in the operating room, e.g., the patient, the surgeon, etc.

[0030] The end effector 120 may be coupled to the robotic arm 104 via an end plate that is locked, for example, by a lever. As can be appreciated, there should be no play between the end effector 120 and the robotic arm 104. While the system 100 may utilize end effectors of various shapes, sizes, and functionality, the depicted end effectors have an opening for removably holding a tool 122. In some embodiments, the navigation array 112 is attached to the end effector. Advantageously, the end effector 120 may be adapted to hold a series of tools, including the tool 122 (e.g., a series of tools used in a particular surgical procedure).

[0031] The tool 122 may be placed within a guide (e.g., or another opening) in the end effector 120. A locking mechanism on the end effector 120 may secure the tool 122 in place. The locking mechanism may be a slider lock mechanism or other feature. While the depicted tool 122 is a dilator, it is understood that the tool may have a probe, a dilator tip (e.g., sharp or blunt), a cutting instrument, a tap, a screw, etc. at its distal end. The cutting instrument may be, for example, a drill, a saw blade, a burr, a reamer, a mill, a scalpel blade, or any other instrument capable of cutting bone or other tissue and appropriate for use in a given operation. In some embodiments, the navigation array 112 may be attached to a robot.

[0032] A navigation array may also be attached to the patient at a known location. Typically, these navigation arrays are attached to the patient's bones. Once the navigation array is attached to the patient's bones, an x-ray or other image may be used to register the location of the array to the patient's bones. The navigation array may be viewed and tracked by the tracking unit 110, so that the tracking unit 110 can track the location of the navigation array attached to the patient and then determine the relative location of the tool 122 with respect to the patient's anatomy by tracking the registration of the navigation array with respect to the patient's bones.

[0033] Computer-assisted surgery can be much more accurate than traditional surgery. See, for example, GW Doan et al., "Image-Free Robotic-Assisted Total Knee Arthroplasty Improves Implant Alignment Accuracy: A Cadaveric Study," The Journal of Athroplasty, Vol. 37, No. 4, pp. 795-801. This accuracy depends on the navigation array being firmly attached to the patient's bone without movement. A source of accuracy error occurs when the bone-mounted navigation array moves relative to the bone during surgery, shifting the frame of reference used by the tracking unit 110 to determine the patient's location relative to the tool. Currently, this potential source of error can be addressed by providing reproducible landmarks on the bone that can be verified at any time during surgery. This specific verification can be included in the surgical workflow, or the surgeon can manually perform this verification and add an additional step to the procedure. However, this approach can only guarantee that the array did not move at a specific point in the surgery or when the user suspects something. In reality, it is more likely that the array moved due to an event unknown to the user because the user was concentrating on something else. This may be the case, for example, during cutting. This potential source of error can also be addressed by attaching a single fiducial to the bone (sometimes called a monitoring marker) and having the system continuously monitor the position of the fiducial relative to the monitoring marker. However, this additional monitoring marker adds time to the procedure and requires additional cutting into the bone. Therefore, an embodiment of a system that automatically detects movement of the navigation array relative to the patient's bones during navigated surgery based on analysis of landmark movement and without monitoring markers is now described.

[0034] In other embodiments, computer-assisted surgery may be performed without a robot.

[0035] The following description uses the example of knee surgery, in which navigation arrays are attached to the tibia and femur. This approach can also be applied to other surgeries. As the array moves relative to the bone to which it is attached ("array-bone relative motion"), the cameras in the tracking system capture the change in array position, but the positions of patient landmarks also change as derived from the array position. This relative motion seen by the tracking unit 110 is similar to other motions that may occur in the system, such as movement of the camera or the patient's leg. To distinguish the movement of the array relative to the bone from these other motions, the motion data can be filtered by amplitude, duration, translation, etc. to distinguish the array-bone relative motion from other types of motion in the system. For example, camera motion is an event within a defined time frame that similarly affects the localization information provided by the camera for all visible arrays. Leg motions can have very different durations or amplitudes, such as large leg movements to reposition the leg before cutting, interactions between a surgical assistant and a retractor, and forces applied to the leg by a saw while performing a cut.

[0036] FIG. 2 illustrates a patient's leg during navigated knee surgery. The leg 200 is shown positioned in an elevated, flexed position. The knee 206 is exposed for surgery. A tibial navigation array 208 is attached to the tibia 202. A femoral navigation array 210 is attached to the femur 204. During a typical knee replacement procedure, different types of anatomical landmarks are acquired intraoperatively. These landmarks can be acquired by the surgeon placing a pointer at specific locations on the patient's anatomy. They can also be calculated by specific rotations of the leg performed by the user. From the localization data acquired during these dedicated surgical steps, the system 100 calculates the hip joint center location 216, the tibial axis 212, and the femoral axis 214. Once these parameters are calculated and the bones are registered to the navigation arrays 208, 210, the system 100 assumes that the navigation arrays 208, 210 are fixed to the bones, so that during the remainder of the surgery, the system 100 can monitor the movement of the navigation arrays 208, 210 and model the movement of the bones in 3D space (and accordingly calculate the locations of the tibia axis 212, femur axis 214, hip center position 216, and any additional acquired landmarks for each bone in 3D space). However, if the spacing between the navigation arrays and the bones changes, for example, due to bumping, the system 100 may calculate bone locations that do not correspond to the actual locations of the bones.

[0037] The hip center location can be used to determine whether the femoral navigation array 204 has moved relative to the femur 204. In an alternative embodiment, a leg holder can maintain the ankle in a stable position, and the ankle position can similarly be used to determine whether the tibial navigation array 208 has moved relative to the tibia 202. FIG. 3 illustrates the perceived movement of the hip center when the femoral navigation array 210 is rotated 5°. Using the hip center location 302 as a determining factor for characterizing the movement detected by the tracking unit 110 has several advantages over other anatomical landmarks. For example, because the hip center is attached to the pelvis, the hip center is unlikely to move significantly during surgery (unless the patient's abdomen is moved). Furthermore, the hip center is located far from the femoral navigation array 210, meaning that small movements of the array result in large movements of the hip center location 216. Therefore, to take advantage of this amplified movement, the navigation array is placed near one end of the bone, and a landmark is chosen at the other end of the bone. Various hip joint center motion metrics using different time spans or granularity may be used. Per-frame motion may be calculated by determining the distance between two consecutive measurements of hip joint center motion. This motion may be sampled, for example, at approximately 30 Hz, although other higher or lower rates may be used. Filtered motion may be calculated by comparing the initial and final positions of the hip joint center over a time window (e.g., 1 second, although other window sizes may be used as well). Absolute motion may be calculated by comparing the current position to a reference position defined at the beginning of the step. The reference position may be defined as the average or median of consecutive positions. Other metrics may be defined that allow the motion of the navigation array relative to the bone to be distinguished from other types of motion of the navigation array. Additionally, other landmarks may be used for other procedures based on the specific anatomical structures used and the steps performed during the procedure. Examples of data collected for various types of motion are presented here.These examples provide insight into how to determine whether a metric indicates movement of the navigation array relative to the bone or other types of array movement events.

[0038] Figures 4A-4C show the motion of several anatomical landmarks during surgical steps without significant motion events. The plots represent the frame-by-frame motion of three specific landmarks in knee surgery: the hip center, the knee center, and the ankle center. These positions are calculated from the array positions provided by the localization system. The frames represent the localization data transmitted at high frequency by the localization system.

[0039] Figure 4A shows three plots of sampled frame-by-frame motion for hip center motion (top plot), knee motion (middle plot), and ankle motion (bottom plot) during a typical surgical step without significant array motion events. The vertical axis represents the sample-to-sample position change in mm for the hip center, knee, and ankle locations, respectively. The horizontal axis represents measurement time. In this example, the hip center, knee, and ankle locations are each sampled at a rate of 30 Hz. The difference between samples is calculated and then plotted. Figure 4B shows a plot of filtered hip center motion. This is calculated by determining the reference position of the hip center at the beginning and end of a time window and calculating the difference between those positions. The value of this metric is to highlight time windows with significant landmark motion while filtering out landmark motion events that are short and may be considered insignificant. The vertical axis represents the difference, and the horizontal axis represents the center of the observed time window. Figure 4C shows a plot of absolute hip center motion compared to a reference position defined at the beginning of the step, when the array was first detected by the localization system. Even in normal steps without significant movement events, the plot shows some low amplitude movements of different landmarks that may be due to minute movements of the legs or noise.

[0040] Figures 5A-5C show plots of landmark motion during surgical steps involving large leg movements. The hip landmark plots show that the hip joint center is a substantially fixed point that should not move beyond a threshold under normal surgical conditions. Figure 5A shows three plots of sampled frame-by-frame motion for hip joint center motion (top plot) and knee movement (middle plot) as the knee moves. Figure 5B shows a plot of filtered hip joint center motion. Figure 5C shows a plot of absolute hip joint center motion.

[0041] 5A-5C show that the hip joint center is a substantially fixed point that can be used as a landmark to detect when the navigation array moves relative to the bone. The use of a fixed landmark helps the navigation array detection algorithm determine that the navigation array is not moving relative to the bone.

[0042] Figures 6A-6C show plots of landmark motion during a surgical step with both camera and leg motion events. A comparison of the frame-by-frame hip joint center motion with other leg landmarks in Figure 6A shows that the hip joint center motion is significantly smaller. Furthermore, Figures 6A-6C show that the amplitude of the filtered and absolute hip joint center motion is limited when the knee moves, meaning that several thresholds can be defined with different metrics to determine what can be explained by normal leg motion and what is suspicious motion. As a result, the hip joint center is a suitable landmark to measure and analyze to determine when the navigation array moves relative to the bone.

[0043] 6A-6C plot the same metrics as FIGS. 4A-4C for the same surgical step, but with two categories of motion events: one camera motion event and one leg motion event.

[0044] Figure 6A shows frame-by-frame metrics for the hip, knee, and ankle center landmarks. Two clusters of motion events can be distinguished for the three landmarks. The first cluster shows nearly identical and simultaneous motion impacts for all observed landmarks, consistent with camera motion event characteristics. The second cluster shows that the ankle and knee centers are similarly affected, albeit with slight differences, while the hip center has very little impact, consistent with leg motion characteristics. Furthermore, Figures 6B and 6C show that cluster 1 has a significant impact on the filtered absolute motion metrics for the hip center, while cluster 2 does not. This confirms that the motion described by cluster 1 is camera motion, while the motion described by cluster 2 is leg motion.

[0045] Figures 7A-7C plot the same metrics as Figures 4A-4C, but for when the femur array moves relative to the bone. Figure 7A shows frame-by-frame motion for hip center motion (top plot), knee motion (middle plot), and ankle motion (bottom plot). In this case, the hip center landmark motion has the largest amplitude and is not synchronized with the motion of the other landmarks. Figure 7B shows a plot of filtered hip center motion. In this case, the filtered motion exceeds a defined threshold when an array motion event occurs. Figure 7C shows a plot of absolute hip center motion. In this case, the filtered motion exceeds a defined threshold when an array motion event occurs.

[0046] The frame-by-frame plot of hip center in Figure 7A shows a large peak at approximately 2:00 and a smaller peak at approximately 2:10. This results in the two simultaneous spikes in Figure 7B. Next, Figure 7C shows two simultaneous jumps in absolute hip center motion, resulting in a plateau where the hip center never returns to its initial position. These plots show a similar but reduced impact on knee location and no impact on ankle location. The motion characteristics can be compared to those of other types of motion events shown in Figure 6. Array motion has a similar impact on the filtered absolute metrics of hip center as camera motion, but array motion does not affect other landmarks in the same way as camera motion. Array motion also has a more significant impact on hip center metrics than knee center metrics, while the opposite is observed for leg motion. Consequently, such plots can be used to develop thresholds for hip center motion; hip center motion above these thresholds for one or more metrics indicates that the detected motion cannot result from normal events during surgery and is likely due to the navigation array moving relative to the bone.

[0047] Based on the above example, other metrics may be examined to determine whether they are useful for distinguishing between motion of the navigation array relative to the bone. An example of another metric is the angle of the femur relative to the tibia in the case of knee surgery. Metrics related to this angle can also be collected and used to determine motion of the navigation array relative to the bone. Once a set of metrics is defined, a significant number of known array motions and normal cases can be collected. This data can then be used to train a navigation array motion detector, which determines thresholds to use for the various metrics to achieve a specified detection / false positive rate. The navigation array motion detector can be implemented using various statistical detection methods or machine learning models. For statistical detection methods, a search can be performed on a set of training data including metric data for normal motion cases to find a set of thresholds for the various metrics that achieves a specified detection / false positive rate. The training data can also include data collected for situations in which the navigation array has moved. For machine learning models, a model architecture can be selected and then trained using the training data until a specified detection / false positive rate is obtained or until an optimization metric is minimized. Any type of machine learning model that achieves the desired performance can be used. Furthermore, several different types of machine learning model architectures can be selected and trained. As a result, the model with the best performance can be used as the motion detector for the navigation array.

[0048] In the above example, absolute difference data was used. Note that each patient has a different sized anatomy. A person with a longer femur will experience more perceived movement of the hip joint center as the navigation array moves, while a person with a shorter femur will experience less movement. As a result, the data can be normalized by femur length or another appropriate bone length. Using normalized data is then less susceptible to variations in the size of a patient's anatomy.

[0049] As mentioned above, navigation array motion detectors can be applied to bones and landmarks other than the hip center during knee surgery, including, for example, elbow surgery, ankle surgery, shoulder surgery, etc. The reference landmarks used for navigation array motion detection should be positioned on the bone as far away as possible from the navigation array, i.e., the reference landmark is near one end of the bone and the navigation array is near the other end of the bone. Furthermore, the landmarks used should be stable, with no or minimal expected movement during normal surgical steps.

[0050] For example, the end of the tibia 202 at the ankle may be used to detect movement of the tibial navigation array 208 relative to the tibia 202 during knee surgery. Because significant ankle movement is common during knee surgery, if a leg holder is used to immobilize the ankle, a navigation array motion detector may be used to detect movement of the tibial navigation array 208 relative to the tibia 202.

[0051] FIG. 8 shows a flow diagram of a high-level method for detecting navigation array motion relative to a bone. The navigation array motion detection method 800 begins by monitoring and collecting real-time array data from a localization device 804 and landmark motion metrics 802, which may include landmark data 806. Various motion data may be measured by the tracking unit 110. This data may then be used to calculate metrics used to detect suspected navigation array motion that may be due to navigation array motion relative to a bone. For example, the data described in FIGS. 4-7 may be calculated as described above, as well as other metrics that provide insight into navigation array motion. The navigation array motion detection method 800 then monitors hip joint center motion (808). The hip joint center motion may be compared to a threshold, as described above. This process occurs continuously. When the hip joint center motion exceeds the threshold, the detection method 800 performs deep data analysis for a time frame 810. This analysis may be based on insights gained based on the plots of FIGS. 4-7. If the deep data analysis determines there is array motion (812), an alert is issued (814). If the deep data analysis determines that the threshold has been exceeded due to another event 816, the detection method 800 does not issue an alert and returns to monitoring 818.

[0052] FIG. 9 shows a flow diagram of another method for detecting navigation array motion relative to a bone. The navigation array motion detection method 900 begins by monitoring and collecting landmark motion metrics 902, which may include real-time array data from a localization device 904 and landmark data 906. Various motion data may be measured by the tracking unit 110. This data may then be used to calculate metrics used to detect suspected navigation array motion that may be due to navigation array motion relative to a bone. For example, the data described in FIGS. 4-7 may be calculated as described above, as well as other metrics that provide insight into navigation array motion. The navigation array motion detection method 900 then monitors hip joint center motion (908). The hip joint center motion may be compared to a threshold, as described above. This process occurs continuously. When the hip joint center motion exceeds the threshold, the detection method 900 determines (920) whether there is sufficient localization data to proceed. If not, the detection method 900 cannot reach a conclusion 922, does not issue an alert, and returns to monitoring 918. If there is enough localization data to proceed, the detection method 900 analyzes the motion for similarity (924). The definition of similarity in this context depends on the underlying camera motion model, which may include different types of transformations, including translation and rotation. Landmark motions are considered similar if they can result from the same transformation among the possibilities described in the model. This may be achieved using various methods, such as thresholding, correlation, etc. If the various motions of the visible landmarks or array conform to the similarity criteria, this indicates camera motion (926), and again, no alert is issued and monitoring is resumed (918). If the motions are not similar, leg motion information may be calculated (928). The detection method 900 may then perform a deep analysis of hip joint center metrics (930) using the leg motion input. If the deep analysis of the hip joint center metrics determines that the hip joint motion is due to leg motion (932), no alert is issued and monitoring is resumed (918). If the deep analysis of the leg hip joint center metrics determines that there is array motion (934), an alert indicating array motion is issued (914).

[0053] The purpose of steps 928 and 930 is to refine the analysis according to the amplitude of leg motion that may have occurred during the analyzed time frame. Step 928 determines metrics useful for characterizing leg motion. This may include, for example, knee center motion or the difference in leg angle (the angle between the tibia axis and the femur axis) within the time frame. Step 930 utilizes the leg motion metric from step 928 along with the hip center metric to determine whether the hip center metric can be explained by leg motion alone or whether array motion has occurred. This may be accomplished, for example, by adaptive thresholding. If the absolute landmark motion metric is less than a threshold associated with the current leg motion, leg motion is indicated 932. Conversely, the navigation array motion detection method 900 issues array motion warnings 934 and 914.

[0054] In an alternative embodiment, steps 920, 924, 928, and 930 may be replaced by a machine learning model. The machine learning model may take motion metrics, such as hip-centered motion, knee-centered motion, and absolute hip-centered motion, as input. Other metrics that indicate the ability to distinguish between various motion scenarios that may occur during surgery may also be used. The output of the machine learning model is an indication of whether the navigation array has moved relative to the bone. This machine learning model is generated by selecting a machine learning model architecture and then using training data to train a model that covers a wide range of motions that may occur during surgery. The model may be trained using standard training and optimization methods. Also, various machine learning model architectures may be selected and trained similarly. The best-performing model may then be used to detect when the navigation array has moved relative to the bone.

[0055] FIG. 10 shows an exemplary hardware diagram 1000 for implementing a navigation array motion detector. The exemplary hardware 1000 may correspond to the tracking unit 110, the external device 108, a connected processor or computer system, or any other processor available in the computer-assisted surgery system 100. As shown, the device 1000 includes a processor 1020, memory 1030, a user interface 1040, a network interface 1050, and storage 1060 interconnected via one or more system buses 1010. It will be understood that FIG. 10 constitutes, in some respects, an abstraction, and the actual organization of the components of the device 1000 may be more complex than depicted.

[0056] Processor 1020 may be any hardware device capable of executing instructions or otherwise processing data stored in memory 1030 or storage 1060. Accordingly, a processor may include a microprocessor, microcontroller, graphics processing unit (GPU), neural network processor, field programmable gate array (FPGA), application specific integrated circuit (ASIC), or other similar device.

[0057] Memory 1030 may include various memories, such as, for example, L1, L2, or L3 cache or system memory, etc. Thus, memory 1030 may include static random access memory (SRAM), dynamic RAM (DRAM), flash memory, read-only memory (ROM), or other similar memory devices.

[0058] The user interface 1040 may include one or more devices for enabling communication with a user. The user interface 1040 may be part of the external device 108. For example, the user interface 1040 may include a display, a touch interface, a mouse, and / or a keyboard for receiving user commands. In some embodiments, the user interface 1040 may include a command line interface or a graphical user interface that may be presented to a remote terminal via the network interface 1050.

[0059] Network interface 1050 may include one or more devices for enabling communication with other hardware devices. For example, network interface 1050 may include a network interface card (NIC) configured to communicate according to an Ethernet protocol or other communication protocol, including a wireless protocol. Additionally, network interface 1050 may implement a TCP / IP stack for communicating according to the TCP / IP protocol. Various alternative or additional hardware or configurations for network interface 1050 will become apparent.

[0060] Storage 1060 may include one or more machine-readable storage media, such as read-only memory (ROM), random-access memory (RAM), magnetic disk storage media, optical storage media, flash memory devices, or similar storage media. In various embodiments, storage 1060 may store instructions for execution by processor 1020 or data on which processor 1020 may operate. For example, storage 1060 may store a base operating system 1061 for controlling various basic operations of hardware 1000. Storage 1062 may store instructions for implementing the navigation array detectors described herein.

[0061] It will be apparent that various information described as being stored in storage 1060 may additionally or alternatively be stored in memory 1030. In this regard, memory 1030 may be considered to constitute a "storage device," and storage 1060 may be considered a "memory." Various other configurations will be apparent. Furthermore, memory 1030 and storage 1060 may both be considered to be "non-transitory machine-readable media." As used herein, the term "non-transitory" will be understood to exclude transitory signals, but to include all forms of storage, including both volatile and non-volatile memory.

[0062] The system bus 1010 enables communication between the processor 1020 , memory 1030 , user interface 1040 , storage 1060 , and network interface 1050 .

[0063] Although host device 1000 is shown as including one of each of the described components, various components may be duplicated in various embodiments. For example, processor 1020 may include multiple microprocessors configured to independently perform the methods described herein, or configured to perform steps or subroutines of the methods described herein, such that the multiple processors cooperate to achieve the functionality described herein. Furthermore, when device 1000 is implemented in a cloud computing system, the various hardware components may reside in separate physical systems. For example, processor 1020 may include a first processor in a first server and a second processor in a second server.

[0064] Although each of the embodiments is described above in terms of their structural arrangement, it should be understood that the present invention also encompasses associated methods of using the above-described embodiments.

[0065] While various exemplary embodiments have been described in detail with particular reference to certain illustrative aspects thereof, it should be understood that the invention is capable of other embodiments and its details are capable of modifications in various obvious respects. As will be readily apparent to those skilled in the art, variations and modifications of the various embodiments and combinations thereof may be effected while remaining within the spirit and scope of the invention. Accordingly, the foregoing disclosure, description, and drawings are for illustrative purposes only and do not in any way limit the invention, which is defined solely by the claims.

[0066] [Embodiment] (1) A method for detecting movement of a first navigation array relative to a bone during computer-assisted surgery, comprising: monitoring a location of a landmark on the bone using the first navigation array, the landmark being at a first end of the bone and the first navigation array being adjacent to a second end of the bone; determining that the location of the landmark has moved a distance that is greater than a threshold; indicating suspicious activity when the distance is above the threshold. (2) monitoring the location of a second navigation array; and analyzing a similarity of motion between the second navigation array and the landmark; and 2. The method of claim 1, further comprising: determining that the suspicious activity is camera movement when the movement of the second navigation array and the landmark is similar. (3) analyzing the similarity of the motion between the second navigation array and the landmarks includes: monitoring frame-to-frame movement of the landmark and the second navigation array; comparing the frame-to-frame movement of the landmark with the frame-to-frame movement of the second navigation array. (4) Calculating bone movement information; 3. The method of claim 2, further comprising: analyzing landmark metrics based on the bone movement information to determine whether the bone movement information indicates that the suspicious activity is array movement. (5) The method of embodiment 3, further comprising analyzing landmark metrics based on bone movement information to determine whether the bone movement information indicates that the suspicious activity is bone movement.

[0067] (6) The method described in embodiment 1, wherein the computer-assisted surgery is knee surgery and the landmark is the hip joint center. (7) The method of embodiment 1, wherein the landmark is located at a location that moves less than a threshold during the computer-assisted surgery. (8) monitoring the location of a second navigation array; and 2. The method of claim 1, further comprising: using a machine learning model to analyze the similarity of the movement of the second navigation array and the landmark; and determining that the suspicious activity is camera movement when the movement of the second navigation array and the landmark is similar. (9) Calculating bone movement information; 9. The method of claim 8, further comprising: using a machine learning model to analyze landmark metrics based on the bone movement information and determine whether the bone movement information indicates that the suspicious activity is array movement or bone movement. (10) A method for detecting movement of a first navigation array relative to a femur during computer-assisted knee surgery, comprising: monitoring a location of a hip joint center of the femur using the first navigation array, the first navigation array being adjacent to the knee; and determining that the location of the hip joint center has moved a distance that is greater than a threshold; indicating suspicious activity when the distance is above the threshold.

[0068] (11) monitoring the location of a second navigation array; and Analyzing a similarity of motion between the second navigation array and the hip joint center; 11. The method of claim 10, further comprising: determining that the suspicious activity is camera movement when the movement of the second navigation array and the hip joint center is similar. (12) analyzing the similarity of the motion between the second navigation array and the hip joint center includes: monitoring frame-to-frame movement of the hip joint center and the second navigation array; comparing the frame-to-frame movement of the hip joint center with the frame-to-frame movement of the second navigation array. (13) calculating leg movement information; 12. The method of claim 11, further comprising: analyzing a hip joint center metric based on the leg movement information; and determining whether the leg movement information indicates that the suspicious activity is array movement. (14) The method of embodiment 13, further comprising analyzing a hip center metric based on the leg movement information to determine whether the leg movement information indicates that the suspicious activity is leg movement. (15) monitoring the location of a second navigation array; and analyzing a similarity of motion between the second navigation array and the hip joint center using a machine learning model; 11. The method of claim 10, further comprising: determining that the suspicious activity is camera movement when the movement of the second navigation array and the hip joint center is similar.

[0069] (16) calculating leg movement information; 16. The method of claim 15, further comprising: using a machine learning model to analyze landmark metrics based on the leg movement information and determine whether the leg movement information indicates that the suspicious activity is array movement or leg movement.

Claims

1. 1. A method for detecting movement of a first navigation array relative to a bone during computer-assisted surgery, comprising: monitoring a location of a landmark on the bone using the first navigation array, the landmark being at a first end of the bone and the first navigation array being adjacent to a second end of the bone; determining that the location of the landmark has moved a distance that is greater than a threshold; indicating suspicious activity when the distance is above the threshold.

2. monitoring the location of the second navigation array; analyzing the similarity of the movement of the second navigation array and the landmark; The method of claim 1 , further comprising: determining that the suspicious activity is camera movement when the movement of the second navigation array and the landmark is similar.

3. Analyzing the similarity of the motion between the second navigation array and the landmarks includes: monitoring frame-to-frame movement of the landmark and the second navigation array; and comparing the frame-to-frame motion of the landmark with the frame-to-frame motion of the second navigation array.

4. Calculating bone movement information; 3. The method of claim 2, further comprising: analyzing landmark metrics based on the bone motion information to determine whether the bone motion information indicates that the suspicious activity is array motion.

5. The method of claim 3 , further comprising analyzing landmark metrics based on bone movement information to determine whether the bone movement information indicates that the suspicious activity is bone movement.

6. The method of claim 1 , wherein the computer-assisted surgery is knee surgery and the landmark is a hip joint center.

7. The method of claim 1 , wherein the landmark is at a location that moves less than a threshold during the computer-assisted surgery.

8. monitoring the location of the second navigation array; 2. The method of claim 1, further comprising: using a machine learning model to analyze similarity of motion between the second navigation array and the landmark; and determining that the suspicious activity is camera motion when the motion between the second navigation array and the landmark is similar.

9. Calculating bone movement information; 10. The method of claim 8, further comprising: using a machine learning model to analyze landmark metrics based on the bone motion information to determine whether the bone motion information indicates that the suspicious activity is array motion or bone motion.

10. 1. A method for detecting movement of a first navigation array relative to a femur during computer-assisted knee surgery, comprising: monitoring a location of a hip joint center of the femur using the first navigation array, the first navigation array being adjacent to the knee; and determining that the location of the hip joint center has moved a distance that is greater than a threshold; indicating suspicious activity when the distance is above the threshold.

11. monitoring the location of the second navigation array; Analyzing a similarity of motion between the second navigation array and the hip joint center; The method of claim 10 , further comprising: determining that the suspicious activity is camera movement when the movement of the second navigation array and the hip center is similar.

12. Analyzing the similarity of the motion between the second navigation array and the hip joint center includes: monitoring frame-to-frame movement of the hip joint center and the second navigation array; and comparing the frame-to-frame movement of the hip joint center with the frame-to-frame movement of the second navigation array.

13. Calculating leg movement information; 12. The method of claim 11, further comprising: analyzing a hip center metric based on the leg movement information to determine whether the leg movement information indicates that the suspicious activity is array movement.

14. 14. The method of claim 13, further comprising: analyzing a hip center metric based on the leg movement information to determine whether the leg movement information indicates that the suspicious activity is a leg movement.

15. monitoring the location of the second navigation array; analyzing a similarity of motion between the second navigation array and the hip joint center using a machine learning model; The method of claim 10 , further comprising: determining that the suspicious activity is camera movement when the movement of the second navigation array and the hip center is similar.

16. Calculating leg movement information; 16. The method of claim 15, further comprising: using a machine learning model to analyze landmark metrics based on the leg movement information to determine whether the leg movement information indicates that the suspicious activity is array movement or leg movement.

Images