Pinless software structure

Abstract

A method can include identifying, using a surgical navigation system, a reference location of a landmark on the bone and determining a location of an ultrasonic sensor relative to the bone by comparing a contour from an ultrasonic image captured by the ultrasonic sensor and a computerized tomography (CT) scan of the bone. The method can also include, aligning, using the reference location and the location of the ultrasonic sensor, the CT scan to a reference coordinate system maintained by the navigation system. The method can also include transforming the location into coordinates within the reference coordinate system to track the orientation of the bone throughout a surgical procedure.

Description

PINLESS SOFTWARE STRUCTURE CLAIM OF PRIORITY

[0001] This application claims the benefit of U.S. Provisional Patent Application Serial No. 63 / 656,446, filed on June 5, 2024, the benefit of priority of which is claimed hereby, and which is incorporated by reference herein in its entirety. TECHNICAL FIELD

[0002] Examples described herein generally relate to software for medical procedures and more specifically to pinless software structure for use in medical procedures. BACKGROUND

[0003] In the field of robotic surgery, precise tracking and navigation of surgical instruments and patient anatomy are critical for successful outcomes. Traditional systems often rely on the use of physical pins or markers inserted into bones to establish reference points for navigation. These pins can be invasive, causing additional trauma to the patient, and may lead to complications such as infection or damage to the bone.

[0004] The use of pins can also limit the mobility of the patient's limb during surgery, which may restrict the surgeon's ability to perform certain procedures. Pins can become loose or shift during the operation, which can compromise the accuracy of the navigation system and potentially lead to suboptimal surgical results.

[0005] Therefore, there is a need for an improved navigation system that can accurately track the orientation of bones without the use of invasive pins. Such a system would benefit from being less invasive, reducing potential complications, and providing greater flexibility in surgical procedures.SUMMARY

[0006] In examples, a method for pinless navigation to track an orientation of a first bone and a second bone of a patient during a robotic surgical procedure, the first bone and the second bone forming a joint, the method including: accessing a computerized tomography scan including each of the first bone and the second bone; obtaining, using a surgical navigation system, a reference location of a landmark on the first bone or the second bone; generating, using a first ultrasonic sensor, a first contour of the first bone at a known location along the first bone; generating, using a second ultrasonic sensor, a second contour of the second bone at a known location along the second bone; determining a first location of the first ultrasonic sensor relative to the first bone by comparing the first contour and the computerized tomography scan of the first bone; determining a second location of the second ultrasonic sensor relative to the second bone by comparing the second contour and the computerized tomography scan of the second bone; aligning, using the reference location, the first location, and the second location, the computerized tomography scan to a reference coordinate system maintained by the surgical navigation system; and transforming the first location and the second location into coordinates within the reference coordinate system to track the orientation of the first bone and the orientation of the second bone throughout a surgical procedure.

[0007] In examples, a method for pinless navigation to track an orientation of a bone by a robotic surgical system, the method including: identifying, using a surgical navigation system, a reference location of a landmark on the bone; determining a location of an ultrasonic sensor relative to the bone by comparing a contour from an ultrasonic image captured by the ultrasonic sensor and a computerized tomography scan of the bone; aligning, using the reference location and the location of the ultrasonic sensor, the computerized tomography scan to a reference coordinate system maintained by the surgical navigation system; and transforming the location into coordinates within the reference coordinate system to track the orientation of the bone throughout a surgical procedure.

[0008] In examples, a robotic surgery system including: a surgical navigation system configured to track a location of one or more objects within anoperating room during a surgical procedure; a surgical robot configured to perform the surgical procedure; and controller circuitry coupled to memory circuitry, including stored instructions, that, when performed by the controller circuitry, cause the controller circuitry to: identify, using the surgical navigation system, a reference location of a landmark on a bone; determine a location of an ultrasonic sensor relative to the bone by comparing a contour from an ultrasonic image captured by the ultrasonic sensor and a computerized tomography scan of the bone; align, using the reference location, the computerized tomography scan to a reference coordinate system maintained by the surgical navigation system; and transform the location into coordinates within the reference coordinate system to track an orientation of the bone throughout a surgical procedure.

[0009] While the systems and methods are discussed herein with respect to a robotic surgical system, the disclosed navigation techniques are also applicable to a navigated surgical procedure using navigated instruments without the need for a surgical robot. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Various examples are illustrated in the figures of the accompanying drawings. Such examples are demonstrative and not intended to be exhaustive or exclusive examples of the present subject matter.

[0011] FIG.1 illustrates a schematic diagram of an example robotic surgery system.

[0012] FIG.2 illustrates a flow diagram of an example of a method for pinless navigation to track an orientation of a first and second bone of a patient during a robotic surgical system.

[0013] FIG.3 illustrates an example of a bone of a patient including landmarking indicators.

[0014] FIG.4 illustrates an example validation of a bone of a patient.

[0015] FIG.5 illustrates an example ultrasound image of a bone of a patient.

[0016] FIG.6 illustrates an example of segmentation of an ultrasound image of a bone of a patient.

[0017] FIG.7 illustrates a schematic diagram of aligning a segmentation of an ultrasound image with an example 3D model of a bone of a patient.

[0018] FIG.8 illustrates a diagram showing an example of point-to-point alignment of an example ultrasound segmentation and 3D model of a bone of a patient.

[0019] FIG.9 illustrates a graphical representation of an example cost function for the alignment of FIG.8.

[0020] FIG.10 illustrates a diagram showing an example point-to-plane alignment of an example ultrasound segmentation and 3D model of a bone of a patient.

[0021] FIG.11 illustrates a matrix representation of a sample cost function for the alignment of FIG. 10.

[0022] FIG.12 illustrates a diagram showing an example plane-to-plane alignment of an example ultrasound segmentation and 3D model of a bone of a patient.

[0023] FIG.13 illustrates a diagram showing an example plane-to-plane alignment of an example ultrasound segmentation and 3D model of a bone of a patient.

[0024] FIG.14 is a block diagram illustrating an example of a machine upon which one or more examples may be implemented. DETAILED DESCRIPTION

[0025] Conventional methods of tracking positions and orientations of bones during robotic surgeries can include inserting physical pins into bones to establish reference points then tracking the locations of the pins, and thus the reference points with a surgical navigation system. Conventional methods of tracking the position and orientation of bones during a robotic surgery can result in surgical complications (e.g., infection, additional trauma to the patient) and can restrict the surgeon's ability to perform certain procedures due to limited mobility of the limbs of the patient. Moreover, the accuracy of these systems can be compromised if the pins become loose or shift during the operation.

[0026] The present disclosure includes a pinless navigation system for use in robotic surgical procedures. This system utilizes non-invasive ultrasonic sensors to accurately track the orientation of bones without the need for physical pins. By comparing contours from ultrasonic images captured by the sensors with preoperative computerized tomography scans of the bones, the system can determine the location of the sensors relative to the bones. This position of the sensors relative to the bones can be used to align the computerized tomography scans of the bone to a reference coordinate system maintained by the navigation system. The alignment between the computerized tomography scans of the bones and the reference coordinate system can allow for real-time tracking of bone orientation throughout the surgical procedure. Therefore, the present disclosure provides a less invasive, more reliable, and more flexible approach to surgical navigation, potentially reducing complications and improving surgical outcomes.

[0027] The above discussion is intended to provide an overview of subject matter of the present patent application. It is not intended to provide an exclusive or exhaustive explanation of the invention. The description below is included to provide further information about the present patent application.

[0028] FIG.1 illustrates a schematic diagram of an example robotic surgery system 100. The robotic surgery system 100 can be used for pinless navigation to track the orientation of a first and second bone of a patient during a robotic surgical procedure. The robotic surgery system 100 can include a pinless system 110, a surgical robot 120, and a controller 130.

[0029] The pinless system 110 can be configured to track the real-time 3D position and orientation of objects within the operating room during a surgical procedure. For example, the pinless system 110 can be configured to track surgical tools, other components of the controller 130, or patient bones using location identifiers to determine the location and orientation of the patient bones in real time during surgery.

[0030] The pinless system 110 can include a tibia sensor assembly 112 and a femur sensor assembly 114. Each belt can contain an array of ultrasonic sensors to capture 2D or 3D ultrasonic images of the underlying bone. Each assembly (e.g., thetibia sensor assembly 112 and the femur sensor assembly 114) can include a belt or other method to secure the sensors related to the bones as well as a navigation array for determining a location of each assembly within the virtual / robotic coordinate system (e.g., the surgical navigation system 122).

[0031] The ultrasonic sensors can communicate with an acquisition box 116 running pinless tracking software 118. The acquisition box 116 can process the ultrasonic signals to determine the 3D position and orientation of the tibia and femur belts relative to the bones. The belts can also include optical tracking markers to communicate with the optical tracking system (e.g., surgical navigation system 122) of the surgical robot 120.

[0032] The surgical robot 120 can be configured to perform surgical procedures, such as orthopedic repair procedures, joint replacements, or other robotically assisted surgeries. The surgical robot 120 can orient tools and manipulate tissue based on the real-time bone position and orientation data from the pinless system 110. As referenced above, the surgical robot 120 can include a surgical navigation system 122. The surgical navigation system 122 can include a computing device (e.g., computer), monitor, and cameras (e.g., including infrared light). The surgical navigation system 122 can track positions and orientations of components of the robotic surgery system 100 throughout the surgical procedure.

[0033] The controller 130 can be coupled to memory circuitry 150. The memory circuitry 150 can include stored instructions 152 that, when performed by the controller 130, cause the controller 130 to perform one or more procedures to track a location or an orientation of one or more bones of a patient without using pins for landmarking.

[0034] The instructions 152 can cause the controller 130 to identify a reference location of anatomical landmark locations on the patient's bones using a preoperative 3D CAD model generated from CT or MRI scans. For example, easily identifiable protrusions, depressions, or surfaces can be selected. Such landmarks will be discussed in more detail with reference to FIG.3.

[0035] The controller 130 can also be configured by the stored instructions 152 to determine a location of an ultrasonic sensor relative to the bone bycomparing a contour from an ultrasonic image captured by the ultrasonic sensor and a computerized tomography (CT) scan of the bone. Here, the controller 130 can receive an ultrasonic signal from either or both of the tibia sensor assembly 112 or the femur sensor assembly 114, or an ultrasonic image from the acquisition box 116 (or any other controller of the robotic surgery system 100), which is derived from the ultrasonic signals from the tibia sensor assembly 112 or the femur sensor assembly 114.

[0036] The controller 130 can also be configured by the stored instructions 152 to align, using the reference location, the CT scan to a reference coordinate system maintained by the navigation system of the surgical robot 120. After the controller 130 determines the location of the ultrasonic sensor relative to the bone, the controller 130 can receive position data of the ultrasonic sensor from the navigation system of the surgical robot 120 to determine how the position of the sensors (e.g., the tibia sensor assembly 112 or the femur sensor assembly 114) have moved during the medical procedure. Aligning the location of the ultrasonic sensor relative to the bone can establish a spatial relationship between the ultrasonic sensors and the bone anatomy of the patient.

[0037] The controller 130 can also be configured by the stored instructions 152 to transform the location into coordinates within the reference coordinate system to track an orientation of the bone throughout a surgical procedure. Transforming the location of the sensors and thus the bones of the patient into coordinates allows the position of the sensors and the bones to be communicated with the navigation system of the surgical robot 120. As the belts (e.g., the tibia sensor assembly 112 or the femur sensor assembly 114) move with movement of the respective bones during the surgical procedure, the controller 130 can receive the updated sensor positions from the optical navigation system of the surgical robot 120 and transform those updated locations into bone orientations within the reference frame, which allows for real-time tracking of bone orientations throughout the medical procedure.

[0038] FIG.2 illustrates a flow diagram of an example of a method 200 for real-time pinless tracking of bone orientation during a robotic surgical procedure. The method 200 can optionally include any of operations 210–280.

[0039] At operation 210, method 200 can optionally include accessing a 3D- model, from a computerized tomography (CT) scan or MRI scan, including each of the first bone and the second bone. The controller (e.g., the acquisition box 116, the controller 130, or any other controller of the robotic surgery system 100 (all shown in FIG.1) can access the CT scan. The controller can download the CT scan from a server, database, or the like, or otherwise obtain the CT scan relating to the bones of the patient. The CT scan can include detailed surface contours and pre-operatively placed landmarks for each of the first bone and the second bone (e.g., tibia and femur). In examples, the controller (e.g., the acquisition box 116, the controller 130, or any other controller of the robotic surgery system 100 (all shown in FIG. 1) can access the CT scan (or other medical imaging of the bone) and generate a 3D model representing the bones of the patient based on the medical imaging.

[0040] At operation 220, method 200 can optionally include obtaining, using a surgical navigation system, a reference location of a landmark on the first bone or the second bone. The surgical navigation system of the surgical robot 120 (FIG.1) can analyze the 3D model of the bone to locate the pre-determined, or pre- operatively marked, landmarks on the bone of the patient. These landmarks can help the robot recognize regions of anatomical importance of the bone. The landmarks can also be used to assist in registering the CT scans the respective bones.

[0041] At operation 230, the method 200 can optionally include generating, using a first ultrasonic sensor, a first contour of the first bone at a known location along the first bone. As discussed herein, the first ultrasonic sensor can be installed within a belt (e.g., the tibia sensor assembly 112) to secure the position of the sensor relative to the bone of the patient. The ultrasonic sensor can capture an ultrasonic profile of the bone at the location of the bone that the belt is installed. In examples, the ultrasonic sensors can be configured to capture more than one (e.g., two or more) contours of the bone of the patient. Each of the contours captured by the ultrasonic sensors can be taken from a different location of the bone of the patient.

[0042] At operation 240, the method 200 can optionally include generating, using a second ultrasonic sensor, a second contour of the second bone at a known location along the second bone. As discussed herein, the second ultrasonic sensor can be installed within a belt (e.g., femur sensor assembly 114) to secure the position of the sensor relative to the bone of the patient. The ultrasonic sensor can capture an ultrasonic profile of the bone at the location of the bone that the belt is installed.

[0043] At operation 250, the method 200 can optionally include determining a first location of the first ultrasonic sensor relative to the first bone by comparing the first contour and the CT scan (or a model derived from a medical image of the first bone) of the first bone. At operation 260, the method 200 can optionally include determining a second location of the second ultrasonic sensor relative to the second bone by comparing the second contour and the CT scan (or a model derived from a medical image of the second bone) of the second bone. In examples, the controller (e.g., controller 130) can be configured to match the profile generated from the ultrasonic profiles and the cross-sectional profiles of the CT scan of the bone. There are various strategies to complete such matching, which will be discussed herein.

[0044] At operation 270, the method 200 can optionally include aligning, using the reference location, the first location, and the second location, the CT scan to a reference coordinate system maintained by the navigation system. Once the ultrasonic profile is matched with a cross-section of the bone, the ultrasonic profile can be aligned to the cross-section of the CT scan to accurately determine a location and orientation of the bone based on the location and orientation of the first ultrasonic sensor (e.g., the tibia sensor assembly 112) and the second ultrasonic sensor (e.g., the femur sensor assembly 114).

[0045] At operation 280, the method 200 can optionally include transforming the first location and the second location into coordinates within the reference coordinate system to track the orientation of the first bone and the orientation of the second bone throughout a surgical procedure. Such alignment from operation 270 can translate the location of the first ultrasonic sensor (e.g., the tibia sensor assembly 112) and the second ultrasonic sensor (e.g., the femur sensorassembly 114) into the coordinate system that the robotic surgical system (e.g., the surgical robot 120) uses to maintain locations of objects during a medical procedure.

[0046] FIG.3 illustrates an example of a first bone 310 and a second bone 320 including landmarks 330 labeled thereon. As discussed with reference to FIGS. 1 and 2, the landmarks 330 can be positioned on the 3D-model (from a CT scan or MRI scan) of the bones (e.g., the first bone 310 or the second bone 320).

[0047] For example, the landmarks 330 can indicate ends, regions, or specific anatomical points of the bone, such as portions of the femur (e.g., medial and lateral condyles, internal and external supracondylar ridges, labium mediale, foramen nutricium, linea pectinea, or the like), portions of the tibia (e.g., lateral condyle, medical condyle, tibial tuberosity, medial malleolus, fibular notch, or the like), or any other portion of any bone that the surgical robot 120 (FIG. 1) can be configured to perform a medical procedure thereon.

[0048] The landmarks 330 can be defined pre-operatively or intraoperatively. The landmarks 330 can be defined manually, or using software programmed to scan the bone and determine the various regions or specific anatomical points of the bone for use by the robotic surgery system 100. In examples, machine learning techniques can be used to determine the landmarks 330 on the bones. In such examples, bones with landmarks can be used to train a convolutional neural network (CNN), and the CNN can be used to locate and label the landmarks 330 as the medical team is setting up for the surgical procedure.

[0049] FIG.4 illustrates an example graphical user interface GUI 400 showing an example validation of a location of bone validation. As shown in FIG.4, the GUI 400 can include validation elements 402 and validation status indicators 404. As shown in FIG. 4, the validation elements 402 can indicate a bone that the medical professional needs to validate before the system can be used. As shown, the validation elements 402 include a femur and a tibia. In examples, the validation elements 402 can be updated to include any bone, or other anatomical landmark that can be validated for the surgical procedure. The validation status indicators 404 can indicate whether each bone of the validation elements 402 have been successfully validated. The graphical user interface GUI 400 can also include other informationthat can be helpful to the medical professionals performing the medical procedure, such as the size of femur, size of bearing, and a size of the tibial. The graphical user interface GUI 400 can also include information about the surgeon and other surgical planning information.

[0050] FIGS. 5 and 6 will be discussed together. FIG. 5 illustrates an example of ultrasound images (e.g., 502 and 504) of a bone (e.g., either of the first bone 310 or the second bone 320) of a patient. FIG.6 illustrates an example of segmentation of an ultrasound image (e.g., 502 and 504) of a bone (e.g., either of the first bone 310 or the second bone 320) of a patient.

[0051] As shown in FIG.5, operation 210 (FIG.2) of method 200 (FIG.2) surface contours 506 and 508 (e.g., the light-colored portion of the images 502 and 504 shown within the indicator box) as shown in each of the images 502 and 504, can be found by the controller (e.g., the acquisition box 116, the controller 130, or any other controller of the robotic surgery system 100 (all shown in FIG.1). As discussed herein, the surface contours 506 and 508 can be a section profiles of the bone of the patient from a 3D-model, a computerized tomography (CT) scan, or MRI scan, including the first bone and / or the second bone.

[0052] As shown in FIG.6, operations 230 – 260 (FIG. 2) of the method 200 (FIG.2) can align the section profiles 610 and 612 of the bone with the surface contours 506 and 508, respectively. FIGS. 7–11 discuss aligning a segmentation of an ultrasound image (e.g., as shown in FIG. 6) to a bone (or a model of a bone) of the patient. More specifically, FIG.7 shows a general example of aligning the segmentation of an ultrasound image to a bone, and FIGS.8–11 show techniques or strategies to obtain the alignment between the segmentation of the ultrasound image and the model of the bone.

[0053] FIG.7 illustrates a schematic diagram 700 of aligning a contour generated from segmentation of an ultrasound image (e.g., contours 702 and 704) with an example 3D model (e.g., models 706 and 708) of a bone (e.g., a tibia or a femur) of a patient. As shown in FIG. 7, the contours 702 and 704 and the models 706 and 708 can be inputs into the segmentation system (e.g., controller 130 (FIG. 1)), and the controller 130 can compare the contours 702 and 704 to the models 706and 708 to generate the outputs 710 and 712, which include contour 702 and model 706 and contour 704 and model 708, respectively. The controller 130 can match the contours to the models to determine a location of the sensors (e.g., sensors installed in the tibia sensor assembly 112 or the femur sensor assembly 114) relative to the bone of the patient (e.g., the tibia or the femur).

[0054] FIG.8 illustrates a diagram showing an example of point-to-point alignment of an example ultrasound segmentation and a 3D model of a bone of a patient. The point-to-point alignment of the segmentation and 3D model of a bone shown in FIG.8 can be used for the first bone or the second bone. As shown in FIG. 8, the contour (e.g., contours 702 or 704 (FIG.7)) can include a plurality of points 802 and the profiles (e.g., section profiles 610 or 612) of the models (e.g., models 706 or 708) can include a plurality of points 804.

[0055] As shown in FIG.8, the point-to-point algorithm can include an iterative process including at least first iteration 806 and second iteration 808. The iterative point-to-point algorithm can include pairing each point of the plurality of points 802 with the closest point of the plurality of points 804. The points of the plurality of points 802 can be connected to form a first centroid and the points of the plurality of points 804 can be connected to form a second centroid (as shown by 702 and 704 (FIG. 7) and represented by the models 706 and 708 (FIG.7)). The controller (e.g., the controller 130 can be configured by the point-to-point algorithm (e.g., the stored instructions 152) to determine for each pair of the plurality of points 802 and the plurality of points 804, a rotation amount using a singular value decomposition of a coordinate-intercorrelation matrix and rotate the first centroid the relative to the second centroid, based on an average of the rotation amount for each pair of first points and second points, as shown in second iteration 808. In examples, the singular value decomposition of a coordinate-intercorrelation matrix can be determined by finding a mean of rotation matrices for each of the paired points of the plurality of points 802 and the plurality of points 804.

[0056] The point-to-point algorithm (e.g., the stored instructions 152 (FIG. 1)) can include more iterations that continually move each point of the plurality of points 802 toward the respective closest points of the plurality of points 804 androtate the points of the plurality of points 802 relative to the points of the plurality of points 804 until the average of the rotation amount is below a threshold (e.g., near zero).

[0057] FIG.9 illustrates a graphical representation 902 and matrix representation 904 of an example cost function for the alignment of FIG.8.

[0058] FIG.10 illustrates a diagram showing an example point-to-plane alignment of an example ultrasound segmentation and 3D model of a bone of a patient. As shown in FIG.10, the first contour can include a plurality of first points (e.g., shown as S1, S2, S3, etc.) and the CT scan of the first bone can include a plurality of second points (e.g., shown as d1, d2, and d3, etc.), and wherein aligning the CT scans of the first bone and the second bone to the reference coordinate system includes an iterative closest point to plane algorithm (e.g., instructions 152) causing the controller 130 to perform operations of determining a minimum distance from each point of the plurality of first points to a tangent plane of each point of the plurality of second points (e.g., shown as l1, l2, and l3, etc.). The minimum distance is measured from a normal direction (e.g., shown as n1, n2, and n3, etc.) of the tangent plane of each point of the plurality of second points. The point-to-plane algorithm can also cause the controller to align the plurality of the first points (e.g., shown as S1, S2, S3, etc.) and the plurality of the second points (e.g., shown as d1, d2, and d3, etc.) by simultaneously translating and rotating the plurality of the first points using a cost function (e.g., optimizing to minimize a loss function).

[0059] FIG.11 illustrates a matrix representation of a sample cost function 1100 for the alignment of FIG.10.

[0060] FIG.12 illustrates a diagram showing an example plane-to-plane alignment of an example ultrasound segmentation and 3D model of a bone of a patient. As shown in FIG.12, the first contour can include a plurality of first points 1202 (e.g., a single point of the plurality shown as s1) and the CT scan of the first bone can include a plurality of second points 1204 (a single point of the plurality shown as d1). Aligning the CT scans of the first bone and the second bone to the reference coordinate system can include an iterative closest point plane-to-plane algorithm (e.g., stored instructions 152 (FIG, 1)) that can cause the controller 130 todetermine a minimum distance (ni + mi) from each point of the plurality of first points 1202 to a tangent plane of each point of the plurality of second points 1204, instead of traditional distance (Di) between the points (d1 and s1). The minimum distance (ni + mi) can be measured from an average of a normal direction (mi) from each point of the plurality of first points and a normal direction (ni) of the tangent plane of each point of the plurality of second points. Based on the minimum distance (ni + mi) the plurality of first points and the plurality of second points can be aligned by simultaneously translating and rotating the plurality of first points relative to the plurality of second points using a cost function (e.g., optimizing to minimize a loss function).

[0061] FIG.13 illustrates a diagram showing an example plane-to-plane alignment of an example ultrasound segmentation and 3D model of a bone of a patient. More specifically, FIG.13 shows a projection of contour-normals (e.g., mi and ni plurality of first points 1202 (as shown in FIG.12)) on local bone transversal section (e.g., the plurality of second points 1204) for compatibility.

[0062] FIG.14 illustrates a block diagram of an example machine 1400 upon which any one or more of the techniques (e.g., methodologies) discussed herein may perform. Examples, as described herein, may include, or may operate by, logic or a number of components, or mechanisms in the machine 1400. Circuitry (e.g., processing circuitry) is a collection of circuits implemented in tangible entities of the machine 1400 that include hardware (e.g., simple circuits, gates, logic, etc.). Circuitry membership may be flexible over time. Circuitries include members that may, alone or in combination, perform specified operations when operating. In examples, hardware of the circuitry may be immutably designed to carry out a specific operation (e.g., hardwired). In examples, the hardware of the circuitry may include variably connected physical components (e.g., execution units, transistors, simple circuits, etc.) including a machine-readable medium physically modified (e.g., magnetically, electrically, moveable placement of invariant massed particles, etc.) to encode instructions of the specific operation. In connecting the physical components, the underlying electrical properties of a hardware constituent are changed, for example, from an insulator to a conductor or vice versa. Theinstructions enable embedded hardware (e.g., the execution units or a loading mechanism) to create members of the circuitry in hardware via the variable connections to carry out portions of the specific operation when in operation. Accordingly, in examples, the machine-readable medium elements are part of the circuitry or are communicatively coupled to the other components of the circuitry when the device is operating. In examples, any of the physical components may be used in more than one member of more than one circuitry. For example, under operation, execution units may be used in a first circuit of a first circuitry at one point in time and reused by a second circuit in the first circuitry, or by a third circuit in a second circuitry at a different time. Additional examples of these components with respect to the machine 1400 follow.

[0063] In alternative examples, the machine 1400 may operate as a standalone device or may be connected (e.g., networked) to other machines. In a networked deployment, the machine 1400 may operate in the capacity of a server machine, a client machine, or both in server-client network environments. In examples, the machine 1400 may act as a peer machine in peer-to-peer (P2P) (or other distributed) network environment. The machine 1400 may be a personal computer (PC), a tablet PC, a set-top box (STB), a personal digital assistant (PDA), a mobile telephone, a web appliance, a network router, switch or bridge, or any machine capable of executing instructions (sequential or otherwise) that specify actions to be taken by that machine. Further, while only a single machine is illustrated, the term “machine” shall also be taken to include any collection of machines that individually or jointly execute a set (or multiple sets) of instructions to perform any one or more of the methodologies discussed herein, such as cloud computing, software as a service (SaaS), other computer cluster configurations.

[0064] The machine (e.g., computer system) 1400 may include a hardware processor 1402 (e.g., a central processing unit (CPU), a graphics processing unit (GPU), a hardware processor core, or any combination thereof), a main memory 1404, a static memory (e.g., memory or storage for firmware, microcode, a basic- input-output (BIOS), unified extensible firmware interface (UEFI), etc.) 1406, and mass storage 1408 (e.g., hard drives, tape drives, flash storage, or other blockdevices) some or all of which may communicate with each other via an interlink (e.g., bus) 1430. The machine 1400 may further include a display unit 1410, an alphanumeric input device 1412 (e.g., a keyboard), and a user interface (UI) navigation device 1414 (e.g., a mouse). In examples, the display unit 1410, input device 1412 and UI navigation device 1414 may be a touch screen display. The machine 1400 may additionally include a storage device (e.g., drive unit) 1408, a signal generation device 1418 (e.g., a speaker), a network interface device 1420, and one or more sensors 1416, such as a global positioning system (GPS) sensor, compass, accelerometer, or other sensor. The machine 1400 may include an output controller 1428, such as a serial (e.g., universal serial bus (USB), parallel, or other wired or wireless (e.g., infrared (IR), near field communication (NFC), etc.) connection to communicate or control one or more peripheral devices (e.g., a printer, card reader, etc.).

[0065] Registers of the processor 1402, the main memory 1404, the static memory 1406, or the mass storage 1408 may be, or include, a machine readable medium 1422 on which is stored one or more sets of data structures or instructions 1424 (e.g., software) embodying or utilized by any one or more of the techniques or functions described herein. The instructions 1424 may also reside, completely or at least partially, within any of registers of the processor 1402, the main memory 1404, the static memory 1406, or the mass storage 1408 during execution thereof by the machine 1400. In examples, one or any combination of the hardware processor 1402, the main memory 1404, the static memory 1406, or the mass storage 1408 may constitute the machine readable media 1422. While the machine readable medium 1422 is illustrated as a single medium, the term “machine readable medium” may include a single medium or multiple media (e.g., a centralized or distributed database, and / or associated caches and servers) configured to store the one or more instructions 1424.

[0066] The term “machine-readable medium” may include any medium that is capable of storing, encoding, or carrying instructions for execution by the machine 1400 and that causes the machine 1400 to perform any one or more of the techniques of the present disclosure, or that is capable of storing, encoding orcarrying data structures used by or associated with such instructions. Non-limiting machine-readable medium examples may include solid-state memories, optical media, magnetic media, and signals (e.g., radio frequency signals, other photon- based signals, sound signals, etc.). In examples, a non-transitory machine-readable medium comprises a machine-readable medium with a plurality of particles having invariant (e.g., rest) mass, and thus are compositions of matter. Accordingly, non- transitory machine-readable media are machine-readable media that do not include transitory propagating signals. Specific examples of non-transitory machine- readable media may include: non-volatile memory, such as semiconductor memory devices (e.g., Electrically Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM)) and flash memory devices; magnetic disks, such as internal hard disks and removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks.

[0067] In examples, information stored or otherwise provided on the machine-readable medium 1422 may be representative of the instructions 1424, such as instructions 1424 themselves or a format from which the instructions 1424 may be derived. This format from which the instructions 1424 may be derived may include source code, encoded instructions (e.g., in compressed or encrypted form), packaged instructions (e.g., split into multiple packages), or the like. The information representative of the instructions 1424 in the machine-readable medium 1422 may be processed by processing circuitry into the instructions to implement any of the operations discussed herein. For example, deriving the instructions 1424 from the information (e.g., processing by the processing circuitry) may include: compiling (e.g., from source code, object code, etc.), interpreting, loading, organizing (e.g., dynamically or statically linking), encoding, decoding, encrypting, unencrypting, packaging, unpackaging, or otherwise manipulating the information into the instructions 1424.

[0068] In examples, the derivation of the instructions 1424 may include assembly, compilation, or interpretation of the information (e.g., by the processing circuitry) to create the instructions 1424 from some intermediate or preprocessed format provided by the machine-readable medium 1422. The information, whenprovided in multiple parts, may be combined, unpacked, and modified to create the instructions 1424. For example, the information may be in multiple compressed source code packages (or object code, or binary executable code, etc.) on one or several remote servers. The source code packages may be encrypted when in transit over a network and decrypted, uncompressed, assembled (e.g., linked) if necessary, and compiled or interpreted (e.g., into a library, stand-alone executable etc.) at a local machine, and executed by the local machine.

[0069] The instructions 1424 may be further transmitted or received over a communications network 1426 using a transmission medium via the network interface device 1420 utilizing any one of a number of transfer protocols (e.g., frame relay, internet protocol (IP), transmission control protocol (TCP), user datagram protocol (UDP), hypertext transfer protocol (HTTP), etc.). Example communication networks may include a local area network (LAN), a wide area network (WAN), a packet data network (e.g., the Internet), LoRa / LoRaWAN, or satellite communication networks, mobile telephone networks (e.g., cellular networks such as those complying with 3G, 4G LTE / LTE-A, or 5G standards), Plain Old Telephone (POTS) networks, and wireless data networks (e.g., Institute of Electrical and Electronics Engineers (IEEE) 502.11 family of standards known as Wi-Fi®, IEEE 502.15.4 family of standards, peer-to-peer (P2P) networks, among others. In examples, the network interface device 1420 may include one or more physical jacks (e.g., Ethernet, coaxial, or phone jacks) or one or more antennas to connect to the communications network 1426. In examples, the network interface device 1420 may include a plurality of antennas to wirelessly communicate using at least one of single-input multiple-output (SIMO), multiple-input multiple-output (MIMO), or multiple-input single-output (MISO) techniques. The term “transmission medium” shall be taken to include any intangible medium that is capable of storing, encoding or carrying instructions for execution by the machine 1400, and includes digital or analog communications signals or other intangible medium to facilitate communication of such software. A transmission medium is a machine-readable medium.

[0070] The following, non-limiting examples, detail certain aspects of the present subject matter to solve the challenges and provide the benefits discussed herein, among others.

[0071] Example 1 is a method for pinless navigation to track an orientation of a first bone and a second bone of a patient during a robotic surgical procedure, the first bone and the second bone forming a joint, the method comprising: accessing a computerized tomography scan including each of the first bone and the second bone; obtaining, using a surgical navigation system, a reference location of a landmark on the first bone or the second bone; generating, using a first ultrasonic sensor, a first contour of the first bone at a known location along the first bone; generating, using a second ultrasonic sensor, a second contour of the second bone at a known location along the second bone; determining a first location of the first ultrasonic sensor relative to the first bone by comparing the first contour and the computerized tomography scan of the first bone; determining a second location of the second ultrasonic sensor relative to the second bone by comparing the second contour and the computerized tomography scan of the second bone; aligning, using the reference location, the first location, and the second location, the computerized tomography scan to a reference coordinate system maintained by the surgical navigation system; and transforming the first location and the second location into coordinates within the reference coordinate system to track the orientation of the first bone and the orientation of the second bone throughout a surgical procedure.

[0072] In Example 2, the subject matter of Example 1 optionally includes accessing one or more first landmarks on the computerized tomography scan of the first bone, each landmark of the one or more first landmarks corresponding to a unique anatomical portion of the first bone; and accessing one or more second landmarks on the computerized tomography scan of the second bone, each landmark of the one or more second landmarks corresponding to a unique anatomical portion of the second bone.

[0073] In Example 3, the subject matter of Example 2 optionally includes wherein the one or more first landmarks and the one or more second landmarks are recognized and labeled preoperatively.

[0074] In Example 4, the subject matter of Example 3 optionally includes wherein obtaining the reference location includes capturing a coordinate using a pointer tracked by the surgical navigation system.

[0075] In Example 5, the subject matter of any one or more of Examples 3–4 optionally include wherein the first contour includes a plurality of first points and the computerized tomography scan of the first bone includes a plurality of second points, and wherein aligning the computerized tomography scan of the first bone and the computerized tomography scan of the second bone to the reference coordinate system includes an iterative closest point to point algorithm comprising: pairing each point of the plurality of first points with a closest point of the plurality of second points; joining a first centroid of the plurality of first points and a second centroid of the plurality of second points; determining, for each pair of the plurality of first points and the plurality of second points, a rotation amount using a singular value decomposition of a coordinate-intercorrelation matrix; and rotating, the first centroid relative to the second centroid, based on an average of the rotation amount for each pair of first points and second points.

[0076] In Example 6, the subject matter of Example 5 optionally includes wherein the second contour includes a plurality of third points and the computerized tomography scan of the second bone includes a plurality of fourth points, and wherein the iterative closest point to point algorithm comprising: pairing each point of the plurality of third points with a closest point of the plurality of fourth points; joining a third centroid of the plurality of third points and a fourth centroid of the plurality of fourth points; determining, for each pair of the plurality of third points and the plurality of fourth points, a second rotation amount using a singular value decomposition of a coordinate-intercorrelation matrix; and rotating, the third centroid relative to the fourth centroid, based on an average of the rotation amount for each pair of third points and fourth points.

[0077] In Example 7, the subject matter of any one or more of Examples 3–6 optionally include wherein the first contour includes a plurality of first points and the computerized tomography scan of the first bone includes a plurality of second points, and wherein aligning the computerized tomography scan of the first boneand the computerized tomography scan of the second bone to the reference coordinate system includes an iterative closest point to plane algorithm comprising: determining a minimum distance from each point of the plurality of first points to a tangent plane of each point of the plurality of second points, the minimum distance is measured from a normal direction of the tangent plane of each point of the plurality of second points; and aligning the plurality of first points and the plurality of second points by simultaneously translating and rotating the plurality of first points using a cost function.

[0078] In Example 8, the subject matter of any one or more of Examples 3–7 optionally include wherein the first contour includes a plurality of first points and the computerized tomography scan of the first bone includes a plurality of second points, and wherein aligning the computerized tomography scan of the first bone and the computerized tomography scan of the second bone to the reference coordinate system includes an iterative closest point plane to plane algorithm: determining a minimum distance from each point of the plurality of first points to a tangent plane of each point of the plurality of second points, the minimum distance is measured from an average of a normal direction from each point of the plurality of first points and a normal direction of the tangent plane of each point of the plurality of second points; and aligning the plurality of first points and the plurality of second points by simultaneously translating and rotating the plurality of first points using a cost function.

[0079] In Example 9, the subject matter of Example 8 optionally includes wherein the second contour includes a plurality of third points and the computerized tomography scan of the second bone includes a plurality of fourth points, and wherein the iterative closest point plane to plane algorithm includes: determining a minimum distance from each point of the plurality of third points to a tangent plane of each point of the plurality of fourth points, the minimum distance is measured from a normal direction of the tangent plane of each point of the plurality of fourth points; and aligning the plurality of third points and the plurality of fourth points by simultaneously translating and rotating the plurality of third points using a cost function.

[0080] In Example 10, the subject matter of any one or more of Examples 8– 9 optionally include wherein the second contour includes a plurality of third points and the computerized tomography scan of the second bone includes a plurality of fourth points, and wherein the iterative closest point plane to plane algorithm includes: determining a minimum distance from each point of the plurality of third points to a tangent plane of each point of the plurality of fourth points, the minimum distance is measured from an average of a normal direction from each point of the plurality of third points and a normal direction of the tangent plane of each point of the plurality of fourth points; and aligning the plurality of third points and the plurality of fourth points by simultaneously translating and rotating the plurality of third points using a cost function.

[0081] In Example 11, the subject matter of any one or more of Examples 1– 10 optionally include wherein accessing a computerized tomography (CT) scan of each of the first bone and the second bone comprises: accessing a computerized tomography scan of a leg of the patient, the computerized tomography scan of the leg including the first bone, the second bone, and a knee joint formed by the first bone and the second bone.

[0082] In Example 12, the subject matter of any one or more of Examples 1– 11 optionally include wherein accessing a computerized tomography scan of each of the first bone and the second bone comprises: accessing a computerized tomography scan of a hip of the patient, the computerized tomography scan of the hip including the first bone, the second bone, and a hip joint formed by the first bone and the second bone.

[0083] In Example 13, the subject matter of any one or more of Examples 1– 12 optionally include wherein accessing a computerized tomography (CT) scan of each of the first bone and the second bone comprises: accessing a computerized tomography scan of an arm of the patient, the computerized tomography scan of the arm including the first bone, the second bone, and an elbow joint formed by the first bone and the second bone.

[0084] In Example 14, the subject matter of any one or more of Examples 1– 13 optionally include wherein a first ultrasonic image, captured by the firstultrasonic sensor, includes a proximal contour captured on a proximal portion of the first bone and a distal contour captured on a distal portion of the first bone.

[0085] In Example 15, the subject matter of any one or more of Examples 1– 14 optionally include wherein a second ultrasonic image, captured by the second ultrasonic sensor, includes a proximal contour captured on a proximal portion of the second bone and a distal contour captured on a distal portion of the second bone.

[0086] In Example 16, the subject matter of Example 15 optionally includes wherein the second bone is a tibia, and wherein the second ultrasonic sensor is installed anterior the tibia.

[0087] Example 17 is a method for pinless navigation to track an orientation of a bone by a robotic surgical system, the method comprising: identifying, using a surgical navigation system, a reference location of a landmark on the bone; determining a location of an ultrasonic sensor relative to the bone by comparing a contour from an ultrasonic image captured by the ultrasonic sensor and a computerized tomography scan of the bone; aligning, using the reference location and the location of the ultrasonic sensor, the computerized tomography scan to a reference coordinate system maintained by the surgical navigation system; and transforming the location into coordinates within the reference coordinate system to track the orientation of the bone throughout a surgical procedure.

[0088] In Example 18, the subject matter of Example 17 optionally includes accessing one or more landmarks on the computerized tomography scan of the bone, each landmark of the one or more landmarks corresponding to a unique anatomical portion of the bone.

[0089] In Example 19, the subject matter of Example 18 optionally includes wherein the one or more landmarks are recognized and labeled preoperatively.

[0090] In Example 20, the subject matter of Example 19 optionally includes wherein identifying the reference location includes capturing a coordinate using a pointer tracked by the surgical navigation system.

[0091] In Example 21, the subject matter of any one or more of Examples 19–20 optionally include wherein aligning the computerized tomography scan to the reference coordinate system includes an iterative closest point to point algorithm.

[0092] In Example 22, the subject matter of Example 21 optionally includes wherein the contour includes a plurality of first points and the computerized tomography scan includes a plurality of second points, and wherein the iterative closest point to point algorithm includes: pairing each point of the plurality of first points with a closest point of the plurality of second points; joining a first centroid of the plurality of first points and a second centroid of the plurality of second points; determining, for each pair of the plurality of first points and the plurality of second points, a rotation amount using a singular value decomposition of a coordinate- intercorrelation matrix; and rotating, the first centroid relative to the second centroid, based on an average of the rotation amount for each pair of first points and second points.

[0093] In Example 23, the subject matter of any one or more of Examples 19–22 optionally include wherein aligning the computerized tomography scan of the bone to the reference coordinate system includes an iterative closest point plane to plane algorithm.

[0094] In Example 24, the subject matter of Example 23 optionally includes wherein the contour includes a plurality of first points and the computerized tomography scan of the bone includes a plurality of second points, and wherein the iterative closest point plane to plane algorithm includes: determining a minimum distance from each point of the plurality of first points to a tangent plane of each point of the plurality of second points, the minimum distance is measured from a normal direction of the tangent plane of each point of the plurality of second points; and aligning the plurality of first points and the plurality of second points by simultaneously translating and rotating the plurality of first points using a cost function.

[0095] In Example 25, the subject matter of any one or more of Examples 23–24 optionally include wherein the contour includes a plurality of first points and the computerized tomography scan of the bone includes a plurality of second points, and wherein the iterative closest point plane to plane algorithm includes: determining a minimum distance from each point of the plurality of first points to a tangent plane of each point of the plurality of second points, the minimum distanceis measured from an average a normal direction from each point of the plurality of first points and a normal direction of the tangent plane of each point of the plurality of second points; and aligning the plurality of first points and the plurality of second points by simultaneously translating and rotating the plurality of first points using a cost function.

[0096] Example 26 is a robotic surgery system comprising: a surgical navigation system configured to track a location of one or more objects within an operating room during a surgical procedure; a surgical robot configured to perform the surgical procedure; and controller circuitry coupled to memory circuitry, including stored instructions, that, when performed by the controller circuitry, cause the controller circuitry to: identify, using the surgical navigation system, a reference location of a landmark on a bone; determine a location of an ultrasonic sensor relative to the bone by comparing a contour from an ultrasonic image captured by the ultrasonic sensor and a computerized tomography scan of the bone; align, using the reference location, the computerized tomography scan to a reference coordinate system maintained by the surgical navigation system; and transform the location into coordinates within the reference coordinate system to track an orientation of the bone throughout a surgical procedure.

[0097] In Example 27, the subject matter of Example 26 optionally includes wherein the stored instructions, when performed by the controller circuitry, cause the controller circuitry to: access one or more landmarks on the computerized tomography scan of the bone, each landmark of the one or more landmarks corresponding to a unique anatomical portion of the bone.

[0098] In Example 28, the subject matter of Example 27 optionally includes wherein the one or more landmarks are recognized and labeled preoperatively.

[0099] In Example 29, the subject matter of Example 28 optionally includes wherein identifying the reference location includes capturing a coordinate using a pointer tracked by the surgical navigation system.

[0100] In Example 30, the subject matter of any one or more of Examples 28–29 optionally include wherein aligning the computerized tomography scan to the reference coordinate system includes an iterative closest point to point algorithm.

[0101] In Example 31, the subject matter of Example 30 optionally includes wherein the contour includes a plurality of first points and the computerized tomography scan includes a plurality of second points, and wherein the iterative closest point to point algorithm includes: pairing each point of the plurality of first points with a closest point of the plurality of second points; joining a first centroid of the plurality of first points and a second centroid of the plurality of second points; determining, for each pair of the plurality of first points and the plurality of second points, a rotation amount using a singular value decomposition of a coordinate- intercorrelation matrix; and rotating, the first centroid relative to the second centroid, based on an average of the rotation amount for each pair of first points and second points.

[0102] In Example 32, the subject matter of any one or more of Examples 28–31 optionally include wherein aligning the computerized tomography scan of the bone to the reference coordinate system includes an iterative closest point plane to plane algorithm.

[0103] In Example 33, the subject matter of Example 32 optionally includes wherein the contour includes a plurality of first points and the computerized tomography scan of the bone includes a plurality of second points, and wherein the iterative closest point plane to plane algorithm includes: determining a minimum distance from each point of the plurality of first points to a tangent plane of each point of the plurality of second points, the minimum distance is measured from a normal direction of the tangent plane of each point of the plurality of second points; and aligning the plurality of first points and the plurality of second points by simultaneously translating and rotating the plurality of first points using a cost function.

[0104] In Example 34, the subject matter of any one or more of Examples 32–33 optionally include wherein the contour includes a plurality of first points and the computerized tomography scan of the bone includes a plurality of second points, and wherein the iterative closest point plane to plane algorithm includes: determining a minimum distance from each point of the plurality of first points to a tangent plane of each point of the plurality of second points, the minimum distanceis measured from an average a normal direction from each point of the plurality of first points and a normal direction of the tangent plane of each point of the plurality of second points; and aligning the plurality of first points and the plurality of second points by simultaneously translating and rotating the plurality of first points using a cost function.

[0105] Example 35 includes a method, apparatus, or system including any element of any of Examples 1–34.

[0106] The above detailed description includes references to the accompanying drawings, which form a part of the detailed description. The drawings show, by way of illustration, specific examples that may be practiced. These embodiments are also referred to herein as “examples.” Such examples may include elements in addition to those shown or described. However, the present inventors also contemplate examples in which only those elements shown or described are provided. Moreover, the present inventors also contemplate examples using any combination or permutation of those elements shown or described (or one or more aspects thereof), either with respect to a particular example (or one or more aspects thereof), or with respect to other examples (or one or more aspects thereof) shown or described herein.

[0107] All publications, patents, and patent documents referred to in this document are incorporated by reference herein in their entirety, as though individually incorporated by reference. In the event of inconsistent usages between this document and those documents so incorporated by reference, the usage in the incorporated reference(s) should be considered supplementary to that of this document; for irreconcilable inconsistencies, the usage in this document controls.

[0108] In this document, the terms “a” or “an” are used, as is common in patent documents, to include one or more than one, independent of any other instances or usages of “at least one” or “one or more.” In this document, the term “or” is used to refer to a nonexclusive or, such that “A or B” includes “A but not B,” “B but not A,” and “A and B,” unless otherwise indicated. In the appended claims, the terms “including” and “in which” are used as the plain-English equivalents of the respective terms “comprising” and “wherein.” Also, in the following claims, theterms “including” and “comprising” are open-ended, that is, a system, device, article, or process that includes elements in addition to those listed after such a term in a claim are still deemed to fall within the scope of that claim. Moreover, in the following claims, the terms “first,” “second,” and “third,” etc. are used merely as labels, and are not intended to impose numerical requirements on their objects.

[0109] The term “about,” as used herein, means approximately, in the region of, roughly, or around. When the term “about” is used in conjunction with a numerical range, it modifies that range by extending the boundaries above and below the numerical values set forth. In general, the term “about” is used herein to modify a numerical value above and below the stated value by a variance of 10%. In one aspect, the term “about” means plus or minus 10% of the numerical value of the number with which it is being used. Therefore, about 50% means in the range of 45%-55%. Numerical ranges recited herein by endpoints include all numbers and fractions subsumed within that range (e.g., 1 to 5 includes 1, 1.5, 2, 2.75, 3, 3.90, 4, 4.24, and 5). Similarly, numerical ranges recited herein by endpoints include subranges subsumed within that range (e.g., 1 to 5 includes 1-1.5, 1.5-2, 2-2.75, 2.75-3, 3-3.90, 3.90-4, 4-4.24, 4.24-5, 2-5, 3-5, 1-4, and 2-4). It is also to be understood that all numbers and fractions thereof are presumed to be modified by the term “about.”

[0110] The above description is intended to be illustrative, and not restrictive. For example, the above-described examples (or one or more aspects thereof) may be used in combination with each other. Other examples may be used, such as by one of ordinary skill in the art upon reviewing the above description. The Abstract is to allow the reader to quickly ascertain the nature of the technical disclosure and is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. Also, in the above Detailed Description, various features may be grouped together to streamline the disclosure. This should not be interpreted as intending that an unclaimed disclosed feature is essential to any claim. Rather, inventive subject matter may lie in less than all features of a particular disclosed embodiment. Thus, the following claims are hereby incorporated into the Detailed Description, with each claim standing on itsown as a separate embodiment. The scope of the examples should be determined with reference to the appended claims, along with the full scope of equivalents to which such claims are entitled.

Claims

CLAIMS Claims:

1. A method for pinless navigation to track an orientation of a first bone and a second bone of a patient during a robotic surgical procedure, the first bone and the second bone forming a joint, the method comprising: accessing a computerized tomography scan including each of the first bone and the second bone; obtaining, using a surgical navigation system, a reference location of a landmark on the first bone or the second bone; generating, using a first ultrasonic sensor, a first contour of the first bone at a known location along the first bone; generating, using a second ultrasonic sensor, a second contour of the second bone at a known location along the second bone; determining a first location of the first ultrasonic sensor relative to the first bone by comparing the first contour and the computerized tomography scan of the first bone; determining a second location of the second ultrasonic sensor relative to the second bone by comparing the second contour and the computerized tomography scan of the second bone; aligning, using the reference location, the first location, and the second location, the computerized tomography scan to a reference coordinate system maintained by the surgical navigation system; and transforming the first location and the second location into coordinates within the reference coordinate system to track the orientation of the first bone and the orientation of the second bone throughout a surgical procedure.

2. The method of claim 1, comprising:accessing one or more first landmarks on the computerized tomography scan of the first bone, each landmark of the one or more first landmarks corresponding to a unique anatomical portion of the first bone; and accessing one or more second landmarks on the computerized tomography scan of the second bone, each landmark of the one or more second landmarks corresponding to a unique anatomical portion of the second bone.

3. The method of claim 2, wherein the one or more first landmarks and the one or more second landmarks are recognized and labeled preoperatively.

4. The method of claim 3, wherein obtaining the reference location includes capturing a coordinate using a pointer tracked by the surgical navigation system.

5. The method of any of claims 3–4, wherein the first contour includes a plurality of first points and the computerized tomography scan of the first bone includes a plurality of second points, and wherein aligning the computerized tomography scan of the first bone and the computerized tomography scan of the second bone to the reference coordinate system includes an iterative closest point to point algorithm comprising: pairing each point of the plurality of first points with a closest point of the plurality of second points; joining a first centroid of the plurality of first points and a second centroid of the plurality of second points; determining, for each pair of the plurality of first points and the plurality of second points, a rotation amount using a singular value decomposition of a coordinate-intercorrelation matrix; and rotating, the first centroid relative to the second centroid, based on an average of the rotation amount for each pair of first points and second points.

6. The method of claim 5, wherein the second contour includes a plurality of third points and the computerized tomography scan of the second bone includes a plurality of fourth points, and wherein the iterative closest point to point algorithm comprising: pairing each point of the plurality of third points with a closest point of the plurality of fourth points; joining a third centroid of the plurality of third points and a fourth centroid of the plurality of fourth points; determining, for each pair of the plurality of third points and the plurality of fourth points, a second rotation amount using a singular value decomposition of a coordinate-intercorrelation matrix; and rotating, the third centroid relative to the fourth centroid, based on an average of the rotation amount for each pair of third points and fourth points.

7. The method of any of claims 3–6, wherein the first contour includes a plurality of first points and the computerized tomography scan of the first bone includes a plurality of second points, and wherein aligning the computerized tomography scan of the first bone and the computerized tomography scan of the second bone to the reference coordinate system includes an iterative closest point to plane algorithm comprising: determining a minimum distance from each point of the plurality of first points to a tangent plane of each point of the plurality of second points, the minimum distance is measured from a normal direction of the tangent plane of each point of the plurality of second points; and aligning the plurality of first points and the plurality of second points by simultaneously translating and rotating the plurality of first points using a cost function.

8. The method of any of claims 3–7, wherein the first contour includes a plurality of first points and the computerized tomography scan of the first bone includes a plurality of second points, and wherein aligning the computerized tomography scan of the first bone and the computerized tomography scan of the second bone to the reference coordinate system includes an iterative closest point plane to plane algorithm: determining a minimum distance from each point of the plurality of first points to a tangent plane of each point of the plurality of second points, the minimum distance is measured from an average of a normal direction from each point of the plurality of first points and a normal direction of the tangent plane of each point of the plurality of second points; and aligning the plurality of first points and the plurality of second points by simultaneously translating and rotating the plurality of first points using a cost function.

9. The method of claim 8, wherein the second contour includes a plurality of third points and the computerized tomography scan of the second bone includes a plurality of fourth points, and wherein the iterative closest point plane to plane algorithm includes: determining a minimum distance from each point of the plurality of third points to a tangent plane of each point of the plurality of fourth points, the minimum distance is measured from a normal direction of the tangent plane of each point of the plurality of fourth points; and aligning the plurality of third points and the plurality of fourth points by simultaneously translating and rotating the plurality of third points using a cost function.

10. The method of any of claims 8–9, wherein the second contour includes a plurality of third points and the computerized tomography scan of the second boneincludes a plurality of fourth points, and wherein the iterative closest point plane to plane algorithm includes: determining a minimum distance from each point of the plurality of third points to a tangent plane of each point of the plurality of fourth points, the minimum distance is measured from an average of a normal direction from each point of the plurality of third points and a normal direction of the tangent plane of each point of the plurality of fourth points; and aligning the plurality of third points and the plurality of fourth points by simultaneously translating and rotating the plurality of third points using a cost function.

11. The method of any of claims 1–10, wherein accessing a computerized tomography (CT) scan of each of the first bone and the second bone comprises: accessing a computerized tomography scan of a leg of the patient, the computerized tomography scan of the leg including the first bone, the second bone, and a knee joint formed by the first bone and the second bone.

12. The method of any of claims 1–11, wherein accessing a computerized tomography scan of each of the first bone and the second bone comprises: accessing a computerized tomography scan of a hip of the patient, the computerized tomography scan of the hip including the first bone, the second bone, and a hip joint formed by the first bone and the second bone.

13. The method of any of claims 1–12, wherein accessing a computerized tomography (CT) scan of each of the first bone and the second bone comprises: accessing a computerized tomography scan of an arm of the patient, the computerized tomography scan of the arm including the first bone,the second bone, and an elbow joint formed by the first bone and the second bone.

14. The method of any of claims 1–13, wherein a first ultrasonic image, captured by the first ultrasonic sensor, includes a proximal contour captured on a proximal portion of the first bone and a distal contour captured on a distal portion of the first bone.

15. The method of any of claims 1–14, wherein a second ultrasonic image, captured by the second ultrasonic sensor, includes a proximal contour captured on a proximal portion of the second bone and a distal contour captured on a distal portion of the second bone.

16. The method of claim 15, wherein the second bone is a tibia, and wherein the second ultrasonic sensor is installed anterior the tibia.

17. A method for pinless navigation to track an orientation of a bone by a robotic surgical system, the method comprising: identifying, using a surgical navigation system, a reference location of a landmark on the bone; determining a location of an ultrasonic sensor relative to the bone by comparing a contour from an ultrasonic image captured by the ultrasonic sensor and a computerized tomography scan of the bone; aligning, using the reference location and the location of the ultrasonic sensor, the computerized tomography scan to a reference coordinate system maintained by the surgical navigation system; and transforming the location into coordinates within the reference coordinate system to track the orientation of the bone throughout a surgical procedure.

18. The method of claim 17, comprising: accessing one or more landmarks on the computerized tomography scan of the bone, each landmark of the one or more landmarks corresponding to a unique anatomical portion of the bone.

19. The method of claim 18, wherein the one or more landmarks are recognized and labeled preoperatively.

20. The method of claim 19, wherein identifying the reference location includes capturing a coordinate using a pointer tracked by the surgical navigation system.

21. The method of any of claims 19–20, wherein aligning the computerized tomography scan to the reference coordinate system includes an iterative closest point to point algorithm.

22. The method of claim 21, wherein the contour includes a plurality of first points and the computerized tomography scan includes a plurality of second points, and wherein the iterative closest point to point algorithm includes: pairing each point of the plurality of first points with a closest point of the plurality of second points; joining a first centroid of the plurality of first points and a second centroid of the plurality of second points; determining, for each pair of the plurality of first points and the plurality of second points, a rotation amount using a singular value decomposition of a coordinate-intercorrelation matrix; and rotating, the first centroid relative to the second centroid, based on an average of the rotation amount for each pair of first points and second points.

23. The method of any of claims 19–22, wherein aligning the computerized tomography scan of the bone to the reference coordinate system includes an iterative closest point plane to plane algorithm.

24. The method of claim 23, wherein the contour includes a plurality of first points and the computerized tomography scan of the bone includes a plurality of second points, and wherein the iterative closest point plane to plane algorithm includes: determining a minimum distance from each point of the plurality of first points to a tangent plane of each point of the plurality of second points, the minimum distance is measured from a normal direction of the tangent plane of each point of the plurality of second points; and aligning the plurality of first points and the plurality of second points by simultaneously translating and rotating the plurality of first points using a cost function.

25. The method of any of claims 23–24, wherein the contour includes a plurality of first points and the computerized tomography scan of the bone includes a plurality of second points, and wherein the iterative closest point plane to plane algorithm includes: determining a minimum distance from each point of the plurality of first points to a tangent plane of each point of the plurality of second points, the minimum distance is measured from an average a normal direction from each point of the plurality of first points and a normal direction of the tangent plane of each point of the plurality of second points; and aligning the plurality of first points and the plurality of second points by simultaneously translating and rotating the plurality of first points using a cost function.

26. A robotic surgery system comprising: a surgical navigation system configured to track a location of one or more objects within an operating room during a surgical procedure; a surgical robot configured to perform the surgical procedure; and controller circuitry coupled to memory circuitry, including stored instructions, that, when performed by the controller circuitry, cause the controller circuitry to: identify, using the surgical navigation system, a reference location of a landmark on a bone; determine a location of an ultrasonic sensor relative to the bone by comparing a contour from an ultrasonic image captured by the ultrasonic sensor and a computerized tomography scan of the bone; align, using the reference location, the computerized tomography scan to a reference coordinate system maintained by the surgical navigation system; and transform the location into coordinates within the reference coordinate system to track an orientation of the bone throughout a surgical procedure.

27. The robotic surgery system of claim 26, wherein the stored instructions, when performed by the controller circuitry, cause the controller circuitry to: access one or more landmarks on the computerized tomography scan of the bone, each landmark of the one or more landmarks corresponding to a unique anatomical portion of the bone.

28. The robotic surgery system of claim 27, wherein the one or more landmarks are recognized and labeled preoperatively.

29. The robotic surgery system of claim 28, wherein identifying the reference location includes capturing a coordinate using a pointer tracked by the surgical navigation system.

30. The robotic surgery system of any of claims 28–29, wherein aligning the computerized tomography scan to the reference coordinate system includes an iterative closest point to point algorithm.

31. The robotic surgery system of claim 30, wherein the contour includes a plurality of first points and the computerized tomography scan includes a plurality of second points, and wherein the iterative closest point to point algorithm includes: pairing each point of the plurality of first points with a closest point of the plurality of second points; joining a first centroid of the plurality of first points and a second centroid of the plurality of second points; determining, for each pair of the plurality of first points and the plurality of second points, a rotation amount using a singular value decomposition of a coordinate-intercorrelation matrix; and rotating, the first centroid relative to the second centroid, based on an average of the rotation amount for each pair of first points and second points.

32. The robotic surgery system of any of claims 28–31, wherein aligning the computerized tomography scan of the bone to the reference coordinate system includes an iterative closest point plane to plane algorithm.

33. The robotic surgery system of claim 32, wherein the contour includes a plurality of first points and the computerized tomography scan of the bone includes a plurality of second points, and wherein the iterative closest point plane to plane algorithm includes:determining a minimum distance from each point of the plurality of first points to a tangent plane of each point of the plurality of second points, the minimum distance is measured from a normal direction of the tangent plane of each point of the plurality of second points; and aligning the plurality of first points and the plurality of second points by simultaneously translating and rotating the plurality of first points using a cost function.

34. The robotic surgery system of any of claims 32–33, wherein the contour includes a plurality of first points and the computerized tomography scan of the bone includes a plurality of second points, and wherein the iterative closest point plane to plane algorithm includes: determining a minimum distance from each point of the plurality of first points to a tangent plane of each point of the plurality of second points, the minimum distance is measured from an average a normal direction from each point of the plurality of first points and a normal direction of the tangent plane of each point of the plurality of second points; and aligning the plurality of first points and the plurality of second points by simultaneously translating and rotating the plurality of first points using a cost function.

Images