Graph convolutional network models for recommendation of orthopedic prosthesis

Abstract

For each node of a plurality of nodes of a graph, a computing system generates a feature vector for the node that initially includes feature data characterizing one or more aspects of a bone of a patient. The graph includes one or more edges. Each of the edges connects a respective pair of the nodes. The computing system applies a graph convolutional network (GCN) model to the graph and the feature vectors for the nodes to generate output. The computing system determines, based on the output, one more recommended prostheses for the patient or recommended values of one or more prosthesis parameters.

Description

GRAPH CONVOLUTIONAL NETWORK MODELS FOR RECOMMENDATION OF ORTHOPEDIC PROSTHESIS

[0001] This application claims priority to U.S. Provisional Patent Application 63 / 535,425, filed August 30, 2023, the entire content of which is incorporated herein by reference.BACKGROUND

[0002] Planning an orthopedic surgery may involve selecting an appropriate prosthesis to implant in a patient. Even among prostheses available to implant on a single bone, there may be prostheses of different shapes and sizes. Selecting an appropriate prosthesis for a patient may be an important factor in whether the orthopedic surgery has a successful outcome or whether complications arise. Because no two patients have exactly the same anatomy, it may be a challenge for surgeons, especially those with less experience, to select an appropriate prosthesis for a patient.SUMMARY

[0003] This disclosure describes techniques in which a computing system uses one or more graph convolutional network models to determine recommended orthopedic prostheses for individual patients. As described herein, the computing system may obtain feature data characterizing one or more aspects of a bone of a patient. A graph includes a plurality of sampling nodes and one or more edges. In some examples, the sampling nodes may correspond to different sampling positions, such as different cross-sections of the bone or different regions within the bone. For each of the sampling nodes in the graph, a feature vector for the sampling node initially includes the feature data characterizing the one or more aspects of the bone at the sampling position corresponding to the node. The computing system may apply a GCN model to the graph and the feature vectors for the nodes to generate an output. In some examples, the computing system determines a recommended prosthesis for the patient based on the output generated by the GCN model. In some examples, the computing system determines recommended values of one or more prosthesis parameters based on the output generated by the GCN model. The prosthesis parameters may specify one or more aspects of one or more prostheses, such as a size and radius of a glenosphere of a glenoid prosthesis, a stem size of a humeral prosthesis, a head sphere size of the humeral prosthesis, type of augment of the glenoid prosthesis, a baseplate type of the glenoid prosthesis, and so on. Using the GCN model in thisway may allow the computing system to take into account aspects of the bone at various sampling positions. Furthermore, the GCN model may include fewer parameters than other types of classifier neural networks, which may conserve computational resources, and may be easier to train.

[0004] In one example, this disclosure describes a method comprising: for each node of a plurality of nodes of a graph, generating, by a computing system, a feature vector for the node that initially includes feature data characterizing one or more aspects of a bone of a patient, wherein the graph includes one or more edges, each of the edges connecting a respective pair of the nodes; applying, by the computing system, a graph convolutional network (GCN) model to the graph and the feature vectors for the nodes to generate an output; and determining, by the computing system, a recommended prosthesis for the patient based on the output.

[0005] The details of various examples of the disclosure are set forth in the accompanying drawings and the description below. Various features, objects, and advantages will be apparent from the description, drawings, and claims.BRIEF DESCRIPTION OF DRAWINGS

[0006] FIG. 1 is a conceptual diagram illustrating an example computing system in which one or more techniques of this disclosure may be performed.

[0007] FIG. 2 is a flowchart illustrating an example operation of a computing system in accordance with one or more techniques of this disclosure.

[0008] FIGS. 3A, 3B, and 3C are conceptual diagrams illustrating example cross-sections of a bone in accordance with one or more techniques of this disclosure.

[0009] FIG. 4 is a conceptual diagram illustrating an example graph that may be used with a graph convolutional network (GCN) model in accordance with one or more techniques of this disclosure.

[0010] FIGS. 5A, 5B, 5C, and 5D are conceptual diagrams illustrating example cross-sections of a talus in accordance with one or more techniques of this disclosure.

[0011] FIG. 6 is a block diagram illustrating an example of a planning system that uses primary and secondary GCNs in accordance with one or more techniques of this disclosure.

[0012] FIG. 7 is a conceptual diagram illustrating an example process for recommending a talar prosthesis based in part on a recommended tibial prosthesis in accordance with one or more techniques of this disclosure.

[0013] FIG. 8 is a conceptual diagram illustrating an example graph that may be used with a GCN model in accordance with one or more techniques of this disclosure.

[0014] FIG. 9 is a flowchart illustrating an example operation of the computing system in which two interacting prostheses are recommended in accordance with one or more techniques of this disclosure.

[0015] FIG. 10 is a conceptual diagram illustrating an example 3 -dimensional lattice graph in accordance with one or more techniques of this disclosure.

[0016] FIG. 11 is a conceptual diagram illustrating an example mesh-based neural network model, in accordance with one or more techniques of this disclosure.

[0017] FIG. 12 is a flowchart illustrating an example operation of a plarming system, in accordance with one or more techniques of this disclosure.

[0018] FIG. 13 is a block diagram illustrating an example architecture for generating quantized face embeddings and generating a reconstructed anatomic model, in accordance with one or more techniques of this disclosure.

[0019] FIG. 14 is a flowchart illustrating an example operation of a prediction unit to generate information indicating one or more recommended prostheses for the patient or recommended values of one or more prosthesis parameters, in accordance with one or more techniques of this disclosure.

[0020] FIG. 15 is a flowchart illustrating an example training process for a machine learning model, in accordance with one or more techniques of this disclosure.

[0021] FIG. 16 is a conceptual diagram illustrating an example surgical planning user interface showing surgical suggestions for a reverse shoulder replacement surgery, in accordance with one or more techniques of this disclosure.DETAILED DESCRIPTION

[0022] A surgeon may select a prosthesis to implant in a patient from among a plurality of available prostheses. The available prostheses may have a variety of prosthesis parameters, such as a size and shape. Selecting an appropriate prosthesis from among the available prostheses may be an important factor in whether the surgery is ultimately successful. Selecting a prosthesis that is too large or too small may result in a limited range of motion, susceptibility to loosening, bone fracture, and other complications. Because patients have differently sized and shaped bones, selecting an appropriate prosthesis for implantation on a bone may not be a straightforward process. Often the ability to select an appropriate prosthesis comes as muchfrom experience as from objective knowledge. As a result, it is often difficult for less experienced surgeons to select an appropriate prosthesis. Computerized surgical planning systems have been developed to help users plan orthopedic surgeries, some of which provide automatic recommendations of prostheses to implant in a patient. Because bones often have complex shapes, it may be difficult for such computerized surgical planning systems to predict an appropriate prosthesis given measurements and deterministic rules, or even conventional machine learning models. For example, accurate prediction of prostheses may depend on information regarding relationships between non-adjacent portions of bones. A computerized surgical planning system that uses a conventional convolutional neural network (CNN) model for prediction of orthopedic prostheses would typically include a hierarchy of convolutional layers that convolve information from spatially adjacent areas in a 3D image. However, this approach may require a significant number of convolutional layers. In existing computerized surgical planning systems may use the output of the CNN to filter a set of available protheses. For instance, output of the CNN may be used to rank available prostheses and information regarding only a subset of the top-ranked available prostheses is initially presented for user (e.g., surgeon) review. Filtering the set of available prostheses may accelerate the planning process. Furthermore, the planning system may generate and display user interfaces in which 3D models of the prostheses are positioned are at suggested implantation positions with respect to 3D models of the patient’s anatomy. A user may use such user interfaces to confirm that a prosthesis is appropriate for a patient. However, generating such user interfaces may be computationally complex and time consuming. It is therefore undesirable to generate such user interfaces for all available prostheses.

[0023] This disclosure describes techniques that may address these issues. Specifically, this disclosure describes techniques in which a computing system uses one or more graph convolutional network (GCN) models to recommend an appropriate prosthesis for a patient to a surgeon. For example, the computing system may obtain feature data characterizing one or more aspects of a bone at a plurality of sampling positions, such as cross-sections of the bone or regions within the bone. A graph includes a plurality of nodes and one or more edges. The nodes include sampling nodes that correspond to different sampling positions. For each of the sampling nodes in the graph, a feature vector for the sampling node initially includes feature data characterizing one or more aspects of the bone at the sampling position corresponding to the sampling node. The computing system may apply a GCN model to the graph and the features vectors for the nodes to generate an output. The computing system may determine, based on the output, recommendations of one or more prosthesis for implantation in the patientor recommendations of values of one or more prosthesis parameters of one or more prostheses for implantation in the patient. The application of a GCN in this way allows the computing system to efficiently account for interrelationships among aspects of the bone at various sampling positions of the bone. Furthermore, the GCN model may include fewer parameters (e.g., weights, layers, etc.) than other types of machine learning models, which may increase computational efficiency and reduce storage requirements. Furthermore, in some examples, the computing system may use the output of the GCN model to filter a set of available prostheses. For at least the reasons set forth above, the techniques of this disclosure improve the efficiency and effectiveness of computing systems that filter sets of available prostheses. Filtering the set of available prostheses may enable the planning system to more quickly and efficiently present user interfaces in which 3D models of the prostheses are positioned are at suggested implantation positions with respect to 3D models of the patient’s anatomy.

[0024] FIG. 1 is a conceptual diagram illustrating an example computing system 100 in which one or more techniques of this disclosure may be performed. In the example of FIG. 1, computing system 100 includes one or more processors 102, a storage system 104, a communication interface 106, and a display 108. In other examples, computing system 100 may include more, fewer, or different components. The components of computing system 100 may be in one or more computing devices. For example, processors 102 may be in a single computing device or distributed among multiple computing devices of computing system 100, storage system 104 may be in a single computing device or distributed among multiple computing devices of computing system 100, and so on. In some examples, computing system 100 is a personal computer, a system of computing devices, one or more server devices, or a system comprising one or more other types of computing devices.

[0025] Processors 102 may be implemented in circuitry and include one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), hardware, or any combinations thereof. In general, processors 102 maybe implemented as fixed-function circuits, programmable circuits, or a combination thereof. Fixed-function circuits refer to circuits that provide particular functionality and are preset on the operations that can be performed. Programmable circuits refer to circuits that can be programmed to perform various tasks and provide flexible functionality in the operations that can be performed. For instance, programmable circuits may execute software or firmware that cause the programmable circuits to operate in the manner defined by instructions of the software or firmware. Fixed-function circuits may execute software instructions (e.g., to receive parameters or output parameters), but the types ofoperations that the fixed-function circuits perform are generally immutable. In some examples, one or more of the units may be distinct circuit blocks (fixed-function or programmable), and in some examples, the one or more units may be integrated circuits.

[0026] Processors 102 may include arithmetic logic units (ALUs), elementary function units (EFUs), digital circuits, analog circuits, and / or programmable cores, formed from programmable circuits. In examples where the operations of processors 102 are performed using software executed by the programmable circuits, storage system 104 may store the object code of the software that processors 102 receives and executes, or another memory within processors 102 (not shown) may store such instructions. Examples of the software include software designed for surgical planning. Processors 102 may perform the actions ascribed in this disclosure to computing system 100.

[0027] Storage system 104 may store various types of data used by processors 102. Storage system 104 may include any of a variety of memory devices, such as dynamic random access memory (DRAM), including synchronous DRAM (SDRAM), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or other types of memory devices. Examples of display 108 include a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or another type of display device.

[0028] Communication interface 106 allows computing system 100 to output data and instructions to and receive data and instructions from a medical imaging system, or other device via one or more communication links or networks. Communication interface 106 may include hardware circuitry that enables computing system 100 to communicate (e.g., wirelessly or using wires) with other computing systems and devices. Example networks may include various types of communication networks including one or more wide-area networks, such as the Internet, local area networks, and so on. The network may include wired and / or wireless communication links.

[0029] In the example of FIG. 1, storage system 104 stores medical image data 110, a plarming system 118, and a training system 120. In other examples, storage system 104 may store more, fewer, or different types of data or units. Moreover, the data and units illustrated in the example of FIG. 1 are provided for purposes of explanation and may not represent how data is actually stored or how software is actually implemented. Planning system 118 may comprise instructions that are executable by processors 102. For ease of explanation, this disclosure may describe planning system 118 as performing various actions when processors 102 execute instructions of planning system 118. Additionally, in the example of FIG. 1, planning system 118 includes feature data 112, graph data 114, and a GCN model 116.

[0030] Planning system 118 is a system that may help a user, such as a surgeon, plan an orthopedic surgery, either as part of a pre-operative planning process or an intra-operative planning process. As part of performing a process to help the user plan an orthopedic surgery, planning system 118 may present a series of user interfaces that help the user select an orthopedic prosthesis for implantation into a patient when there are multiple available orthopedic prostheses. For ease of explanation, this disclosure may refer to orthopedic prostheses as simply prostheses. The user interfaces may include user interfaces that display 3D models one or more of the patient’s bones, measurement data regarding the bones, and other information regarding the bones or other anatomical features of the patient. Additionally, the user interfaces may display information regarding a set of available prostheses for implantation in the patient. Planning system 118 may perform an automatic process of ranking and / or filtering the available prostheses. The user interfaces that display the information regarding the set of available prostheses may, at least initially, limit the information to one or more topranked prostheses and / or remaining prostheses after filtering.

[0031] Plarming system 118 may be applicable in helping a user select a prosthesis in the context of a variety of orthopedic surgeries. For example, in an ankle arthroplasty, plarming system 118 may help the surgeon select a tibial prosthesis and / or a talar prosthesis. In a shoulder arthroplasty, planning system 118 may help the surgeon select a glenoid prosthesis and / or a humeral prosthesis. In a knee arthroplasty, planning system 118 may help the surgeon select a femoral prosthesis and / or a tibial prosthesis. In a hip arthroplasty, planning system 118 may help the surgeon select an acetabular prosthesis and / or a femoral prosthesis. Other examples may apply with respect to other bones and / or joints.

[0032] Prostheses for implantation on a bone may be available in a variety of shapes and sizes. For example, there may be several differently sized tibial prostheses from which a surgeon may select when performing an ankle arthroplasty. A tibial prosthesis may be implanted at the distal end of a patient’s tibia. Differently sized tibial prostheses may be appropriate depending on the size of the patient’s tibia and other factors. In some examples, prostheses may be customized so that the prostheses are specific to a patient. In other words, the prostheses may be patientspecific prostheses.

[0033] As part of the process of recommending one or more prostheses for implantation in a patient, or values of one or more prosthesis parameters, planning system 118 obtain feature data 112 characterizing one or more aspects of one or more bones at a plurality of sampling positions. In some examples, the sampling positions include cross-sections of the one or more bones. Example aspects of a cross-section of a bone may include dimensions (e.g., width,length, height, etc.) of the bone at the cross-section, an average density of the bone (or cortical bone) at the cross-section, a minimum density of the bone at the cross-section, data indicating the presence of voids in the bone, and other types of data characterizing the bone at the crosssection. In some examples, the sampling positions include 2-dimensional or 3 -dimensional regions. In such examples, the aspects of a bone may include bone density within the region, the presence or absence of bone tissue within the region, data describing a texture (e.g., cortical, cancellous, etc.) of the bone, and so on. In some examples, the sampling positions may include a combination of cross-sections and regions.

[0034] In examples where the sampling positions include cross-sections of a bone, each of the cross-sections may be perpendicular to an axis through the bone. For instance, in an example where the bone is a tibia, each of the cross-sections may be perpendicular to a mechanical axis of the tibia. In an example where the bone is a talus, each of the cross-sections may be perpendicular to a mechanical axis of the talus. The mechanical axis of the tibia may be defined as a line connecting a center of the knee with a center of the talus. The cross-sections may be separated from each other by one or more amounts. For instance, the cross-sections may be separated from each other by 1 millimeter (mm) or another amount.

[0035] In some examples, planning system 118 may use medical image data 110 to obtain feature data 112. Medical image data 110 may include computed tomography (CT) data, dualenergy x-ray absorptiometry (DXA or DEXA) data, 3 -dimension bone modeling data, magnetic resonance imaging (MRI) data, and / or other types of medical imaging data. Planning system 118 may use medical image data 110 to generate a 3-dimensional model or multiple 2- dimensional models of one or more bones of the patient. For instance, planning system 118 may perform a segmentation process to determine boundaries of the one or more bones and, in some examples othertissues, within medical image data 110. Planning system 118 may use the determined boundaries to generate one or more models (e.g., 2- or 3-dimensional models) of the one or more bones (and, in some examples, other tissues). Planning system 118 may then use the one or more models to measure one or more aspects of the one or more bones. The feature data may indicate the measured aspects of the one or more bones. Planning system 118 may perform the segmentation process in one of a variety of ways. For example, planning system 118 may use a neural network-based process to perform the segmentation. In some examples, planning system 118 may additionally or alternatively obtain feature data directly from 2-dimensional medical images (e.g., x-rays, CT slices).

[0036] Graph data 114 defines a graph that includes a plurality of nodes and one or more edges. Each of the edges connects a respective pair of the nodes. In other words, each of the edges isa connection between exactly two of the nodes. In some examples, the edges connect each of the nodes. In other examples, the edges do not connect each of the nodes. The nodes include sampling nodes that correspond to different sampling positions, such as cross-sections or regions. Sampling nodes that correspond to cross-sections may be referred to herein as “crosssection nodes.” For each of the sampling nodes of the graph, a feature vector for the sampling node initially includes the feature data characterizing the one or more aspects of the bone at the sampling position corresponding to the node. For example, the feature vector corresponding to a specific cross-section may include elements indicating a length and width of the specific cross-section of the bone. In some examples, the nodes of the graph may include nodes in addition to the sampling nodes, such as nodes that correspond to a bone as a whole or nodes that correspond to neighboring bones, and so on.

[0037] Planning system 118 may apply GCN model 116 to a graph and the feature vectors for the nodes to generate an output. In some examples, planning system 118 may determine one or more recommended prostheses for the patient based on the output. In some examples, planning system 118 may determine recommended values of one or more prosthesis parameters based on the output. The prosthesis parameters may specify aspects of one or more prostheses. When planning system 118 applies GCN model 116 to the graph and the feature vectors, planning system 118 may perform one or more message passing rounds. When performing at least one or each of the message passing rounds, planning system 118 may, for at least one or for each respective node of the plurality of nodes, pass the feature vector for the respective node to each node of the plurality of nodes that is connected in the graph to the respective node. Thus, each node may receive the feature vectors of its neighboring nodes.

[0038] Furthermore, as part of performing a message passing round, planning system 118 may, for at least one of or each respective node of the plurality of nodes, modify the feature vector of the respective node based on the feature vector for the respective node and the feature vectors passed to the respective node. In other words, planning system 118 may aggregate the feature vector for the respective node and the feature vectors passed to the respective node. Planning system 118 may modify the feature vector of the respective node in one of a variety of ways. For example, planning system 118 may calculate each element of the modified feature vector as an average (e.g., a weighted average) of corresponding elements of the feature vectors. In another example, planning system 118 may calculate each element of the modified feature vector as a sum (or weighted sum) of corresponding elements of the feature vectors.

[0039] In some examples where planning system calculates a weighted average or weighted sum, planning system 118 may determine weights for feature vectors according to the following formula. if i * jotherwiseIn this example, for each edge of the graph linking a node i and a node j, the weight Cy applied to features of the feature vector may be equal to -7- if the edge connects different nodes. In this “i; equation d indicates the distance between the sampling positions corresponding to the nodes. For example, if node i corresponds to the first cross-section and node j corresponds to the fourth cross-section, the value of d may be equal to the 3. In some examples where the sampling positions include 2- or 3 -dimensional regions, the value d may indicate the physical Euclidean distance between centroids or closest boundaries of the regions. In other examples, the value d may indicate another type of distance measure. The weight C applied to features of the feature vector may be equal to 1 if node i and node j are the same node (i.e., the weight applied to features of a node’s own feature vector is 1). Thus, the feature vectors of nodes corresponding to sampling positions more distant from a sampling position corresponding to a current node have less impact on the modified features of the feature vector of the current node than the feature vectors of nodes corresponding to closer sampling positions.

[0040] In some examples, specific sampling positions may be more important than other sampling positions to determining one or more recommended prostheses or determining recommended values of one or more prosthesis parameters. For instance, in an example where the sampling positions correspond to cross-sections of a tibia, cross-sections closer to the distal end of the tibia may be more important for recommending a tibial prosthesis than cross-sections further from the distal end of the tibia. Accordingly, when aggregating the feature vector of a node with feature vectors of other nodes, planning system 118 may apply weights to the feature vectors that are based on the importance of the sampling positions corresponding to the nodes.

[0041] Furthermore, in some examples where the sampling positions include cross-sections, cross-sections that have bone voids or low bone mineral density may have less importance than cross-sections that do not have bone voids or have higher bone mineral density (BMD). Accordingly, when aggregating the feature vector of a node with feature vectors of other nodes,planning system 118 may apply weights to the feature vectors that are based on the presence of voids and / or BMD of the cross-sections corresponding to the nodes. In some examples, the weights can be based on a combination of factors, such as voids, BMD, distance between crosssections, relative importance, and so on.

[0042] After completion of the one or more message passing rounds, planning system 118 may, for each respective node of the nodes of the graph, apply a series of one or more graph convolutional layers (GCL) of GCN model 116 to the potentially modified feature vector of the respective node to obtain an embedding for the respective node. Each of the GCLs may include a set of artificial neurons. At least some of the artificial neurons of the GCL may receive, as input, some or all features of the feature vector of a node or a feature vector generated by a previous GCL. The artificial neurons of the GCL may output the result of a transfer function applied to a weighted sum of the inputs according to machine-learned weights .

[0043] In some examples, some or all of the GCLs increase the dimensional embedding space. Lor instance, in an example where each of the feature vectors includes two features (e.g., a feature corresponding to a width of a cross-section of the bone and a feature corresponding to a length of the cross-section of the bone), a first GCL may increase the dimensional embedding space from two dimensions to four dimensions. In this example, a second GCL may increase the dimensional embedding space from four dimensions to eight dimensions. In some examples, planning system 118 applies batch normalization to outputs of each of the GCLs. In some examples, planning system 118 applies an activation function, such as a Rectified Linear Unit (ReLU) activation function, sigmoid activation function, or other type of activation function, to outputs of the GCLs.

[0044] Planning system 118 may generate an output based on the embeddings. Lor example, planning system 118 may apply a pooling layer to the embeddings for the nodes. The pooling layer generates a first intermediate vector comprising first intermediate features corresponding to prostheses having different sizes. Lor example, the number of nodes in the graph may be equal to 10 and the final GCL layer may output an 8-dimensional embedding for each of the nodes. Thus, there may be 10x8 features after the final GCL layer. In this example, the pooling layer may reduce the number of features to 1x8. Lor instance, the pooling layer may determine an average (e.g., mean) of each of the corresponding features of each of the 10 nodes, resulting in a first intermediate vector that includes 8 features.

[0045] Additionally, planning system 118 may apply a fully connected layer to the first intermediate vector to generate a second intermediate vector. The fully connected layer may include a specific number of output neurons. In some examples, the specific number of outputneurons may correspond to the number of available prostheses. For instance, each of the output neurons may correspond to a difference size of a prothesis. Thus, the second intermediate vector may comprise second intermediate features corresponding to the prostheses having different sizes. In some examples, the output neurons correspond to different prosthesis parameters. The prosthesis parameters may specify aspects of one or more prostheses. For instance, in an example involving shoulder arthroplasty, the prosthesis parameters may include a size or radius of a glenosphere of a glenoid prosthesis, an eccentricity of the glenosphere, a baseplate type of the glenoid prosthesis, an augment type of the glenoid prosthesis, a stem size of a humeral prosthesis, a head sphere size of the humeral prosthesis, and so on. In an example involving a total ankle replacement surgery, the prosthesis parameters may include a length of a tibial tray, a size of a talar prosthesis, a size of an articulation element (e.g., a polyethylene articulating element coupled to the tibial tray), and so on.

[0046] Some or all of the output neurons of the fully connected layer may receive, as input, some or all of the features of the first intermediate vector. Some or all of the output neurons of the fully connected layer may calculate the result of a transfer function applied to a weighted sum of the inputs according to machine-learned weights. Furthermore, in some examples, planning system 118 may apply a softmax layer to the second intermediate vector to generate an output that includes a final vector. In some examples, the final vector includes final features corresponding to the prostheses with different sizes. The softmax layer may convert the second intermediate vector into a probability distribution of possible outcomes. The softmax layer may normalize the second intermediate vector to a probability distribution over the output classes (e.g., available prostheses). In this example, planning system 118 may determine a recommended prosthesis based on the final vector. For example, planning system 118 may determine that the recommended prosthesis is the prosthesis corresponding to a highest (or lowest) valued feature in the final vector. In some examples, the final vector includes features that correspond to different sets of mutually compatible prostheses and planning system 118 may determine a recommended set of mutually compatible prostheses as the set of mutually compatible prostheses corresponding to a highest (or lowest) valued feature in the final vector.

[0047] In some examples, the final vector includes features that correspond to different combinations of potential values of prosthesis parameters of one or more prostheses. For instance, a first feature of the final vector may correspond to a first radius of a glenosphere of a glenoid prosthesis, a first baseplate type of the glenoid prosthesis, and a first augment type of the glenoid prosthesis; a second feature of the final vector may correspond to the first radius of the glenosphere of the glenoid prosthesis, the first baseplate type of the glenoid prosthesis,and a second augment type of the glenoid prosthesis; a third feature of the final vector may correspond to the first radius of the glenosphere of the glenoid prosthesis, the second baseplate type of the glenoid prosthesis, and the second augment type of the glenoid prosthesis; and so on. An augment component of a glenoid prosthesis is positioned between a baseplate of the glenoid prosthesis and the scapula of the patient. Different augment types have different shapes . For instance, a first type of augment element has a bone-facing surface parallel to the baseplate, a second type of augment element has a bone-facing surface angled relative to the baseplate across a full diameter of the baseplate, a third type of augment element is partially angled relative to the baseplate and partially parallel to the baseplate, a fourth type of augment element has a bone-facing surface that has a patient-specific shape that conforms with a shape of the patient’s scapula. Different baseplate types may have different sizes. For instance, a first type of baseplate may have a first width and height to engage relatively small glenoid fossae, a second type of baseplate may have a second width and height to engage relatively larger glenoid fossae, a third type of baseplate may engage both a glenoid fossa and one or more other parts of the patient’s scapula, such as an acromion process or coracoid process. The third type of baseplate may be used in cases of complex deformity or trauma of the patient’s scapula and extra support is desirable.

[0048] In examples where the final vector includes features that correspond to different combinations of potential values of prosthesis parameters of two or more prostheses, the features are limited to potential combinations of values of the prosthesis parameters that are compatible among the prostheses. For example, a tibial tray having a first size may not be compatible with a talar prosthesis having a second size. Accordingly, in this example, the final vector does not include a feature that corresponds to the tibial tray having the first size and the talar prosthesis having the second size. Planning system may determine recommended values of the prosthesis parameters of the one or more prostheses as the combination of values of the prosthesis parameters corresponding to a highest (or lowest) valued feature in the final vector.

[0049] This process may be summarized by the following equation:Z = f X, A) = softmax(MLP (Readout(A ReLU (AXWW) IV(1)) )In the equation above, Z is a convolved signal matrix, A is a signal (e.g., a matrix of node feature vectors), IF0' are weights of the first GCL layer (i.e., the input-to-hidden layer), and are weights of the second GCL layer (i.e., the hidden-to-output layer). A may be a 1 1 normalized adjacency matrix defined as A = D~~AD~~, where A = A + INwith A being theadjacency matrix and IN being the identity matrix. may be equal to Z ' j=o Aij ■ Inthe equation above, ReLU (AXIVmay represent the first GCL layer and A ReLU (AXIVmay represent the second GCL layer. Readout represents a layer (i.e., a readout layer) that applies mean pooling of the node embeddings of the second GCL layer. MLP represents the fully connected layer before the final softmax layer.

[0050] In some examples, planning system 118 may automatically perform one or more checks to verify that the one or more recommended prostheses or recommended values of one or more prosthesis parameters of one or more prostheses are appropriate for the patient. For instance, after determining a recommended prosthesis based on the output of GCN model 116, planning system 118 may verify that the recommended prosthesis is appropriately seated on cortical bone. For instance, if the recommended prosthesis is not seated on cortical bone, the prosthesis may, in effect, sink into the cancellous bone. Thus, there must be enough overlap of the prosthesis and cortical bone to ensure stable positioning of the prosthesis.

[0051] After planning system 118 determines a recommended prosthesis, the surgeon may determine whether to accept the recommendation or choose another prosthesis. If the surgeon accepts the recommendation, the surgeon may surgically implant the recommended prosthesis. For instance, if planning system 118 recommends a specific glenoid prosthesis, the surgeon may surgically implant the specific glenoid prosthesis in the patient. In examples where planning system 118 determines recommended values of prosthesis parameters of a prosthesis, the surgeon may evaluate the recommended values of the prosthesis parameters, accept the recommended values, or elect different values of the prosthesis parameters. The surgeon may implant the prostheses according to the recommended or elected values of the prosthesis parameters.

[0052] Training system 120 may train GCN model 116. Training system 120 may be included in the same computing system or a different computing device or system from computing devices / systems that implement planning system 118. In some examples, training system 120 may perform one or more training epochs. In each of the training epochs, training system 120 may present one or more batches (e.g., full batches, mini-batches, or individual training examples) of training input examples to GCN model 116. For each training input example, training system 120 may compare the outputs produced by GCN model 116 to corresponding labeled training output examples using an error function (e.g., a loss function). In some examples, the error function is a cross-entropy error function, e.g., as defined in the following equation:In the equation above, In is the natural log function, J’L is the set of node indices that have labels, Y is the set of labels, and Z is the convolved signal matrix (i.e., the output of GCN model 116). After each batch, training system 120 may update weights of GCN model 116, e.g., by applying a backpropagation process. During each of the epochs, training system 120 may present the same training input examples to GCN model 116. A training dataset may include verification examples in addition to training input examples. Training system 120 may use the verification examples to verify GCN model 116 but does not use the verification examples to update the weights of GCN model 116.

[0053] Additionally, the system of FIG. 1 includes a fulfillment system 122. In some examples, fulfillment system 122 is configured to automatically (e.g., robotically) select a selected prosthesis from a rack of available prostheses for shipment. In some examples, fulfillment system 122 is configured to manufacture a selected, patient-specific prosthesis. For instance, fulfillment system 122 may include an additive manufacturing system (e.g., a 3D printer) to manufacture one or more components of a patient-specific prosthesis.

[0054] FIG. 2 is a flowchart illustrating an example operation of computing system 100 in accordance with one or more techniques of this disclosure. The flowcharts of this disclosure are presented as examples. In other examples, the flowcharts may include more, fewer, or different actions, or actions may be performed in different orders.

[0055] In the example of FIG. 2, planning system 118 may, for each node of a plurality of nodes of a graph, generate a feature vector for the node that initially includes feature data characterizing one or more aspects of a bone of a patient (200). The graph includes one or more edges, each of the edges connecting a respective pair of the nodes. The graph may be structured in one of a variety of ways. For instance, in some examples, the nodes may include crosssection nodes that correspond to a different cross-section in a plurality of cross-sections of the bone. In such examples, for each of the cross-section nodes in the graph, the feature data initially included in the feature vector for the cross-section node characterizes one or more aspects of the bone at the cross-section corresponding to the node. In some examples, the crosssections are perpendicular to a mechanical axis of the bone. In other examples, the crosssections may be perpendicular to another axis of the bone (e.g., an anatomic axis) or otherwise arranged with respect to the bone.

[0056] In some examples, the plurality of nodes may include nodes that have feature vectors that initially include feature data characterizing one or more aspects of two or more bones of the patient. For example, the plurality of nodes may include nodes that have feature vectors that initially include feature data characterizing one or more aspects of a tibia of the patient and nodes that have feature vectors that initially include feature data characterizing one or more aspects of a talus of the patient. In another example, the plurality of nodes may include nodes that have feature vectors that initially include feature data characterizing one or more aspects of a scapula of the patient and nodes that have feature vectors that initially include feature data characterizing one or more aspects of a humerus of the patient.

[0057] For each of the nodes in the graph, a feature vector for the node initially includes the feature data characterizing the one or more aspects of a bone at the cross-section corresponding to the node. In some examples, for each of the cross-sections of a bone, the one or more aspects of the bone at the cross-section of the bone include one or more measurements of dimensions of the bone at the cross-section of the bone. For instance, in an example (e.g., where the bone is a tibia), the one or more measurements of the dimensions of the bone at the cross-section of the bone include a medial width of the bone at the cross-section and an anterior length of the bone at the cross-section (e.g., as shown below with respect to FIGS. 3A-3C). In another example, the one or more measurements of the bone at the cross-section of the bone may include one or more of an anterior distance (i.e., a distance from a center of the cross-section to an anterior edge of the cross-section), a posterior distance (i.e., a distance from the center of the cross-section to a posterior edge of the cross-section), a lateral distance (i.e., a distance from the center of the cross-section to a lateral edge of the cross-section), and a medial distance (i.e., a distance from the center of the cross-section to a medial edge of the cross-section). The use of anterior distance, posterior distance, lateral distance, and medial distance as features for input to GCN model 116 may account for asymmetry of the bone and may therefore improve the prediction of the recommended prosthesis, e.g., by accounting for overhang or underhang of a prosthesis with respect to the bone in a way that may not be represented with only two distances, or by recommending an asymmetric prosthesis.

[0058] In some examples, such as examples where the bone is a talus, the feature vector for the node initially includes feature data characterizing a single measurement (e.g., a medio- lateral width) of the talus. In other words, in examples where the bone is a talus, the one or more measurements may include a measurement of a medio-lateral width of the talus. The medio-lateral width measurement may be the only measurement needed for the talus in examples where the width of the talus is sufficient to select a talar prosthesis from among aplurality of available talar prostheses. In other examples where one or more additional measurements (e.g., an anterior-posterior length) of the talus is needed to select a talar prosthesis from among a plurality of available talar prostheses, the feature vector for a node may include one or more additional measurements of the talus.

[0059] In some examples, planning system 118 obtains feature data 112 based on medical image data 110. For example, planning system 118 may obtain 2-dimensional or 3 -dimensional medical images, such as individual CT images, a 3D model built from 2-dimensional CT images, a 3-dimensional MRI image, other types of medical images. In examples where planning system 118 obtains 2-dimensional medical images, planning system 118 may select 2-dimensional medical images corresponding to cross-sections of the bone that planning system 118 will use to obtain the feature data. In some examples where planning system 118 obtains 2-dimensional medical images, planning system 118 may interpolate one or more of the cross-sections of the bone that planning system 118 will use to obtain the feature data based on two or more of the 2-dimensional medical images. In some examples, where planning system 118 obtains a 3 -dimensional medical image, planning system 118 may take slices of the 3-dimensional medical image to obtain the cross-sections of the bone. The cross-sections of the bone may be orthogonal to a mechanical axis of the bone. In some examples where planning system 118 obtains a 2D image or a 3D model, planning system 118 may interpolate pixel or voxel values for desired cross-section locations.

[0060] In some examples, the feature vectors of nodes may include information about a bone mineral density (BMD) of the bone at the corresponding cross-sections. Information about the BMD of the bone may be useful in determining a recommended prosthesis. For instance, larger prostheses may be more appropriate when BMD is relatively low in order to increase contact of the prosthesis with denser cortical bone and to otherwise spread the load of the prosthesis across more area. Furthermore, in some examples, voids may be present at one or more crosssections of the bone. Voids may be areas of zero or very low BMD within a bone. In general, it is desirable to have the prosthesis be in contact with solid bone and not bone voids. Thus, including voids and other BMD information in the feature vectors may help planning system 118 recommend a prosthesis.

[0061] In some examples, planning system 118 may preprocess feature data 112. For example, prior to using any of the features, planning system 118 may apply a normalization function to the features. Application of the normalization function may improve numerical stability and gradient descent convergence. The following is an example normalization function:x — . x = - oIn this normalization function, x is a normalized value of a feature, x is an original value of the feature, . is a mean of the original values of the features in a training dataset, and o is a standard deviation of the features in the training dataset.

[0062] In other examples, the graph is a 3 -dimensional lattice graph and, for each respective sampling position in a 3 -dimensional arrangement of sampling positions, the plurality of nodes includes a node corresponding to the respective sampling position. The 3-dimensional arrangement of sampling positions may include sampling positions corresponding to positions within one or more bones. For some or all of the nodes, the feature data initially included in the feature vector for the node characterizes one or more aspects of the one or more bones at the sampling position corresponding to the node.

[0063] Additionally, in the example of FIG. 2, planning system 118 applies GCN model 116 to the graph and feature vectors of the nodes to generate an output (202). As part of applying GCN model 116, planning system 118 may perform one or more message passing rounds (204). In a graph where each node is connected to each other node, it may not be necessary to perform more than one message passing round. In other examples, such as examples where the graph is a 3 -dimensional lattice graph, multiple rounds of message passing may be performed because not all of the nodes are connected to one another.

[0064] After completion of the one or more message passing rounds, planning system 118 may apply one or more GCLs to the feature vectors of nodes to generate embeddings for the nodes (206). After generating the embeddings for the nodes, planning system 118 may generate the output based on the embeddings for the nodes (208). Planning system 118 may determine, based on the output, one or more recommended prostheses or recommended values of one or more prosthesis parameters of one or more prostheses (210). For example, planning system 118 may apply one or more layers (e.g., a pooling layer, a fully connected layer, and a softmax layer) of GCN model 116 to the embeddings to obtain a final vector. In some examples, features in the final vector correspond to respective prostheses (or sets of mutually compatible prostheses) and each of the features in the final vector may indicate a level of confidence (i.e., a confidence score) that the recommended prosthesis (or set of mutually compatible prostheses) should be the prosthesis corresponding to the feature. In such examples, planning system 118 may determine that the recommended prosthesis is the prosthesis corresponding to a highest (or lowest) valued feature in the final vector. In some examples, features in the final vectorcorrespond to potential combinations of potential values of one or more prosthesis parameters of one or more prostheses. In such examples, planning system 118 may determine that the recommended values of the one or more prosthesis parameters of the one or more prostheses are the values of the one or more prosthesis parameters in the combination of values corresponding to a highest (or lowest) valued feature in the final vector.

[0065] In some examples, planning system 118 may use a regression process to determine, based on the output of GCN model 116, one or more recommended prostheses or recommended values of one or more prosthesis parameters. For instance, in such examples, the output of a final layer of GCN model 116 does not indicate levels of confidence that the recommended prosthesis should be a specific prosthesis. Rather, after the GCL layers and a readout layer, GCN model 116 may include a final multi-layer perceptron (MLP) that outputs one or more dimensions (e.g., a width, a length, height, size, size of glenosphere, radius of glenosphere, offset of glenosphere, stem size, head sphere size, etc.) of an appropriate prosthesis. Thus, GCN model 116 as a whole outputs the one or more dimensions of the appropriate prosthesis. In some examples, a first layer of the final MLP may include eight neurons (as noted above, in some examples, a second GCL layer has 8 dimensions) and a second layer of the final MLP may include two neurons. Planning system 118 may then determine one or more recommended prostheses from among a plurality of available prostheses based on the one or more dimensions (e.g., the width and length). For instance, planning system 118 may determine the one or more recommended prostheses as the prostheses that have smallest differences in dimensions from the dimensions output by GCN model 116, without violating one or more constraints. Such constraints may include avoiding situations in which an edge of the prosthesis extends beyond a corresponding edge of a bone. In this way, planning system 118 uses the output of GCN model 116 to perform a regression of the dimensions output by GCN model 116 to one of the available prostheses. In this way, planning system 118 may filter one or more recommended prostheses from the plurality of available prostheses. In other words, planning system 118 may eliminate, or deprioritize, prostheses that have larger differences in dimensions from the dimensions output by GCN model 116. Planning system 118 may display user interfaces that provide information about the recommended prostheses. Presenting information about multiple recommended prostheses may enable the surgeon compare the recommended prostheses and make an educated decision when selecting a prosthesis for implantation in the patient. In some examples, fulfillment system 112 manufactures a patient-specific prosthesis to have the dimensions output by GCN model 116. In examples where GCN model 116 outputs dimensionsof an appropriate prosthesis, the ground truths in the training data used for training GCN model 116 may be specify dimensions of the appropriate prostheses for different patients.

[0066] In some examples, the output of GCN model 116 (e.g., the output of a final MLP of GCN model 116) includes values of one or more prosthesis parameters of an appropriate prosthesis. For instance, in some such examples, a first layer of the final MLP may include eight neurons and a second layer of the final MLP may include a neuron for each of the prosthesis parameters. The prosthesis parameters of the appropriate prosthesis may characterize a shape and / or other characteristics of the appropriate prosthesis. In some examples, the prosthesis parameters may include a width and a length of the appropriate prosthesis. In another example, the prosthesis parameters may include a set of prosthesis parameters for two or more cross-sections of the bone. The set of prosthesis parameters for a cross-section of the bone may characterize a shape of the appropriate prosthesis at the crosssection. For instance, the set of prosthesis parameters for a cross-section of the bone may include a given number (e.g., 4, 8, 12, etc.) of prosthesis parameters, each of which indicates a distance from a center of the cross-section to an edge of the appropriate prosthesis. In an example where the number of prosthesis parameters for a cross-section is 8, the distances may correspond to lines radiating from the center of the cross-section separated by 45° angles.

[0067] In some examples where the output of GCN model 116 includes values of one or more prosthesis parameters of an appropriate prosthesis, planning system 118 may determine one or more recommended prosthesis from among a plurality of available prostheses based on the values of the one or more prosthesis parameters. For instance, planning system 118 may determine the recommended prosthesis as the prosthesis that has smallest difference in values of the prosthesis parameters from the values of the prosthesis parameters output by GCN model 116, without violating one or more constraints. In this way, plarming system 118 uses the output of GCN model 116 to perform a regression of the dimensions output by GCN model 116 to one of the available prostheses. In some examples, planning system 118 outputs information regarding the one or more recommended prostheses for display. Planning system 118 may receive an indication of user input to select one of the recommended prostheses or another one of the available prostheses. In some examples, fulfillment system 122 uses data generated by planning system 118 to select the selected prostheses from a rack of available prostheses for packaging and shipment. In some examples, fulfillment system 122 manufactures a patientspecific prosthesis based on the values of the prosthesis parameters output by GCN model 116. In examples where GCN model 116 outputs values of prosthesis parameters of an appropriateprosthesis, the ground truths in the training data used for training GCN model 116 may be specify prosthesis parameters of the appropriate prostheses for different patients.

[0068] FIGS. 3A, 3B, and 3C are conceptual diagrams illustrating example cross-sections 300A, 300B, 300C (collectively, “cross-sections 300”) of a model of a bone 302 in accordance with one or more techniques of this disclosure. Specifically, the examples of FIGS. 3A, 3B, and 3C show three different cross-sections of a tibia of a patient. Each of cross-sections 300 may correspond to a node of the graph. The model of bone 302 may represent a current (e.g., morbid) shape of bone 302.

[0069] For each of cross-sections 300A, 300B, 300C, planning system 118 may determine medial widths 304A, 304B, 304C (collectively, “medial widths 304”) and lateral widths 306A, 306B, 306C (collectively, “lateral widths 306”). Medial widths 304 and lateral widths 306 are defined by a distance between a projection point 308 and points on the cortical bone of the tibia. Projection point 308 may be defined as a projection of the tibia plafond onto crosssections 300.

[0070] Additionally, for each of cross-sections 300A, 300B, 300C, planning system 118 may determine anterior lengths 310A, 310B, 310C (collectively, “anterior lengths 310”) and posterior lengths 312A, 312B, 312C (collectively, “posterior lengths 312”). Anterior lengths 312 and posterior lengths 312 are defined by a distance between projection point 308 and points on the cortical bone of the tibia.

[0071] For each of cross-sections 300, planning system 118 may determine a first feature of the feature vector of the node corresponding to the cross-section by adding the medial width and the lateral width for the cross-section. Planning system 118 may determine a second feature of the feature vector of the node corresponding to the cross-section by adding the anterior width and the lateral width for the cross-section.

[0072] FIG. 4 is a conceptual diagram illustrating an example graph 400 that may be used with a GCN model in accordance with one or more techniques of this disclosure. In the example of FIG. 4, circles represent nodes of graph 400. Lines between the nodes represent edges of graph 400. In the example of FIG. 4, there are ten nodes, so graph 400 may be suitable for use with ten cross-sections of a bone, or ten cross-sections of a set of two or more bones.

[0073] FIGS. 5A, 5B, 5C, and 5D are conceptual diagrams illustrating example cross-sections 500 of a model of a talus 502 in accordance with one or more techniques of this disclosure. The model of talus 502 may represent a current (e.g., morbid) shape of talus 502. Each of crosssections 500 may correspond to a node of the graph. Furthermore, each of cross-sections 500 may be perpendicular to a mechanical axis of talus 502. In some examples, cross-sections 500are computed at depths of 5mm to 8mm following the mechanical axis of talus 502. Each of cross-sections 500 may be separated by 1mm.

[0074] For each of cross-sections 500A, 500B, 500C, planning system 118 may determine medial widths 504A, 50BA, 504C, 504D (collectively, “medial widths 504”) and lateral widths 506A, 506B, 506C, 506D (collectively, “lateral widths 506”). Medial widths 504 and lateral widths 506 are defined by a distance between a projection point 508 and points on the cortical bone of talus 502. In some examples, projection point 508 may be defined by determining a centroid of the two high-point landmarks of talus 502 and projecting the centroid onto crosssections 500 along the mechanical axis of talus 502.

[0075] Additionally, for each of cross-sections 500A, 500B, 500C, 500D planning system 118 may determine anterior lengths 510A, 510B, 510C, 510D (collectively, “anterior lengths 510”) and posterior lengths 512A, 512B, 512C, 512D (collectively, “posterior lengths 512”). Anterior lengths 512 and posterior lengths 512 are defined by a distance between projection point 508 and points on the cortical bone of the tibia.

[0076] For each of cross-sections 500, planning system 118 may determine a first feature of the feature vector of the node corresponding to the cross-section by adding the medial width and the lateral width for the cross-section. Planning system 118 may determine a second feature of the feature vector of the node corresponding to the cross-section by adding the anterior width and the lateral width for the cross-section.

[0077] FIG. 6 is a block diagram illustrating an example of planning system 118 that uses primary and secondary GCNs in accordance with one or more techniques of this disclosure. In some examples, recommendation of one prosthesis may be dependent on the recommendation of another prosthesis. For instance, recommendation of a talar prosthesis may be dependent on the recommendation of a tibial prosthesis. In other words, planning system 118 may determine which talar prosthesis to recommend based in part on which tibial prosthesis is recommended (or selected by a user). In another example, recommendation of a humeral prosthesis may be dependent on a glenoid prosthesis. For ease of explanation, if a first prosthesis needs to be selected / recommended before a second prosthesis, this disclosure may refer to the first prosthesis as a primary prosthesis and the second prosthesis as a secondary prosthesis. The primary prosthesis may be designed for implantation in a first bone and the secondary prosthesis may be designed for implantation in the second bone. There may be challenges associated with how a machine learning model may take this dependency among prothesis into account. For instance, a single machine-learning model that predicts both primary andsecondary protheses from a single set of input data may be difficult to train and / or may include an excessive number of weights.

[0078] In accordance with one or more techniques of this disclosure, planning system 118 may include primary feature data 612, primary graph data 614, and a primary GCN model 616. Additionally, planning system 118 may include secondary feature data 622, secondary graph data 624, and one or more secondary GCN models 626A through 626N (collectively, “GCN models 626”). Primary feature data 612, primary graph data 614, and primary GCN model 616 may be similar to feature data 112, graph data 114, and GCN model 116. Planning system 118 may use primary feature data 612, primary graph data 614, and primary GCN model 616 in the same way as feature data 112, graph data 114, and GCN model 116 to generate first output. Planning system 118 may determine a recommended primary prosthesis for implantation on a first bone based on the first output.

[0079] Secondary feature data 622 characterizes one or more aspects of a second bone at a second plurality of cross-sections of the second bone. Moreover, secondary graph data 624 may define a secondary graph that is structured in one of a variety of ways. For instance, in some examples, the secondary graph may include sampling nodes corresponding to cross-sections of the second bone (i.e., cross-section nodes that correspond to different cross-sections in a plurality of cross-sections of the second bone), e.g., in a manner similar to FIG. 4. In other examples, the secondary graph may include sampling nodes arranged in a 3-dimensional lattice. For each of the cross-section nodes in the secondary graph, the feature data initially included in the feature vector for the cross-section node characterizes one or more aspects of the second bone at the cross-section corresponding to the cross-section node.

[0080] For instance, in an example where the sampling nodes correspond to cross-sections of the second bone and the second bone is the talus, the aspects of the second bone may include the length and width of the talus at each of cross-sections 500 as shown in the example of FIG. 5. Secondary graph data 624 defines a second graph that includes a second plurality of nodes and one or more second edges. Each of the second plurality of nodes corresponds to a different cross-section in the cross-sections of the second bone. For each of nodes in the second graph, a feature vector for the node initially includes the feature data characterizing the one or more second aspects of the second bone at the cross-section of the second bone corresponding to the node. In an example where the second graph is a 3 -dimensional lattice, the sampling nodes of the second graph may correspond to regions within the second bone (or regions within an image of the second bone).

[0081] Each of secondary GCN models 626 may correspond to a different available primary prosthesis. For example, there may be a plurality of differently sized tibial protheses. Accordingly, there may be one secondary GCN model for each tibial prosthesis in the plurality of tibial prostheses. Thus, after determining a recommended primary prosthesis (or after receiving an indication of user input to select a primary prosthesis), planning system 118 may select one of a secondary GCN model from among GCN models 626 based on the first prosthesis. After selecting the second GCN model, planning system 118 may apply the secondary GCN model to the second graph and secondary feature data 622 to generate second output. In some examples, planning system 118 may determine a recommended second prosthesis for the patient based on the second output from among a set of second prostheses that are compatible with the primary prosthesis to which the secondary GCN model corresponds. In this way, the recommended second prosthesis is necessary compatible with the primary prosthesis. In some examples, planning system 118 determines, based on the second output, recommended values of one or more prosthesis parameters that specify aspects of a second prosthesis. For instance, the output of the secondary GCN model may include a vector that includes values of features corresponding to different combinations of potential values of prosthesis parameters of the second prosthesis. Planning system 118 may determine the recommended values of the prosthesis parameters of the second prosthesis based on the values in the output vector of the secondary GCN model. Planning system 118 may apply the secondary GCN model in the same way as described above for applying a GCN model. Planning system 118 may determine one or more recommended prostheses or determine recommended values of one or more prosthesis parameters in the same way as described above with respect to the GCN model.

[0082] FIG. 7 is a conceptual diagram illustrating an example process for recommending a talar prosthesis based in part on a recommended tibial prosthesis in accordance with one or more techniques of this disclosure. In the example of FIG. 7, there are eight available tibial prostheses 700, labeled “1”, “2*”, “3”, “3 Long”, “4”, “4 Long”, “5”, and “5 Long.”. There are 5 available talar prostheses 702A, 702B, 702C, 702D, and 702E (collectively, “talar prostheses 702”) that have different talar dome sizes. For any given one of tibial prosthesis 700, a user or planning system 118 may select from among two of talar prostheses 702. In other words, talar prosthesis 702A and talar prosthesis 702B are compatible with tibial prostheses 700A (i.e., tibial prosthesis 1 and 2*, talar prosthesis 702B and talar prosthesis 702C are compatible with tibial prostheses 700B (i.e., tibial prostheses 3 and 3 Long), talar prosthesis 702C and talar prosthesis 702D are compatible with tibial prostheses 700C (i.e., tibial prostheses 4 and 4Long), and talar prosthesis 702D and talar prosthesis 702E are compatible with tibial prostheses 700D (i.e., tibial prostheses 5 and 5 Long). Furthermore, differently sized polyethylene (poly) articulating components may be attachable to different tibial prostheses 700. For example, either a 1 or a 1+ poly articulating component may be used with tibial prostheses 700A and talar prosthesis 702A, poly articulating component 2 may be used with the combination of tibial prostheses 700A and talar prosthesis 702B, poly articulating component 2+ may be used with the combination of tibial prostheses 700B and talar prosthesis 702B, poly articulating component 3 may be used with the combination of tibial prostheses 700B and talar prosthesis 702C, poly articulating component 3+ may be used with the combination of tibial prostheses 700C and talar prosthesis 702C, poly articulating component 4 may be used with the combination of tibial prostheses 700C and talar prosthesis 702D, poly articulating component 4+ may be used with the combination of tibial prostheses 700D and talar prosthesis 702D, and poly articulating component 5 may be used with the combination of tibial prostheses 700D and talar prosthesis 702E.

[0083] In some examples, planning system 118 may apply a primary GCN model 616 to determine which of tibial prostheses 700 to recommend. Different secondary GCN models 626 may correspond to different sets of one or more tibial prostheses 700. For instance, if one of tibial prostheses 700A is the recommended tibial prosthesis, planning system 118 may use a first secondary GCN model to determine which of talar prostheses 702A or 702B (and, in some examples, poly articulating components) to recommend; if one of tibial prostheses 700B is the recommended tibial prosthesis, planning system 118 may use a second secondary GCN model to determine which of talar prostheses 702B or 702C (and, in some examples, poly articulating components) to recommend; and so on.

[0084] In some examples, there is a single GCN model (e.g., GCN model 116) and a final vector generated based on output of the GCN model includes values of features corresponding to different combinations of tibial prostheses, talar prostheses, and poly sizes. For example, the final vector may include a first feature corresponding to the tibial prosthesis 1, poly size 1, and talar prosthesis 702A; a second feature corresponding to the tibial prosthesis 1, poly size 1+, and talar prosthesis 702A; a third feature corresponding to the tibial prosthesis 1, poly size 2, and talar prosthesis 702B; a fourth feature corresponding to the tibial prosthesis 2*, poly size 1, and talar prosthesis 702A; a fifth feature corresponding to the tibial prosthesis 2*, poly size 1+, and talar prosthesis 702A; a sixth feature corresponding to the tibial prosthesis 2*, poly size 1, and talar prosthesis 702A; and so on. Since the final vector only includes features corresponding to mutually compatible prostheses, planning system 118 will not recommendsets of mutually incompatible prostheses or mutually incompatible sets of prosthesis parameters.

[0085] FIG. 8 is a conceptual diagram illustrating an example graph 800 that may be used with a GCN model in accordance with one or more techniques of this disclosure. Graph 800 is similar to graph 400 (FIG. 4) but includes fewer nodes. Planning system 118 may use graph 400 in instances where ten cross-sections are used and may use graph 800 in instances where four cross-sections are used. For example, planning system 118 may use ten cross-sections when recommending a tibial prosthesis and may use four cross-sections when recommending a corresponding, compatible talar prosthesis.

[0086] FIG. 9 is a flowchart illustrating an example operation of computing system 100 in which two interacting prostheses are recommended in accordance with one or more techniques of this disclosure. In the example of FIG. 9, planning system 118 may determine a first recommended prosthesis or recommended values of one or more prosthesis parameters of a first prosthesis, e.g., a prosthesis for implantation in a first bone, such as bone 302 (900). Planning system 118 may determine the first recommended prosthesis or the recommended values of the one or more prosthesis parameters of the first prosthesis in the same manner as described elsewhere, such as with respect to FIG. 2.

[0087] Additionally, planning system 118 may obtain secondary feature data 622 characterizing one or more second aspects of a second bone (e.g., talus 502) at a second plurality of cross-sections of the second bone (902). A second graph (e.g., graph 800) includes a second plurality of nodes and one or more second edges. The second graph may be defined by secondary graph data 624. Sampling nodes of the second graph may correspond to different sampling positions. For instance, one or more sampling nodes of the second plurality of nodes correspond to a different cross-section in the cross-sections (e.g., cross-sections 500 of the second bone. In some examples, one or more sampling nodes of the second plurality of nodes correspond to different regions. For each of the nodes in the second graph, a feature vector for the node initially includes the feature data (e.g., secondary feature data 622) characterizing the one or more second aspects of the second bone at the cross-section of the second bone corresponding to the node. For example, where the second bone is a talus, the one or more second aspects of the talus at each cross-section of the talus include a single measurement of the talus, such as a medio-lateral width of the talus at the cross-section. Thus, in this example, in the feature vector for a node may initially include the medio-lateral width of the talus at the cross-section of the talus corresponding to the node.

[0088] Planning system 118 may select a second GCN model from a plurality of secondary GCN models (e.g., secondary GCN models 626) based on the first prosthesis (904). For example, planning system 118 may use a predefined mapping from first prostheses to specific ones of the secondary GCN models. Planning system 118 may use output of the secondary GCN model mapped to a primary prosthesis to determine a recommendation of a secondary prosthesis that is compatible with the primary prosthesis. For example, there may be two secondary prostheses compatible with the primary prosthesis, and planning system 118 may use the output of the selected secondary GCN model to determine which of these two secondary prostheses to recommend for the patient. In some examples there may only be a single secondary GCN model and selection of the second GCN model may be omitted.

[0089] After selecting the second GCN model, planning system 118 may apply the second GCN model to the second graph and the feature vectors for the second plurality of nodes to generate a second output (906). As part of applying the second GCN model, planning system 118 may perform one or more message passing rounds (908). In a graph where each node is connected to each other node, it may not be necessary to perform more than one message passing round. Planning system 118 may then apply one or more GCLs to feature vectors of nodes to generate embeddings for the nodes (910). In some examples where the second bone is a talus and the feature vector for each node may be a 1 -dimensional vector that initially includes the medio-lateral width of the talus at the cross-section of the talus corresponding to the node. In such examples, a first GCL of the second GCN model may increase the dimensionality of the feature vectors from a 1 -dimensional embedding space to a 2- dimensional embedding space. A second GCL of the second GCN model may modify values within the 2-dimensional embedding space.

[0090] After generating the embeddings for the nodes, planning system 118 may generate the second output based on the embeddings for the nodes (912). For instance, planning system 118 may apply a pooling layer to the embeddings for the nodes. The pooling layer generates a first intermediate vector comprising first intermediate features corresponding to prostheses having different sizes. For example, the number of nodes in the graph may be equal to 4 and the final GCL layer may output a 2-dimensional embedding for each of the nodes. Thus, there may be 4x2 features after the final GCL layer. In this example, the pooling layer may reduce the number of features to 1x2. For instance, the pooling layer may determine a mean (average) of each of the corresponding features of each of the four nodes, resulting in a first intermediate vector that includes 2 features.

[0091] Additionally, planning system 118 may apply a fully connected layer to the first intermediate vector to generate a second intermediate vector. The fully connected layer may include a specific number of output neurons. The specific number of output neurons may correspond to the number of available secondary prostheses. For instance, the output neurons may include output neurons that correspond to a different sizes of a secondary prothesis. Thus, the second intermediate vector may comprise second intermediate features corresponding to the different available sizes of the secondary prosthesis. Similarly, in some examples, the output neurons may include output neurons that correspond to different values of one or more prosthesis parameters (e.g., size, radius, etc.) ofthe second prosthesis. Some or all ofthe output neurons of the fully connected layer may receive, as input, each of the features of the first intermediate vector. Each of the output neurons of the fully connected layer may calculate the result of a transfer function applied to a weighted sum of the inputs according to machine- learned weights.

[0092] Furthermore, in some examples, planning system 118 may apply a softmax layer to the second intermediate vector to generate the output, which includes a final vector. The final vector may include final features corresponding to the prostheses having different sizes. The softmax layer may convert the second intermediate vector into a probability distribution of possible outcomes. The softmax layer may normalize the second intermediate vector to a probability distribution over the output classes (e.g., available prostheses).

[0093] Plarming system 118 may determine, based on the second output, a second prosthesis to recommend for implantation in the patient or a recommended values of one or more prosthesis parameters of the secondary prosthesis (914). For instance, in an example where the second output includes a final vector that includes final features corresponding to the prostheses having different sizes, planning system 118 may determine that the recommended second prosthesis is the prosthesis corresponding to a highest (or lowest) valued feature in the final vector. In other examples, planning system 118 may use regression processes to determine the primary and secondary prostheses based on the final vectors. In such examples, the regression processes may be similar to that described above (e.g., with respect to FIG. 2). In other examples the second output may include dimensions and / or parameters of an appropriate prosthesis, and planning system 118 may determine the recommended prosthesis from among a plurality of available prostheses based on the dimensions and / or parameters. In some examples, planning system 118 may determine recommended values of the one or more prosthesis parameters of the secondary prosthesis in the same way as described elsewhere in this disclosure for determining recommended values of prosthesis parameters. In someexamples where the second output includes dimensions and / or parameters, planning system 118 may determine a custom, patient-specific prosthesis based on the dimensions and / or parameters.

[0094] FIG. 10 is a conceptual diagram illustrating an example 3 -dimensional lattice graph 1000 in accordance with one or more techniques of this disclosure. In the example of FIG. 10, dots correspond to nodes and lines correspond to edges. Furthermore, in the example of FIG. 10, lattice graph 1000 is a 4x4x4 lattice. In other examples, lattice graph 1000 may have other dimensions.

[0095] In the example of FIG. 10, as many as six edges may be connected to each of the nodes. Thus, for each of the nodes, the edges of lattice graph 1000 may include a set of edges that connect the node with up to six other nodes in the plurality of nodes. In other examples, there may be other limits on the number of edges that may be connected to a single node. For instance, for each of the nodes, the edges of the lattice graph may include a set of edges that connect the node with up to eight other nodes in the plurality of nodes. In another example, for each of the nodes, the edges of the lattice graph may include a set of edges that connect the node with up to twelve other nodes in the plurality of nodes.

[0096] Plarming system 118 may obtain a 3 -dimensional arrangement of sampling positions. In some examples, the 3 -dimensional arrangement of sampling positions is at least partially predefined. Thus, the sampling positions may include a set of sampling positions corresponding to predefined coordinates within 3-dimensional image data of the bone. In some examples, the 3 -dimensional arrangement of sampling positions may be a 3 -dimensional arrangement of equally spaced positions. For instance, each of the sampling positions may be separated by a distance of 2 mm, 5 mm, or another distance, in each of the width, length, and height dimensions. In some examples, the distances between the sampling positions may be different in one or more dimensions. For instance, the distances between the sampling points may be equal to a first value in the width and length dimensions and a second, different value in the height dimension. The sampling positions may be defined at coordinates relative to a specific anatomical landmark (e.g., most-distal point on tibia, centroid of talus, etc.) or a user-defined location.

[0097] In other examples, the 3-dimensional arrangement of sampling positions is not predefined. For instance, in some examples, planning system 118 may apply a segmentation process to image data (e.g., 2-dimensional or 3-dimensional image data) of a bone to identify a plurality of regions. Borders of the regions may correspond to anatomical boundaries. The regions may correspond to sampling positions in the plurality of sampling positions. Thus, thesampling positions may correspond to different regions. Two or more of the regions may have different sizes. In some examples, the segmentation process is an over-segmentation process that identifies regions even within anatomical structures. For instance, the segmentation process may identify a plurality of 3-dimensional regions within a tibia. In some examples, the over-segmentation process used by planning system 118 may be the so-called SLIC algorithm (Radhakrishna Achanta, Appu Shaji, Kevin Smith, Aurelien Lucchi, Pascal Fua, and Sabine Stisstrunk, SLIC Superpixels Compared to State-of-the-art Superpixel Methods, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 34, num. 11, p. 2274 - 2282, May 2012).

[0098] In some examples where planning system 118 applies a segmentation process, the number of regions identified by the segmentation process is not available prior to applying the segmentation process. Accordingly, planning system 118 may adaptively generate graph data 114 such that the number of nodes in the graph is determined based on (e.g., equal to, in proportion to, etc.) the number of regions identified by the segmentation process. In some examples, planning system 118 does not generate nodes for regions that are not of interest. For instance, in some examples, planning system 118 does not generate nodes for regions that correspond to soft tissue, open space, bones that are not of interest, and so on. Examples of bones that are not of interest may include the fibula, navicular bone, calcaneus, etc., for usecases involving total ankle replacement. Excluding nodes for regions that are not of interest may reduce graph size and, accordingly, may reduce processing time when applying GCN model 116. In other examples, the number of regions identified by the segmentation process may be predetermined.

[0099] Furthermore, for each of the regions, planning system 118 may generate edges that connect a node corresponding to the region to nodes corresponding to regions that border the region. In some examples, planning system 118 may limit the number of edges that connect to any specific node to a specific limit. In other words, planning system 118 may generate the edges such that no more than a specific number of nodes is allowed to connect to any given node. For example, the number of edges that are allowed to connect to a given node may be limited to six, eight, twelve, etc.

[0100] Planning system 118 may generate feature data for each of the nodes. For example, for each of the nodes, the feature data initially included in the feature vector for the node characterizes one or more aspects of the bone at the sampling position corresponding to the node.

[0101] In examples, where planning system 118 applies a segmentation process to 3- dimensional image data of the bone to identify a plurality of regions, planning system 118 may, for at least one of the regions, generate, based on data associated with one or more pixels of the 3 -dimensional image data within the region, the feature vector of the node corresponding to the sampling position corresponding to the region. In some examples, the one or more pixels of the 3 -dimensional image data include a plurality of pixels of the 3 -dimensional image data. Furthermore, in some examples, the region is a 3 -dimensional region and each of the coordinates of two or more of the plurality of pixels included in the region are different. In other words, the pixels in the 3-dimensional region do not necessarily form a straight line in one of the dimensions.

[0102] In some examples, the feature data initially included in the feature vector for a node may include a measure of bone density for the sampling position corresponding to the node. The measure of bone density may be an average of bone density measurements (e.g., measured in Hounsfield units), or other number based on the bone density measurements, within a region corresponding to the node. In other examples, the feature data of a node may include a measure of bone density corresponding to a single point (e.g., the centroid of a region corresponding to the node, a predefined position, etc.).

[0103] In some examples, for one or more of the nodes, the feature data initially included in the feature vector for the node may describe a local texture (e.g., cortical, cancellous, etc.) of the bone. The description of the local texture may be an average (or other statistic) of values indicating the local texture at points within a region corresponding to the node. In other examples, the description of the local texture may correspond to a single point (e.g., the centroid of a region corresponding to the node, a predefined position, etc.).

[0104] In some examples, for one or more of the nodes, the feature data initially included in the feature vector for the node includes a gray level co-occurrence matrix. A gray level cooccurrence matrix (GLCM) is based on gray level pixel values, which may also be referred to as intensity values. The intensity values may correspond to bone density measurements or other information. In general, planning system 118 may use a predefined spatial relationship to generate a GLCM. The predefined spatial relationship defines a relationship between a reference pixel and a neighbor pixel. Examples of spatial relationships may include the neighbor pixel being 1 pixel to the right of the reference pixel, the neighbor pixel being 3 pixels above the reference pixels, the neighbor pixel being 2 pixels above and 2 pixels left of the reference pixel, and so on. Planning system 118 may generate a matrix (i.e., a GLCM) having a size of (Range of Intensities x Range of Intensities), where each cell of the matrix is initializedto 0. For example, for an 8-bit single channel image, planning system 118 may generate a 256x256 matrix. Planning system 118 may traverse through the image and, for every unique ordered pair of intensity values found for the defined spatial relationship, planning system 118 may increment a cell of the matrix having coordinates matching the ordered pair. For example, if the spatial relationship is that the neighbor pixel is 1 pixel to the right of the reference pixel, and there are 4 instances in the image where a neighbor pixel and the reference pixel have values of 1 and 3, respectively, planning system 118 sets the value in the matrix at coordinates (1, 3) equal to 4. In some examples, the gray level co-occurrence matrix in the feature vector of a node may represent texture within a region corresponding to the node. In other words, the “image” upon which planning system 118 calculates the GLCM is the region corresponding to the node. In some examples, the image on which plarming system 118 generates the GLCM for a node may include the entire image or a subset of the image. Use of a lattice graph may be advantageous in some use cases because the feature vectors of nodes of the lattice graph may include types of information other than distances and dimensions, which may increase recommendation quality.

[0105] As discussed above, there are several challenges associated with existing computerized systems for recommending (e.g., filtering) prostheses. In addition to the GCN models discussed above, this disclosure also describes generative pre-trained transformer (GPT)-based techniques for recommending prostheses or recommended prosthesis parameters. The GPT- based techniques may allow the computing system to efficiently account for interrelationships among aspects of the bone at various sampling positions of the bone.

[0106] FIG. 11 is a block diagram illustrating example components of planning system 118, in accordance with one or more techniques of this disclosure. In the example of FIG. 11, the components of planning system 118 include machine learning model 1130, a prediction unit 1102, a training unit 1104, and a feature enhancement unit 1106. Machine learning model 1130 includes a feature encoder 1110, a quantization module 1112, a feature decoder 1114, a generative pre-trained transformer (GPT) 1134, and a codebook 1118. In other examples, planning system 118 may be implemented using more, fewer, or different components. For instance, training unit 1104 may be omitted in instances where ML model 1130 has already been trained. In some examples, one or more of the components of planning system 118 are implemented as software modules. Moreover, the components of FIG. 11 are provided as examples and planning system 118 may be implemented in other ways.

[0107] In general, prediction unit 1102 applies ML model 1130 to an input anatomic model to determine one or more recommended prostheses for the patient or recommended values of oneor more prosthesis parameters. Feature encoder 1110 of ML model 1130 generates a feature graph and input features based on an input anatomic model. The input anatomic model may be a lattice graph, such as lattice graph 1000. Feature encoder 1110 may then apply a graph convolutional encoder to the feature graph and input features to generate face embeddings. Quantization module 1112 uses codebook 1118 to generate quantized face embeddings based on the face embeddings. Prediction unit 1102 may sequence the quantized face embeddings to form a sequence of input tokens for GPT 1134. GPT 1134 generates a sequence of output tokens (e.g., predicted codebook indices) based on the sequence of input tokens (i.e., the sequence of quantized face embeddings). Prediction unit 1102 uses codebook 1118 to generate a sequence of quantized face embeddings based on the predicted codebook indices (e.g., the sequence of output tokes). Feature decoder 1114 uses the sequence of quantized face embeddings to generate a predicted model.

[0108] Training unit 1104 may train ML model 1130. Training unit 1104 may use a set of training examples 1132 to train ML model 1130. In one or more examples, training unit 1104 may be configured to train ML model 1130 using a plurality of training epochs. Feature enhancement unit 1106 may generate synthetic training examples to enhance training examples 1132. In some examples, ML model 1130 may be trained by a device or system other than computing system 100. In such examples, planning system 118 may receive ML model 1130, including GPT 1134.

[0109] FIG. 12 is a flowchart illustrating an example operation of planning system 118, in accordance with one or more techniques of this disclosure. In the example of FIG. 12, plarming system 118 may obtain an input anatomic model that comprises a first 3D mesh representing a surface of at least a first portion of an anatomic structure of a patient (1200). Plarming system 118 may generate input tokens based on the input anatomic model (1202). Planning system 118 may apply GPT 1134 to the input tokens to generate output tokens (1204). Planning system 118 may determine, based on the output tokens, one or more recommended prostheses for the patient or recommended values of one or more prosthesis parameters (1206).

[0110] FIG. 13 is a block diagram illustrating an example architecture for generating quantized face embeddings and generating a reconstructed anatomic model, in accordance with one or more techniques of this disclosure. In the example of FIG. 13, feature encoder 1110 includes a feature generator 1300 and a graph convolutional encoder 1302. Feature decoder 1114 includes a decoder 1342. The architecture of FIG. 13 may be used for training graph convolutional encoder 1302 and decoder 1342 separately from GPT 1134.

[0111] Feature encoder 1110 obtains an input anatomic model 1304 representing one or more anatomic structures (e.g., bones or soft tissue structures) of a patient. Feature generator 1300 generates a face graph and input features 1306 based on an input anatomic model 1304. In the face graph, each face corresponds to a node of a graph. The nodes corresponding to neighboring faces are connected in the graph by undirected edges. Each of the nodes has a feature vector. For each of the nodes, the feature vector of the node may include 9 coordinate values (i.e., 3 coordinate values for each of the 3 dimensions) for each of the vertices of the corresponding face, data defining a face normal vector, data defining angles between edges of the corresponding face, and data indicating an area of the corresponding face.

[0112] Graph convolutional encoder 1302 generates face embeddings 1308 based on the face graph and input features 1306. Graph convolutional encoder 1302 performs one or more rounds of message passing. During each round of message passing, graph convolutional encoder 1302 collects, for each node of the graph, feature vectors of nodes that neighbor the node. Graph convolutional encoder 1302 then aggregates the feature vector of the node with the feature vectors of the nodes that neighbor the node, thereby updating the feature vector of the node. Aggregating the feature vectors may involve determining averages of corresponding features in the feature vectors. After completing the one or more rounds of message passing, graph convolutional encoder 1302 may, for each node of the graph, use the updated feature vector of the node as input for a neural network 1307 that is trained to generate a face embedding for the face corresponding to the node. In this way, graph convolutional encoder 1302 may generate a face embedding for each face of the input anatomic model.

[0113] In some examples, neural network 1307 includes a series of graph convolutional layers, such as SAGE-Conv graph convolutional layers. In some such examples, a first layer of neural network 1307 takes 16 features as input: 9 features indicating point coordinates of a face, 3 features indicating a normal vector of the face, 3 features indicating angles at the vertices of the face, and feature indicating an area of the face.

[0114] Quantization module 1112 generates quantized face embeddings 1308 based on face embeddings 1308. In the example of FIG. 13, quantization module 1112 includes feature splitting unit 520 configured to split the face embeddings 1308 of each of the faces into a plurality of sub-vectors 1312. Each of sub-vectors 1312 may correspond to a vertex of a face. An aggregation unit 1322 of quantization module 1112 may then aggregate sub-vectors 1312 for corresponding vertices into aggregated sub-vectors 1323. For instance, each vertex of the model may have an index. Thus, two faces that share a vertex and the information about the faces may indicate the same index for the shared vertex. When aggregating the sub -vectors forcorresponding vertices, quantization module 1112 may calculate averages of the features in the sub-vectors for vertices having the same index.

[0115] For each of the aggregated sub-vectors 1323, a codebook lookup unit 1324 of quantization module 1112 may look up a codebook index in codebook 1118 for the aggregated sub-vector. Codebook 1118 includes a plurality of representative vectors. Each of the representative vectors may have the same number of elements as each of the aggregated subvectors. Each representative vector is associated with a unique codebook index. To look up a codebook index in codebook 1118 for an aggregated sub-vector, codebook lookup unit 1324 may determine which of the representative vectors is closest to the aggregated sub-vector. For instance, codebook lookup unit 1324 may treat the aggregated sub-vector and the representative vectors as specifying points in a multi-dimensional space, calculate a Euclidean distance between the aggregated sub-vector and the representative vectors, and determine which of the representative vectors has the lowest Euclidean distance. In this way, codebook lookup unit 1324 may output a plurality of codebook indexes 1326 for each face. Furthermore, in some examples, codebook lookup unit 1324 may apply a tier, residual quantization process that determines multiple codebook indexes for each aggregated sub-vector of a face. In some examples, the representative vectors in codebook 1118 are learned. Training unit 1104 may use a -means clustering algorithm to learn the representative vectors. In examples where training unit 1104 uses the -means clustering algorithm to learn the representative vectors, the representative vectors may be cluster-center vectors that indicate the centers of clusters.

[0116] A reshaper unit 1310 of quantization module 1112 uses codebook indexes 1326 to generate quantized face embeddings 1328. That is, reshaper unit 1310 receives a stack of codebook indexes 1326 for each face. For each of the faces, reshaper unit 1310 may reduce the stack of codebook indexes for the face to a single feature embedding per face through summation across representative vectors, and concatenation across vertices, thereby generating quantized face embeddings 1328 for the faces. For example, reshaper unit 1310 may use the following equation to determine quantized face embeddings 1328:In the equation above, Z indicates a vector containing quantized face embeddings 1328 for the faces, z1;... zNindicates the individual quantized face embeddings forfaces 1 through N, tf-v+dis a -th codebook index (i.e., a token) of vertex v of face i, e(tf’v+d) indicates therepresentative vector in codebook 1118 corresponding to the codebook index tf'v+d. and ©v=o indicates concatenation of features for vertices v = 0 through v = 2 of face i.

[0117] Feature decoder 1114 includes a sequence generator 1340 and a decoder 1342. Sequence generator 1340 generates a sequence of quantized face embeddings 1344 for the faces. The quantized face embeddings in the sequence of quantized face embeddings 1344 may be sequenced in the same way as face embeddings 1308.

[0118] Decoder 1342 generates a reconstructed anatomic model 1346 based on the sequence of quantized face embeddings 1344. For example, decoder 1342 outputs a set of 9 coordinate values for each face of reconstructed anatomic model 1346. Reconstructed anatomic model 1346 is a reconstructed version of input anatomic model 1304. Thus, reconstructed anatomic model 1346 does not include additional information, such as information indicating one or more recommended prostheses or recommended values of one or more prosthesis parameters. Decoder 1342 may be implemented as a ID ResNet decoder. In some examples, the ResNet decoder includes 34 layers, i.e., decoder 1342 may be implemented using a ResNet34 model. In other examples, other decoders may be used, such as ResNetl8 model, a ResNetlOl model, or another type of neural network model. In general, decoder 1342 may be able to reconstruct structures with more variability if decoder 1342 has more layers. In examples where ML model 1130 is used only for generating a limited number of structures, such as a scapula or tibia, an implementation of decoder 1342 with 18 or 34 layers may be sufficient, as opposed to a 101- layer ResNet 101 model. Using fewer layers may conserve computing resources (e.g., processing cycles and memory) if performance is similar. Inference time may also be reduced in models having fewer layers.

[0119] FIG. 14 is a flowchart illustrating an example operation of prediction unit 1102 to generate information indicating one or more recommended prostheses for the patient or recommended values of one or more prosthesis parameters, in accordance with one or more techniques of this disclosure. In the example of FIG. 14, feature encoder 1110 obtains an input anatomic model and generates face embeddings based on the input anatomic model, as described above with respect to FIG. 13 (1400). Additionally, planning system 118 applies graph convolutional encoder 1302 to the feature graph and input features to generate face embeddings (1402). Quantization model 1112 generates quantized face embeddings 1328 based on the face embeddings (1404).

[0120] Prediction unit 1102 may then generate a sequence of features (i.e., a sequence of input tokens) based on the quantized face embeddings (1406). The sequence of features may includea feature for each of the quantized face embeddings. The feature for a quantized face embedding may be based on the quantized face embedding and a learned discrete positional encoding for the quantized face embedding. The learned discrete positional encoding for the quantized face embedding provides information about the position of the quantized face embedding in the sequence of features and an index of each embedding within the quantized face embedding. In other words, for each embedding within the quantized face embedding, a position encoding value for the embedding is determined and then added to (i.e., summed with) the embedding. In some examples, a function for determining position encoding values is learned as part of a training process for GPT 1134. In some examples, functions for determining position encoding values include sinusoidal functions. Additionally, prediction unit 1102 may include a start embedding at the start of the sequence and an end embedding at the end of the sequence. The start embedding indicates the start ofthe sequence. The end embedding indicates the end of the sequence.

[0121] Prediction unit 1102 may autoregressively apply GPT 1134 to the sequence of features to generate output tokens (1408). The sequence of features may include the quantized face embeddings. The output tokens may include a sequence of codebook indices for faces of at least a portion of an anatomic structure or prosthesis. For example, prediction unit 1102 may apply GPT 1134 to the sequence of features to generate a set of predicted codebook indexes for a face. If the set of predicted codebook indices does not include a stop token, prediction unit 1102 may use codebook 1118 to determine a quantized face embedding for the face. Prediction unit 1102 may then update the sequence of quantized face embeddings to include the quantized face embedding. Prediction unit 1102 may use the updated sequence of quantized face embeddings as input to GPT 1134 to generate a next set of predicted codebook indices. This cycle may continue until the set of predicted codebook indices includes the stop token. GPT 1134 may be a GPT-2 medium architecture. That is, GPT transformer may have 24 multiheaded self-attention layers, 16 heads, 768 feature width and a context length of 4608, e.g., as described in Siddiqui et al., “MeshGPT: Generating Triangle Meshes with Decoder-Only Transformers”, arXiv:2311.15475vl [cs.CV] 27 Nov 2023.

[0122] In some examples, prediction unit 1102 may apply GPT 1134 to first input tokens and second input tokens to generate the output tokens. The first input tokens may be based on the input anatomic model (e.g., the first input tokens may include the quantized face embeddings). The second input tokens may include information describing the patient and not describing the first 3D mesh. The information describing the patient includes one or more of: a gender of the patient, an age of the patient, a glenoid type of a scapula of the patient, a pathology of theanatomic structure, or other information describing the patient. Inclusion of the second input tokens may improve the accuracy of predicted anatomic model 132.

[0123] Prediction unit 1102 may determine a sequence of quantized face embeddings based on the codebook indices (1410). Prediction unit 1102 may determine the quantized face embeddings in the same as way described above with respect to FIG. 13.

[0124] Prediction unit 1102 may then apply feature decoder 1114 to the sequence of quantized face embeddings to generate a predicted model (1412). The predicted model may include faces corresponding to the one or more anatomic structures and one or more prostheses. In some examples, prediction unit 1102 may update the sequence of face embeddings and predicted anatomic model 132 as prediction unit 1102 uses GPT 1134 to progressively predict additional sets of predicted codebook indices.

[0125] Prediction unit 1102 may determine, based on the predicted model, one or more recommended prostheses for the patient or recommended values of one or more prosthesis parameters (1414). For example, prediction unit 1102 may compare dimensions of a prosthesis in the predicted model to dimensions of available prostheses to determine which of the available prostheses is a closest match. Prediction unit 1102 may determine that a recommended prosthesis is the available prosthesis whose dimensions most closely match those of the prosthesis in the predicted model. In some examples, prediction unit 1102 may determine that recommended values of prosthesis parameters as the values of the corresponding prosthesis parameters of a prosthesis in the predicted model.

[0126] FIG. 15 is a flowchart illustrating an example training process for ML model 1130, in accordance with one or more techniques of this disclosure. As mentioned above, training unit 1104 may train ML model 1130. That is, training unit 1104 may train ML model 1130 to generate input tokens based on the input anatomic model, apply GPT 1134 of ML model 1130 to the input tokens to generate output tokens; and generate, based on the output tokens, a predicted anatomic model that includes a second 3D mesh representing surfaces of one or more prostheses.

[0127] In a first phase of training ML model 1130, training unit 1104 may train graph convolutional encoder 1302 and feature decoder 1114 and learn the representative vectors of codebook 1118 (1500). In a second phase of training ML model 1130, training unit 1104 may training unit 1104 may train GPT 1134 (1502).

[0128] In the first phase, training unit 1104 may perform one or more training epochs. In each training epoch, training unit 1104 may perform a series of training iterations. In each training iteration, training unit 1104 provides an input anatomic model from training examples 1132 asinput to feature encoder 1110. Feature encoder 1110, quantization module 1112, and feature decoder 1114 may generate a reconstructed anatomic model based on the input anatomic model. During the first phase, the input anatomic model may be a complete model of one or more anatomic structures with one or more implanted prostheses. After generating the reconstructed anatomic model, training unit 1104 may calculate a reconstruction loss value based on a comparison of the input anatomic model and the reconstructed anatomic model. For example, training unit 1104 may calculate the reconstruction loss value using a cross-entropy loss on discrete mesh coordinates. Training unit 1104 may use a backpropagation method to update parameters (e.g., weights) of feature encoder 1110 and feature decoder 1114 based on the reconstruction loss value.

[0129] In some examples, training unit 1104 updates the representative vectors of codebook 1118 as part of the training process. For example, training unit 1104 may add aggregated subvectors 1323 generated during a training iteration to a training set of aggregated sub-vectors. Training unit 1104 may assign a weight to each of the aggregated sub-vectors in the training set such that the aggregated sub-vectors have exponentially less weight the longer the aggregated sub-vectors have remained in the training set. In some examples, training unit 1104 uses the -means clustering process to update the representative vectors based on the aggregated sub-vectors in the training set.

[0130] In some examples, quantization module 1112 applies a tiered, residual vector quantization process. For instance, training unit 1104 may initialize a set of k first-tier representative vectors, assign generated aggregated sub-vectors (i.e., first-tier vectors) to their closest first-tier representative vectors, and then update the first-tier representative vectors based on an average position of the first-tier vectors assigned to the first-tier representative vectors. Training unit 1104 may repeat the assignment and updating steps until a termination condition is reached. For each of the first-tier vectors, training unit 1104 may calculate second- tier residual vectors indicating differences between the first-tier vectors and their first-tier representative vectors. Training unit 1104 then initializes a set of ki second-tier representative vectors, assigns the second-tier residual vectors to their closes second-tier representative vectors, updates the second-tier representative vectors based on an average position of the second-tier residual vectors assigned to the second-tier representative vectors, and repeats the assignment and updating steps until the termination condition is reached. Training unit 1104 may repeat this process for one or more tiers. Thus, when codebook lookup unit 1324 receives an aggregated sub-vector (i.e., a first-tier vector), codebook lookup unit 1324 determines a codebook index of a first-tier representative vector for the first-tier vector, determines a second-tier residual vector indicating a difference between the first-tier vector and the first-tier representative vector, determines a codebook index of a second-tier representative vector for the second-tier residual vector, determines a third-tier residual vector indicating a difference between the second-tier residual vector and the second-tier representative vector, determines a codebook index of a third -tier representative vector for the third-tier residual vector, and so on. In this way, for each aggregated sub-vector, codebook lookup unit 1324 identifies a stack of codebook indexes for the aggregated sub-vector.

[0131] In some examples, training unit 1104 applies a straight-through estimator (STE) technique to optimize vector quantization. During a training iteration, training unit 1104 quantizes face embeddings 1308 to generate quantized face embeddings 1328. For each of face embeddings 1308, training unit 1104 calculates a commitment loss value that characterizes the difference between the face embedding and the quantized face embedding. For instance, training unit 1104 may calculate the commitment loss value as:where z is the face embedding, z is the quantized face embedding, D is the number of dimensions of the face embedding, and sg is the stop gradient operation. After the reconstructed anatomical model 546 is generated for the training iteration, training unit 1104 calculates a general loss value as a sum of the reconstruction loss value (as described above) and the commitment loss values. Training unit 1104 uses the backpropagation method to update parameters of feature encoder 1110 and feature decoder 1114 based on the general loss value. However, the vector quantization function applied by quantization module 1112 cannot be directly learned by using backpropagation because the vector quantization function is not differentiable. Hence, during backpropagation, the vector quantization function is regarded as an identity function, allowing modification of parameters of neurons of neural network 1307 that generate face embeddings 1308. This alters the values in face embeddings 1308 that are fed into the vector quantization function applied by quantization module 1112. This ultimately has the effect of reducing the commitment loss values produced by the vector quantization function because face embeddings 1308 are generated so that aggregated sub-vectors generated from face embeddings 1308 are made closer to fixed representative vectors in codebook 1118. The STE technique may be used with the tiered residual vector quantization process as described above but the representative vectors at the various tiers are not updated.

[0132] In this way, feature encoder 1110, quantization module 1112, and feature decoder 1114 may learn to generate mesh models of anatomic structures and feature vectors that provide compact representations of the one or more anatomic structures.

[0133] As noted above, training unit 1104 may calculate a reconstruction loss value based on a comparison of an input anatomic model and a reconstructed anatomic model. Training unit 1104 may calculate the reconstruction loss value in one of a variety of ways. For example, training unit 1104 calculate a mesh regularization loss value (e.g., a chamfer loss, an Earth's Mover Distance loss (also known as Wasserstein loss), an edge regularization loss, a normal consistency loss, or a Laplacian loss. In some examples, training unit 1104 may calculate a curvature-weighted chamfer loss, e.g., as described in Bongratz et al., “Vox2Cortex: Fast Explicit Reconstruction of Cortical Surfaces from 3D MRI Scans with Geometric Deep Neural Networks, arxiv:2203.09446v2 [cs.CV] 18 Mar. 2022, where areas with high curvature are present, such as certain areas of a scapula.

[0134] In some examples, training unit 1104 calculates different multiple types of reconstruction loss values and calculates a weighted reconstruction loss value as a weighted sum of the reconstruction loss values. The losses may have a different weight according to the location of the faces or points. For example, training unit 1104 may use different weights for the reconstruction loss values for different areas of the anatomic structure. In some examples, training unit 1104 calculates a single type of reconstruction loss value for the whole anatomic structure as a sum of reconstruction losses for individual locations on the initial and reconstructed anatomic models. In some such examples, training unit 1104 may use different weights for different regions. In this way, more emphasis may be placed on accurate reconstruction of important areas. For example, greater weight can be used for areas of a scapula closer to the glenoid fossa of the scapula than areas further from the glenoid fossa.

[0135] As noted above, training unit 1104 may train GPT 1134 after completing training of feature encoder 1110 and feature decoder 1114 and learning codebook 1118. In other words, the parameters of feature encoder 1110, the parameters of feature decoder 1114, and learning codebook 1118 may be frozen while training GPT 1134. Training unit 1104 may train GPT 1134 using self-attention to autoregressively generate an ordered sequence of triangles which define a final mesh. Given a target token sequence (i.e., a target sequence of codebook indices) T = (t0, t1;tN) with tj = (f , t , ...' ) and s- is the corresponding predicted sequence element, training unit 1104 may train GPT 1134 with a loss function of:log p(s?c= t )where N is the number of tokens in the target token sequence, D is the number of sub-vectors per face embedding, |C| is the number of entries on codebook 1118, and p(s = t. is the probability of the predicted sequence element s being equal to the corresponding token in the target token sequence. Thus, after training feature encoder 1110 and learning codebook 1118, training unit 1104 may use feature encoder 1110 and quantization module 1112 to generate quantized face embeddings based on training examples 1132. As previously mentioned, training examples 1132 may include models of healthy or complete anatomical structures.

[0136] Training unit 1104 may use training examples 1132 to train feature encoder 1110, quantization model 212, feature decoder 1114, and GPT 1134. Training examples 1132 may include example anatomic models, such as anatomic models. Training examples 1132 may include models of anatomic structures of real patients. For example, in the first training phase, training unit 1104 may, for each first training example of a set of first training examples, apply feature encoder 1110 to an input anatomic model associated with the training example to generate face embeddings associated with the first training example. Quantization module 1112 may apply generate input tokens associated with the first training example based on the face embeddings associated with the first training example. Training unit 1104 may apply feature decoder 1114 to the input tokens associated with the first training example to generate a reconstructed anatomic model associated with the training example. Training unit 1104 may apply a first loss function to generate a loss value associated with the first training example based on the input anatomic model associated with the first training example and the reconstructed anatomic model associated with the first training example. Training unit 1104 may perform a first backpropagation process that modifies parameters of the feature encoder and the feature decoder based on the loss value associated with the first training example. In the second training phase following the first training phase, for each second training example of a set of second training examples, training unit 1104 may apply feature encoder 1110 to generate second face embeddings associated with the second training example. Training unit 1104 may apply quantization module 1112 to the face embeddings associated with the second training example to generate input tokens based on the second face embeddings. Training unit 1104 may apply GPT 1134 to generate output tokens associated with the second training example based on the input tokens associated with the second training example. Training unit 1104 may apply feature decoder 1114 to the input tokens associated with the second trainingexample to generate a predicted anatomic model associated with the training example. Training unit 1104 may apply a second loss function to generate a loss value associated with the second training example. Training unit 1104 may perform a second backpropagation process that modifies parameters of the GPT based on the loss value associated with the second training example.

[0137] In some examples, feature enhancement unit 1106 performs data augmentation to improve the training of machine learning model 1130. In some examples, feature enhancement unit 1106 performs scaling, rotation, mirroring (e.g., generating a synthetic left scapula model from an actual right scapula model), unit sphere normalization, jitter, multiresolution remeshing, or other techniques to generates synthetic training examples. Training unit 1104 may use the synthetic training examples along with original training examples 1132 fortraining of machine learning model 1130.

[0138] In some examples, training unit 1104 may perform a mesh decimation process on the anatomic models of training examples 1132. For example, training unit 1104 may use a Quadric Error Metrics (QEM) mesh simplification algorithm to reduce the number of faces in the anatomic models. Reducing the number of faces may reduce training time. In some examples, training unit 1104 may reduce the number of faces in some areas of the anatomic models and not other areas of the anatomic models. This may reduce a potential bottleneck of processing circuitry capabilities and / or may reduce inference time of the mesh completion algorithm when processing a mesh derived from a high-resolution CT scan.

[0139] FIG. 16 is a conceptual diagram illustrating an example surgical planning user interface 1600 showing surgical suggestions for a reverse shoulder replacement surgery, in accordance with one or more techniques of this disclosure. Planning system 118 may generate user interface 1600 for display (e.g., on display 108). In some examples, planning system 118 may generate user interface 1600 after determining one or more recommended prostheses from among a plurality of available prostheses.

[0140] In the example of FIG. 16, user interface 1600 shows surgical suggestions 1602A, 1602B (collectively, “surgical suggestions 1602”) for a reverse shoulder replacement surgery. Each of surgical suggestions 1602 may correspond to a different recommended prosthesis. In the example of FIG. 16, each of surgical suggestions 1602 indicates atype of aglenoid implant, a diameter of the glenoid implant, a glenosphere diameter and type (e.g., centered, eccentric, tilted, etc.) of the glenoid implant, a neck shaft angle of a corresponding humerus implant, a version of the glenoid implant, a seating percentage of the glenoid implant, and a peg depth of the glenoid implant. A user may select one of surgical suggestions 602. In the example of FIG.16, the black background is used to indicate that surgical suggestion 602A is the selected surgical suggestion. In other examples, user interface 1600 may relate to other types of prostheses.

[0141] Furthermore, user interface 1600 includes superior view 1606, frontal view 1608, and model 1610. Superior view 1606 shows an x-ray image of the shoulder of a patient from a superior perspective (i.e., looking in the inferior direction from a superior position). Superior view 1606 shows an outline 1612 of a glenoid implant of the type indicated by the selected surgical suggestion at the positions indicated by the selected surgical suggestion. Frontal view 1608 shows an x-ray image of the shoulder of the patient from an anterior perspective (i.e., looking in the posterior direction from an anterior position). Frontal view 1608 shows an outline 1614 of the glenoid implant of the type indicated by the surgical suggestion at the positions indicated by the selected surgical suggestion. Model 1610 shows a 3D model of the patient’s scapula with a phantom image of a glenoid implant. Generation of views 1608 and models 1610 may be computationally complex. Accordingly, it would be undesirable and timeconsuming to generate views 1608 and models 1610 for all available prostheses. Rather, planning system 118 may generate views 1608 and models 1610 for only the one or more recommended prostheses. Determining the one or more recommended prostheses as described in this disclosure may avoid this issue, improving the ease of use of planning system 118. Planning system 118 may generate views 1608 and models 1610 for other ones of the available prostheses on demand.

[0142] The following is a non-limiting list of clauses in accordance with one or more techniques of this disclosure.

[0143] Clause 1A. A method comprising: for each node of a plurality of nodes of a graph, generating, by a computing system, a feature vector for the node that initially includes feature data characterizing one or more aspects of a bone of a patient, wherein the graph includes one or more edges, each of the edges connecting a respective pair of the nodes; applying, by the computing system, a graph convolutional network (GCN) model to the graph and the feature vectors for the nodes to generate an output; and determining, by the computing system, a recommended prosthesis for the patient based on the output.

[0144] Clause 2A. The method of clause 1A, wherein: the plurality of nodes includes cross-section nodes corresponding to different cross-sections in a plurality of cross-sections of the bone, and for each of the cross-section nodes in the graph, the feature data initially included in the feature vector for the cross-section node characterizes one or more aspects of the bone at the cross-section corresponding to the cross-section node.

[0145] Clause 3A. The method of clause 2A, wherein, for each of the cross-sections of the bone, the one or more aspects of the bone at the cross-section of the bone include one or more measurements of dimensions of the bone at the cross-section of the bone.

[0146] Clause 4A. The method of clause 3A, wherein the bone is a tibia and the one or more measurements include measurements of the dimensions of the bone at the cross-section of the bone include a medial width of the tibia at the cross-section and an anterior length of the tibia at the cross-section.

[0147] Clause 5A. The method of clause 3A, wherein the bone is a talus and the one or more measurements include a measurement of a medio-lateral width of the talus.

[0148] Clause 6A. The method of any of clauses 2A-5A, wherein the cross-sections are perpendicular to a mechanical axis of the bone.

[0149] Clause 7A. The method of any of clauses 1A-6A, wherein: the GCN model is a first GCN model, the bone is a first bone, the feature data is first feature data, the one or more aspects are one or more first aspects, the graph is a first graph, the plurality of nodes is a first plurality of nodes, the recommended prosthesis is a first prosthesis, and the output is a first output, and the method further comprises: generating, by the computing system, for each node of a second plurality of nodes of a second graph, a feature vector for the node that initially includes secondary feature data characterizing one or more second aspects of a second bone of the patient, wherein the second graph include one or more second edges, each of the second edges connecting a respective pair of the nodes in the second graph, applying, by the computing system, a second GCN model to the second graph and the feature vectors for the second plurality of nodes to generate second output; and determining, by the computing system, a recommended second prosthesis for the patient based on the second output.

[0150] Clause 8A. The method of clause 7A, further comprising selecting, by the computing system, based on the first prosthesis, the second GCN model from among a plurality of secondary GCN models.

[0151] Clause 9A. The method of any of clauses 7A-8A, wherein: the second plurality of nodes includes cross-section nodes that correspond to a different cross-section in a plurality of cross-sections of the second bone, and for each of the cross-section nodes in the second graph, the feature data initially included in the feature vector for the cross-section node characterizes one or more aspects of the second bone at the cross-section corresponding to the cross-section node.

[0152] Clause 10A. The method of any of clauses 7A-9A, wherein the first prosthesis is designed for implantation in the first bone and the second prosthesis is designed for implantation in the second bone.

[0153] Clause 11A. The method of any of clauses 7A-10A, wherein the first bone is a tibia and the second bone is a talus.

[0154] Clause 12A. The method of any of clauses 1A-10A, wherein applying the GCN model comprises: performing, by the computing system, one or more message passing rounds, wherein performing a message passing round comprises: for each respective node of the plurality of nodes, passing, by the computing system, the feature vector for the respective node to each node of the plurality of nodes that is connected in the graph to the respective node; and for each respective node of the plurality of nodes, modifying, by the computing system, the feature vector of the respective node based on the feature vector for the respective node and the feature vectors passed to the respective node; and after completion of the one or more message passing rounds, for each respective node of the plurality of nodes, applying, by the computing system, a series of one or more graph convolutional layers (GCLs) of the GCN model to the feature vector of the respective node to obtain an embedding for the respective node; and generating, by the computing system, the output based on the embeddings for the nodes.

[0155] Clause 13A. The method of clause 12A, wherein generating the output based on the embeddings for the nodes comprises: applying, by the computing system, a pooling layer to the embeddings for the nodes, wherein the pooling layer generates a first intermediate vector comprising first intermediate features corresponding to prostheses having different sizes; applying, by the computing system, a fully connected layer to the first intermediate vector to generate a second intermediate vector, wherein the second intermediate vector comprises second intermediate features corresponding to the prostheses having different sizes; and applying, by the computing system, a softmax layer to the second intermediate vector to generate the output, wherein the output includes a final vector comprising final features corresponding to the prostheses having different sizes.

[0156] Clause 14A. The method of clause 12A, wherein: the nodes include cross-section nodes that correspond to different cross-sections in a plurality of cross-sections of the bone, and for each of the cross-section nodes in the graph: the feature data initially included in the feature vector for the cross-section node characterizes one or more aspects of the bone at the cross-section corresponding to the cross-section node, and modifying the feature vector of the cross-section node comprises, modifying, by the computing system, the feature vector of thecross-section node based on a weighted average of features in the feature vector of the crosssection node and features in the feature vectors passed to the cross-section node, wherein weights used in the weighted average are based on importance of the cross-section corresponding to the cross-section node and the cross-sections corresponding to adjacent nodes, the adjacent nodes being connected to the respective node in the graph data.

[0157] Clause 15A. The method of any of clauses 1A-14A, wherein: the graph is a 3- dimensional lattice graph, for each respective sampling position in a 3 -dimensional arrangement of sampling positions, the plurality of nodes includes a node corresponding to the respective sampling position, the 3 -dimensional arrangement of sampling positions includes sampling positions corresponding to positions within the bone, and for each of the nodes, the feature data initially included in the feature vector for the node characterizes one or more aspects of the bone at the sampling position corresponding to the node.

[0158] Clause 16A. The method of clause 15 A, wherein, for one or more of the nodes, the feature data initially included in the feature vector for the node includes one or more of: a measure of bone density for the sampling position corresponding to the node, data describing a local texture of the bone, or a gray level co-occurrence matrix.

[0159] Clause 17A. The method of any of clauses 15A-16A, wherein: for each of the nodes, the plurality of edges includes a set of edges that connect the node with up to six other nodes in the plurality of nodes, for each of the nodes, the plurality of edges includes a set of edges that connect the node with up to eight other nodes in the plurality of nodes, or for each of the nodes, the plurality of edges includes a set of edges that connect the node with up to twelve other nodes in the plurality of nodes.

[0160] Clause 18A. The method of any of clauses 15A-17A, wherein the sampling positions include a set of sampling positions corresponding to predefined coordinates within 3 -dimensional image data of the bone.

[0161] Clause 19A. The method of any of clauses 15A-18A, further comprising: applying, by the computing system, a segmentation process to image data of the bone to identify a plurality of regions, wherein the regions correspond to sampling positions in the plurality of sampling positions; and for at least one of the regions, generating, by the computing system, based on data associated with one or more pixels of the image data within the region, the feature vector of the node corresponding to the sampling position corresponding to the region.

[0162] Clause 20A. The method of clause 19A, wherein the one or more pixels of the image data include a plurality of pixels of the image data.

[0163] Clause 21 A. The method of clause 20A, wherein the region is a 3-dimensional region and each of the coordinates of two or more of the plurality of pixels included in the region are different.

[0164] Clause 22A. The method of any of clauses 1A-12A or 14A-21A, wherein: the output includes one or more dimensions of an appropriate prosthesis, and determining the recommended prosthesis for the patient comprises determining, by the computing system, the recommended prosthesis for the patient from among a plurality of available prostheses based on the one or more dimensions of the appropriate prosthesis.

[0165] Clause 23A. The method of any of clauses 1A-12A or 14A-21A, wherein: the output includes one or more parameters of an appropriate prosthesis, and determining the recommended prosthesis for the patient comprises determining, by the computing system, the recommended prosthesis for the patient based on the one or more parameters of the appropriate prosthesis.

[0166] Clause 24A. A computing system comprising: a storage system; and one or more processors implemented in circuitry and configured to perform the methods of any of clauses 1-23.

[0167] Clause 25A. A computing system comprising means for performing the methods of any of clauses 1A-23A.

[0168] Clause 26A. Non-transitory computer-readable storage media having instructions stored thereon that, when executed, cause a computing system to perform the methods of any of clauses 1-23 A.

[0169] Clause IB. A method comprising: for each node of a plurality of nodes of a graph, generating, by one or more processors implemented in circuitry, a feature vector for the node that initially includes feature data characterizing one or more aspects of a bone of a patient, wherein the graph includes one or more edges, each of the edges connecting a respective pair of the nodes; applying, by the one or more processors, a graph convolutional network (GCN) model to the graph and the feature vectors for the nodes to generate an output; and determining, by the one or more processors, based on the output, one or more recommended prostheses for the patient or recommended values of one or more prosthesis parameters.

[0170] Clause 2B. The method of clause IB, wherein: the plurality of nodes includes cross-section nodes corresponding to different cross-sections in a plurality of cross-sections of the bone, and for each of the cross-section nodes in the graph, the feature data initially included in the feature vector for the cross-section node characterizes one or more aspects of the bone at the cross-section corresponding to the cross-section node.

[0171] Clause 3B. The method of clause 2B, wherein, for each of the cross-sections of the bone, the one or more aspects of the bone at the cross-section of the bone include one or more measurements of dimensions of the bone at the cross-section of the bone.

[0172] Clause 4B. The method of clause 3B, wherein the bone is a tibia and the one or more measurements include measurements of the dimensions of the bone at the cross-section of the bone include a medial width of the tibia at the cross-section and an anterior length of the tibia at the cross-section.

[0173] Clause 5B. The method of clause 3B, wherein the bone is a talus and the one or more measurements include a measurement of a medio-lateral width of the talus.

[0174] Clause 6B. The method of any of clauses 2B-5B, wherein the cross-sections are perpendicular to a mechanical axis of the bone.

[0175] Clause 7B. The method of any of clauses 1B-6B, wherein: the GCN model is a first GCN model, the bone is a first bone, the feature data is first feature data, the one or more aspects are one or more first aspects, the graph is a first graph, the plurality of nodes is a first plurality of nodes, the prosthesis is a first prosthesis, and the output is a first output, and the method further comprises: generating, by the computing system, for each node of a second plurality of nodes of a second graph, a feature vector for the node that initially includes secondary feature data characterizing one or more second aspects of a second bone of the patient, wherein the second graph include one or more second edges, each of the second edges connecting a respective pair of the nodes in the second graph, applying, by the computing system, a second GCN model to the second graph and the feature vectors for the second plurality of nodes to generate second output; and determining, by the computing system, based on the second output, a second prosthesis to recommend for implantation in the patient or recommended values of one or more second prosthesis parameters, wherein the one or more second prosthesis parameters specify one or more aspects of a secondary prosthesis.

[0176] Clause 8B. The method of clause 7B, further comprising selecting, by the computing system, based on the first prosthesis, the second GCN model from among a plurality of secondary GCN models.

[0177] Clause 9B. The method of any of clauses 7B-8B, wherein: the second plurality of nodes includes cross-section nodes that correspond to a different cross-section in a plurality of cross-sections of the second bone, and for each of the cross-section nodes in the second graph, the feature data initially included in the feature vector for the cross-section node characterizes one or more aspects of the second bone at the cross-section corresponding to the cross-section node.

[0178] Clause 10B. The method of any of clauses 7B-9B, wherein the first prosthesis is designed for implantation in the first bone and the second prosthesis is designed for implantation in the second bone.

[0179] Clause 1 IB. The method of any of clauses 7B-10B, wherein the first bone is a tibia and the second bone is a talus.

[0180] Clause 12B. The method of any of clauses 1B-10B, wherein applying the GCN model comprises: performing, by the computing system, one or more message passing rounds, wherein performing a message passing round comprises: for each respective node of the plurality of nodes, passing, by the computing system, the feature vector for the respective node to each node of the plurality of nodes that is connected in the graph to the respective node; and for each respective node of the plurality of nodes, modifying, by the computing system, the feature vector of the respective node based on the feature vector for the respective node and the feature vectors passed to the respective node; and after completion of the one or more message passing rounds, for each respective node of the plurality of nodes, applying, by the computing system, a series of one or more graph convolutional layers (GCLs) of the GCN model to the feature vector of the respective node to obtain an embedding for the respective node; and generating, by the computing system, the output based on the embeddings for the nodes.

[0181] Clause 13B. The method of clause 12B, wherein generating the output based on the embeddings for the nodes comprises: applying, by the computing system, a pooling layer to the embeddings for the nodes, wherein the pooling layer generates a first intermediate vector comprising first intermediate features corresponding to prostheses having different sizes; applying, by the computing system, a fully connected layer to the first intermediate vector to generate a second intermediate vector, wherein the second intermediate vector comprises second intermediate features corresponding to the prostheses having different sizes; and applying, by the computing system, a softmax layer to the second intermediate vector to generate the output, wherein the output includes a final vector comprising final features corresponding to one of: the prostheses having different sizes, sets of compatible prostheses, or combinations of potential values of the one or more prosthesis parameters.

[0182] Clause 14. The method of clause 12, wherein: the nodes include cross-section nodes that correspond to different cross-sections in a plurality of cross-sections of the bone, and for each of the cross-section nodes in the graph: the feature data initially included in the feature vector for the cross-section node characterizes one or more aspects of the bone at the cross-section corresponding to the cross-section node, and modifying the feature vector of thecross-section node comprises, modifying, by the computing system, the feature vector of the cross-section node based on a weighted average of features in the feature vector of the crosssection node and features in the feature vectors passed to the cross-section node, wherein weights used in the weighted average are based on importance of the cross-section corresponding to the cross-section node and the cross-sections corresponding to adjacent nodes, the adjacent nodes being connected to the respective node in the graph data.

[0183] Clause 15. The method of any of clauses 1-14, wherein: the graph is a 3- dimensional lattice graph, for each respective sampling position in a 3-dimensional arrangement of sampling positions, the plurality of nodes includes a node corresponding to the respective sampling position, the 3-dimensional arrangement of sampling positions includes sampling positions corresponding to positions within the bone, and for each of the nodes, the feature data initially included in the feature vector for the node characterizes one or more aspects of the bone at the sampling position corresponding to the node.

[0184] Clause 16. The method of clause 15, wherein, for one or more of the nodes, the feature data initially included in the feature vector for the node includes one or more of: a measure of bone density for the sampling position corresponding to the node, data describing a local texture of the bone, or a gray level co-occurrence matrix.

[0185] Clause 17. The method of any of clauses 15-16, wherein: for each of the nodes, the plurality of edges includes a set of edges that connect the node with up to six other nodes in the plurality of nodes, for each of the nodes, the plurality of edges includes a set of edges that connect the node with up to eight other nodes in the plurality of nodes, or for each of the nodes, the plurality of edges includes a set of edges that connect the node with up to twelve other nodes in the plurality of nodes.

[0186] Clause 18B. The method of any of clauses 15B-17B, wherein the sampling positions include a set of sampling positions corresponding to predefined coordinates within 3 -dimensional image data of the bone.

[0187] Clause 19B. The method of any of clauses 15B-18B, further comprising: applying, by the computing system, a segmentation process to image data of the bone to identify a plurality of regions, wherein the regions correspond to sampling positions in the plurality of sampling positions; and for at least one of the regions, generating, by the computing system, based on data associated with one or more pixels of the image data within the region, the feature vector of the node corresponding to the sampling position corresponding to the region.

[0188] Clause 20B. The method of clause 19B, wherein the one or more pixels of the image data include a plurality of pixels of the image data.

[0189] Clause 2 IB. The method of clause 20B, wherein the region is a 3-dimensional region and each of the coordinates of two or more of the plurality of pixels included in the region are different.

[0190] Clause 22B. The method of any of clauses 1B-12B or 14B-21B, wherein: the output includes one or more dimensions of an appropriate prosthesis, and determining the recommended prosthesis for the patient comprises determining, by the computing system, the recommended prosthesis for the patient from among a plurality of available prostheses based on the one or more dimensions of the appropriate prosthesis.

[0191] Clause 23B. The method of any of clauses 1B-12B or 14B-21B, wherein: the output includes one or more parameters of an appropriate prosthesis, and determining the recommended prosthesis for the patient comprises determining, by the computing system, the recommended prosthesis for the patient based on the one or more parameters of the appropriate prosthesis.

[0192] Clause 24B. The method of any of clauses 1B-23B, wherein the one or more prosthesis parameters include one or more of: a size of a glenosphere of a glenoid prosthesis, a radius of the glenosphere of the glenoid prosthesis, a baseplate type of the glenoid prosthesis, an augment type of the glenoid prosthesis, a glenoid eccentricity of the glenoid prosthesis, a stem size of a humeral prosthesis, or a head sphere size of the humeral prosthesis.

[0193] Clause 25B. The method of any of clauses 1B-23B, wherein the one or more prosthesis parameters include one or more of: a size of a tibial component, a size of a talar prosthesis, or a size of an articulating component attachable to the tibial component.

[0194] Clause 26B. The method of any of clauses 1B-24B, further comprising surgically implanting the recommended prosthesis.

[0195] Clause 27B. A computing system comprising: a storage system; and one or more processors implemented in circuitry and configured to perform the methods of any of clauses 1B-26B.

[0196] Clause 28B. A computing system comprising means for performing the methods of any of clauses 1B-26B.

[0197] Clause 29B. Non-transitory computer-readable storage media having instructions stored thereon that, when executed, cause a computing system to perform the methods of any of clauses 1B-26B.

[0198] While the techniques been disclosed with respect to a limited number of examples, those skilled in the art, having the benefit of this disclosure, will appreciate numerous modifications and variations there from. For instance, it is contemplated that any reasonable combination of the described examples may be performed. It is intended that the appended claims cover such modifications and variations as fall within the true spirit and scope of the invention.

[0199] It is to be recognized that depending on the example, certain acts or events of any of the techniques described herein can be performed in a different sequence, may be added, merged, or left out altogether (e.g., not all described acts or events are necessary for the practice of the techniques). Moreover, in certain examples, acts or events may be performed concurrently, e.g., through multi-threaded processing, interrupt processing, or multiple processors, rather than sequentially.

[0200] In one or more examples, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium and executed by a hardware -based processing unit. Computer-readable media may include computer-readable storage media, which corresponds to a tangible medium such as data storage media, or communication media including any medium that facilitates transfer of a computer program from one place to another, e.g., according to a communication protocol. In this manner, computer-readable media generally may correspond to (1) tangible computer- readable storage media which is non-transitory or (2) a communication medium such as a signal or carrier wave. Data storage media may be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, code and / or data structures for implementation of the techniques described in this disclosure. A computer program product may include a computer-readable medium.

[0201] By way of example, and not limitation, such computer-readable storage media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage, or other magnetic storage devices, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. Also, any connection is properly termed a computer-readable medium. For example, if instructions are transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave areincluded in the definition of medium. It should be understood, however, that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transitory media, but are instead directed to non-transitory, tangible storage media. Disk and disc, as used herein, includes compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and Blu-ray disc, where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0202] Operations described in this disclosure may be performed by one or more processors, which may be implemented as fixed-function processing circuits, programmable circuits, or combinations thereof, such as one or more digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. Fixed-function circuits refer to circuits that provide particular functionality and are preset on the operations that can be performed. Programmable circuits refer to circuits that can programmed to perform various tasks and provide flexible functionality in the operations that can be performed. For instance, programmable circuits may execute instructions specified by software or firmware that cause the programmable circuits to operate in the manner defined by instructions of the software or firmware. Fixed-function circuits may execute software instructions (e.g., to receive parameters or output parameters), but the types of operations that the fixed-function circuits perform are generally immutable. Accordingly, the terms “processor” and “processing circuity,” as used herein may refer to any of the foregoing structures or any other structure suitable for implementation of the techniques described herein.

Claims

What is claimed is:

1. A method compri sing : for each node of a plurality of nodes of a graph, generating, by one or more processors implemented in circuitry, a feature vector for the node that initially includes feature data characterizing one or more aspects of a bone of a patient, wherein the graph includes one or more edges, each of the edges connecting a respective pair of the nodes; applying, by the one or more processors, a graph convolutional network (GCN) model to the graph and the feature vectors for the nodes to generate an output; and determining, by the one or more processors, based on the output, one or more recommended prostheses for the patient or recommended values of one or more prosthesis parameters.

2. The method of claim 1, wherein: the plurality of nodes includes cross-section nodes corresponding to different crosssections in a plurality of cross-sections of the bone, and for each of the cross-section nodes in the graph, the feature data initially included in the feature vector for the cross-section node characterizes one or more aspects of the bone at the cross-section corresponding to the cross-section node.

3. The method of claim 2, wherein, for each of the cross-sections of the bone, the one or more aspects of the bone at the cross-section of the bone include one or more measurements of dimensions of the bone at the cross-section of the bone.

4. The method of claim 3, wherein the bone is a tibia and the one or more measurements include measurements of the dimensions of the bone at the cross-section of the bone include a medial width of the tibia at the cross-section and an anterior length of the tibia at the crosssection.

5. The method of claim 3, wherein the bone is a talus and the one or more measurements include a measurement of a medio-lateral width of the talus.

6. The method of any of claims 2-5, wherein the cross-sections are perpendicular to a mechanical axis of the bone.

7. The method of any of claims 1-6, wherein: the GCN model is a first GCN model, the bone is a first bone, the feature data is first feature data, the one or more aspects are one or more first aspects, the graph is a first graph, the plurality of nodes is a first plurality of nodes, the prosthesis is a first prosthesis, and the output is a first output, and the method further comprises: generating, by the computing system, for each node of a second plurality of nodes of a second graph, a feature vector for the node that initially includes secondary feature data characterizing one or more second aspects of a second bone of the patient, wherein the second graph include one or more second edges, each of the second edges connecting a respective pair of the nodes in the second graph, applying, by the computing system, a second GCN model to the second graph and the feature vectors for the second plurality of nodes to generate second output; and determining, by the computing system, based on the second output, a second prosthesis to recommend for implantation in the patient or recommended values of one or more second prosthesis parameters, wherein the one or more second prosthesis parameters specify one or more aspects of a secondary prosthesis.

8. The method of claim 7, further comprising selecting, by the computing system, based on the first prosthesis, the second GCN model from among a plurality of secondary GCN models.

9. The method of any of claims 7-8, wherein: the second plurality of nodes includes cross-section nodes that correspond to a different cross-section in a plurality of cross-sections of the second bone, and for each of the cross-section nodes in the second graph, the feature data initially included in the feature vector for the cross-section node characterizes one or more aspects of the second bone at the cross-section corresponding to the cross-section node.

10. The method of any of claims 7-9, wherein the first prosthesis is designed for implantation in the first bone and the second prosthesis is designed for implantation in the second bone.

11. The method of any of claims 7-10, wherein the first bone is a tibia and the second bone is a talus.

12. The method of any of claims 1-10, wherein applying the GCN model comprises: performing, by the computing system, one or more message passing rounds, wherein performing a message passing round comprises: for each respective node of the plurality of nodes, passing, by the computing system, the feature vector for the respective node to each node of the plurality of nodes that is connected in the graph to the respective node; and for each respective node of the plurality of nodes, modifying, by the computing system, the feature vector of the respective node based on the feature vector for the respective node and the feature vectors passed to the respective node; and after completion of the one or more message passing rounds, for each respective node of the plurality of nodes, applying, by the computing system, a series of one or more graph convolutional layers (GCLs) of the GCN model to the feature vector of the respective node to obtain an embedding for the respective node; and generating, by the computing system, the output based on the embeddings for the nodes.

13. The method of claim 12, wherein generating the output based on the embeddings for the nodes comprises: applying, by the computing system, a pooling layer to the embeddings for the nodes, wherein the pooling layer generates a first intermediate vector comprising first intermediate features corresponding to prostheses having different sizes; applying, by the computing system, a fully connected layer to the first intermediate vector to generate a second intermediate vector, wherein the second intermediate vector comprises second intermediate features corresponding to the prostheses having different sizes; and applying, by the computing system, a softmax layer to the second intermediate vector to generate the output, wherein the output includes a final vector comprising final features corresponding to one of: the prostheses having different sizes, sets of compatible prostheses, orcombinations of potential values of the one or more prosthesis parameters.

14. The method of claim 12, wherein: the nodes include cross-section nodes that correspond to different cross-sections in a plurality of cross-sections of the bone, and for each of the cross-section nodes in the graph: the feature data initially included in the feature vector for the cross-section node characterizes one or more aspects of the bone at the cross-section corresponding to the cross-section node, and modifying the feature vector of the cross-section node comprises, modifying, by the computing system, the feature vector of the cross-section node based on a weighted average of features in the feature vector of the cross-section node and features in the feature vectors passed to the cross-section node, wherein weights used in the weighted average are based on importance of the cross-section corresponding to the cross-section node and the cross-sections corresponding to adjacent nodes, the adjacent nodes being connected to the respective node in the graph data.

15. The method of any of claims 1-14, wherein: the graph is a 3 -dimensional lattice graph, for each respective sampling position in a 3 -dimensional arrangement of sampling positions, the plurality of nodes includes a node corresponding to the respective sampling position, the 3 -dimensional arrangement of sampling positions includes sampling positions corresponding to positions within the bone, and for each of the nodes, the feature data initially included in the feature vector for the node characterizes one or more aspects of the bone at the sampling position corresponding to the node.

16. The method of claim 15, wherein, for one or more of the nodes, the feature data initially included in the feature vector for the node includes one or more of: a measure of bone density for the sampling position corresponding to the node, data describing a local texture of the bone, or a gray level co-occurrence matrix.

17. The method of any of claims 15-16, wherein: for each of the nodes, the plurality of edges includes a set of edges that connect the node with up to six other nodes in the plurality of nodes, for each of the nodes, the plurality of edges includes a set of edges that connect the node with up to eight other nodes in the plurality of nodes, or for each of the nodes, the plurality of edges includes a set of edges that connect the node with up to twelve other nodes in the plurality of nodes.

18. The method of any of claims 15-17, wherein the sampling positions include a set of sampling positions corresponding to predefined coordinates within 3 -dimensional image data of the bone.

19. The method of any of claims 15-18, further comprising: applying, by the computing system, a segmentation process to image data of the bone to identify a plurality of regions, wherein the regions correspond to sampling positions in the plurality of sampling positions; and for at least one of the regions, generating, by the computing system, based on data associated with one or more pixels of the image data within the region, the feature vector of the node corresponding to the sampling position corresponding to the region.

20. The method of claim 19, wherein the one or more pixels of the image data include a plurality of pixels of the image data.

21. The method of claim 20, wherein the region is a 3 -dimensional region and each of the coordinates of two or more of the plurality of pixels included in the region are different.

22. The method of any of claims 1-12 or 14-21, wherein: the output includes one or more dimensions of an appropriate prosthesis, and determining the recommended prosthesis for the patient comprises determining, by the computing system, the recommended prosthesis for the patient from among a plurality of available prostheses based on the one or more dimensions of the appropriate prosthesis.

23. The method of any of claims 1-12 or 14-21, wherein: the output includes one or more parameters of an appropriate prosthesis, anddetermining the recommended prosthesis for the patient comprises determining, by the computing system, the recommended prosthesis for the patient based on the one or more parameters of the appropriate prosthesis.

24. The method of any of claims 1-23, wherein the one or more prosthesis parameters include one or more of: a size of a glenosphere of a glenoid prosthesis, a radius of the glenosphere of the glenoid prosthesis, a baseplate type of the glenoid prosthesis, an augment type of the glenoid prosthesis, a glenoid eccentricity of the glenoid prosthesis, a stem size of a humeral prosthesis, or a head sphere size of the humeral prosthesis.

25. The method of any of claims 1-23, wherein the one or more prosthesis parameters include one or more of: a size of a tibial component, a size of a talar prosthesis, or a size of an articulating component attachable to the tibial component.

26. The method of any of claims 1-24, further comprising surgically implanting the recommended prosthesis.

27. A computing system comprising: a storage system; and one or more processors implemented in circuitry and configured to perform the methods of any of claims 1-26.

28. A computing system comprising means for performing the methods of any of claims 1-26.

29. Non-transitory computer-readable storage media having instructions stored thereon that, when executed, cause a computing system to perform the methods of any of claims 1-26.

Images