Graphic convolutional network model for recommending orthopedic prostheses

Abstract

For each node in the graph, the computing system generates a feature vector for the node, initially containing feature data that characterizes one or more aspects of the patient's bone. The graph includes one or more edges, each of which connects each pair of nodes. The computing system applies a graph convolutional network (GCN) model to the graph and the feature vectors for the nodes to generate an output. Based on the output, the computing system determines one or more recommended prostheses or recommended values ​​for one or more prosthesis parameters for the patient.

Description

Technical Field

[0001]

[0001] This application claims priority to U.S. Provisional Patent Application No. 63 / 535,425, filed on August 30, 2023, the entire content of which is incorporated herein by reference. Background Art

[0002]

[0002] Planning orthopedic surgery may involve selecting an appropriate prosthesis for implantation in a patient. Even among prostheses available for implantation on a single bone, there may be prostheses of different shapes and sizes. Selecting an appropriate prosthesis for a patient can be an important factor in whether an orthopedic surgery has a successful outcome or whether complications occur. Since no two patients have exactly the same anatomy, selecting an appropriate prosthesis for a patient can be a challenge for surgeons, especially less experienced surgeons. Summary of Invention

[0003]

[0003] This disclosure describes a technique in which a computing system uses one or more graph convolutional network models to determine a recommended orthopedic prosthesis for an individual patient. As described herein, the computing system may acquire feature data that characterizes one or more aspects of the patient's bone. The graph includes a plurality of sampling nodes and one or more edges. In some examples, the sampling nodes may correspond to different sampling locations, such as different cross-sections of the bone or different regions within the bone. For each sampling node in the graph, a feature vector for the sampling node initially includes feature data that characterizes one or more aspects of the bone at the sampling location corresponding to that node. The computing system may apply a GCN model to the graph and the feature vectors for the nodes to generate an output. In some examples, the computing system determines a recommended prosthesis for the patient based on the output generated by the GCN model. In some examples, the computing system determines recommended values ​​for one or more prosthesis parameters based on the output generated by the GCN model. Prosthesis parameters can specify one or more aspects of one or more prostheses, such as the size and radius of the glenosphere of a glenoid prosthesis, the stem size of a humeral prosthesis, the head sphere size of a humeral prosthesis, the type of extension of a glenoid prosthesis, and the baseplate type of a glenoid prosthesis. Using a GCN model in this way can allow a computing system to take into account the morphology of bone at various sampling locations. Furthermore, a GCN model may contain fewer parameters than other types of classifier neural networks, which can save computational resources and make it easier to train.

[0004]

[0004] In one example, the present disclosure describes a method comprising: for each node of a plurality of nodes in a graph, a computing system first generates a feature vector for the node, which includes feature data characterizing one or more aspects of the patient's bone; a computing system applies a graph convolutional network (GCN) model to the graph and the feature vector for the nodes to generate an output, wherein the graph includes one or more edges, each of which connects each pair of nodes; and a computing system determines a recommended prosthesis for the patient based on the output.

[0005]

[0005] Details of various examples of the present disclosure are described in the accompanying drawings and the following description. Various features, purposes, and advantages will become apparent from the description, drawings, and claims. [Brief explanation of the drawing]

[0006] [Figure 1]

[0006] A conceptual diagram showing an exemplary computing system in which one or more of the techniques of this disclosure may be implemented. [Figure 2]

[0007] A flowchart illustrating exemplary operation of a computing system using one or more of the techniques of this disclosure. [Figure 3A]

[0008] A conceptual diagram showing an exemplary cross-section of bone using one or more techniques of the present disclosure. [Figure 3B] A conceptual diagram showing an exemplary cross-section of bone using one or more techniques of the present disclosure. [Figure 3C] A conceptual diagram showing an exemplary cross-section of bone using one or more techniques of the present disclosure. [Figure 4]

[0009] A conceptual diagram showing an exemplary graph that may be used with a graph convolutional network (GCN) model using one or more of the techniques of this disclosure. [Figure 5A]

[0010] A conceptual diagram showing an exemplary cross-section of the talus using one or more techniques of the present disclosure. [Figure 5B] A conceptual diagram showing an exemplary cross-section of the talus using one or more techniques of the present disclosure. [Figure 5C] A conceptual diagram showing an exemplary cross-section of the talus using one or more techniques of the present disclosure. [Figure 5D] A conceptual diagram showing an exemplary cross-section of the talus using one or more techniques of the present disclosure. [Figure 6]

[0011] A block diagram illustrating an example of a planning system using a primary GCN and a secondary GCN, based on one or more techniques of the present disclosure. [Figure 7]

[0012] A conceptual diagram illustrating an exemplary process for recommending a talar prosthesis on a partial basis to a recommended tibial prosthesis using one or more techniques of the present disclosure. [Figure 8]

[0013] A conceptual diagram showing an exemplary graph that may be used with a GCN model using one or more techniques of the present disclosure. [Figure 9]

[0014] A flowchart illustrating exemplary operation of a computing system in which two interacting prostheses are recommended using one or more techniques of the present disclosure. [Figure 10]

[0015] A conceptual diagram showing an exemplary three-dimensional grid graph using one or more techniques of the present disclosure. [Figure 11]

[0016] A conceptual diagram illustrating an exemplary mesh-based neural network model using one or more techniques of the present disclosure. [Figure 12]

[0017] A flowchart illustrating exemplary operation of a planning system using one or more of the techniques of this disclosure. [Figure 13]

[0018] A block diagram illustrating an exemplary architecture for generating quantized face embeddings and reconstructing anatomical models using one or more techniques of the present disclosure. [Figure 14]

[0019] A flowchart illustrating an exemplary operation of a prediction unit for generating information indicating one or more recommended prostheses for a patient, or recommended values of one or more prosthesis parameters, in accordance with one or more techniques of the present disclosure. [Figure 15]

[0020] A flowchart illustrating an exemplary training process for a machine learning model in accordance with one or more techniques of the present disclosure. [Figure 16]

[0021] A conceptual diagram illustrating an exemplary surgical planning user interface that shows a surgical proposal for reverse shoulder replacement surgery in accordance with one or more techniques of the present disclosure. Mode for Carrying Out the Invention

[0007]

[0022] A surgeon can select a prosthesis to implant in a patient from among several available options. Available prostheses can have various parameters, such as size and shape. Selecting the appropriate prosthesis from the available options can be a crucial factor in the ultimate success of the surgery. Choosing a prosthesis that is too large or too small can lead to limited range of motion, susceptibility to loosening, fracture, and other complications. Because patients have bones of different sizes and shapes, selecting the appropriate prosthesis for implantation on bone can be a complex process. Often, the ability to select the appropriate prosthesis comes from experience as well as objective knowledge. Consequently, selecting the appropriate prosthesis is often difficult for less experienced surgeons. To assist users in planning orthopedic surgeries, computerized surgical planning systems have been developed, some of which offer automated recommendations for prostheses to be implanted in patients. Because bones often have complex shapes, it can be difficult for such computerized surgical planning systems to predict appropriate prostheses given measurements, deterministic rules, and even conventional machine learning models. For example, accurate prosthesis prediction may depend on information about the relationships between non-adjacent parts of the bone. Computerized surgical planning systems that use conventional convolutional neural network (CNN) models for orthopedic prosthesis prediction generally involve a hierarchy of convolutional layers that convolve information from spatially adjacent areas in a 3D image. However, this method can require a considerable number of convolutional layers. Existing computerized surgical planning systems may use the output of a CNN to filter the set of available prostheses. For example, the output of a CNN may be used to rank the available prostheses, and information on only the top-ranked subset of available prostheses is initially presented for user (e.g., surgeon) review.Filtering the set of available prostheses can accelerate the planning process. Furthermore, the planning system may generate and display a user interface in which a three-dimensional model of a prosthesis is positioned at a proposed implantation site relative to a three-dimensional model of the patient's anatomy. A user may use such a user interface to confirm that the prosthesis is suitable for the patient. However, generating such a user interface is computationally complex and can be time-consuming. Therefore, it is not desirable to generate such a user interface for all available prostheses.

[0008]

[0023] This disclosure describes a technique that can address these problems. More specifically, this disclosure describes a technique in which a computing system uses one or more graph convolutional network (GCN) models to recommend an appropriate prosthesis to a surgeon for a patient. For example, the computing system may acquire feature data characterizing one or more aspects of bone at multiple sampling locations, such as a cross-section of bone or a region within bone. The graph includes multiple nodes and one or more edges. The nodes include sampling nodes corresponding to different sampling locations. For each sampling node in the graph, a feature vector for the sampling node initially includes feature data characterizing one or more aspects of bone at the sampling location corresponding to that sampling node. The computing system may apply a GCN model to the graph and the feature vectors for the nodes to generate an output. Based on the output, the computing system may determine a recommendation for one or more prostheses for implantation in the patient, or a recommendation for the values ​​of one or more prosthesis parameters of one or more prostheses for implantation in the patient. Such an application of GCN allows the computing system to efficiently consider the interrelationships between aspects of bone at various sampling locations of bone. Furthermore, GCN models may contain fewer parameters (e.g., weights, layers, etc.) than other types of machine learning models, which can increase computational efficiency and reduce memory requirements. In addition, in some examples, a computing system may use the output of a GCN model to filter the set of available prostheses. For at least the reasons described above, the techniques of this disclosure improve the efficiency and effectiveness of computing systems for filtering the set of available prostheses. Filtering the set of available prostheses may enable a planning system to propose a user interface in which 3D models of prostheses are positioned at proposed implantation locations relative to 3D models of the patient's anatomical structures, more quickly and efficiently.

[0009]

[0024] Figure 1 is a conceptual diagram showing an exemplary computing system 100 in which one or more techniques of the present disclosure may be implemented. In the example of Figure 1, the computing system 100 includes one or more processors 102, a storage system 104, a communication interface 106, and a display 108. In other examples, the computing system 100 may include more, fewer, or different components. The components of the computing system 100 may be in one or more computing devices. For example, the processor 102 may be in a single computing device or distributed among multiple computing devices of the computing system 100, the storage system 104 may be in a single computing device or distributed among multiple computing devices of the computing system 100, and so on. In some examples, the computing system 100 is a personal computer, a system of computing devices, one or more server devices, or a system comprising one or more other types of computing devices.

[0010]

[0025] The processor 102 is implemented in a circuit and may include one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), hardware, or any combination thereof. Generally, the processor 102 may be implemented as fixed-function circuits, programmable circuits, or a combination thereof. Fixed-function circuits refer to circuits that provide a specific functionality and are preset to operations that can be performed. Programmable circuits refer to circuits that can be programmed to perform a variety of tasks and provide flexible functionality in operations that can be performed. For example, a programmable circuit may execute software or firmware that operates the programmable circuit in a manner defined by software or firmware instructions. Fixed-function circuits may execute software instructions (for example, to receive or output parameters), but the type of operation performed by a fixed-function circuit is generally immutable. In some examples, one or more of the units may be separate circuit blocks (fixed-function or programmable), and in some examples, one or more units may be integrated circuits.

[0011]

[0026] The processor 102 may include an arithmetic logic unit (ALU), an elementary function unit (EFU), digital circuits, analog circuits, and / or a programmable core, formed from programmable circuits. In an example where the operation of the processor 102 is carried out using software executed by the programmable circuits, the memory system 104 may store object code of the software that the processor 102 receives and executes, or another memory (not shown) within the processor 102 may store such instructions. An example of software includes software designed for surgical planning. The processor 102 may perform actions originating from the computing system 100 in this disclosure.

[0012]

[0027] The memory system 104 may store various types of data used by the processor 102. The memory system 104 may include any of various memory devices, such as dynamic random access memory (DRAM), which includes synchronous DRAM (SDRAM), magnetoresistive RAM (MRAM), resistive RAM (RRAM®), or other types of memory devices. An example of the display 108 may include a liquid crystal display (LCD), a plasma display, an organic light-emitting diode (OLED) display, or another type of display device.

[0013]

[0028] The communication interface 106 enables the computing system 100 to output data and commands to and receive data and commands from a medical imaging system or other device via one or more communication links or networks. The communication interface 106 may include hardware circuitry that enables the computing system 100 to communicate with other computing systems and devices (for example, wirelessly or using wires). Exemplary networks may include various types of communication networks, including one or more wide area networks such as the Internet and local area networks. The network may include wired and / or wireless communication links.

[0014]

[0029] In the example in Figure 1, the memory system 104 stores medical image data 110, a planning system 118, and a training system 120. In other examples, the memory system 104 may store more, less, or different types of data or units. Furthermore, the data and units shown in the example in Figure 1 are provided for illustrative purposes only and may not represent how the data is actually stored or how the software is actually implemented. The planning system 118 may comprise instructions that can be executed by the processor 102. For simplicity of explanation, the disclosure may describe the planning system 118 as performing various actions when the processor 102 executes instructions for the planning system 118. Furthermore, in the example in Figure 1, the planning system 118 includes feature data 112, graph data 114, and a GCN model 116.

[0015]

[0030] The planning system 118 is a system that can assist users, such as surgeons, in planning orthopedic surgery as part of either the preoperative planning process or the intraoperative planning process. As part of performing a process to assist users in planning orthopedic surgery, the planning system 118 may present a set of user interfaces to assist users in selecting an orthopedic prosthesis to implant in a patient when there are multiple available orthopedic prostheses. For simplicity of explanation, this disclosure may refer to orthopedic prostheses simply as prostheses. The user interface may include a user interface that displays one or more 3D models of the patient's bones, measurement data about the bones, and other information about the patient's bones or other anatomical features. Furthermore, the user interface may display information about a set of available prostheses for implantation in the patient. The planning system 118 may perform an automated process of ranking and / or filtering the available prostheses. The user interface that displays information about a set of available prostheses may, at least initially, limit the information to one or more top-ranked prostheses and / or the remaining prostheses after filtering.

[0016]

[0031] The planning system 118 may be applicable in various orthopedic surgery contexts to assist users in selecting prostheses. For example, in ankle arthroplasty, the planning system 118 may assist surgeons in selecting tibial and / or talar prostheses. In shoulder arthroplasty, the planning system 118 may assist surgeons in selecting glenoid prostheses and / or humeral prostheses. In knee arthroplasty, the planning system 118 may assist surgeons in selecting femoral and / or tibial prostheses. In hip arthroplasty, the planning system 118 may assist surgeons in selecting acetabular and / or femoral prostheses. Other examples may be applicable to other bones and / or joints.

[0017]

[0032] Prostheses for implantation on bone can be available in a variety of shapes and sizes. For example, there may be several different sizes of tibial prostheses that a surgeon can choose from when performing ankle arthroplasty. The tibial prosthesis can be implanted in the distal end of the patient's tibia. Depending on the size of the patient's tibia and other factors, different sizes of tibial prostheses may be appropriate. In some cases, the prosthesis can be customized so that it is unique to the patient. In other words, the prosthesis can be a patient-specific prosthesis.

[0018]

[0033] As part of the process of recommending one or more prostheses or values ​​for one or more prosthesis parameters for implantation in a patient, the planning system 118 acquires feature data 112 that characterize one or more aspects of one or more bones at multiple sampling locations. In some examples, the sampling locations include cross-sections of one or more bones. Exemplary aspects of a bone cross-section may include dimensions of the bone at the cross-section (e.g., width, length, height, etc.), average density of bone (or cortical bone) at the cross-section, minimum density of bone at the cross-section, data indicating the presence of voids in the bone, and other types of data characterizing the bone at the cross-section. In some examples, the sampling locations include two-dimensional or three-dimensional regions. In such examples, aspects of the bone may include data describing bone density within the region, the presence or absence of bone tissue within the region, and the texture of the bone (e.g., cortical, cancellous, etc.). In some examples, the sampling locations may include combinations of cross-sections and regions.

[0019]

[0034] In examples where the sampling location includes a cross-section of bone, each cross-section may be perpendicular to the axis passing through the bone. For example, in one example where the bone is the tibia, each cross-section may be perpendicular to the mechanical axis of the tibia. In another example where the bone is the talus, each cross-section may be perpendicular to the mechanical axis of the talus. The mechanical axis of the tibia can be defined as the line connecting the center of the knee to the center of the talus. The cross-sections may be spaced apart from each other by one or more amounts. For example, the cross-sections may be spaced apart by 1 millimeter (mm) or another amount.

[0020]

[0035] In some examples, the planning system 118 may use medical image data 110 to obtain feature data 112. Medical image data 110 may include computed tomography (CT) data, dual-energy X-ray absorptiometry (DXA or DEXA) data, three-dimensional bone modeling data, magnetic resonance imaging (MRI) data, and / or other types of medical imaging data. The planning system 118 may use medical image data 110 to generate a three-dimensional model or multiple two-dimensional models of one or more bones of a patient. For example, the planning system 118 may perform a segmentation process to determine the boundaries of one or more bones, and in some examples, other tissues, within the medical image data 110. The planning system 118 may use the determined boundaries to generate one or more models (e.g., two-dimensional or three-dimensional models) of one or more bones (and, in some examples, other tissues). The planning system 118 may then use one or more models to measure one or more aspects of one or more bones. The feature data may represent the measured aspects of one or more bones. The planning system 118 may perform the segmentation process in one of several ways. For example, the planning system 118 may use a neural network-based process to perform segmentation. In some examples, the planning system 118 may, as an addition or alternative, obtain feature data directly from two-dimensional medical images (e.g., X-rays, CT slices).

[0021]

[0036] Graph data 114 defines a graph containing multiple nodes and one or more edges. Each edge connects each pair of nodes. In other words, each edge is a connection between exactly two of the nodes. In some examples, edges connect each of the nodes. In other examples, edges do not connect each of the nodes. Nodes include sampling nodes corresponding to different sampling locations, such as cross-sections or regions. Sampling nodes corresponding to cross-sections are sometimes referred to herein as “cross-section nodes.” For each sampling node in the graph, a feature vector for the sampling node initially contains feature data that characterizes one or more aspects of the bone at the sampling location corresponding to that node. For example, a feature vector corresponding to a particular cross-section may include elements indicating the length and width of a particular cross-section of the bone. In some examples, in addition to sampling nodes, the nodes in the graph may include nodes such as nodes corresponding to entire bones or nodes corresponding to neighboring bones.

[0022]

[0037] The planning system 118 may apply the GCN model 116 to the graph and feature vectors for the nodes in order to generate an output. In some examples, the planning system 118 may determine one or more recommended prostheses for a patient based on the output. In some examples, the planning system 118 may determine recommended values ​​for one or more prosthesis parameters based on the output. The prosthesis parameters may specify one or more prosthesis configurations. When the planning system 118 applies the GCN model 116 to the graph and feature vectors, the planning system 118 may perform one or more message passing rounds. When performing at least one or each of the message passing rounds, the planning system 118 may pass the feature vector for each node to each node of the multiple nodes connected to each node in the graph, for at least one of the multiple nodes, or for each of each of those nodes. Thus, each node may receive the feature vectors of its neighboring nodes.

[0023]

[0038] Furthermore, as part of performing a message passing round, the planning system 118 may modify the feature vector of at least one of the multiple nodes, or each of each of them, based on the feature vector for that node and the feature vector passed to that node. In other words, the planning system 118 may aggregate the feature vector for each node and the feature vector passed to that node. The planning system 118 may modify the feature vector of each node in one of several ways. For example, the planning system 118 may calculate each element of the modified feature vector as the average (e.g., weighted average) of the corresponding elements of the feature vector. In another example, the planning system 118 may calculate each element of the modified feature vector as the sum (or weighted sum) of the corresponding elements of the feature vector.

[0024]

[0039] In some examples where the planning system calculates a weighted average or weighted sum, the planning system 118 may determine the weights for the feature vectors according to the following formula:

[0025]

number

[0026] In this example, for each edge of the graph linking node i and node j, a weight e is applied to the features of the feature vector. ij When an edge connects different nodes,

[0027]

number

[0028] It can be equal to . In this formula, d represents the distance between sampling locations corresponding to nodes. For example, if node i corresponds to the first cross section and node j corresponds to the fourth cross section, the value of d can be equal to 3. In some examples where the sampling locations include two-dimensional or three-dimensional regions, the value d may represent the physical Euclidean distance between the centroids or nearest boundaries of the region. In other examples, the value d may represent a different type of distance measure. Weight e applied to the features of the feature vector ij This can be equal to 1 if nodes i and j are the same node (i.e., the weight applied to the features of the node's own feature vector is 1). Therefore, the feature vectors of nodes corresponding to sampling locations further away from the sampling location corresponding to the current node have less influence on the modified features of the current node's feature vector than the feature vectors of nodes corresponding to sampling locations closer to it.

[0029]

[0040] In some cases, a particular sampling location may be more important than other sampling locations for determining one or more recommended prostheses or recommended values ​​for one or more prosthesis parameters. For example, in one case where the sampling locations correspond to cross-sections of the tibia, a cross-section closer to the distal end of the tibia may be more important for recommending a tibial prosthesis than a cross-section further from the distal end of the tibia. Therefore, when aggregating the feature vectors of one node with those of another, the planning system 118 may apply weights to the feature vectors based on the importance of the sampling locations corresponding to the nodes.

[0030]

[0041] Furthermore, in some examples where the sampling location includes a cross-section, a cross-section with bone voids or a low bone mineral density may have lower importance than a cross-section without bone voids or with a higher bone mineral density (BMD). Therefore, when aggregating the feature vectors of one node with those of another, the planning system 118 may apply weights to the feature vectors based on the presence and / or BMD of the cross-sections corresponding to the nodes. In some examples, the weights may be based on a combination of factors, such as voids, BMD, distance between cross-sections, and relative importance.

[0031]

[0042] After the completion of one or more message-passing rounds, the planning system 118 may apply a series of one or more graph convolutional layers (GCLs) of the GCN model 116 to each node of the graph to obtain an embedding for that node. Each GCL may contain a set of artificial neurons. At least some of the artificial neurons in the GCL may take as input some or all of the features of the node's feature vector or the feature vector generated by the previous GCL. The artificial neurons in the GCL may output the result of a transfer function applied to a weighted sum of the inputs according to machine-learned weights.

[0032]

[0043] In some examples, part or all of the GCL increases the dimensional embedding space. For example, in one example where each feature vector contains two features (e.g., a feature corresponding to the width of the bone cross-section and a feature corresponding to the length of the bone cross-section), the first GCL may increase the dimensional embedding space from 2 dimensions to 4 dimensions. In this example, the second GCL may increase the dimensional embedding space from 4 dimensions to 8 dimensions. In some examples, the planning system 118 applies batch normalization to each output of the GCL. In some examples, the planning system 118 applies an activation function such as a Rectified Linear Unit (ReLU) activation function, a sigmoid activation function, or another type of activation function to the output of the GCL.

[0033]

[0044] The planning system 118 may generate an output based on the embeddings. For example, the planning system 118 may apply a pooling layer to the embeddings for the nodes. The pooling layer generates a first intermediate vector having a first intermediate feature corresponding to protheses of different sizes. For example, the number of nodes in the graph may be equal to 10, and the final GCL layer may output an 8-dimensional embedding for each node. Thus, there may be 10 × 8 features after the final GCL layer. In this example, the pooling layer may reduce the number of features to 1 × 8. For example, the pooling layer may determine the average (e.g., mean) of each corresponding feature for each of the 10 nodes, resulting in a first intermediate vector containing 8 features.

[0034]

[0045] Furthermore, the planning system 118 may apply a fully connected layer to the first intermediate vector to generate a second intermediate vector. The fully connected layer may contain a certain number of output neurons. In some examples, a certain number of output neurons may correspond to the number of available prostheses. For example, each output neuron may correspond to a different size of prosthesis. Thus, the second intermediate vector may have a second intermediate feature corresponding to prostheses of different sizes. In some examples, the output neurons may correspond to different prosthesis parameters. Prosthesis parameters may specify one or more prosthese characteristics. For example, in an example involving shoulder arthroplasty, prosthesis parameters may include the size or radius of the glenosphere of the glenosphere prosthesis, the eccentricity of the glenosphere, the baseplate type of the glenosphere prosthesis, the expansion type of the glenosphere prosthesis, the stem size of the humeral prosthesis, the headsphere size of the humeral prosthesis, and so on. In one case involving total ankle replacement surgery, prosthesis parameters may include the length of the tibial tray, the size of the talar prosthesis, and the size of the articular elements (e.g., polyethylene articular elements attached to the tibial tray).

[0035]

[0046] Some or all of the output neurons in the fully connected layer may receive some or all of the features of the first intermediate vector as input. Some or all of the output neurons in the fully connected layer may compute the result of a transfer function applied to the weighted sum of the inputs according to machine-learned weights. Furthermore, in some examples, the planning system 118 may apply a softmax layer to the second intermediate vector to generate an output containing the final vector. In some examples, the final vector contains final features corresponding to prothesis of different sizes. The softmax layer may transform the second intermediate vector into a probability distribution of possible outcomes. The softmax layer may normalize the second intermediate vector into a probability distribution across output classes (e.g., available prothesis). In this example, the planning system 118 may determine a recommended prothesis based on the final vector. For example, the planning system 118 may determine that the recommended prothesis is the one corresponding to the highest (or lowest) valued feature in the final vector. In some examples, the final vector may include features corresponding to different sets of mutually compatible prostheses, and the planning system 118 may determine the recommended set of mutually compatible prostheses as the set of mutually compatible prostheses corresponding to the highest (or lowest) value feature in the final vector.

[0036]

[0047] In some examples, the final vector includes features corresponding to different combinations of possible values ​​for the prosthesis parameters of one or more prostheses. For example, the first feature of the final vector may correspond to the first radius of the glenosphere of the glenosphere prosthesis, the first baseplate type of the glenosphere prosthesis, and the first expansion type of the glenosphere prosthesis; the second feature of the final vector may correspond to the first radius of the glenosphere of the glenosphere prosthesis, the first baseplate type of the glenosphere prosthesis, and the second expansion type of the glenosphere prosthesis; the third feature of the final vector may correspond to the first radius of the glenosphere of the glenosphere prosthesis, the second baseplate type of the glenosphere prosthesis, and the second expansion type of the glenosphere prosthesis, and so on. The expansion component of the glenosphere prosthesis is positioned between the baseplate of the glenosphere prosthesis and the patient's scapula. Different expansion types have different shapes. For example, a first type of extension element has a bone-facing surface parallel to the baseplate; a second type of extension element has a bone-facing surface angled to the baseplate across the entire diameter of the baseplate; a third type of extension element is partially angled to the baseplate and partially parallel to the baseplate; and a fourth type of extension element has a bone-facing surface with a patient-specific shape that conforms to the shape of the patient's scapula. Different baseplate types may have different sizes. For example, a first type of baseplate may have a first width and height for engaging with a relatively small glenoid cavity; a second type of baseplate may have a second width and height for engaging with a relatively large glenoid cavity; and a third type of baseplate may engage with both the glenoid cavity and one or more other parts of the patient's scapula, such as the acromion process or coracoid process. A third type of baseplate may be used in cases of complex deformities or trauma to the patient's scapula, where additional support is desirable.

[0037]

[0048] In examples where the final vector includes features corresponding to different combinations of possible values ​​for the prosthesis parameters of two or more prostheses, the features are limited to possible combinations of prosthesis parameter values ​​that are compatible between the prostheses. For example, a tibial tray of size 1 may not fit a talus prosthesis of size 2. Therefore, in this example, the final vector does not include features corresponding to a tibial tray of size 1 and a talus prosthesis of size 2. The planning system may determine the recommended values ​​for the prosthesis parameters of one or more prostheses as combinations of prosthesis parameter values ​​corresponding to the highest (or lowest) value feature in the final vector.

[0038]

[0049] This process can be summarized by the following formula:

[0039]

number

[0040] In the above equation, Z is the convolutional signal matrix, X is the signal (for example, a matrix of node feature vectors), and W (0) is the weight of the first GCL layer (i.e., from the input layer to the hidden layer), and W (1) This is the weight of the second GCL layer (i.e., from the hidden layer to the output layer).

[0041]

number

[0042] teeth,

[0043]

number

[0044] It can be a normalized adjacency matrix defined as, where,

[0045]

number

[0046] A is the adjacency matrix, and I N This is the identity matrix.

[0047]

number

[0048] teeth,

[0049]

number

[0050] It can be equal to the above equation.

[0051]

number

[0052] This can represent the first GCL layer,

[0053]

number

[0054] can represent the second GCL layer. Readout represents the layer that applies average pooling of node embeddings in the second GCL layer (i.e., the read layer). MLP represents the fully connected layer before the final softmax layer.

[0055]

[0050] In some cases, the planning system 118 may automatically perform one or more tests to verify that one or more recommended prostheses, or the recommended values ​​for one or more prosthesis parameters of one or more prostheses, are appropriate for the patient. For example, after determining a recommended prosthesis based on the output of the GCN model 116, the planning system 118 may verify that the recommended prosthesis is properly seated on the cortical bone. For example, if the recommended prosthesis is not seated on the cortical bone, the prosthesis may effectively sink into the cancellous bone. Therefore, there must be sufficient overlap between the prosthesis and the cortical bone to ensure stable positioning of the prosthesis.

[0056]

[0051] After the planning system 118 has determined a recommended prosthesis, the surgeon may decide whether to accept the recommendation or choose a different prosthesis. If the surgeon accepts the recommendation, the surgeon may surgically implant the recommended prosthesis. For example, if the planning system 118 recommends a particular glenoid prosthesis, the surgeon may surgically implant that particular glenoid prosthesis in the patient. In an example where the planning system 118 determines recommended values ​​for the prosthesis parameters of the prosthesis, the surgeon may evaluate the recommended values ​​for the prosthesis parameters and accept the recommended values ​​or choose different values ​​for the prosthesis parameters. The surgeon may implant the prosthesis according to the recommended or selected values ​​for the prosthesis parameters.

[0057]

[0052] The training system 120 can train the GCN model 116. The training system 120 may be contained in the same computing device / system that implements the planning system 118, or in a different computing device or system. In some examples, the training system 120 may perform one or more training epochs. In each training epoch, the training system 120 may present the GCN model 116 with one or more batches (e.g., a full batch, a mini-batch, or individual training examples) of training input examples. For each training input example, the training system 120 may use an error function (e.g., a loss function) to compare the output produced by the GCN model 116 with the corresponding labeled training output example. In some examples, the error function is a cross-entropy error function, such as defined in the following equation:

[0058]

number

[0059] In the above equation, ln is the natural logarithm function, and y L is a set of node indices with labels, Y is a set of labels, and Z is the convolutional signal matrix (i.e., the output of the GCN model 116). After each batch, the training system 120 may update the weights of the GCN model 116, for example, by applying a backpropagation process. During each epoch, the training system 120 may present the same training input examples to the GCN model 116. The training dataset may include validation examples in addition to the training input examples. The training system 120 may use the validation examples to validate the GCN model 116, but does not use the validation examples to update the weights of the GCN model 116.

[0060]

[0053] Furthermore, the system in Figure 1 includes a fulfillment system 122. In some examples, the fulfillment system 122 is configured to automatically (for example, by robotic control) select a prosthesis from a rack of available prostheses for shipment. In some examples, the fulfillment system 122 is configured to manufacture the selected patient-specific prosthesis. For example, the fulfillment system 122 may include an additive manufacturing system (for example, a 3D printer) for manufacturing one or more components of the patient-specific prosthesis.

[0061]

[0054] Figure 2 is a flowchart illustrating exemplary operation of the computing system 100 using one or more techniques of the present disclosure. The flowcharts of the present disclosure are presented as examples. In other examples, the flowcharts may include more, fewer, or different actions, or the actions may be performed in a different order.

[0062]

[0055] In the example of Figure 2, the planning system 118 may generate a feature vector for each node of a plurality of nodes in the graph, initially containing feature data that characterizes one or more aspects of the patient's bone (200). The graph includes one or more edges, each of which connects each pair of nodes. The graph can be structured in one of several ways. For example, in some examples, the nodes may include cross-sectional nodes corresponding to different cross-sections in a plurality of cross-sections of bone. In such examples, for each cross-sectional node in the graph, the feature data initially included in the feature vector for the cross-sectional node characterizes one or more aspects of the bone in the cross-section corresponding to that node. In some examples, the cross-sections are perpendicular to the functional axis of the bone. In other examples, the cross-sections may be perpendicular to another axis of the bone (e.g., the anatomical axis) or may be arranged differently relative to the bone.

[0063]

[0056] In some examples, the plurality of nodes may include a node having a feature vector that first contains feature data characterizing one or more aspects of two or more bones of the patient. For example, the plurality of nodes may include a node having a feature vector that first contains feature data characterizing one or more aspects of the patient's tibia and a node having a feature vector that first contains feature data characterizing one or more aspects of the patient's talus. In another example, the plurality of nodes may include a node having a feature vector that first contains feature data characterizing one or more aspects of the patient's scapula and a node having a feature vector that first contains feature data characterizing one or more aspects of the patient's humerus.

[0064]

[0057] For each node in the graph, the feature vector for the node initially includes feature data that characterizes one or more aspects of the bone in the cross-section corresponding to that node. In some examples, for each cross-section of the bone, one or more aspects of the bone in the cross-section include one or more measurements of the bone dimensions in the cross-section. For example, in one example (where the bone is the tibia), one or more measurements of the bone dimensions in the cross-section include the medial width of the bone in the cross-section and the anterior length of the bone in the cross-section (as shown below with respect to Figures 3A to 3C). In another example, one or more measurements of the bone in the cross-section may include one or more of the following: anterior distance (i.e., the distance from the center of the cross-section to the anterior edge of the cross-section), posterior distance (i.e., the distance from the center of the cross-section to the posterior edge of the cross-section), lateral distance (i.e., the distance from the center of the cross-section to the lateral edge of the cross-section), and medial distance (i.e., the distance from the center of the cross-section to the medial edge of the cross-section). The use of anterior, posterior, lateral, and medial distances as features for input to the GCN Model 116 can take into account bone asymmetry and thus improve the prediction of recommended prostheses by considering, for example, the overhang or underhang of the prosthesis relative to the bone in a way that cannot be expressed by two distances alone, or by recommending asymmetrical prostheses.

[0065]

[0058] In some cases, such as when the bone is the talus, the feature vector for the node first includes feature data that characterizes a single measurement of the talus (e.g., medial-lateral width). In other words, in cases where the bone is the talus, one or more measurements may include the medial-lateral width measurement of the talus. The medial-lateral width measurement may be the only measurement required for the talus in cases where the width of the talus is sufficient to select a talus prosthesis from among several available talus prostheses. In other cases where one or more additional measurements of the talus (e.g., anterior-posterior length) are required to select a talus prosthesis from among several available talus prostheses, the feature vector for the node may include one or more additional measurements of the talus.

[0066]

[0059] In some examples, the planning system 118 acquires feature data 112 based on medical image data 110. For example, the planning system 118 may acquire two-dimensional or three-dimensional medical images, such as individual CT images, a 3D model constructed from two-dimensional CT images, a three-dimensional MRI image, or other types of medical images. In examples where the planning system 118 acquires two-dimensional medical images, the planning system 118 may select two-dimensional medical images corresponding to bone cross-sections that the planning system 118 will use to acquire feature data. In some examples where the planning system 118 acquires two-dimensional medical images, the planning system 118 may interpolate one or more bone cross-sections that the planning system 118 will use to acquire feature data based on two or more of the two-dimensional medical images. In some examples where the planning system 118 acquires three-dimensional medical images, the planning system 118 may take slices of a three-dimensional medical image to acquire bone cross-sections. The bone cross-sections may be orthogonal to the functional axes of the bone. In some examples where the planning system 118 acquires a 2D image or a 3D model, the planning system 118 may interpolate pixel or voxel values ​​for the desired cross-sectional location.

[0067]

[0060] In some cases, the feature vector of a node may include information about the bone mineral density (BMD) of the bone in the corresponding cross-section. Information about the bone's BMD can be useful in determining the recommended prosthesis. For example, a larger prosthesis may be more appropriate when the BMD is relatively low, in order to increase contact of the prosthesis with denser cortical bone, and otherwise to diffuse the load of the prosthesis over a larger area. Furthermore, in some cases, voids may be present in one or more cross-sections of the bone. Voids may be areas in the bone with zero or very low BMD. Generally, it is desirable to have the prosthesis in contact with solid bone rather than voids in the bone. Therefore, including void and other BMD information in the feature vector can help the planning system 118 recommend a prosthesis.

[0068]

[0061] In some examples, the planning system 118 may preprocess the feature data 112. For example, before using any of the features, the planning system 118 may apply a normalization function to the features. Applying a normalization function can improve numerical stability and gradient descent convergence. The following are example normalization functions.

[0069]

number

[0070] In this normalization function,

[0071]

number

[0072] σ is the normalized value of the feature, x is the original value of the feature, μ is the mean of the original values ​​of the feature in the training dataset, and σ is the standard deviation of the feature in the training dataset.

[0073]

[0062] In another example, the graph is a three-dimensional grid graph, where for each sampling position in the three-dimensional arrangement of sampling positions, multiple nodes include a node corresponding to each sampling position. The three-dimensional arrangement of sampling positions may include sampling positions corresponding to positions within one or more bones. For some or all of the nodes, the first feature data included in the feature vector for the node characterizes one or more aspects of one or more bones at the sampling position corresponding to the node.

[0074]

[0063] Furthermore, in the example of Figure 2, the planning system 118 applies the GCN model 116 to the graph and the node feature vectors to generate an output (202). As part of applying the GCN model 116, the planning system 118 may perform one or more message-passing rounds (204). In a graph where each node is connected to the others, it may not be necessary to perform two or more message-passing rounds. In other examples, such as the example where the graph is a three-dimensional grid graph, not all nodes are connected to each other, so multiple rounds of message-passing may be performed.

[0075]

[0064] After one or more message passing rounds are completed, the planning system 118 may apply one or more GCLs to the node feature vector to generate an embedding for the node (206). After generating the embedding for the node, the planning system 118 may generate an output based on the embedding for the node (208). Based on the output, the planning system 118 may determine one or more recommended prostheses, or recommended values ​​for one or more prosthesis parameters of one or more prostheses (210). For example, the planning system 118 may apply one or more layers of the GCN model 116 (e.g., a pooling layer, a fully connected layer, and a softmax layer) to the embedding to obtain the final vector. In some examples, the features in the final vector correspond to each prosthesis (or a set of mutually compatible prostheses), and each feature in the final vector may indicate a level of confidence (i.e., confidence score) that the recommended prosthesis (or set of mutually compatible prostheses) should be the prosthesis corresponding to that feature. In such cases, the planning system 118 may determine that the recommended prosthesis is the prosthesis corresponding to the highest (or lowest) value feature in the final vector. In some cases, the features in the final vector correspond to possible combinations of possible values ​​for one or more prosthesis parameters of one or more prostheses. In such cases, the planning system 118 may determine that the recommended values ​​for one or more prosthesis parameters of one or more prostheses are the values ​​of one or more prosthesis parameters in the combination of values ​​corresponding to the highest (or lowest) value feature in the final vector.

[0076]

[0065] In some examples, the planning system 118 may use a regression process to determine one or more recommended prostheses or recommended values ​​for one or more prosthesis parameters based on the output of the GCN model 116. For example, in such an example, the output of the final layer of the GCN model 116 does not indicate a level of confidence that the recommended prosthesis should be a particular prosthesis. Rather, after the GCL layer and readout layer, the GCN model 116 may include a final multilayer perceptron (MLP) that outputs one or more dimensions of a suitable prosthesis (e.g., width, length, height, size, glenosphere size, glenosphere radius, glenosphere offset, stem size, headsphere size, etc.). Thus, the GCN model 116 as a whole outputs one or more dimensions of a suitable prosthesis. In some examples, the first layer of the final MLP may contain eight neurons (as mentioned above, in some examples the second GCL layer has eight dimensions), and the second layer of the final MLP may contain two neurons. The planning system 118 can then determine one or more recommended prostheses from among several available prostheses based on one or more dimensions (e.g., width and length). For example, the planning system 118 may determine one or more recommended prostheses as the prosthesis with the smallest dimensional difference from the dimensions output by the GCN model 116 without violating one or more constraints. Such constraints may include avoiding situations where the edge of the prosthesis extends beyond the corresponding edge of the bone. In this way, the planning system 118 can use the output of the GCN model 116 to perform a regression of the dimensions output by the GCN model 116 to one of the available prostheses. In this way, the planning system 118 can filter one or more recommended prostheses from among several available prostheses. In other words, the planning system 118 may eliminate or lower the priority of prostheses that have a larger dimensional difference from the dimensions output by the GCN model 116. The planning system 118 may display a user interface that provides information about recommended prostheses.Presenting information on multiple recommended prostheses can enable surgeons to compare recommended prostheses and make experience-based decisions when selecting a prosthesis for implantation in a patient. In some examples, the execution system 112 manufactures a patient-specific prosthesis to have dimensions output by the GCN model 116. In examples where the GCN model 116 outputs appropriate prosthesis dimensions, the ground truth in the training data used to train the GCN model 116 may specify appropriate prosthesis dimensions for different patients.

[0077]

[0066] In some examples, the output of the GCN model 116 (for example, the output of the final MLP of the GCN model 116) includes values ​​for one or more prosthesis parameters of the appropriate prosthesis. For example, in some such examples, the first layer of the final MLP may include eight neurons, and the second layer of the final MLP may include neurons for each of the prosthesis parameters. The prosthesis parameters of the appropriate prosthesis may characterize the shape and / or other properties of the appropriate prosthesis. In some examples, the prosthesis parameters may include the width and length of the appropriate prosthesis. In other examples, the prosthesis parameters may include a set of prosthesis parameters for two or more cross-sections of bone. A set of prosthesis parameters for cross-sections of bone may characterize the shape of the appropriate prosthesis in the cross-section. For example, a set of prosthesis parameters for cross-sections of bone may include a given number (e.g., four, eight, twelve, etc.) of prosthesis parameters, each representing the distance from the center of the cross-section to the edge of the appropriate prosthesis. In one example where there are eight prosthesis parameters for the cross-section, the distance can correspond to lines radiating from the centers of the cross-sections separated at a 45° angle.

[0078]

[0067] In some examples, where the output of the GCN model 116 includes the values ​​of one or more prosthesis parameters for a suitable prosthesis, the planning system 118 may determine one or more recommended prostheses from among several available prostheses based on the values ​​of one or more prosthesis parameters. For example, the planning system 118 may determine a recommended prosthesis as the prosthesis with the smallest difference between the prosthesis parameter values ​​output by the GCN model 116 and the values ​​of the prosthesis parameters output by the GCN model 116, without violating one or more constraints. In this way, the planning system 118 may use the output of the GCN model 116 to perform a regression of the dimensions output by the GCN model 116 to one of the available prostheses. In some examples, the planning system 118 outputs information about one or more recommended prostheses for display. The planning system 118 may receive user input instructions to select one of the recommended prostheses or another of the available prostheses. In some examples, the execution system 122 uses data generated by the planning system 118 to select a prosthesis from a rack of available prostheses for packaging and shipping. In some examples, the execution system 122 manufactures a patient-specific prosthesis based on the prosthesis parameter values ​​output by the GCN model 116. In examples where the GCN model 116 outputs the prosthesis parameter values ​​for the appropriate prosthesis, the ground truth in the training data used to train the GCN model 116 may specify the prosthesis parameters for the appropriate prosthesis for different patients.

[0079]

[0068] Figures 3A, 3B, and 3C are conceptual diagrams showing exemplary cross-sections 300A, 300B, and 300C (collectively, “cross-sections 300”) of a model of bone 302 by one or more techniques of the present disclosure. In detail, the examples in Figures 3A, 3B, and 3C show three different cross-sections of a patient’s tibia. Each of the cross-sections 300 may correspond to a node in a graph. The model of bone 302 may represent the current (e.g., diseased) shape of bone 302.

[0080]

[0069] For each of sections 300A, 300B, and 300C, the planning system 118 may determine the medial widths 304A, 304B, and 304C (collectively, "medial width 304") and the lateral widths 306A, 306B, and 306C (collectively, "lateral width 306"). The medial width 304 and lateral width 306 are defined by the distance between the projection point 308 and a point on the cortical bone of the tibia. The projection point 308 may be defined as the projection of the tibia plafond onto section 300.

[0081]

[0070] Furthermore, for each of the sections 300A, 300B, and 300C, the planning system 118 may determine the anterior lengths 310A, 310B, and 310C (collectively, "anterior length 310") and the posterior lengths 312A, 312B, and 312C (collectively, "posterior length 312"). The anterior length 310 and the posterior length 312 are defined by the distance between the projection point 308 and a point on the cortical bone of the tibia.

[0082]

[0071] For each of the cross-sections 300, the planning system 118 may determine a first feature of the feature vector of the node corresponding to the cross-section by adding the inner width and outer width of the cross-section. The planning system 118 may determine a second feature of the feature vector of the node corresponding to the cross-section by adding the front width and outer width of the cross-section.

[0083]

[0072] Figure 4 is a conceptual diagram showing an exemplary graph 400 that may be used with a GCN model according to one or more techniques of the present disclosure. In the example of Figure 4, circles represent nodes of graph 400. Lines between nodes represent edges of graph 400. In the example of Figure 4, there are 10 nodes, and therefore graph 400 may be suitable for use with 10 cross-sections of a bone, or 10 cross-sections of two or more sets of bones.

[0084]

[0073] Figures 5A, 5B, 5C, and 5D are conceptual diagrams showing exemplary cross-sections 500 of a model of the talus 502 by one or more techniques of the present disclosure. The model of the talus 502 may represent the current (e.g., diseased) shape of the talus 502. Each of the cross-sections 500 may correspond to a node on a graph. Furthermore, each of the cross-sections 500 may be perpendicular to the functional axis of the talus 502. In some examples, the cross-sections 500 are calculated at a depth of 5 mm to 8 mm along the functional axis of the talus 502. Each of the cross-sections 500 may be spaced only 1 mm apart.

[0085]

[0074] For each of the sections 500A, 500B, and 500C, the planning system 118 may determine the medial widths 504A, 504B, 504C, and 504D (collectively, "medial width 504") and the lateral widths 506A, 506B, 506C, and 506D (collectively, "lateral width 506"). The medial width 504 and lateral width 506 are defined by the distance between the projection point 508 and a point on the cortical bone of the talus 502. In some examples, the projection point 508 may be defined by determining the centroids of two high-point landmarks of the talus 502 and projecting those centroids onto the section 500 along the functional axis of the talus 502.

[0086]

[0075] Furthermore, for each of the sections 500A, 500B, 500C, and 500D, the planning system 118 may determine the anterior lengths 510A, 510B, 510C, and 510D (collectively, the "anterior length 510") and the posterior lengths 512A, 512B, 512C, and 512D (collectively, the "posterior length 512"). The anterior length 510 and the posterior length 512 are defined by the distance between the projection point 508 and a point on the cortical bone of the tibia.

[0087]

[0076] For each of the cross-sections 500, the planning system 118 may determine a first feature of the feature vector of the node corresponding to the cross-section by adding the inner width and outer width of the cross-section. The planning system 118 may determine a second feature of the feature vector of the node corresponding to the cross-section by adding the front width and outer width of the cross-section.

[0088]

[0077] Figure 6 is a block diagram showing an example of a planning system 118 using primary and secondary GCNs according to one or more techniques of the present disclosure. In some examples, the recommendation of one prosthesis may depend on the recommendation of another prosthesis. For example, the recommendation of a talus prosthesis may depend on the recommendation of a tibial prosthesis. In other words, the planning system 118 may determine which talus prosthesis should be recommended based in part on which tibial prosthesis is recommended (or selected by the user). In another example, the recommendation of a humerus prosthesis may depend on a glenoid prosthesis. For simplicity of explanation, when a first prosthesis needs to be selected / recommended before a second prosthesis, the present disclosure may refer to the first prosthesis as the primary prosthesis and the second prosthesis as the secondary prosthesis. The primary prosthesis may be designed for implantation in a first bone, and the secondary prosthesis may be designed for implantation in a second bone. A potential challenge lies in how machine learning models can account for this dependency between proteses. For example, a single machine learning model that predicts both primary and secondary proteses from a single set of input data may be difficult to train and / or may contain an excessive number of weights.

[0089]

[0078] According to one or more techniques of the present disclosure, the planning system 118 may include primary feature data 612, primary graph data 614, and primary GCN model 616. Furthermore, the planning system 118 may include secondary feature data 622, secondary graph data 624, and one or more secondary GCN models 626A to 626N (collectively, "GCN model 626"). The primary feature data 612, primary graph data 614, and primary GCN model 616 may be the same as the feature data 112, graph data 114, and GCN model 116. The planning system 118 may use the primary feature data 612, primary graph data 614, and primary GCN model 616 in the same manner as the feature data 112, graph data 114, and GCN model 116 to generate a first output. Based on the first output, the planning system 118 may determine a recommended primary prosthesis for a first implantation on bone.

[0090]

[0079] The secondary feature data 622 characterizes one or more aspects of the second bone in a second set of cross-sections of the second bone. Furthermore, the secondary graph data 624 can define a secondary graph structured in one of a variety of ways. For example, in some examples, the secondary graph may include sampling nodes corresponding to cross-sections of the second bone (i.e., cross-section nodes corresponding to different cross-sections in the set of cross-sections of the second bone), in a manner similar to, for example, Figure 4. In other examples, the secondary graph may include sampling nodes arranged in a three-dimensional grid. For each cross-section node in the secondary graph, the first feature data included in the feature vector for the cross-section node characterizes one or more aspects of the second bone in the cross-section corresponding to that cross-section node.

[0091]

[0080] For example, in one example where the sampling nodes correspond to cross-sections of a second bone and the second bone is the talus, the aspects of the second bone may include the length and width of the talus in each of the cross-sections 500, as shown in the example in Figure 5. The secondary graph data 624 defines a second graph including a second plurality of nodes and one or more second edges. Each of the second plurality of nodes corresponds to a different cross-section in the cross-section of the second bone. For each node in the second graph, the feature vector for the node initially includes feature data that characterizes one or more second aspects of the second bone in the cross-section of the second bone corresponding to that node. In one example where the second graph is a three-dimensional grid, the sampling nodes of the second graph may correspond to regions in the second bone (or regions in the image of the second bone).

[0092]

[0081] Each of the secondary GCN models 626 may correspond to a different available primary prosthesis. For example, there may be multiple tibial prostheses of different sizes. Therefore, there may be one secondary GCN model for each tibial prosthesis among multiple tibial prostheses. Thus, after determining a recommended primary prosthesis (or after receiving user input instructions for selecting a primary prosthesis), the planning system 118 may select one of the secondary GCN models from the GCN models 626 based on the first prosthesis. After selecting a second GCN model, the planning system 118 may apply the secondary GCN model to a second graph and secondary feature data 622 to generate a second output. In some examples, the planning system 118 may determine a recommended second prosthesis for the patient based on a second output from a set of second prostheses in which the secondary GCN model fits the corresponding primary prosthesis. In this way, the recommended second prosthesis must fit the primary prosthesis. In some examples, the planning system 118 determines recommended values ​​for one or more prosthesis parameters that specify the configuration of the second prosthesis, based on the second output. For example, the output of the secondary GCN model may include a vector containing feature values ​​corresponding to different combinations of possible values ​​for the prosthesis parameters of the second prosthesis. The planning system 118 may determine recommended values ​​for the prosthesis parameters of the second prosthesis based on the values ​​in the output vector of the secondary GCN model. The planning system 118 may apply the secondary GCN model in the same manner as described above for applying the GCN model. The planning system 118 may determine one or more recommended prostheses or recommended values ​​for one or more prosthesis parameters in the same manner as described above with respect to the GCN model.

[0093]

[0082] Figure 7 is a conceptual diagram illustrating an exemplary process for recommending a talar prosthesis on a partial basis to a recommended tibial prosthesis using one or more techniques of the present disclosure. In the example of Figure 7, there are eight available tibial prostheses 700 labeled “1”, “2*”, “3”, “3Long”, “4”, “4Long”, “5”, and “5Long”. There are five available talar prostheses 702A, 702B, 702C, 702D, and 702E (collectively “talar prosthesis 702”) having different talar dome sizes. For any given one of the tibial prostheses 700, the user or planning system 118 may select from two of the talar prostheses 702. In other words, talus prostheses 702A and 702B fit tibial prostheses 700A (i.e., tibial prostheses 1 and 2*), talus prostheses 702B and 702C fit tibial prostheses 700B (i.e., tibial prostheses 3 and 3Long), talus prostheses 702C and 702D fit tibial prostheses 700C (i.e., tibial prostheses 4 and 4Long), and talus prostheses 702D and 702E fit tibial prostheses 700D (i.e., tibial prostheses 5 and 5Long). Furthermore, polyethylene (poly) joint components of different sizes may be attachable to different tibial prostheses 700.For example, either 1 or 1+ poly-joint components may be used with the tibial prosthesis 700A and talus prosthesis 702A, poly-joint component 2 may be used with the combination of tibial prosthesis 700A and talus prosthesis 702B, poly-joint component 2+ may be used with the combination of tibial prosthesis 700B and talus prosthesis 702B, and poly-joint component 3 may be used with the combination of tibial prosthesis 700B and talus prosthesis 702C. Poly joint component 3+ may be used in combination with tibial prosthesis 700C and talus prosthesis 702C, poly joint component 4 may be used in combination with tibial prosthesis 700C and talus prosthesis 702D, poly joint component 4+ may be used in combination with tibial prosthesis 700D and talus prosthesis 702D, and poly joint component 5 may be used in combination with tibial prosthesis 700D and talus prosthesis 702E.

[0094]

[0083] In some cases, the planning system 118 may apply a primary GCN model 616 to determine which of the tibial prostheses 700 should be recommended. Different secondary GCN models 626 may correspond to different sets of one or more tibial prostheses 700. For example, if one of the tibial prostheses 700A is the recommended tibial prosthesis, the planning system 118 may use a first secondary GCN model to determine which of the talus prostheses 702A or 702B (and, in some cases, polyarticular components) should be recommended; if one of the tibial prostheses 700B is the recommended tibial prosthesis, the planning system 118 may use a second secondary GCN model to determine which of the talus prostheses 702B or 702C (and, in some cases, polyarticular components) should be recommended; and so on.

[0095]

[0084] In some examples, there is a single GCN model (for example, GCN model 116), and the final vector generated based on the output of the GCN model includes feature values ​​corresponding to different combinations of tibial prosthesis, talar prosthesis, and polysize. For example, the final vector may include a first feature corresponding to tibial prosthesis 1, polysize 1, and talar prosthesis 702A; a second feature corresponding to tibial prosthesis 1, polysize 1+, and talar prosthesis 702A; a third feature corresponding to tibial prosthesis 1, polysize 2, and talar prosthesis 702B; a fourth feature corresponding to tibial prosthesis 2*, polysize 1, and talar prosthesis 702A; a fifth feature corresponding to tibial prosthesis 2*, polysize 1+, and talar prosthesis 702A; a sixth feature corresponding to tibial prosthesis 2*, polysize 1, and talar prosthesis 702A, and so on. Since the final vector includes only features corresponding to mutually compatible prostheses, the planning system 118 will not recommend a set of mutually incompatible prostheses or a set of mutually incompatible prosthesis parameters.

[0096]

[0085] Figure 8 is a conceptual diagram showing an exemplary graph 800 that may be used with a GCN model according to one or more techniques of the present disclosure. Graph 800 is similar to graph 400 (Figure 4) but contains fewer nodes. The planning system 118 may use graph 400 in cases where 10 sections are used, and may use graph 800 in cases where 4 sections are used. For example, the planning system 118 may use 10 sections when recommending a tibial prosthesis, and 4 sections when recommending a corresponding, compatible talar prosthesis.

[0097]

[0086] Figure 9 is a flowchart illustrating exemplary operation of a computing system 100 in which two interacting prostheses are recommended using one or more techniques of the present disclosure. In the example of Figure 9, the planning system 118 may determine a first recommended prosthesis, or recommended values ​​for one or more prosthesis parameters of a first prosthesis, for example, a prosthesis for implantation in a first bone, such as bone 302 (900). The planning system 118 may determine the first recommended prosthesis, or the recommended values ​​for one or more prosthesis parameters of a first prosthesis, in the same manner as described elsewhere, such as with respect to Figure 2.

[0098]

[0087] Furthermore, the planning system 118 may acquire secondary feature data 622 characterizing one or more second aspects of the second bone (e.g., the talus 502) in a second plurality of cross-sections of the second bone (902). A second graph (e.g., graph 800) includes a second plurality of nodes and one or more second edges. The second graph may be defined by secondary graph data 624. The sampling nodes of the second graph may correspond to different sampling locations. For example, one or more sampling nodes among the second set of nodes correspond to different cross-sections of the second bone (e.g., different cross-sections within cross-section 500). In some examples, one or more sampling nodes among the second set of nodes correspond to different regions. For each node in the second graph, the feature vector for the node initially includes feature data (e.g., secondary feature data 622) that characterizes one or more second aspects of the second bone in the cross-section of the second bone corresponding to that node. For example, if the second bone is the talus, one or more second aspects of the talus in each cross-section of the talus include a single measurement of the talus, such as the medial-lateral width of the talus in the cross-section. Thus, in this example, the feature vector for the node might initially include the medial-lateral width of the talus in the cross-section of the talus corresponding to that node.

[0099]

[0088] The planning system 118 may select a second GCN model from a plurality of secondary GCN models (e.g., secondary GCN model 626) based on the first prosthesis (904). For example, the planning system 118 may use a predefined mapping from the first prosthesis to a specific prosthesis in the secondary GCN models. The planning system 118 may use the output of the secondary GCN models mapped to the primary prosthesis to determine a recommendation for a secondary prosthesis that fits the primary prosthesis. For example, there may be two secondary prostheses that fit the primary prosthesis, and the planning system 118 may use the output of the selected secondary GCN model to determine which of these two secondary prostheses should be recommended for the patient. In some examples, there may be only a single secondary GCN model, and the selection of a second GCN model may be omitted.

[0100]

[0089] After selecting a second GCN model, the planning system 118 may apply the second GCN model to a second graph and feature vectors for a second set of nodes to generate a second output (906). As part of applying the second GCN model, the planning system 118 may perform one or more message-passing rounds (908). In a graph where each node is connected to the other nodes, it may not be necessary to perform two or more message-passing rounds. The planning system 118 may then apply one or more GCLs to the feature vectors of the nodes to generate embeddings for the nodes (910). In some examples where the second bone is a talus, the feature vector for each node may be a one-dimensional vector that initially includes the medial-lateral width of the talus in the cross-section of the talus corresponding to the node. In such examples, the first GCL of the second GCN model may increase the dimension of the feature vector from a one-dimensional embedding space to a two-dimensional embedding space. The second GCL of the second GCN model may modify the values ​​in the two-dimensional embedding space.

[0101]

[0090] After generating embeddings for the nodes, the planning system 118 may generate a second output based on the embeddings for the nodes (912). For example, the planning system 118 may apply a pooling layer to the embeddings for the nodes. The pooling layer generates a first intermediate vector having first intermediate features corresponding to prothesis having different sizes. For example, the number of nodes in the graph may be equal to 4, and the final GCL layer may output a 2D embedding for each of the nodes. Thus, there may be 4 × 2 features after the final GCL layer. In this example, the pooling layer may reduce the number of features to 1 × 2. For example, the pooling layer may determine the mean (average) of each of the corresponding features for each of the 4 nodes, resulting in a first intermediate vector containing 2 features.

[0102]

[0091] Furthermore, the planning system 118 may apply a fully connected layer to the first intermediate vector to generate a second intermediate vector. The fully connected layer may contain a certain number of output neurons. The certain number of output neurons may correspond to the number of available secondary prostheses. For example, the output neurons may include output neurons corresponding to secondary prostheses of different sizes. Thus, the second intermediate vector may have second intermediate features corresponding to secondary prostheses of different available sizes. Similarly, in some examples, the output neurons may include output neurons corresponding to different values ​​of one or more prosthesis parameters (e.g., size, radius, etc.) of the second prosthesis. Some or all of the output neurons of the fully connected layer may receive each of the features of the first intermediate vector as input. Each of the output neurons of the fully connected layer may compute the result of a transfer function applied to the weighted sum of the inputs according to machine-learned weights.

[0103]

[0092] Furthermore, in some examples, the planning system 118 may apply a softmax layer to a second intermediate vector to generate an output that includes a final vector. The final vector may include final features corresponding to prothesis having different sizes. The softmax layer may transform the second intermediate vector into a probability distribution of possible outcomes. The softmax layer may normalize the second intermediate vector into a probability distribution across output classes (e.g., available prothesis).

[0104]

[0093] Based on the second output, the planning system 118 may determine recommended values ​​for one or more prosthesis parameters of a second or secondary prosthesis to recommend for implantation in the patient (914). For example, in one example, if the second output includes a final vector containing final features corresponding to prostheses of different sizes, the planning system 118 may determine that the recommended second prosthesis is the prosthesis corresponding to the highest (or lowest) value feature in the final vector. In other examples, the planning system 118 may use a regression process to determine primary and secondary prostheses based on the final vector. In such examples, the regression process may be similar to that described above (with respect to Figure 2, for example). In other examples, the second output may include dimensions and / or parameters of a suitable prosthesis, and the planning system 118 may determine a recommended prosthesis from among several available prostheses based on the dimensions and / or parameters. In some examples, the planning system 118 may determine recommended values ​​for one or more prosthesis parameters of a secondary prosthesis in the same manner as described elsewhere in this disclosure for determining recommended values ​​for prosthesis parameters. In some examples where the second output includes dimensions and / or parameters, the planning system 118 may determine a custom, patient-specific prosthesis based on the dimensions and / or parameters.

[0105]

[0094] Figure 10 is a conceptual diagram showing an exemplary three-dimensional grid graph 1000 by one or more techniques of the present disclosure. In the example of Figure 10, points correspond to nodes and lines correspond to edges. Furthermore, in the example of Figure 10, the grid graph 1000 is a 4×4×4 grid. In other examples, the grid graph 1000 may have other dimensions.

[0106]

[0095] In the example of Figure 10, each node may have up to six edges connected to it. Therefore, for each node, the edges of the grid graph 1000 may include a set of edges that connect the node to up to six other nodes in a group of nodes. In other examples, there may be other limitations on the number of edges that can be connected to a single node. For example, for each node, the edges of the grid graph may include a set of edges that connect the node to up to eight other nodes in a group of nodes. In another example, for each node, the edges of the grid graph may include a set of edges that connect the node to up to twelve other nodes in a group of nodes.

[0107]

[0096] The planning system 118 can obtain a three-dimensional arrangement of sampling locations. In some examples, the three-dimensional arrangement of sampling locations is at least partially predefined. Thus, the sampling locations may include a set of sampling locations corresponding to predefined coordinates in the three-dimensional image data of the bone. In some examples, the three-dimensional arrangement of sampling locations may be a three-dimensional arrangement of equally spaced locations. For example, each sampling location may be spaced at a distance of 2 mm, 5 mm, or another distance in each of the width, length, and height dimensions. In some examples, the distance between sampling locations may differ in one or more dimensions. For example, the distance between sampling locations may be equal to a first value in the width and length dimensions and equal to a second different value in the height dimension. Sampling locations may be defined in coordinates to specific anatomical landmarks (e.g., the distal point on the tibia, the centroid of the talus, etc.) or user-defined locations.

[0108]

[0097] In other examples, the three-dimensional arrangement of sampling locations is not predefined. For example, in some examples, the planning system 118 may apply a segmentation process to bone image data (e.g., two-dimensional or three-dimensional image data) to identify multiple regions. Region borders may correspond to anatomical boundaries. Regions may correspond to sampling locations among multiple sampling locations. Therefore, sampling locations may correspond to different regions. Two or more of the regions may have different sizes. In some examples, the segmentation process is an over-segmentation process that identifies regions even within anatomical structures. For example, the segmentation process may identify multiple three-dimensional regions within the tibia. In some examples, the oversegmentation process used by the planning system 118 may be the so-called SLIC algorithm (Radhakrishna Achanta, Appu Shaji, Kevin Smith, Aurelien Lucchi, Pascal Fua, and Sabine Susstrunk, SLIC Superpixels Compared to State-of-the-art Superpixel Methods, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 34, No. 11, pp. 2274-2282, May 2012).

[0109]

[0098] In some examples where the planning system 118 applies a segmentation process, the number of regions identified by the segmentation process is not available before the segmentation process is applied. Therefore, the planning system 118 can adaptively generate graph data 114 such that the number of nodes in the graph is determined based on (e.g., equal to, proportional to, etc.) the number of regions identified by the segmentation process. In some examples, the planning system 118 does not generate nodes for regions of no interest. For example, in some examples, the planning system 118 does not generate nodes for regions corresponding to soft tissue, open space, bones of no interest, etc. Examples of bones of no interest may include the fibula, navicular bone, calcaneus, etc., in a use case involving total ankle replacement. Excluding nodes for regions of no interest can reduce the graph size and therefore reduce the processing time when applying the GCN model 116. In other examples, the number of regions identified by the segmentation process can be predetermined.

[0110]

[0099] Furthermore, for each region, the planning system 118 may generate edges connecting the node corresponding to the region to the node corresponding to the region adjacent to that region. In some examples, the planning system 118 may limit the number of edges connecting to any particular node to a certain limit. In other words, the planning system 118 may generate edges such that it is not possible for more than a certain number of nodes to connect to any given node. For example, the number of edges that are allowed to connect to a given node may be limited to 6, 8, 12, etc.

[0111]

[0100] The planning system 118 can generate feature data for each node. For example, for each node, the first feature data included in the feature vector for the node characterizes one or more aspects of the bone at the sampling location corresponding to the node.

[0112]

[0101] In an example where the planning system 118 applies a segmentation process to three-dimensional image data of a bone to identify multiple regions, the planning system 118 may generate a feature vector of a node corresponding to the sampling location corresponding to the region for at least one of the regions, based on data associated with one or more pixels of the three-dimensional image data within the region. In some examples, one or more pixels of the three-dimensional image data include multiple pixels of the three-dimensional image data. Furthermore, in some examples, the region is a three-dimensional region where the coordinates of two or more of the multiple pixels contained within the region are different. In other words, pixels in a three-dimensional region do not necessarily form a straight line in one of their dimensions.

[0113]

[0102] In some examples, the first feature data included in the feature vector for a node may include a measure of bone density for the sampling location corresponding to the node. The measure of bone density may be the average of bone density measurements (e.g., measured with a Hounsfield unit) within the region corresponding to the node, or other numbers based on bone density measurements. In other examples, the feature data for a node may include a measure of bone density for a single point (e.g., the centroid of the region corresponding to the node, a predefined location, etc.).

[0114]

[0103] In some examples, for one or more of the nodes, the first feature data included in the feature vector for the node may describe the local texture of the bone (e.g., cortex, spongy tissue, etc.). The description of the local texture may be the average (or other statistical value) of the values ​​indicating the local texture at points in the region corresponding to the node. In other examples, the description of the local texture may correspond to a single point (e.g., the centroid of the region corresponding to the node, a predefined location, etc.).

[0115]

[0104] In some examples, for one or more of the nodes, the first feature data included in the feature vector for the node includes a gray-level co-occurrence matrix. The gray-level co-occurrence matrix (GLCM) is based on gray-level pixel values, sometimes called intensity values. Intensity values ​​may correspond to bone density measurements or other information. Generally, the planning system 118 may use predefined spatial relationships to generate the GLCM. Predefined spatial relationships define the relationship between a reference pixel and neighboring pixels. Examples of spatial relationships may include neighboring pixels one pixel to the right of a reference pixel, neighboring pixels three pixels above a reference pixel, neighboring pixels two pixels above and two pixels to the left of a reference pixel, and so on. The planning system 118 may generate a matrix (i.e., the GLCM) with a size of (intensity range × intensity range), where each cell of the matrix is ​​initialized to 0. For example, for an 8-bit single-channel image, the planning system 118 may generate a 256 × 256 matrix. The planning system 118 can traverse across an image, and for each unique ordered pair of intensity values ​​found for a defined spatial relationship, the planning system 118 may increment a cell in a matrix having the coordinates corresponding to the ordered pair. For example, if the spatial relationship is that a neighboring pixel is one pixel to the right of a reference pixel, and there are four instances in the image where the neighboring pixel and the reference pixel have values ​​of 1 and 3, the planning system 118 sets the value in the matrix at coordinates (1,3) to equal 4. In some examples, the gray-level co-occurrence matrix in the node's feature vector may represent the texture within the region corresponding to the node. In other words, the “image” from which the planning system 118 computes the GLCM is the region corresponding to the node. In some examples, the image from which the planning system 118 generates the GLCM for a node may include the entire image or a subset of the image. The use of a grid graph may be advantageous in some use cases because the feature vectors of the nodes in the grid graph may contain types of information other than distance and dimension, which can increase the recommendation quality.

[0116]

[0105] As described above, there are several challenges associated with existing computerized systems for recommending (e.g., filtering) prostheses. In addition to the GCN model described above, this disclosure also describes generative pre-trained transformer (GPT)®-based techniques for recommending prostheses or recommended prosthesis parameters. GPT-based techniques enable computing systems to efficiently consider the interrelationships between bone morphologies at various bone sampling locations.

[0117]

[0106] Figure 11 is a block diagram showing exemplary components of a planning system 118 according to one or more techniques of the present disclosure. In the example of Figure 11, the components of the planning system 118 include a machine learning model 1130, a prediction unit 1102, a training unit 1104, and a feature enhancement unit 1106. The machine learning model 1130 includes a feature encoder 1110, a quantization module 1112, a feature decoder 1114, a generative pre-trained transformer (GPT) 1134, and a codebook 1118. In other examples, the planning system 118 may be implemented using more, fewer, or different components. For example, the training unit 1104 may be omitted if the ML model 1130 has already been trained. In some examples, one or more of the components of the planning system 118 are implemented as software modules. Furthermore, the components of Figure 11 are provided as examples, and the planning system 118 may be implemented in other ways.

[0118]

[0107] Generally, the prediction unit 1102 applies the ML model 1130 to an input anatomical model to determine one or more recommended prostheses or recommended values ​​for one or more prosthesis parameters for a patient. The feature encoder 1110 of the ML model 1130 generates a feature graph and input features based on the input anatomical model. The input anatomical model can be a grid graph, such as grid graph 1000. The feature encoder 1110 may then apply a graph convolution encoder to the feature graph and input features to generate a surface embedding. The quantization module 1112 uses the codebook 1118 to generate a quantized surface embedding based on the surface embedding. The prediction unit 1102 may sequence the quantized surface embedding to form a sequence of input tokens for the GPT 1134. The GPT 1134 generates a sequence of output tokens (e.g., a predicted codebook index) based on the sequence of input tokens (i.e., the sequence of quantized surface embeddings). The prediction unit 1102 uses the codebook 1118 to generate a sequence of quantized surface embeddings based on the predicted codebook index (e.g., a sequence of output tokens). The feature decoder 1114 uses the sequence of quantized surface embeddings to generate the predicted model.

[0119]

[0108] The training unit 1104 can train the ML model 1130. The training unit 1104 can use a set of training examples 1132 to train the ML model 1130. In one or more examples, the training unit 1104 may be configured to train the ML model 1130 using multiple training epochs. The feature enhancement unit 1106 may generate synthetic training examples to enhance the training examples 1132. In some examples, the ML model 1130 may be trained by a device or system other than the computing system 100. In such examples, the planning system 118 may receive the ML model 1130 including GPT 1134.

[0120]

[0109] Figure 12 is a flowchart illustrating exemplary operation of the planning system 118 using one or more techniques of the present disclosure. In the example of Figure 12, the planning system 118 may obtain an input anatomical model comprising a first 3D mesh representing the surface of at least a first portion of the patient's anatomical structure (1200). The planning system 118 may generate input tokens based on the input anatomical model (1202). The planning system 118 may apply GPT 1134 to the input tokens to generate output tokens (1204). Based on the output tokens, the planning system 118 may determine one or more recommended prostheses or recommended values ​​for one or more prosthesis parameters for the patient (1206).

[0121]

[0110] Figure 13 is a block diagram illustrating an exemplary architecture for generating quantized surface embeddings and reconstructing an anatomical model using one or more techniques of the present disclosure. In the example of Figure 13, the feature encoder 1110 includes a feature generator 1300 and a graph convolution encoder 1302. The feature decoder 1114 includes a decoder 1342. The architecture of Figure 13 may be used to train the graph convolution encoder 1302 and the decoder 1342 separately from the GPT 1134.

[0122]

[0111] The feature encoder 1110 obtains an input anatomical model 1304 representing one or more anatomical structures of a patient (e.g., bone or soft tissue structures). The feature generator 1300 generates a face graph and input features 1306 based on the input anatomical model 1304. In the face graph, each face corresponds to a node in the graph. Nodes corresponding to neighboring faces are connected in the graph by undirected edges. Each node has a feature vector. For each node, the node's feature vector may include nine coordinate values ​​for each vertex of the corresponding face (i.e., three coordinate values ​​for each of the three dimensions), data defining the face normal vector, data defining the angles between the edges of the corresponding face, and data indicating the area of ​​the corresponding face.

[0123]

[0112] The graph convolution encoder 1302 generates a face embedding 1308 based on the face graph and the input features 1306. The graph convolution encoder 1302 performs one or more rounds of message passing. Between each round of message passing, the graph convolution encoder 1302 collects the feature vectors of neighboring nodes for each node in the graph. The graph convolution encoder 1302 then updates the node's feature vector by aggregating it with the feature vectors of neighboring nodes. Aggregating the feature vectors may involve determining the mean of the corresponding features in the feature vectors. After completing one or more rounds of message passing, the graph convolution encoder 1302 may use the updated feature vector of the node as input for a neural network 1307 trained to generate a face embedding for the face corresponding to the node for each node in the graph. In this way, the graph convolution encoder 1302 can generate a face embedding for each face of the input anatomical model.

[0124]

[0113] In some examples, the neural network 1307 includes a series of graph convolutional layers, such as a SAGE-Conv graph convolutional layer. In some such examples, the first layer of the neural network 1307 takes 16 features as input: 9 features representing the point coordinates of a face, 3 features representing the normal vector of a face, 3 features representing the angles at the vertices of a face, and 1 feature representing the area of ​​a face.

[0125]

[0114] The quantization module 1112 generates a quantized face embedding 1328 based on the face embedding 1308. In the example in Figure 13, the quantization module 1112 includes a feature partitioning unit 1320 configured to partition each face embedding 1308 of a face into a plurality of subvectors 1312. Each of the subvectors 1312 may correspond to a vertex of a face. The aggregation unit 1322 of the quantization module 1112 may then aggregate the subvectors 1312 for the corresponding vertices into an aggregated subvector 1323. For example, each vertex of a model may have an index. Thus, two faces that share a vertex and information about the faces may show the same index for the shared vertex. When aggregating the subvectors for the corresponding vertices, the quantization module 1112 may calculate the average of the features in the subvectors for vertices that have the same index.

[0126]

[0115] For each of the aggregated sub-vectors 1323, the codebook lookup unit 1324 of the quantization module 1112 may look up a codebook index in the codebook 1118 for the aggregated sub-vector. The codebook 1118 contains multiple representative vectors. Each representative vector may have the same number of elements as each of the aggregated sub-vectors. Each representative vector is associated with a unique codebook index. In order to look up a codebook index in the codebook 1118 for the aggregated sub-vector, the codebook lookup unit 1324 may determine which of the representative vectors is closest to the aggregated sub-vector. For example, the codebook lookup unit 1324 may treat the aggregated sub-vector and the representative vector as specified points in multidimensional space, calculate the Euclidean distance between the aggregated sub-vector and the representative vector, and determine which of the representative vectors has the lowest Euclidean distance. In this way, the codebook lookup unit 1324 may output multiple codebook indices 1326 for each face. Furthermore, in some examples, the codebook lookup unit 1324 may apply a tiered, residual quantization process to determine multiple codebook indices for each aggregated sub-vector of a face. In some examples, representative vectors in codebook 1118 are learned. The training unit 1104 may use the k-means clustering algorithm to learn representative vectors. In examples where the training unit 1104 uses the k-means clustering algorithm to learn representative vectors, the representative vectors may be cluster center vectors indicating the centers of clusters.

[0127]

[0116] The reshaper unit 1310 of the quantization module 1112 uses the codebook index 1326 to generate the quantized face embedding 1328. That is, the reshaper unit 1310 receives a stack of the codebook index 1326 for each face. For each face, the reshaper unit 1310 can reduce the stack of the codebook index for the face to a single feature embedding for each face by summing across representative vectors and concatenating across vertices, thereby generating the quantized face embedding 1328 for the face. For example, the reshaper unit 1310 may use the following formula to determine the quantized face embedding 1328.

[0128]

number

[0129] In the above formula,

[0130]

number

[0131] This shows a vector containing quantized surface embeddings 1328 for the surface,

[0132]

number

[0133] This shows the individual quantized surface embeddings of surfaces 1 to N.

[0134]

number

[0135] This is the d-th codebook index (i.e., token) of vertex v on face i,

[0136]

number

[0137] This is a codebook index.

[0138]

number

[0139] The representative vector in codebook 1118 corresponding to this is shown.

[0140]

number

[0141] This shows the connectivity of features for vertices v=0 to v=2 of face i.

[0142]

[0117] The feature decoder 1114 includes a sequence generator 1340 and a decoder 1342. The sequence generator 1340 generates a sequence 1344 of quantized surface embeddings for a surface. The quantized surface embeddings in the sequence 1344 can be sequenced in the same way as the surface embeddings 1308.

[0143]

[0118] The decoder 1342 generates a reconstructed anatomical model 1346 based on the sequence of quantized face embeddings 1344. For example, the decoder 1342 outputs a set of nine coordinate values ​​for each face of the reconstructed anatomical model 1346. The reconstructed anatomical model 1346 is a reconstructed version of the input anatomical model 1304. Therefore, the reconstructed anatomical model 1346 does not contain additional information such as information indicating one or more recommended prostheses or recommended values ​​for one or more prosthesis parameters. The decoder 1342 can be implemented as a 1D ResNet decoder. In some examples, the ResNet decoder includes 34 layers, i.e., the decoder 1342 can be implemented using the ResNet34 model. In other examples, other decoders may be used, such as the ResNet18 model, the ResNet101 model, or another type of neural network model. In general, the decoder 1342 may be capable of reconstructing structures with more variability if the decoder 1342 has more layers. In examples where the ML model 1130 is used only to generate a limited number of structures, such as the scapula or tibia, an implementation of the decoder 1342 with 18 or 34 layers may suffice, in contrast to the 101-layer ResNet101 model. Using fewer layers can save computing resources (e.g., processing cycles and memory) if performance is similar. Inference time can also be reduced in models with fewer layers.

[0144]

[0119] Figure 14 is a flowchart illustrating the exemplary operation of the prediction unit 1102 for generating information indicating one or more recommended prostheses or recommended values ​​for one or more prosthesis parameters for a patient using one or more techniques of the present disclosure. In the example of Figure 14, the feature encoder 1110 acquires an input anatomical model as described above with respect to Figure 13 and generates a plane embedding based on the input anatomical model (1400). Furthermore, the planning system 118 applies the graph convolution encoder 1302 to the feature graph and input features to generate a plane embedding (1402). The quantization module 1112 generates a quantized plane embedding 1328 based on the plane embedding (1404).

[0145]

[0120] The prediction unit 1102 can then generate a sequence of features (i.e., a sequence of input tokens) based on the quantized surface embeddings (1406). The sequence of features may include features for each of the quantized surface embeddings. Features for a quantized surface embedding may be based on the quantized surface embedding and learned discrete position coding for the quantized surface embedding. Learned discrete position coding for a quantized surface embedding provides information about the position of the quantized surface embedding in the sequence of features and the index of each embedding in the quantized surface embedding. In other words, for each embedding in the quantized surface embedding, a position coding value for the embedding is determined and then added to (i.e., summed) the embedding. In some examples, a function for determining the position coding value is learned as part of the training process for GPT1134. In some examples, the function for determining the position coding value includes a sinusoidal function. Furthermore, the prediction unit 1102 may include a start embedding at the beginning of the sequence and a end embedding at the end of the sequence. The start embedding indicates the beginning of the sequence. Termination embedding indicates the end of a sequence.

[0146]

[0121] The prediction unit 1102 may autoregressively apply GPT 1134 to a sequence of features to generate output tokens (1408). The sequence of features may include quantized face embeddings. The output tokens may include a sequence of codebook indices for faces of at least a portion of an anatomical structure or prosthesis. For example, the prediction unit 1102 may apply GPT 1134 to a sequence of features to generate a predicted set of codebook indices for a face. If the predicted set of codebook indices does not include a stopping token, the prediction unit 1102 may use codebook 1118 to determine a quantized face embedding for the face. The prediction unit 1102 may then update the sequence of quantized face embeddings to include a quantized face embedding. The prediction unit 1102 may use the updated sequence of quantized face embeddings as input to GPT 1134 to generate the next predicted set of codebook indices. This cycle may continue until the predicted set of codebook indices includes a stopping token. GPT1134 may be a GPT-2 medium architecture. That is, the GPT transformer may have 24 multi-head self-aware layers, 16 heads, 768 feature widths, and 4608 context lengths, as described, for example, in Siddiqui et al., "MeshGPT: Generating Triangle Meshes with Decoder-Only Transformers", arXiv:2311.15475v1[cs.CV] November 27, 2023.

[0147]

[0122] In some examples, the prediction unit 1102 may apply GPT 1134 to a first input token and a second input token to generate an output token. The first input token may be based on an input anatomical model (for example, the first input token may include quantized surface embeddings). The second input token may include information describing the patient but not describing the first 3D mesh. Information describing the patient may include one or more of the patient's sex, the patient's age, the patient's scapular glenoid type, pathology of the anatomical structure, or other information describing the patient. Including a second input token may improve the accuracy of the predicted anatomical model 132.

[0148]

[0123] The prediction unit 1102 may determine the sequence of quantized surface embeddings based on the codebook index (1410). The prediction unit 1102 may determine the quantized surface embeddings in the same manner as described above with respect to Figure 13.

[0149]

[0124] Next, the prediction unit 1102 may apply the feature decoder 1114 to the sequence of quantized surface embeddings to generate a predicted model (1412). The predicted model may include surfaces corresponding to one or more anatomical structures and one or more prostheses. In some examples, the prediction unit 1102 may update the sequence of surface embeddings and the predicted anatomical model 132 when the prediction unit 1102 uses the GPT 1134 to progressively predict an additional set of predicted codebook indices.

[0150]

[0125] The prediction unit 1102 may determine one or more recommended prostheses or recommended values ​​for one or more prosthesis parameters for a patient based on the predicted model (1414). For example, the prediction unit 1102 may compare the dimensions of a prosthesis in the predicted model with the dimensions of available prostheses to determine which of the available prostheses is the closest match. The prediction unit 1102 may determine that the recommended prosthesis is the available prosthesis whose dimensions most closely match those of the prosthesis in the predicted model. In some examples, the prediction unit 1102 may determine recommended values ​​for prosthesis parameters as the values ​​of the corresponding prosthesis parameters for the prosthesis in the predicted model.

[0151]

[0126] Figure 15 is a flowchart illustrating an exemplary training process for ML model 1130 using one or more techniques of the present disclosure. As described above, training unit 1104 can train ML model 1130. That is, training unit 1104 can train ML model 1130 to generate input tokens based on an input anatomical model, apply GPT 1134 of ML model 1130 to the input tokens to generate output tokens, and generate a predicted anatomical model including a second 3D mesh representing the surface of one or more prostheses based on the output tokens.

[0152]

[0127] In the first phase of training the ML model 1130, the training unit 1104 may train the graph convolution encoder 1302 and the feature decoder 1114 to learn representative vectors of the codebook 1118 (1500). In the second phase of training the ML model 1130, the training unit 1104 may train the GPT 1134 (1502).

[0153]

[0128] In the first phase, the training unit 1104 may perform one or more training epochs. In each training epoch, the training unit 1104 may perform a series of training iterations. In each training iteration, the training unit 1104 provides an input anatomical model from training example 1132 as input to the feature encoder 1110. The feature encoder 1110, the quantization module 1112, and the feature decoder 1114 may generate a reconstructed anatomical model based on the input anatomical model. During the first phase, the input anatomical model may be a complete model of one or more anatomical structures with one or more implanted prostheses. After generating the reconstructed anatomical model, the training unit 1104 may calculate a reconstruction loss value based on a comparison between the input anatomical model and the reconstructed anatomical model. For example, the training unit 1104 may calculate the reconstruction loss value using cross-entropy loss on discrete mesh coordinates. The training unit 1104 may use a backpropagation method to update the parameters (e.g., weights) of the feature encoder 1110 and feature decoder 1114 based on the reconstruction loss value.

[0154]

[0129] In some examples, the training unit 1104 updates the representative vectors of the codebook 1118 as part of the training process. For example, the training unit 1104 may add aggregated subvectors 1323 generated during training iterations to the training set of aggregated subvectors. The training unit 1104 may assign weights to each of the aggregated subvectors in the training set such that the longer the aggregated subvectors remain in the training set, the smaller the weights the aggregated subvectors have exponentially. In some examples, the training unit 1104 uses a k-means clustering process to update the representative vectors based on the aggregated subvectors in the training set.

[0155]

[0130] In some examples, the quantization module 1112 applies a tiered, residual vector quantization process. For example, the training unit 1104 may initialize a set of k1 first tier representative vectors, assign the generated aggregated sub-vectors (i.e., first tier vectors) to their nearest first tier representative vectors, and then update the first tier representative vectors based on the average position of the first tier vectors assigned to the first tier representative vectors. The training unit 1104 may repeat the assignment and update steps until a termination condition is reached. For each of the first tier vectors, the training unit 1104 may compute a second tier residual vector that shows the difference between the first tier vector and those first tier representative vectors. Next, the training unit 1104 initializes a set of k2 representative vectors for the second tier, assigns the residual vectors of the second tier to the nearest representative vector of the second tier, updates the representative vectors of the second tier based on the average position of the residual vectors of the second tier assigned to the representative vector of the second tier, and repeats the assignment and update steps until the termination condition is reached. The training unit 1104 may repeat this process for one or more tiers. Therefore, when the codebook lookup unit 1324 receives aggregated sub-vectors (i.e., vectors of the first tier), the codebook lookup unit 1324 determines the codebook index of the representative vector of the first tier for the vector of the first tier, determines the residual vector of the second tier showing the difference between the vector of the first tier and the representative vector of the first tier, determines the codebook index of the representative vector of the second tier for the residual vector of the second tier, determines the residual vector of the third tier showing the difference between the residual vector of the second tier and the representative vector of the second tier, determines the codebook index of the representative vector of the third tier for the residual vector of the third tier, and so on.In this way, for each aggregated sub-vector, the codebook lookup unit 1324 identifies a stack of codebook indices for the aggregated sub-vector.

[0156]

[0131] In some examples, the training unit 1104 applies a straight-through estimator (STE) technique to optimize vector quantization. During training iterations, the training unit 1104 quantizes the surface embedding 1308 to generate the quantized surface embedding 1328. For each of the surface embeddings 1308, the training unit 1104 calculates a commitment loss value that characterizes the difference between the surface embedding and the quantized surface embedding. For example, the training unit 1104 calculates the commitment loss value,

[0157]

number

[0158] It can be calculated as follows, where z is a surface embedding,

[0159]

number

[0160] is the quantized surface embedding, D is the dimensionality of the surface embedding, and sg is the stop gradient operation. For training iterations, after the reconstructed anatomical model 1346 is generated, the training unit 1104 calculates the general loss value as the sum of the reconstructed loss value (as described above) and the commitment loss value. The training unit 1104 uses a backpropagation method to update the parameters of the feature encoder 1110 and feature decoder 1114 based on the general loss value. However, the vector quantization function applied by the quantization module 1112 cannot be learned directly by using backpropagation because the vector quantization function is not differentiable. Therefore, during backpropagation, the vector quantization function is considered an identity function, which allows for modification of the parameters of the neurons in the neural network 1307 that generate the surface embedding 1308. This modifies the values ​​in the surface embedding 1308 that are fed into the vector quantization function applied by the quantization module 1112. This has the effect of reducing the commitment loss value produced by the vector quantization function, as the surface embedding 1308 is ultimately generated such that the aggregated sub-vectors generated from the surface embedding 1308 are closer to the fixed representative vectors in the codebook 1118. The STE technique can be used in conjunction with the tiered residual vector quantization process as described above, but the representative vectors at the different tiers are not updated.

[0161]

[0132] In this way, the feature encoder 1110, the quantization module 1112, and the feature decoder 1114 can be trained to generate a mesh model of an anatomical structure and feature vectors that provide a compact representation of one or more anatomical structures.

[0162]

[0133] As described above, the training unit 1104 may calculate a reconstruction loss value based on a comparison between the input anatomical model and the reconstructed anatomical model. The training unit 1104 may calculate the reconstruction loss value in one of several ways. For example, the training unit 1104 may calculate a mesh regularization loss value (e.g., chamfer loss, Earth's Mover Distance loss (also known as Wasserstein loss), edge regularization loss, normal consistency loss, or Laplacian loss). In some examples, training unit 1104 can calculate curvature-weighted chamfer loss, as described, for example, in Bongratz et al., "Vox2Cortex: Fast Explicit Reconstruction of Cortical Surfaces from 3D MRI Scans with Geometric Deep Neural Networks," arxiv:2203.09446v2[cs.CV] March 18, 2022, where there are areas with high curvature, such as certain areas of the scapula.

[0163]

[0134] In some examples, the training unit 1104 calculates multiple different types of reconstruction loss values ​​and calculates a weighted reconstruction loss value as a weighted sum of the reconstruction loss values. The losses may have different weights depending on the location of a face or point. For example, the training unit 1104 may use different weights for the reconstruction loss values ​​for different areas of an anatomical structure. In some examples, the training unit 1104 calculates a single type of reconstruction loss value for the entire anatomical structure as the sum of the reconstruction losses for individual locations on the initial anatomical model and the reconstructed anatomical model. In some such examples, the training unit 1104 may use different weights for different regions. In this way, further emphasis may be placed on the accurate reconstruction of important areas. For example, a greater weight may be used for the area of ​​the scapula closer to the glenoid fossa than for the area further from the glenoid fossa.

[0164]

[0135] As described above, the training unit 1104 may train the GPT 1134 after completing the training of the feature encoder 1110 and the feature decoder 1114 and the learning of the codebook 1118. In other words, the parameters of the feature encoder 1110, the parameters of the feature decoder 1114, and the learning of the codebook 1118 may be frozen while the GPT 1134 is being trained. The training unit 1104 may train the GPT 1134 using self-attention to autoregressively generate an ordered sequence of triangles that define the final mesh.

[0165]

number

[0166] And,

[0167]

number

[0168] However, the corresponding predicted sequence element is the target token sequence (i.e., the target sequence of the codebook index) T=(t0,t1,...,t N When given, training unit 1104,

[0169]

number

[0170] GPT1134 can be trained using the following loss function, where N is the number of tokens in the target token sequence, D is the number of subvectors per surface embedding, and |C| is the number of entries on Codebook 1118.

[0171]

number

[0172] is the predicted sequence element

[0173]

number

[0174] This is the probability equal to the corresponding token in the target token sequence. Thus, after training the feature encoder 1110 and learning the codebook 1118, the training unit 1104 may use the feature encoder 1110 and the quantization module 1112 to generate quantized surface embeddings based on the training example 1132. As previously mentioned, the training example 1132 may include a model of a healthy or complete anatomical structure.

[0175]

[0136] The training unit 1104 may use training examples 1132 to train the feature encoder 1110, the quantization module 1112, the feature decoder 1114, and the GPT 1134. Training examples 1132 may include exemplary anatomical models, such as anatomical models. Training examples 1132 may include models of the anatomical structures of real patients. For example, in the first training phase, for each first training example in the set of first training examples, the training unit 1104 may apply the feature encoder 1110 to the input anatomical model associated with the training example to generate a surface embedding associated with the first training example. The quantization module 1112 may generate input tokens associated with the first training example based on the surface embedding associated with the first training example. The training unit 1104 may apply the feature decoder 1114 to the input tokens associated with the first training example to generate a reconstructed anatomical model associated with the training example. The training unit 1104 may apply a first loss function to generate a loss value associated with the first training example, based on the input anatomical model associated with the first training example and the reconstructed anatomical model associated with the first training example. The training unit 1104 may perform a first backpropagation process to modify the parameters of the feature encoder and feature decoder based on the loss value associated with the first training example. In the second training phase following the first training phase, for each second training example in the set of second training examples, the training unit 1104 may apply the feature encoder 1110 to generate a second surface embedding associated with the second training example. The training unit 1104 may apply the quantization module 1112 to the surface embedding associated with the second training example to generate input tokens based on the second surface embedding. The training unit 1104 may apply the GPT 1134 to generate output tokens associated with the second training example, based on the input tokens associated with the second training example. The training unit 1104 may apply the feature decoder 1114 to input tokens associated with a second training example in order to generate a predicted anatomical model associated with the training example.The training unit 1104 may apply a second loss function to generate a loss value associated with a second training example. The training unit 1104 may perform a second backpropagation process to modify the GPT parameters based on the loss value associated with the second training example.

[0176]

[0137] In some examples, the feature enhancement unit 1106 performs data augmentation to improve the training of the machine learning model 1130. In some examples, the feature enhancement unit 1106 performs scaling, rotation, mirroring (for example, generating a synthetic left scapula model from an actual right scapula model), unit sphere normalization, jitter, multi-resolution remeshing, or other techniques to generate synthetic training examples. The training unit 1104 may use the synthetic training examples along with the original training examples 1132 for training the machine learning model 1130.

[0177]

[0138] In some examples, the training unit 1104 may perform a mesh decimation process on the anatomical model of training example 1132. For example, the training unit 1104 may use a Quadric Error Metrics (QEM) mesh simplification algorithm to reduce the number of faces in the anatomical model. Reducing the number of faces can reduce training time. In some examples, the training unit 1104 may reduce the number of faces in some areas of the anatomical model but not in other areas of the anatomical model. This can reduce a potential bottleneck in processing circuit capacity and / or reduce the inference time of the mesh completion algorithm when processing meshes derived from high-resolution CT scans.

[0178]

[0139] Figure 16 is a conceptual diagram showing an exemplary surgical planning user interface 1600 that illustrates a surgical proposal for reverse shoulder joint replacement surgery using one or more techniques of the present disclosure. The planning system 118 may generate the user interface 1600 for display (for example, on a display 108). In some examples, the planning system 118 may generate the user interface 1600 after determining one or more recommended prostheses from among several available prostheses.

[0179]

[0140] In the example of Figure 16, the user interface 1600 displays surgical proposals 1602A and 1602B (collectively, "Surgical Proposal 1602") for reverse shoulder joint replacement surgery. Each of the surgical proposals 1602 may correspond to a different recommended prosthesis. In the example of Figure 16, each of the surgical proposals 1602 shows the type of glenoid implant, the diameter of the glenoid implant, the glenosphere diameter and type of the glenoid implant (e.g., central, eccentric, inclined, etc.), the neck-shaft angle of the corresponding humeral implant, the version of the glenoid implant, the seating percentage of the glenoid implant, and the pegging depth of the glenoid implant. The user may select one of the surgical proposals 602. In the example of Figure 16, a black background is used to indicate that surgical proposal 602A is the selected surgical proposal. In other examples, the user interface 1600 may relate to other types of prostheses.

[0180]

[0141] Furthermore, the user interface 1600 includes a superior view 1606, a front view 1608, and a model 1610. The superior view 1606 shows an X-ray image of the patient's shoulder from a superior viewpoint (i.e., viewed downwards from an upper position). The superior view 1606 shows the contour 1612 of the glenoid implant of the type indicated by the selected surgical proposal at the position indicated by the selected surgical proposal. The front view 1608 shows an X-ray image of the patient's shoulder from an anterior viewpoint (i.e., viewed backwards from an anterior position). The front view 1608 shows the contour 1614 of the glenoid implant of the type indicated by the surgical proposal at the position indicated by the selected surgical proposal. The model 1610 shows a 3D model of the patient's scapula along with a phantom image of the glenoid implant. The generation of views 1608 and model 1610 may be computationally complex. Therefore, generating views 1608 and models 1610 for all available prostheses would be undesirable and time-consuming. Rather, the planning system 118 may generate views 1608 and models 1610 for only one or more recommended prostheses. Determining one or more recommended prostheses, as described in this disclosure, can avoid this problem and improve the usability of the planning system 118. The planning system 118 may, upon request, generate views 1608 and models 1610 for other available prostheses.

[0181]

[0142] The following is a non-exclusive list of the provisions of one or more techniques of this disclosure.

[0182]

[0143] Clause 1A. A method comprising: for each node of a plurality of nodes in a graph, a computing system generates a feature vector for the node, first including feature data that characterizes one or more aspects of a patient's bone; a computing system applying a graph convolutional network (GCN) model to the graph and the feature vector for the nodes to generate an output, wherein the graph includes one or more edges, each of which connects each pair of nodes; and a computing system determining a recommended prosthesis for the patient based on the output.

[0183]

[0144] Clause 2A. The method according to Clause 1A, wherein multiple nodes include cross-sectional nodes corresponding to different cross-sections in multiple cross-sections of bone, and for each cross-sectional node in the graph, the first feature data included in the feature vector for the cross-sectional node characterizes one or more aspects of bone in the cross-section corresponding to the cross-sectional node.

[0184]

[0145] Clause 3A. The method according to Clause 2A, wherein for each cross-section of the bone, one or more aspects of the bone in the cross-section include one or more measurements of the dimensions of the bone in the cross-section.

[0185]

[0146] Clause 4A. The method according to Clause 3A, wherein one or more measurements, including measurements of bone dimensions in a cross-section of the bone, include the medial width of the tibia in the cross-section and the anterior length of the tibia in the cross-section.

[0186]

[0147] Clause 5A. The method according to Clause 3A, wherein the bone is the talus and one or more measurements include a medial-lateral width measurement of the talus.

[0187]

[0148] Clause 6A. The method according to any of Clauses 2A to 5A, wherein the cross-section is perpendicular to the functional axis of the bone.

[0188]

[0149] Clause 7A. The method according to any one of Clauses 1A to 6A, wherein GCN model is a first GCN model, bone is a first bone, feature data is first feature data, one or more embodiments is one or more first embodiments, graph is a first graph, plurality of nodes is a first plurality of nodes, recommended prosthesis is a first prosthesis, output is a first output, and the method further comprises: generating a feature vector for the node by a computing system, first including secondary feature data characterizing one or more second embodiments of the patient's second bone for each node of a second plurality of nodes in a second graph; applying a second GCN model to the feature vector for the second plurality of nodes by a computing system to generate a second output, wherein the second graph includes one or more second edges, each of which connects each pair of nodes in the second graph; and determining a recommended second prosthesis for the patient by a computing system based on the second output.

[0189]

[0150] Clause 8A. The method of Clause 7A, further comprising a computing system selecting a second GCN model from among a plurality of secondary GCN models based on a first prosthesis.

[0190]

[0151] Clause 9A. The method according to any one of Clauses 7A to 8A, wherein a second plurality of nodes include cross-sectional nodes corresponding to different cross-sections in a plurality of cross-sections of a second bone, and for each of the cross-sectional nodes in the second graph, the first feature data included in the feature vector for the cross-sectional node characterizes one or more aspects of the second bone in the cross-section corresponding to the cross-sectional node.

[0191]

[0152] Clause 10A. The method according to any of Clauses 7A to 9A, wherein the first prosthesis is designed for implantation in a first bone and the second prosthesis is designed for implantation in a second bone.

[0192]

[0153] Clause 11A. The method according to any of Clauses 7A to 10A, wherein the first bone is the tibia and the second bone is the talus.

[0193]

[0154] Clause 12A. The method according to any one of Clauses 1A to 10A, wherein applying a GCN model comprises: a computing system performing one or more message-passing rounds, wherein the performance of a message-passing rounds comprises: for each node of a plurality of nodes, the computing system passing a feature vector for each node to each node of a plurality of nodes connected to each node in the graph; and for each node of the plurality of nodes, the computing system modifying the feature vector for each node based on the feature vector for each node and the feature vector passed to each node; after the completion of one or more message-passing rounds, for each node of the plurality of nodes, the computing system applying a series of one or more graph convolutional layers (GCLs) of the GCN model to the feature vector of each node in order to obtain an embedding for each node; and the computing system generating an output based on the embedding for the node.

[0194]

[0155] Clause 13A. The method according to Clause 12A, wherein generating an output based on an embedding of a node comprises: applying a pooling layer to the embedding of a node by a computing system, wherein the pooling layer generates a first intermediate vector having a first intermediate feature corresponding to a prosthesis of different sizes; applying a fully connected layer to the first intermediate vector by a computing system to generate a second intermediate vector, wherein the second intermediate vector has a second intermediate feature corresponding to a prosthesis of different sizes; and applying a softmax layer to the second intermediate vector by a computing system to generate an output, wherein the output includes a final vector having a final feature corresponding to a prosthesis of different sizes.

[0195]

[0156] Clause 14A. The method according to Clause 12A, wherein a node includes cross-section nodes corresponding to different cross-sections in a plurality of cross-sections of bone, and for each cross-section node in the graph, the first feature data included in the feature vector for the cross-section node characterizes one or more aspects of bone in the cross-section corresponding to the cross-section node, and the feature vector of the cross-section node is modified by a computing system based on a weighted average of the features in the feature vector of the cross-section node and the features in the feature vector passed to the cross-section node, wherein the weights used in the weighted average are based on the importance of the cross-section corresponding to the cross-section node and the cross-section corresponding to the adjacent node, and the adjacent nodes are connected to the respective nodes in the graph data.

[0196]

[0157] Clause 15A. The method according to any one of Clauses 1A to 14A, wherein the graph is a three-dimensional grid graph, and for each sampling position in the three-dimensional arrangement of sampling positions, a plurality of nodes include a node corresponding to each sampling position, the three-dimensional arrangement of sampling positions includes sampling positions corresponding to positions within a bone, and for each node, the first feature data included in the feature vector for the node characterizes one or more aspects of the bone at the sampling position corresponding to the node.

[0197]

[0158] Clause 16A. The method according to Clause 15A, wherein for one or more of the nodes, the first feature data included in the feature vector for the node includes one or more of the following: a measure of bone density for the sampling location corresponding to the node, data describing the local texture of the bone, or a gray-level co-occurrence matrix.

[0198]

[0159] Clause 17A. The method according to any one of Clauses 15A to 16A, wherein for each node, the multiple edges include a set of edges that connect the node to up to six other nodes in the multiple nodes, or for each node, the multiple edges include a set of edges that connect the node to up to eight other nodes in the multiple nodes, or for each node, the multiple edges include a set of edges that connect the node to up to twelve other nodes in the multiple nodes.

[0199]

[0160] Clause 18A. The method according to any one of Clauses 15A to 17A, wherein the sampling locations include a set of sampling locations corresponding to predefined coordinates in three-dimensional image data of a bone.

[0200]

[0161] Clause 19A. The method according to any one of Clauses 15A to 18A, further comprising: applying a segmentation process to bone image data by a computing system to identify multiple regions, wherein for at least one of the regions where the region corresponds to a sampling position among multiple sampling positions, the computing system generates a feature vector of a node corresponding to the sampling position corresponding to the region, based on data associated with one or more pixels of image data in the region.

[0201]

[0162] Clause 20A. The method according to Clause 19A, wherein one or more pixels of the image data include multiple pixels of the image data.

[0202]

[0163] Clause 21A. The method according to Clause 20A, wherein the region is a three-dimensional region and each of two or more coordinates of multiple pixels contained within the region is different.

[0203]

[0164] Clause 22A. The method according to any one of Clauses 1A to 12A or 14A to 21A, wherein the output includes dimensions of one or more appropriate prostheses, and the method for determining a recommended prosthesis for a patient is the method for determining a recommended prosthesis for a patient from among several available prostheses based on the dimensions of one or more appropriate prostheses.

[0204]

[0165] Clause 23A. The method according to any one of Clauses 1A to 12A or 14A to 21A, wherein the output includes one or more parameters of a suitable prosthesis, and the method for determining a recommended prosthesis for a patient is the method for determining a recommended prosthesis for a patient based on one or more parameters of a suitable prosthesis by a computing system.

[0205]

[0166] Clause 24A. A computing system comprising a memory system and one or more processors implemented in a circuit and configured to perform the method described in any of Clauses 1A to 23A.

[0206]

[0167] Clause 25A. A computing system comprising means for carrying out the method described in any of Clauses 1A to 23A.

[0207]

[0168] Clause 26A. A non-temporary computer-readable storage medium that stores instructions causing a computing system to perform any of the methods described in Clauses 1A to 23A when executed.

[0208]

[0169] Clause 1B. A method comprising: generating a feature vector for each node of a plurality of nodes in a graph, using one or more processors implemented in the circuit, first including feature data that characterizes one or more aspects of a patient's bone; applying a graph convolutional network (GCN) model to the graph and the feature vector for the nodes to generate an output, wherein the graph includes one or more edges, each of which connects each pair of nodes; and determining, based on the output, one or more recommended prostheses or recommended values ​​for one or more prosthesis parameters for a patient, using one or more processors.

[0209]

[0170] Clause 2B. The method according to Clause 1B, wherein a plurality of nodes include cross-sectional nodes corresponding to different cross-sections in a plurality of cross-sections of bone, and for each cross-sectional node in the graph, the first feature data included in the feature vector for the cross-sectional node characterizes one or more aspects of bone in the cross-section corresponding to the cross-sectional node.

[0210]

[0171] Clause 3B. The method according to Clause 2B, wherein for each cross-section of the bone, one or more aspects of the bone in the cross-section include one or more measurements of the dimensions of the bone in the cross-section.

[0211]

[0172] Clause 4B. The method according to Clause 3B, wherein the bone is a tibia, and one or more measurements include measurements of the dimensions of the bone in cross-section, including the medial width of the tibia in cross-section and the anterior length of the tibia in cross-section.

[0212]

[0173] Clause 5B. The method according to Clause 3B, wherein the bone is the talus and one or more measurements include a medial-lateral width measurement of the talus.

[0213]

[0174] Clause 6B. The method according to any of Clauses 2B to 5B, wherein the cross-section is perpendicular to the functional axis of the bone.

[0214]

[0175] Clause 7B. GCN model is a first GCN model, bone is a first bone, feature data is first feature data, one or more embodiments is one or more first embodiments, graph is a first graph, plurality of nodes are first plurality of nodes, prosthesis is a first prosthesis, output is a first output, method is to generate feature vectors for the nodes by a computing system, first including secondary feature data characterizing one or more second embodiments of the patient's second bone for each node of the second plurality of nodes of the second graph, wherein the second graph includes one or more second edges, and the second edges The method according to any one of the clauses 1B to 6B, further comprising: applying a second GCN model to a second graph and feature vectors for a second set of nodes, each of which connects each pair of nodes in the second graph, to generate a second output; and determining, based on the second output, a second prosthesis or recommended values ​​for one or more second prosthesis parameters to be recommended for implantation in a patient, wherein one or more second prosthesis parameters specify one or more embodiments of a secondary prosthesis.

[0215]

[0176] Clause 8B. The method of Clause 7B, further comprising a computing system selecting a second GCN model from among a plurality of secondary GCN models based on a first prothesis.

[0216]

[0177] Clause 9B. The method according to any one of Clauses 7B to 8B, wherein a second plurality of nodes include cross-sectional nodes corresponding to different cross-sections in a plurality of cross-sections of a second bone, and for each of the cross-sectional nodes in the second graph, the first feature data included in the feature vector for the cross-sectional node characterizes one or more aspects of the second bone in the cross-section corresponding to the cross-sectional node.

[0217]

[0178] Clause 10B. The method according to any of Clauses 7B to 9B, wherein the first prosthesis is designed for implantation in a first bone and the second prosthesis is designed for implantation in a second bone.

[0218]

[0179] Clause 11B. The method according to any of Clauses 7B to 10B, wherein the first bone is the tibia and the second bone is the talus.

[0219]

[0180] Clause 12B. The method according to any one of Clauses 1B to 10B, wherein applying a GCN model comprises: a computing system performing one or more message-passing rounds, wherein the performance of a message-passing rounds comprises: for each node of a plurality of nodes, the computing system passing a feature vector for each node to each node of a plurality of nodes connected to each node in the graph; and for each node of the plurality of nodes, the computing system modifying the feature vector for each node based on the feature vector for each node and the feature vector passed to each node; after the completion of one or more message-passing rounds, for each node of the plurality of nodes, the computing system applying a series of one or more graph convolutional layers (GCLs) of the GCN model to the feature vector of each node in order to obtain an embedding for each node; and the computing system generating an output based on the embedding for the node.

[0220]

[0181] Clause 13B. The method according to Clause 12B, wherein generating an output based on an embedding for a node comprises: applying a pooling layer to the embedding for a node by a computing system, wherein the pooling layer generates a first intermediate vector having a first intermediate feature corresponding to a prosthesis of different sizes; applying a fully connected layer to the first intermediate vector by a computing system to generate a second intermediate vector, wherein the second intermediate vector has a second intermediate feature corresponding to a prosthesis of different sizes; and applying a softmax layer to the second intermediate vector by a computing system to generate an output, wherein the output includes a final vector having a final feature corresponding to one of the following: a prosthesis of different sizes, a set of compatible prostheses, or a combination of possible values ​​of one or more prosthesis parameters.

[0221]

[0182] Clause 14B. The method of Clause 12B, wherein a node includes cross-section nodes corresponding to different cross-sections in a plurality of cross-sections of bone, and for each cross-section node in the graph, the first feature data included in the feature vector for the cross-section node characterizes one or more aspects of bone in the cross-section corresponding to the cross-section node, and the feature vector of the cross-section node is modified by a computing system based on a weighted average of the features in the feature vector of the cross-section node and the features in the feature vector passed to the cross-section node, wherein the weights used in the weighted average are based on the importance of the cross-section corresponding to the cross-section node and the cross-section corresponding to the adjacent node, and the adjacent nodes are connected to the respective nodes in the graph data.

[0222]

[0183] Clause 15B. The method according to any one of Clauses 1B to 14B, wherein the graph is a three-dimensional grid graph, and for each sampling position in the three-dimensional arrangement of sampling positions, a plurality of nodes include a node corresponding to each sampling position, the three-dimensional arrangement of sampling positions includes sampling positions corresponding to positions within a bone, and for each node, the first feature data included in the feature vector for the node characterizes one or more aspects of the bone at the sampling position corresponding to the node.

[0223]

[0184] Clause 16B. The method according to Clause 15B, wherein for one or more of the nodes, the first feature data included in the feature vector for the node includes one or more of the following: a measure of bone density for the sampling location corresponding to the node, data describing the local texture of the bone, or a gray-level co-occurrence matrix.

[0224]

[0185] Clause 17B. The method according to any one of Clauses 15B to 16B, wherein for each node, the multiple edges include a set of edges that connect the node to up to six other nodes in the multiple nodes, or for each node, the multiple edges include a set of edges that connect the node to up to eight other nodes in the multiple nodes, or for each node, the multiple edges include a set of edges that connect the node to up to twelve other nodes in the multiple nodes.

[0225]

[0186] Clause 18B. The method of any one of Clauses 15B to 17B, wherein the sampling locations include a set of sampling locations corresponding to predefined coordinates in three-dimensional image data of a bone.

[0226]

[0187] Clause 19B. The method of any one of Clauses 15B to 18B, further comprising: applying a segmentation process to bone image data to identify multiple regions by a computing system, wherein for at least one of the regions where the region corresponds to a sampling position among multiple sampling positions, the computing system generates a feature vector of a node corresponding to the sampling position corresponding to the region based on data associated with one or more pixels of image data in the region.

[0227]

[0188] Clause 20B. The method according to Clause 19B, wherein one or more pixels of the image data include multiple pixels of the image data.

[0228]

[0189] Clause 21B. The method according to Clause 20B, wherein the region is a three-dimensional region and each of two or more coordinates of multiple pixels contained within the region is different.

[0229]

[0190] Clause 22B. The method according to any one of Clauses 1B to 12B or 14B to 21B, wherein the output includes dimensions of one or more appropriate prostheses, and the method for determining a recommended prosthesis for a patient is the method for determining a recommended prosthesis for a patient from among several available prostheses based on the dimensions of one or more appropriate prostheses.

[0230]

[0191] Clause 23B. The method according to any one of Clauses 1B to 12B or 14B to 21B, wherein the output includes one or more parameters of a suitable prosthesis, and the method for determining a recommended prosthesis for a patient is the method by which a computing system determines a recommended prosthesis for a patient based on one or more parameters of a suitable prosthesis.

[0231]

[0192] Clause 24B. The method according to any one of Clauses 1B to 23B, wherein one or more prosthesis parameters include one or more of the following: size of the glenosphere of a glenosphere prosthesis, radius of the glenosphere of a glenosphere prosthesis, base plate type of a glenosphere prosthesis, expansion type of a glenosphere prosthesis, degree of glenofoss eccentricity of a glenosphere prosthesis, stem size of a humeral prosthesis, or head sphere size of a humeral prosthesis.

[0232]

[0193] Clause 25B. The method according to any one of claims 1B to 23B, wherein one or more prosthesis parameters include one or more of the sizes of tibial components, talar prostheses, or joint components that can be attached to tibial components.

[0233]

[0194] Clause 26B. The method of any of Clauses 1B to 24B, further comprising surgically implanting a recommended prosthesis.

[0234]

[0195] Clause 27B. A computing system comprising a memory system and one or more processors implemented in a circuit and configured to perform the method described in any of Clauses 1B to 26B.

[0235]

[0196] Clause 28B. A computing system comprising means for carrying out the method described in any of Clauses 1B to 26B.

[0236]

[0197] Clause 29B. A non-temporary computer-readable storage medium that stores instructions causing a computing system to perform any of the methods described in Clauses 1B through 26B when executed.

[0237]

[0198] Although the technique has been disclosed in relation to a limited number of examples, those skilled in the art who are interested in this disclosure will understand that there are numerous modifications and variations therefrom. For example, it is intended that any reasonable combination of the examples described may be implemented. The appended claims are intended to include modifications and variations that fall within the true spirit and scope of the invention.

[0238]

[0199] Depending on the example, some actions or events of any of the techniques described herein may be performed in different sequences, added, merged, or completely excluded (for example, not all described actions or events are necessary for the practice of this technique). Furthermore, in some examples, actions or events may be performed not sequentially, but simultaneously, for example, through multithreading, interrupt handling, or across multiple processors.

[0239]

[0200] In one or more examples, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored as one or more instructions or codes on a computer-readable medium or transmitted through a computer-readable medium and executed by a hardware-based processing unit. The computer-readable medium may include a computer-readable storage medium corresponding to a tangible medium such as a data storage medium, or a communication medium including any medium that facilitates the transfer of a computer program from one place to another according to a communication protocol, for example. Thus, the computer-readable medium may generally correspond to (1) a non-transient tangible computer-readable storage medium, or (2) a communication medium such as a signal or carrier wave. The data storage medium may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, codes and / or data structures for implementing the techniques described herein. A computer program product may include a computer-readable medium.

[0240]

[0201] As an example, not an limitation, such computer-readable storage media may include RAM, ROM, EEPROM®, CD-ROM or other optical disk storage devices, magnetic disk storage devices or other magnetic storage devices, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and can be accessed by a computer. Any connection is also appropriately called a computer-readable medium. For example, if instructions are transmitted from a website, server or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of a medium. However, it should be understood that computer-readable storage media and data storage media do not include connections, carriers, signals, or other temporary media, but instead cover non-temporary tangible storage media. The terms "disk" and "disc" as used herein include Compact Disc (CD), LaserDisc® (disc), Optical Disc (disc), Digital Multipurpose Disc (disc) (DVD), Floppy Disk (disk), and Blu-ray® Disc (disc), where a disk typically reproduces data magnetically, and a disc reproduces data optically using a laser. Any combination of the above should also be included within the scope of computer-readable media.

[0241]

[0202] The operations described herein may be performed by one or more processors, which may be implemented as fixed-function processing circuits, programmable circuits, or combinations thereof, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Fixed-function circuits refer to circuits that provide a specific functionality and are preset to operations that may be performed. Programmable circuits refer to circuits that can be programmed to perform a variety of tasks and provide flexible functionality in operations that may be performed. For example, a programmable circuit may execute instructions specified by software or firmware that operate the programmable circuit in a manner defined by software or firmware instructions. A fixed-function circuit may execute software instructions (for example, to receive or output parameters), but the type of operation performed by a fixed-function circuit is generally immutable. Accordingly, the terms “processor” and “processing circuit” as used herein may refer to any of the above structures or any other structure suitable for implementing the techniques described herein.

Claims

1. For each node of a plurality of nodes in the graph, one or more processors implemented in the circuit generate a feature vector for the node, which initially includes feature data characterizing one or more aspects of the patient's bone, wherein the graph includes one or more edges, each of which connects each pair of nodes. The one or more processors apply a graph convolutional network (GCN) model to the graph and the feature vectors for the nodes in order to generate an output. The one or more processors determine, based on the output, one or more recommended prostheses or recommended values ​​for one or more prosthesis parameters for the patient. A method for providing this.

2. The plurality of nodes include cross-sectional nodes corresponding to different cross-sections in the plurality of cross-sections of the bone, For each of the cross-sectional nodes in the graph, the first feature data included in the feature vector for the cross-sectional node characterizes one or more aspects of the bone in the cross-section corresponding to the cross-sectional node. The method according to claim 1.

3. The method according to claim 2, wherein, for each of the cross-sections of the bone, one or more embodiments of the bone in the cross-section of the bone include one or more measured values ​​of the dimensions of the bone in the cross-section of the bone.

4. The method according to claim 3, wherein the bone is a tibia, and the one or more measurements include measurements of the dimensions of the bone in the cross-section of the bone, and include the medial width of the tibia in the cross-section and the anterior length of the tibia in the cross-section.

5. The method according to claim 3, wherein the bone is the talus, and the one or more measurements include a measurement of the medial-lateral width of the talus.

6. The method according to any one of claims 2 to 5, wherein the cross-section is perpendicular to the functional axis of the bone.

7. The GCN model is the first GCN model, the bone is the first bone, the feature data is the first feature data, the one or more embodiments are one or more first embodiments, the graph is the first graph, the plurality of nodes are the first plurality of nodes, the prosthesis is the first prosthesis, and the output is the first output. The aforementioned method, The computing system generates feature vectors for each node of a second plurality of nodes in a second graph, initially including secondary feature data characterizing one or more second aspects of the patient's second bone, wherein the second graph includes one or more second edges, each of which connects each pair of nodes in the second graph. The computing system applies a second GCN model to the second graph and the feature vectors for the second set of nodes in order to generate a second output. The computing system determines, based on the second output, a second prosthesis or recommended values ​​for one or more second prosthesis parameters that should be recommended for implantation in the patient, wherein the one or more second prosthesis parameters specify one or more forms of the secondary prosthesis. The method according to any one of claims 1 to 6, further comprising:

8. The method according to claim 7, further comprising selecting the second GCN model from among a plurality of secondary GCN models based on the first prothesis using the computing system.

9. The second plurality of nodes include cross-sectional nodes corresponding to different cross-sections in the plurality of cross-sections of the second bone, For each of the cross-sectional nodes in the second graph, the first feature data included in the feature vector for the cross-sectional node characterizes one or more aspects of the second bone in the cross-section corresponding to the cross-sectional node. The method according to any one of claims 7 to 8.

10. The method according to any one of claims 7 to 9, wherein the first prosthesis is designed for the first implantation in bone, and the second prosthesis is designed for the second implantation in bone.

11. The method according to any one of claims 7 to 10, wherein the first bone is the tibia and the second bone is the talus.

12. Applying the aforementioned GCN model means The computing system performs one or more message passing rounds, and in this case, the performance of message passing rounds is For each of the multiple nodes, the computing system transfers the feature vector for each node to each node of the multiple nodes connected to each node in the graph. For each of the plurality of nodes, the computing system modifies the feature vector of each node based on the feature vector for each node and the feature vector passed to each node. Equipped with, After the completion of the one or more message passing rounds, the computing system applies a series of one or more graph convolutional layers (GCLs) of the GCN model to the feature vector of each node in order to obtain an embedding for each node. The computing system generates the output based on the embedding for the node. The method according to any one of claims 1 to 10, comprising:

13. To generate the output based on the embedding for the node is to The computing system applies a pooling layer to the embedding for the node, wherein the pooling layer generates a first intermediate vector having a first intermediate feature corresponding to prostheses of different sizes. The computing system applies a fully connected layer to the first intermediate vector in order to generate a second intermediate vector, wherein the second intermediate vector comprises a second intermediate feature corresponding to the prothesis having a different size. The computing system applies a softmax layer to the second intermediate vector in order to generate the output, and in this case, the output is The prostheses having different sizes, A set of compatible prostheses, or Possible combinations of values ​​for one or more of the aforementioned prosthesis parameters, The final vector includes a final feature that corresponds to one of the following: The method according to claim 12, comprising:

14. The node includes cross-sectional nodes corresponding to different cross-sections among the multiple cross-sections of the bone, For each of the cross-sectional nodes in the graph, The first feature data included in the feature vector for the cross-sectional node characterizes one or more aspects of the bone in the cross-section corresponding to the cross-sectional node, Modifying the feature vector of the section node comprises the computing system modifying the feature vector of the section node based on a weighted average of the features in the feature vector of the section node and the features in the feature vector passed to the section node, wherein the weights used in the weighted average are based on the importance of the section corresponding to the section node and the section corresponding to the adjacent node, and the adjacent node is connected to the respective node in the graph data. The method according to claim 12.

15. The aforementioned graph is a three-dimensional grid graph, For each sampling position in the three-dimensional arrangement of sampling positions, the plurality of nodes include the node corresponding to each sampling position. The three-dimensional arrangement of sampling positions includes sampling positions corresponding to positions within the bone, For each of the nodes, the first feature data included in the feature vector for the node characterizes one or more aspects of the bone at the sampling position corresponding to the node. The method according to any one of claims 1 to 14.

16. For one or more of the aforementioned nodes, the first feature data included in the feature vector for that node is Measurement of bone density for the sampling position corresponding to the node, Data describing the local texture of the bone, or Gray level co-occurrence matrix The method according to claim 15, comprising one or more of the above.

17. For each of the nodes, the plurality of edges include a set of edges that connect the node to up to six other nodes in the plurality of nodes, For each of the nodes, the plurality of edges include a set of edges that connect the node to up to eight other nodes in the plurality of nodes, or For each of the nodes, the plurality of edges include a set of edges that connect the node to up to 12 other nodes in the plurality of nodes. The method according to any one of claims 15 to 16.

18. The method according to any one of claims 15 to 17, wherein the sampling position includes a set of sampling positions corresponding to predefined coordinates in the three-dimensional image data of the bone.

19. The computing system applies a segmentation process to the bone image data in order to identify multiple regions, wherein the region corresponds to a sampling position among the multiple sampling positions. For at least one of the aforementioned regions, the computing system generates the feature vector of the node corresponding to the sampling position corresponding to the region, based on data associated with one or more pixels of the image data within the region. The method according to any one of claims 15 to 18, further comprising the above.

20. The method according to claim 19, wherein one or more pixels of the image data include a plurality of pixels of the image data.

21. The method according to claim 20, wherein the region is a three-dimensional region, and each of the coordinates of two or more of the plurality of pixels contained in the region is different.

22. The output includes one or more dimensions of a suitable prosthesis, Determining the recommended prosthesis for the patient comprises the computing system determining the recommended prosthesis for the patient from among a plurality of available prostheses based on the one or more dimensions of the appropriate prosthesis. The method according to any one of claims 1 to 12 or 14 to 21.

23. The output includes one or more parameters of a suitable prosthesis. Determining the recommended prosthesis for the patient comprises determining the recommended prosthesis for the patient based on one or more parameters of the appropriate prosthesis by the computing system. The method according to any one of claims 1 to 12 or 14 to 21.

24. The method according to any one of claims 1 to 23, wherein the one or more prosthesis parameters include one or more of the following: the size of the glenosphere of the glenosphere prosthesis, the radius of the glenosphere of the glenosphere prosthesis, the base plate type of the glenosphere prosthesis, the expansion type of the glenosphere prosthesis, the degree of glenofoss eccentricity of the glenosphere prosthesis, the stem size of the humeral prosthesis, or the head sphere size of the humeral prosthesis.

25. The method according to any one of claims 1 to 23, wherein the one or more prosthesis parameters include one or more of the sizes of tibial components, talar prostheses, or joint components that can be attached to the tibial components.

26. The method according to any one of claims 1 to 24, further comprising surgically implanting the recommended prosthesis.

27. Memory system and, One or more processors implemented in a circuit and configured to carry out the method according to any one of claims 1 to 26 A computing system equipped with [the following features].

28. A computing system comprising means for carrying out the method described in any one of claims 1 to 26.

29. A non-temporary computer-readable storage medium storing instructions that, when executed, cause a computing system to perform the method according to any one of claims 1 to 26.

Images