Graph convolutional network model for orthopaedic prosthesis recommendation

Abstract

For each of a plurality of nodes in a graph, the computing system generates a feature vector for the node that initially includes feature data characterizing one or more aspects of the patient’s bone. The graph includes one or more edges. Each edge connects a respective pair of nodes. The computing system applies a graph convolutional network (GCN) model to the graph and the feature vectors for the nodes to generate an output. The computing system determines, based on the output, one or more recommended prostheses or recommended values for one or more prosthesis parameters for the patient.

Description

[0001] Cross-references to related applications

[0002] This application claims priority to U.S. Provisional Patent Application No. 63 / 535,425, filed August 30, 2023, the entire contents of which are incorporated herein by reference. Background Technology

[0003] Planning orthopedic surgery may involve selecting the appropriate prosthesis to implant in a patient. Even among prostheses that can be implanted into a single bone, there may be different shapes and sizes. Choosing the right prosthesis for a patient can be a crucial factor in determining the success of orthopedic surgery or the occurrence of complications. Because no two patients have exactly the same anatomy, selecting the right prosthesis can be challenging for surgeons, especially less experienced ones. Summary of the Invention

[0004] This disclosure describes a technique for a computational system to determine recommended orthopedic prostheses for an individual patient using one or more graph convolutional network (GCN) models. As described herein, the computational system can obtain feature data characterizing one or more aspects of the patient's bone. The graph includes multiple sampling nodes and one or more edges. In some examples, the sampling nodes may correspond to different sampling locations, such as different cross-sections of the bone or different regions within the bone. For each sampling node in the graph, the feature vector of the sampling node initially includes feature data characterizing one or more aspects of the bone at the sampling location corresponding to the node. The computational system can apply the GCN model to the graph and the feature vectors of the nodes to generate an output. In some examples, the computational system determines a recommended prosthesis for the patient based on the output generated by the GCN model. In some examples, the computational system determines recommended values ​​for one or more prosthesis parameters based on the output generated by the GCN model. Prosthesis parameters may specify one or more aspects of one or more prostheses, such as the size and radius of the glenoid ball of the glenoid prosthesis, the stem size of the humeral prosthesis, the head-sphere size of the humeral prosthesis, the reinforcement type of the glenoid prosthesis, the base plate type of the glenoid prosthesis, and so on. Using the GCN model in this way allows the computational system to take into account multiple aspects of the bone at various sampling locations. In addition, GCN models can include fewer parameters than other types of classifier neural networks, which saves computational resources and makes them easier to train.

[0005] In one example, this disclosure describes a method comprising: for each of a plurality of nodes in a graph, generating a feature vector for that node by a computational system, the feature vector initially including feature data characterizing one or more aspects of a patient’s bone, wherein the graph includes one or more edges, each of which connects a corresponding pair of nodes; applying a graph convolutional network (GCN) model to the graph and the feature vector of the nodes by the computational system to generate an output; and determining a recommended prosthesis for the patient by the computational system based on the output.

[0006] Details of various examples of this disclosure are set forth in the accompanying drawings and the description below. Various features, objects, and advantages will become apparent from the specification, drawings, and claims. Attached Figure Description

[0007] Figure 1 This is a conceptual diagram illustrating an example computing system in which one or more techniques of this disclosure can be implemented.

[0008] Figure 2 This is a flowchart illustrating an example operation of a computing system according to one or more techniques disclosed herein.

[0009] Figure 3A , Figure 3B and Figure 3C This is a conceptual diagram showing an example cross-section of bone according to one or more techniques of this disclosure.

[0010] Figure 4 This is a conceptual diagram illustrating example diagrams that can be used with graph convolutional network (GCN) models according to one or more techniques disclosed herein.

[0011] Figure 5A , Figure 5B , Figure 5C and Figure 5D This is a conceptual diagram showing an example cross-section of the talus according to one or more techniques of this disclosure.

[0012] Figure 6 This is a block diagram illustrating an example of a planning system using primary GCN and secondary GCN according to one or more techniques of this disclosure.

[0013] Figure 7 This is a conceptual diagram illustrating an example process of recommending a talus prosthesis based in part on a recommended tibial prosthesis, according to one or more techniques of this disclosure.

[0014] Figure 8 This is a conceptual diagram illustrating example diagrams of one or more technologies according to this disclosure that can be used with a GCN model.

[0015] Figure 9This is a flowchart illustrating an example operation of a computational system for recommending two interacting stigmas according to one or more techniques of this disclosure.

[0016] Figure 10 This is a conceptual diagram illustrating an example 3D grid map according to one or more techniques of this disclosure.

[0017] Figure 11 This is a conceptual diagram illustrating an example grid-based neural network model according to one or more techniques of this disclosure.

[0018] Figure 12 This is a flowchart illustrating an example operation of a planning system based on one or more techniques according to this disclosure.

[0019] Figure 13 This is a block diagram illustrating an example architecture for generating quantized patch embeddings and generating reconstructed anatomical models according to one or more techniques of this disclosure.

[0020] Figure 14 This is a flowchart illustrating an example operation of a prediction unit for generating information indicating recommended values ​​for one or more recommended prostheses or one or more prosthesis parameters for a patient, according to one or more techniques of this disclosure.

[0021] Figure 15 This is a flowchart illustrating an example training process for a machine learning model according to one or more techniques disclosed herein.

[0022] Figure 16 This is a conceptual diagram illustrating an example surgical planning user interface according to one or more techniques of this disclosure, which displays surgical recommendations for a reverse shoulder replacement surgery. Detailed Implementation

[0023] Surgeons can choose from a variety of available prostheses to implant in a patient. Available prostheses can have various parameters, such as size and shape. Choosing the right prosthesis from the available options can be a crucial factor in the ultimate success of the surgery. Selecting a prosthesis that is too large or too small can lead to limited range of motion, easy loosening, fractures, and other complications. Because patients have bones that vary in size and shape, selecting the right prosthesis for bone implantation can not be a simple process. Often, the ability to select the right prosthesis comes from both experience and objective knowledge. Therefore, less experienced surgeons often struggle to select the appropriate prosthesis. Computerized surgical planning systems have been developed to assist users in planning orthopedic surgeries, and some of these systems provide automated recommendations for prostheses to be implanted in patients. Because bones often have complex shapes, such computerized surgical planning systems may struggle to predict the right prosthesis based on measurements and deterministic rules, or even traditional machine learning models. For example, accurate prediction of the prosthesis may depend on information about the relationships between non-adjacent parts of the bone. Computerized surgical planning systems that use traditional convolutional neural network (CNN) models to predict orthopedic prostheses typically include a hierarchical structure of convolutional layers that convolve information from spatially adjacent regions in a 3D image. However, this approach can require a large number of convolutional layers. In existing computerized surgical planning systems, the output of a CNN can be used to filter the set of available prostheses. For example, the CNN output can be used to rank available prostheses, and information about only a subset of the top-ranked available prostheses is initially presented for review by the user (e.g., the surgeon). Filtering the set of available prostheses can accelerate the planning process. Furthermore, the planning system can generate and display user interfaces in which a 3D model of the prosthesis is positioned at the suggested implantation location relative to a 3D model of the patient's anatomy. Users can use such user interfaces to confirm that the prosthesis is suitable for the patient. However, generating such user interfaces can be computationally complex and time-consuming. Therefore, generating such user interfaces for all available prostheses is not ideal.

[0024] This disclosure describes techniques that can address these problems. Specifically, this disclosure describes techniques in which a computational system uses one or more Graph Convolutional Network (GCN) models to recommend suitable prostheses to a surgeon. For example, the computational system can acquire feature data characterizing one or more aspects of bone at multiple sampling locations, such as cross-sections of bone or regions within bone. The graph includes multiple nodes and one or more edges. Nodes include sampling nodes corresponding to different sampling locations. For each sampling node in the graph, the feature vector of the sampling node initially includes feature data characterizing one or more aspects of bone at the sampling location corresponding to the sampling node. The computational system can apply the GCN model to the feature vectors of the graph and nodes to generate an output. The computational system can determine a recommendation for one or more prostheses to be implanted in the patient, or a recommendation for one or more prosthesis parameter values ​​for one or more prostheses to be implanted in the patient, based on the output. Applying GCN in this way allows the computational system to efficiently consider the interrelationships of various aspects of bone at different sampling locations. Furthermore, the GCN model can include fewer parameters (e.g., weights, layers, etc.) than other types of machine learning models, which can improve computational efficiency and reduce storage requirements. In addition, in some examples, the computational system can use the output of the GCN model to filter a set of available prostheses. For at least the reasons stated above, the technology disclosed herein improves the efficiency and effectiveness of computational systems for filtering a set of available prostheses. Filtering the set of available prostheses allows the planning system to present the user interface faster and more efficiently, wherein a 3D model of the prosthesis is positioned at the suggested implantation location relative to a 3D model of the patient's anatomy.

[0025] Figure 1 This is a conceptual diagram illustrating an example computing system 100 in which one or more techniques of this disclosure can be implemented. Figure 1 In one example, computing system 100 includes one or more processors 102, a storage system 104, a communication interface 106, and a display 108. In other examples, computing system 100 may include more, fewer, or different components. The components of computing system 100 may reside in one or more computing devices. For example, processor 102 may be in a single computing device or distributed across multiple computing devices, storage system 104 may be in a single computing device or distributed across multiple computing devices, and so on. In some examples, computing system 100 is a personal computer, a system of computing devices, one or more server devices, or a system including one or more other types of computing devices.

[0026] Processor 102 may be implemented in a circuit and includes one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), hardware, or any combination thereof. Typically, processor 102 may be implemented as a fixed-function circuit, a programmable circuit, or a combination thereof. A fixed-function circuit refers to a circuit that provides a specific function and whose operations are preset. A programmable circuit refers to a circuit that can be programmed to perform various tasks and provide flexible functionality in the operations it can perform. For example, a programmable circuit may execute software or firmware that causes the programmable circuit to operate in a manner defined by the instructions of the software or firmware. A fixed-function circuit may execute software instructions (e.g., receiving or outputting parameters), but the type of operation performed by a fixed-function circuit is typically immutable. In some examples, one or more units may be discrete circuit blocks (fixed-function or programmable), and in some examples, one or more units may be integrated circuits.

[0027] Processor 102 may include an arithmetic logic unit (ALU), an essential function unit (EFU), digital circuitry, analog circuitry, and / or a programmable core, these components being formed by programmable circuitry. In an example where the operation of processor 102 is performed using software executed by programmable circuitry, storage system 104 may store object code of the software received and executed by processor 102, or another memory (not shown) within processor 102 may store such instructions. Examples of software include software designed for surgical planning. Processor 102 may perform actions attributed to computing system 100 in this disclosure.

[0028] Storage system 104 can store various types of data used by processor 102. Storage system 104 may include any of a variety of memory devices, such as dynamic random access memory (DRAM), including synchronous DRAM (SDRAM), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or other types of memory devices. Examples of display 108 include liquid crystal display (LCD), plasma display, organic light-emitting diode (OLED) display, or another type of display device.

[0029] Communication interface 106 allows computing system 100 to output data and instructions to and receive data and instructions from medical imaging systems or other devices via one or more communication links or networks. Communication interface 106 may include hardware circuitry that enables computing system 100 to communicate with other computing systems and devices (e.g., wirelessly or via a wired connection). Example networks may include various types of communication networks, including one or more wide area networks, such as the Internet, local area networks, etc. The network may include wired communication links and / or wireless communication links.

[0030] exist Figure 1 In one example, storage system 104 stores medical image data 110, planning system 118, and training system 120. In other examples, storage system 104 may store more, less, or different types of data or units. Furthermore, in... Figure 1 The data and units shown in the examples are provided for illustrative purposes and may not represent the actual way the data is stored or the actual way the software is implemented. The planning system 118 may include instructions executable by the processor 102. For ease of explanation, this disclosure may describe the planning system 118 as performing various actions when the processor 102 executes the instructions of the planning system 118. Additionally, in Figure 1 In the example, the planning system 118 includes feature data 112, graph data 114, and GCN model 116.

[0031] Planning system 118 is a system that assists users (such as surgeons) in planning orthopedic surgeries, and can be part of a preoperative planning process or an intraoperative planning process. As part of the process of assisting users in planning orthopedic surgeries, planning system 118 can present a series of user interfaces that help users select an orthopedic prosthesis for implantation into the patient when multiple available orthopedic prostheses are available. For ease of explanation, this disclosure may simply refer to an orthopedic prosthesis as a prosthesis. The user interface may include a user interface displaying one or more 3D models of the patient's bones, measurement data about the bones, and other information about the patient's bones or other anatomical features. Additionally, the user interface may display information about a set of available prostheses for implantation into the patient. Planning system 118 can perform a process of automatically ranking and / or filtering available prostheses. The user interface displaying information about the set of available prostheses may at least initially limit the information to one or more top-ranked prostheses and / or the remaining prostheses after filtering.

[0032] The planning system 118 can be applied to assist users in selecting prostheses in a variety of orthopedic surgical settings. For example, in ankle arthroplasty, the planning system 118 can help surgeons select tibial and / or talar prostheses. In shoulder arthroplasty, the planning system 118 can help surgeons select glenoid and / or humeral prostheses. In knee arthroplasty, the planning system 118 can help surgeons select femoral and / or tibial prostheses. In hip arthroplasty, the planning system 118 can help surgeons select acetabular and / or femoral prostheses. Other examples can be applied to other bones and / or joints.

[0033] Prostheses that can be used for bone implantation can come in a variety of shapes and sizes. For example, several different sizes of tibial prostheses may be available, from which surgeons can choose when performing ankle arthroplasty. A tibial prosthesis can be implanted at the distal end of a patient's tibia. Depending on the patient's tibial size and other factors, different sizes of tibial prostheses may be appropriate. In some cases, the prosthesis can be custom-made so that it is specific to the patient. In other words, the prosthesis can be a patient-specific prosthesis.

[0034] As part of the process of recommending one or more prostheses or one or more prosthesis parameter values ​​for implantation in a patient, the planning system 118 acquires feature data 112 that characterizes one or more aspects of one or more bones at multiple sampling locations. In some examples, the sampling locations include cross-sections of one or more bones. Example aspects of a bone cross-section may include: the size of the bone at the cross-section (e.g., width, length, height, etc.), the average density of the bone (or cortical bone) at the cross-section, the minimum density of the bone at the cross-section, data indicating the presence of voids in the bone, and other types of data characterizing the bone at the cross-section. In some examples, the sampling locations include 2D or 3D regions. In such examples, aspects of the bone may include bone density within the region, the presence of bone tissue within the region, data describing bone texture (e.g., cortical bone, cancellous bone, etc.), and so on. In some examples, the sampling locations may include a combination of cross-sections and regions.

[0035] In the example of sampling locations including cross-sections of bone, each cross-section may be perpendicular to the axis passing through the bone. For example, in the example where the bone is the tibia, each cross-section may be perpendicular to the mechanical axis of the tibia. In the example where the bone is the talus, each cross-section may be perpendicular to the mechanical axis of the talus. The mechanical axis of the tibia may be defined as the line connecting the center of the knee joint to the center of the talus. The cross-sections may be spaced apart from each other by one or more amounts. For example, the cross-sections may be spaced apart by 1 millimeter (mm) or another distance.

[0036] In some examples, the planning system 118 may use medical image data 110 to acquire feature data 112. Medical image data 110 may include computed tomography (CT) data, dual-energy X-ray absorptiometry (DXA or DEXA) data, 3D bone modeling data, magnetic resonance imaging (MRI) data, and / or other types of medical imaging data. The planning system 118 may use the medical image data 110 to generate 3D models or multiple 2D models of one or more bones of a patient. For example, the planning system 118 may perform a segmentation process to determine the boundaries of one or more bones, and in some examples, determine the boundaries of other tissues within the medical image data 110. The planning system 118 may use the determined boundaries to generate one or more models (e.g., 2D or 3D models) of one or more bones, and in some examples, may also generate models of other tissues. The planning system 118 may then use one or more models to measure one or more aspects of one or more bones. Feature data may indicate the measurement aspects of one or more bones. The planning system 118 may perform the segmentation process in one of a variety of ways. For example, the planning system 118 may use a neural network-based process to perform segmentation. In some examples, the planning system 118 may additionally or alternatively acquire feature data directly from 2D medical images (e.g., X-rays, CT slices).

[0037] Graph data 114 defines a graph comprising multiple nodes and one or more edges. Each edge connects a corresponding pair of nodes. In other words, each of these edges is a connection between exactly two of these nodes. In some examples, an edge connects every single node. In other examples, an edge does not connect every single node. Nodes include sampling nodes corresponding to different sampling locations, such as cross-sections or regions. A sampling node corresponding to a cross-section may be referred to herein as a "cross-section node". For each sampling node of the graph, the feature vector of the sampling node initially includes feature data characterizing one or more aspects of the bone at the sampling location corresponding to the node. For example, the feature vector corresponding to a particular cross-section may include elements indicating the length and width of that particular cross-section of the bone. In some examples, the nodes of the graph may include other nodes besides sampling nodes, such as nodes corresponding to the entire bone or nodes corresponding to adjacent bones, and so on.

[0038] The planning system 118 can apply the GCN model 116 to the graph and the feature vectors of the nodes to generate output. In some examples, the planning system 118 can determine one or more recommended prostheses for the patient based on the output. In some examples, the planning system 118 can determine recommended values ​​for one or more prosthesis parameters based on the output. The prosthesis parameters can specify aspects of one or more prostheses. When the planning system 118 applies the GCN model 116 to the graph and the feature vectors, the planning system 118 can perform one or more message passing rounds. During at least one or each message passing round, for at least one or each of a plurality of nodes, the planning system 118 passes the feature vector of the corresponding node to each node in the plurality of nodes in the graph that is connected to the corresponding node. Thus, each node can receive the feature vectors of its neighboring nodes.

[0039] Furthermore, as part of executing message passing rounds, for at least one or each of multiple nodes, the planning system 118 can modify the feature vector of the corresponding node based on the feature vector of the corresponding node and the feature vector passed to the corresponding node. In other words, the planning system 118 can aggregate the feature vector of the corresponding node and the feature vector passed to the corresponding node. The planning system 118 can modify the feature vector of the corresponding node in one of several ways. For example, the planning system 118 can calculate each element of the modified feature vector as the average of the corresponding elements of the feature vector (e.g., a weighted average). In another example, the planning system 118 can calculate each element of the modified feature vector as the sum (or weighted sum) of the corresponding elements of the feature vector.

[0040] In some examples of planning systems that calculate weighted averages or weighted sums, planning system 118 can determine the weights of the eigenvectors according to the following formula.

[0041]

[0042] In this example, if the edge connects different nodes, for the connecting nodes i and nodes j For each edge of the graph, the weights of the features applied to the feature vectors. e ij It can be equal to In this equation, d This represents the distance between the sampling location corresponding to the node. For example, if the node... i Corresponding to the first cross section and node j Corresponding to the fourth cross section, d The value can be equal to 3. In some examples where the sampling location includes a 2D or 3D region, the value... dThis can indicate the physical Euclidean distance between the centroids or nearest boundaries of these regions. In other examples, the value... d This can indicate another type of distance measurement. If the node i and nodes j If the features applied to the same node (i.e., the weight of the features applied to the node's own feature vector is 1), then the weight of the features applied to the feature vector... e ij It can be equal to 1. Therefore, compared to the feature vector corresponding to the node that is closer to the sampling position, the feature vector corresponding to the node that is farther away from the current node has a smaller impact on the modification of the feature vector of the current node.

[0043] In some examples, a particular sampling location may be more important than other sampling locations in determining one or more recommended prostheses or in determining recommended values ​​for one or more prosthesis parameters. For example, in an example where the sampling location corresponds to a cross-section of the tibia, a cross-section closer to the distal tibia may be more important for recommending a tibial prosthesis than a cross-section farther from the distal tibia. Therefore, when aggregating the feature vectors of a node with the feature vectors of other nodes, the planning system 118 can apply weights to the feature vectors based on the importance of the sampling location corresponding to the node.

[0044] Furthermore, in some examples where the sampling location includes cross-sections, cross-sections with bone voids or lower bone mineral density may be less important than cross-sections without bone voids or with higher bone mineral density (BMD). Therefore, when aggregating the feature vectors of a node with those of other nodes, the planning system 118 can impose weights on the feature vectors based on the presence of voids and / or BMD in the cross-section corresponding to the node. In some examples, the weights can be based on a combination of factors such as voids, BMD, distance between cross-sections, relative importance, etc.

[0045] After completing one or more message passing rounds, for each corresponding node in the graph, the planning system 118 can apply a series of one or more graph convolutional layers (GCLs) of the GCN model 116 to the latent modified feature vector of the corresponding node to obtain the embedding of the corresponding node. Each GCL may include a set of artificial neurons. At least some of the artificial neurons of the GCL can receive the following as input: some or all of the features of the node's feature vector, or the feature vector generated by the previous GCL. The artificial neurons of the GCL can output the result of a weighted transfer function applied to the input according to machine learning weights.

[0046] In some examples, some or all of the dimensions of the embedding space are increased in the GCL. For example, in an example where each feature vector includes two features (e.g., a feature corresponding to the width of the bone cross-section and a feature corresponding to the length of the bone cross-section), the first GCL can increase the dimensionality of the embedding space from 2 dimensions to 4 dimensions. In this example, the second GCL can increase the dimensionality of the embedding space from 4 dimensions to 8 dimensions. In some examples, the planning system 118 applies batch normalization to the output of each GCL. In some examples, the planning system 118 applies activation functions (such as the rectified linear unit (ReLU) activation function, the sigmoid activation function, or other types of activation functions) to the output of the GCL.

[0047] The planning system 118 can generate output based on embeddings. For example, the planning system 118 can apply a pooling layer to the node embeddings. The pooling layer generates a first intermediate vector that includes first intermediate features corresponding to spurs of different sizes. For example, the number of nodes in the graph can be equal to 10, and the final GCL layer can output an 8-dimensional embedding for each node. Therefore, there can be 10 × 8 features after the final GCL layer. In this example, the pooling layer can reduce the number of features to 1 × 8. For example, the pooling layer can determine the average value (e.g., mean) of the corresponding features for each of the 10 nodes, thus obtaining a first intermediate vector including 8 features.

[0048] Additionally, the planning system 118 can apply a fully connected layer to the first intermediate vector to generate a second intermediate vector. The fully connected layer may include a specific number of output neurons. In some examples, the specific number of output neurons may correspond to the number of available prostheses. For example, each of the output neurons may correspond to a different size of the prosthesis. Therefore, the second intermediate vector may include second intermediate features corresponding to prostheses of different sizes. In some examples, the output neurons correspond to different prosthesis parameters. Prosthesis parameters may specify aspects of one or more prostheses. For example, in an example involving shoulder arthroplasty, prosthesis parameters may include the size or radius of the glenoid ball of the glenoid prosthesis, the eccentricity of the glenoid ball, the base plate type of the glenoid prosthesis, the reinforcement type of the glenoid prosthesis, the stem size of the humeral prosthesis, the head ball size of the humeral prosthesis, and so on. In an example involving total ankle replacement surgery, prosthesis parameters may include the length of the tibial support, the size of the talus prosthesis, the size of the articular movable component (e.g., a polyethylene articular movable component coupled to the tibial support), and so on.

[0049] Some or all of the output neurons of the fully connected layer can receive some or all of the features of the first intermediate vector as input. Some or all of the output neurons of the fully connected layer can compute the result of a transfer function applied to a weighted sum of the inputs according to machine learning weights. Furthermore, in some examples, the planning system 118 can apply a softmax layer to the second intermediate vector to generate an output that includes the final vector. In some examples, the final vector includes final features corresponding to spurs of different sizes. The softmax layer can transform the second intermediate vector into a probability distribution of possible outcomes. The softmax layer can normalize the second intermediate vector to a probability distribution over the output class (e.g., available spurs). In this example, the planning system 118 can determine the recommended spurs based on the final vector. For example, the planning system 118 can determine that the recommended spur is the spur corresponding to the highest (or lowest) value feature in the final vector. In some examples, the final vector includes features corresponding to different sets of mutually compatible spurs, and the planning system 118 can determine the set of mutually compatible spurs corresponding to the highest (or lowest) value feature in the final vector as the set of recommended mutually compatible spurs.

[0050] In some examples, the final vector includes features corresponding to different combinations of potential values ​​for the prosthesis parameters of one or more prostheses. For example, a first feature of the final vector may correspond to a first radius of the glenoid bulb of the glenoid prosthesis, a first substrate type of the glenoid prosthesis, and a first reinforcement type of the glenoid prosthesis; a second feature of the final vector may correspond to a first radius of the glenoid bulb of the glenoid prosthesis, a first substrate type of the glenoid prosthesis, and a second reinforcement type of the glenoid prosthesis; a third feature of the final vector may correspond to a first radius of the glenoid bulb of the glenoid prosthesis, a second substrate type of the glenoid prosthesis, and a second reinforcement type of the glenoid prosthesis; and so on. The reinforcement components of the glenoid prosthesis are positioned between the substrate of the glenoid prosthesis and the patient's scapula. Different reinforcement types have different shapes. For example, a first type of reinforcement has a bone contact surface parallel to the substrate, a second type of reinforcement has a bone contact surface inclined relative to the substrate over the entire diameter of the substrate, a third type of reinforcement is partially angled relative to the substrate and partially parallel to the substrate, and a fourth type of reinforcement has a patient-specific bone contact surface that conforms to the shape of the patient's scapula. Different substrate types may have different dimensions. For example, a first type of substrate may have a first width and height to engage a relatively small glenoid fossa, a second type of substrate may have a second width and height to engage a relatively large glenoid fossa, and a third type of substrate may engage the glenoid fossa and one or more other parts of the patient's scapula, such as the acromion or coracoid process. The third type of substrate can be used when the patient's scapula has complex deformities or trauma and requires additional support.

[0051] In examples where the final vector includes features corresponding to different combinations of potential values ​​for prosthesis parameters of two or more prostheses, these features are restricted to combinations of potential values ​​for prosthesis parameters that ensure compatibility between prostheses. For example, a tibial prosthesis with a first size may not be compatible with a talus prosthesis with a second size. Therefore, in this example, the final vector does not include features corresponding to a tibial prosthesis with a first size and a talus prosthesis with a second size. The planning system can determine recommended values ​​for the prosthesis parameters of one or more prostheses as combinations of prosthesis parameter values ​​corresponding to the highest (or lowest) value feature in the final vector.

[0052] This process can be summarized by the following equation:

[0053] In the above equation, Z It is a convolutional signal matrix. X It is a signal (e.g., a matrix of node eigenvectors). W （0） These are the weights of the first GCL layer (i.e., the input to the hidden layer), and W （1） These are the weights of the second GCL layer (i.e., hidden to the output layer). It can be defined as The normalized adjacency matrix, where, , A It is an adjacency matrix and I N It is an identity matrix. It can be equal to In the above equation, It can represent the first GCL layer and It can represent the second GCL layer. Readout The presentation layer (i.e., the readout layer) applies mean pooling to the node embeddings of the second GCL layer. MLP This indicates a fully connected layer before the final softmax layer.

[0054] In some examples, the planning system 118 can automatically perform one or more checks to verify that one or more recommended prostheses, or recommended values ​​for one or more prosthesis parameters, are appropriate for the patient. For example, after determining a recommended prosthesis based on the output of the GCN model 116, the planning system 118 can verify whether the recommended prosthesis is properly positioned on the cortical bone. For example, if the recommended prosthesis is not positioned on the cortical bone, the prosthesis may actually sink into the cancellous bone. Therefore, the prosthesis must have sufficient overlap with the cortical bone to ensure stable positioning of the prosthesis.

[0055] After the planning system 118 determines a recommended prosthesis, the surgeon can decide whether to accept the recommendation or choose another prosthesis. If the surgeon accepts the recommendation, they can surgically implant the recommended prosthesis. For example, if the planning system 118 recommends a specific glenoid prosthesis, the surgeon can surgically implant that specific glenoid prosthesis at the patient's site. In the example where the planning system 118 determines recommended values ​​for the prosthesis parameters, the surgeon can evaluate the recommended values, accept the recommended values, or choose different prosthesis parameter values. The surgeon can implant the prosthesis based on either the recommended or selected values ​​of the prosthesis parameters.

[0056] Training system 120 can train GCN model 116. Training system 120 may be included in the same computing system as the computing device / system implementing planning system 118, or in a different computing device or system. In some examples, training system 120 can perform one or more training epochs. In each training epoch, training system 120 can present one or more batches (e.g., full batches, mini-batches, or individual training examples) of training input examples to GCN model 116. For each training input example, training system 120 can compare the output generated by GCN model 116 with the corresponding labeled training output example using an error function (e.g., a loss function). In some examples, the error function is a cross-entropy error function, for example, as defined in the following equation:

[0057] In the above equation, ln is the natural logarithm function. y L It is a collection of node indexes with labels. Y It is a collection of tags, and Z This is the convolutional signal matrix (i.e., the output of GCN model 116). After each batch, the training system 120 can update the weights of the GCN model 116, for example, by applying a backpropagation process. During each epoch, the training system 120 can present the same training input examples to the GCN model 116. In addition to the training input examples, the training dataset may include validation examples. The training system 120 can use validation examples to validate the GCN model 116 but does not use validation examples to update the weights of the GCN model 116.

[0058] in addition, Figure 1The system also includes a fulfillment system 122. In some examples, the fulfillment system 122 is configured to automatically (e.g., robotically) select a chosen prosthesis from an available prosthesis shelf for shipment. In some examples, the fulfillment system 122 is configured to manufacture the selected patient-specific prosthesis. For example, the fulfillment system 122 may include an additive manufacturing system (e.g., a 3D printer) to manufacture one or more components of the patient-specific prosthesis.

[0059] Figure 2 This is a flowchart illustrating example operation of a computing system 100 according to one or more techniques of this disclosure. The flowchart of this disclosure is presented as an example. In other examples, the flowchart may include more, fewer, or different actions, and the actions may be performed in different orders.

[0060] exist Figure 2 In the example, for each of the multiple nodes in the graph, the planning system 118 can generate a feature vector for the node, which initially includes feature data (200) characterizing one or more aspects of the patient's bone. The graph includes one or more edges, where each edge connects a corresponding pair of nodes. The graph can be constructed in one of a variety of ways. For example, in some examples, nodes may include cross-sectional nodes corresponding to different cross-sections among multiple cross-sections of the bone. In such examples, for each of the cross-sectional nodes in the graph, the feature data initially included in the feature vector of the cross-sectional node characterizes one or more aspects of the bone at the cross-section corresponding to the node. In some examples, the cross-section is perpendicular to the mechanical axis of the bone. In other examples, the cross-section may be perpendicular to another axis of the bone (e.g., an anatomical axis) or otherwise arranged relative to the bone.

[0061] In some examples, multiple nodes may include nodes with feature vectors that initially include feature data characterizing one or more aspects of two or more bones in the patient. For example, multiple nodes may include nodes with feature vectors that initially include feature data characterizing one or more aspects of the patient's tibia, and nodes with feature vectors that initially include feature data characterizing one or more aspects of the patient's talus. In another example, multiple nodes may include nodes with feature vectors that initially include feature data characterizing one or more aspects of the patient's scapula, and nodes with feature vectors that initially include feature data characterizing one or more aspects of the patient's humerus.

[0062] For each node in the diagram, the node's feature vector initially includes feature data characterizing one or more aspects of the bone at the cross-section corresponding to the node. In some examples, for each cross-section of the bone, one or more aspects of the bone at the cross-section include one or more measurements of the bone's dimensions at the cross-section. For example, in one example (e.g., in the case of the tibia), one or more measurements of the bone's dimensions at the cross-section include the inner width of the bone at the cross-section and the anterior length of the bone at the cross-section (e.g., as described below regarding...). Figures 3A to 3C (As shown). In another example, one or more measurements of the bone at a cross-section of the bone may include one or more of the following: anterior distance (i.e., the distance from the center of the cross-section to the anterior side of the cross-section), posterior distance (i.e., the distance from the center of the cross-section to the posterior side of the cross-section), lateral distance (i.e., the distance from the center of the cross-section to the lateral side of the cross-section), and medial distance (i.e., the distance from the center of the cross-section to the medial side of the cross-section). Using the anterior, posterior, lateral, and medial distances as features input to the GCN model 116 can account for bone asymmetry and thus improve the prediction of recommended prostheses, for example by explaining prosthesis overhang or underhang relative to the bone in a way that may not be represented by just two distances, or by recommending asymmetrical prostheses.

[0063] In some examples, such as when the bone is the talus, the node's feature vector initially includes feature data characterizing a single measurement of the talus (e.g., medial-lateral width). In other words, in examples where the bone is the talus, one or more measurements may include a measurement of the medial-lateral width of the talus. In examples where the talus width is sufficient to select a talus prosthesis from multiple available talus prostheses, the medial-lateral width measurement may be the only required measurement of the talus. In other examples where one or more additional measurements of the talus (e.g., anteroposterior length) are required to select a talus prosthesis from multiple available talus prostheses, the node's feature vector may include one or more additional measurements of the talus.

[0064] In some examples, the planning system 118 acquires feature data 112 based on medical image data 110. For example, the planning system 118 can acquire 2D or 3D medical images, such as individual CT images, 3D models constructed from 2D CT images, 3D MRI images, and other types of medical images. In examples where the planning system 118 acquires 2D medical images, the planning system 118 can select 2D medical images corresponding to cross-sections of bone, which the planning system 118 will use to acquire feature data. In some examples where the planning system 118 acquires 2D medical images, the planning system 118 can interpolate one or more cross-sections of bone that the planning system 118 will use to acquire feature data based on two or more 2D medical images. In some examples where the planning system 118 acquires 3D medical images, the planning system 118 can slice the 3D medical images to obtain cross-sections of bone. The cross-sections of bone can be perpendicular to the mechanical axis of the bone. In some examples where the planning system 118 acquires 2D images or 3D models, the planning system 118 can interpolate pixel or voxel values ​​for desired cross-sectional locations.

[0065] In some examples, the feature vector of a node may include information about the bone mineral density (BMD) of the bone at the corresponding cross-section. Information about bone BMD can help determine the recommended prosthesis. For example, when BMD is relatively low, a larger prosthesis may be more suitable to increase the contact between the prosthesis and the denser cortical bone, and otherwise distribute the load of the prosthesis over a larger area. Furthermore, in some examples, voids may exist at one or more cross-sections of the bone. Voids can be areas where the intraosseous BMD is zero or very low. Typically, it is desirable for the prosthesis to contact solid bone rather than bone voids. Therefore, including void and other BMD information in the feature vector can help plan the system118 to recommend a prosthesis.

[0066] In some examples, the planning system 118 can preprocess the feature data 112. For instance, before using any features, the planning system 118 can apply a normalization function to the features. Applying a normalization function can improve numerical stability and gradient descent convergence. Here are some exemplary normalization functions:

[0067] In this normalization function, It is the normalized value of the feature. x These are the original values ​​of the features. It is the mean of the original values ​​of the features in the training dataset, and It is the standard deviation of the features in the training dataset.

[0068] In other examples, the graph is a 3D grid, and for each corresponding sampling location in the 3D arrangement of sampling locations, multiple nodes include nodes corresponding to the corresponding sampling location. The 3D arrangement of sampling locations may include sampling locations corresponding to one or more intraosseous locations. For some or all nodes, feature data initially included in the feature vector of the node represents one or more aspects of one or more bones at the sampling location corresponding to the node.

[0069] In addition, Figure 2 In the example, planning system 118 applies GCN model 116 to the graph and the feature vectors of the nodes to generate output (202). As part of applying GCN model 116, planning system 118 may perform one or more message passing rounds (204). In a graph where every node is connected to every other node, it may not be necessary to perform more than one message passing round. In other examples, such as in examples where the graph is a 3D grid graph, multiple message passing rounds may be performed because not all nodes are connected to each other.

[0070] After completing one or more message passing rounds, the planning system 118 may apply one or more GCLs to the feature vectors of nodes to generate node embeddings (206). After generating the node embeddings, the planning system 118 may generate an output based on the node embeddings (208). The planning system 118 may determine recommended values ​​for one or more recommended spurious entities or one or more spurious entity parameters based on the output (210). For example, the planning system 118 may apply one or more layers (e.g., pooling layers, fully connected layers, and softmax layers) of the GCN model 116 to the embeddings to obtain a final vector. In some examples, features in the final vector correspond to corresponding spurious entities (or sets of compatible spurious entities), and each feature in the final vector may indicate that the recommended spurious entity (or set of compatible spurious entities) should be the confidence level (i.e., confidence score) of the spurious entity corresponding to that feature. In such examples, the planning system 118 may determine that the recommended spurious entity is the spurious entity in the final vector corresponding to the feature with the highest (or lowest) value. In some examples, features in the final vector correspond to potential combinations of potential values ​​of one or more spurious entity parameters of one or more spurious entities. In such examples, planning system 118 can determine that the recommended values ​​of one or more prosthesis parameters for one or more prostheses are the values ​​of one or more prosthesis parameters in the final vector that correspond to the highest (or lowest) value feature.

[0071] In some examples, the planning system 118 may use a regression process to determine recommended values ​​for one or more prostheses or one or more prosthesis parameters based on the output of the GCN model 116. For example, in such examples, the output of the final layer of the GCN model 116 does not indicate the confidence level at which a particular prosthesis should be recommended. Instead, after the GCL layer and the readout layer, the GCN model 116 may include a final multilayer perceptron (MLP) that outputs one or more dimensions of a suitable prosthesis (e.g., width, length, height, size, globus sphere size, globus sphere radius, globus sphere offset, handle size, head sphere size, etc.). Thus, the GCN model 116 outputs one or more dimensions of a suitable prosthesis overall. In some examples, the first layer of the final MLP may include eight neurons (as mentioned above, in some examples, the second GCL layer has eight dimensions), and the second layer of the final MLP may include two neurons. The planning system 118 can then determine one or more recommended prostheses from a pool of available prostheses based on one or more dimensions (e.g., width and length). For example, planning system 118 can identify one or more recommended prostheses as those with the smallest dimensional difference from the output of GCN model 116 and that do not violate one or more constraints. Such constraints may include preventing the prosthesis's edges from extending beyond the corresponding edges of the bone. In this way, planning system 118 uses the output of GCN model 116 to regress the dimensions output by GCN model 116 to one of the available prostheses. In this way, planning system 118 can filter one or more recommended prostheses from multiple available prostheses. In other words, planning system 118 can eliminate or reduce the priority of prostheses with a large dimensional difference from the dimensions output by GCN model 116. Planning system 118 can display a user interface that provides information about the recommended prostheses. Presenting information about multiple recommended prostheses allows surgeons to compare these recommended prostheses and make informed decisions when selecting an implant for a patient. In some examples, execution system 112 manufactures a patient-specific prosthesis with the dimensions output by GCN model 116. In the example where GCN model 116 outputs a suitable prosthesis size, the true values ​​in the training data used to train GCN model 116 can specify the appropriate prosthesis size for different patients.

[0072] In some examples, the output of GCN model 116 (e.g., the output of the final MLP of GCN model 116) includes values ​​for one or more prosthesis parameters of the suitable prosthesis. For example, in some such examples, the first layer of the final MLP may include eight neurons, and the second layer of the final MLP may include neurons for each prosthesis parameter. The prosthesis parameters of the suitable prosthesis may characterize the shape and / or other properties of the suitable prosthesis. In some examples, the prosthesis parameters may include the width and length of the suitable prosthesis. In another example, the prosthesis parameters may include a set of prosthesis parameters for two or more cross-sections of the bone. The set of prosthesis parameters for the bone cross-sections may characterize the shape of the suitable prosthesis at the cross-section. For example, the set of prosthesis parameters for the bone cross-sections may include a given number (e.g., 4, 8, 12, etc.) of prosthesis parameters, where each prosthesis parameter indicates the distance from the center of the cross-section to the edge of the suitable prosthesis. In an example where the number of prosthesis parameters for the cross-section is 8, the distance may correspond to lines radiating from the center of the cross-section at 45° intervals.

[0073] In some examples where the output of GCN model 116 includes values ​​of one or more prosthesis parameters suitable for a prosthesis, planning system 118 can determine one or more recommended prostheses from a pool of available prostheses based on the values ​​of one or more prosthesis parameters. For example, planning system 118 can determine a recommended prosthesis as one that has the smallest difference in prosthesis parameter values ​​from the prosthesis parameter values ​​output by GCN model 116 and does not violate one or more constraints. In this way, planning system 118 uses the output of GCN model 116 to regress the dimensions output by GCN model 116 to one of the available prostheses. In some examples, planning system 118 outputs information about one or more recommended prostheses for display. Planning system 118 can receive user input indicating to select one of the recommended prostheses or another of the available prostheses. In some examples, fulfillment system 122 uses data generated by planning system 118 to select a chosen prosthesis from a shelf of available prostheses for packaging and shipping. In some examples, fulfillment system 122 manufactures a patient-specific prosthesis based on the prosthesis parameter values ​​output by GCN model 116. In the example where GCN model 116 outputs appropriate prosthesis parameter values, the true values ​​in the training data used to train GCN model 116 can specify the appropriate prosthesis parameters for different patients.

[0074] Figure 3A , Figure 3B and Figure 3C This is a conceptual diagram illustrating example cross-sections 300A, 300B, and 300C (collectively, "cross-section 300") of a model of bone 302 according to one or more technologies of this disclosure. Specifically, Figure 3A , Figure 3B and Figure 3CThe example shows three different cross-sections of a patient's tibia. Each cross-section 300 can correspond to a node in the graph. The model of bone 302 can represent the current (e.g., pathological) shape of bone 302.

[0075] For each of the cross sections 300A, 300B, and 300C, the planning system 118 can determine the medial widths 304A, 304B, and 304C (collectively referred to as "medial width 304") and the lateral widths 306A, 306B, and 306C (collectively referred to as "lateral width 306"). The medial width 304 and the lateral width 306 are defined by the distance between the projection point 308 and a point on the cortical bone of the tibia. The projection point 308 can be defined as the projection of the tibial plateau onto the cross section 300.

[0076] Furthermore, for each of the cross sections 300A, 300B, and 300C, the planning system 118 can determine the anterior lengths 310A, 310B, and 310C (collectively referred to as "anterior length 310") and the posterior lengths 312A, 312B, and 312C (collectively referred to as "posterior length 312"). The anterior length 310 and the posterior length 312 are defined by the distance between the projection point 308 and a point on the cortical bone of the tibia.

[0077] For each cross section 300, the planning system 118 can determine the first feature of the feature vector of the node corresponding to the cross section by adding the inner width and outer width of the cross section. The planning system 118 can determine the second feature of the feature vector of the node corresponding to the cross section by adding the front width and outer width of the cross section.

[0078] Figure 4 This is a conceptual diagram, illustrating example figure 400 of one or more technologies according to this disclosure that can be used with a GCN model. Figure 4 In the example, circles represent nodes in Figure 400. Lines between nodes represent edges in Figure 400. Figure 4 In the example, there are ten nodes, so Figure 400 can be adapted to be used at ten cross-sections of a bone, or at ten cross-sections of a set of two or more bones.

[0079] Figure 5A , Figure 5B , Figure 5C and Figure 5DThis is a conceptual diagram illustrating an example cross-section 500 of a model of the talus 502 according to one or more techniques of this disclosure. The model of the talus 502 can represent the current (e.g., pathological) shape of the talus 502. Each cross-section 500 may correspond to a node in the diagram. Furthermore, each of the cross-sections 500 may be perpendicular to the mechanical axis of the talus 502. In some examples, the cross-sections 500 are calculated at a depth of 5 mm to 8 mm from the mechanical axis of the talus 502. Each cross-section 500 may be spaced 1 mm apart.

[0080] For each of the cross sections 500A, 500B, 500C, and 504D, the planning system 118 can determine the inner widths 504A, 504B, 504C, and 504D (collectively referred to as "inner width 504") and the outer widths 506A, 506B, 506C, and 506D (collectively referred to as "outer width 506"). The inner width 504 and the outer width 506 are defined by the distance between the projection point 508 and a point on the cortical bone of the talus 502. In some examples, the projection point 508 can be defined by determining the centroid of two high-point markers of the talus 502 and projecting that centroid onto the cross section 500 along the mechanical axis of the talus 502.

[0081] Furthermore, for each of the cross sections 500A, 500B, 500C, and 500D, the planning system 118 can determine the anterior lengths 510A, 510B, 510C, and 510D (collectively referred to as "anterior length 510") and the posterior lengths 512A, 512B, 512C, and 512D (collectively referred to as "posterior length 512"). The anterior length 510 and the posterior length 512 are defined by the distance between the projection point 508 and a point on the cortical bone of the tibia.

[0082] For each cross section 500, the planning system 118 can determine the first feature of the feature vector of the node corresponding to the cross section by adding the inner width and outer width of the cross section. The planning system 118 can determine the second feature of the feature vector of the node corresponding to the cross section by adding the front width and back width of the cross section.

[0083] Figure 6This is a block diagram illustrating an example of a planning system 118 using primary and secondary GCNs according to one or more techniques of this disclosure. In some examples, the recommendation of one prosthesis may depend on the recommendation of another prosthesis. For example, the recommendation of a talus prosthesis may depend on the recommendation of a tibia prosthesis. In other words, the planning system 118 may determine which talus prosthesis to recommend based at least in part on the recommended (or user-selected) tibia prosthesis. In another example, the recommendation of a humeral prosthesis may depend on a glenoid prosthesis. For ease of explanation, if a first prosthesis needs to be selected / recommended before a second prosthesis, this disclosure may refer to the first prosthesis as the primary prosthesis and the second prosthesis as the secondary prosthesis. The primary prosthesis may be designed to be implanted in the first bone, and the secondary prosthesis may be designed to be implanted in the second bone. Challenges may exist related to how the machine learning model incorporates this dependency between prostheses. For example, a single machine learning model predicting primary and secondary prostheses from the same set of input data may be difficult to train and / or may include too many weights.

[0084] According to one or more techniques disclosed herein, the planning system 118 may include primary feature data 612, primary graph data 614, and a primary GCN model 616. Additionally, the planning system 118 may include secondary feature data 622, secondary graph data 624, and one or more secondary GCN models 626A to 626N (collectively referred to as "GCN model 626"). The primary feature data 612, primary graph data 614, and primary GCN model 616 may be similar to feature data 112, graph data 114, and GCN model 116. The planning system 118 may use the primary feature data 612, primary graph data 614, and primary GCN model 616 to generate a first output in the same manner as feature data 112, graph data 114, and GCN model 116. The planning system 118 may determine a recommended primary prosthesis for implantation on the first bone based on the first output.

[0085] Secondary feature data 622 characterizes one or more aspects of the second bone at a second plurality of cross-sections of the second bone. Furthermore, secondary map data 624 can define a secondary map constructed in one of a variety of ways. For example, in some examples, the secondary map may include sampling nodes corresponding to cross-sections of the second bone (i.e., cross-section nodes corresponding to different cross-sections among the plurality of cross-sections of the second bone), for example, in a manner similar to... Figure 4 In other examples, the secondary graph may include sampled nodes arranged as a 3D grid. For each of the cross-sectional nodes in the secondary graph, feature data initially included in the feature vector of the cross-sectional node characterizes one or more aspects of the second bone at the cross-section corresponding to the cross-sectional node.

[0086] For example, in an example where the sampling node corresponds to a cross-section of a second bone and the second bone is the talus, aspects of the second bone may include the length and width of the talus at each cross-section 500 as shown in the example of Figure 5. Secondary graph data 624 defines a second graph comprising a second plurality of nodes and one or more second sides. Each of the second plurality of nodes corresponds to a different cross-section in the cross-section of the second bone. For each node in the second graph, the node's feature vector initially includes feature data characterizing one or more second aspects of the second bone at the cross-section of the second bone corresponding to the node. In an example where the second graph is a 3D grid, the sampling nodes of the second graph may correspond to regions within the second bone (or regions within an image of the second bone).

[0087] Each of the secondary GCN models 626 can correspond to a different available primary prosthesis. For example, multiple tibial prostheses of different sizes may exist. Therefore, among the multiple tibial prostheses, there can be a secondary GCN model for each tibial prosthesis. Thus, after determining the recommended primary prosthesis (or after receiving an instruction from the user to select a primary prosthesis), the planning system 118 can select a secondary GCN model from the GCN models 626 based on the first prosthesis. After selecting the secondary GCN model, the planning system 118 can apply the secondary GCN model to the second graph and secondary feature data 622 to generate a second output. In some examples, the planning system 118 can determine a recommended second prosthesis for the patient based on the second output from a set of second prostheses compatible with the primary prosthesis corresponding to the primary GCN model. In this way, the recommended second prosthesis must be compatible with the primary prosthesis. In some examples, the planning system 118 determines recommended values ​​for one or more prosthesis parameters based on the second output, which specify aspects of the second prosthesis. For example, the output of the secondary GCN model may include a vector containing feature values ​​corresponding to different combinations of potential values ​​for the prosthesis parameters of the second prosthesis. The planning system 118 may determine recommended values ​​for the prosthesis parameters of the second prosthesis based on the values ​​in the output vector of the secondary GCN model. The planning system 118 may apply the secondary GCN model in the same manner as described above for applying the GCN model. The planning system 118 may determine one or more recommended prostheses or determine recommended values ​​for one or more prosthesis parameters in the same manner as described above for the GCN model.

[0088] Figure 7 This is a conceptual diagram illustrating an example process of recommending a talus prosthesis based in part on a recommended tibial prosthesis, according to one or more techniques of this disclosure. Figure 7 In the example, there are eight available tibial prostheses 700, labeled "1", "2", etc. The five available talus prostheses are 702A, 702B, 702C, 702D, and 702E (collectively referred to as "talus prostheses 702"), each with a different talus top size. For a given tibial prosthesis 700, the user or planning system 118 can select from two talus prostheses 702. In other words, talus prostheses 702A and 702B are paired with tibial prostheses 700A (i.e., tibial prostheses 1 and 2). The talus prostheses 702B and 702C are compatible with the tibial prosthesis 700B (i.e., tibial prostheses 3 and 3 extended), the talus prostheses 702C and 702D are compatible with the tibial prosthesis 700C (i.e., tibial prostheses 4 and 4 extended), and the talus prostheses 702D and 702E are compatible with the tibial prosthesis 700D (i.e., tibial prostheses 5 and 5 extended). Furthermore, polyethylene (polyethylene) joint movable parts of different sizes can be attached to different tibial prostheses 700. For example, the articulated joint component 1 or 1+ can be used with the tibial prosthesis 700A and the talus prosthesis 702A; the articulated joint component 2 can be used with a combination of the tibial prosthesis 700A and the talus prosthesis 702B; the articulated joint component 2+ can be used with a combination of the tibial prosthesis 700B and the talus prosthesis 702B; the articulated joint component 3 can be used with a combination of the tibial prosthesis 700B and the talus prosthesis 702C; the articulated joint component 3+ can be used with a combination of the tibial prosthesis 700C and the talus prosthesis 702C; the articulated joint component 4 can be used with a combination of the tibial prosthesis 700C and the talus prosthesis 702D; the articulated joint component 4+ can be used with a combination of the tibial prosthesis 700D and the talus prosthesis 702D; and the articulated joint component 5 can be used with a combination of the tibial prosthesis 700D and the talus prosthesis 702E.

[0089] In some examples, the planning system 118 may apply a primary GCN model 616 to determine which of the tibial prostheses 700 is recommended. Different secondary GCN models 626 may correspond to different sets of one or more tibial prostheses 700. For example, if one of the tibial prostheses 700A is a recommended tibial prosthesis, the planning system 118 may use a primary GCN model to determine which of the talus prostheses 702A or 702B is recommended (and, in some examples, also includes the joint assembly); if one of the tibial prostheses 700B is a recommended tibial prosthesis, the planning system 118 may use a secondary GCN model to determine which of the talus prostheses 702B or 702C is recommended (and, in some examples, also includes the joint movement component); and so on.

[0090] In some examples, a single GCN model exists (e.g., GCN model 116), and the final vector generated based on the output of the GCN model includes feature values ​​corresponding to different combinations of tibial prostheses, talar prostheses, and aggregate sizes. For example, the final vector may include: a first feature corresponding to tibial prosthesis 1, aggregate size 1, and talar prosthesis 702A; a second feature corresponding to tibial prosthesis 1, aggregate size 1+, and talar prosthesis 702A; a third feature corresponding to tibial prosthesis 1, aggregate size 2, and talar prosthesis 702B; and a fourth feature corresponding to tibial prosthesis 2... Size 1 corresponds to talus prosthesis 702A; fifth feature, corresponding to tibial prosthesis 2. The sixth feature corresponds to size 1+ and talus prosthesis 702A; and the tibial prosthesis 2. The size 1 corresponds to the talus prosthesis 702A; and so on. Since the final vector only includes features corresponding to compatible prostheses, the planning system 118 will not recommend sets of incompatible prostheses or sets of incompatible prosthesis parameters.

[0091] Figure 8 This is a conceptual diagram illustrating example Figure 800 that can be used with a GCN model according to one or more techniques of this disclosure. Figure 800 is similar to Figure 400 ( Figure 4 However, it includes fewer nodes. The planning system 118 can use Figure 400 with ten cross sections and Figure 800 with four cross sections. For example, the planning system 118 can use ten cross sections when recommending a tibial prosthesis and four cross sections when recommending a corresponding and compatible talus prosthesis.

[0092] Figure 9 This is a flowchart illustrating an example operation of a computational system 100 that recommends two interacting simulants according to one or more techniques of this disclosure. Figure 9 In the example, planning system 118 can determine (900) recommended values ​​for a first recommended prosthesis or one or more prosthesis parameters of a first prosthesis, for example, a prosthesis for implantation in a first bone (such as bone 302). Planning system 118 can determine recommended values ​​for a first recommended prosthesis or one or more prosthesis parameters of a first recommended prosthesis in the same manner as described elsewhere, such as regarding... Figure 2 .

[0093] Additionally, the planning system 118 can obtain (902) secondary feature data 622 characterizing one or more second aspects of the second bone (e.g., talus 502) at multiple second cross sections of the second bone. The second graph (e.g., Figure 800) includes a second plurality of nodes and one or more second sides. The second graph can be defined by secondary graph data 624. The sampling nodes of the second graph can correspond to different sampling locations. For example, one or more sampling nodes among the second plurality of nodes correspond to different cross sections (e.g., cross section 500 of the second bone). In some examples, one or more sampling nodes among the second plurality of nodes correspond to different regions. For each node in the second graph, the node's feature vector initially includes feature data (e.g., secondary feature data 622) characterizing one or more second aspects of the second bone corresponding to the node at the cross section of the second bone. For example, in the case where the second bone is the talus, one or more second aspects of the talus at each cross section of the talus include a single measurement of the talus, such as the inner-outer width of the talus at the cross section. Therefore, in this example, the node's feature vector initially may include the inner-outer width of the talus at the cross section of the talus corresponding to the node.

[0094] Planning system 118 can select (904) a second GCN model from multiple secondary GCN models (e.g., secondary GCN model 626) based on the first prosthesis. For example, planning system 118 can use a predefined mapping from the first prosthesis to a specific one of the secondary GCN models. Planning system 118 can use the output of the secondary GCN model mapped to the primary prosthesis to determine the recommendation of a secondary prosthesis compatible with the primary prosthesis. For example, there may be two secondary prostheses compatible with the primary prosthesis, and planning system 118 can use the output of the selected secondary GCN model to determine which of the two secondary prostheses to recommend to the patient. In some examples, only a single secondary GCN model may exist and the selection of the second GCN model may be omitted.

[0095] After selecting the second GCN model, the planning system 118 can apply the second GCN model (906) to the second graph and the feature vectors for the second plurality of nodes to generate a second output. As part of applying the second GCN model, the planning system 118 can perform one or more message passing rounds (908). In a graph where each node is connected to every other node, it may not be necessary to perform more than one message passing round. The planning system 118 can then apply one or more GCLs (910) to the feature vectors of the nodes to generate the embeddings of the nodes. In some examples where the second bone is the talus and the feature vector of each node can be a 1-dimensional vector, the feature vector initially includes the inner-outer width of the talus at the cross-section of the talus corresponding to the node. In such examples, the first GCL of the second GCN model can increase the dimension of the feature vector from a 1-dimensional embedding space to a 2-dimensional embedding space. The second GCL of the second GCN model can modify the values ​​within the 2-dimensional embedding space.

[0096] After generating the node embeddings, the planning system 118 can generate a second output (912) based on the node embeddings. For example, the planning system 118 can apply a pooling layer to the node embeddings. The pooling layer generates a first intermediate vector that includes first intermediate features corresponding to spurs of different sizes. For example, the number of nodes in the graph can be equal to 4, and the final GCL layer can embed 2D features for each output of the nodes. Therefore, there can be 4×2 features after the final GCL layer. In this example, the pooling layer can reduce the number of features to 1×2. For example, the pooling layer can determine the mean (e.g., average) of the corresponding features for each of the four nodes, thus obtaining a first intermediate vector including 2 features.

[0097] Additionally, the planning system 118 can apply a fully connected layer to the first intermediate vector to generate a second intermediate vector. The fully connected layer may include a specific number of output neurons. This specific number of output neurons may correspond to the number of available secondary prostheses. For example, the output neurons may include output neurons corresponding to different sizes of the secondary prostheses. Therefore, the second intermediate vector may include second intermediate features corresponding to different available sizes of the secondary prostheses. Similarly, in some examples, the output neurons may include output neurons corresponding to different values ​​of one or more prosthesis parameters (e.g., size, radius, etc.) of the secondary prosthesis. Some or all of the output neurons of the fully connected layer may receive each of the features of the first intermediate vector as input. Each of the output neurons of the fully connected layer may compute the result of a transfer function applied to a weighted sum of inputs according to machine learning weights.

[0098] Furthermore, in some examples, the planning system 118 can apply a softmax layer to the second intermediate vector to generate an output that includes a final vector. The final vector may include final features corresponding to prostheses of different sizes. The softmax layer can transform the second intermediate vector into a probability distribution of possible outcomes. The softmax layer can normalize the second intermediate vector to a probability distribution over the output category (e.g., available prostheses).

[0099] The planning system 118 can determine (914) recommended values ​​for one or more prosthesis parameters for a secondary prosthesis or a secondary prosthesis for implantation into a patient based on the second output. For example, in an example where the second output includes a final vector containing final features corresponding to prostheses of different sizes, the planning system 118 can determine that the recommended secondary prosthesis is the one corresponding to the highest (or lowest) value feature in the final vector. In other examples, the planning system 118 can use a regression process to determine the primary and secondary prostheses based on the final vector. In such examples, the regression process can be similar to the regression process described above (e.g., regarding...). Figure 2 In other examples, the second output may include the size and / or parameters of a suitable prosthesis, and the planning system 118 may determine a recommended prosthesis from a pool of available prostheses based on the size and / or parameters. In some examples, the planning system 118 may determine recommended values ​​for one or more prosthesis parameters of a secondary prosthesis in the same manner as described elsewhere in this disclosure for determining recommended values ​​for prosthesis parameters. In some examples where the second output includes size and / or parameters, the planning system 118 may determine a customized, patient-specific prosthesis based on the size and / or parameters.

[0100] Figure 10 This is a conceptual diagram illustrating an example 3D grid map 1000 according to one or more techniques of this disclosure. Figure 10 In the example, points correspond to nodes and lines correspond to edges. Furthermore, in... Figure 10 In the example, grid plot 1000 is a 4×4×4 grid. In other examples, grid plot 1000 may have other sizes.

[0101] exist Figure 10 In the example, up to six edges can connect to each of the nodes. Therefore, for each node, the edges of the grid graph 1000 can include a set of edges connecting the node to up to six other nodes out of a plurality of nodes. In other examples, there may be additional restrictions on the number of edges that can connect to a single node. For example, for each node, the edges of the grid graph can include a set of edges connecting the node to up to eight other nodes out of a plurality of nodes. In another example, for each node, the edges of the grid graph can include a set of edges connecting the node to up to twelve other nodes out of a plurality of nodes.

[0102] The planning system 118 can acquire a 3D arrangement of sampling locations. In some examples, the 3D arrangement of sampling locations is at least partially predefined. Therefore, sampling locations may include a set of sampling locations corresponding to predefined coordinates within the 3D image data of the bone. In some examples, the 3D arrangement of sampling locations may be an equidistant 3D arrangement. For example, each of the sampling locations may be spaced 2 mm, 5 mm, or other distances in each aspect of the width, length, and height dimensions. In some examples, the distance between sampling locations may differ in one or more dimensions. For example, the distance between sampling points may be equal to a first value in the width and length dimensions, and equal to a second different value in the height dimension. Sampling locations may be defined at coordinates relative to a specific anatomical landmark (e.g., the distal end of the tibia, the centroid of the talus, etc.) or a user-defined location.

[0103] In other examples, the 3D arrangement of the sampling locations is not predefined. For instance, in some examples, the planning system 118 may apply a segmentation process to image data of bone (e.g., 2D or 3D image data) to identify multiple regions. The boundaries of these regions may correspond to anatomical boundaries. Regions may correspond to sampling locations among multiple sampling locations. Therefore, sampling locations may correspond to different regions. Two or more regions may have different sizes. In some examples, the segmentation process is an oversegmentation process that can even identify regions within anatomical structures. For example, the segmentation process may identify multiple 3D regions within the tibia. In some examples, the oversegmentation process used by the planning system 118 can be the so-called SLIC algorithm (Radhakrishna Achanta, Appu Shaji, Kevin Smith, Aurelien Lucchi, Pascal Fua and Sabine Süsstrunk, “SLIC Superpixels Compared to State-of-the-art Superpixel Methods”, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 34, No. 11, pp. 2274-2282, May 2012).

[0104] In some examples where the segmentation process is applied to the planning system 118, the number of regions identified by the segmentation process is not available before the segmentation process is applied. Therefore, the planning system 118 can adaptively generate graph data 114 such that the number of nodes in the graph is determined based on (e.g., equal to, proportional to, etc.) the number of regions identified by the segmentation process. In some examples, the planning system 118 does not generate nodes for regions of no interest. For example, in some examples, the planning system 118 does not generate nodes corresponding to regions such as soft tissue, open spaces, or bones of no interest. In use cases involving total ankle replacement, examples of bones of no interest could include the fibula, scaphoid, calcaneus, etc. Excluding nodes from regions of no interest can reduce the graph size and, correspondingly, reduce processing time when applying the GCN model 116. In other examples, the number of regions identified by the segmentation process can be predetermined.

[0105] Furthermore, for each region, the planning system 118 can generate edges that connect the node corresponding to that region to nodes in regions adjacent to that region. In some examples, the planning system 118 can limit the number of edges connected to any particular node to a specific constraint. In other words, the planning system 118 can generate edges such that the number of nodes allowed to connect to any given node does not exceed a specific number. For example, the number of edges allowed to connect to a given node can be limited to six, eight, twelve, etc.

[0106] The planning system 118 can generate feature data for each of the nodes. For example, for each node, the feature data initially included in the node's feature vector represents one or more aspects of the bone at the sampling location corresponding to the node.

[0107] In an example where the planning system 118 applies a segmentation process to 3D image data of bone to identify multiple regions, the planning system 118 can generate, for at least one region, a feature vector corresponding to a node at a sampling location corresponding to the region, based on data associated with one or more pixels of the 3D image data within the region. In some examples, one or more pixels of the 3D image data comprise multiple pixels of the 3D image data. Furthermore, in some examples, the region is a 3D region, and each of the coordinates of two or more pixels included in the region is different. In other words, the pixels in the 3D region do not necessarily form a straight line in one dimension.

[0108] In some examples, the feature data initially included in the feature vector of a node may include bone mineral density measurements at the sampling location corresponding to the node. Within the region corresponding to the node, the bone mineral density measurement may be the average of bone mineral density measurements (e.g., measured in Henle units) or other values ​​based on the bone mineral density measurement. In other examples, the feature data of a node may include bone mineral density measurements relative to a single point (e.g., the centroid of the region corresponding to the node, a predefined location, etc.).

[0109] In some examples, for one or more nodes, the feature data initially included in the node's feature vector can describe the local texture of the bone (e.g., cortical bone, cancellous bone, etc.). The description of the local texture can be the average (or other statistic) of values ​​indicating the local texture at points within the region corresponding to the node. In other examples, the description of the local texture can correspond to a single point (e.g., the centroid of the region corresponding to the node, a predefined location, etc.).

[0110] In some examples, for one or more nodes, the feature data initially included in the node's feature vector includes a gray-level co-occurrence matrix (GLCM). The GLCM is based on gray-level pixel values, which can also be referred to as intensity values. Intensity values ​​may correspond to bone density measurements or other information. Typically, the planning system 118 can generate the GLCM using predefined spatial relationships. These predefined spatial relationships define the relationship between a reference pixel and its neighboring pixels. Examples of spatial relationships could include a neighboring pixel being one pixel to the right of the reference pixel, a neighboring pixel being three pixels above the reference pixel, a neighboring pixel being two pixels above the reference pixel and two pixels to the left of the reference pixel, and so on. The planning system 118 can generate a matrix (i.e., the GLCM) of size (intensity range × intensity range), where each cell of the matrix is ​​initialized to 0. For example, for an 8-bit single-channel image, the planning system 118 can generate a 256 × 256 matrix. The planning system 118 can traverse the image, and for each unique ordered pair of intensity values ​​found for the defined spatial relationships, the planning system 118 can increment the cells in the matrix whose coordinates match that ordered pair. For example, if the spatial relationship is that an adjacent pixel is 1 pixel to the right of a reference pixel, and there are 4 instances in the image where the adjacent pixel and the reference pixel have values ​​of 1 and 3 respectively, then the planning system 118 sets the value at coordinates (1, 3) in the matrix to be equal to 4. In some examples, the gray-level co-occurrence matrix in the node's feature vector can represent the texture within the region corresponding to the node. In other words, the "image" on which the planning system 118 calculates the GLCM is based is the region corresponding to the node. In some examples, the image on which the planning system 118 generates the GLCM for a node can include the entire image or a subset of the image. Using a grid map can be advantageous in some use cases because the feature vectors of grid map nodes can include other types of information besides distance and size, which can improve recommendation quality.

[0111] As discussed above, several challenges exist associated with existing computerized systems used for recommending (e.g., filtering) prostheses. In addition to the GCN models discussed above, this disclosure also describes techniques based on generative pre-trained transformers (GPTs) for recommending prostheses or recommending prosthesis parameters. GPT-based techniques allow computational systems to efficiently consider the interrelationships of various aspects of bone at different sampling locations within the bone.

[0112] Figure 11 This is a block diagram illustrating example components of a planning system 118 according to one or more technologies of this disclosure. Figure 11In the example, the components of the planning system 118 include a machine learning model 1130, a prediction unit 1102, a training unit 1104, and a feature augmentation unit 1106. The machine learning model 1130 includes a feature encoder 1110, a quantization module 1112, a feature decoder 1114, a generative pre-trained transformer (GPT) 1134, and a codebook 1118. In other examples, the planning system 118 can be implemented using more, fewer, or different components. For example, if the ML model 1130 has already been trained, the training unit 1104 can be omitted. In some examples, one or more components of the planning system 118 are implemented as software modules. Furthermore, Figure 11 The components are provided as examples, and the planning system 118 can be implemented in other ways.

[0113] Typically, prediction unit 1102 applies ML model 1130 to the input anatomical model to determine recommended values ​​for one or more prostheses or one or more prosthesis parameters for the patient. Feature encoder 1110 of ML model 1130 generates feature maps and input features based on the input anatomical model. The input anatomical model can be a grid map, such as grid map 1000. Feature encoder 1110 can then apply a graph convolutional encoder to the feature maps and input features to generate face embeddings. Quantization module 1112 uses codebook 1118 to generate quantized face embeddings based on the face embeddings. Prediction unit 1102 can sort the quantized face embeddings to form an input label sequence for GPT 1134. GPT 1134 generates an output label sequence (e.g., a predicted codebook index) based on the input label sequence (i.e., the quantized face embedding sequence). Prediction unit 1102 uses codebook 1118 to generate a sequence of quantized face embeddings (e.g., a sequence of output labels) based on the predicted codebook index. The feature decoder 1114 uses quantized patch embedding sequences to generate a prediction model.

[0114] Training unit 1104 can train ML model 1130. Training unit 1104 can train ML model 1130 using a set of training examples 1132. In one or more examples, training unit 1104 can be configured to train ML model 1130 using multiple training epochs. Feature augmentation unit 1106 can generate synthetic training examples to augment training example 1132. In some examples, ML model 1130 can be trained by a device or system outside of computing system 100. In such examples, planning system 118 can receive ML model 1130, including GPT 1134.

[0115] Figure 12 This is a flowchart illustrating an example operation of a planning system 118 according to one or more techniques of this disclosure. Figure 12In the example, the planning system 118 may acquire (1200) an input anatomical model, which includes a first 3D mesh representing at least a first portion of the patient's anatomical structure. The planning system 118 may generate (1202) an input token based on the input anatomical model. The planning system 118 may apply GPT 1134 to (1204) the input token to generate an output token. The planning system 118 may determine (1206) recommended values ​​for one or more recommended prostheses or one or more prosthesis parameters for the patient based on the output token.

[0116] Figure 13 This is a block diagram illustrating an example architecture for generating quantized patch embeddings and generating reconstructed anatomical models according to one or more techniques of this disclosure. Figure 13 In the example, feature encoder 1110 includes feature generator 1300 and graph convolution encoder 1302. Feature decoder 1114 includes decoder 1342. Figure 13 The architecture can be used to train the graph convolutional encoder 1302 and decoder 1342 separately from the GPT 1134.

[0117] The feature encoder 1110 acquires an input anatomical model 1304 representing one or more anatomical structures (e.g., bone or soft tissue structures) of the patient. The feature generator 1300 generates a patch map and input features 1306 based on the input anatomical model 1304. In the patch map, each patch corresponds to a node in the graph. Nodes corresponding to adjacent patches are connected in the graph by undirected edges. Each node has a feature vector. For each node, the feature vector may include nine coordinate values ​​for each vertex of the corresponding patch (i.e., three coordinate values ​​for each of the three dimensions), data defining the patch normal vector, data defining the angles between the edges of the corresponding patch, and data indicating the area of ​​the corresponding patch.

[0118] Graph convolutional encoder 1302 generates patch embeddings 1308 based on a patch map and input features 1306. Graph convolutional encoder 1302 performs one or more message passing rounds. During each message passing round, graph convolutional encoder 1302 collects feature vectors of neighboring nodes for each node in the graph. Then, graph convolutional encoder 1302 aggregates the feature vectors of a node with the feature vectors of its neighboring nodes to update the node's feature vector. Aggregating feature vectors may include determining the average value of corresponding features in the feature vector. After completing one or more message passing rounds, graph convolutional encoder 1302 may use the updated feature vector of each node in the graph as input to neural network 1307, where neural network 1307 is trained to generate patch embeddings of the patches corresponding to the nodes. In this way, graph convolutional encoder 1302 can generate patch embeddings for each patch of the input anatomical model.

[0119] In some examples, the neural network 1307 includes a series of graph convolutional layers, such as the SAGE-Conv graph convolutional layer. In some such examples, the first layer of the neural network 1307 takes 16 features as input: 9 features indicating the point coordinates of the patch, 3 features indicating the normal vector of the patch, 3 features indicating the angle of the vertices of the patch, and one feature indicating the area of ​​the patch.

[0120] Quantization module 1112 generates quantized patch embedding 1328 based on patch embedding 1308. Figure 13 In the example, quantization module 1112 includes a feature segmentation unit 1320 configured to segment the face embedding 1308 of each face into multiple sub-vectors 1312. Each of the sub-vectors 1312 may correspond to a vertex of the face. Aggregation unit 1322 of quantization module 1112 can then aggregate the sub-vectors 1312 of corresponding vertices into an aggregated sub-vector 1323. For example, each vertex of the model may have an index. Therefore, two faces sharing a vertex, along with information about those faces, can indicate the same index for the shared vertex. When aggregating the sub-vectors of corresponding vertices, quantization module 1112 can calculate the average of the features of vertices with the same index in the sub-vectors.

[0121] For each of the aggregate sub-vectors 1323, the codebook lookup unit 1324 of the quantization module 1112 can look up the codebook index of that aggregate sub-vector in the codebook 1118. The codebook 1118 includes multiple representative vectors. Each representative vector may have the same number of elements as each aggregate sub-vector. Each representative vector is associated with a unique codebook index. To look up the codebook index of the aggregate sub-vector in the codebook 1118, the codebook lookup unit 1324 can determine which representative vector is closest to the aggregate sub-vector. For example, the codebook lookup unit 1324 can treat the aggregate sub-vector and the representative vector as points in a specified multidimensional space, calculate the Euclidean distance between the aggregate sub-vector and the representative vector, and determine which of the representative vectors has the shortest Euclidean distance. In this way, the codebook lookup unit 1324 can output multiple codebook indices 1326 for each patch. Furthermore, in some examples, the codebook lookup unit 1324 can apply a hierarchical residual quantization process, which determines multiple codebook indices for each aggregate sub-vector of the patch. In some examples, the representative vectors in codebook 1118 are learned. Training unit 1104 can use... k - The mean clustering algorithm is used to learn the representative vectors. In training unit 1104, it is used... k - In the example of using the mean clustering algorithm to learn representative vectors, the representative vectors can be cluster center vectors that indicate the cluster centers.

[0122] The shaper unit 1310 of the quantization module 1112 uses codebook index 1326 to generate quantized patch embeddings 1328. That is, the shaper unit 1310 receives a stack of codebook indices 1326 for each patch. For each patch, the shaper unit 1310 can reduce the stack of codebook indices for each patch to a single feature embedding for each patch by summing over the representative vectors and concatenating them at the vertices, thereby generating the quantized patch embedding 1328. For example, the shaper unit 1310 can determine the quantized patch embedding 1328 using the following equation:

[0123] In the above equation, This indicates that the vector containing the facets is a 1328-bit vector containing the facet quantization facet embedding. Indicator faceplate 1 to N Single quantized patch embedding, It's a sheet of dough. i vertex v The d Each codebook index (i.e., tag). Indicator codebook 1118 and codebook index The corresponding representative vector, and Indicator face i vertex v =0 tov =2 feature splicing.

[0124] The feature decoder 1114 includes a sequence generator 1340 and a decoder 1342. The sequence generator 1340 generates a quantized patch embedding sequence 1344. The quantized patch embeddings in the quantized patch embedding sequence 1344 can be ordered in the same manner as the patch embedding 1308.

[0125] Decoder 1342 generates a reconstructed anatomical model 1346 based on the quantized patch embedding sequence 1344. For example, decoder 1342 outputs a set of nine coordinate values ​​for each patch of the reconstructed anatomical model 1346. The reconstructed anatomical model 1346 is a reconstructed version of the input anatomical model 1304. Therefore, the reconstructed anatomical model 1346 does not include additional information, such as information indicating recommended values ​​for one or more prostheses or one or more prosthesis parameters. Decoder 1342 can be implemented as a 1DResNet decoder. In some examples, the ResNet decoder includes 34 layers; that is, decoder 1342 can be implemented using a ResNet34 model. In other examples, other decoders can be used, such as a ResNet18 model, a ResNet101 model, or another type of neural network model. Generally, if decoder 1342 has more layers, decoder 1342 can be able to reconstruct structures with greater variability. In the example where ML model 1130 is only used to generate a limited number of structures (such as the scapula or tibia), an implementation of decoder 1342 with 18 or 34 layers may be sufficient, as opposed to the 101-layer ResNet 101 model. Using fewer layers can save computational resources (e.g., handling loops and memory) while maintaining similar performance. Inference time can also be reduced in models with fewer layers.

[0126] Figure 14 This is a flowchart illustrating an example operation of a prediction unit 1102 for generating information indicating recommended values ​​for one or more recommended prostheses or prosthesis parameters for a patient, according to one or more techniques of this disclosure. Figure 14 In the example, the feature encoder 1110 acquires the input anatomy model and generates patch embeddings based on the input anatomy model, as described above regarding... Figure 13 As described in (1400). Additionally, the planning system 118 applies a graph convolutional encoder 1302 to the feature map and input features to generate patch embeddings (1402). The quantization model 1112 generates quantized patch embeddings 1328 (1404) based on the patch embeddings.

[0127] The prediction unit 1102 can then generate a feature sequence (i.e., an input label sequence) (1406) based on the quantized patch embeddings. The feature sequence can include features for each of the quantized patch embeddings. The features of the quantized patch embeddings can be based on the quantized patch embeddings and discrete positional encodings learned for the quantized patch embeddings. The learned discrete positional encodings for the quantized patch embeddings provide information about the position of the quantized patch embedding in the feature sequence and an index for each embedding within the quantized patch embedding. In other words, for each embedding within a quantized patch embedding, a positional encoding value for that embedding is determined, and then that positional encoding value is added to that embedding (i.e., summed with the embedding). In some examples, the function used to determine the positional encoding value is learned as part of the training process for GPT 1134. In some examples, the function used to determine the positional encoding value includes a sine function. Additionally, the prediction unit 1102 can include a start embedding at the beginning of the sequence and an end embedding at the end of the sequence. The start embedding indicates the beginning of the sequence. The end embedding indicates the end of the sequence.

[0128] Prediction unit 1102 can autoregressively apply GPT 1134 to the feature sequence to generate output labels (1408). The feature sequence may include quantized patch embeddings. The output labels may include a sequence of codebook indexes for patches of at least a portion of an anatomical structure or prosthesis. For example, prediction unit 1102 can apply GPT 1134 to the feature sequence to generate a set of predicted codebook indices for patches. If the set of predicted codebook indices does not include stop markers, prediction unit 1102 can use codebook 1118 to determine the quantized patch embeddings of the patches. Prediction unit 1102 can then update the quantized patch embedding sequence to include the quantized patch embeddings. Prediction unit 1102 can use the updated quantized patch embedding sequence as input to GPT 1134 to generate a set of predicted codebook indices. This loop can continue until the set of predicted codebook indices includes stop markers. GPT 1134 may be a GPT-2 medium architecture. That is, the GPT transformer can have 24 multi-head self-attention layers, 16 heads, 768 feature widths and 4608 context lengths, for example, as described by Siddiqui et al. in “MeshGPT: Generating Triangle Meshes with Decoder-OnlyTransformers” (arXiv:2311.15475v1 [cs.CV] November 27, 2023).

[0129] In some examples, prediction unit 1102 may apply GPT 1134 to a first input label and a second input label to generate an output label. The first input label may be based on an input anatomical model (e.g., the first input label may include quantized patch embeddings). The second input label may include information describing the patient but not the first 3D mesh. The information describing the patient includes one or more of the following: the patient's sex, the patient's age, the glenoid type of the patient's scapula, the pathology of the anatomical structure, and other information describing the patient. Including the second input label can improve the accuracy of the predicted anatomical model 132.

[0130] Prediction unit 1102 can determine the quantized patch embedding sequence (1410) based on the codebook index. Prediction unit 1102 can proceed according to the above-mentioned... Figure 13 The same method is used to determine the quantization patch embedding.

[0131] The prediction unit 1102 can then apply the feature decoder 1114 to quantize the patch embedding sequence to generate a prediction model (1412). The prediction model may include patches corresponding to one or more anatomical structures and one or more prostheses. In some examples, the prediction unit 1102 may update the patch embedding sequence and the predicted anatomy model 132 as it progressively predicts an additional set of prediction codebook indices using GPT 1134.

[0132] Prediction unit 1102 can determine recommended values ​​(1414) for one or more recommended prostheses or one or more prosthesis parameters for the patient based on a prediction model. For example, prediction unit 1102 can compare the size of the prosthesis in the prediction model with the size of available prostheses to determine which of the available prostheses is the closest match. Prediction unit 1102 can determine that the recommended prosthesis is the available prosthesis whose size is closest to the prosthesis size in the prediction model. In some examples, prediction unit 1102 can determine the recommended value of the prosthesis parameter as the value of the corresponding prosthesis parameter in the prediction model.

[0133] Figure 15 This is a flowchart illustrating an example training process for an ML model 1130 according to one or more techniques of this disclosure. As described above, training unit 1104 can train ML model 1130. That is, training unit 1104 can train ML model 1130 to generate input labels based on an input anatomical model, apply GPT 1134 of ML model 1130 to the input labels to generate output labels; and generate a predictive anatomical model based on the output labels, the predictive anatomical model including a second 3D mesh representing the surfaces of one or more prostheses.

[0134] In the first stage of training the ML model 1130, the training unit 1104 can train the graph convolutional encoder 1302 and the feature decoder 1114 and learn the representative vector (1500) of the codebook 1118. In the second stage of training the ML model 1130, the training unit 1104 can train GPT 1134 (1502).

[0135] In the first phase, training unit 1104 may execute one or more training cycles. Within each training cycle, training unit 1104 may execute a series of training iterations. In each training iteration, training unit 1104 provides an input anatomical model as input to feature encoder 1110 based on training example 1132. Feature encoder 1110, quantization module 1112, and feature decoder 1114 may generate a reconstructed anatomical model based on the input anatomical model. During the first phase, the input anatomical model may be a complete model of one or more anatomical structures along with one or more implanted prostheses. After generating the reconstructed anatomical model, training unit 1104 may compute a reconstruction loss value based on a comparison between the input anatomical model and the reconstructed anatomical model. For example, training unit 1104 may use cross-entropy loss on discrete grid coordinates to compute the reconstruction loss value. Training unit 1104 may use backpropagation to update the parameters (e.g., weights) of feature encoder 1110 and feature decoder 1114 based on the reconstruction loss value.

[0136] In some examples, training unit 1104 updates the representative vector of codebook 1118 as part of the training process. For example, training unit 1104 may add aggregated sub-vectors 1323 generated during training iterations to the training set of aggregated sub-vectors. Training unit 1104 may assign weights to each aggregated sub-vector in the training set such that the weight decreases exponentially the longer the aggregated sub-vector is retained in the training set. In some examples, training unit 1104 uses... k - The mean clustering process updates the representative vector based on the aggregated subvectors in the training set.

[0137] In some examples, quantization module 1112 applies a hierarchical residual vector quantization process. For example, training unit 1104 can initialize... k A set of first-level representative vectors is used to generate aggregated sub-vectors (i.e., first-level vectors) that are closest to the first-level representative vectors. The first-level representative vectors are then updated based on the average position of the first-level vectors assigned to them. Training unit 1104 can repeat the assignment and update steps until a termination condition is met. For each of the first-level vectors, training unit 1104 can compute second-level residual vectors that indicate the difference between the first-level vectors and their first-level representative vectors. Training unit 1104 then initializes...k A set of two second-level representative vectors is used. A second-level residual vector is assigned to its nearest second-level representative vector. The second-level representative vector is updated based on the average position of the assigned second-level residual vectors, and this assignment and update step is repeated until a termination condition is met. The training unit 1104 can repeat this process for one or more levels. Therefore, when the codebook lookup unit 1324 receives an aggregated sub-vector (i.e., a first-level vector), the codebook lookup unit 1324 determines the codebook index of the first-level representative vector of the first-level vector, determines the second-level residual vector indicating the difference between the first-level vector and the first-level representative vector, determines the codebook index of the second-level representative vector of the second-level residual vector, determines the third-level residual vector indicating the difference between the second-level residual vector and the second-level representative vector, determines the codebook index of the third-level representative vector of the third-level residual vector, and so on. In this way, for each aggregated sub-vector, the codebook lookup unit 1324 identifies the codebook index stack for that aggregated sub-vector.

[0138] In some examples, training unit 1104 applies the straight-through estimator (STE) technique to optimize vector quantization. During training iterations, training unit 1104 quantizes patch embeddings 1308 to generate quantized patch embeddings 1328. For each patch embedding 1308, training unit 1104 computes a commitment loss value representing the difference between the patch embedding and the quantized patch embedding. For example, training unit 1104 may compute the commitment loss value as:

[0139] Where z represents patch embedding. It is a quantized patch embedding. Dis the number of dimensions of the patch embeddings, and sg is the stopping gradient operation. After generating the reconstructed anatomy model 1346 for the training iterations, the training unit 1104 calculates the total loss as the sum of the reconstruction loss (as described above) and the commitment loss. The training unit 1104 updates the parameters of the feature encoder 1110 and the feature decoder 1114 based on the total loss using backpropagation. However, the vector quantization function applied by the quantization module 1112 cannot be learned directly using backpropagation because the vector quantization function is not differentiable. Therefore, during backpropagation, the vector quantization function is treated as an identity function, allowing modification of the parameters of the neurons in the neural network 1307 that generates the patch embeddings 1308. This changes the values ​​fed into the patch embeddings 1308 by the vector quantization function applied by the quantization module 1112. This ultimately has the effect of reducing the commitment loss value generated by the vector quantization function, because generating the patch embeddings 1308 makes the aggregated subvectors generated from the patch embeddings 1308 closer to the fixed representative vectors in the codebook 1118. STE technology can be used in conjunction with the hierarchical residual vector quantization process described above, but the representative vectors at each level are not updated.

[0140] In this way, the feature encoder 1110, the quantization module 1112, and the feature decoder 1114 can learn to generate a mesh model of the anatomical structure and feature vectors that provide a compact representation of the one or more anatomical structures.

[0141] As described above, training unit 1104 can compute a reconstruction loss value based on a comparison between the input anatomical model and the reconstructed anatomical model. Training unit 1104 can compute the reconstruction loss value in one of several ways. For example, training unit 1104 computes a mesh regularization loss value (e.g., chamfer loss, earth movement distance loss (also known as Wasserstein loss), edge regularization loss, normal consistency loss, or Laplacian loss). In some examples, training unit 1104 can compute a curvature-weighted chamfer loss, for example, as described by Bongratz et al. in “Vox2Cortex: Fast Explicit Reconstruction of Cortical Surfaces from 3D MRI Scans with Geometric Deep Neural Networks”, arxiv: 2203.09446v2 [cs.CV] March 18, 2022, where there are high curvature regions, such as certain areas of the scapula.

[0142] In some examples, training unit 1104 calculates multiple types of reconstruction loss values ​​and computes a weighted reconstruction loss value as a weighted sum of the reconstruction loss values. The loss can have different weights depending on the location of the patch or point. For example, training unit 1104 can use different weights to calculate reconstruction loss values ​​for different regions of the anatomical structure. In some examples, training unit 1104 calculates a single type of reconstruction loss value for the entire anatomical structure as the sum of the reconstruction losses at various locations on the initial anatomical model and the reconstructed anatomical model. In some such examples, training unit 1104 can use different weights for different regions. In this way, more attention can be paid to the accurate reconstruction of important regions. For example, a larger weight can be used for the scapular region near the glenoid fossa than for regions farther from the glenoid fossa.

[0143] As described above, training unit 1104 can train GPT 1134 after completing the training of feature encoder 1110 and feature decoder 1114 and learning codebook 1118. In other words, the parameters of feature encoder 1110, the parameters of feature decoder 1114, and the learned codebook 1118 can be frozen during the training of GPT 1134. Training unit 1104 can use self-attention to train GPT 1134 to autoregressively generate an ordered sequence of triangles that defines the final mesh. Given a target label sequence (i.e., the target sequence of codebook indices)... ,in and These are the corresponding predicted sequence elements. Training unit 1104 can use the following loss function to train GPT 1134:

[0144] in, N It is the number of tags in the target tag sequence. D is the number of subvectors embedded for each facet, |C| is the number of entries in codebook 1118, and It is a predicted sequence element This is equal to the probability of the corresponding label in the target label sequence. Therefore, after training the feature encoder 1110 and the learning codebook 1118, the training unit 1104 can use the feature encoder 1110 and the quantization module 1112 to generate quantized patch embeddings based on the training example 1132. As previously mentioned, the training example 1132 can include models of healthy or complete anatomical structures.

[0145] Training unit 1104 can use training example 1132 to train feature encoder 1110, quantization model 212, feature decoder 1114, and GPT 1134. Training example 1132 may include example anatomical models, such as anatomical models. Training example 1132 may include models of anatomical structures of real patients. For example, in a first training phase, training unit 1104 may apply feature encoder 1110 to the input anatomical model associated with the training example for each first training example in a first training example set to generate a patch embedding associated with the first training example. Quantization module 1112 may generate input labels associated with the first training example based on the patch embedding associated with the first training example. Training unit 1104 may apply feature decoder 1114 to the input labels associated with the first training example to generate a reconstructed anatomical model associated with the training example. Training unit 1104 may apply a first loss function to generate a loss value associated with the first training example based on the input anatomical model associated with the first training example and the reconstructed anatomical model associated with the first training example. Training unit 1104 may perform a first backpropagation process, which modifies the parameters of the feature encoder and feature decoder based on the loss value associated with the first training example. In a second training phase following the first training phase, for each second training example in the second training example set, training unit 1104 may apply feature encoder 1110 to generate a second patch embedding associated with the second training example. Training unit 1104 may apply quantization module 1112 to the patch embedding associated with the second training example to generate input labels based on the second patch embedding. Training unit 1104 may apply GPT 1134 to generate output labels associated with the second training example based on the input labels associated with the second training example. Training unit 1104 may apply feature decoder 1114 to the input labels associated with the second training example to generate a predictive anatomy model associated with that training example. Training unit 1104 may apply a second loss function to generate a loss value associated with the second training example. Training unit 1104 may perform a second backpropagation process, which modifies the parameters of the GPT based on the loss value associated with the second training example.

[0146] In some examples, feature augmentation unit 1106 performs data augmentation to improve the training of machine learning model 1130. In some examples, feature augmentation unit 1106 performs scaling, rotation, mirroring (e.g., generating a synthetic left scapula model from an actual right scapula model), unit sphere normalization, jittering, multi-resolution remeshing, or other techniques to generate synthetic training examples. Training unit 1104 can use the synthetic training examples as well as the original training example 1132 to train machine learning model 1130.

[0147] In some examples, training unit 1104 may perform a mesh simplification process on the anatomical model used to train example 1132. For example, training unit 1104 may use a quadratic error metric (QEM) mesh simplification algorithm to reduce the number of faces in the anatomical model. Reducing the number of faces can reduce training time. In some examples, training unit 1104 may reduce the number of faces in some regions of the anatomical model without reducing the number of faces in other regions of the anatomical model. This can reduce potential bottlenecks in processing circuitry and / or reduce the inference time of the mesh completion algorithm when processing meshes derived from high-resolution CT scans.

[0148] Figure 16 This is a conceptual diagram illustrating an example surgical planning user interface 1600 according to one or more techniques of this disclosure, which displays surgical recommendations for a reverse shoulder replacement surgery. A planning system 118 can generate the user interface 1600 for display (e.g., on a display 108). In some examples, the planning system 118 may generate the user interface 1600 after determining one or more recommended prostheses from a plurality of available prostheses.

[0149] exist Figure 16 In the example, user interface 1600 displays surgical recommendations 1602A and 1602B (collectively referred to as "surgical recommendations 1602") for a reverse shoulder replacement surgery. Each of the surgical recommendations 1602 can correspond to a different recommended prosthesis. Figure 16 In the example, each of surgical recommendations 1602 indicates the type of glenoid implant, the diameter of the glenoid implant, the diameter and type of the glenoid ball of the glenoid implant (e.g., centered, off-center, tilted, etc.), the neck-shaft angle of the corresponding humeral implant, the version of the glenoid implant, the percentage of placement of the glenoid implant, and the pin depth of the glenoid implant. The user can select one of surgical recommendations 602. Figure 16 In the example, the black background is used to indicate that surgical recommendation 602A is the selected surgical recommendation. In other examples, the user interface 1600 may involve other types of prostheses.

[0150] In addition, the user interface 1600 includes a top view 1606, a front view 1608, and a model 1610. The top view 1606 displays an X-ray image of the patient's shoulder taken from an overhead perspective (i.e., looking down from an overhead position). The top view 1606 displays the outline 1612 of a glenoid implant of the type indicated by the selected surgical recommendation at the location indicated by the selected surgical recommendation. The front view 1608 displays an X-ray image of the patient's shoulder taken from a frontal perspective (i.e., looking back from a frontal position). The front view 1608 displays the outline 1614 of a glenoid implant of the type indicated by the selected surgical recommendation at the location indicated by that surgical recommendation. The model 1610 displays a 3D model of the patient's scapula and a phantom image of the glenoid implant. The generation of views 1608 and model 1610 can be computationally complex. Therefore, generating views 1608 and model 1610 for all available prostheses would be undesirable and time-consuming. Conversely, planning system 118 can generate views 1608 and models 1610 only for one or more recommended spurious models. Determining one or more recommended spurious models as described in this disclosure avoids this problem, thereby improving the usability of planning system 118. Planning system 118 can generate views 1608 and models 1610 as needed for at least one of other available spurious models.

[0151] The following is a list of non-restrictive terms applicable to one or more technologies disclosed herein.

[0152] Clause 1A. A method comprising: for each of a plurality of nodes in a graph, generating a feature vector for the node by a computational system, the feature vector initially including feature data characterizing one or more aspects of a patient’s bone, wherein the graph includes one or more edges, each edge connecting a corresponding pair of nodes; applying a graph convolutional network (GCN) model to the graph and the feature vector of the nodes by the computational system to generate an output; and determining a recommended prosthesis for the patient by the computational system based on the output.

[0153] Clause 2A. The method according to Clause 1A, wherein: the plurality of nodes include cross-sectional nodes corresponding to different cross-sections in a plurality of cross-sections of the bone, and for each cross-sectional node in the figure, feature data initially included in the feature vector of the cross-sectional node characterizes one or more aspects of the bone at the cross-section corresponding to the cross-sectional node.

[0154] Clause 3A. The method according to Clause 2A, wherein, for each cross section of the bone, one or more aspects of the bone at the cross section of the bone include one or more measurements of the size of the bone at the cross section of the bone.

[0155] Clause 4A. The method according to Clause 3A, wherein the bone is the tibia, and one or more measurements include measurements of the dimensions of the bone at a cross-section of the bone, including the medial width of the tibia at the cross-section and the anterior length of the tibia at the cross-section.

[0156] Clause 5A. The method according to Clause 3A, wherein the bone is the talus, and one or more measurements include a measurement of the medial-lateral width of the talus.

[0157] Clause 6A. The method according to any one of Clauses 2A to 5A, wherein the cross section is perpendicular to the mechanical axis of the bone.

[0158] Clause 7A. A method according to any one of Clauses 1A to 6A, wherein: the GCN model is a first GCN model, the bone is a first bone, the feature data is first feature data, one or more aspects are one or more first aspects, the graph is a first graph, the plurality of nodes are first plurality of nodes, the recommended prosthesis is a first prosthesis, and the output is a first output, and the method further comprises: for each second node in a second plurality of nodes of a second graph, generating a feature vector of the node by a computational system, the feature vector initially including secondary feature data characterizing one or more second aspects of the second bone of the patient, wherein the second graph includes one or more second edges, each second edge connecting a corresponding pair of nodes in the second graph; applying the second GCN model to the feature vectors of the second graph and the second plurality of nodes by the computational system to generate an output; and determining a recommended second prosthesis for the patient by the computational system based on the second output.

[0159] Clause 8A. The method pursuant to Clause 7A further includes: selecting a second GCN model from a plurality of secondary GCN models by a computing system based on a first spur.

[0160] Clause 9A. The method according to any one of Clauses 7A to 8A, wherein: the second plurality of nodes includes cross-sectional nodes corresponding to different cross-sections in a plurality of cross-sections of the second bone, and for each cross-sectional node in the second figure, feature data initially included in the feature vector of the cross-sectional node characterizes one or more aspects of the second bone at the cross-section corresponding to the cross-sectional node.

[0161] Clause 10A. The method according to any one of Clauses 7A to 9A, wherein a first prosthesis is designed for implantation in a first bone and a second prosthesis is designed for implantation in a second bone.

[0162] Clause 11A. The method according to any one of Clauses 7A to 10A, wherein the first bone is the tibia and the second bone is the talus.

[0163] Clause 12A. The method according to any one of Clauses 1A to 10A, wherein applying the GCN model comprises: performing one or more message passing rounds by a computational system, wherein performing a message passing round comprises: for each corresponding node among a plurality of nodes, the computational system passing the feature vector of the corresponding node to each of a plurality of nodes connected to the corresponding node in the graph; and for each corresponding node among the plurality of nodes, the computational system modifying the feature vector of the corresponding node based on the feature vector of the corresponding node and the feature vector passed to the corresponding node; and after completing one or more message passing rounds, for each corresponding node among the plurality of nodes, the computational system applying a series of one or more graph convolutional layers (GCLs) of the GCN model to the feature vector of the corresponding node to obtain the embedding of the corresponding node; and generating an output by the computational system based on the node embedding.

[0164] Clause 13A. The method of Clause 12A, wherein generating output based on node embedding comprises: applying a pooling layer to the node embedding by a computational system, wherein the pooling layer generates a first intermediate vector including first intermediate features corresponding to spurs of different sizes; applying a fully connected layer to the first intermediate vector by a computational system to generate a second intermediate vector, wherein the second intermediate vector includes second intermediate features corresponding to spurs of different sizes; and applying a softmax layer to the second intermediate vector by a computational system to generate an output, wherein the output comprises a final vector including final features corresponding to spurs of different sizes.

[0165] Clause 14A. The method according to Clause 12A, wherein: nodes include cross-sectional nodes corresponding to different cross-sections among a plurality of cross-sections of the bone, and for each cross-sectional node in the graph: feature data initially included in the feature vector of the cross-sectional node characterizes one or more aspects of the bone at the cross-section corresponding to the cross-sectional node, and modifying the feature vector of the cross-sectional node includes: modifying the feature vector of the cross-sectional node by a computational system based on a weighted average of features in the feature vector of the cross-sectional node and features passed to the feature vector of the cross-sectional node, wherein the weights used in the weighted average are based on the importance of the cross-section corresponding to the cross-sectional node and the cross-sections corresponding to adjacent nodes connected to corresponding nodes in the graph data.

[0166] Clause 15A. The method according to any one of Clauses 1A to 14A, wherein: the graph is a 3D grid graph, for each corresponding sampling location in a 3D arrangement of sampling locations, a plurality of nodes include nodes corresponding to the corresponding sampling location, the 3D arrangement of sampling locations includes sampling locations corresponding to locations within the bone, and for each node, feature data initially included in the feature vector of the node characterizes one or more aspects of the bone at the sampling location corresponding to the node.

[0167] Clause 16A. The method according to Clause 15A, wherein, for one or more nodes, the feature data initially included in the feature vector of the node includes one or more of the following: bone density measurements at the sampling location corresponding to the node, data describing the local texture of the bone, or a gray-level co-occurrence matrix.

[0168] Clause 17A. The method according to any one of Clauses 15A to 16A, wherein: for each node, the plurality of edges includes a set of edges that connects the node to at most six other nodes among the plurality of nodes; for each node, the plurality of edges includes a set of edges that connects the node to at most eight other nodes among the plurality of nodes; or for each node, the plurality of edges includes a set of edges that connects the node to at most twelve other nodes among the plurality of nodes.

[0169] Clause 18A. The method according to any one of Clauses 15A to 17A, wherein the sampling location comprises a set of sampling locations corresponding to predefined coordinates within the 3D image data of the bone.

[0170] Clause 19A. The method according to any one of Clauses 15A to 18A further comprises: applying segmentation processing to image data of bone by a computing system to identify multiple regions, wherein the regions correspond to sampling positions among a plurality of sampling positions; and for at least one region, generating, by the computing system, a feature vector of a node corresponding to a sampling position of the region based on data associated with one or more pixels of image data within the region.

[0171] Clause 20A. The method according to Clause 19A, wherein one or more pixels of the image data comprise a plurality of pixels of the image data.

[0172] Clause 21A. The method according to Clause 20A, wherein the region is a 3-dimensional region, and each of the coordinates of two or more pixels among the plurality of pixels included in the region is different.

[0173] Clause 22A. A method according to any one of Clauses 1A to 12A or 14A to 21A, wherein: the output includes one or more sizes of a suitable prosthesis, and determining a recommended prosthesis for the patient includes: a calculation system determining a recommended prosthesis for the patient from a plurality of available prostheses based on one or more sizes of a suitable prosthesis.

[0174] Clause 23A. A method according to any one of Clauses 1A to 12A or 14A to 21A, wherein: the output includes one or more parameters of a suitable prosthesis, and determining a recommended prosthesis for the patient comprises: a calculation system determining a recommended prosthesis for the patient based on one or more parameters of a suitable prosthesis.

[0175] Clause 24A. A computing system comprising: a storage system; and one or more processors implemented in a circuit and configured to perform a method according to any one of Clauses 1 to 23.

[0176] Clause 25A. A computing system comprising means for performing a method according to any one of Clauses 1A to 23A.

[0177] Clause 26A. A non-transitory computer-readable storage medium having instructions stored thereon that, when executed, cause a computing system to perform a method according to any one of Clauses 1 to 23A.

[0178] Clause 1B. A method comprising: for each of a plurality of nodes in a graph, generating a feature vector of the node by one or more processors implemented in a circuit, the feature vector initially including feature data characterizing one or more aspects of a patient’s bone, wherein the graph includes one or more edges, each edge connecting a corresponding pair of nodes; applying a graph convolutional network (GCN) model to the graph and the feature vector of the nodes by one or more processors to generate an output; and determining, based on the output, recommended values ​​for one or more recommended prostheses or one or more prosthesis parameters for the patient by one or more processors.

[0179] Clause 2B. The method according to Clause 1B, wherein: the plurality of nodes include cross-sectional nodes corresponding to different cross-sections in a plurality of cross-sections of the bone, and for each cross-sectional node in the figure, feature data initially included in the feature vector of the cross-sectional node characterizes one or more aspects of the bone at the cross-section corresponding to the cross-sectional node.

[0180] Clause 3B. The method according to Clause 2B, wherein, for each cross section of the bone, one or more aspects of the bone at the cross section of the bone include one or more measurements of the size of the bone at the cross section of the bone.

[0181] Clause 4B. The method according to Clause 3B, wherein the bone is the tibia, and one or more measurements include measurements of the dimensions of the bone at a cross-section of the bone, including the inner width of the tibia at the cross-section and the anterior length of the tibia at the cross-section.

[0182] Clause 5B. The method according to Clause 3B, wherein the bone is the talus, and one or more measurements include a measurement of the medial-lateral width of the talus.

[0183] Clause 6B. The method according to any one of Clauses 2B to 5B, wherein the cross section is perpendicular to the mechanical axis of the bone.

[0184] Clause 7B. The method according to any one of Clauses 1B to 6B, wherein: the GCN model is a first GCN model, the bone is a first bone, the feature data is first feature data, one or more aspects are one or more first aspects, the graph is a first graph, the plurality of nodes are a first plurality of nodes, the prosthesis is a first prosthesis, and the output is a first output, and the method further comprises: for each node in a second plurality of nodes of a second graph, generating a feature vector of the node by a computational system, the feature vector initially including secondary feature data characterizing one or more second aspects of the second bone of the patient, wherein the second graph includes one or more second edges, each second edge connecting a corresponding pair of nodes in the second graph; applying the second GCN model to the feature vectors of the second graph and the second plurality of nodes by the computational system to generate a second output; and determining, based on the second output, recommended values ​​for a second prosthesis or one or more second prosthesis parameters for implantation in the patient, wherein the one or more second prosthesis parameters specify one or more aspects of the secondary prosthesis.

[0185] Clause 8B. The method pursuant to Clause 7B further includes: selecting a second GCN model from a plurality of secondary GCN models by a computing system based on a first spur.

[0186] Clause 9B. The method according to any one of Clauses 7B to 8B, wherein: the second plurality of nodes includes cross-sectional nodes corresponding to different cross-sections in a plurality of cross-sections of the second bone, and for each cross-sectional node in the second figure, feature data initially included in the feature vector of the cross-sectional node characterizes one or more aspects of the second bone at the cross-section corresponding to the cross-sectional node.

[0187] Clause 10B. The method according to any one of Clauses 7B to 9B, wherein a first prosthesis is designed for implantation in a first bone and a second prosthesis is designed for implantation in a second bone.

[0188] Clause 11B. The method according to any one of Clauses 7B to 10B, wherein the first bone is the tibia and the second bone is the talus.

[0189] Clause 12B. The method according to any one of Clauses 1B to 10B, wherein applying the GCN model comprises: performing one or more message passing rounds by a computational system, wherein performing a message passing round comprises: for each corresponding node among a plurality of nodes, the computational system passing the feature vector of the corresponding node to each of a plurality of nodes connected to the corresponding node in the graph; and for each corresponding node among the plurality of nodes, the computational system modifying the feature vector of the corresponding node based on the feature vector of the corresponding node and the feature vector passed to the corresponding node; and after completing one or more message passing rounds, for each corresponding node among the plurality of nodes, the computational system applying a series of one or more graph convolutional layers (GCLs) of the GCN model to the feature vector of the corresponding node to obtain the embedding of the corresponding node; and generating an output by the computational system based on the node embedding.

[0190] Clause 13B. The method of Clause 12A, wherein generating output based on node embedding comprises: applying a pooling layer to the node embedding by a computational system, wherein the pooling layer generates a first intermediate vector, the first intermediate vector including first intermediate features corresponding to spurs of different sizes; applying a fully connected layer to the first intermediate vector by a computational system to generate a second intermediate vector, wherein the second intermediate vector includes second intermediate features corresponding to spurs of different sizes; and applying a softmax layer to the second intermediate vector by a computational system to generate an output, wherein the output comprises a final vector including a final feature corresponding to one of the following: spurs of different sizes, a set of compatible spurs, or a combination of potential values ​​of one or more spur parameters.

[0191] Clause 14. The method according to Clause 12, wherein: nodes include cross-sectional nodes corresponding to different cross-sections among a plurality of cross-sections of the bone, and for each cross-sectional node in the graph: feature data initially included in the feature vector of the cross-sectional node characterizes one or more aspects of the bone at the cross-section corresponding to the cross-sectional node, and modifying the feature vector of the cross-sectional node includes: modifying the feature vector of the cross-sectional node by a computational system based on a weighted average of features in the feature vector of the cross-sectional node and features passed to the feature vector of the cross-sectional node, wherein the weights used in the weighted average are based on the importance of the cross-section corresponding to the cross-sectional node and the cross-sections corresponding to adjacent nodes connected to corresponding nodes in the graph data.

[0192] Clause 15. The method according to any one of Clauses 1 to 14, wherein: the graph is a 3D grid graph, for each corresponding sampling location in a 3D arrangement of sampling locations, a plurality of nodes include nodes corresponding to the corresponding sampling location, the 3D arrangement of sampling locations includes sampling locations corresponding to locations within the bone, and for each node, feature data initially included in the feature vector of the node characterizes one or more aspects of the bone at the sampling location corresponding to the node.

[0193] Clause 16. The method according to Clause 15, wherein, for one or more nodes, the feature data initially included in the feature vector of the node includes one or more of the following: bone density measurements at the sampling location corresponding to the node, data describing the local texture of the bone, or a gray-level co-occurrence matrix.

[0194] Clause 17. The method according to any one of Clauses 15 to 16, wherein: for each node, the plurality of edges includes a set of edges that connects the node to at most six other nodes among the plurality of nodes; for each node, the plurality of edges includes a set of edges that connects the node to at most eight other nodes among the plurality of nodes; or for each node, the plurality of edges includes a set of edges that connects the node to at most twelve other nodes among the plurality of nodes.

[0195] Clause 18B. The method according to any one of Clauses 15B to 17B, wherein the sampling locations comprise a set of sampling locations corresponding to predefined coordinates within the 3D image data of the bone.

[0196] Clause 19B. The method according to any one of Clauses 15B to 18B further includes: applying segmentation processing by a computing system to image data of bone to identify multiple regions, wherein the regions correspond to sampling positions among a plurality of sampling positions; and for at least one region, generating, by the computing system, a feature vector of a node corresponding to a sampling position of the region based on data associated with one or more pixels of image data within the region.

[0197] Clause 20B. The method according to Clause 19B, wherein one or more pixels of the image data comprise a plurality of pixels of the image data.

[0198] Clause 21B. The method according to Clause 20B, wherein the region is a 3-dimensional region, and each of the coordinates of two or more pixels among the plurality of pixels included in the region is different.

[0199] Clause 22B. The method according to any one of Clauses 1B to 12B or 14B to 21B, wherein: the output includes one or more sizes of a suitable prosthesis, and determining a recommended prosthesis for the patient includes: the calculation system determining a recommended prosthesis for the patient from a plurality of available prostheses based on one or more sizes of a suitable prosthesis.

[0200] Clause 23B. The method according to any one of Clauses 1B to 12B or 14B to 21B, wherein: the output includes one or more parameters of a suitable prosthesis, and determining the recommended prosthesis for the patient includes: the calculation system determining the recommended prosthesis for the patient based on one or more parameters of the suitable prosthesis.

[0201] Clause 24B. The method according to any one of Clauses 1B to 23B, wherein one or more prosthesis parameters include one or more of the following: the size of the glenoid ball of the glenoid prosthesis, the radius of the glenoid ball of the glenoid prosthesis, the base plate type of the glenoid prosthesis, the reinforcement type of the glenoid prosthesis, the glenoid eccentricity of the glenoid prosthesis, the stem size of the humeral prosthesis, or the head ball size of the humeral prosthesis.

[0202] Clause 25B. The method according to any one of Clauses 1B to 23B, wherein one or more prosthetic parameters include one or more of the following: the size of the tibial component, the size of the talar prosthesis, or the size of the articular movable component that can be attached to the tibial component.

[0203] Clause 26B. The method pursuant to any one of Clauses 1B to 24B also includes: surgical implantation of a recommended prosthesis.

[0204] Clause 27B. A computing system comprising: a storage system; and one or more processors implemented in a circuit and configured to perform a method according to any one of Clauses 1B to 26B.

[0205] Clause 28B. A computing system comprising means for performing a method according to any one of Clauses 1B to 26B.

[0206] Clause 29B. A non-transitory computer-readable storage medium having instructions stored thereon that, when executed, cause a computing system to perform a method according to any one of Clauses 1B to 26B.

[0207] Although these techniques are disclosed in combination with a limited number of examples, those skilled in the art will recognize the numerous modifications and variations that arise from this disclosure. For example, any reasonable combination of the described examples is contemplated. It is intended that the appended claims cover these modifications and variations, provided they fall within the true spirit and scope of the invention.

[0208] It should be recognized that, depending on the example, certain actions or events of any technique described herein may be performed in a different order, and may be added, combined, or ignored entirely (e.g., not all described actions or events are necessary for the practical technique). Furthermore, in some examples, actions or events may be performed in parallel, rather than sequentially, for example, through multithreaded processing, interrupt handling, or multiple processors.

[0209] In one or more examples, the described functionality may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functionality may be stored as one or more instructions or code on and / or transmitted via a computer-readable medium and executed by a hardware-based processing unit. A computer-readable medium may include a computer-readable storage medium corresponding to a tangible medium such as a data storage medium, or a communication medium, including any medium that facilitates the transfer of a computer program from one place to another (e.g., according to a communication protocol). In this way, a computer-readable medium may generally correspond to: (1) a non-transitory tangible computer-readable storage medium; or (2) a communication medium such as a signal or carrier wave. A data storage medium may be any available medium accessible by one or more computers or one or more processors to retrieve instructions, code, and / or data structures for implementing the techniques described in this disclosure. Computer program products may include computer-readable media.

[0210] By way of example, and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disc storage devices, disk storage devices, or other magnetic storage devices, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. Furthermore, any connection is properly referred to as a computer-readable medium. For example, coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium if instructions are transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave. However, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transient media, but rather refer to non-transient tangible storage media. As used herein, disks and optical discs include compact optical discs (CDs), laser discs, optical discs, digital versatile optical discs (DVDs), floppy disks, and Blu-ray discs, wherein disks typically reproduce data magnetically, while optical discs reproduce data optically using lasers. The combination of the above should also be included within the scope of computer-readable media.

[0211] The operations described in this disclosure can be performed by one or more processors, which can be implemented as fixed-function processing circuits, programmable circuits, or combinations thereof, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Fixed-function circuits refer to circuits that provide a specific function and whose operations are preset. Programmable circuits refer to circuits that can be programmed to perform various tasks and provide flexible functionality in the operations that can be performed. For example, a programmable circuit can execute instructions specified by software or firmware that cause the programmable circuit to operate in a manner defined by the software or firmware instructions. Fixed-function circuits can execute software instructions (e.g., receiving or outputting parameters), but the type of operations performed by a fixed-function circuit is generally immutable. Therefore, the terms "processor" and "processing circuit" as used herein can refer to any of the foregoing structures or any other structure suitable for implementing the techniques described herein.

Claims

1. A method comprising: For each of the multiple nodes in the graph, a feature vector of the node is generated by one or more processors implemented in circuitry. The feature vector initially includes feature data characterizing one or more aspects of the patient’s bone, wherein the graph includes one or more edges, each edge connecting a corresponding pair of nodes. The one or more processors apply a Graph Convolutional Network (GCN) model to the graph and the feature vectors of the nodes to generate an output; and The one or more processors determine, based on the output, one or more recommended prostheses or recommended values ​​for one or more prosthesis parameters for the patient.

2. The method according to claim 1, wherein: The plurality of nodes includes cross-sectional nodes corresponding to different cross-sections among the plurality of cross-sections of the bone, and For each cross-sectional node in the figure, the feature data initially included in the feature vector of the cross-sectional node characterizes one or more aspects of the bone at the cross-section corresponding to the cross-sectional node.

3. The method according to claim 2, wherein, For each cross section of the bone, the one or more aspects of the bone at the cross section of the bone include one or more measurements of the size of the bone at the cross section of the bone.

4. The method according to claim 3, wherein, The bone is the tibia, and the one or more measurements include measurements of the dimensions of the bone at the cross-section of the bone, including the inner width of the tibia at the cross-section and the anterior length of the tibia at the cross-section.

5. The method according to claim 3, wherein, The bone is the talus, and the one or more measurements include a measurement of the medial-lateral width of the talus.

6. The method according to any one of claims 2 to 5, wherein, The cross-section is perpendicular to the mechanical axis of the bone.

7. The method according to any one of claims 1 to 6, wherein: The GCN model is the first GCN model, the bone is the first bone, the feature data is the first feature data, the one or more aspects are one or more first aspects, the graph is the first graph, the multiple nodes are the first multiple nodes, the prosthesis is the first prosthesis, and the output is the first output, graph and graph. The method further includes: For each of the second plurality of nodes in the second graph, a feature vector is generated by the computing system. This feature vector initially includes secondary feature data characterizing one or more second aspects of the patient's second bone. The second graph includes one or more second edges, each connecting a corresponding pair of nodes in the second graph. The computing system applies the second GCN model to the feature vectors of the second graph and the second plurality of nodes to generate a second output; and The computing system determines recommended values ​​for a second prosthesis or one or more second prosthesis parameters to be implanted in the patient based on the second output, wherein the one or more second prosthesis parameters specify one or more aspects of the secondary prosthesis.

8. The method according to claim 7, further comprising: The computing system selects the second GCN model from multiple secondary GCN models based on the first spurious model.

9. The method according to any one of claims 7 to 8, wherein: The second plurality of nodes includes cross-sectional nodes corresponding to different cross-sections among the plurality of cross-sections of the second bone, and For each cross-sectional node in the second figure, the feature data initially included in the feature vector of the cross-sectional node characterizes one or more aspects of the second bone at the cross-section corresponding to the cross-sectional node.

10. The method according to any one of claims 7 to 9, wherein, The first prosthesis is designed to be implanted in the first bone, and the second prosthesis is designed to be implanted in the second bone.

11. The method according to any one of claims 7 to 10, wherein, The first bone is the tibia, and the second bone is the talus.

12. The method according to any one of claims 1 to 10, wherein, Applications of GCN models include: The computing system performs one or more rounds of message passing, wherein performing one round of message passing includes: For each corresponding node among the plurality of nodes, the computing system transmits the feature vector of the corresponding node to each node in the plurality of nodes connected to the corresponding node in the graph; and For each of the plurality of nodes, the computing system modifies the feature vector of the corresponding node based on the feature vector of the corresponding node and the feature vector passed to the corresponding node; and After completing one or more rounds of message passing, for each corresponding node among the plurality of nodes, the computing system applies a series of one or more graph convolutional layers (GCLs) of the GCN model to the feature vector of the corresponding node to obtain the embedding of the corresponding node; and The output is generated by the computing system based on the embedding of the node.

13. The method according to claim 12, wherein, Generating the output based on the embedding of the node includes: The computing system applies a pooling layer to the embedding of the node, wherein the pooling layer generates a first intermediate vector, the first intermediate vector including a first intermediate feature corresponding to a prosthesis with a different size; The computing system applies a fully connected layer to the first intermediate vector to generate a second intermediate vector, wherein the second intermediate vector includes second intermediate features corresponding to prostheses of different sizes; and The computational system applies a softmax layer to the second intermediate vector to generate the output, wherein the output includes a final vector comprising a final feature corresponding to one of the following: Prostheses of different sizes, A collection of compatible prostheses, and The combination of potential values ​​for one or more prosthesis parameters.

14. The method according to claim 12, wherein: The nodes include cross-sectional nodes corresponding to different cross-sections among a plurality of cross-sections of the bone, and For each cross-sectional node in the diagram: The feature data initially included in the feature vector of the cross-sectional node characterizes one or more aspects of the bone at the cross-section corresponding to the cross-sectional node, and Modifying the feature vector of the cross-sectional node includes: the computing system modifying the feature vector of the cross-sectional node based on a weighted average of the features in the feature vector of the cross-sectional node and the features passed to the feature vector of the cross-sectional node, wherein the weights used in the weighted average are based on the importance of the cross-section corresponding to the cross-sectional node and the importance of the cross-section corresponding to the adjacent node, which is connected to the corresponding node in the graph data.

15. The method according to any one of claims 1 to 14, wherein: The image is a 3D grid map. For each corresponding sampling position in the 3D arrangement of sampling positions, the plurality of nodes includes a node corresponding to the corresponding sampling position. The 3D arrangement of sampling positions includes sampling positions corresponding to the locations within the bone, and For each node, the feature data initially included in the feature vector of the node characterizes one or more aspects of the bone at the sampling location corresponding to the node.

16. The method according to claim 15, wherein, For one or more of the nodes, the feature data initially included in the feature vector of the node includes one or more of the following: Bone density measurement value at the sampling location corresponding to the node. Data describing the local texture of the bone, and Gray-level co-occurrence matrix.

17. The method according to any one of claims 15 to 16, wherein: For each node, the plurality of edges includes a set of edges connecting the node to at most six other nodes among the plurality of nodes. For each node, the plurality of edges includes a set of edges connecting the node to at most eight other nodes among the plurality of nodes, or For each node, the plurality of edges includes a set of edges connecting the node to a maximum of twelve other nodes among the plurality of nodes.

18. The method according to any one of claims 15 to 17, wherein, The sampling locations include a set of sampling locations corresponding to predefined coordinates within the 3D image data of the bone.

19. The method according to any one of claims 15 to 18, further comprising: A computing system applies segmentation processing to the image data of the bone to identify multiple regions, wherein the regions correspond to sampling positions among a plurality of sampling locations; and For at least one region, the computing system generates a feature vector corresponding to a node at a sampling location within the region, based on data associated with one or more pixels of the image data within the region.

20. The method according to claim 19, wherein, The one or more pixels of the image data include a plurality of pixels of the image data.

21. The method according to claim 20, wherein, The region is a 3D region, and the coordinates of each of two or more pixels among the plurality of pixels included in the region are different.

22. The method according to any one of claims 1 to 12 or 14 to 21, wherein: The output includes one or more sizes of a suitable prosthesis, and Determining the recommended prosthesis for the patient includes: a calculation system determining the recommended prosthesis for the patient from a plurality of available prostheses based on one or more dimensions of the suitable prosthesis.

23. The method according to any one of claims 1 to 12 or 14 to 21, wherein: The output includes one or more parameters of a suitable prosthesis, and Determining the recommended prosthesis for the patient includes: a calculation system determining the recommended prosthesis for the patient based on one or more parameters of the suitable prosthesis.

24. The method according to any one of claims 1 to 23, wherein, The one or more prosthesis parameters include one or more of the following: the size of the glenoid ball of the glenoid prosthesis, the radius of the glenoid ball of the glenoid prosthesis, the base plate type of the glenoid prosthesis, the reinforcement type of the glenoid prosthesis, the glenoid eccentricity of the glenoid prosthesis, the stem size of the humeral prosthesis, and the head ball size of the humeral prosthesis.

25. The method according to any one of claims 1 to 23, wherein, The one or more prosthesis parameters include one or more of the following: the size of the tibial component, the size of the talus prosthesis, and the size of the articular movement component that can be attached to the tibial component.

26. The method according to any one of claims 1 to 24, further comprising: The recommended prosthesis was surgically implanted.

27. A computing system, comprising: Storage system; as well as One or more processors, implemented in circuitry and configured to perform the method according to any one of claims 1 to 26.

28. A computing system comprising means for performing the method according to any one of claims 1 to 26.

29. A non-transitory computer-readable storage medium having instructions stored thereon, which, when executed, cause a computing system to perform the method according to any one of claims 1 to 26.

Images