Implant Identification

Abstract

Implant identification involves obtaining a machine learning (ML) model trained to select an anatomical implant model of a physical implant for replacement of a target anatomical structure based on characteristics of the anatomical surface of an anatomical structure adjacent to the target anatomical structure to be replaced, obtaining image data of a patient's anatomical region, where the anatomical region has the target anatomical structure and other anatomical structures adjacent to the target anatomical structure, determining characteristics of the anatomical surface of the other anatomical structures, where the anatomical surface exists at each of the interfaces between the other anatomical structures and the target anatomical structure, applying the ML model using the determined characteristics of the anatomical surface, and obtaining an implant model selected by the ML model as a designation of a physical implant for at least partial replacement of the target anatomical structure as a surgical implant candidate within the patient.

Description

Background Art

[0001] Among anatomical injuries, some are more difficult to heal than others. For example, for various bones, due to insufficient blood supply to promote rapid healing, patients may be unable to perform certain body functions if the healing period is prolonged. In some cases, especially for certain bones in the human body, an implant device that replaces the bone or a part thereof can address the injury. Due to the interaction of such bones with other anatomical structures, such as other bones in the case of joints, the accurate identification, selection, and creation of an appropriate implant for a specific patient are important.

Summary of the Invention

[0002] By providing a method implemented on a computer, the drawbacks of the prior art are overcome and further advantages are provided. The method involves obtaining a machine learning model trained to select an anatomical implant model of a physical implant for partial or total replacement of a target anatomical structure based on the characteristics of the anatomical surface of the anatomical structure adjacent to the target anatomical structure to be partially or totally replaced. The method obtains image data of a patient's anatomical region, where the anatomical region has the patient's target anatomical structure and the patient's other anatomical structures, and the other anatomical structures are adjacent to the patient's target anatomical structure. The method determines the characteristics of at least one anatomical surface of the other anatomical structures from the image data, and at least one anatomical surface exists at each of at least one interface between the patient's other anatomical structures and the target anatomical structure. The method utilizes the determined characteristics of the at least one anatomical surface, applies the machine learning model, and obtains the implant model selected based on the application. The selected implant model is selected by the machine learning model as a candidate for surgical implantation in the patient, at least partially, as a partial or total replacement of the patient's target anatomical structure, as a specification of a physical implant.

[0003] Furthermore, a computer system is provided that includes a memory and a processor that communicates with the memory, and the computer system is configured to execute a method. The method includes obtaining a machine learning model trained to select an anatomical implant model of a physical implant for partial or total replacement of a target anatomical structure based on characteristics of an anatomical surface of an anatomical structure adjacent to the target anatomical structure to be partially or totally replaced. The method includes obtaining image data of an anatomical region of a patient, the anatomical region having the patient's target anatomical structure and other anatomical structures of the patient, the other anatomical structures being adjacent to the patient's target anatomical structure. The method includes determining characteristics of at least one anatomical surface of the other anatomical structures from the image data, the at least one anatomical surface being present at each of at least one interface between the other anatomical structures of the patient and the target anatomical structure. The method includes utilizing the determined characteristics of the at least one anatomical surface, applying the machine learning model, and obtaining an implant model selected based on the application, the selected implant model being selected by the machine learning model as a designation of a physical implant for at least partial candidacy for surgical implantation within the patient as a partial or total replacement of the patient's target anatomical structure.

[0004] Furthermore, a computer program product is provided for performing a method, the computer program product including a computer-readable storage medium readable by a processing circuit and storing instructions for execution by the processing circuit. The method includes obtaining a machine learning model trained to select an anatomical implant model of a physical implant for partial or total replacement of a target anatomical structure based on characteristics of an anatomical surface of an anatomical structure adjacent to the target anatomical structure to be partially or totally replaced. The method includes obtaining image data of an anatomical region of a patient, the anatomical region having the patient's target anatomical structure and other anatomical structures of the patient, the other anatomical structures being adjacent to the patient's target anatomical structure. The method includes determining characteristics of at least one anatomical surface of the other anatomical structures from the image data, the at least one anatomical surface being present at each of at least one interface between the patient's other anatomical structures and the target anatomical structure. The method includes utilizing the determined characteristics of the at least one anatomical surface, applying the machine learning model, and obtaining an implant model selected based on the application, the selected implant model being selected by the machine learning model as a partial or total replacement of the patient's target anatomical structure and as a designation of a physical implant for at least partial candidacy for surgical implantation within the patient.

[0005] Additionally or alternatively, the method can include providing the selected implant model to a model candidate designation module for designating a model candidate for verification.

[0006] Additionally or alternatively, the method comprises presenting an implant model selected by a user on a graphical user interface and receiving an operation on the selected implant model, the operation modifying the selected implant model and generating an implant model candidate for verification, where the implant model candidate specifies a physical implant having one or more physical characteristics different from the physical implant specified by the selected implant model, receiving, and providing the implant model candidate for verification to a verification module, the verification determining whether the physical implant specified by the implant model candidate is suitable for surgical implantation within a patient.

[0007] Additionally or alternatively, the operation on the selected implant model can include at least one operation specified by the user and / or at least one operation automatically determined by artificial intelligence.

[0008] Additionally or alternatively, based on a verification determining that the physical implant specified by the implant model candidate is not suitable, the method can include repeating (i) receiving an operation and (ii) providing the implant model candidate for verification to the verification module one or more times, where in each iteration of the repetition, the implant model candidate that did not pass is provided as the selected implant model for the next iteration of the repetition.

[0009] Additionally or alternatively, the method can include receiving an operation on at least one anatomical surface characteristic of another anatomical structure, and the verification for determining whether the physical implant specified by the implant model candidate is suitable for surgical implantation within a patient is at least partially based on the operated characteristics of the at least one anatomical surface.

[0010] Additionally or alternatively, the manipulation of the characteristics of at least one anatomical surface can include at least one manipulation specified by the user and / or at least one manipulation automatically determined by artificial intelligence.

[0011] Additionally or alternatively, the method can include receiving a manipulation of the characteristics of at least one anatomical surface of another anatomical structure and providing the selected implant model to a verification module as an implant model candidate for verification, where the verification determines whether the physical implant specified by the implant model candidate is suitable for surgical implantation in the patient, and the verification is at least partially based on the manipulated characteristics of the at least one anatomical surface.

[0012] Additionally or alternatively, based on (i) a manipulation of the selected implant model that changes the selected implant model and generates an implant model candidate for verification, where different implant model candidates specify physical implants having one or more different physical characteristics compared to the physical implant specified by the selected implant model, and / or (ii) identifying the implant model candidate after receiving a manipulation of the characteristics of at least one anatomical surface of another anatomical structure, the method can include providing the implant model candidate to a verification module for verification, where the verification determines whether the physical implant specified by the implant model candidate is suitable for surgical implantation in the patient, and based on the verification determining that the physical implant specified by the implant model candidate is suitable for surgical implantation in the patient, the method can include presenting the implant model candidate in a training dataset as part of an example of training to associate the implant model candidate with at least one anatomical surface of another anatomical structure.

[0013] Additionally or alternatively, the method can include providing a selected implant model as an implant model candidate to a verification module for verification, the verification determining whether the physical implant specified by the implant model candidate is suitable for surgical implantation within a patient.

[0014] Additionally or alternatively, the implant model candidate may be an initial implant model candidate. Based on verification that determines that the physical implant specified by the initial implant model candidate fails to qualify, the method includes: (a) (i) performing an operation on the initial implant model candidate that modifies the initial implant model candidate to generate a different implant model candidate for verification, where the different implant model candidate specifies a physical implant having one or more different physical characteristics as compared to the physical implant specified by the initial implant model candidate, and / or (ii) receiving an operation on the characteristics of at least one anatomical surface of another anatomical structure; (b) determining the next implant model candidate to provide to a verification module for verification, where, based on having received an operation on the initial implant model candidate, the next implant model candidate is determined to be a different implant model generated from the operation on the initial implant model candidate, or, based on having received an operation on the characteristics of at least one anatomical surface of another anatomical structure and not having received an operation on the initial implant model candidate, the next implant model candidate is determined to be the initial implant model candidate; and (c) providing the next implant model candidate to the verification module for verification of whether the physical implant specified by the next implant model candidate qualifies for surgical implantation within a patient, where the verification for determining whether the physical implant specified by the next implant model candidate qualifies for surgical implantation within a patient is at least partially based on at least one selected from the group having the operation on the initial implant model candidate and the operation on the characteristics at at least one anatomical surface.

[0015] Additionally or alternatively, the method can include training a machine learning model to select an anatomical implant model, the training using samples from a library of implant models to train the machine learning model to select an anatomical implant model from the library of implant models, and the selected implant model being selected by the machine learning model from the library of implant models.

[0016] Additionally or alternatively, the machine learning model can include a trained generator of an adversarial generative network (GAN), the generator being trained using samples from a library of implant models to generate implant models, and the selected implant model being generated by the generator and having the implant model selected by the generator for output as the selected implant model.

[0017] Additionally or alternatively, the obtained image data can include three-dimensional digital model data representing an anatomical region of a patient, and determining characteristics of at least one anatomical surface of another anatomical structure can include processing the image data to present the at least one anatomical surface as at least one digital three-dimensional surface and converting the at least one digital three-dimensional surface to at least one two-dimensional projection, and the determined characteristics of the at least one anatomical surface being determined from the at least one two-dimensional projection.

[0018] Additionally or alternatively, the method can include preprocessing the image data to generate a three-dimensional digital model of the anatomical region of the patient with the target anatomical structure omitted therefrom.

[0019] Additionally or alternatively, one or more anatomical surfaces can include one or more articular surfaces where a physical implant will engage based on being at least partially surgically implanted within the patient.

[0020] Additionally or alternatively, the anatomical region can include the patient's ankle, the target anatomical structure has a talus, and at least one anatomical surface has at least one articular surface of at least one bone adjacent to the talus.

[0021] Additionally or alternatively, determining the characteristics of at least one anatomical surface can be based on (i) manual indication by the user of at least one anatomical surface provided based on user input to a graphical user interface that displays a model including at least other anatomical structures of the patient, and / or (ii) automatic analysis of image data to identify at least one anatomical surface.

[0022] Further features and advantages are realized by the concepts described herein.

[0023] The aspects described herein are particularly pointed out and distinctly claimed as examples in the claims at the end of this specification. The foregoing and other objects, features, and advantages of the present disclosure will be apparent from the following detailed description when taken in conjunction with the accompanying drawings.

Brief Description of the Drawings

[0024]

Figure 1

Figure 2A

Figure 2B

Figure 2C

Figure 3A

Figure 3B

Figure 4A

Figure 4B

Figure 5A

Figure 5B

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

DETAILED DESCRIPTION OF THE INVENTION

[0025] This specification describes methods for anatomical analysis and optimal anatomical structure implant hardware selection. Aspects can identify an optimal anatomical implant model based on an existing patient's anatomical structure. In certain embodiments, a machine learning (ML) model, such as a deep neural network, is trained to select at least one optimal or "best fit" anatomical structure implant model and / or its characteristics based on one or more surfaces of the surrounding patient's anatomical structure that are expected to engage or interact when the corresponding physical implant is provided within the patient. By way of example, an optimal talus implant model selection using artificial intelligence (AI) is provided, where software identifies one or more best fit models for a talus implant from a database of talus models. The identified implant model, if available, can inform the selection of a physical implementation of that model or optionally be used as the designation of the physical implant to be generated / manufactured. An "anatomical structure implant model" (also interchangeably referred to herein as an "implant model", "anatomical structure model" or "anatomical model") means a digital (i.e., data-formed) model of an anatomical structure. This digital model can function as the designation of a physical implant, which can be used for the manufacture of the physical implant and / or the selection of an existing, e.g., commercially available, physical implant (if such an implant has already been manufactured).

[0026] Accordingly, in some embodiments, a best-fit talus model identified and selected from a database or library of models can be presented for use in selecting or manufacturing a hardware implant having desired characteristics. For example, an implant model of an anatomical structure verified for a particular patient can function as a basis for identifying existing implants or as the specification of an implant to be generated (e.g., manufactured). In this latter regard, an identified anatomical structure implant model can be loaded from a file such as an.stl file, optionally manipulated, verified, and provided to a manufacturing apparatus such as a three-dimensional (3D) printer.

[0027] The search for a best-fit implant model may be based on the anatomical surface of the anatomical structure adjacent to the target anatomical structure for replacement. For example, the anatomical surface may be the articular surface of another anatomical structure with which the target anatomical structure for replacement engages.

[0028] In this specification, various aspects are discussed and presented with reference to the talus (the "ankle"). The talus is one of a group of bones in the foot called the tarsal bones. The tarsal bones form the lower part of the ankle joint through the joint. The talus can transmit the entire body weight, or a significant portion of the body weight, to the foot. The talus is widely covered with cartilage and forms the talocalcaneonavicular joint together with the calcaneus (the "heel bone") and the navicular bone. The talus can include three basic parts: (i) the head, (ii) the neck, and (iii) the body part. Examples of such articular surfaces in talus replacement include the calcaneal articular surface, the navicular articular surface, the tibial articular surface, and / or the fibular articular surface.

[0029] As described above, there are cases where various injuries are difficult to heal. For example, since the talus does not receive sufficient blood supply, when the talus is fractured, it may be impossible to walk for several months without crutches. One approach is to treat the injury with a talus implant. To insert a talus implant as a substitute for the patient's natural talus, first, the ankle joint is incised and exposed, and then the obstructive extensor digitorum longus tendon, anterior tibial artery, and extensor digitorum longus are moved or removed. The talus may be fixed not only by the anterior talofibular ligament and superficial deltoid ligament but also by the syndesmosis and anterior tibiofibular ligament. The exposed and damaged talus is removed from the ankle joint, and a talus implant can be inserted and replaced in the space that was previously occupied by the patient's natural talus. However, since the talus interacts with other bones of the ankle, an accurate talus replacement and implantation method / system are desired for the accurate selection, optional fabrication, and implantation of the replacement talus in the patient. In existing talus replacement surgeries, a ground-up approach of designing a custom-made talus for each patient is used. This is time-consuming and costly. According to some aspects described herein, machine learning is utilized to assist in the selection and generation of the best-fit anatomical implant.

[0030] The aspects described herein are presented with reference to the human talus, but this is for illustrative purposes only. Those skilled in the art will recognize that the aspects presented herein can be used to inform the selection / generation of implants for use with other anatomical features within the human body and / or other anatomical structures, and further, that the aspects described herein are applicable to both total and partial replacement of anatomical structures.

[0031] Accordingly, by way of illustration and not limitation, some aspects described herein utilize machine learning to support the selection and / or generation of a best-fit implant, such as a talus implant. Artificial intelligence, for example in the form of one or more trained ML models, may be utilized to cognitively analyze the specific anatomical parameters (e.g., joint surface) of a particular patient and select from this analysis a best-fit anatomical structure implant model, such as a talus design. In some examples, the selection is made from a library / database of existing designs, such as a digital three-dimensional (3D) model of the talus (or other form of specification). In some examples, the analysis of the anatomical parameters and the selection of candidate best-fit models may inform operations on models that result in new models for providing to the database. The ML models employed herein are capable of self-learning and can be improved by use. Further, the parameters used to select an anatomical model, such as a talus model for an implant, can be adjusted by additional processing to select and generate a best-fit solution tailored to an individual patient.

[0032] Program code can determine the characteristics of an anatomical (e.g., joint) surface, such as dimensions, distances, profiles, and other characteristics, from an image of the anatomical structure of a given patient and utilize these characteristics to select a best-fit anatomical structure implant model from a library of options and can use machine learning. The ML models utilized by the aspects discussed herein may be of various types, such as neural networks including, for example, recurrent neural networks and / or convolutional neural networks. If specific data that would assist the program code in selecting a best-fit existing model is not available, the aspect can generate a simulation of the missing data and utilize the simulation to select (and optionally manufacture) the implant. For example, if a portion of the talus anatomical structure is missing, that portion can be simulated for selection / manufacturing purposes.

[0033] In an exemplary embodiment, program code executed on one or more processors processes and analyzes data from an image, such as a digital imaging and communications in medicine (DICOM) image of a patient's anatomical structure, generates a digital model of the anatomical structure, and automatically determines / selects an implant model from a library of implant models based on the digital model of the anatomical structure. This selected model may perhaps have operations applied to it and serve as a candidate to replace the patient's anatomical structure. The candidate can be the subject of verification for a particular patient. If verified, the corresponding physical implant is selected and can be obtained, for example, if it is available as an existing "off-the-shelf" product. Alternatively, the physical implant can be manufactured using a manufacturing technique such as 3D printing, for example, according to the specifications of the anatomical structure implant model.

[0034] Thus, in some embodiments, the software implementing the AI can provide recommendations regarding suitable anatomical models for the replacement of a patient's anatomical structure and thus help identify an implant solution for selection or any manufacturing, based on the anatomical structure of a particular patient. Current approaches for generating implants are labor-intensive, potentially inaccurate, and customized, which can result in a large amount of work and an increase in the cost of the overall process. In contrast, the aspects discussed herein automate part of the implant design / selection process based on the training and application of machine learning models and identify the best existing designs for selecting / generating patient-specific implants, allowing for the fitting / design of a custom implant prototype for a particular set of conditions.

[0035] FIG. 1 shows an exemplary environment for using and incorporating the aspects described herein. The environment shows an exemplary technical architecture and data flow that illustrate certain general aspects described in more detail herein. The environment includes a controller 110, such as a computer system, having a processing circuit 120 that includes one or more processors (by way of example) and a memory 130. The memory 130 can include program code / instructions executable by the processing circuit to cause the controller 110 to perform functions such as the processes described herein. The program code of the memory 130 can include various sets of code / instructions configured to perform particular activities. Examples of such sets are shown in FIG. 1 as modules or components. These are shown as separate elements within the memory 130 for purposes of conceptual illustration, but in other examples, these modules can be further separated and / or combined. The specific delineations between the modules shown in FIG. 1 are for purposes of illustration.

[0036] A data processing module 140 obtains image data 102. In other examples, it is obtained from a data store that stores the image data, but in the example, the image data is obtained from an imaging device such as a medical imaging device. The image data can include images of anatomical regions and structures in any of various digital formats. One such format is based on the Digital Imaging and Communications in Medicine (DICOM) standard. Exemplary image data is scan data (e.g., a set of images) generated from computed tomography (CT) or other imaging diagnostic procedures.

[0037] The data processing module 140 processes the obtained image data 102 to perform desired standardization, cleaning, or initial processing. For example, a CT scan generates a stack of two-dimensional (2D) images that divide an anatomical structure into very thin "slices". For example, a CT scan of a foot yields 600 images. The program code of the data processing module 140 can segment the image data, which in this example refers to converting the 2D images into a 3D model. Thus, the data processing module 140 can create a 3D model from the input image data 102.

[0038] The anatomical regions presented by the image data can include the patient's anatomical structure (referred to herein as the "target anatomical structure") that is an anatomical structure that may be replaced by an implant, and other patient anatomical structures that are at least partially adjacent to the target anatomical structure.

[0039] By way of specific example, FIGS. 2A-2C show examples of computer tomography (CT) images that have been loaded and segmented in accordance with the aspects described herein. FIG. 2A shows a coronal plane image of a patient's ankle, FIG. 2B shows an axial plane image of a patient's ankle, and FIG. 2C shows a sagittal plane image of a patient's ankle. The image data has been processed into segments of bone that are different from the patient's anatomical structure. The images shown present different bones.

[0040] Considering that the program code is ultimately used to select / specify an implant suitable for the target anatomical structure (the talus in these examples), the image data can be processed by the data processing device 140 and / or the data analyzer 142 to remove the image data of the specific target anatomical structure that the implant will replace. The resulting model can be presented to the user for viewing. For example, FIGS. 3A-3B show examples of views of the anterolateral (FIG. 3A) and posteromedial (FIG. 3B) of a digital model of a patient's ankle region with the talus omitted. Specifically, the tibia 302, fibula 304, calcaneus 306, and metatarsal / tarsal bones 308 are depicted surrounding a space 310 where the talus would fit but is programmatically removed.

[0041] The processed data from the data processing device 140 is provided to the data analyzer 142 for its analysis. The data analyzer 142 is used to identify the characteristics of the anatomical surface adjacent to the target anatomical structure. In the example, the anatomical surface is the articular surface, i.e., the surface of the anatomical structure with which (in this example) the patient's talus engages, and thus the surface where a suitable talus implant is expected to contact when provided within the patient. Thus, as part of the data analysis by the analyzer 142, the patient's articular surface surrounding the anatomical structure is identified. In some embodiments, the articular surface is identified / designated in whole or in part manually by a user (such as a physician, an engineer specializing in virtual surgical planning, etc.) using a computer system, a display, and an input device to identify the boundaries and other characteristics of these articular surfaces. Additionally or alternatively, AI based on machine learning can perform the identification by a model trained to take the image data as input and identify the articular surfaces of the adjacent anatomical structures presented in the image data. For example, the program code of the data analyzer 142 can apply a machine learning algorithm utilizing a neural network to detect and determine the characteristics of the articular surfaces presented in the image. In the context of the embodiments presented herein, the articular surface is the other bone surface of at least some of the bones that the talus contacts, such as the surfaces of the navicular bone, calcaneus, fibula, and / or tibia.

[0042] Figures 4A - 4B show examples of the patient bone models of Figures 3A - 3B with the aforementioned three joint surfaces highlighted. The models with the highlighted surfaces can be rendered for user viewing. In Figures 4A - 4B, the fibula 404, the calcaneus 406, and the navicular bone 412 are shown in their relative positions with a space 410 for the talus implant. 414a shows the calcaneal joint surface between the calcaneus and the talus, i.e., the portion of the surface of the calcaneus where the talus physically contacts / engages to form a joint. 414b shows the navicular joint surface between the navicular bone and the talus, i.e., the portion of the surface of the navicular bone where the talus physically contacts / engages. 414c shows the fibular joint surface between the fibula and the talus, i.e., the portion of the bottom surface of the fibula where the talus physically contacts / articulates. The software can determine and / or grasp various characteristics (such as size, shape, perimeter, area, surface contour, etc.) of these surfaces based on user input and utilize them in the model selection / generation process as described.

[0043] Figures 5A - 5B show the characteristics (dimensions, perimeters, and positions relative to each other, etc.) of the joint surfaces of Figures 4A - 4B where the surfaces 414a, 414b, 414c are separated from the anatomical structures of other patients, according to the aspects described herein. The aspects can display this figure on a display for the user to view if necessary.

[0044] In some situations, there may be no anatomical structure that identifies the joint surface. In such cases, the program code can infer the missing part of the anatomical structure from the available image data 102. For example, when the program code determines an implant to be used on the left side of a patient and the desired joint surface is not present in the image data 102, the program code may infer the missing surface based on the available data.

[0045] Returning to FIG. 1, once the articular surface has been identified by the data analyzer 142, the processing of the controller can proceed to the model candidate specifying module 144. In some aspects, the model candidate selection module 144 can use the articular surface identified from 142 to inform limits (such as distances, perimeters, and other parameters) for selecting implant model candidates that fill the implant space. The module 144 can utilize the model selection module 146 to select / specify the best-fit anatomical structure implant model from the model library 148, which is a library / database of anatomical implant models. The models in the model library 148 may include models of actual human talus bones and / or models that are at least partially specified / manipulated by the user, as described below. In the example, the model selection module 146 uses the ML model described herein to find the closest point in the n-dimensional latent space to select the best-fit implant model present in the library 148.

[0046] Optionally, as described herein, operations can be applied to the selected implant model, and such operations are facilitated by the model manipulation module 152.

[0047] In any case, the model candidate designating module is used to designate implant model candidates to be verified. In the verification, it is determined whether the candidate is an appropriate designation for use in a specific patient for whom the image data 102 was taken. As an example, the user selects the best-fit model identified by 146 as a candidate, and this selected candidate is provided for verification. Optionally, the selected best-fit model returned from 146 is presented to an interface for the user to view, and the user can select such a best-fit model as the initially selected implant model. Alternatively, the best-fit model is automatically selected (e.g., by 146 or 144) and optionally confirmed by the user. Feedback in the form of the user's selection or confirmation regarding the best-fit design for a given input can, if desired, be used by program code to update the machine learning algorithms utilized to select the best-fit design from the model library.

[0048] Once the model candidate is designated, the model candidate is provided to the model verification module 150. The verification module 150 receives the model candidate and attempts verification. The verification may be manual, automatic, or a combination of the two. In a particular example, the user performs at least part of the verification by digitally identifying, e.g., using a controller, whether the implant fit is appropriate or whether some adjustment is needed. Additionally or alternatively, in some embodiments, the verification is at least partially automated (e.g., using a machine learning model) to verify whether the selected best-fit implant model (to which operations may also be applied) is an acceptable fit. The verification can confirm whether the physical implant designated by the implant model candidate is suitable for surgical implantation within the patient. The verification may include one or more desired verification processes, tests, checks, etc. performed on the selected implant model or collection of selection candidates.

[0049] In one example, the verification includes an evaluation (fully manual, fully automated, or a combination thereof) of the distances between a selected implant and the anatomical structures of various patients when the implant is in the surgical position. For example, based on surface mapping, it may be desirable to examine the average size of the gap between the surface of the implant and the corresponding articular surface of the patient's anatomical structure that forms an interface with the surface of the implant. In the example, the desired gap size may be 0 to 3 millimeters, more specifically 1 to 2 millimeters. Generally, spatial and / or volumetric analysis may be useful in determining the fit of the implant with the surrounding anatomical structures. As another example, the verification may simulate and evaluate how the fitness is informed by the patient's movement using an operating simulation in which the implant model is placed within the patient. For example, a walking cycle or other operating simulation can simulate the movement, shifting, etc. of the implant based on the simulated user movement. Thereby, the appropriateness of the selected implant for a specific patient can be known.

[0050] In particular, before making a replacement plan, if the patient has contracted an infectious disease or had the anatomical structure removed due to other events, changes in the surrounding anatomical structures (such as soft tissues) that may change the characteristics of the space for implanting the implant may be considered before considering the selection of the implant. For example, if soft tissue begins to enter the void left by the removed anatomical structure, joint distraction may be desirable, although a talus implant sized to fit the patient's natural anatomical structure may not be as appropriate. The talus implant may be selected to be smaller than the original talus and larger than the space it currently occupies to achieve the desired level of joint distraction. Verification in this context can confirm whether the implant has a chance of achieving the desired amount of distraction.

[0051] Verification may incorporate the opinions of physicians and other medical experts. For example, verification may be based on responses from a collection after presenting a selected implant model in combination with a particular patient's anatomical structure / anatomical structures. In one embodiment, the physician's collection may vote to indicate a proper fit of the implant and anatomical structure pair. In a specific example, a collection of implant models is presented (e.g., in the form of a grid) as candidates for one or more corresponding patient anatomical structures, and the physician is prompted to indicate which shows a proper fit. For proper fit of the implant, responses can be collected in the form of crowdsourcing input. The proposed combination of implant and anatomical structure may be presented in any desired way, including cross-sectional views or other static diagrams, and / or as the above-described motion simulations. In some examples, the responses are used to train a machine learning model to verify a selected implant model against an anatomical structure. Additionally or alternatively, the responses can inform statistical "rules" (tolerances, averages, etc.) useful for implant verification.

[0052] When a model candidate is verified, a physical implant model selector / generator 154 is used to select an existing physical implant (e.g., a commercially available product) if its manufacturer / model can be identified from the specification, or to start the generation / manufacture of the model by passing the implant specification to the implant creation device 160 for manufacturing the implant. This specification can instruct the creation of the implant. For example, in some embodiments, a digital form specification can be provided to the implant creation device 160, which may be, by way of example, a 3D printer or other additive manufacturing device or its support control device. The implant creation device 160 is controlled by program code of a controller and / or other computer system and can automatically create the implant.

[0053] If the candidate is not verified, the process can return to the model candidate designation module 144 via the operation module 152. The operation module 152 enables any operation on the model as part of the designation of the model candidate. For example, the user can arbitrarily operate on the model candidate, such as a model selected from a model library, and / or a model candidate that has failed verification. The operation may be performed when the user and / or the process identifies that a particular model candidate is not sufficient for a particular patient. As an example, the user or the process may observe that the model candidate is unacceptably small (e.g., exceeding one or more thresholds) for the space it occupies, or that the area of the implant that contacts the articular surface of the surrounding anatomical structure is not satisfactorily aligned with those surfaces. Additionally or alternatively, the user may want to promote an anatomical change such as joint separation by the selection of the implant. Thus, the operation module can be used for the user to manually adjust the size, shape, and other characteristics of the implant model in order to better fit the model to the patient. Optionally, such adjustment may be performed automatically in some embodiments.

[0054] Instead of or in addition to operating on the implant model, the manipulation module 152 may manipulate the size, position, distances between features, and / or other characteristics of the patient's anatomical structure reflected by the input data 102, thereby being used to modify the patient's anatomical structure on which model selection and / or verification is based. This may include changes to the characteristics (size, position, distances between features, etc.) of the joint surface identified from 142. By changing the patient's anatomical structure, whether it is an implant model candidate presented to the verification module 150 and fails verification in the verification module 150, or an implant model candidate identified after updating and searching the model library 148 via the selection module 146, there may be a better implant model candidate identified. Further, an implant candidate that initially fails verification may pass without additional operations on the implant model itself if appropriate manipulation of the patient's anatomical structure is applied. Since the patient's anatomical structure may naturally adapt to the implant, in appropriate situations, partial manipulation of the patient's anatomical structure may be acceptable.

[0055] In some examples, the library model selection module 146 selects a plurality of implant models from the library that are the "best-fitting" models for a given patient's anatomy. The selection module 146 may rank these plurality of best-fit models by confidence, or an indication of how well the ML model used by the selection module 146 fits them. The "best" model of such ranking can be selected automatically or manually by the user as the initially selected implant model. This initially selected model can be sent as is to the verification module 150 for verification, and in effect serves as an initial implant model candidate to attempt verification of the selected implant model. If this candidate is rejected, the process may return to the model candidate designation module 144 to select the next best-fit model in the ranking as the next candidate and / or implant model candidate, operation on the patient's anatomy, or both. The automated or user operation via 152 may or may not be performed at each iteration, and either or both may be optional. A specific example of the process is to return to 144 without modifying the candidate and select the next ranked model among the plurality of models selected by the library model selection module 146 as the next candidate to try. However, as described above, the user may apply an operation to the selected / implant model candidate at any iteration or when the next model is selected. The user can have criteria for understanding whether operating on the anatomy or the model candidate that failed verification may have a chance of succeeding in verification, or instead whether it would be better to select another model from the library as the next model. AI may also be used in this decision-making, for example, to identify situations where working with the current model and applying an operation to the current model and / or the anatomy may be superior to switching to the next selected model selected by 146 from the library 148, for example.

[0056] The model candidate specifying module 144 can be seen to be used to specify model candidates for verification. The candidates for verification may remain provided in the library 148 and / or, after being automatically or manually operated, may be sent for verification. If a model candidate that has passed verification (i) does not exist in the library 148 or (ii) exists in the library but has not been previously verified for the anatomical structure corresponding to the anatomical structure of the patient for which verification was performed, this can be useful training feedback for the system. For example, a specified verified implant model that is not yet in the library 148 can be added to the library (151). The operated model may not represent the corresponding natural anatomical structure from the patient, but nevertheless, since it has been found to be valid and verified for a particular patient's anatomical structure, it can be added to the library 148 as a legitimate implant model. Next, or if the implant model was previously known in the library but was not related to the anatomical structure of that patient or the anatomical structure to which the operation applied, this can be provided as additional training data for training the ML model used by the library model selection module 146 to identify the best fit implant model for a particular patient's anatomical structure input (e.g., characteristics of the joint surface).

[0057] In some examples, the first model candidate is selected as the best fit from a library, and the user or system may manipulate the model as an initial adjustment before attempting verification. In either case, the process according to FIG. 1 can iterate candidates and / or manipulations until a validated model is identified, after which a physical implant is obtained / manufactured and used. In this way, the process can iterate until an appropriate and effective implant model candidate is identified. This process may include manipulation of the patient's anatomical structure reflected by the input data and / or an implant model placed for verification. Manipulation of the implant model may result in the definition of a new implant model not in the library. In this case, assuming the new model is validated against the anatomical structure of a patient with the new model, the new model can be saved in the library for later selection. On the other hand, it can also be provided as an example of further training along with the characteristics of the corresponding articular surface for which the model has been validated.

[0058] In an example, a graphical user interface (GUI) is provided, and the user calls the processes described herein, including selection and specification of an implant model based on one or more best-fit options selected by program code (e.g., library model selection module 146) applying one or more trained machine learning algorithms. The GUI may enable the user to upload digital files that make up the image data (e.g., FIG. 1, 102) and specify particular search criteria and / or search methods to use. The GUI can call a process as described herein to select a best-fit model and present it to the user. Based on these search results, the user can select a predetermined result as the selected implant model to provide to the model candidate designation module 144, and / or the next selected implant model to provide to the model candidate designation module 144 may be automatically selected. The user may have the option to perform operations on the model and / or the patient's anatomical structure (including, e.g., the articular surface).

[0059] The program code can search based on one or more of the articular surfaces, display the best-fit model itself, from which the user can select the next implant model to provide as a candidate to 144. In other examples, the program code obtains image data and automatically searches / selects a model.

[0060] FIG. 6 shows an exemplary interface presented by such a GUI for display and selection of a best-fit implant model, in accordance with the aspects described herein. Element 602 presents a 2D rendering of the articular surface of the scaphoid of the patient's existing anatomical structure, i.e., the portion of the surface of the patient's scaphoid that is expected to physically engage with the talus implant. The characteristics of this surface may serve as an input to the ML model of the library model selection module 146 and may also be utilized by the library model selection module 146. The library model selection module 146 determines one or more (five in this example) of the top best-fit models, indicated by 604, which present 2D renderings of the scaphoid engagement surfaces of the top five best-fit models.

[0061] A 2D rendering of the surface is shown in FIG. 6, which is an example of a way to display the input and the selection result of the library model. In this regard, when using 3D model data, neural networks and other ML models for AI may not perform optimally. In such cases, the 3D curved surface representing the articular surface used as the input to the model can be provided as a 2D projection. 602 in FIG. 6 is a 2D projection view of the 3D scaphoid articular surface of a patient. Different colors, shades, numerical values, or other data can be used to represent a third "depth" dimension where each point corresponds to the distance between the point on the articular surface and the plane behind the surface. In the examples of 602 and 604 in FIG. 6, the points "closest" to the 2D plane are shown with the darkest shade, and the points farthest from the 2D plane are shown with the lightest shade. By representing the 3D articular surface as 2D data, AI can handle the characteristics of the articular surface more efficiently. This 2D rendering may be optional. This is because existing or future-developed other AI may be able to handle 3D data as it is.

[0062] As described above, 604 presents the top five best-fit models represented by the surface that engages with the patient's scaphoid. The user can select one of these results, and the GUI can display the selected implant along with the surrounding anatomical structure. 606 in FIG. 6 shows a front view and a posterior-medial view of the patient's ankle incorporating the selected implant model 608.

[0063] Therefore, the program code of FIG. 1 can identify the articular surface of the patient's anatomical structure that engages with the implant. The library model selection module 146 can determine the best-fit model of the implant selected from the library based on the characteristics of these articular surfaces. To select the best-fit model (e.g., the one expected to most appropriately engage with the identified articular surface of the patient's anatomical structure), the library model selection module 146 can utilize a classifier including an artificial neural network (ANN), a convolutional neural network (CNN) (such as Mask-RCNN), an autoencoder neural network (AE), a deep convolutional network (DCN), and / or other image classifiers and / or segmentation models, and combinations thereof. In an example of using a CNN, the CNN can be configured using an AI instruction set (e.g., a native AI instruction set or an appropriate AI instruction set). In a specific example herein, the classifier utilized is a deep learning model. The connections of the nodes of the deep learning model can be trained and retrained without redesigning their number, arrangement, interface with the image input, etc. In some examples, these nodes collectively form a neural network. In a particular embodiment, the nodes of the classifier may not have a layer structure. To configure the neural network, the program code can, by way of example, connect the layers of the network, define skip connections of the network layers, set the coefficients (e.g., convolutional coefficients) to learned values, set the filter length, and determine the patch size (e.g., an area of n×n pixels of the image, where n is 3, 4, etc.).

[0064] In some embodiments, the program code utilizes a neural network to select a model from a model library in real time so that a user providing input to the GUI can obtain results almost immediately. In some examples, the program code includes a pre-trained CNN configured to classify aspects of an image obtained by the program code. In some examples, the CNN may be MobileNet or MobileNet2 and may be available in Keras, TensorFlow, and / or other types of libraries. The classifier of the model selection module can be generated in embodiments by the program code from a pre-trained CNN that, for example, deletes, modifies, and / or replaces layers of neurons (e.g., input layer, output layer).

[0065] In accordance with the aspects described herein, program code, such as the program code of the library model selection module 146, can implement an ML model trained to learn the main features of an anatomical structure centered on an articular surface and identify one or more best-fit designs from a model library. In some examples, the program code includes a deep neural network trained to learn the features of the designs of a model library 148 that includes articular surfaces that correspondingly form pairs in order to appropriately classify the input.

[0066] FIG. 7 shows a conceptual diagram of the training and use of a machine learning model according to the aspects described herein. A training dataset 702 is used for training an ML model 704. The training dataset 702 can include a model of an exemplary adjacent anatomical structure (e.g., an articular surface) and / or its characteristics, along with a correlated anatomical implant model that "fits" its adjacent anatomical structure. Thus, using the examples described herein, the training dataset can include a model of a talus (which may be a model of an actual human talus) and a model of the articular surface (e.g., 3D and / or 2D) paired with those taluses. The characteristics of the articular surface and the talus paired with the articular surface can be determined, for example, by preprocessing image data of an actual patient.

[0067] The ML model 704 can be any type of ML model, a specific example of which is an autoencoder, which is a type of neural network. By training the ML model 704, the ML model 704 learns to be able to identify a talus model that fits a given input articular surface. Such a talus model can correspond to an existing implant or an implant planned for manufacture. The ML model 704 can be trained with any amount of training data, but generally, the larger the number of training examples, the higher the accuracy compared to when the number of training examples is small. In a specific example, the ML model 704 may be trained based on approximately 200 training cases (pairs of articular surface - bone models) to obtain acceptable results.

[0068] Generally, the training causes the ML model 704 to learn to identify the most relevant patterns in the data such that when the characteristics of the articular surface adjacent to a given target anatomical structure for replacement are input into the model, the model returns an appropriate implant model for anatomical replacement. The dimensionality reduction into the "low-dimensional space" picks up the most representative patterns in the anatomical structure as the features to look for when the input 706 is provided. In an example, the input data may be a 3D model of an anatomical structure, which is converted to 2D data (e.g., a projection as described above) and supplied to the ML model 704 as the input 706. The model 704 is trained to identify an implant model from a model library 708 that fits a given input. The model library 708 can include implant models corresponding to examples of the training dataset and / or other implant models. The ML model 704 provides zero or more selected models as the output 710. If multiple models are identified as the "best fit" implants for a given input, they may or may not be ranked.

[0069] The models in the model library 708 and any "hits" based on the input can be represented by an "n-dimensional latent space" 720. An example of 720 shows three dimensions, but in many instances the latent space is n-dimensional, where n is greater than or equal to 3. Each point in the latent space represents a possible output label, e.g., an implant model in these examples. Thus, each talus model may be represented as a specific point in the latent space. In this example, two dots 721, 722 represent two "hits", the best fit implant models, based on the virtual input surfaces fed into the trained ML model.

[0070] In some embodiments, the ML model may be provided as a 3D adversarial generative network (GAN) structure. In the GAN approach, a discriminator that discriminates between real / actual samples and fake / generated samples generated from a generator is trained. On the other hand, the generator is trained to generate better samples. Once the generator is sufficiently trained, it can be used as a classifier / ML model for the input anatomical features. In the context of the aspects described herein, the generator can be trained to generate an implant model (e.g., a total talus model) from scratch, instead of selecting a best-fit model from an existing library or in attrition to, thereby providing a way to select an implant model. The generator effectively learns from the patterns present in the input with the goal of generating an anatomical structure (e.g., the talus) that a human would naturally generate for a given anatomical structure. The result generated by the generator is a point cloud. The point cloud can be processed using various filtering and interpolation techniques to generate a closed surface and ultimately a continuous volume representing a complete anatomical structure such as the talus.

[0071] FIG. 8 shows an exemplary process flow for implant selection based on artificial intelligence, according to aspects described herein. All or part of this process can be performed by a computer system executing program code. In this process, a first part (such as a talus) of the anatomical structure of a first anatomical region (such as an ankle / lower limb) to be replaced by an implant is identified (802). Optionally, the process can make a determination (804) that the patient's contralateral side (e.g., having a second part of the anatomical structure in a second anatomical region) alone does not provide sufficient image data to form the basis of a desired implant to replace the first part of the anatomical structure. Thus, in this case, the process acquires (806) image data of the first part and the first anatomical region of the anatomical structure and generates (808) a 3D model incorporating the first part and the first anatomical region of the anatomical structure. The process operates the 3D model to hide (810) the first part of the anatomical structure with respect to the first anatomical region and then identifies and separates (812) the articular surfaces of the components (e.g., tibia, fibula, talo-navicular joint, calcaneus) of the first anatomical region with which the first part of the anatomical structure interacts. The process can generate (814) a partial volume based on the articular surfaces, i.e., the articular surfaces inform a part of the periphery of the volume / implant to occupy space. The partial volume represents limits regarding at least some of the limits / dimensions of the implant. Next, the process selects (816) one or more implant models of the first part of the anatomical structure that best-fit the partial volume from a library of models. The process can fit (818) one or more such selected models for desired engagement with one or more components of the first anatomical region, and these components may be the articular surfaces and / or other components of the patient's anatomical structure.Finally, the model specification may be verified, at which point the process generates an implant specification file (820) based on the verified model specification, transmits the implant specification file to a 3D printer or other implant manufacturing device (822), and the implant manufacturing device generates an implant (824). As an alternative to 820, 822, 824, an existing physical implant corresponding to the verified model specification can be selected and obtained. In either case, once the physical implant is obtained, the physical implant can be surgically implanted into the patient.

[0072] FIG. 9 is a diagram showing an example of a graphical user interface (GUI) of software for implementing the aspects described herein. The GUI can be used, for example, as part of at least the library model selection module 146 of FIG. 1, to identify the best fit model and present it to the user for selection. The interface 900 is displayed when the software is loaded. The TOP-RESULTS section 902 includes an option 904 for the user to select how many outputs (e.g., the best fit model) the ML model selects and provides as implant models that may be used. The option 904 can be configured with a default, for example 5 as shown. The option 904 is implemented as a number picker with an up-down selector in this example.

[0073] The interface 900 also includes an anatomical structure type selection unit 906 that enables the user to select the anatomical surface (here, the joint surface) that the software should use in comparing the models in the library against the anatomical structure model loaded by the file input unit 908. In this example, the anatomical structure type selection unit 906 is implemented as three radio buttons corresponding to the calcaneus, navicular, or fibular joint surface, and the user is to select one of the three. In embodiments where multiple joint surfaces may be considered in the selection of the best-fit model, the surface type selection unit 906 may be implemented using different interface element types to enable the user to indicate multiple joint surfaces. In the example of FIG. 9, the user has selected (and / or the program is using by default) the calcaneal joint surface.

[0074] The file input unit 908 enables the user to provide the file to be processed. In this example, the software imports the.stl file of the anatomical structure. The user selects the search button and references the appropriate.stl file containing the patient's anatomical structure model from the file system. When the user selects the file, the file may be opened / loaded.

[0075] The user can also select a search method in the interface unit 910, which is implemented as three radio buttons corresponding to a pre-trained, custom, and histogram search method. In the pre-trained search method, the search utilizes an ML model (e.g., a convolutional neural network) trained on a dataset, and then a final layer with classes / labels corresponding to the available implant models in the database is added. The training can be based on a desired anatomical structure, specialized for an anatomical region (e.g., shoulder, ankle, knee, etc.), or more general (e.g., whole body CT or scan). The custom search method may train the ML model in a more customized way for specific symptoms, deformities, etc. (i.e., a less generalizable method than the pre-trained method). The histogram search method, instead, may be based on statistics and statistical modeling that compares the selected anatomical structure to a "normal" (known by the mean value) anatomical structure, measurements, curvatures, etc. stored in a database of normal anatomical structures. This allows knowing the degree of correlation between the actual patient's anatomical structure and the average / normal anatomical structure, and identifying the characteristics of the implant that are desired for the patient's anatomical structure and thus useful for the selection of the appropriate implant.

[0076] Once these parameters are specified, the user selects the OK button, which causes the system to start processing and display the best-fit results. In an embodiment, the result display can be output to the command line interface as shown in FIG. 10, where in this example, five results (#0 - #4) are identified by the name or other identifier of the implant model. Additionally or alternatively, the interface presents a 2D graphical rendering of the input joint surface and a 2D rendering of the profiles of the specified joint surfaces corresponding to the five best-fit results, as shown in FIG. 11.

[0077] Thus, after performing the search, based on the search results selected by the program code, the software can display one or more best-fit results. In some examples, the user selects a given result. Any one or more of these results can be used in combination with other aspects described herein, such as the aspects of FIG. 1, such as the model candidate designation module 144. In embodiments where a search based on multiple articular surfaces is performed, different sets of best-fit results, each corresponding to one of the articular surfaces, may be provided. Additionally or alternatively, the software can display the full model and the user can select from there.

[0078] FIG. 12 shows an example of a process for implant selection according to the aspects described herein. In some examples, this process is performed by one or more computer systems as described herein.

[0079] This process obtains (1202) a machine learning (ML) model trained to select an anatomical implant model of a physical implant for partial or total replacement of a target anatomical structure based on the characteristics of the anatomical surface of the anatomical structure adjacent to the target anatomical structure that is partially or fully replaced. The method also obtains (1204) image data of the anatomical region of a particular patient. The anatomical region includes the patient's target anatomical structure and the patient's other anatomical structures, and the other anatomical structures are adjacent to the patient's target anatomical structure. In an example, the target anatomical structure is the patient's talus, and the other anatomical structures are the navicular bone, calcaneus, tibia, and / or fibula.

[0080] This process proceeds by determining (1206) the characteristics of at least one anatomical surface of other anatomical structures from the image data. The at least one anatomical surface is at least one interface between the other anatomical structures of the patient and the target anatomical structure. In an example, the anatomical surface is an articular surface. The anatomical surface can include an articular surface where a physical implant will engage based on being at least partially surgically implanted within the patient. In a particular embodiment, the anatomical region has the patient's ankle, the target anatomical structure has the talus, and the at least one anatomical surface has at least one articular surface of at least one bone adjacent to the talus, such as the articular surfaces of the navicular bone, calcaneus bone, tibia, and / or fibula of each bone around the patient's talus.

[0081] The user may manually identify the surface and / or, for example, a process applying a machine learning model may automatically identify the surface. Further, the user may optionally adjust the identified surface. Thus, determining (1206) the characteristics of at least one anatomical surface can be based on (i) manual indication by the user of at least one anatomical surface provided based on user input to a graphical user interface that displays a model including at least the other anatomical structures of the patient, and / or (ii) automatic analysis of the image data to identify at least one anatomical surface.

[0082] In certain embodiments, preprocessing is performed to convert 3D data to 2D data. Thus, the resulting image data can include three-dimensional digital model data representing the anatomical region of the patient, and determining (1206) the characteristics of at least one anatomical surface of other anatomical structures can include processing the image data to present at least one anatomical surface as at least one digital three-dimensional surface and converting at least one digital three-dimensional surface to at least one two-dimensional projection, and the determined characteristics of the at least one anatomical surface are determined from the at least one two-dimensional projection.

[0083] Continuing to refer to FIG. 12, the process applies an ML model (1208) using determined characteristics of at least one anatomical surface and, based on the application, obtains a selected implant model (1210). The selected implant model is selected by the ML model as a partial or whole replacement of the patient's target anatomical structure and as a designation of a physical implant for at least partial surgical implantation within the patient.

[0084] The ML model can be trained using samples from a library of implant models. This training can train the ML model to select an anatomical implant model from the library of implant models, and thus the selected implant model can be selected by the ML model from the library of implant models.

[0085] Alternatively, the ML model can include a trained generator of a generative adversarial network, where the generator is trained using samples from a library of implant models and is trained to generate implant models. In this case, the selected implant model is generated by the generator and can include the implant model selected by the generator for output as the selected implant model.

[0086] The selected implant model may or may not be appropriate for the patient. Thus, the process proceeds by providing (1212) the selected implant model to a model candidate specifying module to specify model candidates for verification. The model candidate specifying module can process the selected model for possible operations and / or other tasks. In this regard, the selected implant model may or may not be the one initially presented for verification. Further, the selected implant model can be presented to the user on a graphical user interface. Any method can preprocess the image data to create a three-dimensional digital model of the patient's anatomical region omitting the target anatomical structure and present this for display to the user.

[0087] The process may also optionally include receiving an operation (1214). The user may wish to operate on the selected model at this point (or after verification fails), which results in a different model specification and thus also a different specification of the physical implant that the model represents. Thus, in one aspect, receiving an operation (1214) can include receiving an operation on the selected implant model, where the operation modifies the selected implant model and generates an implant model candidate for verification. Based on such an operation, the implant model candidate specifies a physical implant having one or more different physical characteristics compared to the physical implant specified by the selected implant model initially selected and provided prior to the user's operation. By way of example, the operation on the selected implant model may include an operation specified by the user and / or an operation automatically determined and optionally automatically applied by artificial intelligence.

[0088] Additionally or alternatively, the user may desire to apply an operation to a representation (e.g., surface) of the patient's anatomical structure. Thus, receiving an operation (1214) can include receiving an operation on at least one anatomical surface characteristic of another anatomical structure. For example, the user may change the shape or other characteristics of the surface and / or the distance between surfaces. When the user moves a bone or other anatomical structure, the position of the anatomical surface of that bone may change. An operation on at least one anatomical surface characteristic can include an operation specified by the user and / or an operation automatically determined and optionally automatically applied by artificial intelligence.

[0089] Regardless of whether the operation has been applied, at this point, an implant model candidate is obtained as acquired and provided by 1210 / 1212, or after that operation. This process proceeds by providing the implant model candidate to a verification module for verification (1216). Verification determines whether a specified physical implant from the implant model candidate is suitable for surgical implantation within the patient. Since verification is performed on the presented candidate, if an operation has been performed, it is performed taking the operation into account. Thus, verification, which determines whether a physical implant specified by the implant model candidate is suitable for surgical implantation within the patient, is performed based at least in part on any manipulated characteristics of the anatomical surface and / or on an operation made to the implant model itself.

[0090] At 1218, the process determines whether the implant model candidate passes the verification, i.e., whether the physical implant specified by the implant model candidate is suitable for surgical implantation in the patient. If not (1218, N), the process can return to 1214, where it may receive, using the candidate implant selection module, the candidate that failed the verification and / or operations on the patient's anatomical structure. This provides information for the next candidate, which may be a modified or unmodified version of the first failed candidate, and this next candidate is provided for verification at 1216. In this regard, the failed candidate is substantially the next iteration of the "selected implant model" initially assigned to the implant candidate designation module. For each iteration, the implant model candidate that did not pass is provided as the "selected implant model" for the next iteration of the iteration, operations are optionally received, the next candidate is obtained, and provided to the verification module for verification. The iteration may be performed any number of times until the candidate is verified.

[0091] After the implant model candidate fails (1218, N), instead of returning to 1214, the process returns to 1220 to obtain the (next) selected implant model and can work from that next model. This may be particularly so if the ML model has identified multiple best-fit implant models. Additionally or alternatively, if the user has manipulated the patient's anatomical structure as described above, it may be desirable to reapply the ML model (return to 1208) to check whether another best-fit implant from the library fits better.

[0092] A specific example where the presented implant model is directly sent for verification (e.g., when no operation is created and / or when the selected implant model chosen by ML is directly provided for verification as an initial task), the selected implant model is provided to the verification module as an implant model candidate for verification. In this case, the implant model candidate is an initial implant model candidate. Based on verification that determines that the physical implant specified by the initial implant model candidate fails (1218, N), the process returns to 1214. Next, the process at 1214 (i) receives an operation on the initial implant model candidate, the operation changes the initial implant model candidate to generate a different implant model candidate for verification, and the different implant model candidate has one or more different physical characteristics compared to the physical implant specified by the initial implant model candidate and / or (ii) specifies a physical implant having an operation on the characteristics of at least one anatomical surface of another anatomical structure. As part of providing the implant model candidate 1216, the process determines the next implant model candidate to provide to the verification module for verification. Based on receiving an operation on the initial implant model candidate, the next implant model candidate to provide is determined to be a different implant model generated from the operation on the initial implant model candidate. However, based on receiving an operation on the properties of at least one anatomical surface of another anatomical structure and not receiving an operation on the initial implant model candidate, the next implant model candidate is determined to be the initial implant model candidate. Thus, the next implant model candidate may be the failed initial implant model candidate or may not be the initial implant model candidate. As described, this next implant model candidate is provided to the verification module for verification (1216) to determine whether the physical implant specified by the next implant model candidate is qualified for surgical implantation in a patient, and the verification is at least partially based on the operation received on the initial implant model candidate and / or the operation on the characteristics of at least one anatomical surface.

[0093] Instead, at 1218, if a candidate is verified (1218, Y), the process provides an implant model candidate as a designation for use by the physical implant (1220). At that point, the physical implant may be obtained if available and may be manufactured if desired.

[0094] The implant model candidate is an engineered version of a selected implant model selected from a library and, in a situation where the engineered version does not separately exist in the library, can be added to the library as a valid implant model. Additionally or alternatively, as part of an example of training to associate the implant model candidate as a model suitable for an anatomical structure, the implant model candidate can be displayed in a training dataset by associating the implant model candidate with at least one anatomical surface of another anatomical structure. Further, even if the implant model candidate already exists in the library but in an example of training is not associated with the anatomical structure of a particular patient for which it was verified (e.g., the patient's anatomical structure is not reflected in existing examples of training and / or the user manipulates the anatomical structure of an actual patient to generate a virtual patient anatomical structure that is not yet reflected), the implant model candidate can be shown in the training dataset as part of an example of training to associate the implant model candidate with the patient's anatomical structure in order to associate the implant model candidate as a model suitable for the anatomical structure.

[0095] As used herein, terms such as "connected", "coupled", "contacted", "joined" are defined broadly to encompass a variety of arrangements and assembly techniques. These arrangements and techniques include (1) direct joining of one component to another without an intervening component therebetween (e.g., the parts are in direct physical contact), and (2) joining of one component to another with one or more additional components therebetween, where one component that "connects", "contacts", or "joins" to the other component is operatively associated with the other component in any manner (e.g., electrically, fluidly, physically, optically, etc.) even in the presence of one or more additional components therebetween, but are not limited thereto. It should be understood that some components that are in direct physical contact with each other may or may not be in electrical contact and / or fluid contact with each other. Further, two components that are electrically connected, electrically coupled, optically connected, optically coupled, fluidly connected, or fluidly coupled may or may not be in direct physical contact with each other, and one or more other components may be disposed therebetween.

[0096] As used herein, the terms "comprising" and "having" mean the same thing.

[0097] The terms "substantially", "about", "approximately", "relatively", or other such similar terms that may be used throughout the present disclosure, including in the claims, are used to describe and account for small variations from a reference or parameter, such as variations due to processing. Such small fluctuations include zero fluctuations from the reference or parameter. For example, it can refer to ±10% or less, such as ±5% or less, such as ±2% or less, such as ±1% or less, such as ±0.5% or less, such as ±0.2% or less, such as ±0.1% or less, such as ±0.05% or less. As used herein, terms such as "substantially", "about", "approximately", "relatively" may also mean no variation.

[0098] As used herein, "electrically coupled" refers to the transfer of electrical energy between any combination of power sources, electrodes, conductive surfaces, droplets, conductive traces, wires, waveguides, nanostructures, other circuit segments, etc. The term "electrically coupled" may be used in connection with direct or indirect connections and may pass through various media such as a fluid medium, an air gap, etc.

[0099] As used herein, "neural network" refers to a programming paradigm inspired by biology that enables a computer to learn from observational data. This learning is called deep learning and is a series of learning techniques in neural networks. Neural networks, including modular neural networks, are capable of pattern recognition with speed, accuracy, and efficiency in situations where there are multiple and extensive datasets, including across the distributed network of the technical environment. Recent neural networks are non-linear statistical data modeling tools. These are typically used to model complex relationships between inputs and outputs or to identify patterns in data (i.e., neural networks are tools for non-linear statistical data modeling or decision-making). Generally, program code utilizing a neural network can model complex relationships between inputs and outputs and identify patterns in data. Particularly due to the high speed and efficiency of neural networks when analyzing multiple complex datasets, neural networks and deep learning provide solutions to many problems in image recognition, speech recognition, and natural language processing. A neural network can model complex relationships between inputs and outputs for classification to identify patterns in data including images. Particularly due to the high speed and efficiency of neural networks when analyzing multiple complex datasets, neural networks and deep learning provide solutions to many problems in image recognition that would otherwise be impossible. As will be described below, the neural networks in some embodiments of the present invention are utilized to learn various features of the design of a talus implant, including but not limited to focusing on the articular surface.

[0100] The "Convolutional Neural Network" (CNN) used in this specification is a class of neural networks. CNN utilizes a feed-forward artificial neural network and is most commonly applied to the analysis of visual images. CNN is named as such because it utilizes convolutional layers that apply a convolution operation (performing a mathematical operation on two functions to generate a third function representing how the shape of one function is changed by the other) to the input and pass the result to the next layer. Convolution mimics the response of individual neurons to visual stimuli. Each convolutional neuron processes only the data within its receptive field. Processing an image using a general (i.e., fully connected feed-forward) neural network requires a very large number of neurons due to the very large input size related to the image, which is not practical. To support the learning process, CNN fine-tunes a large number of parameters and an enormous pre-labeled dataset, enabling the use of a consistent number of trainable parameters regardless of the image size, reducing the number of free parameters and allowing the network to be deeper with fewer parameters, thus addressing this problem by using CNN. CNN uses backpropagation to solve the problem of vanishing or exploding gradients in the training of traditional multi-layer neural networks. Therefore, CNN can be used in large-scale (image) recognition systems, providing state-of-the-art results in segmentation, object detection, and object retrieval. CNN is possible in any dimension, but many of the existing CNNs are two-dimensional and process a single image. These images contain pixels in a two-dimensional (2D) space (vertical, horizontal) and are processed through a series of 2D filters to understand the set of pixels that most correspond to the final output classification. 3D Convolutional Neural Network (3D-CNN) is an extension of the more traditional 2D-CNN and is generally used in problems related to video classification. 3D-CNN accepts multiple images, often consecutive image frames of a video, and uses 3D filters to understand the 3D set of presented pixels. In the current context, as described in this specification, the images provided to the CNN include anatomical images of patients.

[0101] As used herein, a "classifier" is composed of various cognitive algorithms, artificial intelligence (AI) instruction sets, and / or machine learning algorithms. Classifiers include, but are not limited to, deep learning models (e.g., neural networks with many layers) and random forest models. The classifier classifies items (data, metadata, objects, etc.) into groups based on the relationships between data elements contained in the metadata from the records. In some embodiments of the present invention, the program code can utilize the frequency of occurrence of features in mutual information to identify and filter false positives. Generally, the program code utilizes a classifier to create a boundary between first-quality data and second-quality data. By continuously using the classifier, its accuracy is improved by testing the classifier. When training the classifier, in some instances, the program code supplies an existing feature set that describes the features of the metadata and / or data to one or more cognitive analysis algorithms being trained. The program code trains the classifier to classify records based on the presence or absence of certain predetermined conditions known prior to tuning. The presence or absence of the conditions is not specified in the records of the dataset. When classifying a source as providing data for a given condition (based on metadata), the program code can utilize the classifier to indicate the probability of the given condition, for example, on a scale between 0 and 1, where 1 indicates certain existence. The classification does not have to be binary and can be a value on an established scale. As disclosed herein, in some instances, a classifier is utilized to select the optimal talus from a database based on program code that cognitively analyzes the patient's anatomical structure.

[0102] As used herein, the term "deep learning model" refers to a type of classifier. A deep learning model can be implemented in various forms such as a neural network (e.g., a convolutional neural network). In some examples, a deep learning model includes multiple layers, and each layer has multiple processing nodes. In some examples, the layers are processed in sequence, and the nodes in the layer closer to the model input layer are processed before the nodes in the layer closer to the model output layer. Thus, a layer feeds the next layer. Internal nodes are often "hidden" nodes in the sense that the values of their inputs and outputs are not visible from outside the model.

[0103] As used herein, the term "conditional adversarial network" ("cGAN") refers to an adversarial generative network (GAN), which is a machine learning framework used for learning a generative model. Specifically, cGAN is utilized to generate images conditionally. GAN relies on a generator that learns to generate new images and a discriminator that learns to distinguish between synthetic and real images. In cGAN, a conditioning is applied, i.e., both the generator and the discriminator are conditioned on auxiliary information from other modalities. Thus, cGAN can learn a multimodal mapping from input to output by being supplied with different context information. In the examples of this specification, the context information can include both anatomical data from a patient and a library of existing talus implants.

[0104] As used herein, the term "processor" refers to a hardware and / or software device capable of executing computer instructions, including but not limited to one or more software processors, hardware processors, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), and / or programmable logic devices (PLDs).

[0105] Although various examples are provided, modifications are possible without departing from the spirit of the aspects of the claims.

[0106] The processes described herein may be implemented singly or in combination by one or more computer systems. FIG. 13 shows an example of such a computer system and related apparatus for incorporating and / or using the aspects described herein. A computer system may herein be referred to as a data processing device / system, a computing device / system / node, or simply a computer. A computer system may be based on one or more of a variety of system architectures and / or instruction set architectures, such as those provided by Intel Corporation (Santa Clara, Calif., USA) or ARM Holdings plc (Cambridge, England, UK) by way of example.

[0107] FIG. 13 shows a computer system 1300 that communicates with an external device 1312. The computer system 1300 includes one or more processors 1302, such as a central processing unit (CPU) for example. The processor can include functional components used for executing instructions, such as a functional component for calling program instructions from locations such as cache and main memory, a functional component for decoding program instructions, a functional component for executing program instructions, a functional component for accessing memory for instruction execution, and a functional component for writing the results of executed instructions. The processor 1302 can also include registers used by one or more functional components. The computer system 1300 also includes a memory 1304, an input / output (I / O) device 1308, and an I / O interface 1310, which are coupled to the processor 1302 and to each other via one or more buses and / or other connections. The bus connection can represent any one or more of multiple types of bus structures, including a memory bus or memory controller, a peripheral device bus, an accelerated graphics port, a processor or local bus using any of various bus architectures. By way of example, such architectures include Industry Standard Architecture (ISA), Micro Channel Architecture (MCA), Enhanced ISA (EISA), Video Electronics Standards Association (VESA) local bus, Peripheral Component Interconnect (PCI), and the like.

[0108] Memory 1304 can be or include the main memory or system memory (e.g., random access memory) used for the execution of program instructions, a storage device such as a hard drive, flash media, or optical media by way of example, and / or cache memory by way of example. Memory 1304 can include a cache such as a shared cache that can be coupled to a local cache (e.g., including L1 cache, L2 cache, etc.) of processor 1302, for example. Further, memory 1304 can be or include at least one computer program product having a set (e.g., at least one) of program modules, instructions, code, etc. configured to execute the functions of the embodiments described herein when executed by one or more processors.

[0109] Memory 1304 can store an operating system 1305 and other computer programs 1306 such as one or more computer programs / applications executed to perform the aspects described herein. Specifically, the program / application can include computer-readable program instructions that may be configured to execute the functions of the embodiments of the aspects described herein.

[0110] Examples of I / O devices 1308 include, but are not limited to, microphones, speakers, global positioning system (GPS) devices, cameras, lights, accelerometers, gyroscopes, magnetometers, sensors configured to sense light, proximity, heart rate, body temperature and / or ambient temperature, blood pressure, and / or skin resistance, and activity monitors. The I / O devices may be incorporated into the computer system as shown, but in some embodiments, the I / O devices may be considered external devices (1312) coupled to the computer system via one or more I / O interfaces 1310.

[0111] Computer system 1300 may communicate with one or more external devices 1312 via one or more I / O interfaces 1310. Exemplary external devices include a keyboard, a pointing device, a display, and / or any other device that enables a user to interact with computer system 1300. Other exemplary external devices include any device that enables computer system 1300 to communicate with one or more other computing systems or peripheral devices such as a printer. A network interface / adapter is an exemplary I / O interface that enables computer system 1300 to communicate with one or more networks such as a local area network (LAN), a general wide area network (WAN), and / or a public network (e.g., the Internet), and provides communication with other computing devices or systems, storage devices, etc. Ethernet-based (such as Wi-Fi) interfaces and Bluetooth® adapters are examples of currently available network adapters used in computer systems (Bluetooth® is a registered trademark of Bluetooth SIG, Inc, Kirkland, Washington, U.S.A.).

[0112] Communication between I / O interface 1310 and external device 1312 may occur via wired and / or wireless communication links 1311 such as an Ethernet-based wired or wireless connection. Examples of wireless connections include cellular, Wi-Fi, Bluetooth®, proximity, near-field wireless, or other wireless connections. More generally, communication link 1311 may be any suitable wireless communication link and / or wired communication link for communicating data.

[0113] The specific external device 1312 may include one or more data storage devices that may store one or more programs, one or more computer-readable program instructions, and / or data, etc. The computer system 1300 may include a removable / non-removable, volatile / non-volatile computer system storage medium, and / or may be coupled to the computer system storage medium, and may communicate with the computer system storage medium (e.g., as an external device of the computer system). For example, it may include, and / or may be coupled to, a non-removable non-volatile magnetic medium (typically called a "hard drive"), a magnetic disk drive for reading from and writing to a removable non-volatile magnetic disk (e.g., a "floppy disk"), and / or a removable non-volatile optical disk such as a CD-ROM, DVD-ROM or other optical medium, and an optical disk drive for reading from or writing to the optical disk.

[0114] The computer system 1300 may be operable in many other general-purpose or special-purpose computing system environments or configurations. The computer system 1300 can take any of various forms, and well-known examples thereof include personal computer (PC) systems, server computer systems such as messaging servers, thin clients, thick clients, workstations, laptops, mobile devices / computers such as handheld devices, smartphones, tablets and wearable devices, multiprocessor systems, microprocessor-based systems, telephony devices, network appliances (such as edge appliances), virtualized devices, storage controllers, set-top boxes, programmable household appliances, network PCs, minicomputer systems, mainframe computer systems, and distributed cloud computing environments including any of the above systems or devices, but are not limited thereto.

[0115] Aspects of the invention may be a system, method, and / or computer program product, any of which may be configured to perform or facilitate the aspects described herein.

[0116] In some embodiments, aspects of the invention may take the form of a computer program product that may be implemented as a computer-readable medium. The computer-readable medium may be a tangible storage device / media storing computer-readable program code / instructions. Examples of computer-readable media include, but are not limited to, electronic, magnetic, optical, or semiconductor storage devices or systems, or combinations thereof. Exemplary embodiments of computer-readable media include hard drives or other mass storage devices, electrical connections with wires, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory such as EPROM or flash memory, optical fibers, portable computer disks / diskettes such as compact disc read-only memory (CD-ROM) or digital versatile disc (DVD), optical storage devices, magnetic storage devices, or any combination of the foregoing. The computer-readable medium may be readable by a processor, or processing unit, etc. to obtain data (e.g., instructions) from the medium for execution. In a particular example, the computer program product is or includes one or more computer-readable media containing / storing computer-readable program code to provide and facilitate one or more aspects described herein.

[0117] As described above, program instructions included in or stored on a computer-readable medium can be obtained and executed by any of various suitable components of a computer system, such as a processor of the computer system, to cause the computer system to operate and function in a particular manner. Such program instructions for performing operations for implementing, achieving, or facilitating the aspects described herein may be written in any desired programming language or may be compiled from code written in any desired programming language. In some embodiments, such programming languages include object-oriented programming languages and / or procedural programming languages such as C, C++, C#, Java, and the like.

[0118] Program code can include one or more program instructions obtained for execution by one or more processors. The computer program instructions may be provided, for example, to one or more processors of one or more computer systems and, when executed by one or more processors, cause the program instructions to manufacture a machine such that the aspects of the invention, such as the operations or functions described in the flowcharts and / or block diagrams described herein, are implemented, achieved, or facilitated. Accordingly, each block, or combination of blocks, of the flowchart diagrams and / or block diagrams depicted and described herein can, in some embodiments, be implemented by computer program instructions.

[0119] Although various embodiments have been described above, these are merely examples.

[0120] The terms used in this specification are for the purpose of describing particular embodiments only and are not intended to be limiting. In this specification, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. As used herein, the terms “comprising” and / or “comprises” specify the presence of the stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0121] All structural, material, acts, and equivalents (if any) corresponding to the means-plus-function or step-plus-function elements in the following claims are intended to include any structure, material, act for performing the function in combination with other claimed elements as specifically claimed. The description of one or more embodiments has been presented for purposes of illustration and description, but is not intended to be exhaustive or limiting. Many modifications and variations will be apparent to those of ordinary skill in the art. Embodiments were chosen and described in order to best explain the various aspects and practical applications, and to enable others of ordinary skill in the art to understand the various embodiments with various modifications as are suited to the particular use contemplated.

Claims

1. A method implemented by a computer, comprising: obtaining a machine learning model trained to select an anatomical implant model of a physical implant for partial or total replacement of a target anatomical structure based on characteristics of an anatomical surface of an anatomical structure adjacent to the target anatomical structure to be partially or totally replaced; obtaining image data of an anatomical region of a patient, the anatomical region having the patient's target anatomical structure and other anatomical structures of the patient, the other anatomical structures being adjacent to the patient's target anatomical structure; determining characteristics of at least one anatomical surface of the other anatomical structures from the image data, the at least one anatomical surface being present at each of at least one interface between the patient's other anatomical structures and the target anatomical structure; utilizing the determined characteristics of the at least one anatomical surface, applying the machine learning model, and obtaining a selected implant model selected based on the application, the selected implant model being selected by the machine learning model as a candidate for a surgical implant in the patient at least partially as a partial or total replacement of the patient's target anatomical structure; A method comprising the above steps.

2. The method according to claim 1, further comprising providing the selected implant model to a model candidate designation module to designate a model candidate for verification.

3. presenting the selected implant model to a user on a graphical user interface; receiving an operation on the selected implant model, the operation changing the selected implant model and generating an implant model candidate for verification, the implant model candidate designating a physical implant having one or more different physical characteristics compared to the physical implant designated by the selected implant model. To provide an implant model candidate to a verification module for verification, the verification further includes determining whether the physical implant specified by the implant model candidate is qualified for surgical implantation in a patient, and providing the method according to claim 2.

4. The operations on the selected implant model are at least one operation specified by the user, and at least one operation automatically determined by artificial intelligence, and the method according to claim 3 having at least one selected from the group including.

5. Based on verification that determines that the physical implant specified by the implant model candidate fails, (i) receiving an operation, and (ii) providing the implant model candidate to the verification module for verification one or more times, further including repeating, In each iteration of the iteration, the unqualified implant model candidate is provided as the selected implant model for the next iteration of the iteration, and the method according to claim 3.

6. Further including receiving an operation on the characteristics of the at least one anatomical surface of the other anatomical structure, The verification for determining whether the physical implant specified by the implant model candidate is qualified for surgical implantation in a patient is at least partially based on the manipulated characteristics of the at least one anatomical surface, and the method according to claim 3, 4 or 5.

7. The operations on the characteristics of the at least one anatomical surface are at least one operation specified by the user, and at least one operation automatically determined by artificial intelligence, and the method according to claim 6 having at least one selected from the group including.

8. Receiving an operation on the characteristics of the at least one anatomical surface of the other anatomical structure, Further including providing the selected implant model as an implant model candidate for verification to a verification module for verification, The verification determines whether the physical implant specified by the implant model candidate is qualified for surgical implantation in a patient, The verification is at least partially based on the manipulated characteristics of the at least one anatomical surface, and the method according to claim 2.

9. (i) An operation on a selected implant model, the operation changing the selected implant model, generating an implant model candidate for verification, and different implant model candidates specifying a physical implant having one or more different physical characteristics as compared to the physical implant specified by the selected implant model, and, (ii) An operation on the characteristics of the at least one anatomical surface of the other anatomical structure, based on receiving at least one selected from the group including, The method further includes providing the implant model candidate to a verification module for verification, Verification determines whether the physical implant specified by the implant model candidate is suitable for surgical implantation in a patient, Based on verification determining that the physical implant specified by the implant model candidate is suitable for surgical implantation in a patient, the method further includes displaying the implant model candidate in a training dataset as part of an example of training to associate the implant model candidate with the at least one anatomical surface of the other anatomical structure. The method according to claim 2.

10. Further including providing the selected implant model as an implant model candidate to a verification module for verification, Verification determines whether the physical implant specified by the implant model candidate is suitable for surgical implantation in a patient. The method according to claim 1.

11. The implant model candidate is an initial implant model candidate, The method, Based on verification determining that the physical implant specified by the initial implant model candidate is not suitable, An operation on the initial implant model candidate, the operation changing the initial implant model candidate, generating a different implant model candidate for verification, and different implant model candidates specifying a physical implant having one or more different physical characteristics as compared to the physical implant specified by the initial implant model candidate, and, An operation on the characteristics of the at least one anatomical surface of the other anatomical structure, Receiving at least one selected from the group having, Determining the next implant model candidate to provide to the verification module for verification, Based on receiving an operation on the initial implant model candidate, determining that the next implant model candidate is a different implant model generated from the operation on the initial implant model candidate, or Receiving an operation on at least one anatomical surface property of the other anatomical structure and determining that the next implant model candidate is the initial implant model candidate based on not receiving an operation on the initial implant model candidate, determining; Providing the next implant model candidate to the verification module for verification of whether the physical implant specified by the next implant model candidate passes surgical implantation in a patient, wherein the verification for determining whether the physical implant specified by the next implant model candidate passes for surgical implantation in a patient is at least partially based on receiving at least one selected from a group having an operation on the initial implant model candidate and an operation on the property at the at least one anatomical surface, providing, the method according to claim 10.

12. Further comprising training the machine learning model to select an anatomical implant model, The training uses samples from a library of implant models and trains the machine learning model to select an anatomical implant model from the library of implant models, The selected implant model is selected by the machine learning model from the library of implant models, the method according to any one of claims 1, 2, 3, 4, 5, 8, 9, 10 or 11.

13. The machine learning model has a trained generator of a generative adversarial network (GAN), The generator is trained using samples from a library of implant models and is trained to generate implant models, The selected implant model is generated by the generator and has the implant model selected by the generator for output as the selected implant model, according to any one of claims 1, 2, 3, 4, 5, 8, 9, 10 or 11.

14. The obtained image data has three-dimensional digital model data representing the anatomical region of the patient, Determining the characteristics of the at least one anatomical surface of the other anatomical structure Processing the image data to present the at least one anatomical surface as at least one digital three-dimensional surface, and Converting the at least one digital three-dimensional surface into at least one two-dimensional projection, and includes, The determined characteristics of the at least one anatomical surface are determined from the at least one two-dimensional projection, according to any one of claims 1, 2, 3, 4, 5, 8, 9, 10 or 11.

15. The method according to claim 14, further comprising preprocessing the image data to generate a three-dimensional digital model of the anatomical region of the patient with the target anatomical structure omitted therefrom.

16. One or more anatomical surfaces have one or more joint surfaces with which a physical implant will engage based on being at least partially surgically implanted within the patient, according to any one of claims 1, 2, 3, 4, 5, 8, 9, 10 or 11.

17. The anatomical region has the patient's ankle, The target anatomical structure has a talus, The at least one anatomical surface has at least one joint surface of at least one bone adjacent to the talus, according to any one of claims 1, 2, 3, 4, 5, 8, 9, 10 or 11.

18. Determining the characteristics of the at least one anatomical surface Based on at least one selected from the group having manual indication by the user of the at least one anatomical surface provided based on user input to a graphical user interface that displays a model including at least the other anatomical structures of the patient, and Automated analysis of the image data to confirm the at least one anatomical surface, according to any one of claims 1, 2, 3, 4, 5, 8, 9, 10 or 11.

19. A memory, A processor that communicates with the memory, configured to execute a method, the method comprising: obtaining a machine learning model trained to select an anatomical implant model of a physical implant for partial or total replacement of a target anatomical structure based on characteristics of an anatomical surface of an anatomical structure adjacent to the target anatomical structure to be partially or totally replaced; obtaining image data of an anatomical region of a patient, the anatomical region having the target anatomical structure of the patient and other anatomical structures of the patient, the other anatomical structures being adjacent to the target anatomical structure of the patient; determining characteristics of at least one anatomical surface of the other anatomical structures from the image data, the at least one anatomical surface being present at each of at least one interface between the other anatomical structures of the patient and the target anatomical structure; utilizing the determined characteristics of the at least one anatomical surface, applying the machine learning model, and obtaining a selected implant model selected based on the application, the selected implant model being selected by the machine learning model as a partial or total replacement of the target anatomical structure of the patient and at least partially designated as a candidate for a surgical implant within the patient as a physical implant; A computer system comprising.

20. The computer system according to claim 19, wherein the method further comprises providing the selected implant model to a model candidate designation module to designate a model candidate for verification.

21. The method further comprises: presenting the selected implant model to the user on a graphical user interface; receiving an operation on the selected implant model, the operation being to change the selected implant model and generate an implant model candidate for verification, the implant model candidate designating a physical implant having one or more physical characteristics different from the physical implant designated by the selected implant model. To provide an implant model candidate to a verification module for verification, the verification further includes determining whether the physical implant specified by the implant model candidate is surgically implantable in a patient, and providing the same, the computer system according to claim 20.

22. The operation on the selected implant model is at least one operation specified by a user, and at least one operation automatically determined by artificial intelligence, the computer system according to claim 21 having at least one selected from the group including the same.

23. Based on verification that determines that the physical implant specified by the implant model candidate fails, the method further includes repeating (i) receiving an operation and (ii) providing the implant model candidate to the verification module for verification one or more times, In each iteration of the repetition, the implant model candidate that has failed is provided as the selected implant model for the next iteration of the repetition, the computer system according to claim 21.

24. The method further includes receiving an operation on the characteristics of the at least one anatomical surface of the other anatomical structure, The verification for determining whether the physical implant specified by the implant model candidate is surgically implantable in a patient is at least partially based on the manipulated characteristics of the at least one anatomical surface, the computer system according to claim 21, 22 or 23.

25. The operation on the characteristics of the at least one anatomical surface is at least one operation specified by a user, and at least one operation automatically determined by artificial intelligence, the computer system according to claim 24 having at least one selected from the group including the same.

26. The method is receiving an operation on the characteristics of the at least one anatomical surface of the other anatomical structure, further including providing the selected implant model as an implant model candidate for verification to a verification module for verification, The verification determines whether the physical implant specified by the implant model candidate is surgically implantable in a patient, Verification is based at least in part on the manipulated characteristics of the at least one anatomical surface, the computer system according to claim 20. **Claim 27** (i) an operation on a selected implant model, the operation changing the selected implant model to generate an implant model candidate for verification, different implant model candidates specifying physical implants having one or more different physical characteristics compared to the physical implant specified by the selected implant model, and (ii) an operation on the characteristics of the at least one anatomical surface of the other anatomical structure, based on identifying an implant model candidate after receiving at least one selected from the group comprising the method further comprising providing the implant model candidate to a verification module for verification verification determining whether the physical implant specified by the implant model candidate is suitable for surgical implantation in a patient based on verification determining that the physical implant specified by the implant model candidate is suitable for surgical implantation in a patient, the method further comprising displaying the implant model candidate in a training dataset as part of an example of training to associate the implant model candidate with the at least one anatomical surface of the other anatomical structure, the computer system according to claim 20. **Claim 28** the method further comprising providing the selected implant model as an implant model candidate to a verification module for verification verification determining whether the physical implant specified by the implant model candidate is suitable for surgical implantation in a patient, the computer system according to claim 19. **Claim 29** the implant model candidate is an initial implant model candidate, the method based on verification determining that the physical implant specified by the initial implant model candidate is not suitable An operation on an initial implant model candidate, where the operation changes the initial implant model candidate, generates different implant model candidates for verification, and the different implant model candidates specify a physical implant having one or more different physical characteristics compared to the physical implant specified by the initial implant model candidate, and, An operation on the characteristics of the at least one anatomical surface of the other anatomical structure, Receiving at least one selected from the group having, Determining the next implant model candidate to provide to the verification module for verification, where, Based on receiving the operation on the initial implant model candidate, determining that the next implant model candidate is a different implant model generated from the operation on the initial implant model candidate, or, Based on receiving the operation on the characteristics of the at least one anatomical surface of the other anatomical structure and not receiving the operation on the initial implant model candidate, determining that the next implant model candidate is the initial implant model candidate, the determining, Providing the next implant model candidate to the verification module for verification to determine whether the physical implant specified by the next implant model candidate is suitable for surgical implantation in a patient, where the verification for determining whether the physical implant specified by the next implant model candidate is suitable for surgical implantation in a patient is at least partially based on receiving at least one selected from the group having the operation on the initial implant model candidate and the operation on the characteristics at the at least one anatomical surface, the providing, included in the computer system of claim 28.

30. The method further includes training the machine learning model to select an anatomical implant model, The training uses samples from a library of implant models and trains the machine learning model to select an anatomical implant model from the library of implant models, The selected implant model is the computer system according to any one of claims 19, 20, 21, 22, 23, 26, 27, 28 or 29, which is selected by the machine learning model from the library of implant models.

31. The machine learning model has a trained generator of a generative adversarial network (GAN), The generator is trained using samples from a library of implant models and is trained to generate implant models, The selected implant model is generated by the generator and has the implant model selected by the generator for output as the selected implant model, the computer system according to any one of claims 19, 20, 21, 22, 23, 26, 27, 28 or 29.

32. The obtained image data has three-dimensional digital model data representing the anatomical region of the patient, Determining the characteristics of the at least one anatomical surface of the other anatomical structures is processing the image data to present the at least one anatomical surface as at least one digital three-dimensional surface, and converting the at least one digital three-dimensional surface into at least one two-dimensional projection, and the determined characteristics of the at least one anatomical surface are determined from the at least one two-dimensional projection, the computer system according to any one of claims 19, 20, 21, 22, 23, 26, 27, 28 or 29.

33. The method further includes preprocessing the image data to generate a three-dimensional digital model of the anatomical region of the patient with the target anatomical structure omitted therefrom, the computer system according to claim 32.

34. One or more anatomical surfaces have one or more articular surfaces with which a physical implant will engage based on being at least partially surgically implanted within the patient, the computer system according to any one of claims 19, 20, 21, 22, 23, 26, 27, 28 or 29.

35. The anatomical region has the patient's ankle, The target anatomical structure has the talus, ​ The computer system according to any one of claims 19, 20, 21, 22, 23, 26, 27, 28 or 29, wherein the at least one anatomical surface has at least one joint surface of at least one bone adjacent to the talus.

36. Determining the characteristics of the at least one anatomical surface comprises manual indication by the user of the at least one anatomical surface provided based on user input to a graphical user interface that displays a model including at least the other anatomical structures of the patient, and automated analysis of the image data to identify the at least one anatomical surface, and is based on at least one selected from the group having, the computer system according to any one of claims 19, 20, 21, 22, 23, 26, 27, 28 or 29.

37. A computer-readable storage medium readable by a processing circuit and storing instructions for execution by the processing circuit to perform a method, The method comprises obtaining a machine learning model trained to select an anatomical implant model of a physical implant for partial or total replacement of a target anatomical structure based on characteristics of anatomical surfaces of anatomical structures adjacent to the target anatomical structure to be partially or totally replaced; obtaining image data of an anatomical region of a patient, the anatomical region having the target anatomical structure of the patient and other anatomical structures of the patient, the other anatomical structures being adjacent to the target anatomical structure of the patient; determining characteristics of at least one anatomical surface of the other anatomical structures from the image data, the at least one anatomical surface being present at each of at least one interface between the other anatomical structures of the patient and the target anatomical structure. Utilize the determined characteristics of the at least one anatomical surface, apply the machine learning model, and obtain an implant model selected based on the application, wherein the selected implant model is selected by the machine learning model as a partial or total replacement of the target anatomical structure of the patient and at least partially as a designation of a physical implant for a surgical implantation candidate within the patient, and obtain the selected implant model. A computer program product including the above. **Claim 38** The computer program product according to claim 37, further comprising providing the selected implant model to a model candidate designation module to designate a model candidate for verification. **Claim 39** The method presents the selected implant model to the user on a graphical user interface, and receives an operation on the selected implant model, the operation being to change the selected implant model, generate an implant model candidate for verification, and the implant model candidate designates a physical implant having one or more different physical characteristics compared to the physical implant designated by the selected implant model, and receives the operation. The computer program product according to claim 38, further comprising providing the implant model candidate for verification to a verification module, wherein the verification determines whether the physical implant designated by the implant model candidate is qualified for surgical implantation within the patient. **Claim 40** The operation on the selected implant model includes at least one selected from the group including at least one operation specified by the user and at least one operation automatically determined by artificial intelligence. The computer program product according to claim 39. **Claim 41** Based on the verification that determines that the physical implant designated by the implant model candidate fails to pass, the method further includes repeating (i) receiving an operation and (ii) providing the implant model candidate to the verification module for verification one or more times. The computer program product according to claim 39, wherein in each iteration of the iteration, the implant model candidate that did not pass is provided as the selected implant model for the next iteration of the iteration.

42. The method further includes receiving an operation on the characteristics of the at least one anatomical surface of the other anatomical structure, The verification for determining whether the physical implant specified by the implant model candidate is qualified for surgical implantation in a patient is at least partially based on the manipulated characteristics of the at least one anatomical surface. The computer program product according to claim 39, 40 or 41.

43. The operation on the characteristics of the at least one anatomical surface is At least one operation specified by the user, and The computer program product according to claim 42, having at least one selected from the group including at least one operation automatically determined by artificial intelligence.

44. The method is Receiving an operation on the characteristics of the at least one anatomical surface of the other anatomical structure, and Further including providing the selected implant model as an implant model candidate for verification to a verification module for verification, The verification determines whether the physical implant specified by the implant model candidate is qualified for surgical implantation in a patient, The verification is at least partially based on the manipulated characteristics of the at least one anatomical surface. The computer program product according to claim 38.

45. (i) An operation on the selected implant model, wherein the operation changes the selected implant model and generates an implant model candidate for verification, and different implant model candidates are compared with the physical implant specified by the selected implant model. An operation that specifies a physical implant having one or more different physical characteristics, and (ii) Based on identifying the implant model candidate after receiving at least one selected from the group including an operation on the characteristics of the at least one anatomical surface of the other anatomical structure, The method further includes providing the implant model candidate to a verification module for verification. Verification determines whether the physical implant specified by the implant model candidate passes surgical implantation within the patient, Based on the verification that determines that the physical implant specified by the implant model candidate passes surgical implantation within the patient, the method further includes displaying the implant model candidate in a training data set as part of an example of training to associate the implant model candidate with the at least one anatomical surface of the other anatomical structure. The computer program product according to claim 38.

46. The method further includes providing the selected implant model as an implant model candidate to a verification module for verification, Verification determines whether the physical implant specified by the implant model candidate passes surgical implantation within the patient. The computer program product according to claim 37.

47. The implant model candidate is an initial implant model candidate, The method is, Based on the verification that determines that the physical implant specified by the initial implant model candidate does not pass, An operation on the initial implant model candidate, the operation changes the initial implant model candidate to generate a different implant model candidate for verification, and the different implant model candidate specifies a physical implant having one or more different physical characteristics compared to the physical implant specified by the initial implant model candidate, and, An operation on the characteristics of the at least one anatomical surface of the other anatomical structure, Receiving at least one selected from the group having, Determining the next implant model candidate to be provided to the verification module for verification, Based on receiving the operation on the initial implant model candidate, determining that the next implant model candidate is a different implant model generated from the operation on the initial implant model candidate, or, Receiving the operation on the characteristics of the at least one anatomical surface of the other anatomical structure and based on not receiving the operation on the initial implant model candidate, determining that the next implant model candidate is the initial implant model candidate. Determining, To provide the next implant model candidate to the verification module for verifying whether the next implant model candidate's designated physical implant meets the criteria for surgical implantation within a patient, the verification for determining whether the physical implant designated by the next implant model candidate meets the criteria for surgical implantation within a patient includes providing, at least in part based on receiving at least one selected from a group having operations on the initial implant model candidate and operations on the characteristics at the at least one anatomical surface. The computer program product according to claim 46.

48. The method further includes training the machine learning model to select an anatomical implant model. The training uses samples from a library of implant models and trains the machine learning model to select an anatomical implant model from the library of implant models. The selected implant model is selected by the machine learning model from the library of implant models. The computer program product according to any one of claims 37, 38, 39, 40, 41, 44, 45, 46 or 47.

49. The machine learning model has a trained generator of a generative adversarial network (GAN). The generator is trained using samples from a library of implant models and is trained to generate implant models. The selected implant model is generated by the generator and has the implant model selected by the generator for output as the selected implant model. The computer program product according to any one of claims 37, 38, 39, 40, 41, 44, 45, 46 or 47.

50. The obtained image data has three-dimensional digital model data representing the anatomical region of the patient. Determining the characteristics of the at least one anatomical surface of the at least one other anatomical structure includes: processing the image data to present the at least one anatomical surface as at least one digital three-dimensional surface; converting the at least one digital three-dimensional surface into at least one two-dimensional projection. The determined characteristic of the at least one anatomical surface is determined from the at least one two-dimensional projection, the computer program product according to any one of claims 37, 38, 39, 40, 41, 44, 45, 46 or 47.

51. The method further includes preprocessing the image data to generate a three-dimensional digital model of the anatomical region of the patient with the target anatomical structure omitted therefrom, the computer program product according to claim 50.

52. One or more anatomical surfaces have one or more articular surfaces with which a physical implant will engage based on being at least partially surgically implanted within a patient, the computer program product according to any one of claims 37, 38, 39, 40, 41, 44, 45, 46 or 47.

53. The anatomical region has the patient's ankle, The target anatomical structure has the talus, The at least one anatomical surface has at least one articular surface of at least one bone adjacent to the talus, the computer program product according to any one of claims 37, 38, 39, 40, 41, 44, 45, 46 or 47.

54. Determining the characteristic of the at least one anatomical surface is based on at least one selected from the group having manual indication by the user of the at least one anatomical surface provided based on user input to a graphical user interface that displays a model including at least the other anatomical structures of the patient, and automatic analysis of the image data for verifying the at least one anatomical surface, the computer program product according to any one of claims 37, 38, 39, 40, 41, 44, 45, 46 or 47.

Images