Systems and methods of using three-dimensional image reconstruction to aid in assessing bone or soft tissue aberrations for orthopedic surgery

Abstract

To provide systems and methods for enhancing pre-surgery and in-surgery imaging techniques for correctly modeling bone aberrations and another physiology when planning and executing an orthopedic surgery.SOLUTION: Systems and methods for calculating external bone loss for alignment of pre-diseased joints comprise: generating a three-dimensional ("3D") computer model of an operative area from at least two two-dimensional ("2D") radiographic images, wherein at least a first radiographic image is captured at a first position, and wherein at least a second radiographic image is captured at a second position, and wherein the first position is different than the second position; identifying an area of bone loss on the 3D computer model; and applying a surface adjustment algorithm to calculate an external missing bone surface fitting the area of bone loss.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] (Reference to related applications) This application claims the interests of U.S. Provisional Patent Application No. 63 / 217,567, filed on July 1, 2021. The disclosures of that related application are incorporated in their entirety into this disclosure.

[0002] (Field of invention) This disclosure relates, in general, to the field of orthopedic joint replacement surgery, and more specifically to the use of photogrammetry and three-dimensional reconstruction techniques to assist surgeons and technicians in planning and performing orthopedic surgery. [Background technology]

[0003] A new objective of joint replacement surgery is to restore the natural alignment and axis of rotation (one or more) of the joint as it existed before the onset of the condition. However, this objective can be difficult to achieve in practice because a joint consists not only of articular bone but also of auxiliary supporting bones and various soft tissues, including cartilage, ligaments, muscles, and tendons. Traditionally, surgeons have avoided fully restoring the natural alignment or have estimated alignment angles and other dimensions based on averages derived from population samples. However, these averages often fail to account for the natural variations in the anatomical structure of a particular patient, especially when the patient suffers from chronic bone deformity diseases such as osteoarthritis.

[0004] In an attempt to address this problem, some healthcare providers have begun using computed tomography (CT) scans and magnetic resonance imaging (MRI) techniques to help plan orthopedic surgery by examining the internal anatomical structure of patients. Data from these CT scans and MRIs have been further used to create three-dimensional (3D) models in digital form. These models can be sent to specialists to design and manufacture patient-specific implants and instruments for the surgery. Additive manufacturing techniques (e.g., 3D printing) and other conventional manufacturing techniques may be used to create physical implants or instruments that conform to the patient's specific anatomical structure.

[0005] However, obtaining CT scans and MRIs can be complex, time-consuming, and ineffective. CT scans also tend to expose patients to higher levels of radiation per session than they might otherwise receive using other non-invasive imaging techniques such as conventional radiography or ultrasound. Furthermore, scheduling considerations may necessitate CT or MRI scans being performed more than a month before the actual surgery. This delay can be exacerbated by the trend of gradually shifting orthopedic procedures to ambulatory surgical centers (ASCs), which are often smaller facilities lacking expensive on-site CT scanners and MRI machines. As a result, patients are often forced to schedule hospital-based examination appointments.

[0006] Increased time between the pre-operative examination and surgery increases the risk that the anatomical structure of the patient's bone and soft tissues may further deteriorate or change under normal use or due to disease progression. Further deterioration can not only cause further discomfort to the patient but also negatively impact the usefulness of the examination data for the surgical team. This can be particularly problematic with patient-specific implants fabricated from outdated data and surgical techniques that aim to restore range of motion based on the natural alignment of the joint before the onset of the disease. Furthermore, increased time between the pre-operative examination and surgery increases the likelihood that extrinsic events may negatively impact the data. For example, accidents such as dislocation or fracture in the planned surgical area usually impair the usefulness of the pre-operative examination data. Such risks may be higher in particularly active or particularly frail individuals.

[0007] Furthermore, not all patients have access to CT scans or MRIs to create patient-specific implants or devices. This may be partly due to the amount of time required to acquire the data, send it to medical device design specialists, generate a 3D model of the desired anatomical structure, create a patient-specific device or implant design based on the data or model, manufacture the patient-specific device or implant, ship and transport the device or implant to the surgical clinic, and sterilize the device or implant before treatment. Availability may also depend on the patient's medical insurance and the type of disease.

[0008] Knowing the precise amount of cartilage and bone loss in a patient can be useful in surgeries aimed at restoring the natural range of motion of the joint before the onset of the condition. Examples include major knee replacement surgeries (typically called "total knee arthroplasty" or "TKA"), total hip arthroplasty ("THA"), and procedures aimed at alleviating the causes of femoroacetabular impingements ("FAI").

[0009] Using the knee joint and TKA procedure as an example: A normal knee joint generally has an articular line (more specifically, a flexion-extension (FE) axis of rotation) that is approximately 2° to 3° varus relative to the mechanical medial-lateral (ML) line of the tibia. In anatomically aligned TKA procedures, the surgeon typically resects a portion of the patient's distal femoral condyle at approximately 3° valgus relative to the ML line of the femur, and then resects the tibia perpendicular to the longitudinal axis of the tibia, thereby resecting approximately 2° to 3° varus of the tibia's ML line. The surgeon then places and tests the components of the prosthesis in the resected area to assess the patient's range of motion and then adjusts them as needed.

[0010] However, the physiology of every patient is slightly different. For this reason, and due to extrinsic variability surrounding the survey data, many TKA surgeons opt for a more patient-specific kinematic alignment approach, using tools and procedures intended to identify the patient's pre-onset articular alignment during surgery. These tools tend to measure the thickness of the hyaline articular cartilage of un-weary or lightly-weary femoral condyles. Such tools tend to have a thickness gauge associated with the measuring end of the tool. The measuring end of the tool is typically inserted into the thickest area of ​​the cartilage of the lightly-weary condyle until the tip of the measuring end reaches the underlying bone. The surgeon then uses the thickness gauge to measure and record the amount of remaining cartilage. The surgeon then uses this measurement as an approximation of the amount of cartilage wear in the worn condyle.

[0011] However, this technology has several limitations. First, the remaining cartilage for accurate measurement may be insufficient in non-worn beads. Second, even when sufficient articular cartilage exists in non-worn beads, this cartilage measurement technique does not consider bone loss that occurs in worn beads. This problem can be even more severe when there is little or no remaining cartilage in adjacent beads. As a result, using existing intraoperative techniques, it is not possible to reliably measure the exact reduction amounts of both cartilage and bone in all kinematic alignment TKAs. Therefore, these techniques, combined with the problem and availability of accurate preoperative data, may endanger the accurate alignment of the artificial joint line and the natural pre-onset joint line. Repeated studies have shown that artificial joints that change the natural axis of rotation of the pre-onset joint tend to cause dysfunction, early implant wear, and patient dissatisfaction.

Prior Art Documents

Patent Documents

[0012]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0013] Therefore, there has long been an unsolved need to enhance preoperative and intraoperative imaging techniques for accurately modeling bone abnormalities and other physiologies when planning and performing orthopedic surgery.

Means for Solving the Problems

[0014] Problems with limited access to conventional preoperative CT and MRI imaging techniques, problems with data accuracy due to deterioration of bone and cartilage between preoperative imaging and surgery time, and problems with limitations in determining the natural joint line of the bone or joint before onset resulting from the use of currently available intraoperative tools and techniques are alleviated by a system and / or method for calculating the degree of bone abnormality, the system and / or method using a deep learning network to identify an area of bone abnormality from the input of at least two separate two-dimensional ("2D") input images of a target orthopedic element, wherein a first image of the at least two separate 2D input images is captured from a first lateral position and a second image of the at least two separate 2D input images is captured from a second lateral position offset by an offset angle from the first lateral position, and calculating a correction region, the correction region removing the area of bone abnormality.

[0015] In certain exemplary embodiments, it is contemplated that the first and second input images can be radiographic input images. Without being bound by theory, it is contemplated that radiographs enable in vivo analysis of the surgical area and can account for the external sum of passive soft tissue structures and dynamic external forces occurring around the surgical area, including the effects of ligament restraint, load-bearing forces, and muscle activity.

[0016] Certain embodiments according to the present disclosure are contemplated to be used to create patient-specific surgical plans, implants, and instruments from data derived from the cartilage and bone anatomical structures of the surgical area and / or data derived from the soft tissue structures of the surgical area.

Brief Description of the Drawings

[0017] The above will become apparent from the following more specific description, shown below, of exemplary embodiments of the present disclosure, as illustrated in the accompanying drawings. The drawings are not necessarily to scale, with emphasis instead being placed on showing the disclosed embodiments. [Figure 1]This is a simplified anterior view of the left knee joint, showing areas of negative bone abnormality (i.e., bone loss) in the medial and lateral femoral condyles. [Figure 2] This is a side view of a femoral resection guide positioning device having an adjustment pad that extends through a negative bone anomaly and is positioned on the worn bone exposed surface of the lateral femoral condyle. [Figure 3] This is a simplified anterior view of the left knee joint, visually representing the application of a surface adjustment algorithm to calculate the external bone defect surface that corrects for negative bone abnormalities. [Figure 4] This is a side view of a femoral resection guide positioning device having an adjustment pad extending to the calculated external bone defect surface on the lateral femoral condyle. [Figure 5] This is a simplified perspective view of the distal femur oriented in a flexed position. The posterior condyle resection guide positioning device is positioned on the posterior portion of the femoral condyle. [Figure 6] This is a perspective view of a femoral resection guide attached to a femoral resection guide positioning device. [Figure 7] This is a side view of a cartilage thickness gauge. [Figure 8] This is a flowchart illustrating the steps of an exemplary method. [Figure 9A] This is an image of the target orthopedic element taken from the AP position, illustrating an example calibration jig. [Figure 9B] This is an image of the target orthopedic element in Figure 8A, taken from the ML position, which shows an example calibration jig. [Figure 10] This is a schematic diagram of a pinhole camera model used to illustrate how the position of a point in 3D space can be determined from two 2D images taken from different reference frames of a calibrated image detector, using the principle of epipolar geometry. [Figure 11] This is a schematic diagram of a system that uses a deep learning network to identify features (e.g., anatomical landmarks) of a target orthopedic element, including bone abnormalities, and to generate a 3D model of that orthopedic element. [Figure 12] These are schematic perspective views of two 3D reconstruction models: the distal aspect of the femur and the inverse volume of the identified negative bone anomaly. [Figure 13] A flowchart illustrating the steps of another exemplary method. [Figure 14] This is a flowchart illustrating the steps of yet another exemplary method. [Figure 15] This is a flowchart illustrating the process of yet another exemplary method. [Figure 16] This is a schematic diagram of a system configured to generate a physical model of a bone anomaly, which is derived from using two or more tissue-penetrating, flattened input images taken at an offset angle for the same orthopedic element of interest from a calibrated detector. [Figure 17] This is a schematic diagram illustrating how a CNN-type deep learning network can be used to identify features (e.g., anatomical landmarks) including bone abnormalities in target orthopedic elements. [Figure 18] This is a schematic diagram of an example system. [Modes for carrying out the invention]

[0018] The following detailed description of preferred embodiments is provided for illustrative purposes only to aid understanding and is not intended to be exhaustive or to limit the scope and spirit of the invention. The embodiments have been selected and described to best illustrate the principles and practical applications of the invention. Those skilled in the art will recognize that many modifications can be made to the invention disclosed herein without departing from the scope and spirit of the invention.

[0019] Unless otherwise specified, similar reference numerals indicate corresponding parts in several drawings. The drawings illustrate embodiments of various features and components of the present disclosure, but the drawings are not necessarily to scale, and certain features may be exaggerated to better illustrate embodiments of the present disclosure, and such examples should not be construed as limiting the scope of the present disclosure.

[0020] Unless otherwise expressly stated herein, the following rules of interpretation shall apply herein: (a) All words used herein shall be construed as having the gender or number (singular or plural) required in such context. (b) The singular terms “a,” “an,” and “the” used herein and in the appended claims shall include plural references unless otherwise clearly stated in context. (c) The antecedent “about” applied to any enumerated range or value shall indicate an approximation of a range or value having deviations from known in the art or expected from measurement. (d) Unless otherwise explicitly stated, the words “herein, hereby, hereto,” “hereinbefore,” and “hereinafter,” and words of similar meaning, refer to the entire herein and not to any particular paragraph, claim, or other detail. (e) Explanatory headings are for convenience only and do not control or affect the meaning of any part of this specification. (f) “or” and “any” are not exclusive, and “include” and “including” are not restrictive. Furthermore, "comprising," "having," "including," and "containing" should be interpreted as unrestricted terms (i.e., meaning "including but not limited to...").

[0021] References in this specification such as "one embodiment" or "exemplary embodiment" indicate that the described embodiments may include certain features, structures, or characteristics, but not all embodiments may necessarily include such features, structures, or characteristics. Furthermore, such phrases do not necessarily refer to the same embodiment. Moreover, where certain features, structures, or characteristics are described in relation to an embodiment, it is implied that any influence on such features, structures, or characteristics in relation to other embodiments, whether explicitly stated or not, is within the knowledge of those skilled in the art.

[0022] To the extent necessary to provide descriptive support, the subject matter and / or text of the attached claims are incorporated herein by reference in their entirety.

[0023] The enumeration of value ranges in this specification is merely intended to serve as a simple means of individually referring to each distinct value within any subrange between them, unless otherwise explicitly indicated herein. Each distinct value within an enumerated range is incorporated herein or into the claims as if each distinct value were individually enumerated herein. Where a particular range of values ​​is provided, each intervening value up to one-tenth of the lower limit between the upper and lower limits of that range, and any other stated or intervening values ​​within the described range of that subrange, are understood to be included herein unless the context explicitly indicates otherwise. All subranges are also included. The upper and lower limits of these smaller ranges are also included therein, subject to any specific and explicitly excluded limitations within the described range.

[0024] It should be noted that some of the terms used herein are relative. For example, the terms “upper” and “lower” are spatially relative to each other; that is, the upper components are located higher than the lower components in their respective orientations, but these terms can change if the orientation is reversed. The terms “inlet” and “outlet” refer to the fluid flowing through a given structure; for example, the fluid flows into the structure through the inlet and then flows out of the structure through the outlet. The terms “upstream” and “downstream” refer to the direction in which the fluid flows through various components before flowing through the downstream components.

[0025] The terms “horizontal” and “vertical” are used to indicate direction relative to an absolute reference point, i.e., the Earth’s surface level. However, these terms should not be interpreted as requiring structures to be absolutely parallel or absolutely perpendicular to each other. For example, a first vertical structure and a second vertical structure are not necessarily parallel to each other. The terms “upper” and “lower” or “base” are used to refer to a place or surface where the upper is always higher relative to an absolute reference point, i.e., the Earth’s surface, than the lower or base. The terms “upward” and “downward” are also relative to an absolute reference point. An upward flow always opposes Earth’s gravity.

[0026] Throughout this disclosure, various positional terms such as “distal,” “proximal,” “medial,” “lateral,” “anterior,” and “posterior” are used in a conventional manner when referring to human anatomical structures. More specifically, “distal” refers to a region away from the attachment point to the main body, while “proximal” refers to a region near the attachment point to the main body. For example, the distal femur refers to the portion of the femur near the tibia, and the proximal femur refers to the portion of the femur near the hip joint. The terms “medial” and “lateral” are also essentially opposite. “Medial” refers to something located closer to the center of the main body. “Lateral” means that something is located to the right or left of the main body rather than to the center. With regard to “anterior” and “posterior,” “anterior” refers to something located closer to the front of the main body, while “posterior” refers to something located closer to the back of the main body.

[0027] "Vasculus" and "valgus" are broad terms and not limited to any particular movement, but they include rotational movements in the medial and / or lateral directions relative to the knee joint.

[0028] As used herein, the term “bone abnormality” should be understood to mean an area of ​​external or internal bone loss, an area of ​​abnormal supernumerary bone (such as bone proliferators (i.e., osteophytes)), or any other area of ​​bone that is discontinuous with the natural area of ​​surrounding bone.

[0029] All methods for identifying areas of bone abnormality for the purpose of calculating a correction area are considered to be within the scope of this disclosure, and the correction area removes the area of ​​bone abnormality from the surrounding bone area. As an example, the following sections describe exemplary embodiments of systems and methods used to restore the natural joint line and axis of rotation of the knee joint before the onset of the condition in total knee arthroplasty ("TKA").

[0030] A general description of a major total knee arthroplasty (TKA) is as follows: The surgeon typically initiates the surgery by making a nearly vertical medial patellar incision on the anterior or anteromedial side of the knee to be operated on. The surgeon then proceeds to incise the adipose tissue to expose the joint capsule. The surgeon may then perform a medial patellar arthotomy to perforate the joint capsule. The patella may then be moved nearly laterally using a retractor to expose the distal femoral condyles (103 and 107, see Figure 1) and the cartilaginous meniscus (generally 112, see Figure 1) resting on the proximal tibia plateau. The surgeon then removes the meniscus and, using measuring instruments, measures and resects the distal femur 105 and proximal tibia 110 to accommodate a test implant. A test implant is a test internal prosthesis that generally has the same functional dimensions as an actual internal prosthesis, but is designed to be temporarily inserted and removed for the purpose of evaluating the fit of the actual internal prosthesis and for kinematic evaluation of the knee joint. Once inserted, test implants and actual internal prosthesis implants are generally positioned adjacent to these resection sites. Therefore, the position and orientation of these femoral and tibial resections significantly influence the position and orientation of the test implant and actual internal prosthesis implant, and consequently, the reconstructed articular line.

[0031] Next, this tibial resection is performed. Once resected, the resected area of ​​the tibia may be recognized as a "tibial plateau." The surgeon may then place a test tibial component on the resected proximal tibial plateau. Generally, the surgeon measures and resectes the distal femoral condyle using different instruments for the purpose of attaching a test femoral component. If the test component does not seat properly, the surgeon uses further instruments to measure and resecte the femoral condyle and / or tibial plateau until the desired seating is achieved.

[0032] Next, the surgeon typically inserts a test meniscus insert between the test tibial tray and the test femoral component to test knee flexion and extension, overall stability, and patellar tracking with the test implant. Once the test movement characteristics are met, the surgeon may, if necessary, permanently attach the actual tibial and femoral components of the internal prosthesis implant using bone cement, or avoid the use of bone cement by using a press-fit implant.

[0033] The latest alignment philosophy is the kinematic alignment principle. The kinematic alignment principle recognizes that every patient's physiology is slightly different and aims to restore the patient's natural joint line before the onset of the condition by performing actual measurements of surgical physiology to confirm the position of the natural joint line. Once these measurements are known, a tool such as a swivel femoral resection guide locator 400 (Figures 2, 4, and 6) or resection guide 424 (Figure 6) is then positioned over the exposed bone. The resection guide 424 may be customized to complement the exposed bone, or it may be selectively locked onto the femoral resection guide locator 400. These femoral resection guide locators 400 may have an adjustable positioning mechanism (see 440) that allows the surgeon to adjust the orientation of the resection guide 424 relative to the exposed bone based on the patient's specific measurements. Once the resection guide 424 is set to the desired orientation, it is then temporarily fixed to the bone. Next, the surgeon inserts a surgical saw into the resection slot 455 of the oriented resection guide 424 to resect the underlying (undying) bone at the desired resection plane. The position and orientation of the resection guide 424 largely determines the position and orientation of the rotation axis of the reconstructed joint, as the position and orientation of the resection guide 424 greatly influences the orientation of the experimental implant and the actual internal prosthesis implant.

[0034] While the methods and systems described herein are intended to be particularly useful for kinematic alignment, this disclosure does not limit the use of the systems and methods described herein to kinematic alignment. For example, the systems and methods described herein may be used in conjunction with anatomical alignment, mechanical alignment, or any other alignment method, provided that any existing bone abnormalities affect the positioning of the alignment device (see excision guide locator 400 and alignment guide 600). Furthermore, this disclosure does not limit the exemplary systems and methods described herein to use in the knee joint. Any orthopedic surgery in which it is desirable for the surgeon to have preoperative or intraoperative knowledge of bone or soft tissue abnormalities is considered to be within the scope of this disclosure. Examples of such orthopedic surgeries include, but are not limited to, hip arthroplasty and procedures aimed at alleviating the causes of femoroacetabular impingement.

[0035] Figure 1 is a front view of a simplified left knee joint (i.e., an exemplary set of subject orthopedic elements 100 in an exemplary surgical area 170). The examples described with reference to Figures 1-5 relate to an exemplary knee joint for illustrative purposes. It will be understood that “orthopedic elements” 100, as referred to throughout this disclosure, may include any skeletal structures and associated soft tissues, such as tendons, ligaments, cartilage, and muscles, but are not limited to the anatomical structures of the knee joint. An indefinite list of exemplary orthopedic elements 100 includes any partial or complete bone of the body, including but not limited to the femur, tibia, pelvis, vertebrae, humerus, ulna, radius, scapula, skull, fibula, clavicle, mandible, ribs, carpal bones, metacarpal bones, metatarsal bones, phalanges, or any associated tendons, ligaments, skin, cartilage, or muscles. It will be understood that the exemplary surgical area 170 may include several subject orthopedic elements 100.

[0036] The exemplary orthopedic element 100 shown in Figure 1 is the distal portion of the femur 105, the proximal portion of the tibia 110, the proximal portion of the fibula 11, the medial collateral ligament ("MCL") 113, the lateral collateral ligament ("LCL") 122, and the articular cartilage 123 positioned over the distal femoral condyles 107 and 103. The area of ​​bone abnormality (generally, 115) is shown above the distal femoral condyles 107 and 103. The medial area of ​​bone abnormality 115a is shown within the medial condyle 107, and the lateral area of ​​bone abnormality 115b is shown within the lateral condyle 103 of the femur 105 (collectively, the "distal femoral condyle"). In Figure 1, the areas of bone abnormality 115a and 115b are "negative bone abnormalities," i.e., areas of bone loss. Figure 1 shows the medial and lateral condyles 107 and 103 of the distal femur 105, positioned on the tibial plateau 112 in the proximal aspect of the tibia 110. Medial M, the MCL 113 engages the distal femur 105 with the proximal tibia 110. Similarly, lateral L, the LCL 122 engages the distal femur 105 with the femur 111. The femorotibial space 120 separates the distal femur 105 from the tibial plateau 112. Hyalinian articular cartilage 123 is shown around the areas of bone abnormality 115a and 115b on the distal femur 105.

[0037] The natural pre-symptomatic location of the articular line is largely determined by the interaction between the soft tissue (e.g., articular cartilage 123) on the femoral condyles 107, 103 and the meniscus, which is supported by the underlying bone (e.g., tibia 110). If areas of bone abnormality 115a, 115b (e.g., areas of bone loss shown in Figure 1) are not included, knowing the thickness of the pre-symptomatic cartilage 123 can be used to closely approximate the location of the pre-symptomatic articular line.

[0038] Figure 7 shows an exemplary cartilage thickness gauge 40. The illustrated thickness gauge 40 includes an elongated handle portion 52 that can be used by a surgeon to hold and manipulate the instrument. A shoulder portion 62 engages the shaft portion 60 with the handle portion 52. The shaft portion 60 includes a proximal solid portion 63 and a distal hollow portion 64. The hollow portion 64 extends to the measuring end 41. The hollow portion 64 receives a piston 80. The piston 80 is located inside the hollow portion 64 and is biased against a spring 79 located between the piston 80 and the solid portion 63 of the shaft portion 60. Markers (not shown) on the piston 80 can be viewed through a viewing portal in the shaft portion 60. These markers are preferably placed at constant increments, such as 1 millimeter ("mm") increments. A reference line may be located adjacent to the viewing portal. Similarly, markers visible through the viewing portal move relative to the reference line.

[0039] All methods for evaluating cartilage wear are considered to be within the scope of this disclosure. One exemplary method of performing kinematic alignment techniques using this exemplary cartilage thickness gauge 40 (Figure 7) and resection guide is further described in U.S. Patent Application No. 16 / 258,340. U.S. Patent Application No. 16 / 258,340 in its entirety is incorporated herein by reference. The method disclosed herein uses the cartilage thickness gauge 40 to measure the thickness T (Figure 1) of hyaline articular cartilage 123 (Figure 1) adjacent to an unworn or lightly worn femoral condyle (i.e., 107 or 103). The measuring end 41 of the cartilage thickness gauge 40 is inserted into the thickest region of the cartilage 123 adjacent to the lightly worn condyle until the tip 86 of the measuring end 41 reaches the underlying bone 106 (Figure 1). In this way, the tip 86 of the measuring end 41 is pushed into the thickest region of the remaining proximal cartilage 123 so that the piston 80 compresses the spring 79. Simultaneously, a marker on the piston 80 moves relative to the baseline and becomes visible through the viewing portal, thereby indicating the thickness of the cartilage 123 located beneath the measuring end 41. The surgeon then records the amount of remaining cartilage (i.e., cartilage thickness T) and uses this measurement as an approximation of the amount of cartilage wear in the worn condyle. This process is preferably repeated for each distal femoral condyle 103, 107 and each posterior femoral condyle (107a, 103a, Figure 5).

[0040] As a different example for evaluating the thickness of cartilage 123, a surgeon or technician can use an intraoperative probe to map the location and physical properties of soft tissue, such as its elasticity and density. The system can then use this data to calculate the amount of cartilage wear across the condyle.

[0041] Referring here to Figure 2, at any point after exposure of the distal femur 105, the surgeon may perforate an intramedullary canal approximately in the center of the distal femur 105 and then insert an intramedullary rod 420 (see also Figure 6) into the emptied intramedullary canal to provide a base for selectively positioning reference instruments relative to the distal femur 105. Once the intramedullary rod 420 is firmly seated, the stabilizing portion 451 of the swivel femoral resection guide locator 400 may slide toward the intramedullary rod 420, thereby positioning the adjustment pads 440A and 440B adjacent to the medial distal femoral condyle 107 and the lateral distal femoral condyle 103, respectively.

[0042] The stabilizing portion 451 may be an intramedullary rod holder member or another device configured to be fixed in a fixed position relative to the pivoting main body portion 412. The main body portion 412 is configured to pivot relative to the stabilizing portion 451. The pin 411 (Figure 6) may be positioned in close contact with annular holes aligned within the stabilizing portion 451 and the main body portion 412, respectively, so that the main body portion 412 of the pivoting femoral resection guide locator 400 can be said to be "configured to pivot" relative to the stabilizing portion 451, or to be "in a pivotal relationship" with the stabilizing portion 451.

[0043] Next, the adjustment pads 440A and 440B (Figure 6) extend from the distal reference plane 485 of the resection guide locator 400 and contact the femoral condyles 103 and 107. The length l of each adjustment pad 440 relative to the reference plane 485 is preferably the same length l as the measured thickness T of the associated adjacent hyaline articular cartilage 123 of each condyle.

[0044] For example, if there is 2 mm of cartilage wear on the medial side M and 0 mm on the lateral side L, the surgeon extends the medial adjustment pad 440A by 2 mm. The surgeon then positions each adjustment pad 440 on the appropriate condyle. In this example, the lateral adjustment pad 440B remains approximately coplanar with the distal reference plane 485. This sets the position of the distal reference plane 485 relative to the intramedullary rod 420, which determines the resection angle. The resection guide 424 (Figure 6) is then fixed to the femur 105 at the desired resection angle, the femoral resection guide locator 400 is removed, and the surgeon resects the distal femoral condyles 103, 107 through the resection slot 455 at the desired resection angle.

[0045] However, these adjustments only consider the existing cartilage wear. These adjustments do not consider areas of bone abnormalities, such as bone loss, that may occur in the worn condyles (e.g., the lateral condyle 103 or the medial condyle 107). The adjustment pads 440A and 440B are typically set based on measurements from the cartilage thickness gauge 40. If the patient has significant bone loss or significant bone proliferators (commonly referred to as "osteophytes"), the distal reference plane 485 of the femoral resection guide locator 400 is not positioned at the exact location of the pre-onset articular surface. Furthermore, if the surgeon extends the adjustment pad 440 to the remaining area of ​​bone (as shown in Figure 2), it is impossible to know precisely where the pre-onset bone ends and where the articular cartilage 123 begins. Therefore, even if the amount of cartilage wear can be reasonably approximated by measuring the thickest area of ​​the adjacent, unworn cartilage 123, it is impossible to know precisely the depth of the bone defect. Therefore, any adjustment of the adjustment pad 440 for contacting the condyle having an area of ​​significant bone loss (and consequently any adjustment of the position of the distal reference plane 485 and the resection guide 424) should be, at best, an approximation of the pre-symptomatic articular surface. Insufficient precision in this area carries the risk of miscalculating the orientation of the joint's natural pre-symptomatic axis of rotation. Consequently, any estimation error can adversely affect patient comfort and the lifespan of the implant.

[0046] As a result, even with the use of the swivel femoral resection guide locator 400 described in U.S. Patent Application No. 16 / 258,340 and existing intraoperative techniques, it is not possible to reliably measure the exact amount of cartilage and bone loss in all TKAs. Therefore, these techniques, coupled with the problems and availability of accurate preoperative data, may impair the accurate reconstruction of the prosthesis line using the natural pre-onset articular line.

[0047] To illustrate this, Figure 2 shows a side view of a femoral resection guide locator 400 having an adjustment pad 440B that extends through a negative region 115b of bone abnormality, positioned on the exposed surface of the worn bone 116 of the lateral condyle 103. The extended length l of the pad adjuster 440 relative to the distal reference plane 485 is first determined by a measurement of cartilage thickness (see 123), which can be obtained using a cartilage thickness gauge 40 as described above, or by other cartilage thickness measurement methods. In the case of femoral condyles 103, 107 suffering from osteopenia, it was previously impossible to reliably determine the amount of bone abnormality (e.g., osteopenia in this case) using the femoral resection guide locator 400 together with pad adjusters 440A, 440B.

[0048] For example, the precise depth of negative bone anomaly (i.e., bone loss) in the lateral condyle 103 cannot be easily determined using conventional methods. When the pad adjuster 440B is extended to the measured cartilage thickness and its end is placed on the exposed surface of the worn bone 116, the pre-onset articular surface is no longer shown at that point, and the depth of the negative bone anomaly is not properly considered, which risks misalignment of the swivel femoral resection guide locator 400. In other words, if the surgeon first sets the extension length l of the illustrated pad adjuster 440B to 2 mm from the distal reference plane 485 (for each chondrometer measurement), and then positions the pad adjuster 440 so that it contacts the exposed surface of the worn bone 116, the pad adjuster extends into the area of ​​bone loss 115b by an unknown depth of the negative bone anomaly 115b, thereby changing the angle of the distal reference plane 485 (and consequently the cutting plane) relative to the intramedullary rod 420. This new cutting angle is not useful for kinematically aligning the knee because it does not reflect the pre-onset articular surface.

[0049] Surgeons may attempt to compensate for the lack of depth of bone loss by adding length (see l) to the pad adjuster 440, but the exact amount of loss is unknown to the surgeon. Therefore, surgeons may overestimate or underestimate the amount of loss, thereby risking misalignment of the femoral resection guide locator 400 relative to the actual articular surface of the joint before the onset. Kinematic alignment is the technique of referencing the femoral articular surface. Therefore, errors made in referencing the articular surface, if not corrected, are transmitted to the proximal tibia, thereby potentially further exacerbating the initial error. Furthermore, as shown in Figure 2, the exposed surfaces of the bone anomaly area 115b and the worn bone 116 may not have uniform depth. This can impair the initial stability of the angled position of the resection guide locator relative to the intramedullary rod 420.

[0050] In recent years, it has become possible to create 3D models of surgical areas using multiple 2D images, such as X-ray images from imaging systems. These models can be used preoperatively to plan surgeries much closer to the actual surgery date. Furthermore, these 3D models can be generated intraoperatively to check the preoperative model and plan, or these 3D models can function as native models from which areas of bone abnormalities can be calculated. However, X-ray images have not been previously used as input for 3D models, typically due to concerns about image resolution and accuracy. An X-ray image is a 2D representation of a 3D space. Therefore, in a 2D X-ray image, the image object is always distorted compared to the actual object that exists in three dimensions. Furthermore, the object through which the X-rays pass can deflect the path of the X-rays as they travel from the X-ray source (typically the anode of an X-ray machine) to the X-ray detector (which may include, as non-limiting examples, X-ray image intensifiers, phosphorus materials, flat panel detectors (FPDs) (including indirect and direct conversion FPDs), or any number of digital or analog X-ray sensors or X-ray films). Defects in the X-ray machine itself or in its calibration can also impair the usefulness of X-ray photogrammetry and 3D model reconstruction. In addition, emitted X-ray photons have different energies. When X-rays interact with material placed between the X-ray source and the detector, noise and artifacts may be generated in part due to Compton scattering and Rayleigh scattering, the photoelectric effect, extrinsic variations in the environment, or intrinsic variations in the X-ray generation unit, X-ray detector, and / or processing unit or display.

[0051] Furthermore, a single 2D image loses the actual 3D data of the object. Therefore, there is no data from a single 2D image that a computer can use to reconstruct a 3D model of an actual 3D object. For this reason, CT scans, MRI, and other imaging techniques that preserve three-dimensional data have often been preferred inputs for reconstructing models of one or more orthopedic elements of an object (i.e., reconstructing 3D models from actual 3D data to generally produce more accurate and higher-resolution models). However, certain exemplary embodiments of the present disclosure, discussed below, overcome these problems by using a deep learning network to improve the accuracy of the reconstructed 3D model generated from radiographic photogrammetry and by identifying areas of bone abnormalities on or within the reconstructed 3D model. In certain exemplary embodiments, areas of bone abnormalities can be corrected or eliminated using a curve fitting algorithm (i.e., an exemplary surface adjustment algorithm).

[0052] An exemplary method for calculating the degree of bone abnormality may include generating a 3D model of a surgical region 170 from at least two 2D images, wherein the first image is captured at a first lateral position and the second image is captured at a second lateral position, the first lateral position being different from the second lateral position; identifying the region of bone abnormality on the 3D model; and calculating a correction region, the correction region being the region of bone abnormality removed from the 3D model (i.e., relative to the surrounding bone region).

[0053] An exemplary system for calculating the degree of bone abnormality is a radiographer 1800 (Figure 18) including a radiator 21 and a detector 33, wherein the detector 33 of the radiographer 1800 captures a first image 30 (Figures 9 and 10) at a first lateral position 30a (Figures 9 and 10) and a second image 50 (Figures 9 and 10) at a second lateral position 50a (Figures 9 and 10), and the first lateral position 30a is offset by an offset angle θ (Figure 10) from the second lateral position 50a. The system may include a radiation imager 1800, a transmitter 29 (Figure 18), and a computer 1600, wherein the transmitter 29 transmits a first image 30 and a second image 50 from a detector 33 to the computer 1600, and the computer 1600 is configured to identify a region of bone abnormality 115 in a 3D model 1100 of the target orthopedic element and calculate a correction region, the correction region being the region of bone abnormality removed from the 3D model 1100 of the target orthopedic element (i.e., relative to the surrounding bone region).

[0054] In certain exemplary embodiments, the exemplary system may further include a display 19.

[0055] In certain exemplary embodiments, the exemplary system may further comprise a manufacturing machine 18. In exemplary embodiments comprising a manufacturing machine 18, the manufacturing machine 18 may be an additive manufacturing machine. In such embodiments, the additive manufacturing machine may be used to manufacture a 3D model 1100 of an orthopedic element of interest or a 3D model 115m of a bone anomaly.

[0056] While radiographs from radiography systems may be desirable because radiographs are relatively inexpensive compared to CT scans, and because some radiography systems, such as fluoroscopy systems, are generally compact enough for intraoperative use, this disclosure does not limit the use of 2D images to radiographs unless otherwise expressly claimed, and nothing in this disclosure limits the type of imaging system to radiography systems. Other 2D images may include, for example, CT images, CT fluoroscopy images, fluoroscopy images, ultrasound images, positron emission tomography (PET) images, and MRI images. Other imaging systems may include, for example, CT, CT fluoroscopy, fluoroscopy, ultrasound, PET, and MRI systems.

[0057] Preferably, the exemplary method may be implemented on a computer platform (e.g., computer 1600) having hardware such as one or more central processing units (CPUs), random access memory (RAM), and / or input / output (I / O) interfaces. An example of the architecture of the exemplary computer 1600 is provided below with reference to Figure 16.

[0058] In certain exemplary embodiments, the 3D model 1100 of the orthopedic element in question and / or the 3D model 115m of the bone anomaly may be computer models. In other exemplary embodiments, the 3D model 1100 of the orthopedic element in question and / or the 3D model 115m of the bone anomaly may be physical models.

[0059] There are various methods for generating a 3D model from 2D preoperative or intraoperative images. For example, one such method may involve receiving a set of 2D radiographic images of the patient's surgical area 170 using a radiographic imaging system, and then computing a first 3D model using the epipolar geometry principle with the coordinate system of the radiographic imaging system and projective geometric data from each 2D image (see Figures 9 and 10). Such an exemplary method may further involve projecting the first 3D model onto the 2D radiographic images, and then adjusting the initial 3D model by aligning first and second radiographic images 30, 50 onto the first 3D model using an image-to-image alignment technique. Once the image-to-image alignment technique is applied, a modified 3D model may be generated. This process may be repeated until the desired clarity is achieved.

[0060] Another example is a deep learning network such as a convolutional neural network (CNN), a recurrent neural network (RNN), a modular neural network, or a Sequence to Sequence model ("deep neural network"). A deep learning network (also known as a DNN) can generate 3D models 1100 of target orthopedic elements and / or 3D models 115m of bone anomalies from a set of at least two 2D images of a patient's surgical region 170 and can be used to identify regions of bone anomalies 115. The 2D images are preferably trans-tissue images such as radiographic images (e.g., X-ray images or fluoroscopic images). In such a method, the deep learning network can generate models from projective geometric data (i.e., spatial data 43 or volumetric data 75) from each 2D image. The deep learning network may have the advantage of being able to generate masks of different target orthopedic elements 100 (e.g., bones) or bone anomalies 115 in the surgical region 170, as well as being able to calculate the volume of one or more imaged orthopedic elements 100 (see Figures 11, 61).

[0061] Figure 8 is a flowchart outlining the steps of an exemplary method that uses a deep learning network to identify a region of bone abnormality 115 on or within an imaged orthopedic element 100 using two flattened input images (30, 50, Figures 9 and 10) taken from an offset angle θ. The exemplary method involves calibrating the imager 1800 (Figure 18) to image point (X L , e L , X R , e R The process includes step 1a, which involves determining a mapping relationship between the image (Figure 10) and the corresponding spatial coordinates (e.g., Cartesian coordinates on the x,y plane) to define the spatial data 43. The imaging device 1800 is preferably a radiographic imaging device capable of generating X-ray images ("X-ray images" may be understood to include fluoroscopic images), but all medical imaging devices are considered to be within the scope of this disclosure.

[0062] Step 2a includes capturing a first image 30 (Figure 9) of the target orthopedic element 100 using an imaging technique (e.g., X-ray imaging, CT imaging, MRI imaging, or ultrasound imaging), the first image 30 defining a first reference frame 30a. In step 3a, a second image 50 (Figure 9) of the target orthopedic element 100 is captured using an imaging technique, the second image 50 defining a second reference frame 50a, and the first reference frame 30a is offset from the second reference frame 50a by an offset angle θ. The first image 30 and the second image 50 are input images from which data (including spatial data 43) can be extracted. In other exemplary embodiments, it will be understood that two or more images may be used. In such embodiments, each input image is preferably separated from other input images by an offset angle θ. Step 4a includes projecting spatial data 43 from a first image 30 of the target orthopedic element 100 and spatial data 43 from a second image 50 of the target orthopedic element 100 to define volume data 75 (Figure 11) using epipolar geometry.

[0063] Step 5a includes detecting bone abnormalities 115 from volume data 75 of the orthopedic element 100 using a deep learning network. Step 6a includes defining a 3D model 1100 of the target orthopedic element by detecting other features (e.g., anatomical landmarks) from the volume data 75 of the target orthopedic element 100 using a deep learning network. Step 7a includes removing the detected bone abnormalities 115 from the 3D model 1100 of the target orthopedic element by applying a surface adjustment algorithm.

[0064] In certain exemplary embodiments, the deep learning network that detects bone abnormalities 115 from volumetric data 75 may be the same deep learning network that detects other features from the volumetric data 75 of the target orthopedic element 100. In other exemplary embodiments, the deep learning network that detects bone abnormalities 115 from volumetric data 75 may be different from the deep learning network that detects other features from the volumetric data 75 of the target orthopedic element 100.

[0065] In certain exemplary embodiments, the first image 30 may show the target orthopedic element 100 in an outward lateral position (i.e., the first image 30 is a side view of the orthopedic element 100). In other exemplary embodiments, the second image 50 may show the orthopedic element 100 in an anterior-posterior ("AP") lateral position (i.e., the second image 50 is an AP view of the orthopedic element 100). In yet another exemplary embodiment, the first image 30 may show the orthopedic element 100 in an AP lateral position. In yet another exemplary embodiment, the second image 50 may show the orthopedic element 100 in an outward lateral position. In yet another exemplary embodiment, neither the first image 30 nor the second image 50 can show the orthopedic element 100 in an AP lateral position or an outward lateral position, as long as the first image 30 is offset from the second image 50 by an offset angle θ. Computer 1600 can calculate the offset angle θ from input images 30 and 50, including calibration fixtures (see 973, Figure 9). The first image 30 and the second image 50 may be collectively referred to as "input images" or individually as "input images". These input images 30 and 50 preferably show the same orthopedic element 100 from different angles. These input images 30 and 50 may be taken along the lateral plane of the orthopedic element 100.

[0066] Certain exemplary methods may further include using a style-transferring deep learning network such as CycleGAN. A method using a style-transferring deep learning network may start with a radiographic input image (e.g., 30) and use a style-transferring neural network to transfer the style of the input image to a DRR type image. Further exemplary methods may include using a deep learning network to identify features (e.g., anatomical landmarks) of the target orthopedic element 100 or bone anomaly 115 and providing a segmentation mask for each target orthopedic element 100 or bone anomaly 115.

[0067] Figures 10 and 11 show a mechanism by which a volume 61 including volume data 75 can be created by combining a first input image 30 and a second input image 50. FIG. 10 shows the basic principle of epipolar geometry that can be used to convert spatial data 43 from each input image 30, 50 into volume data 75 (FIG. 11). The spatial data 43 is understood to be defined by a set of image points (e.g., X L , X R ) that are mapped to corresponding spatial coordinates (e.g., x and y coordinates) of a given input image 30, 50.

[0068] FIG. 10 is a simplified schematic diagram of perspective projection described by the pinhole camera model. Although FIG. 10 conveys basic concepts related to computer stereo vision, it does not mean the only way a 3D model can be reconstructed from 2D stereo images. In this simplified model, light rays are emitted from the optical center (i.e., the point within the lens where it is assumed that light rays of electromagnetic radiation from the object (e.g., visible light, X-rays, etc.) intersect within the sensor or detector array 33 (FIG. 18) of the imager). The optical center is represented by points O L , O R in FIG. 10. In reality, the image planes (see 30a, 50a) are usually behind the optical center (e.g., O L , O R ), and the actual optical center is projected as a point onto the detector array 33, but a virtual image plane (see 30a, 50a) is presented here for a more concise illustration of the principle.

[0069] The first input image 30 is taken from a first reference frame 30a, and the second input image 50 is taken from a second reference frame 50a that is different from the first reference frame 30a. Each image includes a matrix of pixel values. The first and second reference frames 30a, 50a are preferably offset from each other by an offset angle θ. The offset angle θ can represent the angle between the x-axis of the first reference frame 30a and the x-axis of the second reference frame 50a. Point e LThe optical center O of the second input image on the first input image 30 R This is the location of point e. R The optical center O of the first input image on the second input image 50 is L This is the location of point e. L This is known as the "epipol" or epipolar point, and is located on line O L -O R It is located above. Point X, O L , O R This defines the epipolar plane.

[0070] The actual optical center is the point where the incident rays of electromagnetic radiation from the object are expected to intersect within the detector lens. Therefore, in this model, the electromagnetic radiation rays are actually located at the optical center O for the purpose of visualizing how the position of a 3D point X in 3D space can be determined from two or more input images 30, 50 captured from detectors 33 at known relative positions. L , O R It can be thought that it is emitted from each point of the first input image 30 (for example, X L If ) corresponds to a line in 3D space, then the corresponding point (for example, X R If these corresponding points (e.g., X) can be detected in the second input image, then L , X R ) must be a projection of a common 3D point X. Therefore, the corresponding image point (e.g., X) L , X R The lines generated by ) must intersect at 3D point X. Generally, the value of X must be such that all corresponding image points in two or more input images 30, 50 (for example, X L , X R When calculated for ), the 3D volume 61 containing volume data 75 can be replicated from two or more input images 30, 50. The value of any given 3D point X can be triangulated in various ways. A non-exclusive list of exemplary calculation methods includes the midpoint method, direct linear transformation method, elementary matrix method, intersection method, and bundle adjustment method.

[0071] "Image points" as described herein (for example, X L , XR It will be understood that a 3D point X can refer to a point in space, a pixel, a portion of a pixel, or a set of adjacent pixels. When used herein, it will also be understood that a 3D point X can represent a point in 3D space. In certain exemplary uses, a 3D point X can be represented as a voxel, a portion of a voxel, or a set of adjacent voxels.

[0072] However, before the principle of epipolar geometry can be applied, the position of each image detector 33 relative to other image detectors 33 must be determined (or, the position of a single image detector 33 must be determined when the first image 30 is taken, and the adjusted position of the single image detector 33 should be recognized when the second image 50 is taken). It is also desirable to determine the focal length and optical center of the imager 1800. To actually verify this, the image detector 33 (or multiple image detectors) are first calibrated. Figures 9A and 9B show calibration fixtures 973A and 973B for the target orthopedic element 100. In these figures, the exemplary orthopedic element 100 is the distal aspect of the femur 105 and the proximal aspect of the tibia 110 including the knee joint. The proximal fibula 111 in Figures 9A and 9B is another orthopedic element 100 that has been imaged. The patella 901 shown in Figure 9B is another orthopedic element 100.

[0073] Figure 9A is a front-to-back view of an exemplary orthopedic element 100 (i.e., Figure 9A represents a first image 30 taken from a first reference frame 30a). The first calibration fixture 973A is attached to the first retaining assembly 974A. The first retaining assembly 974A may include a first padded support 971A engaged with a first strap 977A. The first padded support 971A is attached to the outside of the patient's thigh via the first strap 977A. The first retaining assembly 974A supports the first calibration fixture 973A, preferably oriented parallel to the first reference frame 30a (i.e., perpendicular to the detector 33). Similarly, a second calibration fixture 973B may be provided attached to a second retaining assembly 974B. The second retaining assembly 974B may include a second padded support 971B engaged with a second strap 977B. The second padded support 971B is attached to the outside of the patient's calf via the second strap 977B. The second retaining assembly 974B supports the second calibration fixture 973B, which is preferably parallel to the first reference frame 30a (i.e., perpendicular to the detector 33). The calibration fixtures 973A and 973B are preferably positioned sufficiently far from the target orthopedic element 100 so that the calibration fixtures 973A and 973B do not overlap with either of the target orthopedic elements 100.

[0074] Figure 9B is an internal and external view of an exemplary orthopedic element 100 (i.e., Figure 9B represents a second image 50 taken from a second reference frame 50a). In the illustrated example, the internal and external reference frames 50a are rotated or “offset” by 90° from the anterior and posterior first reference frames 30a. The first calibration fixture 973A is attached to the first retaining assembly 974A. The first retaining assembly 974A may include a first padded support 971A engaged with a first strap 977A. The first padded support 971A is attached to the lateral aspect of the patient’s thigh via the first strap 977A. The first retaining assembly 974A supports the first calibration fixture 973A, preferably parallel to the second reference frame 50a (i.e., perpendicular to the detector 33). Similarly, a second calibration fixture 973B attached to the second retaining assembly 974B may be provided. The second retaining assembly 974B may include a second padded support 971B engaged with a second strap 977B. The second padded support 971B is attached to the outside of the patient's calf via the second strap 977B. The second retaining assembly 974B supports a second calibration fixture 973B, preferably parallel to the second reference frame 50a (i.e., perpendicular to the detector 33). The calibration fixtures 973A, 973B are preferably positioned sufficiently far from the target orthopedic element 100 so that the calibration fixtures 973A, 973B do not overlap with any of the target orthopedic elements 100.

[0075] Because the knee joint is stable in this orientation (see Figure 18), the patient can be positioned preferably in an upright position (i.e., with the leg extended). Preferably, the patient's distance from the imaging device should not be changed during the acquisition of input images 30, 50. The first and second images 30, 50 do not need to capture the entire leg; rather, the images can focus on the target joint in the surgical area 170.

[0076] Depending on the orthopedic element 100 being imaged and modeled, it will be understood that only a single calibration fixture 973 may be used. Similarly, two or more calibration fixtures may be used, especially when a long set of orthopedic elements 100 are being imaged and modeled.

[0077] Each calibration fixture 973A, 973B is preferably of a known size. Each calibration fixture 973A, 973B preferably has at least four calibration points 978 distributed throughout. The calibration points 978 are distributed in a known pattern, where the distance from one point 978 to other points is known. The distance of the calibration fixture 973 from the orthopedic element 100 may also preferably be known. For the calibration of the X-ray photogrammetry system, the calibration points 978 may preferably be defined by a metallic structure on the calibration fixture 973. Metals typically absorb most of the X-ray beam that comes into contact with them. Therefore, metals typically appear much brighter than materials that absorb little X-ray (such as air cavities or fatty tissue). Common exemplary structures for defining calibration points include reseau crosses, circles, triangles, cones, and spheres.

[0078] These calibration points 978 may reside on the 2D surface of the calibration jig 973, or the 3D calibration points 978 may be captured as a 2D projection from a given image reference frame. In either case, the 3D coordinate (commonly called the z-coordinate) may be set to be equal to zero for all calibration points 978 captured in the image. The distance between each calibration point 978 is known. These known distances may be expressed as x,y coordinates in the image sensor / detector 33. To map points in 3D space to 2D coordinate pixels on the sensor 33, the dot product of the detector's calibration matrix, extrinsic matrix, and homologous coordinate vectors of the real 3D points may be used. This maps the real-world coordinates of points in 3D space to the calibration jig 973. In other words, this generally allows the x,y coordinates of a real point in 3D space to be precisely converted to the 2D coordinate plane of the image detector's sensor 33 in order to define the spatial data 43 (see Figure 10).

[0079] The calibration method described above is provided as an example. It will be understood that all suitable methods for calibrating a radiographic system are considered to be within the scope of this disclosure. A non-exclusive list of other radiographic system calibration methods includes the use of reseau plates, Zhang's method, bundle adjustment methods, direct linear transformation methods, maximum likelihood estimation, k-nearest neighbor regression approach ("kNN"), other deep learning methods, or combinations thereof.

[0080] Figure 11 shows how calibrated input images 30, 50 can be back-projected onto a 3D volume 61 containing two channels 65, 66 when oriented along a known offset angle θ. The first channel 65 contains all image points (e.g., X) of the first input image 30. L The second channel 66 includes all image points of the second input image 50 (e.g., X RThis includes, for example, each image point (e.g., a pixel) is replicated on its associated back-projected 3D ray. Then, using epipolar geometry, a volume 61 of the imaged surgical region 170, including volume data 75, can be generated from these back-projected 2D input images 30, 50.

[0081] Referring to Figure 11, the first image 30 and the second image 50 preferably have known image dimensions. The dimensions may be in pixels. For example, the first image 30 may have dimensions of 128 × 128 pixels. The second image 50 may have dimensions of 128 × 128 pixels. The dimensions of the input images 30, 50 used in a particular calculation preferably have consistent dimensions. Consistent dimensions may be desirable for later defining a cubic working area of ​​normal volume 61 (e.g., a 128 × 128 × 128 cube). As seen in Figure 10, the offset angle θ is preferably 90°. However, in other exemplary embodiments, other offset angles θ may be used.

[0082] In the illustrated example, each of the 128×128 pixel input images 30 and 50 is duplicated 128 times across the length of the adjacent input image to create a volume 61 with dimensions of 128×128×128 pixels. That is, the first image 30 is copied and stacked 128 pixels behind itself, one copy per pixel, while the second image 50 is copied and stacked 128 pixels behind itself so that the stacked images overlap and thereby create volume 61. Thus, it can be said that volume 61 contains two channels 65, 66, where the first channel 65 contains the first image 30 duplicated n times over the length of the second image 50 (i.e., the x-axis of the second image 50), and the second channel 66 contains the second image 50 duplicated m times over the length of the first image 30 (i.e., the x-axis of the first image 30), where "n" and "m" are the lengths of the indicated images, expressed as the number of pixels (or other dimensions in other exemplary embodiments) containing the length of the indicated image. Given an offset angle θ, each transverse slice of volume 61 (also known to some radiologists as an "axial slice") produces an epipolar plane containing voxels that are back-projected from pixels containing two epipolar lines. In this way, the spatial data 43 from the first image 30 of the target orthopedic element 100 and the spatial data 43 from the second image 50 of the target orthopedic element 100 are projected to define volume data 75. By using this volumetric data 75, the 3D representation can be reconstructed using the epipolar geometry principle as described above, and the 3D representation is geometrically consistent with the information in the input images 30 and 50.

[0083] An exemplary system and method for computing bone anomaly regions 115 using a deep learning network, where the deep learning network is a CNN, provides a detailed example showing how the CNN can be structured and trained. All CNN architectures are considered to be within the scope of this disclosure. Common CNN architectures include, for example, LeNet, GoogLeNet, AlexNet, ZFNet, ResNet, and VGGNet.

[0084] Figure 17 is a schematic diagram of a CNN showing how it may be used to identify regions of bone anomalies 115. While not bound by theory, it is intended that a CNN may be desirable in some cases to reduce the size of volume data 75 without losing the features necessary to identify a desired orthopedic element or desired region of bone anomaly 115. The volume data 75 of multiple back-projected input images 30, 50 is a multidimensional array that may be known as the “input tensor”. This input tensor contains the input data for the first convolution (in this example, the volume data 75). A filter (also known as a kernel 69) is shown placed on the volume data 75. The kernel 69 is a tensor (i.e., a multidimensional array) that defines the filter or function (this filter or function may be known as the “weight” given to the kernel). In the illustrated embodiment, the kernel tensor 69 is three-dimensional. The filter or function containing the kernel 69 may be programmed manually or learned through a CNN, RNN, or other deep learning network. In the illustrated embodiment, the kernel 69 is a 3 × 3 × 3 tensor, but all tensor sizes and dimensions are considered to be within the scope of this disclosure as long as the kernel tensor size is smaller than the size of the input tensor.

[0085] Each cell or voxel in kernel 69 has a numerical value. These values ​​define the filter or function of kernel 69. Convolution or cross-correlation operations are performed between two tensors. In Figure 17, the convolution is represented by path 76. Path 76, which kernel 69 follows, is a visualization of the mathematical operation. By following this path 76, kernel 69 sequentially traverses the entire volume 61 (e.g., volume data 75) of the input tensor. The goal of this operation is to extract features from the input tensor.

[0086] A convolutional layer 72 typically includes one or more of the convolutional stage 67, the detector stage 68, and the pooling stage 58. While each of these operations is visually represented in the first convolutional layer 72a in Figure 17, it will be understood that subsequent convolutional layers 72b, 72c, etc., may also include one or more, or all, of the operations of the convolutional stage 67, the detector stage 68, and the pooling layer 58, or combinations or permutations thereof. Furthermore, while Figure 17 shows five convolutional layers 72a, 72b, 72c, 72d, and 72e with varying resolutions, it will be understood that other exemplary embodiments may use more or fewer convolutional layers.

[0087] In the convolution stage 67, the kernel 69 is sequentially multiplied by multiple patches of pixels in the input data (i.e., volume data 75 in the illustrated example). The patches of pixels extracted from the data are known as receptive fields. The multiplication of kernel 69 with the receptive fields involves element-wise multiplication between each pixel in the receptive field and kernel 69. After multiplication, the results are summed to form one element of the convolution output. This kernel 69 is then shifted to adjacent receptive fields, and the element-wise multiplication and summing operations continue until all pixels in the input tensor undergo this operation.

[0088] Up to this point, the input data of the input tensor (e.g., volume data 75) is linear. Next, a nonlinear activation function is used to introduce nonlinearity into this data. The use of such a nonlinear function marks the beginning of the detector stage 68. A common nonlinear activation function is a normalized linear unit function ("ReLU") given by the function.

[0089]

number

[0090] When used with a bias, the nonlinear activation function serves as a threshold for detecting the presence of features extracted by kernel 69. For example, a convolutional output tensor is generated by applying a convolutional or cross-correlation operation between the input tensor and kernel 69, where kernel 69 includes a low-level edge filter in the convolution stage 67. A feature-mapped output tensor is then returned by applying a nonlinear activation function with a bias to the convolutional output tensor. The bias is sequentially added to each cell of the convolutional output tensor. For a given cell, if the sum is greater than or equal to 0 (assuming ReLU is used in this example), the sum is returned to the corresponding cell in the feature-mapped output tensor. Similarly, if the sum is less than 0 for a given cell, the corresponding cell in the feature-mapped output tensor is set to 0. Thus, applying a nonlinear activation function to the convolutional output behaves like a threshold for determining whether, and to what extent, the convolutional output matches the filter of kernel 69. In this way, the nonlinear activation function detects the presence of desired features from the input data (e.g., volume data 75 in this example).

[0091] All nonlinear activation functions are considered to be within the scope of this disclosure. Other examples include sigmoid, TanH, Leaky ReLU, parametric ReLU, Softmax, and Switch activation functions.

[0092] However, a drawback of this approach is that the feature map output of this first convolutional layer 72a records the exact location of the desired feature (edges in the example above). Therefore, small movements of features in the input data will generate different feature maps. To address this problem and reduce computational power, downsampling is used to reduce the resolution of the input data while still preserving important structural elements. Downsampling can be achieved by changing the stride of the convolution along the input tensor. Downsampling can also be achieved by using the pooling layer 58.

[0093] Effective padding may be applied to reduce the size of the convolved tensor (see 72b) compared to the input tensor (see 72a). A pooling layer 58 is preferably applied to reduce the spatial size of the convolved data, which reduces the computational power required to process the data. Common pooling techniques, including max pooling and mean pooling, may be used. Max pooling returns the maximum value of the portion of the input tensor covered by kernel 69, while mean pooling returns the average of all values ​​of the portion of the input tensor covered by kernel 69. Image noise may be reduced using max pooling.

[0094] In certain exemplary embodiments, a fully connected layer may be added after the final convolutional layer 72e to learn a nonlinear combination of high-level features represented by the output of the convolutional layer (such as profiles of imaged proximal tibia 110 or bone anomalies 115).

[0095] The upper half of Figure 17 represents the compression of the input volume data 75, and the lower half represents the decompression until the input volume data 75 reaches its original size. The output feature maps of each convolutional layer 72a, 72b, 72c, etc., are used as input to subsequent convolutional layers 72b, 72c, etc., to enable increasingly complex feature extraction. For example, the first kernel 69 may detect edges, the kernel in the first convolutional layer 72b may detect a set of edges in a desired orientation, the kernel in the third convolutional layer 72c may detect a longer set of edges in a desired orientation, and so on. This process may continue until the entire profile of the medial distal femoral condyle is detected by the downstream convolutional layer 72.

[0096] The lower half of Figure 17 is an upsample (i.e., an expansion of the spatial support of the lower-resolution feature map). The deconvolution operation is performed to increase the size of the input for the next downstream convolutional layer (see 72c, 72d, and 72e). In the case of the final convolutional layer 72e, the convolution may be employed with a 1×1×1 kernel 69 to produce a multi-channel output volume 59 of the same size as the input volume 61. Each channel of the multi-channel output volume 59 may represent a desired extracted high-level feature. Subsequently, the Softmax activation function may detect the desired orthopedic element 100. For example, the illustrated embodiment may include six output channels numbered 0, 1, 2, 3, 4, and 5, where channel 0 represents the identified background volume, channel 1 represents the identified distal femur 105, channel 2 represents the identified proximal tibia 110, channel 3 represents the identified proximal fibula 111, channel 4 represents the patella 901, and channel 5 represents the identified bone anomaly 115.

[0097] In an exemplary embodiment, a 3D model 1100 of the target orthopedic element or a 3D model 115m of the bone anomaly can be created using a selected output channel containing output volume data 59 of the desired orthopedic element 100 or bone anomaly 115b.

[0098] The above example illustrates the use of a three-dimensional tensor kernel 69 for convolving input volume data 75, but it will be understood that the general model described above can be used with 2D spatial data 43 from a first calibrated input image 30 and a second calibrated input image 50, respectively. In other exemplary embodiments, a machine learning algorithm (i.e., a deep learning network (e.g., CNN)) may be used after the imaging device has been calibrated, but before the 2D-to-3D reconstruction. That is, a CNN may be used to detect features (e.g., anatomical landmarks) of the target orthopedic element 100 from the first reference frame 30a and the second reference frame 50a of the respective 2D input images 30, 50. In an exemplary embodiment, a CNN may be used to identify high-level orthopedic elements (e.g., distal femur 105 and any bone anomalies 115) from the 2D input images 30, 50. The CNN may then optionally apply a mask or outline to the detected orthopedic element 100 or bone anomaly 115. The imaging device 1800 is configured, and the CNN analyzes multiple corresponding image points (e.g., X) of features between two input images 30 and 50. L , X R When an element is identified, it is intended that multiple corresponding image points in 3D space can be aligned using a transformation matrix between reference frames 30a and 50a of the target orthopedic element 100.

[0099] In a particular exemplary embodiment, which includes using a deep learning network to add a mask or outline to the 2D orthopedic elements 100 or bone anomalies 115 detected from each input image 30, 50, only the 2D mask or outline of the identified orthopedic elements 100 or bone anomalies 115 can be sequentially back-projected in the manner described with reference to Figures 10 and 11 above to define the volume 61 of the identified orthopedic elements 100 or bone anomalies 115. In this exemplary method, a 3D model 1100 of the target orthopedic element or a 3D model 115m of the bone anomaly can be produced.

[0100] In embodiments where the first image 30 and the second image 50 are X-ray images, training the CNN may present several challenges. For comparison, a CT scan typically generates a series of images of a desired volume. Each CT image, including a typical CT scan, can be considered a segment of the imaged volume. From these segments, a 3D model can be constructed relatively easily by adding regions of the desired elements as the elements are shown in their respective consecutive CT images. The modeled elements can then be compared to the CT scan data to ensure accuracy.

[0101] In contrast, radiographic imaging systems typically do not produce sequential images that capture different segments of the imaged volume. Rather, all the information in the image is flattened in a 2D plane. In addition, since a single radiographic image 30 inherently lacks 3D data, it is difficult to check the model generated by the epipolar geometry reconstruction technique described above using the actual geometric shape of the target orthopedic element 100. To address this problem, a CNN can be trained using CT images, such as digitally reconstructed radiograph ("DRR") images. By training the neural network in this way, the neural network can develop its own weights (e.g., filters) for the kernel 69 to identify the desired orthopedic element 100 or bone anomaly 115b. Since X-ray images have a different appearance from DRRs, an inter-image transformation can be performed to render the input X-ray image to have a DRR-style appearance. An exemplary inter-image transformation method is the CycleGAN inter-image transformation technique. In embodiments where an inter-image style transfer method is used, the style transfer method is preferably used before inputting the data into the deep learning network for feature detection.

[0102] The above examples are provided for illustrative purposes only and are not intended to limit the scope of this disclosure. All methods for generating a 3D model 1100 of the same orthopedic element 100 or a 3D model 115m of a bone anomaly from 2D radiographic images (e.g., 30a, 50a) of the same orthopedic element 100 taken from at least two lateral positions are considered to be within the scope of this disclosure.

[0103] Figure 12 is a perspective view showing the distal femur 105 and 3D models 115m1 and 115m2 of bone anomalies. It will be understood that the 3D models 115m1 and 115m2 of bone anomalies can be generated according to any method or system of this disclosure. In the illustrated embodiment, the distal femur 105 had two negative bone anomalies 115a and 115c (i.e., areas of bone loss) on the medial condyle 107. The method according to this disclosure can be used to identify the bone anomalies 115a and 115c as described herein. An output channel containing bone anomaly volume data 59 can be used to generate the 3D model 115m1 of the bone anomaly. In the illustrated example, each of the 3D models 115m1 and 115m2 of the bone anomalies contains the inverse volume of the actual bone anomalies 115a and 115c. To depict the volume boundaries containing 3D models 115m1 and 115m2 of bone anomalies, a deep learning network can be trained to detect the edges of the actual negative bone anomalies 115a and 115c and the curvature of the surface of adjacent orthopedic elements 100 where the negative bone anomalies 115a and 115c reside. In the illustrated example, the surface of the adjacent orthopedic element is the medial condylar surface. Surface adjustment algorithms, such as curve fitting algorithms, can then be applied to estimate the curvature of the defect surface, thereby correcting the negative bone anomaly 115. In embodiments, the estimated curvature can then be added to the identified surface region of the negative bone anomaly (e.g., 115a). The space between the identified surface region of the negative bone anomaly and the estimated curvature defines the modeled volume of the negative bone anomaly 115a. Data including this modeled volume can be used to generate a 3D model 115m1 of the anomaly, which has a modeled inverse volume relative to the actual volume of the negative bone anomaly 115a.

[0104] In embodiments where the bone anomaly 115 is a positive bone anomaly (e.g., a bone proliferator), it will be understood that a deep learning network can be trained to detect the edges of the actual positive bone anomaly 115 and the curvature of the surface of the adjacent orthopedic element 100 where the positive bone anomaly 115 is present. If the surface of the adjacent orthopedic element 100 is curved, a curve fitting algorithm can be used to estimate the curvature of the surface where the positive bone anomaly 115 is absent, thereby correcting the positive bone anomaly 115.

[0105] In certain exemplary embodiments, the 3D model 115m1 of the bone anomaly may be produced as a physical 3D model of the bone anomaly. When used intraoperatively, the physical 3D model 115m1 of the bone anomaly may be fabricated at a 1:1 scale. In such exemplary embodiments, the physical 3D model 115m1 of the bone anomaly may be fabricated from medical-grade polyamide (informally known as nylon). The material of the physical 3D model 115m1 of the bone anomaly should be sterilizable and preferably have properties suitable for autoclaving. Autoclaves are generally small and compact, making them particularly useful for sterilizing the physical 3D model 115m1 of the bone anomaly in or near a surgical clinic.

[0106] Other examples of suitable materials for the 3D model 115m1 of the bone anomaly include medical polyethylene (e.g., ultra-high molecular weight polyethylene ("UHMWPE")), polyether ether ketone ("PEEK"), or other clinically proven biocompatible materials including, but not limited to, cobalt-chromium-molybdenum alloys, titanium alloys, and ceramic materials including, but not limited to, zirconia-reinforced alumina ("ZTA") ceramics. In situations where the bone anomaly is an area of ​​bone loss, the advantage of a 1:1 physical 3D model 115m1 of the bone anomaly is that the physical 3D model 115m1 of the bone anomaly has a complementary surface to the exposed surface 116 of the actual bone anomaly (see 115b in Figures 2 and 4). Provided that the physical 3D bone anomaly model 115m1 is properly sterilized, the physical 3D bone anomaly model 115m1 can be placed adjacent to the complementary surface 116 of the actual bone anomaly 115b during surgery. In this way, the aforementioned uncertainties are avoided by referring to Figure 2 above.

[0107] In exemplary embodiments, a physical 3D model 115m2 of the bone anomaly can be selectively attached to one or both posterior pads 693a, 693b of the alignment guide 600. By providing a physically sterilized model of the bone defect, the uncertainties described below with reference to Figure 5 are intended to be eliminated.

[0108] Figure 3 is a simplified front view of a left knee joint, visually representing the process of detecting bone abnormalities 115a, 115b on or within volume data 75 of an orthopedic element 100 using a deep learning network. The deep learning network may detect other landmarks from the volume data 75 of the target orthopedic element 100 to define a 3D model 1100 of the target orthopedic element. A surface adjustment algorithm is then applied to remove the bone abnormalities 115a, 115b from the 3D model 1100 of the target orthopedic element. In the illustrated embodiment, the bone abnormalities 115a, 115b are removed by calculating external defect bone surfaces 117a, 117b (in this example, the lateral surface region of bone loss) that coincide with the lateral surface of the bone abnormalities 115a, 115b. A computer platform performing the exemplary method may run software trained via artificial intelligence to identify features (i.e., landmarks) on worn condyles that indicate bone loss in the 3D model. In other exemplary embodiments, the boundaries of bone loss can be manually identified via a computer interface to identify the areas of bone anomalies 115a, 115b. Once identified, in situations where the bone anomalies constitute bone loss, a surface adjustment algorithm may be applied to calculate external defect bone surfaces 117a, 117b that conform to the outer surface areas of the bone anomalies 115a, 115b.

[0109] In other exemplary embodiments, the step of using a deep learning network to detect bone anomalies 115a, 115b on or within volumetric data 75 of orthopedic elements 100 further includes generating a 3D model 115m of the bone anomaly. If the 3D bone anomaly model 115m is a computer model, the 3D bone anomaly model 115m may optionally be projected onto a display 19 such as a screen. In certain exemplary embodiments, the 3D bone anomaly computer model 115m may be projected into the surgeon's field of view so as to be superimposed on the actual bone anomaly 115 in an image of the surgical area 170. In other exemplary embodiments, the 3D bone anomaly computer model 115m may be projected into the surgeon's field of view so as to be superimposed on the patient's actual bone anomaly 115, such as within an exposed surgical area 170. Such a display 19 may be achieved via an augmented reality device, preferably a head-mounted augmented reality device.

[0110] In yet another exemplary embodiment, the physical 3D bone anomaly model 115m may be fabricated through a manufacturing technique (see Figure 18). This manufacturing technique may include a reduction manufacturing method, such as the use of a computer numerical control ("CNC") machine or milling machine. In yet another exemplary embodiment, the manufacturing technique may include an additive manufacturing technique, such as a 3D printing technique. If the manufacturing technique is an additive manufacturing technique, the physical 3D bone anomaly model 115m may be fabricated at a preoperative center, at a location away from the site, or at or near the surgical facility.

[0111] Figure 4 is a side view of a swivel femoral resection guide positioning device 400 having an adjustment pad 440 with an adjustment knob 442 extending to the calculated external defect bone surface 117b on the lateral condyle 103. As can be seen, if the depth of cartilage wear is properly confirmed, the adjustment pad 440 may be considered to extend to the pre-symptomatic surface of the bone. However, in practice, the surgeon may instead choose to add the depth of bone loss to the confirmed depth of cartilage loss, set the length l of the adjustment pad 440 to reflect the sum of the maximum bone loss depth and the cartilage loss depth, and then choose to position the adjustment pad 440 on the remaining exposed bone 116 of the lateral condyle. In this way, the distal reference plane 485 is precisely positioned on the articular surface of the pre-symptomatic joint at this point.

[0112] In another exemplary embodiment, a 1:1 physical 3D bone anomaly model 115m may be fixed to the distal end of an adjustment knob 440 such that the complementary surface of the 1:1 physical 3D bone anomaly model 115m engages with the surface 116 of the bone anomaly 115b when the adjustment knob 440 is positioned adjacent to the exposed bone anomaly 115b (see Figure 17). In yet another exemplary embodiment, where one of the target orthopedic elements 100 is the articular cartilage 123 of the distal femur 105, the physical 3D bone anomaly model 115m1 may include a modeled surface of the cartilage defect in addition to the volume of bone loss as described above. A surface adjustment algorithm may be used to define the surface of the cartilage defect relative to the surface of the surrounding cartilage.

[0113] Figure 5 is a simplified perspective view of the distal end of the femur 105 after distal resection has been performed using the resection guide 424. The alignment guide 600, including posterior pads 693a, 693b, is positioned below the posterior portions 103a, 107a of the femoral condyles 103, 107. For simplification, the posterior portions of the femoral condyles will be referred to as “posterior condyles” 103a, 107a. The alignment guide 600 may be a composite sizing and alignment guide as shown herein, or it may be the alignment guide 600 alone. The alignment guide 600 may include a body 602. The posterior pads 693a, 693b extend from the lower portion of the body 602, and the drill bores 642a, 642b extend through the body 602 above the posterior pads 693a, 693b. In the illustrated embodiment, the swivel drill bore 642a extends through the body 602 of the alignment guide 600, and the radial drill bore 642b is positioned radially distal to the swivel drill bore 642a. The radial drill bore 642b also extends through the body 602 of the alignment guide 600. In practice, the surgeon places the posterior pads 693a, 693b beneath the respective posterior condyles 103a, 107a, so that the body 602 is positioned adjacent to the resection surface 603 of the distal femur 105. The surgeon measures the thickness of the remaining articular cartilage on the posterior condyles 103a, 107a and sets the length of the posterior pads 693a, 693b to reflect the amount of cartilage wear (similar to the method described above with reference to Figure 2). By adjusting the positions of the rear pads 693a and 693b relative to the main body 602, the radial drill bore 642b is rotated around the swivel drill bore 642a. Once the surgeon is satisfied with the angle, the surgeon can lock the angle of the swivel portion of the alignment guide 600 in place.

[0114] Next, the surgeon can drill holes in the resection surface 603 through drill bores 642a and 642b, and then insert pins into the respective drill bores 642a and 642b. The surgeon can then remove the alignment guide 600, leaving the pins in place. The angles of these pins define the angles at which a further resection guide (commonly known as a "4-in-1 resection block") can be positioned next to the resection surface. The 4-in-1 resection block has additional resection slots that allow the surgeon to perform anterior, posterior, and chamfered resections using a single "resection block". These additional resections create a profile on the distal femur 105 where a test implant (and eventually the actual internal prosthesis implant) can be placed.

[0115] Negative bone anomalies (e.g., osteopenia) 115c may be less common in the posterior portions 103a and 107a of the femoral condyles 103 and 107, but such osteopenia is still possible, especially in progressive degenerative diseases. Negative bone anomalies 115c of the posterior condyles 103a and 107a present similar problems in accurately replicating the natural articular surfaces of the pre-symptomatic joint, particularly in kinematic alignment.

[0116] For example, if the medial posterior condyle 107a has a negative bone anomaly 115 as shown in Figure 5, it was previously impossible to be certain how to adjust the illustrated alignment guide to account for the amount of bone wear present. Over-adjusting the medial posterior pad 693a can change the position in which each drill bore 642a, 642b is positioned relative to the resection surface 603, and the rotation angle of each drill bore 642a, 642b may change. As a result, the pin may be mispositioned. Consequently, the position of the 4-in-1 resection block will also be shifted to slide over the mispositioned pin. Incorrect positioning of the anterior, posterior, and chamfered resections can result in the improper seating of the femoral components of the internal prosthesis implant.

[0117] To address this problem, surgeons can measure the amount of articular cartilage wear on the posterior condyles 103a, 107a using the method described above or other known methods. In another exemplary embodiment, a 1:1 physical 3D bone anomaly model 115m may be fixed to the distal end of the posterior pads 693a, 693b such that the complementary surface (also known as the "mating surface") of the 1:1 physical 3D bone anomaly model 115m engages with the surface 116c of the bone anomaly 115c when the posterior pad 693a is positioned adjacent to the exposed bone anomaly 115c.

[0118] In other exemplary embodiments, the mating surface of the physical 3D bone anomaly model may include one or more protrusions (e.g., spikes, pins, or other protections). These protrusions can be hammered into or otherwise forcibly inserted into the abrasion surface 116c of the bone anomaly 115c, thereby fixing the physical model 115m of the bone anomaly within the negative bone anomaly 115c, and thereby eliminating the negative bone anomaly 115c. Using the alignment guide 600 in this way with the physical 3D bone anomaly model ensures more accurate reference. Furthermore, some 4-in-1 cutting blocks have markings designed to reference the surfaces of the posterior condyles 107a, 103a. Using the physical 3D bone anomaly model in this way can effectively recreate the pre-symptomatic surfaces of the posterior condyles 107a, 103a and serve as a visual guide to properly align the 4-in-1 cutting block (or possibly other instrument) to the referenced index.

[0119] In yet another exemplary embodiment, where one of the target orthopedic elements 100 is the articular cartilage 123 of the distal femur 105, the physical 3D bone anomaly model 115m2 may include a modeled surface of the cartilage defect in addition to the volume of bone loss as described above. A surface adjustment algorithm may be used to define the surface of the cartilage defect relative to the surface of the surrounding cartilage. In this way, the articular surface of the condyle can be accurately recreated, thereby substantially improving the accuracy of the articular surface reference.

[0120] In other exemplary embodiments, shims having a height equal to the maximum depth of the negative bone anomaly, the depth of the articular cartilage defect, or the sum of the maximum depth of the negative bone anomaly and the depth of the articular cartilage defect are added to one or more of the posterior pads 693a, 693b to offset the amount of wear and substantially recreate the position of the pre-onset articular surface.

[0121] A computer platform having hardware such as one or more central processing units ("CPU"), random access memory ("RAM"), and input / output ("I / O") interfaces may receive at least two 2D radiographic images taken in different orientations along a transverse plane, preferably orthogonal to each other. The computer platform may then run a machine learning software application that identifies areas of bone loss 115a, 115b, and 115c and applies a surface adjustment algorithm to compute external defect bone surfaces 117a, 117b, and 117c that fit the areas of bone loss 115a, 115b, and 115c.

[0122] The computer platform may optionally display a 3D computer model 1100. In an exemplary embodiment where the 3D model is displayed, the computer platform may further display the external defect bone surfaces 117a, 117b, 117c over the bone loss areas 115a, 115b, 115c, allowing an observer to visualize the external defect bone surfaces 117a, 117b, 117c before the onset. Referring back to Figure 5, the surgeon can use this data to set the posterior pads 693a, 693b of the alignment guide 600 to reflect the joint surfaces of the posterior condyles 103a, 107a before the onset.

[0123] Figure 16 shows a block diagram of an exemplary computer 1600 in which, generally speaking, one or more of the methods discussed herein can be performed according to several exemplary embodiments. In certain exemplary embodiments, computer 1600 may operate as a single machine. In other exemplary embodiments, computer 1600 may include connected (e.g., networked) machines. Examples of networked machines that may include exemplary computer 1600 include, for example, cloud computing configurations, distributed host configurations, and other computer cluster configurations. In a network configuration, one or more machines of computer 1600 may operate as a client machine, a server machine, or both a server and a client machine. In exemplary embodiments, computer 1600 may reside in a personal computer ("PC"), a mobile phone, a tablet PC, a web appliance, a personal digital assistant ("PDA"), a network router, a bridge, a switch, or any machine capable of executing instructions that specify actions to be taken by or controlled by the machine.

[0124] An exemplary machine, which may include exemplary computer 1600, may, for example, be a component, module, or similar mechanism capable of performing a logical function. Such a machine may be a tangible entity (e.g., hardware) capable of performing a specified operation during operation. For example, hardware may be wired (e.g., specifically configured) to perform a particular operation. For example, such hardware may include a configurable execution medium (e.g., circuits, transistors, logic gates, etc.) and a computer-readable medium having instructions, the instructions configuring the execution medium to perform a particular operation during operation. Configuration may be done via a loading mechanism or under the direction of the execution medium. The execution medium selectively communicates with the computer-readable medium when the machine is operating. For example, when the machine is operating, the execution medium may be configured by a first set of instructions to perform a first action or set of actions at a first time point, and then reconfigured by a second set of instructions to perform a second action or set of actions at a second time point.

[0125] An exemplary computer 1600 may include a hardware processor 1697 (e.g., a CPU, a graphics processing unit ("GPU"), a hardware processor core, or any combination thereof), main memory 1696, and static memory 1695, some or all of which may communicate with each other via an interlink (e.g., a bus) 1694. The computer 1600 may further include a display unit 1698, an input device 1691 (preferably an alphanumeric or character-number input device such as a keyboard), and a user interface ("UI") navigation device 1699 (e.g., a mouse or stylus). In exemplary embodiments, the input device 1691, the display unit 1698, and the UI navigation device 1699 may be touchscreen displays. In exemplary embodiments, the display unit 1698 may include holographic lenses, glasses, goggles, other eyewear, or other AR or VR display components. For example, the display unit 1698 may be worn on the user's head to provide the user with a head-up display. The input device 1691 may be a virtual keyboard (for example, a keyboard that is virtually displayed in a virtual reality ("VR") or augmented reality ("AR") setting) or another virtual input interface.

[0126] Computer 1600 may further include a memory device (e.g., a drive unit) 1692, a signal generator 1689 (e.g., a speaker), a network interface device 1688, and one or more sensors 1687 such as a Global Positioning System ("GPS") sensor, an accelerometer, a compass, or other sensors. Computer 1600 may also include an output controller 1684 for communicating with or controlling one or more auxiliary devices, such as a serial (e.g., Universal Serial Bus ("USB")), parallel, or other wired or wireless (e.g., infrared ("IR"), near-field communication ("NFC"), radio, etc.) connection.

[0127] The storage device 1692 may include a non-temporary, machine-readable medium 1683 on which one or more sets 1682 (e.g., software) of data structures or instructions embodied or utilized by one or more of the functions or methods described herein are stored. The instructions 1682 may reside entirely or at least partially within the main memory 1696, the static memory 1695, or the hardware processor 1697 during their execution by the computer 1600. For example, one or any combination of the hardware processor 1697, the main memory 1696, the static memory 1695, or the storage device 1692 may constitute a machine-readable medium.

[0128] Although machine-readable medium 1683 is shown as a single medium, the term “machine-readable medium” may include a single or multiple mediums configured to store one or more instructions 1682 (e.g., a distributed or centralized database, or associated caches and servers).

[0129] The term “machine-readable medium” may include any medium capable of storing, encoding, or transmitting instructions executed by the computer 1600, and capable of causing the computer 1600 to execute any one or more of the methods of the Disclosure, or capable of storing, encoding, or transmitting data structures used by or associated with such instructions. A non-exclusive exemplary list of machine-readable mediums may include magnetic media, optical media, solid-state memory, non-volatile memory, for example, semiconductor memory devices (e.g., electronically erasable programmable read-only memory, “EEPROM”, electronically programmable read-only memory, “EPROM”, and magnetic disks such as internal hard disks and removable disks, flash memory, magneto-optical disks, CD-ROM disks, and DVD-ROM disks).

[0130] Instruction 1682 may further be transmitted or received over a communication network 1681 using a transmission medium via a network interface device 1688 that utilizes one of several transport protocols (e.g., Internet Protocol ("IP"), User Datagram Protocol ("UDP"), Frame Relay, Transmission Control Protocol ("TCP"), Hypertext Transfer Protocol ("HTTP"), etc.). Examples of communication networks may include wide area networks ("WAN"), plain old telephone ("POTS") networks, local area networks ("LAN"), packet data networks, mobile phone networks, wireless data networks, and peer-to-peer ("P2P") networks. For example, the network interface device 1688 may include one or more physical jacks (e.g., Ethernet, coaxial, or telephone jacks) or one or more antennas to connect to the communication network 1681.

[0131] For example, the network interface device 1688 may include multiple antennas for wireless communication using at least one of the single-input multiple-output ("MIMO") or multiple-input single output ("MISO") methods. The term "transmission medium" includes any intangible medium on which instructions executed by the computer 1600 can be stored, encoded, or transmitted, and includes analog or digital communication signals or other intangible medium to facilitate the communication of such software.

[0132] The exemplary methods described herein may be at least partially mechanical or computer-implemented. Some examples may include computer-readable or machine-readable media encoded with instructions operable to configure an electronic device to perform the exemplary methods described herein. Exemplary implementations of such exemplary methods may include code, such as assembly language code, microcode, high-level language code, or other code. Such code may include computer-readable instructions for performing various methods. The code may form part of a computer program product. Furthermore, in one example, the code may be tangibly stored on or within a volatile, non-temporary, or non-volatile tangible computer-readable medium, such as during execution or at other times. Examples of such tangible computer-readable media include, but are not limited to, removable optical disks (e.g., compact disks and digital video disks), hard drives, removable magnetic disks, memory cards or sticks, removable flash memory drives, magnetic cassettes, random-access memory (RAM), read-only memory (ROMS), and other media.

[0133] In certain exemplary embodiments, the surface adjustment algorithm may be a curve fitting algorithm. The exemplary curve fitting algorithm may involve interpolation or smoothing. In other exemplary embodiments, the curve fitting algorithm may be used to extrapolate the position of the pre-onset articular surface of the bone. In other exemplary embodiments, the surface adjustment algorithm may identify the dimensions of an unabraded contralateral orthopedic element 100, such as an unabraded contralateral condyle. The surface adjustment algorithm may calculate the external defect bone surfaces 117a, 117b by adding the surface of the unabraded orthopedic element to the corresponding area of ​​bone loss on the abraded orthopedic element 100. In the relevant exemplary embodiments, the initial defect bone surface calculation based on measurements of the unabraded orthopedic element 100 may be increased or decreased to fit the curve of the unabraded portion of the abraded orthopedic element 100.

[0134] In other exemplary embodiments, the surface adjustment algorithm may calculate the maximum depth of bone loss. In such embodiments, this maximum depth may be added to the depth of articular cartilage loss to calculate the pre-symptomatic articular surface location for each condyle. In yet another embodiment, the volume 61 of the bone loss regions 115a, 115b may be calculated. It will be understood that any disclosed calculation or the results of any such calculation may be optionally displayed on the display 19. In other exemplary embodiments, this method may further include determining the depth of the defective articular cartilage overlapping the external bone defect surface and defining the pre-symptomatic articular condyle surface by adding the depth of the defective articular cartilage to the external bone defect surface.

[0135] Figure 13 is a flowchart outlining the steps of an exemplary method for identifying a region of bone abnormality 115 on or within an imaged orthopedic element 100 using two flattened input images taken from an offset angle θ, using a deep learning network. The exemplary method includes step 1b, which involves calibrating the imager to define spatial data 43 by determining the mapping relationship between image points and corresponding spatial coordinates. Step 2b includes capturing a first image 30 of the target orthopedic element 100 using an imaging technique, the first image 30 defining a first reference frame 30a. Step 3b includes capturing a second image 50 of the target orthopedic element 100 using an imaging technique, the second image 50 defining a second reference frame 50a, the first reference frame 30a being offset from the second reference frame 50a by an offset angle θ.

[0136] Step 4b includes defining volumetric data 75 by projecting spatial data 43 from a first image 30 of the target orthopedic element 100 and spatial data 43 from a second image 50 of the target orthopedic element 100. Step 5b includes detecting the target orthopedic element 100 using the volumetric data 75 with a deep learning network, where the volumetric data 75 defines anatomical landmarks on or within the target orthopedic element 100.

[0137] Step 6b includes using a deep learning network to detect bone abnormalities 115 on or within the orthopedic element 100 using volumetric data 75. Step 7b includes using a deep learning network to apply a mask to the orthopedic element 100 defined by anatomical landmarks. Step 8b includes applying the deep learning network to the volumetric data 75 to generate a 3D model of the orthopedic element 100. Step 9a includes applying a surface adjustment algorithm to remove bone abnormalities 115 from the 3D model 1100 of the orthopedic element.

[0138] In methods and systems deemed to be within the scope of this disclosure, it will be understood that the deep learning network for detecting the target orthopedic element 100, the deep learning network for detecting the bone anomaly 115, the deep learning network for applying a mask or generating a 3D model 115m of the bone anomaly or a 3D model 1100 of the orthopedic element, or the deep learning network for applying a surface adjustment algorithm may be the same deep learning network or may be different deep learning networks. In embodiments where the deep learning networks are different deep learning networks, these deep learning networks may be referred to as the "first deep learning network," the "second deep learning network," the "third deep learning network," and so on.

[0139] Figure 14 is a flowchart outlining the steps of an exemplary method for identifying a region of bone abnormality 115 on or within an imaged orthopedic element 100 using two flattened input images taken from an offset angle θ, using a deep learning network. The exemplary method includes step 1c of calibrating a radiographer to define spatial data 43 by determining the mapping relationship between radiograph points and corresponding spatial coordinates. Step 2c includes capturing a first radiograph 30 of the orthopedic element 100 using radiographing techniques, the first radiograph 30 defining a first reference frame 30a.

[0140] Step 3c includes capturing a second radiographic image 50 of the target orthopedic element 100 using radiographic imaging techniques, wherein the second radiographic image 50 defines a second reference frame 50a, and the first reference frame 30a is offset from the second reference frame 50a by an offset angle θ. Step 4c includes defining volumetric data 75 by projecting spatial data 43 from the first radiographic image 30 of the target orthopedic element 100 and spatial data 43 from the second radiographic image 50 of the target orthopedic element 100. Step 5c includes detecting the target orthopedic element 100 using a deep learning network, wherein the spatial data 43 defines anatomical landmarks on or within the target orthopedic element 100.

[0141] Step 6c includes applying a mask to the bone anomaly 155 using a deep learning network, wherein spatial data 43 containing image points located within the masked region of either the first or second image has a first value (e.g., a positive value, or "1"), and spatial data 43 containing image points located outside the masked region of either the first image 30 or the second image 50 has a second value (e.g., a negative value, or "0"), where the first value is different from the second value. Step 7c includes calculating a correction region to remove the bone anomaly region 115.

[0142] Figure 15 is a flowchart outlining the steps of an exemplary method for identifying a region of bone abnormality 115 on or within an imaged orthopedic element 100 using two flattened input images taken from an offset angle θ, using a deep learning network. The exemplary method includes step 1d of calibrating a radiographer to define spatial data 43 by determining the mapping relationship between radiograph points and corresponding spatial coordinates.

[0143] Step 2d includes capturing a first radiographic image 30 of the target orthopedic element 100 using radiographic imaging techniques, wherein the first radiographic image 30 defines a first reference frame 30a.

[0144] Step 3d includes capturing a second radiographic image 50 of the target orthopedic element 100 using radiographic imaging techniques, wherein the second radiographic image 50 defines a second reference frame 50a, and the first reference frame 30a is offset from the second reference frame 50a by an offset angle θ. Step 4d includes defining volumetric data 75 by projecting spatial data 43 from the first radiographic image 30 of the target orthopedic element 100 and spatial data 43 from the second radiographic image 50 of the target orthopedic element 100.

[0145] Step 5d includes using a deep learning network to detect the target orthopedic element 100 using volumetric data 75, where the volumetric data 75 defines anatomical landmarks on or within the orthopedic element 100. Step 6d includes using a deep learning network to apply a mask to the target orthopedic element 100 defined by the anatomical landmarks, where spatial data 43 containing image points located within the masked region of either the first or second image has a positive value, and spatial data 43 containing image points located outside the masked region of either the first or second image has a negative value. Step 7d includes using a deep learning network to detect bone abnormalities 115 on or within the orthopedic element 100 using volumetric data 75. Step 8d includes applying the deep learning network to the volumetric data 75 to generate a 3D model of the bone abnormality.

[0146] The exemplary systems and methods disclosed herein are further intended to be used for preoperative planning, intraoperative planning or execution, or postoperative evaluation of implant placement and function.

[0147] Figure 18 is a schematic diagram of an exemplary system comprising a radiographer 1800 including an X-ray source 21, such as an X-ray tube, a filter 26, a collimator 27, and a detector 33. In Figure 18, the radiographer 1800 is shown from top to bottom. Patient 1 is positioned between the X-ray source 21 and the detector 33. The radiographer 1800 may be mounted on a rotatable gantry 28. The radiographer 1800 may take a radiographic image of patient 1 from a first reference frame 30a. The gantry 28 may then rotate the radiographer 1800 by an offset angle (preferably 90°). The radiographer 1800 may then take a second radiographic image 50 from a second reference frame 50a. It will be understood that other exemplary embodiments may include using multiple input images taken at multiple offset angles θ. In such embodiments, the offset angles may be less than 90° or greater than 90° between adjacent input images.

[0148] Next, the transmitter 29 transmits the first image 30 and the second image 50 to the computer 1600. The computer 1600 may apply a deep learning network to identify regions 115 of bone abnormalities on or within the orthopedic element 100 in any manner consistent with the present disclosure. Figure 18 further illustrates that the output of the computer 1600 is transmitted to a manufacturing machine 18. The manufacturing machine 18 may be an additive manufacturing machine such as a 3D printer, or the manufacturing machine may be a subtractive manufacturing machine such as a CNC machine. In yet another exemplary embodiment, the manufacturing machine 18 may be a casting mold. The manufacturing machine 18 may use the output data from the computer 1600 to generate physical models 1100 of one or more 3D models of the orthopedic element in question. In an embodiment, this manufacturing machine may be used to generate a physical 3D model 115m of a bone abnormality.

[0149] Figure 18 also shows another embodiment in which output data from the computing machine 1600 is transmitted to the display 19. The first display 19a shows a virtual 3D model 115m of a bone anomaly. The second display 19b shows a virtual 3D model 1100 of the identified orthopedic element of interest.

[0150] In other exemplary embodiments, the 3D model may be displayed on a display 19. This display 19 may take the form of a screen. In other exemplary embodiments, the display 19 may include a glass or plastic surface worn or held by a surgeon or other person in an operating room. Such a display 19 may include part of an augmented reality device, so that the display shows the 3D model in addition to the bearer's field of view. In certain embodiments, such a 3D model may be superimposed on an actual surgical joint. In yet another exemplary embodiment, the 3D model can be "locked" to one or more features of the surgical orthopedic element 100, thereby maintaining the virtual position of the 3D model relative to one or more features of the surgical orthopedic element 100, independently of the movement of the display 19.

[0151] It is further intended that the display 19 may include part of a virtual reality system in which the entire field of view is simulated.

[0152] An exemplary method for calculating external bone loss for pre-onset joint alignment includes generating a 3D model of the surgical area from at least two 2D radiographic images. At least the first radiographic image is captured at a first lateral position. At least the second radiographic image is captured at a second lateral position. The first lateral position is different from the second lateral position. Preferably, the first lateral position is positioned orthogonally from the second lateral position. The method further includes identifying areas of bone loss on the 3D computer model and applying a surface adjustment algorithm to calculate an external bone defect surface that fits the areas of bone loss.

[0153] A method for calculating the degree of external bone loss includes capturing a first image of a desired orthopedic element using radiographic imaging techniques, wherein the first image defines a first reference frame; capturing a second image of the desired orthopedic element using radiographic imaging techniques, wherein the second image defines a second reference frame, and the first reference frame is offset from the second reference frame by an offset angle; generating a 3D model of the desired orthopedic element by applying 3D reconstruction techniques; identifying areas of bone loss within the 3D model of the desired orthopedic element; identifying intact areas of bone adjacent to the areas of bone loss; and displaying the filled areas of bone loss by applying an adjustment algorithm.

[0154] An exemplary method for calculating external bone loss for pre-symptomatic joint alignment includes generating a 3D model of the surgical area from at least two 2D radiographic images, wherein at least a first radiographic image is captured at a first position and at least a second radiographic image is captured at a second position, the first lateral position being different from the second lateral position; identifying areas of bone abnormality on the 3D model; and applying a surface adjustment algorithm to calculate an external bone defect surface configured to replace the areas of bone abnormality.

[0155] In an exemplary embodiment, the surface adjustment algorithm is a curve fitting algorithm. In an exemplary embodiment, the method further includes calculating the maximum depth of the region of bone abnormality. In an exemplary embodiment, the method further includes defining the pre-symptomatic articular surface by adding the maximum depth of the region of bone abnormality to the depth of cartilage wear. In an exemplary embodiment, during surgery, a 3D model is displayed on an augmented reality device over actual orthopedic elements. In an exemplary embodiment, the region of bone abnormality is a region of bone loss.

[0156] In an exemplary embodiment, the method further includes identifying an intact area of ​​the contralateral orthopedic element, where the intact area corresponds to an aggravated area of ​​the surgical orthopedic element.

[0157] An exemplary method for calculating external bone loss for the purpose of kinematically aligning the knee joint before the onset of injury includes generating a 3D model of the knee joint surgical area from at least two 2D radiographic images, wherein at least a first radiographic image is captured at a first position and at least a second radiographic image is captured at a second position, the first position being different from the second position; identifying the area of ​​bone loss on the 3D model; applying a surface adjustment algorithm to calculate an external defect bone surface that fits the area of ​​bone loss; confirming the depth of the defective articular cartilage overlapping the external defect bone surface; and defining the pre-onset condylar surface by adding the depth of the defective articular cartilage to the external defect bone surface.

[0158] In exemplary embodiments, this method further includes adjusting the adjustable pad of the resection guide locator to contact the remaining external bone surface such that the guide surface of the resection guide locator is positioned on the pre-symptomatic condyle surface.

[0159] An exemplary method for calculating the degree of orthopedic deterioration in vivo includes: capturing a first image of a desired orthopedic element using non-invasive imaging techniques, the first image defining a first reference frame; capturing a second image of the desired orthopedic element using non-invasive imaging techniques, the second image defining a second reference frame, the first reference frame being offset from the second reference frame by an offset angle; applying 3D reconstruction techniques to generate a 3D model of the desired orthopedic element; identifying areas of bone loss within the 3D model of the desired orthopedic element; and applying a surface adjustment algorithm to calculate the surface of the deteriorated areas.

[0160] In an exemplary embodiment, the method further includes projecting a 3D reconstructed model onto a display. In an exemplary embodiment, the non-invasive imaging technique is a radiographic imaging technique.

[0161] An exemplary method for calculating cartilage wear and bone loss in a kinematic alignment procedure involves calibrating a radiographer to determine the mapping relationship between image points and corresponding spatial coordinates and define spatial data; capturing a first image of a desired orthopedic element using radiographing techniques, the first image defining a first reference frame; and capturing a second image of the desired orthopedic element using radiographing techniques, the second image defining a second reference frame, the first reference frame being offset from the second reference frame by an angle. This includes offsetting, identifying the spatial data of a desired orthopedic element in a first image and the spatial data of an orthopedic element in a second image, transforming the spatial data of the desired orthopedic element in the first and second images into a single coordinate system and defining the transformed spatial data, projecting the transformed spatial data of the desired orthopedic element onto a display to generate a 3D model of the desired orthopedic element, identifying aggravated areas within the 3D model of the desired orthopedic element, and applying a surface adjustment algorithm to calculate the surface of the aggravated areas.

[0162] In exemplary embodiments, the adjustment algorithm is a curve fitting algorithm. In exemplary embodiments, the method further includes displaying the volume of a degraded area on a 3D model. In exemplary embodiments, the method further includes identifying an intact area adjacent to the degraded area.

[0163] An exemplary method for calculating articular cartilage wear and external bone loss in the distal femoral condyle for kinematic alignment total knee arthroplasty is to calibrate a radiographer to determine the mapping relationship between image points and corresponding spatial coordinates and define spatial data; to capture a first image of the distal femur using radiographic techniques, wherein the first image defines a first reference frame; and to capture a second image of the distal femur using radiographic techniques, wherein the second image defines a second reference frame, and the first reference frame is This includes offsetting from a second reference frame by an offset angle, identifying spatial data of the distal femur in the first image and spatial data of the distal femur in the second image, converting the spatial data of the distal femur in the first and second images to a single coordinate system and defining the converted spatial data, projecting the converted spatial data of the distal femur onto a display to generate a 3D model of the distal femur, identifying aggravated areas within the 3D model of the distal femur, and applying an adjustment algorithm to calculate the volume of the aggravated areas.

[0164] In exemplary embodiments, the first reference frame is in the anterior-posterior direction. In exemplary embodiments, the second reference frame is in the medial-lateral direction. In exemplary embodiments, the aggravated region includes a bone defect in one of the distal femoral condyles. In exemplary embodiments, the aggravated region includes a cartilage defect on the distal femur. In exemplary embodiments, the adjustment algorithm identifies an intact region adjacent to the aggravated region.

[0165] In an exemplary embodiment, the method further includes identifying an intact region of a contralateral orthopedic element, the intact region of the contralateral orthopedic element corresponding to an aggravated region of the desired orthopedic element. In an exemplary embodiment, the adjustment algorithm is a curve fitting algorithm. In an exemplary embodiment, the method further includes displaying the volume of the aggravated region on a 3D model of the distal femur.

[0166] An exemplary method for calculating the area of ​​bone abnormality includes generating a 3D model of the joint surgery area from at least two 2D images, wherein the first image is captured at a first lateral position and the second image is captured at a second lateral position, the first lateral position being different from the second lateral position; identifying the area of ​​bone abnormality on the 3D model; and calculating a correction region, the correction region being the area of ​​bone abnormality removed relative to the surrounding bone region.

[0167] In exemplary embodiments, the method further includes generating a 3D model of the correction region. In exemplary embodiments, the method further includes generating a physical 3D model of the correction region. In exemplary embodiments, the method further includes generating an orthopedic drill guide, which includes a physical 3D model of the correction region configured to seat in the corresponding region of a negative bone anomaly.

[0168] In exemplary embodiments, the process of generating a physical 3D model of the corrective region is achieved through additive manufacturing technology. In exemplary embodiments, the physical 3D model of the corrective region is manufactured from a material selected from the group consisting of polyamide (i.e., nylon), titanium, cobalt-chromium, or another clinically proven biocompatible material. In exemplary embodiments, the method further includes fixedly engaging the physical 3D model of the corrective region with a surgical instrument. In exemplary embodiments, the surgical instrument is an orthopedic drill guide, and the physical 3D model of the corrective region is configured to seat in the corresponding area of ​​the negative bone anomaly. In exemplary embodiments, the method further includes generating a physical 3D model of the joint surgery region, the physical 3D model of the joint surgery region including one or more bone elements of the joint surgery region.

[0169] An exemplary method for calculating the region of bone abnormality includes using a deep learning network to generate a 3D model of the joint surgery region from at least two 2D images, wherein the first image is captured at a first lateral position and the second image is captured at a second lateral position, the first lateral position being different from the second lateral position; identifying the region of bone abnormality on the 3D model; and calculating a correction region, the correction region being the region of bone abnormality removed relative to the surrounding bone region.

[0170] An exemplary alignment guide includes a main body, a posterior pad extending from the lower part of the main body, a drill bore extending through the main body above the posterior pad, and a patient-specific physical 3D model of a bone anomaly engaged with one of the posterior pads.

[0171] In exemplary embodiments, a patient-specific 3D model of a bone abnormality is generated by any system or method of the present disclosure.

[0172] In an exemplary embodiment, a patient-specific 3D model of a bone abnormality is generated by a process which includes calibrating a radiographer to determine the mapping relationship between radiographic points and corresponding spatial coordinates to define spatial data, capturing a first radiographic image of the target orthopedic element using radiographic imaging techniques, wherein the first radiographic image defines a first reference frame, and capturing a second radiographic image of the target orthopedic element using radiographic imaging techniques, wherein the second radiographic image defines a second reference frame, and the first reference frame is offset from the second reference frame. The method includes offsetting by an angle, projecting spatial data from a first radiographic image of the target orthopedic element and spatial data from a second radiographic image of the target orthopedic element, detecting the target orthopedic element using the spatial data with respect to a deep learning network, wherein the spatial data defines anatomical landmarks on or within the target orthopedic element, detecting bone abnormalities on or within the target orthopedic element using the spatial data with respect to a deep learning network, and applying the deep learning network to the spatial data to generate a 3D model of the bone abnormality.

[0173] In another exemplary embodiment, a patient-specific 3D model of a bone abnormality is generated by a process which includes calibrating a radiographer to determine the mapping relationship between radiographic points and corresponding spatial coordinates to define spatial data, capturing a first radiographic image of the target orthopedic element using radiographic imaging techniques, wherein the first radiographic image defines a first reference frame, and capturing a second radiographic image of the target orthopedic element using radiographic imaging techniques, wherein the second radiographic image defines a second reference frame, and the first reference frame is offset from the second reference frame at an offset angle. The method includes setting, projecting spatial data from a first radiographic image of the target orthopedic element and spatial data from a second radiographic image of the target orthopedic element to define volumetric data, using a deep learning network to detect the target orthopedic element using the volumetric data, wherein the volumetric data defines anatomical landmarks on or within the target orthopedic element, using a deep learning network to detect bone abnormalities on or within the target orthopedic element using the volumetric data, and applying the deep learning network to the volumetric data to generate a 3D model of the bone abnormality.

[0174] In an exemplary embodiment, the physical 3D model of the bone anomaly includes a mating surface that engages with the exposed surface of the worn bone. In an exemplary embodiment, the physical 3D model of the bone anomaly includes a mating surface, the mating surface further includes a projection.

[0175] An exemplary system comprises: a 3D model of an orthopedic element including a surgical region generated from at least two 2D radiographic images, wherein at least a first radiographic image is captured at a first position and at least a second radiographic image is captured at a second position, the first position being different from the second position; and a computer further configured to identify areas of bone abnormalities on the 3D model and to apply a surface adjustment algorithm, the surface adjustment algorithm being configured to remove areas of bone abnormalities from the 3D model and to estimate a bone surface topography to replace the areas of bone abnormalities.

[0176] In exemplary embodiments, the surface adjustment algorithm is a curve fitting algorithm. In exemplary embodiments, the system further comprises a display on which the 3D model is displayed. In exemplary embodiments, the display is an augmented reality or virtual reality device. In exemplary embodiments, the system further comprises an X-ray imager. In specific exemplary embodiments, the system further comprises a manufacturing apparatus configured to generate a physical model of at least a portion of the 3D model.

[0177] In an exemplary embodiment, the manufacturing apparatus is configured to generate a physical model of a bone anomaly. In the exemplary embodiment, the physical model of the bone anomaly is the inverse volume of a negative bone anomaly. In the exemplary embodiment, the manufacturing apparatus is an additive manufacturing apparatus. In the exemplary embodiment, the physical model of the bone anomaly includes medical-grade polyamide.

[0178] The present invention is not limited to the specific configurations and methods disclosed herein or shown in the drawings, but should be understood to include any modifications or equivalents within the claims known in the art. Those skilled in the art will understand that the apparatus and methods disclosed herein will be useful.

Claims

1. A method for generating a 3D model (115m) of a bone abnormality, comprising: calibrating a radiographic imaging device (1800) to determine a mapping relationship between radiographic image points and corresponding spatial coordinates and to define spatial data (43); acquiring a first radiographic image (30) of a target orthopedic element (100) using said radiographic imaging device (1800), said first radiographic image (30) defining a first frame of reference (30a); acquiring a second radiographic image (50) of the target orthopedic element (100) using the radiographic imaging device (1800), the second radiographic image (50) defining a second frame of reference (50a), the first frame of reference (30a) being offset from the second frame of reference (50a) by an offset angle (θ); projecting spatial data (43) from the first radiological image (30) of the target orthopaedic element (100) and spatial data (43) from the second radiological image (50) of the target surgical element (100); detecting the target orthopedic element (100) using the spatial data (43) with a deep learning network, the spatial data (43) defining anatomical landmarks on or within the target orthopedic element (100); detecting bony abnormalities (115) on or within the target orthopedic element (100) using the spatial data (43) with the deep learning network; applying the deep learning network to the spatial data (43) to generate a 3D model (115m) of the bone abnormality; A method for generating a 3D model (115m) of a bone abnormality comprising:

2. The method of claim 1, further comprising using a surface adjustment algorithm to remove a 3D model (115m) of the bone abnormality from the orthopedic element (100) and estimate a bone surface topography (117) to replace the bone abnormality region (115).

3. The method described in claim 2, wherein the surface adjustment algorithm is a curve fitting algorithm.

4. The method of claim 1, further comprising using manufacturing techniques to generate a physical 3D model (115m) of the bone abnormality.

5. The method described in claim 4, wherein the physical 3D model (115m) of the bone abnormality includes a mating surface that mates with an exposed surface (116) of the bone abnormality (115).

6. The method of claim 4, wherein the physical 3D model (115m) of the bone abnormality includes the mating surfaces, and the mating surfaces further include protrusions.

7. A system comprising: A computer (1600), A radiation imaging device (1800); Equipped with The radiation imaging device (1800) storing the first radiographic image (30) of the target orthopaedic element (100), the first radiographic image (30) defining the first frame of reference (30a); storing the second radiographic image (50) of the target orthopedic element (100), the second radiographic image (50) defining the second frame of reference (50a), the first frame of reference (30a) being offset from the second frame of reference (50a) by the offset angle (θ); configured to run The computer includes: projecting the spatial data (43) from the first radiological image (30) of the target surgical element (100) and the spatial data (43) from the second radiological image (50) of the target orthopaedic element (100), the spatial data being defined by the mapping relationship between the radiological image points and the corresponding spatial coordinates; using the deep learning network to detect a target orthopedic element (100) using the spatial data (43), the spatial data (43) defining anatomical landmarks on or within the target orthopedic element (100); detecting, with the deep learning network, the bone abnormality (115) on or within the target orthopedic element (100) using the spatial data (43); applying the deep learning network to the spatial data (43) to generate a 3D model (115m) of the bone abnormality; A system configured to run

8. The system described in claim 7, wherein the computer (1600) is configured to perform the steps of: using a surface adjustment algorithm to remove a 3D model of the bone abnormality from the orthopedic element (100) and estimating a bone surface topography (117) to replace the bone abnormality region (115).

9. The system of claim 8, wherein the surface adjustment algorithm is the curve fitting algorithm.

10. A system described in any one of claims 7 to 9, further comprising a display (19), wherein a 3D model of the orthopedic element (1100) is displayed on the display (19), and the display (19) is an augmented reality device or a virtual reality device.

11. The system described in any one of claims 7 to 10, wherein the radiation imaging device (1800) is an X-ray imaging device.

12. The system described in any one of claims 7 to 11, further comprising a manufacturing device (18) configured to generate the physical model of at least a portion of the 3D model of the orthopedic element (1100), the manufacturing device (18) configured to generate a physical model (115m) of a bone abnormality, and the manufacturing device (18) is an additive manufacturing device.

13. The system described in claim 12, wherein the physical model (115m) of the bone abnormality is the negative inverse of the volume of the bone abnormality (115).

14. The system of claim 13, wherein the physical model (115m) of the bone abnormality comprises medical grade polyamide.

Images