A system and method for using photogrammetry to align surgical elements during surgery.

Abstract

A system and method for verifying a position of an orthopedic element in space includes capturing first and second images of the orthopedic element in different reference frames using radiographic imaging techniques, detecting spatial data defining anatomical landmarks on or within the orthopedic element using a deep learning network, applying a mask to the orthopedic element defined by the anatomical landmarks, projecting the spatial data from the first and second images to define volumetric data, applying the deep learning network to the volumetric data to generate a reconstructed three-dimensional model of the orthopedic element, and mapping the three-dimensional model of the orthopedic element to the spatial data to determine a position of the three-dimensional model of the orthopedic element in three-dimensional space.

Description

[Technical Field]

[0001] Reference to related applications This application claims priority to U.S. Provisional Application No. 63 / 250,906, filed on 30 September 2021. The disclosure of that related application is incorporated in its entirety into this disclosure.

[0002] This disclosure relates, in general, to the field of orthopedic joint replacement surgery, and more specifically to the use of photogrammetry and three-dimensional ("3D") reconstruction techniques to assist surgeons and technicians in planning and performing orthopedic surgery. [Background technology]

[0003] The goal of hip replacement surgery is to restore the patient's natural hip joint position and range of motion as it was before the injury. However, because the hip joint consists not only of articular bone but also various soft tissues including cartilage, muscles, ligaments, and tendons, achieving this goal in practice can be difficult. In all hip replacement surgeries, and especially in minimally invasive hip replacement surgeries, the presence of these soft tissues can severely limit the surgeon's field of view. This problem is even more pronounced in patients with a high body mass index.

[0004] In hip arthroplasty, the pelvis itself is almost entirely surrounded by soft tissue. In minimally invasive procedures, the main incision ultimately exposes the junction of the acetabulum and the proximal femoral head, which typically traverses the rim (i.e., margin) of the acetabulum to direct the surgeon's view. A portal incision extending through one or more quadriceps muscles of the operating leg may align with the concave surface of the acetabulum, but the proximal end of the femur must be moved and rotated away from the acetabulum to expose this view.

[0005] To further complicate matters, the field of view of the surgical area through a portal incision is generally far more limited than that through a primary incision. While an endoscope camera may be positioned through a portal incision to capture images of the concave acetabular surface, the concave surface of the acetabulum lacks bone markers (i.e., landmarks) that can be used to reliably indicate the position of the acetabulum and pelvis. Furthermore, any movement of the femur is likely to be transmitted to the pelvis via connective soft tissue, thereby impairing the usefulness of any images captured by the endoscope camera. Consequently, the use of an endoscope camera unnecessarily prolongs the procedure and has very limited effectiveness in accurately reflecting the position of the acetabulum relative to the proximal femur.

[0006] A prosthesis typically includes an acetabular shell, which the surgeon places within the reamed acetabulum of the hip joint. The acetabular shell may house a liner that essentially functions as a bearing for the substantially spherical head of the femoral component. The femoral component generally includes a stem, a neck, and a head. When placed, the stem is inserted into the resected and reamed proximal end of the femur. The neck connects the proximal end of the stem to the head. The head is then placed within the prosthesis, typically against the liner of the acetabular cup.

[0007] Because the surgical field of view of a surgeon is often obstructed by soft tissue, surgeons in the past have relied on external markers to estimate the proper alignment of the acetabular cup within the acetabulum. U.S. Patent Publication 2013 / 0165941 (Murphy) is one such example. Other providers have offered positioning guides that include external horizontal and vertical positioning bars designed to resemble the axes of the Cartesian plane. To attempt to achieve an acetabular cup abduction angle of about 40° to about 45°, the surgeon positions the positioning guide approximately diagonally with respect to the patient's longitudinal axis of the body (i.e., the imaginary centerline of the body extending from the patient's head to the groin) so that the horizontal positioning bar is positioned approximately parallel to the longitudinal axis of the body. To attempt to achieve an anteversion angle of about 10° to about 15°, the surgeon slightly elevates the positioning device along the vertical positioning bar with respect to the longitudinal axis of the body.

[0008] These external indicators do not take into account the patient's specific anatomical structure, nor do they consider the movement of the pelvis relative to these indicators. For example, when a given patient is lying supine, it is entirely possible that the patient's left acetabulum may be positioned slightly lower than the patient's right acetabulum. Furthermore, many hip arthroplasty procedures require the patient to be repositioned in order to make a specific incision or to access a specific part of the surgical area. As mentioned above, femoral movement is likely to be transmitted to the pelvis via soft tissue. If the patient needs to be repositioned multiple times through standard hip arthroplasty, it is unlikely that the pelvis will always be positioned within the proposed usage parameters for existing acetabular cup positioning guides that rely on external indicators.

[0009] Allowing surgeons to have a direct view of the proximal femur requires removing (and thus displacing) the proximal femur from the acetabulum (or, in some cases, the acetabular cup), making it even more difficult to properly align, size, and place the femoral components. As a result, many surgeons have relied on sound and touch to approach an acceptable femoral stem placement. Both the femoral stem and the acetabular cup are embedded in their respective bones. If the femoral stem is too large, it can easily fracture the proximal femur. If the femoral stem is too small, it can sink into the intramedullary canal of the femur over time as a result of normal use. Sinking can shorten the patient's gait and place excessive pressure on the neck, head, and liner portions, thereby accelerating wear.

[0010] Furthermore, even when the acetabular cup is positioned within the reamed acetabulum at the desired abduction and anteversion angles, and even when an appropriately sized femoral stem is seated within the proximal femur, the position of the femoral components relative to the acetabular cup could not be determined using conventional techniques. While intraoperative fluoroscopy can generate two-dimensional ("2D") images of the femoral components relative to the acetabular components, these fluoroscopic images lacked sufficient 3D information to ensure accurate alignment. For example, in classical fluoroscopy, pelvic tilt was unknown. Consequently, the orientation of any bone landmarks on the pelvis was also unknown. Without accurately determining the pelvic orientation, it was impossible to accurately calculate the position of the natural affected anterior joint line using fluoroscopy alone. Moreover, long-term use of fluoroscopy exposes patients to excessive radiation.

[0011] Improper alignment of the femoral head relative to the acetabular cup can result in shortening of the surgical leg relative to the contralateral leg, displacement of the head relative to the acetabular cup, and increased stress on the acetabular cup, liner, head, or part of the neck (thus increasing the rate of wear and reducing implant life). Any of these drawbacks can contribute to patient discomfort.

[0012] As a result, the surgeon had to maintain the manipulation within a rather large error range for the placement of the acetabular cup. Despite the available tools and procedures, aligning the reconstructed hip joint in a typical hip arthroplasty is based on experience, educated guesswork, and chance. This problem can be particularly pronounced in minimally invasive hip arthroplasty, partly because the surgeon's field of view is so restricted. SUMMARY OF THE INVENTION

[0013] Therefore, there has long been a need, but no solution, to enhance preoperative and intraoperative imaging techniques to accurately model the anatomical structure of the surgical joint and the artificial internal implant when planning and performing hip arthroplasty.

[0014] The problem of limited surgeon visualization of the surgical area in minimally invasive surgery using currently available preoperative or intraoperative tools and techniques, and the associated misalignment problems that can be caused by such lack of visualization, can be alleviated by an exemplary system or method for confirming the position of orthopedic elements within a space, using a deep learning network to identify and model the components of the orthopedic elements and internal implants, and mapping the models of the orthopedic elements and internal implants to spatial data from the input of at least two separate two-dimensional ("2D") input images of the orthopedic elements of interest, wherein the first image of the at least two separate 2D input images is captured from a first lateral position, and the second image of the at least two separate 2D input images is captured from a second lateral position offset from the first lateral position by an offset angle.

[0015] In an exemplary embodiment, the input image can be a radiographic image. Without being bound by theory, radiographs may be desirable because they enable in vivo analysis that can account for the external summation of passive soft tissue structures and dynamic forces occurring around the hip joint, including the effects of ligament restraint, load-bearing forces, and muscle activity.

[0016] Although not bound by theory, it is believed that by mapping models of orthopedic elements and components of an internal artificial implant to spatial data, the positions of the mapped and modeled orthopedic elements can be calculated relative to the mapped and modeled implant components. When this system is applied to two or more orthopedic elements and two or more components of an internal artificial implant, the components of the internal artificial implant can be desirably implanted into their respective orthopedic elements at desired positions, and the respective internal artificial implant components can be desirably aligned with each other.

[0017] It is further contemplated that the specific exemplary systems and methods described herein can be configured to accurately predict the desired sizes of implant components relative to adjacent orthopedic elements.

[0018] It is further contemplated that the specific exemplary systems and methods described herein can be configured to accurately orient the placement of internal artificial implant components relative to the orthopedic elements into which the internal artificial implant components are implanted.

Brief Description of the Drawings

[0019] The above will become apparent from the following more specific description of exemplary embodiments of the disclosure, as illustrated in the accompanying drawings. The drawings are not necessarily to scale, and instead emphasis is placed on illustrating the disclosed embodiments. [Figure 1] A simplified frontal X-ray view of a patient showing exemplary internal artificial hip implants in the right hip joint and the natural left hip joint. [Figure 2] Shows a typical view of a surgeon in minimally invasive hip arthroplasty. [Figure 3] A perspective view of an exemplary acetabular component disposed within a reamed acetabulum. Figure 3 shows the principles of the abduction angle and the anteversion angle. [Figure 4]This shows a poorly positioned acetabular cup and an incorrectly sized femoral stem. [Figure 5] This is a flowchart illustrating the steps of an exemplary method. [Figure 6] This is a flowchart illustrating the steps of a further exemplary method. [Figure 7] This is a schematic diagram of a system that uses a deep learning network to identify the features (e.g., anatomical landmarks) of a target orthopedic element and generate a 3D model of that element. [Figure 8] This is a schematic diagram of a pinhole camera model used to illustrate how the position of a point in 3D space can be determined from two 2D images taken from different reference frames of a calibrated image detector, using the principle of epipolar geometry. [Figure 9A] These are images of the target orthopedic element taken from the front-to-back ("AP") position, illustrating an example calibration jig. [Figure 9B] These are images of the orthopedic elements in Figure 9A, taken at approximately 45° clockwise from the reference frame in Figure 9A using a calibration jig. [Figure 9C] These are images of the orthopedic elements in Figure 9A, taken using a calibration jig at approximately 45° counterclockwise from the reference frame in Figure 9A. [Figure 10] This is a schematic diagram illustrating how a convolutional neural network ("CNN") type deep learning network can be used to identify features, including surface features (e.g., anatomical landmarks), of a target orthopedic element. [Figure 11] This is an exploded view of a modeled internal artificial implant. [Figure 12] This is a schematic diagram of an example system. [Figure 13] This is a schematic diagram of a system configured to generate a model of an orthopedic element and align components of an internal prosthetic implant by acquiring two or more tissue transmission flattening input images of the same orthopedic element from a calibrated detector at an offset angle. [Modes for carrying out the invention]

[0020] The following detailed description of preferred embodiments is provided for illustrative purposes only to aid understanding and is not intended to be exhaustive or to limit the scope and spirit of the invention. The embodiments have been selected and described to best illustrate the principles and practical applications of the invention. Those skilled in the art will recognize that many modifications can be made to the invention disclosed herein without departing from the scope and spirit of the invention.

[0021] Unless otherwise specified, similar reference numerals indicate corresponding parts in several drawings. The drawings illustrate embodiments of various features and components of the present disclosure, but the drawings are not necessarily to scale, and certain features may be exaggerated to better illustrate embodiments of the present disclosure, and such examples should not be construed as limiting the scope of the present disclosure.

[0022] Unless otherwise expressly stated herein, the following rules of interpretation shall apply herein: (a) All words used herein shall be construed as having the gender or number (singular or plural) required in such context. (b) The singular terms “a,” “an,” and “the” used herein and in the appended claims shall include plural references unless otherwise clearly stated in context. (c) The antecedent “about” applied to any enumerated range or value shall indicate an approximation of a range or value having deviations from known in the art or expected from measurement. (d) Unless otherwise explicitly stated, the words “herein, hereby, hereto,” “hereinbefore,” and “hereinafter,” and words of similar meaning, refer to the entire herein and not to any particular paragraph, claim, or other detail. (e) Explanatory headings are for convenience only and do not control or affect the meaning of any part of this specification. (f) “or” and “any” are not exclusive, and “include” and “including” are not restrictive. Furthermore, "comprising," "having," "including," and "containing" should be interpreted as unrestricted terms (i.e., meaning "including but not limited to...").

[0023] References in this specification such as "one embodiment" or "exemplary embodiment" indicate that the described embodiments may include certain features, structures, or characteristics, but not all embodiments may necessarily include such features, structures, or characteristics. Furthermore, such phrases do not necessarily refer to the same embodiment. Moreover, where certain features, structures, or characteristics are described in relation to an embodiment, it is implied that any influence on such features, structures, or characteristics in relation to other embodiments, whether explicitly stated or not, is within the knowledge of those skilled in the art.

[0024] To the extent necessary to provide descriptive support, the subject matter and / or text of the attached claims are incorporated herein by reference in their entirety.

[0025] The enumeration of value ranges in this specification is merely intended to serve as a simple means of individually referring to each distinct value within any subrange between them, unless otherwise explicitly indicated herein. Each distinct value within an enumerated range is incorporated herein or into the claims as if each distinct value were individually enumerated herein. Where a particular range of values ​​is provided, each intervening value up to one-tenth of the lower limit between the upper and lower limits of that range, and any other stated or intervening values ​​within the described range of that subrange, are understood to be included herein unless the context explicitly indicates otherwise. All subranges are also included. The upper and lower limits of these smaller ranges are also included therein, subject to any specific and explicitly excluded limitations within the described range.

[0026] It should be noted that some of the terms used herein are relative. For example, the terms “upper” and “lower” are spatially relative to each other; that is, the upper component is located higher than the lower component in its respective orientation, but these terms may change if the orientation is reversed.

[0027] The terms “horizontal” and “vertical” are used to indicate direction relative to an absolute reference point, i.e., the Earth’s surface level. However, these terms should not be interpreted as requiring structures to be absolutely parallel or absolutely perpendicular to each other. For example, a first vertical structure and a second vertical structure are not necessarily parallel to each other. The terms “upper” and “lower” or “base” are used to refer to a place or surface where the upper is always higher relative to an absolute reference point, i.e., the Earth’s surface, than the lower or base. The terms “upward” and “downward” are also relative to an absolute reference point. An upward flow always opposes Earth’s gravity.

[0028] Orthopedic procedures often involve surgery on a patient's joints. It will be understood that joints typically contain numerous orthopedic elements. It will be further understood that the exemplary methods and systems described herein may be applied to a variety of orthopedic elements. The examples described with reference to Figures 1–4, 9A–9C, and 11 relate to the hip joint for illustrative purposes. It will be understood that the “orthopedic elements”100 referred to throughout this disclosure are not limited to the anatomical structure of the hip joint but may include any skeletal structure or associated soft tissue such as tendons, ligaments, cartilage, and muscles. An unexclusive list of examples of skeletal orthopedic elements100 includes, but are not limited to, the femur, tibia, pelvis, vertebrae, humerus, ulna, radius, scapula, skull, fibula, clavicle, mandible, ribs, carpal bones, metacarpal bones, tarsal bones, metatarsal bones, phalanges, or any associated tendons, ligaments, skin, cartilage, or muscles, including but not limited to any partial or complete bone of the body. It will be understood that the exemplary surgical area 170 may include several target orthopedic elements 100. Similarly, it will be understood that the surgical area 170 is not limited to the hip surgical area used as the primary example herein, but rather can return to any area of ​​the body that is the target of surgery. This may include, in non-limiting examples, the knee, ankle, spine, shoulder, wrist, hand, foot, mandible, skull, ribs, and phalanges.

[0029] Figure 1 is a simplified representation of an anterior radiographic image of an exemplary patient's pelvis 110. Both the patient's right hip joint 101a and left hip joint 101b are shown. Both exemplary hip joints 101a and 101b contain numerous orthopedic elements 100, including the femur 105, the acetabulum of the pelvis 110 (see 108 and 111), and connective tissue. The illustrated right hip joint 101a (i.e., the patient's right hip joint shown on the left side of the page) shows an exemplary internal prosthesis 102 surgically placed in the patient. The illustrated left hip joint 101b shows an exemplary natural hip joint for comparison.

[0030] Referring to the illustrated right hip joint 101a, the exemplary internal prosthesis hip implant 102 generally includes an acetabular component 103 and a femoral component 104. It will be understood that an internal prosthesis implant can generally include multiple components (e.g., an acetabular component 103 and a femoral component 104), and these components may consist of multiple subcomponents. In the illustrated example, the acetabular component 103 typically includes a generally hemispherical acetabular shell 106 and an internal liner 107. The acetabular shell 106 is typically made from cobalt-chromium, titanium, or other biocompatible metals. The internal liner 107 is typically made from ceramic, metal, polymer, or other biocompatible material having a low coefficient of friction and low wear rate.

[0031] To prepare the natural acetabulum (see 108) for the placement of the acetabular shell 106, the surgeon first uses a hemispherical reamer to create a roughly concave surface within the patient's natural acetabulum 108, defining a “reamed acetabulum” 111. The reamed acetabulum 111 is generally complementary to the convex outer surface 109 of the acetabular shell 106. The outer surface 109 of the acetabular shell 106 is usually roughened to facilitate engagement with the reamed acetabulum 111. The roughened surface is also thought to promote bone formation into the spaces of the roughened surface, thereby increasing the strength of the joint over time.

[0032] The medial liner 107 is typically located adjacent to the medial concave surface 112 of the acetabular shell 106 when the medial liner 107 is in its assembled and installed configuration. Generally, when installed, the medial liner 107 functions as a bearing upon which the femoral head 113 of the femoral component 104 articulates.

[0033] The femoral component 104 typically includes a femoral stem 115 having a distal stem end 115a positioned distal to a proximal stem end 115b, and a neck 116 having a distal cervical end 116a that engages with the proximal stem end 115b. The neck 116 extends to a cranial end 116b. A generally spherical artificial femoral head 113 is positioned at the cranial end 116b of the neck 116 in the assembled configuration. In certain exemplary embodiments, the neck 116 may be selectively removable from the proximal stem end 115b. Such a selectively removable neck 116 may be known as a “modular neck”.

[0034] It will be understood that the acetabular component 103 and the femoral component 104, as well as the subcomponents including the acetabular component 103 or the femoral component 104 (e.g., acetabular shell 106, medial liner 107, optional fixation fasteners, femoral stem 115, artificial femoral head 113, etc.), are typically supplied in one or more surgical kits as disassembled and unassembled configurations. In an unassembled and uninstalled configuration, components or subcomponents do not physically engage with other components or subcomponents. In other words, forces are not transmitted directly from one component or subcomponent to another in an unassembled and uninstalled configuration. In an assembled configuration, components or subcomponents are in physical contact with each other, and forces can be transmitted through two or more proximal components or subcomponents. In an assembled and installed configuration, components or subcomponents are in an assembled configuration and are surgically implanted in the patient.

[0035] For comparison, the illustrated left hip joint 101b shows the natural femoral head 126 at the proximal end of the femur 105. The natural femoral head 126 is located within the natural acetabulum 108 of the pelvis 110. Articular cartilage 123 covers the articular surfaces of both the healthy femoral head 126 and the healthy acetabulum 108.

[0036] While there are many surgical approaches to typical hip arthroplasty, most minimally invasive procedures begin with the surgeon making a 6-8 cm incision in the surgical leg, radially adjacent to the hip capsule. Various muscles and tendons are then retracted using surgical instruments, ultimately exposing the joint capsule. The capsule is then perforated, and the surgeon displaces the natural femoral head from the natural acetabulum.

[0037] Preparation of the femur involves excising and removing the natural femoral head 126 from the femur 105. After the removal of the natural femoral head 126, the surgeon may perforate a canal into the intramedullary space of the newly exposed proximal end 105b of the femur 105. The surgeon can then use a femoral broach to expand the space within the intramedullary canal necessary to accommodate the femoral stem 115. A trial stem can be used to test the sizing and positioning of the femoral component 104. The trial component generally has the same dimensions as the actual implant component, but is designed to be easier to install and remove.

[0038] Preparation of the acetabulum involves reaming the natural acetabulum 108 to define the reamed acetabulum 111. The goal is to form a substantially uniform hemispherical space complementary to the substantially hemispherical outer surface 109 of the acetabular shell 106. While it is possible to attempt to test the alignment of the acetabular component 103 relative to the femoral component 104 using a trial acetabular component, visibility is limited and the nature of the procedure typically does not allow for comprehensive testing of the alignment. Furthermore, because visibility is limited, the actual implant components 103, 104 may not be oriented in exactly the same way as the trial components.

[0039] It will be understood that there are various surgical approaches for a typical total hip arthroplasty (for example, some surgeons choose to approach the hip posteriorly, while others choose to approach it laterally or anteriorly). Figure 2 illustrates and illustrates a typical surgeon's view of the surgical area 170 of a typical hip arthroplasty through the main incision. Several retractors 14, 16 (which may include a Hoffman retractor or Cobb elevator in some procedures) are used to contract the fascia 11 located between the initial incision area and the hip capsule. An electrocautery device 40 may be used to excise and cauterize the tissue and prevent excessive bleeding. The natural femoral head 126 is also shown for reference.

[0040] Figure 2 illustrates how the 6–8 cm main incision in the surgical area 170, the position of the hip joint relative to the incision point (see 101a, 101b), and the presence of typical surgical instruments (e.g., retractors 14, 16, mallets, broaches, reamers, pins, implant components, etc.) can significantly obstruct the surgeon's already limited field of view. This problem can be exacerbated by attempting to align the acetabular component 103 and femoral component 104 of the internal prosthesis hip implant 102 with external markers separated from the orientation of the target transplant anatomical structure (e.g., in this hip example, the reamed acetabulum 111 or the resected proximal femur 105), which can lead to inaccurate alignment of the implant components 103, 104 with respect to the bone they are transplanted into and with respect to each other. This, in turn, can contribute to the risk of implant displacement, suboptimal force distribution, faster wear, implant failure, altered gait, general patient discomfort, and the need for further corrective surgery with the same limitations.

[0041] Figure 3 is a perspective view of an exemplary acetabular component 103 positioned within a reamed acetabulum 111. The abduction angle α and anteversion angle υ are shown with respect to the acetabular component 103, but it will be understood that the femoral component 104 is also positioned with an abduction angle α and anteversion angle υ in the proximal femur 105. The abduction angle α and anteversion angle υ of the components of the internal prosthesis (e.g., the acetabular component 103 and the femoral component 104) relative to the orthopedic elements (e.g., the pelvis 110 and 105 and the proximal femur, respectively) into which the components of the internal prosthesis are implanted can be calculated and determined by those skilled in the art.

[0042] It will be understood that the “components of the internal prosthesis” may vary depending on the type of internal prosthesis and the type of surgical area 170. For example, if the surgical area 170 is the hip joint, the “components of the internal prosthesis” may be selected from the group including the acetabular component 103, the femoral component 104, a trial construct, instruments used for or to facilitate the placement of the internal prosthesis or trial implant at the patient placement site, or combinations thereof. In embodiments where the surgical area 170 is the knee, the “components of the internal prosthesis” may be the femoral component of an internal knee prosthesis, the tibial component of an internal knee prosthesis, a trial construct, instruments used for or to facilitate the placement of the internal prosthesis or trial implant at the patient placement site, or combinations thereof.

[0043] To more clearly illustrate the principle of the abduction angle α of the acetabular component 103 relative to the pelvis 110, soft tissues are omitted in Figure 3. The abduction angle α can be measured by several methods known to those skilled in the art. One such method for visualizing the abduction angle α of the acetabular component 103 is to draw a diameter line D extending through the diameter of the rim of the acetabular shell 106 on the coronal plane CP, relative to a roughly horizontal medial-lateral reference line R that lies coplane with the coronal plane CP of the diameter line D. In Figure 3, a reference line R connecting the most distal portions 117a and 117b of the left and right ischial tuberosities is shown. However, it will be understood that other reference markers may be used, as long as the reference line R extends horizontally, medial-laterally, and coronally on the same plane as the diameter line D.

[0044] The shell plane SP is also shown to extend coplanarly through the edge 2 of the acetabular shell 106. Aligning the acetabular shell 106 in three-dimensional space can be considered to involve the selection of an appropriate composite angle, which includes the abduction angle α and the anteversion angle υ. The shell plane SP is shown to more clearly illustrate the concept of acetabular shell alignment in three dimensions. The diameter line D, coronal plane CP, shell plane SP, and medial-lateral reference line R will be understood as geometric reference elements drawn to generally illustrate the abduction angle α and the concept of acetabular alignment. These geometric reference elements do not actually need to be visible.

[0045] Many acetabular shells 106 are designed to fit into a reamed acetabulum 111 with an abduction angle α of approximately 30° to 50°. However, this wide margin highlights the difficulty in properly positioning the acetabular shell 106 within a reamed acetabulum 111 using conventional methods. Furthermore, the general guidance of having an abduction angle α of approximately 30° to 50° does not account for the variability in specific patients.

[0046] Figure 3 also illustrates the concept of the anteversion angle υ. The anteversion angle υ can be calculated by several methods known to those skilled in the art. One such method for visualizing the anteversion angle υ of the acetabular shell 106 is to imagine it as the rotation of the acetabular shell 106 around the central diameter line D used in the visualization of the abduction angle α. A typical acetabular shell 106 may have an anteversion angle υ in the range of approximately 10° to approximately 30°, or approximately 10° to approximately 20°, or approximately 15° to approximately 25°. In practice, it will be understood that the alignment of the acetabular shell 106 within the reamed acetabulum 111 is a composite angle that includes both the abduction angle α and the anteversion angle υ. Similarly, the alignment of the femoral stem 115 within the intramedullary bore 119 is a composite angle that includes both the abduction angle α and the anteversion angle υ.

[0047] Having a femoral stem 115 that aligns with the acetabular shell 106 along a common anteversion angle (or anterior tilt plane) is one of the alignment parameters for a properly aligned artificial hip implant 102; therefore, the anteversion angle υ of the femoral stem 115 typically has the same range of values ​​as the anteversion angle υ of the acetabular shell 106 (i.e., in the range of approximately 10° to approximately 30°, or approximately 10° to approximately 20°, or approximately 15° to approximately 25°). Positioning the femoral stem 115 in the intramedullary canal of the proximal femur 105 such that the longitudinal axis of the femoral stem 115 is collinear with the anatomical axis of the femur 105 in which the femoral stem 115 is positioned is another alignment parameter for a properly aligned femoral component 104 to properly aligned acetabular component 103 to define a properly aligned artificial hip implant 102 together. A third alignment parameter for the femoral stem 115 is the vertical position of the artificial femoral head 113 relative to the natural femoral head of the surgical hip joint prior to resection (see 126).

[0048] Figure 4 shows an acetabular component 103 that is not aligned with the reamed acetabulum 111 and a femoral component 104 that is too short for the reamed femoral canal (also known as the intramedullary bore 119). As shown in the figure, the abduction angle α and anteversion angle υ are excessive. When the patient moves their hip joint during normal use, the neck 116 may come into contact with the edge 2 of the acetabular shell 106. Collectively, the edge 2 and the neck 116 can become fulcrums of a lever that can displace the artificial femoral head 113 from the acetabular component 103. In addition, even if the femoral component 104 does not displace from the acetabular component 103, the force distribution of the femoral head 113 against the medial liner 107 may be excessively concentrated in a relatively small area, thereby increasing wear and shortening the lifespan of the implant 102.

[0049] Figure 4 also shows a femoral stem 115 that is improperly sized and aligned with respect to the intramedullary bore 119. Improper sizing can occur when the femoral reaming (also known as "broaching") tool does not create an intramedullary bore 119 large enough to remove the cancellous bone surrounding the medial cortical wall 120 of the femur 105. Over time, the femoral stem 115 compresses any intermediate cancellous bone between the lateral surface of the femoral stem 115 and the medial cortical wall 120, which causes the femoral stem to sink within the intramedullary bore 119 and displace relative to the proximal femur 105. Variations in broach placement can also result in a varus inclination, where the longitudinal axis of the femoral stem 115 is positioned at a varus angle with respect to the anatomical axis (i.e., the central axis, or longitudinal axis) of the distal femur 105. For example, it is common for a surgeon to make contact with the lateral cortical wall 120 of the intramedullary canal 119 over the desired position of the femoral stem 115. The surgeon may stop reaming once they make contact with the lateral cortical wall 120 of the intramedullary canal 119, assuming that the patient has a narrow intramedullary canal 119 that accommodates only a small femoral stem 115. In practice, the longitudinal axis of the femoral stem 115 is positioned at a varus angle with respect to the anatomical axis of the distal femur 105. This sinking and misalignment can ultimately alter the length of one of the patient's legs relative to the other, and consequently alter the patient's gait. The stopped gait alters the force distribution through the patient's body, which can further accelerate wear of the internal prosthesis implant 102 as well as wear of the healthy cartilage 123 on the remaining natural hip joint 101b.

[0050] The sinking and misalignment of femoral components 104 relative to the distal femur 105 can be particularly difficult to achieve and confirm with conventional 2D radiography. This is because femoral components 104 are inserted into the proximal femur 105 through a 6-8 inch main incision. The surgeon's field of view during insertion is limited by the minimally invasive nature of the procedure, and once the femoral components enter the intramedullary bore 119, they are no longer visible to the unassisted eye. Conventional 2D intraoperative radiography (such as fluoroscopic images) does not show three dimensions and therefore cannot provide an accurate, real-world depiction of the surgical area 170 in 3D space.

[0051] Patient comfort and implant lifespan are thought to depend in part on the placement and size of the artificial hip joint implant 102. Generally, the more closely the placement of an appropriately sized implant replicates the natural movement of the joint before the injury, the longer the implant is expected to last, and the more comfortable the patient is expected to feel.

[0052] In recent years, it has become possible to create 3D models of surgical areas using 2D images such as X-rays. These models can be used preoperatively to plan surgeries much closer to the actual date of the operation. These models can also be used intraoperatively (for example, when projected onto a display or projected across the surgeon's field of view).

[0053] However, X-ray images have not traditionally been used as input for 3D models, typically due to concerns regarding image resolution and accuracy. An X-ray image is a 2D representation of a 3D space. Therefore, in a 2D X-ray image, the image object will inevitably be distorted compared to the actual object existing in three dimensions. Furthermore, the object through which the X-rays pass can deflect the path of the X-rays as it moves from the X-ray source 21 (typically the anode of the X-ray machine; see Figure 12) to the X-ray detector 33 (as a non-limiting example, an X-ray image intensifier, phosphorus material, a flat panel detector "FPD" (which may include indirect and direct conversion FPDs), or any number of digital or analog X-ray sensors or X-ray film; see Figure 12). Defects in the X-ray machine itself (1800, see Figure 12) or its calibration can also impair the usefulness of X-ray photogrammetry and 3D model reconstruction. In addition, emitted X-ray photons have different energies. When X-rays interact with material placed between the X-ray source 21 and the detector 33, noise and artifacts may be generated in part due to Compton scattering and Rayleigh scattering, the photoelectric effect, extrinsic variations in the environment, or intrinsic variations in the X-ray generation unit, X-ray detector, and / or processing unit or display.

[0054] Furthermore, a single 2D image loses the 3D data of the actual object. Therefore, there is no data from a single 2D image that a computer (e.g., a PC) can use to reconstruct a 3D model of the actual 3D object. For this reason, CT scans, MRI, and other imaging techniques that preserve three-dimensional data have often been preferred inputs for reconstructing models of one or more orthopedic elements of an object (i.e., reconstructing 3D models from actual 3D data to generally produce more accurate and higher-resolution models). However, certain exemplary embodiments of the present disclosure, discussed below, overcome these problems by using a deep learning network to improve the accuracy of the reconstructed 3D models generated from X-ray input images.

[0055] There are various methods for generating a 3D model from 2D preoperative or intraoperative images. For example, one such method may involve receiving a set of 2D radiographic images of the patient's surgical area 170 using a radiographic imaging system, and then computing a first 3D model using the epipolar geometry principle with the coordinate system of the radiographic imaging system and projective geometric data from each 2D image (see Figures 8 and 9A, 9B, and 9C). Such an exemplary method may further involve projecting the first 3D model onto the 2D radiographic images, and then adjusting the initial 3D model by aligning first and second radiographic images 30, 50 onto the first 3D model using an image-to-image alignment technique. Once the image-to-image alignment technique is applied, a modified 3D model may be generated. This process may be repeated until the desired clarity is achieved.

[0056] Another example is deep learning networks such as convolutional neural networks (CNNs), recurrent neural networks (RNNs), modular neural networks, or Sequence-to-Sequence models. A deep learning network (also known as a DNN) can be used to generate a 3D model of a target orthopedic element (i.e., a modeled orthopedic element 100b) from a set of at least two 2D images of the patient's surgical region 170. The 2D input images 30, 50, etc., are preferably tissue transmission images such as radiographic images (e.g., X-ray or fluoroscopic images). In such a method, the deep learning network can generate a model from projective geometric data (i.e., spatial data 43 or volumetric data 75) from each 2D image. The deep learning network may have the advantage of being able to generate masks of different target orthopedic elements 100 (e.g., bone, soft tissue, etc.) in the surgical region 170, as well as being able to calculate the volume of one or more imaged orthopedic elements 100 (see 61 in Figure 7). In exemplary embodiments, the identified orthopedic element 100 or internal The dimensions of the components of the artificial implant assembly 102 are mapped to spatial data 43 (Figure 8) derived from input images 30, 50 (Figure 8) to confirm the position of the identified orthopedic element 100 or the components of the internal artificial implant assembly in 3D space. In this way, the positions of the identified orthopedic element 100 and the components of the internal artificial implant (e.g., the acetabular component 104 or the femoral component 103) can be confirmed relative to each other. If this information is displayed to the surgeon and updated in real time or near real time based on the surgeon's repositioning of the implant components relative to the identified orthopedic element, the surgeon can use the exemplary embodiments of this disclosure to accurately align the implant components with the identified orthopedic element in three dimensions while avoiding the limited field of view provided by the main incision.

[0057] Once the system is calibrated as described below, it is intended that new tissue radiographs (i.e., fewer than the number of input images required to calibrate the system) can be acquired intraoperatively to update the reconstructed model of the surgical area (for example, to refresh the position of an identified component of an internal prosthesis relative to another component of the internal prosthesis or to an identified orthopedic element within the system). In other exemplary embodiments, the position of a component of an internal prosthesis relative to another component of the internal prosthesis or to an identified orthopedic element within the system can be refreshed using the same number of new tissue radiographs as the number of input images selected to calibrate the system.

[0058] Figure 5 is a flowchart outlining the steps of an exemplary method for determining the location of orthopedic elements in space. The method involves image points (e.g., X L , X RStep 1a: To determine the mapping relationship between the (Figure 8) and the corresponding spatial coordinates (e.g., x and y coordinates; Figure 8), a tissue transmission device such as a radiographer 1800 is calibrated to define spatial data 43; a first image 30 (Figure 8) of the orthopedic element 100 is captured using radiographic imaging technology, wherein the first image 30 defines a first reference frame 30a; a second image 50 (Figure 8) of the orthopedic element 100 is captured using radiographic imaging technology, wherein the second image 50 defines a second reference frame 50a, and the first reference frame 30a is offset from the second reference frame 50a by an offset angle θ; and the orthopedic element is detected using spatial data 43. A step of using a deep learning network to define spatial data 43, wherein spatial data 43 defines anatomical landmarks on or within an orthopedic element 100, and the detected orthopedic element defines the identified orthopedic element 100a; a step 4a of using a deep learning network to apply a mask to the identified orthopedic element 100a defined by the anatomical landmarks; and a step of projecting spatial data 43 from a first image 30 of the identified orthopedic element 100a and spatial data 43 from a second image 50 of the identified orthopedic element 100a to define volumetric data 75 (Figure 7), wherein image points (e.g., X) located within a masked region of either the first image 30 or the second image 50 are used. L , X R The spatial data 43, which includes ), has a first value and is an image point (e.g., X) located outside the masked region of either the first image 30 or the second image 50. L , X RThe process includes steps 6a, in which spatial data 43 containing ) has a second value, and the first value is different from the second value; step 7a, in which a deep learning network is applied to volumetric data 75 to define the modeled orthopedic element 100b to generate a reconstructed 3D model of the orthopedic element; and step 8a, in which the three-dimensional modeled orthopedic element 100b is mapped to the spatial data 43. In other exemplary embodiments, step 4a may include using a deep learning network to detect spatial data 43 that define anatomical landmarks on or within the orthopedic element 100.

[0059] Figure 6 is a flowchart outlining the steps of another exemplary method for determining the location of orthopedic elements in space. The method involves image points (e.g., X L , X R Step 1b: Calibrate a tissue transmission imaging device such as a radiographer to determine the mapping relationship between the orthopedic element 100 and the corresponding spatial coordinates (e.g., x-coordinate and y-coordinate) and define spatial data 43; Step 2b: Capture a first image 30 of the orthopedic element 100 using radiographic imaging technology, wherein the first image 30 defines a first reference frame 30a; Step 3b: Capture a second image 50 of the orthopedic element 100 using radiographic imaging technology, wherein the second image 50 defines a second reference frame 50a, and the first reference frame 30a is offset from the second reference frame 50a by an offset angle θ; and Use a deep learning network to identify the A step of detecting an orthopedic element 100 using spatial data 43 to define an orthopedic element 100a, comprising: step 4b, wherein the spatial data 43 defines anatomical landmarks on or within the orthopedic element 100; step 5b, which uses a deep learning network to apply a mask to the identified orthopedic element 100a defined by the anatomical landmarks; and step of projecting spatial data 43 from a first image 30 of the identified orthopedic element 100a and spatial data 43 from a second image 50 of the identified orthopedic element 100a to define volumetric data 75, wherein image points (e.g., X) are placed within a masked region of either the first image 30 or the second image 50.L , X R ) The spatial data 43 including an image point (e.g., X L , X R ) outside the masked area of either the first image 30 or the second image 50 has a second value, and the first value is different from the second value, step 6b, step 7b of applying a deep learning network to the volume data 75 to generate a reconstructed 3D model of the orthopedic element to define the orthopedic element 100b, and step 8b of mapping the modeled orthopedic element 100b to the spatial data 43, wherein the orthopedic element is the reamed acetabulum 111 of the pelvis 110. In other exemplary embodiments, step 4b can include detecting spatial data 43 that defines an anatomical landmark on or within the identified orthopedic element 100a using a deep learning network.

[0060] In certain exemplary embodiments, it will be understood that the deep learning network can be the same deep learning network separately trained to perform individual tasks (e.g., identification of the orthopedic element 100 to define the identified orthopedic element 100a, applying a mask to the identified orthopedic element 100a, modeling the identified orthopedic element 100a to define the modeled orthopedic element 100b, etc.). In other exemplary embodiments, different deep learning networks can be used to perform one or more of the discrete tasks.

[0061] The exemplary methods and systems described herein are intended to be used in connection with total hip arthroplasty ("THA"). In such exemplary embodiments, the orthopedic element 100 may be the femur 105, the femoral head 126, the pelvis 110, the acetabular cavity of the pelvis (e.g., the natural acetabulum 108 or the reamed acetabulum 111), and anatomical landmarks of other bones located within or near the surgical area 170. However, it will be understood that nothing in this disclosure limits the application of the exemplary systems and methods for use in THA procedures. The exemplary systems and methods are intended to be useful in any surgical procedure in which the presence of a considerable amount of tissue obstructs the overall view of the orthopedic element 100 or the surgical area 170. Surgeries involving the shoulder, knee, or spine may be primary examples. Pediatric cardiothoracic procedures may be another example. The systems and methods described herein may also be useful in wrist and ankle procedures, even though the surgeon's view is generally less obscured by surrounding tissue than in shoulder, hip, and spine procedures.

[0062] The above example is provided for illustrative purposes only and is not intended to limit the scope of this disclosure. All methods for generating a 3D model from 2D radiographic images of the same object taken from at least two lateral positions are considered to be within the scope of this disclosure.

[0063] Figures 7 and 8 illustrate how a volume 61 containing volume data 75 can be created by combining a first input image 30 and a second input image 50 (Figure 7). In Figure 7, the imaged surgical region 170 is that of the knee joint. Figure 7 provides an example of how a deep learning network can acquire volume data 75 from two calibrated input images 30, 50 offset from each other by an offset angle θ, and generate one or more modeled orthopedic elements 100b from the volume data 75. In Figure 7, the surgical region 170 is the region of the knee joint.

[0064] Figure 8 illustrates the basic principle of epipolar geometry, which can be used to convert spatial data 43 from each input image 30, 50 into volumetric data 75. The spatial data 43 is mapped to image points (e.g., x and y coordinates) of the given input images 30, 50. L , X R It will be understood that it is defined by the set of ).

[0065] Figure 8 is a simplified schematic diagram of an oblique projection described by a pinhole camera model. While Figure 8 conveys a basic concept related to computer stereo vision, it does not imply that a 3D model can be reconstructed from a 2D stereo image in only one way. In this simplified model, the light rays are emitted from the optical center (i.e., a point in the lens where electromagnetic radiation rays from the object (e.g., visible light, X-rays, etc.) are assumed to intersect within the sensor or detector array 33 (Figure 12) of the imager). The optical center is point O in Figure 8. L , O R It is represented as follows. In reality, the image plane (see 30a, 50a) is usually the optical center (e.g., O L , O R The actual optical center is located behind the detector array 33 and is projected as a point onto the detector array 33, but a virtual image plane (see 30a, 50a) is presented here to illustrate the principle more simply.

[0066] The first input image 30 is taken from a first reference frame 30a, and the second input image 50 is taken from a second reference frame 50a, which is different from the first reference frame 30a. Each image contains a matrix of pixel values. The first and second reference frames 30a and 50a are preferably offset from each other by an offset angle θ. The offset angle θ may represent the angle between the x-axis of the first reference frame 30a and the x-axis of the second reference frame 50a. In other words, the angle between the orientation of the orthopedic elements in the first image and the orthopedic elements in the second image may be known as the "offset angle".

[0067] point eL The optical center O of the second input image on the first input image 30 R This is the location of point e. R The optical center O of the first input image on the second input image 50 is L This is the location of point e. L and e R This is known as the "epipol" or epipolar point, and is located on line O L -O R It is located above. Point X, O L , O R This defines the epipolar plane.

[0068] The actual optical center is the point where the incident rays of electromagnetic radiation from the object are expected to intersect within the detector lens. Therefore, in this model, the electromagnetic radiation rays are actually located at the optical center O for the purpose of visualizing how the position of a 3D point X in 3D space can be determined from two or more input images 30, 50 captured from detectors 33 at known relative positions. L , O R It can be thought that it is emitted from each point of the first input image 30 (for example, X L If ) corresponds to a line in 3D space, then the corresponding point (for example, X R If these corresponding points (e.g., X) can be detected in the second input image, then L , X R ) must be a projection of a common 3D point X. Therefore, the corresponding image point (e.g., X) L , X R The lines generated by ) must intersect at 3D point X. Generally, the value of X must be such that all corresponding image points in two or more input images 30, 50 (for example, X L , X RWhen calculated for a 3D volume 61 containing volume data 75, the 3D volume 61 can be replicated from two or more input images 30, 50. The value of any given 3D point X can be triangulated in various ways. A non-limiting list of exemplary calculation methods includes the midpoint method, the direct linear transformation method, the elementary matrix method, the intersection method, and the bundle adjustment method. Furthermore, in certain exemplary embodiments, a deep learning network may be trained on a set of input images to establish a model for determining the position of a given point in 3D space based on two or more input images of the same object, where the first input image 30 is offset from the second input image 50 by an offset angle θ. It will be further understood that any combination of the above methods is within the scope of this disclosure.

[0069] "Image points" as described herein (for example, X L , X R It will be understood that a 3D point X can refer to a point in space, a pixel, a portion of a pixel, or a set of adjacent pixels. When used herein, it will also be understood that a 3D point X can represent a point in 3D space. In certain exemplary uses, a 3D point X can be represented as a voxel, a portion of a voxel, or a set of adjacent voxels.

[0070] However, before the principle of epipolar geometry can be applied, the position of each image detector 33 relative to other image detectors 33(or more) must be determined (or the position of a single image detector 33 must be determined when the first image 30 is taken, and the adjusted position of a single image detector 33 must be known when the second image 50 is taken). It may also be desirable to determine the focal length and optical center of the imager 1800. To actually verify this, the image detector 33(or more) is first calibrated. Figures 9A, 9B, and 9C show calibration fixtures 973A, 973B, and 973C for the orthopedic element 100 in question. In these figures, the exemplary orthopedic element 100 includes the proximal surface of the femur 105 and the natural acetabulum 108 of the pelvis 110, which includes the hip joint 101.

[0071] At least two input images 30, 50 are technically required to calibrate the exemplary system described herein, but at least three input images 30, 50, 70 may be desirable if the input images are radiographic input images and the target surgical area 170 involves contralateral joints that cannot be easily isolated from radiographic imaging. For example, the pelvis 110 includes the contralateral acetabulum 108. A direct medial-lateral radiograph of the pelvis 110 would show both the acetabulum proximal to the detector 33 and the acetabulum distal to the detector 33. However, due to the positioning of the pelvis 110 relative to the detector 33, and because a single 2D radiograph lacks 3D data, the relative acetabular joints appear superimposed on each other, making it difficult for a person or computer 1600 to distinguish which is the proximal acetabulum and which is the distal acetabulum.

[0072] To address this problem, at least three input images 30, 50, and 70 can be used. In one exemplary embodiment, the first input image 30 may be a radiograph capturing an anterior-posterior oblique view of the surgical area 170 (i.e., an example of a first reference frame 30a). For the second input image 50, the patient or detector 33 can be rotated clockwise (which can be specified by a positive angle) or counterclockwise (which can be specified by a negative angle) with respect to the patient's orientation relative to the first input image 30. For example, for the second input image 50, the patient may be rotated ±45° from the patient's orientation in the first input image 30. Similarly, the patient can be rotated clockwise or counterclockwise with respect to the patient's orientation relative to the first input image 30. For example, for the third input image 70, the patient may be rotated ±45° from the patient's orientation in the first input image 30. If the second input image 50 has a positive offset angle (e.g., +45°) with respect to the orientation of the first input image 30, then it will be understood that the third input angle 70 preferably has a negative offset angle (e.g., -45°) with respect to the orientation of the first input image 30, and vice versa.

[0073] In exemplary embodiments, the principle or epipolar geometry can be applied to at least three input images 30, 50, 70 taken from at least three different reference frames 30a, 50a, 70a in order to calibrate the exemplary system.

[0074] Figure 9A is an anterior-posterior view of an exemplary orthopedic element 100 (e.g., proximal femur 105, natural acetabulum 108, pelvis 110, articular cartilage, other soft tissues, etc.) within an exemplary surgical area 170. That is, Figure 9A represents a first image 30 taken from a first reference frame 30a (e.g., a first lateral position). A first calibration fixture 973A is attached to a first retaining assembly 974A. The first retaining assembly 974A may include a first padded support 971A engaged with a first strap 977A. The first padded support 971A is attached to the lateral aspect of the patient's thigh via the first strap 977A. The first retaining assembly 974A supports the first calibration fixture 973A preferably oriented parallel to the first reference frame 30a (i.e., orthogonal to the detector 33). It is desirable that the calibration jig 973A be positioned sufficiently far from the desired orthopedic element 100 so that the calibration jig 973A does not overlap with any of the target orthopedic elements 100. Overlapping would obscure the desired image data.

[0075] Figure 9B is a diagram of exemplary orthopedic elements 100 (e.g., proximal femur 105, natural acetabulum 108, pelvis 110, articular cartilage, other soft tissues, etc.) of the exemplary surgical area 170 of Figure 9A, offset by 45° in the positive direction from the first reference frame 30a. That is, Figure 9B represents a second input image 50 taken from a second reference frame 50a (e.g., a second lateral position). The second calibration fixture 973B is attached to the second retaining assembly 974B. The second retaining assembly 974B may include a second padded support 971B engaged with a second strap 977B. The second padded support 971B is attached to the lateral side of the patient's thigh via the second strap 977B. The second holding assembly 974B supports the second calibration fixture 973B, which is preferably oriented parallel to the second reference frame 50a (i.e., perpendicular to the detector 33). The calibration fixture 973B is preferably positioned far enough away from the orthopedic elements 100 of the target so that the calibration fixture 973B does not overlap with any of the target orthopedic elements 100.

[0076] Figure 9C is a diagram of exemplary orthopedic elements 100 (e.g., proximal femur 105, natural acetabulum 108, pelvis 110, articular cartilage, other soft tissues, etc.) of the exemplary surgical area 170 of Figure 9A, offset by 45° in the negative direction from the first reference frame 30a. That is, Figure 9C represents a third input image 70 taken from a third reference frame 70a (e.g., a third lateral position). The third calibration fixture 973C is attached to the third retaining assembly 974C. The third retaining assembly 974C may include a third padded support 971C engaged with a third strap 977C. The third padded support 971C is attached to the lateral side of the patient's thigh via the third strap 977C. The third holding assembly 974C supports the third calibration fixture 973C, which is preferably oriented parallel to the third reference frame 70a (i.e., perpendicular to the detector 33). The calibration fixture 973C is preferably positioned far enough away from the orthopedic element 100 in question so that the calibration fixture 973C does not overlap with the orthopedic element 100 in question.

[0077] If the system is calibrated preoperatively, the hip joint is stable in this orientation (see Figure 12), so the patient can be positioned in an upright position (i.e., with legs extended). If the system is calibrated during surgery, the patient may lie supine on the operating table. Preferably, the patient's distance from the imaging device should not be changed while acquiring input images 30, 50, and 70. The first, second, and third input images 30, 50, and 70 do not need to capture the entire leg; rather, the images can focus on the target joint in the surgical area 170.

[0078] Depending on the orthopedic element 100 being imaged and modeled, it will be understood that only a single calibration fixture 973 may be used. Similarly, if a particularly long collection of orthopedic elements 100 is being imaged and modeled, multiple calibration fixtures 973 may be used.

[0079] Calibration fixtures 973A, 973B, and 973C are preferably of known size. Each calibration fixture 973A, 973B, and 973C is preferably having at least four calibration points 978 distributed throughout. The calibration points 978 are distributed in a known pattern, where the distance from one point 978 to other points is known. The distance from the calibration fixture 973 to the orthopedic element 100 is also preferably known. For the calibration of the X-ray photogrammetry system, the calibration points 978 may preferably be defined by a metallic structure on the calibration fixture 973. Metals typically absorb most of the X-ray beam that comes into contact with them. Therefore, metals typically appear much brighter compared to materials that absorb little X-ray (such as air cavities or fatty tissue). Common exemplary structures for defining calibration points include, but are not limited to, reseau crosses, circles, triangles, pyramids, and spheres.

[0080] These calibration points 978 may reside on the 2D surface of the calibration jig 973, or 3D calibration points 978 may be captured as a 2D projection from a given image reference frame. In either case, the 3D coordinates (commonly referred to as the z-coordinate) may be set to be equal to zero for all calibration points 978 captured in the image. The distances between each calibration point 978 are known. These known distances may be expressed as x,y coordinates in the image sensor / detector 33. To map points in 3D space to 2D coordinate pixels on the sensor 33, the dot product of the detector's calibration matrix, extrinsic matrix, and homologous coordinate vectors of the real 3D points may be used. This maps the real-world coordinates of points in 3D space to the calibration jig 973. In other words, this generally allows the x,y coordinates of real points in 3D space to be precisely converted to the 2D coordinate plane of the image detector's sensor 33 in order to define spatial data 43 (see Figure 8).

[0081] The calibration method described above is provided as an example. It will be understood that all suitable methods for calibrating X-ray photogrammetry systems are considered to be within the scope of this disclosure. A non-exclusive list of other X-ray photogrammetry system calibration methods includes the use of lithoplate, Zhang's method, bundle adjustment methods, direct linear transformation methods, maximum likelihood estimation, k-nearest neighbor regression methods ("KNN"), convolutional neural network ("CNN")-based methods, other deep learning methods, or combinations thereof.

[0082] Figure 7 illustrates the principle of how two calibrated input images 30 and 50 can be back-projected onto a 3D volume 61 containing two channels 65 and 66 when oriented along a known offset angle θ. The first channel 65 contains all image points (e.g., X) of the first input image 30. L The second channel 66 includes all image points of the second input image 50 (e.g., X RThis includes, for example, each image point (e.g., a pixel) is duplicated on its associated back-projected 3D ray. Next, using epipolar geometry, a volume 61 of the imaged surgical region 170 containing volume data 75 can be generated from these back-projected 2D input images 30, 50. If a third input image 70 is used, a third channel may exist containing all the image points of the third input image 70.

[0083] Referring to Figure 7, it is desirable that the first input image 30 and the second input image 50 have known image dimensions. The dimensions may be in pixels. For example, the first image 30 may have dimensions of 164 × 164 pixels. The second image 50 may have dimensions of 164 × 164 pixels. The dimensions of the input images 30, 50 used in a particular calculation are preferably consistent. Consistent dimensions may be desirable for later defining a cubic working area of ​​normal volume 61 (e.g., a 164 × 164 × 164 cube). In this embodiment, the offset angle θ is preferably 45° between each adjacent input image. However, in other exemplary embodiments, other offset angles θ may be used. For example, in Figure 7, the offset angle θ is 90°.

[0084] In the illustrated example, each of the 164×164 pixel input images 30 and 50 is duplicated 164 times across the length of the adjacent input image to create a volume 61 with dimensions of 164×164×164 pixels. That is, the first image 30 is copied and stacked 164 pixels behind itself, one copy per pixel, while the second image 50 is copied and stacked 164 pixels behind itself so that the copied and stacked images overlap, thereby creating volume 61. Thus, it can be said that volume 61 contains two channels 65, 66, where the first channel 65 contains the first image 30 duplicated n times over the length of the second image 50 (i.e., the x-axis of the second image 50), and the second channel 66 contains the second image 50 duplicated m times over the length of the first image 30 (i.e., the x-axis of the first image 30), where "n" and "m" are the lengths of the indicated images, expressed as the number of pixels (or other dimensions in other exemplary embodiments) that contain the length of the indicated image. Given an offset angle θ, each transverse slice of volume 61 (also known to some radiologists as an "axial slice") creates an epipolar plane containing voxels that are back-projected from pixels containing two epipolar lines. In this way, the spatial data 43 from the first image 30 of the target orthopedic element 100 and the spatial data 43 from the second image 50 of the target orthopedic element 100 are projected to define volume data 75. By using this volumetric data 75, the 3D representation can be reconstructed using the epipolar geometry principle as described above, and the 3D representation is geometrically consistent with the information in the input images 30 and 50.

[0085] Exemplary systems and methods for identifying components of orthopedic elements and / or internal prosthetic implants in space, and for determining the location of components of orthopedic elements and internal prosthetic implants in space, using a deep learning network, where the deep learning network is a CNN, are provided with detailed examples of how CNNs can be structured and trained. All CNN architectures are considered to be within the scope of this disclosure. Common CNN architectures include, for example, LeNet, GoogLeNet, AlexNet, ZFNet, ResNet, and VGGNet.

[0086] Preferably, the methods disclosed herein can be implemented on a computer platform having hardware such as one or more central processing units (CPUs), random access memory (RAM), and / or input / output (I / O) interfaces (see 1600).

[0087] Figure 10 is a schematic diagram of a CNN that shows how edges of target orthopedic elements 100 can be identified using a CNN. While not theoretically bound, a CNN may be desirable in some cases to reduce the size of volumetric data 75 without losing the features necessary to identify the desired orthopedic elements 100 or their surface topography. The volumetric data 75 of multiple back-projected input images 30, 50 is a multidimensional array that may be known as the “input tensor”. This input tensor contains the input data for the first convolution (in this example, the volumetric data 75). A filter (also known as a kernel 69) is shown placed on the volumetric data 75. The kernel 69 is a tensor (i.e., a multidimensional array) that defines the filter or function (this filter or function may be known as the “weight” given to the kernel). In the illustrated embodiment, the kernel tensor 69 is three-dimensional. The filter or function containing the kernel 69 may be programmed manually or learned through a CNN, RNN, or other deep learning network. In the illustrated embodiment, the kernel 69 is a 3 × 3 × 3 tensor, but all tensor sizes and dimensions are considered to be within the scope of this disclosure as long as the kernel tensor size is smaller than the size of the input tensor.

[0088] Each cell or voxel in kernel 69 has a numerical value. These values ​​define the filter or function of kernel 69. The convolution or cross-correlation operation is performed between two tensors. In Figure 10, the convolution is represented by path 76. Path 76, which kernel 69 follows, is a visualization of the mathematical convolution operation. As kernel 69 follows this path 76, it eventually traverses the entire volume 61 (e.g., volume data 75) of the input tensor. The goal of this operation is to extract features from the input tensor.

[0089] A convolutional layer 72 typically includes one or more of the convolutional stage 67, the detector stage 68, and the pooling stage 58. Each of these operations is visually represented in the first convolutional layer 72a in Figure 10, but it will be understood that subsequent convolutional layers 72b, 72c, etc., may also include one or more of the operations of the convolutional stage 67, the detector stage 68, and the pooling layer 58, or combinations or permutations thereof, or all of them. Furthermore, although Figure 10 shows five convolutional layers 72a, 72b, 72c, 72d, and 72e with varying resolutions, it will be understood that other exemplary embodiments may use more or fewer convolutional layers.

[0090] In the convolution stage 67, the kernel 69 is sequentially multiplied by multiple patches of pixels in the input data (i.e., volume data 75 in the illustrated example). The patches of pixels extracted from the data are known as receptive fields. The multiplication of kernel 69 with the receptive fields involves element-wise multiplication between each pixel in the receptive field and kernel 69. After multiplication, the results are summed to form one element of the convolution output. This kernel 69 is then shifted to adjacent receptive fields, and the element-wise multiplication and summing operations continue until all pixels in the input tensor undergo this operation.

[0091] Up to this stage, the input data of the input tensor (e.g., volume data 75) is linear. Next, a nonlinear activation function is used to introduce nonlinearity into this data. The use of such a nonlinear function marks the beginning of the detector stage 68. A common nonlinear activation function is a normalized linear unit function (Rectified Linear Unit function, "ReLU") given by the function.

[0092]

number

[0093] When used with a bias, the nonlinear activation function serves as a threshold for detecting the presence of features extracted by kernel 69. For example, a convolutional output tensor is generated by applying a convolutional or cross-correlation operation between the input tensor and kernel 69, where kernel 69 includes a low-level edge filter in the convolution stage 67. A feature-mapped output tensor is then returned by applying a nonlinear activation function with a bias to the convolutional output tensor. The bias is sequentially added to each cell of the convolutional output tensor. For a given cell, if the sum is greater than or equal to 0 (assuming ReLU is used in this example), the sum is returned to the corresponding cell in the feature-mapped output tensor. Similarly, if the sum is less than 0 for a given cell, the corresponding cell in the feature-mapped output tensor is set to 0. Thus, applying a nonlinear activation function to the convolutional output behaves like a threshold for determining whether, and to what extent, the convolutional output matches a given filter in kernel 69. In this way, the nonlinear activation function detects the presence of desired features from the input data (for example, volume data 75 in this example).

[0094] All nonlinear activation functions are considered to be within the scope of this disclosure. Other examples include sigmoid, TanH, Leaky ReLU, parametric ReLU, Softmax, and Switch activation functions.

[0095] However, a drawback of this approach is that the feature map output of this first convolutional layer 72a records the exact location of the desired feature (edges in the example above). Therefore, small movements of features in the input data will generate different feature maps. To address this problem and reduce computational power, downsampling is used to reduce the resolution of the input data while still preserving important structural elements. Downsampling can be achieved by changing the stride of the convolution along the input tensor. Downsampling can also be achieved by using the pooling layer 58.

[0096] Effective padding may be applied to reduce the size of the convolved tensor (see 72b) compared to the input tensor (see 72a). A pooling layer 58 is preferably applied to reduce the spatial size of the convolved data, which reduces the computational power required to process the data. Common pooling techniques, including max pooling and mean pooling, may be used. Max pooling returns the maximum value of the portion of the input tensor covered by kernel 69, while mean pooling returns the average of all values ​​of the portion of the input tensor covered by kernel 69. Image noise may be reduced using max pooling.

[0097] In certain exemplary embodiments, a fully coupled layer may be added after the final convolutional layer 72e to learn a nonlinear combination of high-level features represented by the output of the convolutional layers (e.g., a profile of an imaged natural acetabulum 108, a profile of a reamed acetabulum 109, or the surface topology of an orthopedic element).

[0098] When used on orthopedic elements 100, the above description of a CNN-type deep learning network is an example of how a deep learning network may be configured to "identify" orthopedic elements 100 in order to define "identified orthopedic elements" 100a.

[0099] The upper half of Figure 10 represents the compression of the input volume data 75, and the lower half represents the decompression until the input volume data 75 reaches its original size. The output feature maps of each convolutional layer 72a, 72b, 72c, etc., are used as input to subsequent convolutional layers 72b, 72c, etc., to enable increasingly complex feature extraction. For example, the first kernel 69 may detect edges, the kernel in the first convolutional layer 72b may detect a set of edges with a desired orientation, the kernel in the third convolutional layer 72c may detect a set of longer edges with a desired orientation, and so on. This process may continue until the entire profile of the desired orthopedic element 100 is detected and identified by the downstream convolutional layer 72.

[0100] The lower half of Figure 10 is an upsample (i.e., an expansion of the spatial support of the lower-resolution feature map). A deconvolution operation is performed to increase the size of the input for the next downstream convolutional layer (see 72c, 72d, and 72e). In the case of the final convolutional layer 72e, the convolution may be employed with a 1×1×1 kernel 69 to produce a multi-channel output volume 59 of the same size as the input volume 61. Each channel of the multi-channel output volume 59 may represent a desired extracted high-level feature. Subsequently, a Softmax activation function may detect the desired orthopedic element 100. For example, the illustrated embodiment may have five output channels numbered 0, 1, 2, 3, and 4, where channel 0 represents an identified background volume, channel 1 represents an identified proximal femur 105, channel 2 represents an identified reamed acetabulum 111, channel 3 represents an identified acetabular component 103, and channel 4 represents an identified femoral component 104.

[0101] It will be understood that in other exemplary embodiments, fewer or more output channels may be used. It will also be understood that the provided output channels may represent components of orthopedic element 100 and internal prosthesis implants different from those enumerated herein.

[0102] For example, in an exemplary embodiment where the system is configured to identify an orthopedic element 100, which is the medial cortical wall 120 of the proximal femur 105, and the system is configured to identify a component of an internal prosthesis, which is a trial component construct, the exemplary embodiment may include three output channels numbered 0, 1, and 2, where channel 0 represents the identified background volume, channel 1 represents the medial cortical wall 120 of the proximal femur 105, and channel 2 represents the identified femoral component 104. The “trial component construct” used in the above example may include a broach, trial neck and trial head assembly, or trial stem, when the trial component construct describes a construct used in the proximal femur 105.

[0103] In an exemplary embodiment in which the system is configured to identify an orthopedic element 100, which is the reamed acetabulum 111 of the pelvis 110, and the system is configured to identify a component of an internal prosthesis, which is the acetabular component 103 or a trial acetabular component, the exemplary embodiment may include three output channels numbered 0, 1, and 2, where channel 0 represents the identified background volume, channel 1 represents the reamed acetabulum 111 of the pelvis 110, and channel 2 represents the acetabular component 103 or a trial acetabular component.

[0104] Such exemplary embodiments may optionally include additional output channels, such as an output channel representing the outer wall of the proximal femur 105. Other output channels may be used to output the abduction angle α and anteversion angle υ of the identified component of the internal prosthesis relative to the identified orthopedic element 100a on which the component of the internal prosthesis is seated, respectively. Further additional output channels may be used to output (as a non-limiting example) the determined size dimensions of the identified orthopedic element, the recommended component type / product model of the internal prosthesis, the recommended component size of the internal prosthesis, the "best fit" output of the recommended component or recommended component size relative to the dimensions of the internal cortical wall 120, the longitudinal axis alignment calculation of the femoral component 104, the construction of a trial component relative to the anatomical axis of the proximal femur 105, the calculated center of the acetabulum, or the longitudinal axis alignment of the neck of the femoral component 104 relative to the center of the prosthetic femoral head 113. Any combination of the foregoing is considered to be within the scope of this disclosure.

[0105] When used in components or subcomponents of an internal artificial implant, the above description of a CNN-type deep learning network is an example of how a deep learning network can be configured to “identify” components (or subcomponents thereof) of an internal artificial implant to define an identified component of the internal artificial implant. When used in an internal artificial implant, the above description of a CNN-type deep learning network is an example of how a deep learning network can be configured to “identify” an internal artificial implant to define an “identified internal artificial implant.” When applied to multiple orthopedic elements, multiple components of an internal artificial implant, multiple internal artificial implants, or combinations thereof, it will be further understood that the above description of a CNN-type deep learning network is an example of how a deep learning network may be configured to “identify” multiple orthopedic elements, multiple components of an internal artificial implant, subcomponents thereof, multiple internal artificial implants, or possibly combinations thereof. Other deep learning network architectures known to those skilled in the art or readily verifiable are also considered to be within the scope of this disclosure.

[0106] In exemplary embodiments, a modeled orthopedic element 100b, a modeled component of an internal prosthesis (e.g., acetabular cup 106, femoral stem 115, etc.), can be generated using a selectable output channel containing output volume data 59 of a desired orthopedic element 100. In certain exemplary embodiments, the modeled orthopedic element 100b is a computer model. In other exemplary embodiments, the modeled orthopedic element 100b is a physical model.

[0107] While the above example illustrates the use of a three-dimensional tensor kernel 69 for convolving input volume data 75, it will be understood that the general model described above can be used with 2D spatial data 43 from any of the calibrated input images 30, 50, or 70. In other exemplary embodiments, a machine learning algorithm (i.e., a deep learning network (e.g., CNN)) may be used after calibration of the imager 1800, but before 2D-to-3D reconstruction. That is, a CNN can be used to detect features (e.g., anatomical landmarks) of the target orthopedic element 100 from a first reference frame 30a, a second reference frame 50a, or a third reference frame 70a of the 2D input images 30, 50, or 70. In an exemplary embodiment, a CNN can be used to identify high-level orthopedic elements (e.g., the proximal femur 105 and part of the surface topology of the target orthopedic element 100), components of an internal prosthesis (e.g., the acetabular cup 106, femoral stem 115, etc.), or the internal prosthesis itself (e.g., hip internal prosthesis 102) from 2D input images 30, 50, 70. The CNN can then optionally apply a mask or contour to the detected orthopedic element 100, components of the internal prosthesis, or the internal prosthesis itself. If the imager 1800 is calibrated, and the CNN can identify multiple corresponding image points (e.g., X) of features between at least two input images 30, 50. L , X RIf an orthopedic element 100a is identified, it is conceivable that a transformation matrix between the reference frames 30a, 50a of the target orthopedic element 100, the components of the internal prosthesis, or the internal prosthesis itself can be used to align multiple corresponding image points in 3D space. In this way, the position of a point in 3D space can be determined to correspond to a set of coordinates in 3D space. In this way, it can be said that the 3D point is "mapped" to spatial data. A deep learning network that can model this relationship in this way or in other ways developed by a deep learning network can be said to be "configured" to "map" the identified orthopedic element, the identified components of the internal prosthesis, and / or (if applicable) the internal prosthesis itself to spatial data confirmed by the first input image 30 and the second input image 50 (and possibly a third input image 70 or further input images or "new" refresh images), thereby determining the position of the identified orthopedic element 100a, the identified components of the internal prosthesis, and / or (if applicable) the internal prosthesis itself in three-dimensional space.

[0108] In embodiments where the first input image 30, the second input image 50, or the third input image 70 is a radiographic X-ray image (including, but not limited to, a fluoroscopic radiographic image), training the CNN may present several challenges. For comparison, a CT scan typically produces a series of images of a desired volume. Each CT image, including a typical CT scan, can be considered a segment of the imaged volume. From these segments, a 3D model can be constructed relatively easily by adding regions of the desired elements as the elements are shown in their respective consecutive CT images. The modeled elements can then be compared to the CT scan data to ensure accuracy. One drawback of CT scans is that they expose patients to excessive amounts of radiation (about 70 times the radiation dose of a single conventional radiograph).

[0109] In contrast, radiographic imaging systems typically do not produce sequential images that capture different segments of the imaged volume. Rather, all the information in the image is flattened in a 2D plane. In addition, since a single radiographic image 30 inherently lacks 3D data, it is difficult to check the model generated by the epipolar geometry reconstruction technique described above using the actual geometric shape of the target orthopedic element 100. To address this problem, a CNN can be trained using CT images, such as digitally reconstructed radiograph ("DRR") images. By training a deep learning network in this way, the deep learning network can develop its own weights (e.g., filters) for the kernel 69 to identify the surface topography of the desired orthopedic element 100 or the target orthopedic element 100. Since X-ray images have a different appearance from DRRs, an image-to-image transformation can be performed to render the input X-ray image to have a DRR-style appearance. An exemplary image-to-image transformation method is the CycleGAN image transformation technique. In embodiments where a style transfer method between images is used, the style transfer method is preferably used before inputting the data into a deep learning network for feature detection.

[0110] The above example is provided for illustrative purposes only and is not intended to limit the scope of this disclosure. All methods for generating a 3D model 100 of the same orthopedic element 100 from 2D radiographic images of the same orthopedic element 100 taken from at least two lateral positions (e.g., 30a, 50a) are considered to be within the scope of this disclosure.

[0111] By determining the thickness and boundaries of specific identified orthopedic elements 100a, components of the internal prosthesis implant, and / or the internal prosthesis implant itself, as well as their precise coordinates in 3D space, the locations of identified orthopedic elements 100a, components of the internal prosthesis implant, and / or the internal prosthesis implant, components of the internal prosthesis implant, and / or the internal prosthesis implant can be known while bypassing the limited field of view provided to the surgeon through the main incision. If the location of identified orthopedic element 100a is known and the location of identified components of the internal prosthesis implant is known, this information can be used to check the desired alignment parameters of the implant components relative to the identified orthopedic element 100a in which the implant components are placed (e.g., acetabular shell 106 placed in a reamed acetabulum 111, femoral stem 115 placed in the intramedullary canal of the proximal femur 105).

[0112] Similarly, if the position of a first component of an internal prosthesis (e.g., an acetabular component) is known in three dimensions relative to a second component of the internal prosthesis (e.g., a femoral component), the surgeon can use the exemplary systems and methods described herein to evaluate the placement, and thus the alignment, of the first component relative to the second component. The surgeon can re-image the surgical area to update the position of the first component relative to the second component at subsequent time intervals until the surgeon is satisfied with the alignment. Such alignment is intended to be performed during surgery to mitigate the problem of misaligned components in multi-component internal prostheses.

[0113] In a particular exemplary embodiment, which involves using a deep learning network to add masks or contours to the 2D orthopedic elements 100 detected from each input image 30, 50, and 70, only the 2D masks or contours of the identified orthopedic element 100 components of the internal prosthesis, and / or the internal prosthesis itself, can be sequentially back-projected in the manner described with reference to Figures 7 and 8 above to define the identified orthopedic element 100, the components of the internal prosthesis, and / or the volume 61 of the internal prosthesis. In this exemplary method, a modeled orthopedic element 100b, a modeled component of the internal prosthesis, and / or the modeled internal prosthesis can be generated.

[0114] Figure 11 is a magnified view of the modeled internal prosthesis 102b, which includes several modeled internal prosthesis components, namely a modeled acetabular component 103b and a modeled femoral component 104b. The modeled acetabular component 103b includes a modeled acetabular shell 106b and a modeled acetabular liner 107b. The modeled femoral component 104b includes a modeled femoral stem 115b, a modeled femoral stem neck 116b, and a modeled prosthesis femoral head 113b.

[0115] An exemplary system or method may further include calculating the center of the acetabulum. Such an exemplary system or method may further include aligning the longitudinal axis of rotation of the femoral stem implant with the center of the acetabulum. In yet another exemplary system or method, the longitudinal axis of the femoral stem 115 may be aligned with the longitudinal axis of the femur 105 (i.e., collinear). In yet another exemplary system and method, the axis of rotation of the neck of the femoral stem 115 may be aligned with the center of the artificial femoral head 113. In yet another exemplary system and method, the position of the artificial head 113 may be aligned (e.g., perpendicularly) with the natural femoral head before the disease, based on an input image of the natural femoral head. In such exemplary embodiments, it is desirable that the longitudinal axis of the femoral stem 115 is collinear with the anatomical axis of the femur 105, and that the femoral stem 115 is positioned at an anteversion angle υ in the range of about 10° to about 30°, preferably about 15° to about 25°.

[0116] A computer platform having hardware such as one or more central processing units ("CPU"), random access memory ("RAM"), and input / output ("I / O") interfaces(s) can receive at least two 2D radiographic images taken in different orientations along a transverse plane. The orientations can be orthogonal to each other (i.e., the first reference frame has an offset angle θ of 90° with respect to the second reference frame). However, in embodiments in which the orthopedic element 100 includes a hip joint 101, at least three 2D radiographic input images may be desirable to avoid interference from the contralateral acetabulum. In such exemplary embodiments, the offset angle θ may preferably be 45° between adjacent reference frames. In other exemplary embodiments, other obtuse or acute offset angles θ may be used.

[0117] Referring to Figure 12, an exemplary system for confirming the position of orthopedic elements 100 and components of internal prosthetic implants in space may include a tissue transmission imager 1800 (e.g., a radiographer, fluoroscopy machine, etc.) including an emitter 21 and a detector 33, the detector 33 of the radiographer 1800 capturing a first input image 30 (Figures 8 and 9A) at a first lateral position 30a (Figures 8 and 9A) and a second input image 50 (Figures 8 and 9B) at a second lateral position 50a (Figures 8 and 9B), where the first lateral position 30a is offset by an offset angle θ (Figure 8) from the second lateral position 50a. In an exemplary embodiment including at least three input images, the detector 33 of the radiographer 1800 captures a third input image 70 at a third lateral position 70a (Figures 8 and 9C), where the third lateral position 70a is offset by two separate offset angles θ1 and θ2 from the second lateral position 50a and the first lateral position 30a.

[0118] The exemplary system may further include a transmitter 29 (Figure 12) and a computer 1600 (see Figure 13 for further details), wherein the transmitter 29 transmits a first input image 30 and a second input image 50 (and optionally a third input image 70, if present) from the detector 33 to the computer 1600, and the computer 1600 is configured to identify orthopedic elements 100a, components of an internal prosthesis, subcomponents of components of an internal prosthesis, or the internal prosthesis itself using one of the deep learning methods discussed herein. It will be understood that the exemplary system disclosed herein may be used preoperatively, intraoperatively, and / or postoperatively.

[0119] In certain exemplary embodiments, the exemplary system may further include a display 19.

[0120] Figure 12 is a schematic diagram of an exemplary system including a radiographer 1800, which includes an X-ray source 21, a filter 26, a collimator 27, and a detector 33, such as an X-ray tube. In Figure 12, the radiographer 1800 is shown from top to bottom. The illustrated radiographer 1800 is a type of tissue transmission radiographer. Patient 1 is positioned between the X-ray source 21 and the detector 33. The radiographer 1800 may be mounted on a rotatable gantry 28. The radiographer 1800 can acquire a first radiographic input image 30 of patient 1 from a first reference frame 30a. The gantry 28 may then rotate the radiographer 1800 by an offset angle. The radiographer 1800 can then acquire a second radiographic input image 50 from a second reference frame 50a. It will be understood that other exemplary embodiments may include using multiple input images taken at multiple offset angles θ. For example, in hip arthroplasty, the radiographer 1800 may be further rotated (or the patient may be rotated) to capture a third radiographic input image 70 from a third reference frame 70a. In such embodiments, the offset angle may be less than 90° or greater than 90° between adjacent input images.

[0121] It will be understood that the offset angle does not need to be exactly 90 degrees in all embodiments. An offset angle having a value within ±45 degrees is intended to be sufficient. In other exemplary embodiments, the operator may take three or more images of the orthopedic element using radiographic imaging techniques. Each subsequent image after the second image is intended to be able to define a subsequent image reference frame. For example, the third image can define the third reference frame, the fourth image can define the fourth reference frame, the nth image can define the nth reference frame, and so on.

[0122] In other exemplary embodiments including three input images and three separate reference frames, each of the three input images may have an offset angle θ of about 60 degrees relative to each other. In some exemplary embodiments including four input images and four separate reference frames, the offset angle θ may be 45 degrees from adjacent reference frames. In exemplary embodiments including five input images and five separate reference frames, the offset angle θ may be about 36 degrees from adjacent reference frames. In exemplary embodiments including n images and n separate reference frames, the offset angle θ may be 180 / n degrees.

[0123] Embodiments involving multiple images, particularly three or more images, do not necessarily need to have regular and consistent offset angles. For example, an exemplary embodiment involving four images and four separate reference frames may have a first offset angle of 85 degrees, a second offset angle of 75 degrees, a third offset angle of 93 degrees, and a fourth offset angle of 107 degrees.

[0124] Next, the transmitter 29 transmits the first input image 30 and the second input image 50 to the computer 1600. The computer 1600 can use a deep learning network to identify orthopedic elements 100a, components of an internal prosthesis, subcomponents of components of an internal prosthesis, or the internal prosthesis itself in any manner consistent with the present disclosure.

[0125] Figure 12 also illustrates another embodiment in which output data from computer 1600 is transmitted to display 19. Display 19 can depict the modeled internal prosthesis 102b. The display can optionally display any of the items identified by the exemplary systems and methods described herein, including but not limited to identified internal prostheses, components or subcomponents of internal prostheses, or one or more orthopedic elements. In exemplary embodiments, it is intended that identified components of internal prostheses, or representative models of components of internal prostheses, can be superimposed onto identified orthopedic elements on which the components of internal prostheses are seated (e.g., femoral components and proximal femur, respectively). The superimposition can be calculated and displayed using mapped spatial data of each identified element (e.g., components of internal prostheses and orthopedic elements on which the components of internal prostheses are seated).

[0126] In this way, surgeons and other personnel in the operating room can visualize the components of the internal artificial implant and the target orthopedic elements in three dimensions in near real-time and align them with each other.

[0127] Furthermore, since spatial data of identified components of the internal prosthesis and spatial data of identified orthopedic elements can be obtained from the exemplary system described herein, in embodiments of the exemplary system, the degree of alignment can be calculated and further displayed on the display 19. For example, the calculated abduction angle α of the identified components of the internal prosthesis can be displayed on the display. As another example, the calculated anteversion angle υ of the identified components of the internal prosthesis can be displayed on the display 19. As yet another example, the vertical position of the artificial femoral head 113 can be displayed and superimposed on a reconstructed 3D image of the natural femoral head of the surgical hip joint based on preoperative planning input images 30, 50, 70 (see 126). As yet another example, the display 19 may optionally display a “best fit” percentage where a percentage reaching or near 100% reflects the alignment of the identified components of the internal prosthesis (e.g., femoral component 104) with respect to a reference orthopedic element.

[0128] For example, in an embodiment where the identified component of the internal prosthesis is the femoral component 104, the reference orthopedic element may be the original proximal femur 105 of the surgical joint identified and reconstructed from a preoperative planning input image according to any embodiment of the present disclosure. In such exemplary embodiments, the best fit percentage of the alignment may take into account the anteversion angle υ of the identified femoral component 104, the varus-valgus position of the femoral stem 115 of the femoral component 104 relative to the anatomical axis of the femur 105, the anteversion angle of the femoral stem 115 in the intramedullary canal of the femur 105 of the surgical hip joint 101, and the vertical, horizontal, and anterior-posterior positions of the prosthesis femoral head 113 relative to the natural femoral head of the surgical hip joint 101 before resection (see 126). Any combination of the above embodiments of what may be displayed on the display 19 is considered to be within the scope of the present disclosure.

[0129] In embodiments where the identified component of the internal prosthesis is the femoral component of the hip joint implant, and the identified orthopedic element is the proximal femur into which the femoral component is inserted and seated, the exemplary system can display the varus or valgus angle of the longitudinal axis of the femoral component with respect to the anatomical axis of the femur (i.e., the central axis of the femur extending through the intramedullary canal of the femur).

[0130] The exemplary system may further include one or more databases. One or more databases may include a list of types of components of the internal prosthesis and associated component size dimensions for the component types in the list of components of the internal prosthesis (e.g., different product models of a particular component). In the exemplary embodiment, the database may include a list of sizes for one particular type of component of the internal prosthesis.

[0131] The computer can compare the dimensions of identified components of the internal prosthesis with values ​​stored in a database. The computer can then select or display a recommended type of component and / or a recommended size for a particular component from the values ​​stored in the database, based on how closely the dimensions of the identified components of the internal prosthesis match the dimensions of the values ​​stored in the database. In this way, the computer 1600 can be said to be "configured to select" a recommended type of component of the internal prosthesis based on the determined size dimensions of the identified orthopedic element in three-dimensional space. Similarly, in this way, the computer 1600 can be said to be "configured to recommend a size for the components of the internal prosthesis based on the determined size dimensions of the identified orthopedic element in three-dimensional space."

[0132] The display 19 may take the form of a screen. In other exemplary embodiments, the display 19 may include a glass or plastic surface worn or held by a surgeon or other person in an operating room. Such a display 19 may include part of an augmented reality device, so that the display shows a 3D model in addition to the wearer's field of view. In certain embodiments, such a 3D model may be superimposed on an actual surgical joint. In yet another exemplary embodiment, the 3D model can be "locked" to one or more features of the surgical orthopedic element 100, thereby maintaining the virtual position of the 3D model relative to one or more features of the surgical orthopedic element 100, independently of the movement of the display 19. It is still intended that the display 19 may include part of a virtual reality system in which the entire field of view is simulated.

[0133] While radiographs from radiography systems may be desirable because radiographs are relatively inexpensive compared to CT scans, and because some radiography systems, such as fluoroscopy systems, are generally compact enough for intraoperative use, this disclosure does not limit the use of 2D images to radiographs unless otherwise expressly claimed, and nothing in this disclosure limits the type of imaging system to radiography systems. Other 2D images may include, for example, CT images, CT fluoroscopy images, fluoroscopy images, ultrasound images, positron emission tomography (PET) images, and MRI images. Other imaging systems may include, for example, CT, CT fluoroscopy, fluoroscopy, ultrasound, PET, and MRI systems.

[0134] Preferably, the exemplary method may be implemented on a computer platform (e.g., computer 1600) having hardware such as one or more central processing units (CPUs), random access memory (RAM), and / or input / output (I / O) interfaces. An example of the architecture of the exemplary computer 1600 is provided below with reference to Figure 7.

[0135] Figure 13 shows a block diagram of an exemplary computer 1600 in which, generally speaking, one or more of the methods discussed herein can be performed according to several exemplary embodiments. In certain exemplary embodiments, computer 1600 may operate as a single machine. In other exemplary embodiments, computer 1600 may include connected (e.g., networked) machines. Examples of networked machines that may include exemplary computer 1600 include, for example, cloud computing configurations, distributed host configurations, and other computer cluster configurations. In a network configuration, one or more machines of computer 1600 may operate as a client machine, a server machine, or both a server and a client machine. In exemplary embodiments, computer 1600 may reside in a personal computer ("PC"), a mobile phone, a tablet PC, a web appliance, a personal digital assistant ("PDA"), a network router, a bridge, a switch, or any machine capable of executing instructions that specify actions to be taken by or controlled by that machine.

[0136] An exemplary machine, which may include exemplary computer 1600, may, for example, be a component, module, or similar mechanism capable of performing a logic function. Such a machine may include a tangible entity (e.g., hardware) capable of performing a specified operation during operation. For example, the hardware may be wired (e.g., specifically configured) to perform a particular operation. For example, such hardware may have a configurable execution medium (e.g., circuits, transistors, logic gates, etc.) and a computer-readable medium having instructions, the instructions configuring the execution medium to perform a particular operation during operation. Configuration may be done via a loading mechanism or under the direction of the execution medium. The execution medium selectively communicates with the computer-readable medium when the machine is operating. For example, when the machine is operating, the execution medium may be configured by a first set of instructions to perform a first action or set of actions at a first time point, and then reconfigured by a second set of instructions to perform a second action or set of actions at a second time point.

[0137] An exemplary computer 1600 may include a hardware processor 1697 (e.g., a CPU, a graphics processing unit ("GPU"), a hardware processor core, or any combination thereof), main memory 1696, and static memory 1695, some or all of which may communicate with each other via an interlink (e.g., a bus) 1694. The computer 1600 may further include a display unit 1698, an input device 1691 (preferably an alphanumeric or character-number input device such as a keyboard), and a user interface ("UI") navigation device 1699 (e.g., a mouse or stylus). In exemplary embodiments, the input device 1691, the display unit 1698, and the UI navigation device 1699 may be touchscreen displays. In exemplary embodiments, the display unit 1698 may include holographic lenses, eyeglasses, goggles, other eyewear, or other AR or VR display components. For example, the display unit 1698 may be worn on the user's head and may provide the user with a head-up display. The input device 1691 may be a virtual keyboard (for example, a keyboard that is virtually displayed in a virtual reality ("VR") or augmented reality ("AR") setting) or other virtual input interface.

[0138] Computer 1600 may further include a memory device (e.g., a drive unit) 1692, a signal generator 1689 (e.g., a speaker), a network interface device 1688, and one or more sensors 1687 such as a global positioning system ("GPS") sensor, an accelerometer, a compass, or other sensors. Computer 1600 may also include an output controller 1684 for communicating with or controlling one or more auxiliary devices, such as a serial (e.g., universal serial bus ("USB")), parallel, or other wired or wireless (e.g., infrared ("IR"), near-field communication ("NFC"), radio, etc.) connection.

[0139] The storage device 1692 may include a non-temporary, machine-readable medium 1683 on which one or more sets 1682 (e.g., software) of data structures or instructions embodied or utilized by one or more of the functions or methods described herein are stored. The instructions 1682 may reside entirely or at least partially within the main memory 1696, the static memory 1695, or the hardware processor 1697 during their execution by the computer 1600. For example, one or any combination of the hardware processor 1697, the main memory 1696, the static memory 1695, or the storage device 1692 may constitute a machine-readable medium.

[0140] Although machine-readable medium 1683 is shown as a single medium, the term “machine-readable medium” may include a single or multiple mediums configured to store one or more instructions 1682 (e.g., a distributed or centralized database, or associated caches and servers).

[0141] The term “machine-readable medium” may include any medium capable of storing, encoding, or transmitting instructions executed by the computer 1600, and capable of causing the computer 1600 to execute any one or more of the methods of the Disclosure, or capable of storing, encoding, or transmitting data structures used by or associated with such instructions. A non-exclusive exemplary list of machine-readable mediums may include magnetic media, optical media, solid-state memory, non-volatile memory, for example, semiconductor memory devices (e.g., electronically erasable programmable read-only memory, “EEPROM”, electronically programmable read-only memory, “EPROM”, and magnetic disks such as internal hard disks and removable disks, flash memory, magneto-optical disks, CD-ROM disks, and DVD-ROM disks).

[0142] Instruction 1682 may further be transmitted or received over a communication network 1681 using a transmission medium via a network interface device 1688 that utilizes one of a number of transport protocols (e.g., Internet Protocol ("IP"), User Datagram Protocol ("UDP"), Frame Relay, Transmission Control Protocol ("TCP"), Hypertext Transport Protocol ("HTTP"), etc.). Illustrative communication networks may include wide area networks ("WAN"), legacy telephone ("POTS") networks, local area networks ("LAN"), packet data networks, mobile telephone networks, wireless data networks, and peer-to-peer ("P2P") networks. For example, the network interface device 1688 may include one or more physical jacks (e.g., Ethernet, coaxial, or telephone jacks) or one or more antennas to connect to the communication network 1681.

[0143] For example, the network interface device 1688 may include multiple antennas for wireless communication using at least one of the single-input multiple-output ("SIMO") or multiple-input single output ("MISO") methods. The term "transmission medium" includes any intangible medium on which instructions executed by the computer 1600 can be stored, encoded, or transmitted, and includes analog or digital communication signals or other intangible mediums to facilitate the communication of such software.

[0144] The exemplary methods described herein may be at least partially mechanical or computer-readable implementations. Some examples may include computer-readable or machine-readable media encoded with instructions operable to configure an electronic device to perform the exemplary methods described herein. Exemplary implementations of such exemplary methods may include code, such as assembly language code, microcode, high-level language code, or other code. Such code may include computer-readable instructions for performing various methods. The code may form part of a computer program product. A computer 1600 capable of executing computer-readable instructions for performing deep learning network methods and calculations can be said to be “configured to perform” deep learning networks. Furthermore, in one example, the code may be tangibly stored on or in a computer-readable medium that is volatile, non-temporary, or non-volatile, such as during execution or at other times. Examples of these tangible computer-readable media include, but are not limited to, removable optical discs (e.g., compact discs and digital video discs), hard drives, removable magnetic disks, memory cards or sticks, removable flash memory drives, magnetic cassettes, random access memory (RAM), read-only memory (ROMS), and other media.

[0145] The exemplary methods disclosed herein are further intended to be used for preoperative planning, intraoperative planning or execution, or postoperative evaluation of implant placement and function.

[0146] An exemplary method for determining the location of orthopedic elements in space is to calibrate a radiographer to define spatial data by determining the mapping relationship between image points and corresponding spatial coordinates; to capture a first image of the orthopedic element using radiographing techniques, the first image defining a first reference frame; to capture a second image of the orthopedic element using radiographing techniques, the second image defining a second reference frame, the first reference frame being offset from the second reference frame by an offset angle; and to detect the orthopedic element using the spatial data with a deep learning network, the spatial data defining anatomical landmarks on or within the orthopedic element. The method may include: applying a mask to orthopedic elements defined by anatomical landmarks using a deep learning network; defining volumetric data by projecting spatial data from a first image of a desired orthopedic element and spatial data from a second image of a desired orthopedic element, wherein spatial data including image points located within the mask region of either the first or second image has a first value, and spatial data including image points located outside the mask region of either the first or second image has a second value, and the first value is different from the second value; applying a deep learning network to the volumetric data to generate a reconstructed three-dimensional model of the orthopedic element; and mapping the three-dimensional model of the orthopedic element to the spatial data.

[0147] In exemplary embodiments, the exemplary method may further include performing style transfer on a first and second image using a deep learning network. In exemplary embodiments, style transfer converts spatial data from radiographic imaging techniques into dynamic digital radiography data.

[0148] In the exemplary embodiment, the first value is a positive value.

[0149] In the exemplary embodiment, the second value is a negative value.

[0150] In an exemplary embodiment, the exemplary method further includes projecting the reconstructed three-dimensional model onto a display.

[0151] In exemplary embodiments, the deep learning network includes a convolutional neural network.

[0152] In an exemplary embodiment, the radiographic imaging technique is fluorescence fluoroscopy.

[0153] In an exemplary embodiment, the method is performed intraoperatively.

[0154] In an exemplary embodiment, the orthopedic element is the acetabulum of the pelvis.

[0155] In an exemplary embodiment, the exemplary method further includes calculating the center of the acetabulum.

[0156] In exemplary embodiments, exemplary methods further include aligning the longitudinal axis of rotation of the neck of the femoral stem (e.g., a component of an internal prosthetic implant) with the center of the acetabulum.

[0157] In exemplary embodiments, exemplary methods further include aligning the acetabular shell (e.g., a component of an internal prosthetic implant) with the patient's reamed acetabulum.

[0158] In exemplary embodiments, exemplary methods further include aligning a femoral stem (e.g., a component of an internal prosthesis) with the intramedullary canal of the reamed proximal femur of a patient.

[0159] In exemplary embodiments, the method further includes aligning the longitudinal axis of the femoral stem (e.g., a component of an internal prosthetic implant) with the anatomical (i.e., central) axis of the femur.

[0160] An exemplary method for determining the location of orthopedic elements in space is to calibrate a radiographer to define spatial data by determining the mapping relationship between image points and corresponding spatial coordinates; to capture a first image of the orthopedic element using radiographing techniques, the first image defining a first reference frame; to capture a second image of the orthopedic element using radiographing techniques, the second image defining a second reference frame, the first reference frame being offset from the second reference frame by an offset angle; and to detect the orthopedic element using spatial data with a neural network, the spatial data defining anatomical landmarks on or within the orthopedic element. The method includes: applying a mask to an orthopedic element defined by anatomical landmarks using a deep learning network; defining volumetric data by projecting spatial data from a first image of a desired orthopedic element and spatial data from a second image of a desired orthopedic element, wherein spatial data containing image points located within the mask region of either the first or second image has a positive value, and spatial data containing image points located outside the mask region of either the first or second image has a negative value; applying the deep learning network to the volumetric data to generate a three-dimensional model of the orthopedic element; and mapping the three-dimensional model of the orthopedic element to spatial data, wherein the orthopedic element is the acetabulum of the pelvis.

[0161] In an exemplary embodiment, the orthopedic element is excised or reamed during the surgical procedure.

[0162] In an exemplary embodiment, the method further includes performing style transfer on a first image and a second image using a deep learning network.

[0163] In an exemplary embodiment, style conversion converts spatial data from radiographic imaging techniques into dynamic digital radiography data.

[0164] In the exemplary embodiment, the first value is a positive value.

[0165] In the exemplary embodiment, the second value is a negative value.

[0166] In an exemplary embodiment, the method further includes projecting the reconstructed three-dimensional model onto a display.

[0167] In exemplary embodiments, the deep learning network includes a convolutional neural network.

[0168] In an exemplary embodiment, the radiographic imaging technique is fluorescence fluoroscopy.

[0169] In an exemplary embodiment, the method is performed intraoperatively.

[0170] In an exemplary embodiment, the method further includes calculating the center of the acetabulum.

[0171] In exemplary embodiments, the method further includes aligning the longitudinal axis of rotation of the neck of the femoral stem with the center of the artificial femoral head, and aligning the longitudinal axis of the femoral stem with the anatomical axis of the intramedullary canal of the femur in which the femoral stem is positioned.

[0172] In exemplary embodiments, exemplary methods further include aligning the acetabular shell (e.g., a component of an internal prosthetic implant) with the patient's reamed acetabulum.

[0173] In exemplary embodiments, exemplary methods further include aligning a femoral stem (e.g., a component of an internal prosthesis) with the intramedullary canal of the reamed proximal femur of a patient.

[0174] In exemplary embodiments, the method further includes aligning the longitudinal axis of the femoral stem (e.g., a component of an internal prosthetic implant) with the anatomical axis of the femur in which the femoral stem is positioned.

[0175] An exemplary system for determining the position of components of orthopedic elements and internal prostheses in space includes a tissue transmission imager; a first input image, the first input image being captured by the tissue transmission imager from a first reference frame and depicting a calibration fixture; a second input image being captured by the tissue transmission imager from a second reference frame offset from the first reference frame and showing the calibration fixture; and a computer configured to run a deep learning network, the deep learning network being configured to identify components of orthopedic elements and internal prostheses, define identified components of identified orthopedic elements and internal prostheses, map identified components of identified orthopedic elements and internal prostheses to spatial data confirmed by the first and second input images, thereby determining the position of identified components of identified orthopedic elements and internal prostheses in three-dimensional space.

[0176] In an exemplary embodiment, the system further includes a third input image, the third input image being captured by a tissue transmission imager from a third reference frame, and the third image depicting a calibration jig.

[0177] In an exemplary embodiment of the system, the deep learning network is further configured to identify multiple components of multiple orthopedic elements and internal prostheses, and to define multiple identified components of multiple identified orthopedic elements and internal prostheses.

[0178] In a further exemplary embodiment of the system, the first identified component of the plurality of identified components of the internal prosthesis is the acetabular component of the hip internal prosthesis, and the second identified component of the plurality of identified components of the internal prosthesis is the femoral component of the hip internal prosthesis.

[0179] In an exemplary embodiment of the system, the identified component of the internal prosthetic implant is the acetabular shell, and the identified orthopedic element is a reamed acetabulum adjacent to the acetabular shell.

[0180] In an exemplary embodiment of the system, the identified component of the internal prosthesis is the femoral stem, and the identified orthopedic element is the intramedullary canal of the femur adjacent to the femoral stem.

[0181] In an exemplary embodiment of the system, the identified orthopedic elements are modeled in three dimensions to define the modeled orthopedic elements.

[0182] In exemplary embodiments, the modeled orthopedic elements are displayed on a screen.

[0183] In an exemplary embodiment, identified components of the internal prosthesis are modeled in three dimensions to define the modeled components of the internal prosthesis.

[0184] In an exemplary embodiment, the modeled components of the internal prosthetic implant are displayed on the screen.

[0185] In the exemplary embodiment, the calculated abduction angle of the identified component of the internal prosthetic implant is displayed on the display.

[0186] In the exemplary embodiment, the calculated anteversion angle of the identified component of the internal prosthesis is displayed on the display.

[0187] An exemplary system for recommending the type of components of an internal artificial implant to be surgically implanted in a patient, comprising: a radiographer; a first input image, the first input image being taken by the radiographer from a first reference frame, the first image depicting a calibration jig; a second input image, the second input image being taken by the radiographer from a second reference frame, the second reference frame being offset from the first reference frame, the second image depicting a calibration jig; and a computer configured to run a deep learning network, the deep learning network recognizing orthopedic elements. A computer configured to define identified orthopedic elements, map identified orthopedic elements to spatial data confirmed by a first input image and a second input image, thereby defining determined size dimensions of identified orthopedic elements in three-dimensional space; and a system comprising a list of types of components of an internal prosthesis, and relevant component size dimensions for the component types in the list of components of an internal prosthesis, wherein the computer is further configured to select a recommended type of component of an internal prosthesis based on the determined size dimensions of identified orthopedic elements in three-dimensional space.

[0188] In an exemplary embodiment, the system further includes a third input image, the third input image being captured by a tissue transmission imager from a third reference frame, and the third image depicting a calibration jig.

[0189] In exemplary embodiments, the identified orthopedic element is the internal geometric shape of the bone before or after reaming, or before or after broaching.

[0190] In an exemplary embodiment, the computer is configured to run a best fit algorithm to select recommended components for an internal prosthetic implant based on the determined size dimensions of the identified orthopedic element.

[0191] An exemplary system for determining the size of components of an internal artificial implant to be surgically implanted in a patient, the system comprising: a transluminal imaging device and a first input image, the first input image being captured by the transluminal imaging device from a first reference frame, the first image depicting a calibration jig; a second input image, the second input image being captured by the transluminal imaging device from a second reference frame, the second reference frame being offset from the first reference frame, the second image depicting a calibration jig; and a computer configured to run a deep learning network. A system comprising a computer configured to identify orthopedic elements, define the identified orthopedic elements, map the identified orthopedic elements to spatial data confirmed by a first input image and a second input image, thereby defining the determined size dimensions of the identified orthopedic elements in three-dimensional space, and a list of components of an internal prosthesis, and related component size dimensions for the components in the list of internal prosthesis components, wherein the computer is further configured to a recommended size for the components of the internal prosthesis based on the determined size dimensions of the identified orthopedic elements in three-dimensional space.

[0192] In an exemplary embodiment, the system further includes a third input image, the third input image being captured by a tissue transmission imager from a third reference frame, and the third image depicting a calibration jig.

[0193] In exemplary embodiments, the identified orthopedic element is the internal geometric shape of the bone before or after reaming, or before or after broaching.

[0194] In an exemplary embodiment, the computer is configured to run a best fit algorithm to select recommended components for an internal prosthetic implant based on the determined size dimensions of the identified orthopedic element.

[0195] The present invention is not limited to the specific configurations and method steps disclosed herein or shown in the drawings, but should be understood to include any modifications or equivalents of the claims known in the art. Those skilled in the art will understand that the apparatus and methods disclosed herein will find useful.

Claims

1. A system for confirming the position of orthopedic elements and components of artificial joint implants within space, A tissue transmission imaging device, The system receives a first two-dimensional input image captured by the tissue transmission imager from a first reference frame, depicting the calibration jig; a second two-dimensional input image captured by the tissue transmission imager from a second reference frame shifted by a first displacement angle from the first reference frame, depicting the calibration jig; and a third two-dimensional input image captured by the tissue transmission imager from a third reference frame shifted by a second displacement angle from the first and second reference frames, depicting the calibration jig. The system is configured to project spatial data from the first two-dimensional input image, the second two-dimensional input image, and the third two-dimensional input image at the first and second displacement angles, and to define volumetric data, and furthermore, a deep learning network is used. A computer configured to perform the following: the deep learning network identifies orthopedic elements and components of an artificial joint implant from the volumetric data, defines the identified orthopedic elements and identified components of the artificial joint implant, maps the identified orthopedic elements and identified components of the artificial joint implant into the spatial data confirmed by the first, second, and third input images, thereby determining the three-dimensional spatial positions of the identified orthopedic elements and identified components of the artificial joint implant, and further calculates the abduction angle or anteversion angle of the identified components of the artificial joint implant. Equipped with, The first two-dimensional input image, the second two-dimensional input image, and the third two-dimensional input image include the spatial data, A system that uses the aforementioned computer to calculate the abduction angle of the identified component of the artificial joint implant after mapping, or the anteversion angle of the identified component of the artificial joint implant.

2. The system according to claim 1, wherein the deep learning network is further configured to identify a plurality of orthopedic elements and a plurality of components of the artificial joint implant, and to define a plurality of identified orthopedic elements and a plurality of identified components of the artificial joint implant.

3. The system according to claim 2, wherein the first identified component of the plurality of identified components of the artificial joint implant is the acetabular component of the hip prosthesis implant, and the second identified component of the plurality of identified components of the artificial joint implant is the femoral component of the hip prosthesis implant.

4. The system according to claim 1, wherein the identified component of the artificial joint implant is an acetabular shell, and the identified orthopedic element is a reamed acetabulum adjacent to the acetabular shell.

5. The system according to claim 1, wherein the identified component of the artificial joint implant is a femoral stem, and the identified orthopedic element is an intramedullary canal of the femur adjacent to the femoral stem.

6. The system according to claim 1, wherein the identified orthopedic elements are modeled in three dimensions to define the modeled orthopedic elements.

7. The system according to claim 6, wherein the modeled orthopedic elements are displayed on a screen.

8. The system according to claim 1, wherein the identified components of the artificial joint implant are modeled in three dimensions to define the modeled components of the artificial joint implant.

9. The system according to claim 8, wherein the modeled components of the artificial joint implant are displayed on a display.

10. The system according to claim 1, wherein the calculated abduction angle of the identified component of the artificial joint implant is displayed on a display.

11. The system according to claim 1, wherein the calculated anteversion angle of the identified component of the artificial joint implant is displayed on a display.

Images