Systems and methods for creating patient-specific guides for orthopaedic surgery using photogrammetry - Patents.com

Abstract

A system and method for generating a patient-specific surgical guide, comprising: capturing first and second images of an orthopedic element in different reference frames using radiographic imaging technology; detecting spatial data defining anatomical landmarks on or within the orthopedic element using a neural network; applying a mask to the orthopedic element defined by the anatomical landmarks; projecting the spatial data from the first and second images to define volumetric data; applying the neural network to the volumetric data to generate a reconstructed three-dimensional ("3D") model of the orthopedic element; and calculating dimensions of a patient-specific surgical guide configured to abut the orthopedic element.

Description

[Technical field]

[0001] (Reference to Related Application) This application claims the benefit of U.S. Provisional Patent Application No. 63 / 223,844, filed July 20, 2021. The disclosure of this related application is incorporated in its entirety into this disclosure.

[0002] FIELD OF THEINVENTION The present disclosure relates generally to the field of orthopaedic joint replacement surgery, and more specifically to the use of photogrammetry and three-dimensional ("3D") reconstruction techniques to assist surgeons and engineers in planning and performing orthopaedic surgical procedures. [Background technology]

[0003] The new goal of joint replacement surgery is to restore the natural alignment and axis or axes of rotation of the pre-symptomatic joint. However, this goal can be difficult to achieve in practice because the joint includes not only the articulating bones, but also the auxiliary supporting bones and various soft tissues, including cartilage, ligaments, muscles, and tendons. Traditionally, surgeons have avoided completely restoring the natural alignment or estimated alignment angles and other dimensions based on averages derived from a sample of the population. However, these averages often do not take into account the natural variations in a particular patient's anatomy, especially when the patient suffers from a chronic bone degenerative disease such as osteoarthritis.

[0004] In an attempt to address this issue, some medical providers have begun to use computed tomography ("CT") scans and magnetic resonance imaging ("MRI") technologies to explore a patient's internal anatomy to help plan orthopedic surgery. Data from these CT scans and MRIs has further been used to create three-dimensional ("3D") models in digital form. These models can be sent to specialists to design and generate patient-specific instruments (such as custom surgical resection guides) for the procedure. Additive manufacturing techniques (e.g., 3D printing) and other traditional manufacturing techniques can be used to create physical instruments that fit the patient's specific anatomy.

[0005] However, obtaining CT scans and MRIs can be complicated, time-consuming, and expensive. CT scans also tend to expose patients to higher levels of radiation per session than they would otherwise receive using other non-invasive imaging techniques, such as conventional radiography or ultrasound. Furthermore, scheduling considerations may result in CT scan or MRI studies being performed a month or more before the actual surgery. This delay may be exacerbated by the trend toward gradually moving orthopedic procedures to ambulatory surgical centers ("ASCs"). ASCs tend to be smaller facilities that often lack expensive on-site CT scanners and MRI machines. This often forces patients to schedule study appointments at the hospital.

[0006] The increased time between the survey appointment and the surgery increases the risk that the patient's bone and soft tissue anatomy will further deteriorate or change under normal use or due to disease progression. Further deterioration can not only cause further discomfort to the patient, but also negatively affect the usefulness of the survey data to the surgical team. This can be particularly problematic for surgical techniques that aim to restore range of motion based on patient-specific guides made from outdated data and the natural alignment of the joints before the disease. Furthermore, the increased time between the pre-operative survey appointment and the surgery increases the possibility that an exogenous event will adversely affect the data. For example, an accident that dislocates or fractures the bone in the planned surgical area will usually compromise the usefulness of the pre-surgical data. Such risks can be increased in particularly active or particularly frail individuals.

[0007] Additionally, not all patients have access to a CT scan or MRI to create a patient-specific instrument, which may be due in part to the amount of time required to acquire the data, send the data to a medical device design specialist, generate a 3D model of the desired anatomy, create a patient-specific instrument design based on the data or model, generate the patient-specific instrument, ship and transport the patient-specific instrument to a surgical clinic, and sterilize the instrument prior to the procedure, and may not be available depending on the patient's medical insurance and type of illness.

[0008] These techniques, combined with the challenges and availability of accurate preoperative data, can therefore compromise the precise alignment of the prosthetic joint line with the natural pre-patient joint line. Multiple studies have shown that prosthetic joints that alter the natural axis of rotation of the pre-patient joint tend to cause poor function, premature implant wear, and patient dissatisfaction. Summary of the Invention

[0009] Thus, there exists a long felt and unmet need to augment pre- and intra-operative imaging techniques to accurately model surgical joints and other physiology, including bone structure, bone loss, soft tissues, when planning and performing orthopaedic surgical procedures.

[0010] Problems of limited access to traditional preoperative CT and MRI imaging techniques, problems with data accuracy due to deterioration of bone and cartilage between the time of preoperative imaging and the time of surgical procedure, and limitations in determining the natural joint line of a pre-symptomatic joint using currently available intraoperative tools and techniques may be alleviated by exemplary systems and methods for generating a patient-specific surgical drill or resection guide, which include using a deep learning network to identify and model an orthopedic feature and using the deep learning network to calculate dimensions of a patient-specific surgical guide configured to abut the orthopedic feature from input of at least two separate two-dimensional ("2D") input images of the target orthopedic feature, where a first image of the at least two separate 2D input images is captured from a first lateral position and a second image of the at least two separate 2D input images is captured from a second lateral position offset by an offset angle from the first lateral position.

[0011] Radiographs allow for in vivo analysis that can take into account the external sum of passive soft tissue structures and dynamic external forces occurring around the knee, including the effects of ligamentous constraints, load-bearing forces, and muscle activity.

[0012] Creating patient-specific surgical plans and instruments typically uses data from cartilage and bone anatomical structures, such as the knee contour, but can also use data from soft tissue structures. [Brief description of the drawings]

[0013] The foregoing will become apparent from the following more particular description of exemplary embodiments of the disclosure, as illustrated in the accompanying drawings, in which: The drawings are not necessarily to scale, emphasis instead being placed upon illustrating the disclosed embodiments. [Figure 1] FIG. 1 is a flow diagram illustrating the steps of an exemplary method. [Diagram 2]FIG. 11 is a flow diagram illustrating further exemplary method steps. [Diagram 3] FIG. 1 illustrates an anterior view of a simplified exemplary left knee joint. [Figure 4] FIG. 1 is a schematic diagram of a pinhole camera model used to convey how the principles of epipolar geometry can be used to ascertain the location of a point in 3D space from two 2D images taken from different reference frames of a calibrated image detector. [Figure 5A] 1 is an image of a target orthopedic component taken from an anterior-posterior ("AP") position showing an exemplary calibration fixture. [Figure 5B] 5B is an image of the target orthopedic element of FIG. 5A taken from a medial-lateral ("ML") position showing an exemplary calibration fixture. [Figure 6] FIG. 1 is a schematic diagram of a system that uses a deep learning network to identify features (e.g., anatomical landmarks) of a target orthopedic element and generate a 3D model of the target orthopedic element. [Figure 7] FIG. 1 is a schematic diagram of a system configured to generate a model of an orthopedic element and calculate dimensions of a patient-specific surgical guide configured to abut the orthopedic element from using two or more tissue-penetrating flattened input images of the same target orthopedic element from detectors calibrated at offset angles. [Figure 8] FIG. 1 is a schematic diagram illustrating how features including the surface of a target orthopedic element (e.g., anatomical landmarks) can be identified using a CNN-type deep learning network. [Figure 9] FIG. 1 is a schematic diagram of an exemplary system. [Figure 10] FIG. 2 is a flow diagram illustrating the steps of an exemplary method. [Figure 11] FIG. 2 is a bottom view of an exemplary patient-specific surgical guide made according to any of the exemplary methods disclosed herein. [Figure 12] FIG. 13 is an underside view of another exemplary patient-specific surgical guide made according to any of the exemplary methods disclosed herein. [Figure 13]An exemplary patient-specific femoral resection guide mount is shown rigidly engaged to the patient's distal femur, and an exemplary patient-specific tibial resection guide mount is shown rigidly secured to the patient's proximal tibia. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] The following detailed description of the preferred embodiments is presented for illustrative purposes only to aid in understanding, and is not intended to be exhaustive or to limit the scope and spirit of the present invention. The embodiments have been chosen and described in order to best explain the principles of the invention and its practical application. Those skilled in the art will recognize that many modifications can be made to the invention disclosed herein without departing from the scope and spirit of the invention.

[0015] Unless otherwise stated, like reference characters designate corresponding parts throughout the several views. Although the drawings depict embodiments of various features and components according to the present disclosure, the drawings are not necessarily to scale and certain features may be exaggerated to better illustrate embodiments of the present disclosure, and such illustrations should not be construed as limiting the scope of the present disclosure.

[0016] Except as otherwise expressly stated herein, the following rules of interpretation apply herein: (a) all words used herein shall be construed as to such gender or number (singular or plural) as required in such context; (b) the singular terms "a," "an," and "the" as used herein and in the appended claims include plural references unless the context clearly indicates otherwise; (c) the antecedent "about" applied to a recited range or value indicates an approximation having a deviation of the range or value known in the art or expected from measurement; (d) unless otherwise indicated, the words "herein," "hereby," "hereinbefore," and "hereinafter," and words of similar import, refer to this specification as a whole and not to any particular paragraph, claim, or other subdivision; (e) the description headings are for convenience only and do not control or affect the meaning of any portion of this specification; (f) "or" and "any" are not exclusive, and "include," "including" are not limiting. Furthermore, the words "comprising," "having," "including," and "containing" are to be construed as open-ended terms (i.e., meaning "including, but not limited to").

[0017] References herein to "one embodiment," "an embodiment," "exemplary embodiment," and the like indicate that the described embodiment may include a particular feature, structure, or characteristic, but not all embodiments may necessarily include the particular feature, structure, or characteristic. Moreover, such phrases do not necessarily refer to the same embodiment. Moreover, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is implied that it is within the knowledge of one of ordinary skill in the art to affect such feature, structure, or characteristic in connection with other embodiments, whether or not explicitly described.

[0018] To the extent necessary to provide narrative support, the subject matter and / or text of the appended claims are incorporated herein by reference in their entirety.

[0019] The recitation of ranges of values ​​herein is merely intended to serve as a shorthand method of individually referring to each separate value within any subrange therebetween, unless otherwise expressly indicated herein. Each separate value within a recited range is incorporated into the specification or claims as if each separate value were individually recited herein. When a specific range of values ​​is provided, it is understood that each intervening value, to the nearest tenth of the unit of the lower limit between the upper and lower limits of that range, and any other recited or intervening value within the recited range of that subrange, is included herein, unless the context clearly dictates otherwise. All subranges are also included. The upper and lower limits of these smaller ranges are also included therein, subject to any specific and expressly excluded limitations in the recited range.

[0020] It should be noted that some of the terms used herein are relative terms: for example, the terms "upper" and "lower" are relative to one another in location, i.e., the upper component is located higher than the lower component in each orientation, but these terms may change if the orientation is reversed.

[0021] The terms "horizontal" and "vertical" are used to indicate orientation relative to an absolute reference, i.e., the earth's surface level. However, these terms should not be interpreted as requiring structures to be absolutely parallel or absolutely perpendicular to each other. For example, a first vertical structure and a second vertical structure are not necessarily parallel to each other. The terms "upper" and "lower" or "base" are used to refer to a place or surface where the upper is always higher than the lower or base, relative to an absolute reference, i.e., the earth's surface. The terms "upward" and "downward" are also relative to an absolute reference. An upward flow always opposes the earth's gravitational force.

[0022] Orthopedic procedures often involve surgery on a patient's joints. It will be appreciated that a joint typically includes multiple orthopedic elements. It will be further appreciated that the exemplary methods and systems described herein may be applied to a variety of orthopedic elements. The example described with reference to Figures 3, 5A, and 5B relates to an exemplary knee joint for illustration purposes. It will be appreciated that the "orthopedic elements" 100 referenced throughout this disclosure are not limited to the anatomy of a knee joint, but may include any skeletal structure and associated soft tissues, such as tendons, ligaments, cartilage, and muscles. A non-limiting list of exemplary orthopedic elements 100 includes any partial or complete bone of the body, including but not limited to the femur, tibia, pelvis, vertebrae, humerus, ulna, radius, scapula, skull, fibula, clavicle, mandible, ribs, carpals, metacarpals, tarsals, metatarsals, phalanges, or any associated tendons, ligaments, skin, cartilage, or muscles. It will be appreciated that the exemplary surgical field 170 may include several target orthopedic elements 100 .

[0023] 3 is an anterior-posterior view of a simplified left knee joint 100 (i.e., an exemplary joint surgical area 170) in extension. The exemplary knee joint 100 includes several orthopedic elements, including the femur 105, the tibia 110, the fibula 111, the patella (not shown), a resected tibial plateau 112, femoral articular cartilage 123, a medial collateral ligament ("MCL") 113 that engages the distal femur 105 to the proximal tibia 110 on the medial side M, and a lateral collateral ligament ("LCL") 122 that engages the distal femur 105 to the fibula 111 on the lateral side L. The femoral articular cartilage 123 has a thickness T, and the femoral articular cartilage 123 engages the bone surface 106 of the distal femur 105. The distal femur further includes a medial condyle 107 and a lateral condyle 103 (collectively the "femoral condyles"). The distal femur 105 is separated from the proximal tibia 110 by the femoro-tibial gap 120. The perspective view of Figure 3 is an example of using radiographic techniques to capture a first image of an orthopaedic element (however, in Figure 3, multiple orthopaedic elements are shown, namely the femur 105, the tibia 110, the fibula 111, the articular cartilage 123, the MCL 113, and the LCL 122) in a first frame of reference (also see Figure 5A, which shows the orthopaedic element of interest taken from the first frame of reference, the first frame of reference capturing the orthopaedic element of interest in an AP position).

[0024] FIG. 5B shows the same target orthopedic feature in a second reference frame, which captures the target orthopedic feature at the ML position.

[0025] In recent years, it has become possible to create 3D models of the surgical area using 2D images, such as x-rays. These models can be used preoperatively to plan the surgery much closer to the date of the actual surgery. Furthermore, these preoperative 3D models serve as native models onto which the surgical instruments themselves can be configured to fit precisely.

[0026] However, radiographs have not typically been used previously as input for 3D models due to concerns about image resolution and accuracy. Radiographs are 2D representations of 3D space. Thus, 2D radiographs necessarily distort the imaged object compared to the actual object present in the third dimension. Furthermore, objects through which the x-rays pass can deflect the path of the x-rays as they travel from the x-ray source (typically the anode of the x-ray machine) to the x-ray detector (which may include, as non-limiting examples, an x-ray image intensifier, phosphorus material, a flat panel detector (FPD) (including indirect conversion FPD and direct conversion FPD), or any number of digital or analog x-ray sensors or x-ray film). Imperfections in the x-ray machine itself or in its calibration can also impair the usefulness of x-ray photogrammetry and 3D model reconstruction. In addition, the emitted x-ray photons have different energies. When X-rays interact with matter placed between the X-ray source and the detector, noise and artifacts can be generated in part due to Compton and Rayleigh scattering, the photoelectric effect, exogenous variations in the environment, or intrinsic variations in the X-ray generating unit, the X-ray detector, and / or the processing unit or display.

[0027] Furthermore, in a single 2D image, the 3D data of the actual object is lost. Thus, there is no data that a computer can use from a single 2D image to reconstruct a 3D model of the actual 3D object. For this reason, CT scans, MRIs, and other imaging techniques that store three-dimensional data have often been the preferred inputs for reconstructing a model of one or more target orthopedic elements (i.e., reconstructing a 3D model from actual 3D data to generally generate a more accurate and higher resolution model). However, certain exemplary embodiments of the present disclosure discussed below overcome these problems by using deep learning networks to improve the accuracy of reconstructed 3D models generated from X-ray input images.

[0028] As an example, a deep learning algorithm, such as a convolutional neural network, can be used to generate a 3D model from a set of at least two 2D radiological images of a patient's surgical field. In such a method, the deep learning algorithm can generate a model from projective geometry data from each of the 2D images. The deep learning algorithm can have the advantage of being able to generate masks of different orthopedic elements (e.g., bones, soft tissues, etc.) in the surgical field, as well as being able to calculate the volume of the imaged or target orthopedic element 100.

[0029] FIG. 1 is a flow diagram outlining steps of an exemplary method for generating a patient-specific surgical guide (eg, a patient-specific drill guide or a patient-specific resection guide). The method includes step 1a of calibrating a radiographic imager 1800 to determine a mapping relationship between radiographic points and corresponding spatial coordinates to define spatial data 43; step 2a of capturing a first image 30 of the orthopedic element 100 using a radiographic technique, the first image 30 defining a first reference frame 30a; step 3a of capturing a second image 50 of the orthopedic element 100 using a radiographic technique, the second image 50 defining a second reference frame 50a, the first reference frame 30a being offset from the second reference frame 50a by an offset angle θ; and step 4a of projecting the spatial data 43 from the first radiographic image 30 of the target orthopedic element 100 and the spatial data 43 from the second radiographic image 50 of the target orthopedic element 100 to define volumetric data 75. The method includes a step 5a, using a deep learning network to detect a target orthopedic element 100 using spatial data 43, the spatial data 43 defining anatomical landmarks on or within the target orthopedic element 100; a step 6a, using a deep learning network to apply a mask to the target orthopedic element 100 defined by the anatomical landmarks, wherein spatial data 43 including image points located within the mask region of either the first image 30 or the second image 50 is given a first value, and spatial data 43 including image points located outside the mask region of both the first image 30 and the second image 50 is given a second value, the second value being different from the first value; and a step 7a, calculating dimensions of a patient-specific surgical guide 500 configured to abut the orthopedic element.

[0030] In an exemplary embodiment, the exemplary method may further include step 8b of applying a deep learning network to the volumetric data 75 to generate a reconstructed 3D model of the orthopedic element. In other exemplary embodiments, steps 5a or 5b may include using a deep learning network to detect spatial data 43 defining anatomical landmarks on or within the orthopedic element 100 (see FIG. 2 ).

[0031] The above examples are provided for purposes of aiding understanding and are not intended to limit the scope of the present disclosure. Any method for generating a 3D model from 2D radiographic images of the same object taken from at least two lateral positions is considered to be within the scope of the present disclosure.

[0032] 4 and 6 show how a first input image 30 and a second input image 50 may be combined to create a volume 61 containing volumetric data 75 (FIG. 6). FIG. 4 illustrates basic principles of epipolar geometry that may be used to transform spatial data 43 from each input image 30, 50 into volumetric data 75. The spatial data 43 is a set of image points (e.g., x, y coordinates) that are mapped to corresponding spatial coordinates (e.g., x and y coordinates) in a given input image 30, 50. L , X R It will be understood that the set of

[0033] FIG. 4 is a simplified schematic diagram of an oblique projection described by the pinhole camera model. FIG. 4 conveys basic concepts related to computer stereo vision, but is not meant to be the only way a 3D model can be reconstructed from 2D stereo images. In this simplified model, rays emanate from the optical center (i.e., the point in the lens where rays of electromagnetic radiation (e.g., visible light, x-rays, etc.) from the target object are assumed to intersect in the imager's sensor or detector array 33 (FIG. 9)). The optical center is indicated in FIG. 4 as point O L , O R In practice, the image plane (see 30a, 50a) is usually located at the optical center (e.g., OL , O R ), and the actual optical centre is projected as a point onto the detector array 33, but a virtual image plane (see 30a, 50a) is presented here to illustrate the principle more concisely.

[0034] The first input image 30 is taken from a first reference frame 30a, and the second input image 50 is taken from a second reference frame 50a that is different from the first reference frame 30a. Each image includes a matrix of pixel values. The first and second reference frames 30a, 50a are preferably offset from each other by an offset angle θ. The offset angle θ may represent the angle between the x-axis of the first reference frame 30a and the x-axis of the second reference frame 50a. In other words, the angle between the orientation of the orthopedic element in the first image and the orthopedic element in the second image may be known as the "offset angle."

[0035] point e L is the optical center O of the second input image on the first input image 30 R This is the location of point e R is the optical center O of the first input image on the second input image 50 L This is the location of point e L and e R is known as an "epipole" or epipolar point, and is the line O L -O R Above. Points X and O L , O R defines the epipolar plane.

[0036] Since the actual optical center is the point where the incident rays of electromagnetic radiation from the target object are assumed to intersect within the detector lens, in this model the rays of electromagnetic radiation are actually referred to as the optical center O for the purpose of visualizing how the location of a 3D point X in 3D space can be ascertained from two or more input images 30, 50 captured from detectors 33 of known relative positions. L , O R Each point of the first input image 30 (e.g., X L) corresponds to a line in 3D space, then the corresponding point (e.g., X R ) can be found in the second input image, then these corresponding points (e.g., X L , X R ) must be the projection of a common 3D point X. Therefore, the corresponding image points (e.g., X L , X R ) should intersect at a 3D point X. In general, the value of X should be the sum of all corresponding image points (e.g., X L , X R ), a 3D volume 61 containing volumetric data 75 may be replicated from two or more input images 30, 50. The value of any given 3D point X may be triangulated in a variety of ways. A non-limiting list of exemplary computational methods includes the midpoint method, the direct linear transformation method, the fundamental matrix method, the intersection method, and the bundle adjustment method.

[0037] As used herein, an "image point" (e.g., X L , X R It will be appreciated that a 3D point X may refer to a point in space, a pixel, a portion of a pixel, or a collection of adjacent pixels. It will also be appreciated that as used herein, a 3D point X may represent a point in 3D space. In certain exemplary applications, the 3D point X may be represented as a voxel, a portion of a voxel, or a collection of adjacent voxels.

[0038] However, before the principles of epipolar geometry can be applied, the position of each image detector 33 relative to the other image detector(s) 33 must be determined (or the position of the single image detector 33 must be determined at the time the first image 30 is taken, and the adjusted position of the single image detector 33 must be known at the time the second image 50 is taken). It is also desirable to determine the focal length and optical center of the imager 1800. To verify this in practice, the image detector 33 (or multiple image detectors) is first calibrated. Figures 5A and 5B show calibration fixtures 973A, 973B for a target orthopedic element 100. In these figures, the exemplary orthopedic element 100 is a distal aspect of the femur 105 and a proximal aspect of the tibia 110 including the knee joint. The proximal fibula 111 in Figures 5A and 5B is another orthopedic element 100 imaged. The patella 901 shown in FIG. 5B is another orthopedic element 100 .

[0039] FIG. 5A is an anterior-posterior view of an exemplary orthopedic element 100 (i.e., FIG. 5A represents a first image 30 taken from a first reference frame 30a (e.g., a first lateral position)). A first calibration fixture 973A is attached to a first holding assembly 974A. The first holding assembly 974A may include a first padded support 971A engaged with a first strap 977A. The first padded support 971A is attached to the outside of the patient's thigh via the first strap 977A. The first holding assembly 974A supports the first calibration fixture 973A, which is desirably oriented parallel to the first reference frame 30a (i.e., perpendicular to the detector 33). Similarly, a second calibration fixture 973B attached to a second holding assembly 974B may be provided. The second holding assembly 974B may include a second padded support 971B engaged with a second strap 977B. The second padded support 971B is attached to the outside of the patient's calf via the second strap 977B. The second holding assembly 974B supports a second calibration fixture 973B that is preferably parallel to the first reference frame 30a (i.e., perpendicular to the detector 33). The calibration fixtures 973A, 973B are preferably positioned sufficiently away from the target orthopedic elements 100 such that the calibration fixtures 973A, 973B do not overlap any of the target orthopedic elements 100.

[0040] FIG. 5B is a medial-lateral view of an exemplary orthopedic element 100 (i.e., FIG. 5B represents a second image 50 taken from a second reference frame 50a (e.g., a second lateral position)). In the illustrated example, the medial-lateral reference frame 50a is rotated or "offset" 90° from the anterior-posterior first reference frame 30a. A first calibration fixture 973A is attached to a first holding assembly 974A. The first holding assembly 974A may include a first padded support 971A engaged to a first strap 977A. The first padded support 971A is attached to the lateral side of the patient's thigh via the first strap 977A. The first holding assembly 974A supports the first calibration fixture 973A, desirably parallel to the second reference frame 50a (i.e., perpendicular to the detector 33). Similarly, a second calibration fixture 973B may be provided attached to a second holding assembly 974B. The second holding assembly 974B may include a second padded support 971B engaged with a second strap 977B. The second padded support 971B is attached to the outside of the patient's calf via the second strap 977B. The second holding assembly 974B supports the second calibration fixture 973B, which is preferably parallel to the second reference frame 50a (i.e., perpendicular to the detector 33). The calibration fixtures 973A, 973B are preferably positioned sufficiently away from the target orthopedic elements 100 such that the calibration fixtures 973A, 973B do not overlap any of the target orthopedic elements 100.

[0041] The patient may desirably be positioned in a standing position (i.e., with the leg extended) since the knee joint is stable in this orientation (see FIG. 9). Preferably, the distance of the patient relative to the imager should not be changed during acquisition of the input images 30, 50. The first and second images 30, 50 need not capture the entire leg; rather, the images may be focused on the joint of interest in the surgical field 170.

[0042] It will be appreciated that only a single calibration fixture 973 may be used depending on the target orthopedic features 100 being imaged and modeled. Similarly, more than one calibration fixture may be used if a particularly long collection of orthopedic features 100 is being imaged and modeled.

[0043] Each calibration fixture 973A, 973B is preferably of a known size. Each calibration fixture 973A, 973B preferably has at least four or more calibration points 978 distributed throughout. The calibration points 978 are distributed in a known pattern in which the distance of one point 978 relative to another point is known. The distance of the calibration fixture 973 from the orthopedic element 100 may also preferably be known. For calibration of a radiogrammetry system, the calibration points 978 may be preferably defined by metal structures on the calibration fixture 973. Metals typically absorb most of the x-ray beam that contacts them. Thus, metals typically appear very bright compared to materials that do not absorb x-rays much (such as air cavities or fatty tissue). Common exemplary structures that define the calibration points include reseau crosses, circles, triangles, pyramids, and spheres.

[0044] These calibration points 978 may be on the 2D surface of the calibration fixture 973, or the 3D calibration points 978 may be captured as 2D projections from a given image reference frame. In either situation, the 3D coordinate (commonly referred to as the z coordinate) may be set equal to zero for all calibration points 978 captured in the image. The distance between each calibration point 978 is known. These known distances may be expressed as x,y coordinates on the image sensor / detector 33. To map a point in 3D space to a 2D coordinate pixel on the sensor 33, a dot product of the detector's calibration matrix, the extrinsic matrix, and the homologous coordinate vector of the real 3D point may be used. This maps the real world coordinates of the point in 3D space to the calibration fixture 973. In other words, this generally allows the x,y coordinates of a real point in 3D space to be accurately transformed into the 2D coordinate plane of the image detector sensor 33 to define spatial data 43 (see FIG. 4).

[0045] The above calibration method is provided as an example. It will be understood that any method suitable for calibrating a radiogrammetry system is considered to be within the scope of this disclosure. A non-limiting list of other radiogrammetry system calibration methods includes the use of rezo plates, Zhang's method, bundle adjustment, direct linear transformation, maximum likelihood estimation, k-nearest neighbor regression approach ("kNN"), other deep learning methods, or combinations thereof.

[0046] FIG. 6 shows how a calibrated input image 30, 50, when oriented along a known offset angle θ, can be backprojected into a 3D volume 61 that contains two channels 65, 66. The first channel 65 projects all image points (e.g., X L etc.), and the second channel 66 contains all image points (e.g., X R3D ray 75. The backprojected 2D input images 30, 50 include a set of 3D ray projections 61, 75, etc. That is, each image point (e.g., pixel) is replicated onto its associated backprojected 3D ray. Then, using epipolar geometry, a volume 61 of the imaged surgical region 170 including volumetric data 75 may be generated from these backprojected 2D input images 30, 50.

[0047] Referring to FIG. 6, the first image 30 and the second image 50 preferably have known image dimensions. The dimensions may be in pixels. For example, the first image 30 may have dimensions of 128×128 pixels. The second image 50 may have dimensions of 128×128 pixels. The dimensions of the input images 30, 50 used in a particular calculation preferably have consistent dimensions. Consistent dimensions may be desirable for later defining a cubic working area of ​​a typical volume 61 (e.g., a 128×128×128 cube). As seen in FIG. 4, the offset angle θ is preferably 90°. However, in other exemplary embodiments, other offset angles θ may be used.

[0048] In the illustrated example, each of the 128×128 pixel input images 30, 50 is replicated 128 times across the length of the adjacent input image to create a volume 61 having dimensions of 128×128×128 pixels. That is, the first image 30 is copied and stacked behind itself by 128 pixels, one copy per pixel, while the second image 50 is copied and stacked behind itself by 128 pixels such that the stacked images overlap, thereby creating the volume 61. In this manner, the volume 61 can be said to include two channels 65, 66, where the first channel 65 includes the first image 30 replicated n times across the length of the second image 50 (i.e., the x-axis of the second image 50) and the second channel 66 includes the second image 50 replicated m times across the length of the first image 30 (i.e., the x-axis of the first image 30), where "n" and "m" are the commanded image length expressed as the number of pixels (or other dimensions in other exemplary embodiments) that comprise the commanded image length. When the offset angle θ is known, each lateral slice (also known as an "axial slice" by some radiologists) of the volume 61 creates an epipolar plane that includes voxels backprojected from the pixels that comprise the two epipolar lines. In this manner, the spatial data 43 from the first image 30 of the target orthopedic element 100 and the spatial data 43 from the second image 50 of the target orthopedic element 100 are projected to define the volumetric data 75. Using this volumetric data 75, a 3D representation can be reconstructed using epipolar geometry principles as described above, which is geometrically consistent with the information in the input images 30,50.

[0049] In the exemplary system and method for generating a patient-specific surgical guide 500 using a deep learning network, where the deep learning network is a CNN, detailed examples are provided showing how the CNN can be structured and trained. All architectures of CNN are considered within the scope of this disclosure. Common CNN architectures include, for example, LeNet, GoogLeNet, AlexNet, ZFNet, ResNet, and VGGNet.

[0050] FIG. 11 is a bottom view of a patient-specific surgical guide 500 made according to any exemplary method disclosed herein. In FIG. 13, the patient-specific surgical guide 500 is rigidly engaged to an orthopedic element 100, which in the illustrated example is a femur 105. The patient-specific surgical guide 500 may be formed from a resilient polymeric material. The patient-specific surgical guide 500 shown in FIG. 11 is a patient-specific femoral resection guide mount 500a configured to rigidly engage the condyles 107, 103 of a patient-specific surgical subject femur 105. The illustrated exemplary patient-specific femoral resection guide mount 500a includes a body 42 having a resection slot 52 extending laterally through the body 42, a bifurcated condyle yoke 25, and a guide receptacle 24. The bifurcated condyle yoke 25 includes a pair of spaced apart arms 31, 41 projecting outwardly from the body 42. The first arm 31 has a first mating surface 36 that is complementary to an anatomical surface feature of a selected region of the patient's natural bone (e.g., one of the patient's distal femoral condyles). Similarly, the second arm 41 has a second mating surface 40 that is complementary to an anatomical surface feature of a selected region of the patient's natural bone (e.g., the other of the patient's distal femoral condyles). A through hole 38 may optionally extend through each of the spaced arms 31, 41. A pin may optionally be inserted through each of the through holes 38 to further secure the illustrated patient-specific surgical guide 500 to the patient's natural bone.

[0051] In an exemplary embodiment, the curved body 42 of the patient-specific surgical guide 500 can store potential energy when the patient-specific surgical guide 500 abuts the surface topography of the patient's exposed natural bone (see 106 in FIG. 3 ). In this manner, the curved body 42 and complementary mating surfaces 36, 40 that match the surface topography of the patient's exposed natural bone can "press-fit" (i.e., frictionally secure) the patient-specific surgical guide 500 into a desired position on the patient's exposed femoral condyle.

[0052] Once the patient-specific surgical guide 500 is abutted and firmly engaged against a complementary portion of the patient's exposed bone at the desired location, the surgeon can insert a surgical saw through the resection slot 52 to resect the patient's distal femur 105 at the desired location in preparation for sizing and fitting the implant. It is contemplated that creating a custom surgical guide 500 in a manner consistent with the present disclosure may enable the surgical saw to be positioned more accurately and precisely, in closer time, and using less energy than previously possible.

[0053] 12 is a bottom view of another exemplary patient-specific surgical guide 500 made according to any exemplary method disclosed herein. In FIG. 12, the patient-specific surgical guide 500 is a tibial resection guide mount 500b.

[0054] The illustrated exemplary patient-specific tibial resection guide mount 500b includes a body 79 having a resection slot 51 extending laterally therethrough, a bifurcated condyle yoke 64, and a guide receptacle 24. The bifurcated condyle yoke 64 includes a pair of spaced apart arms 62, 63 projecting outwardly from the body 79. The first arm 62 has a first mating surface 53 that is complementary to an anatomical surface feature of a selected region of the patient's natural bone (e.g., one of the patient's proximal tibial half-plateau condyles). Similarly, the second arm 63 has a second mating surface 54 that is complementary to an anatomical surface feature of a selected region of the patient's natural bone (e.g., the other of the patient's proximal tibial half-plateau). Through holes 38 may optionally extend through the body 79. A pin may optionally be inserted through each of the through holes 38 to further secure the illustrated patient-specific tibial resection guide mount 500b to the patient's natural bone.

[0055] In an embodiment, the first and second mating surfaces 53, 54 of the patient-specific tibial resection guide mount 500b may enable the patient-specific tibial resection guide mount 500b to be secured in a precise location on the patient's proximal tibia via friction. Once properly seated and secured, the surgeon may insert a surgical saw through the tibial resection slot 51 to resect the proximal tibia plateau.

[0056] FIG. 13 shows a patient-specific femoral resection guide mount 500a rigidly engaged to the patient's distal femur 105 and a patient-specific tibial resection guide mount 500b rigidly secured to the patient's proximal tibia 110.

[0057] Because the patient-specific surgical guide 500 is designed and manufactured using technical specifications derived from the 3D spatial data, which was derived from two radiographic images of the orthopedic element 100 taken from different reference frames, the patient-specific surgical guide 500 precisely fits the orthopedic element 100 per the pre-operative planning. Furthermore, because radiographic images are generally more efficient and easier to obtain than CT or MRI scans, it is contemplated that pre-operative planning can be performed closer to the scheduled surgical procedure date, thereby mitigating the possibility of changes between the pre-operative plan and the actual anatomy on the day of surgery.

[0058] It is further contemplated that pre-operative planning may occur on the same day as the scheduled surgery, particularly if additive manufacturing (e.g., 3D printing machines) or subtractive manufacturing (e.g., CNC machines) machines are present on-site or locally. For example, a patient may undergo pre-operative imaging and planning in the morning and have surgery scheduled in the afternoon.

[0059] Preferably, the methods disclosed herein may be implemented on a computer platform having hardware such as one or more central processing units (CPU), a random access memory (RAM), and / or input / output (I / O) interface(s) (see FIG. 7 ).

[0060] In yet other embodiments, the volume of the orthopedic element may be calculated. It will be appreciated that any of the disclosed calculations or the results of any such calculations may, optionally, be displayed on a display.

[0061] It is further contemplated that the exemplary methods disclosed herein may be used for pre-operative planning, intra-operative planning or execution, or post-operative evaluation of implant placement and function.

[0062] 9, an exemplary system for calculating dimensions of a patient-specific surgical guide 500 configured to abut or be firmly engaged with a target orthopedic element 100 includes a radiographer 1800 (FIG. 9) including an emitter 21 and a detector 33, the detector 33 of the radiographer 1800 capturing a first image 30 (FIGS. 4 and 5A) at a first translation position 30a (FIGS. 4 and 5A) and a second image 50 (FIGS. 4 and 5B) at a second lateral position 50a (FIGS. 4 and 5B). The orthopedic surgical system may include a radiographer 1800 that captures an image of the patient-specific surgical guide 500 (FIGS. 4 and 5B), where the first lateral position 30a is offset from the second lateral position 50a by an offset angle θ (FIG. 4), a transmitter 29 (FIG. 9), and a calculator 1600 (see FIG. 7 for further details), where the transmitter 29 transmits the first image 30 and the second image 50 from the detector 33 to the calculator 1600, which is configured to calculate a surface topography of the target orthopedic element 100. In certain exemplary embodiments, the calculator 1600 may be configured to calculate dimensions of a mating surface of the patient-specific surgical guide 500 that is complementary to the surface topography of a portion of the target orthopedic element 100.

[0063] In certain exemplary embodiments, the exemplary system may further comprise a display 19 .

[0064] In certain exemplary embodiments, the exemplary system may further include a manufacturing machine 18. In exemplary embodiments including a manufacturing machine 18, the manufacturing machine 18 may be an additive manufacturing machine. In such embodiments, the additive manufacturing machine may be used to manufacture the 3D model 1100 of the target orthopaedic element or a physical 3D model of the patient-specific surgical guide 500. By way of example, 3D manufacturing techniques may include, but are not limited to, stereolithography and laser sintering.

[0065] FIG. 9 is a schematic diagram of an exemplary system with a radiographer 1800 including an X-ray source 21, such as an X-ray tube, a filter 26, a collimator 27, and a detector 33. In FIG. 9, the radiographer 1800 is shown from top to bottom. A patient 1 is placed between the X-ray source 21 and the detector 33. The radiographer 1800 may be mounted on a rotatable gantry 28. The radiographer 1800 may take a radiograph of the patient 1 from a first reference frame 30a. The gantry 28 may then rotate the radiographer 1800 by an offset angle (preferably 90°). The radiographer 1800 may then take a second radiograph 50 from a second reference frame 50a. It will be understood that other exemplary embodiments may include using multiple input images taken at multiple offset angles θ. In such embodiments, the offset angle may be less than 90° or more than 90° between adjacent input images.

[0066] It will be understood that the offset angle need not be exactly 90 degrees in all embodiments. An offset angle having a value within the range of ±45 degrees is contemplated to be sufficient. In other exemplary embodiments, the operator may take three or more images of the orthopedic element using radiographic imaging techniques. It is contemplated that each subsequent image after the second image may define a subsequent image frame of reference. For example, a third image may define a third frame of reference, a fourth image may define a fourth frame of reference, an nth image may define an nth frame of reference, and so on.

[0067] In an exemplary embodiment including three input images and three separate reference frames, each of the three input images desirably has an offset angle θ of approximately 60 degrees relative to one another. In an exemplary embodiment including four input images and four separate reference frames, the offset angle θ is desirably 45 degrees from the adjacent reference frame. In an exemplary embodiment including five input images and five separate reference frames, the offset angle θ is desirably approximately 36 degrees from the adjacent reference frame. In an exemplary embodiment including n images and n separate reference frames, the offset angle θ is desirably 180 / n degrees.

[0068] It is further contemplated that embodiments involving multiple images, particularly three or more images, do not necessarily have to have regular and consistent offset angles. For example, an exemplary embodiment involving four images and four separate reference frames may have a first offset angle of 85 degrees, a second offset angle of 75 degrees, a third offset angle of 93 degrees, and a fourth offset angle of 107 degrees.

[0069] The transmitter 29 then transmits the first image 30 and the second image 50 to the computer 1600. The computer 1600 can apply a deep learning network to calculate the dimensions of the mating surface of the patient-specific surgical guide 500 that is complementary to the surface topography of the portion of the target orthopedic element 100 in any manner consistent with this disclosure. FIG. 9 further illustrates that the output of the computer 1600 is transmitted to the manufacturing machine 18. The manufacturing machine 18 can be an additive manufacturing machine such as a 3D printer (e.g., stereolithography and laser sintering manufacturing machine), or the manufacturing machine can be a subtractive manufacturing machine such as a computer numerical control ("CNC") machine. In yet another exemplary embodiment, the manufacturing machine 18 can be a casting mold. The manufacturing machine 18 can use the output data from the computer 1600 to generate a physical model of one or more 3D models 1100 of the target orthopedic element. In this manner, the manufacturing machine 18 can be said to be "configured to generate" at least a partial physical model of the identification surface of the orthopedic element 100. In an embodiment, the manufacturing machine can be used to generate a physical 3D model of the patient-specific surgical guide 500.

[0070] 9 also illustrates another embodiment in which output data from the calculator 1600 is transmitted to the displays 19. A first display 19a shows a virtual 3D model of the patient-specific surgical guide 500. A second display 19b shows a virtual 3D model 1100 of the identified target orthopedic features.

[0071] The display 19 may take the form of a screen. In other exemplary embodiments, the display 19 may include a glass or plastic surface worn or held by the surgeon or other people in the operating room. Such a display 19 may include a portion of an augmented reality device, such that the display shows a 3D model in addition to the bearer's field of view. In certain embodiments, such a 3D model may be superimposed on the actual surgical joint. In still other exemplary embodiments, the 3D model may be "locked" to one or more features of the surgical orthopedic element 100, thereby maintaining the virtual position of the 3D model relative to one or more features of the surgical orthopedic element 100 regardless of the movement of the display 19. It is still further contemplated that the display 19 may include a portion of a virtual reality system in which the entire field of view is simulated.

[0072] Although x-rays from x-ray imaging systems may be desirable because x-rays are relatively inexpensive compared to CT scans and because some x-ray imaging system equipment, such as fluoroscopy systems, is generally compact enough to be used intraoperatively, nothing in this disclosure limits the use of 2D images to x-rays, and nothing in this disclosure limits the type of imaging system to x-ray imaging systems, unless expressly claimed otherwise. Other 2D images may include, for example, CT images, CT fluoroscopy images, fluoroscopy images, ultrasound images, positron emission tomography ("PET") images, and MRI images. Other imaging systems may include, for example, CT, CT fluoroscopy, fluoroscopy, ultrasound, PET, and MRI systems.

[0073] Preferably, the exemplary method may be implemented on a computer platform (e.g., computer 1600) having hardware such as one or more central processing units (CPUs), a random access memory (RAM), and / or input / output (I / O) interface(s). An example of the architecture of the exemplary computer 1600 is provided below with reference to FIG.

[0074] FIG. 7 generally illustrates a block diagram of an exemplary computing device 1600 on which one or more of the methods discussed herein may be performed according to some exemplary embodiments. In certain exemplary embodiments, computing device 1600 may operate on a single machine. In other exemplary embodiments, computing device 1600 may include connected (e.g., networked) machines. Examples of networked machines that may include exemplary computing device 1600 include, for example, cloud computing configurations, distributed host configurations, and other computer cluster configurations. In a network configuration, one or more of computing devices 1600 may operate as a client machine, a server machine, or both a server-client machine. In an exemplary embodiment, computing device 1600 may reside on a personal computer ("PC"), a mobile phone, a tablet PC, a web appliance, a personal digital assistant ("PDA"), a network router, a bridge, a switch, or any machine capable of executing instructions that specify actions to be taken by the machine or by a second machine controlled by the machine.

[0075] Exemplary machines that may include the exemplary computing device 1600 may include, by way of example, components, modules, or similar mechanisms capable of performing logical functions. Such machines may include tangible entities (e.g., hardware) capable of performing specified operations during operation. As an example, the hardware may be hardwired (e.g., specifically configured) to perform certain operations. As an example, such hardware may include a configurable execution medium (e.g., circuits, transistors, logic gates, etc.) and a computer-readable medium having instructions, which, when operated, configure the execution medium to perform certain operations. The configuration may be via a loading mechanism or under the direction of the execution medium. The execution medium selectively communicates to the computer-readable medium when the machine is operating. As an example, when the machine is in operation, the execution medium may be configured by a first set of instructions to perform a first action or set of actions at a first time, and then reconfigured by a second set of instructions to perform a second action or set of actions at a second time.

[0076] The exemplary computing device 1600 may include a hardware processor 1697 (e.g., a CPU, a graphics processing unit (“GPU”), a hardware processor core, or any combination thereof), a main memory 1696, and a static memory 1695, some or all of which may communicate with each other via an interlink (e.g., a bus) 1694. The computing device 1600 may further include a display unit 1698, an input device 1691 (preferably an alphanumeric or alphanumeric input device such as a keyboard), and a user interface (“UI”) navigation device 1699 (e.g., a mouse or stylus). In an exemplary embodiment, the input device 1691, the display unit 1698, and the UI navigation device 1699 may be touch screen displays. In an exemplary embodiment, the display unit 1698 may include holographic lenses, glasses, goggles, other eyewear, or other AR or VR display components. For example, the display unit 1698 may be worn on the user's head and provide a head-up display to the user. The input device 1691 may include a virtual keyboard (e.g., a keyboard displayed virtually in a virtual reality ("VR") or augmented reality ("AR") setting) or other virtual input interface.

[0077] The computing device 1600 may further include a storage device (e.g., a drive unit) 1692, a signal generator 1689 (e.g., a speaker), a network interface device 1688, and one or more sensors 1687, such as a global positioning system ("GPS") sensor, an accelerometer, a compass, or other sensor. The computing device 1600 may include an output controller 1684, such as a serial (e.g., universal serial bus ("USB")), parallel, or other wired or wireless (e.g., infrared ("IR"), near field communication ("NFC"), radio, etc.) connection, for communicating with or controlling one or more auxiliary devices.

[0078] Storage 1692 may include a non-transitory, machine-readable medium 1683 on which is stored one or more sets of data structures or instructions 1682 (e.g., software) embodied or utilized by any one or more of the functions or methods described herein. The instructions 1682 may reside, completely or at least partially, within main memory 1696, within static memory 1695, or within hardware processor 1697 during their execution by computer 1600. By way of example, one or any combination of hardware processor 1697, main memory 1696, static memory 1695, or storage 1692 may constitute a machine-readable medium.

[0079] Although the machine-readable medium 1683 is shown as a single medium, the term “machine-readable medium” may include a single medium or multiple media (e.g., a distributed or centralized database, or associated caches and servers) configured to store one or more instructions 1682.

[0080] The term "machine-readable medium" may include any medium capable of storing, encoding, or transmitting instructions for execution by the computer 1600 and causing the computer 1600 to perform any one or more of the methods of this disclosure, or capable of storing, encoding, or transmitting data structures used by or associated with such instructions. A non-limiting, exemplary list of machine-readable media may include magnetic media, optical media, solid-state memory, non-volatile memory, such as semiconductor memory devices (e.g., electronically erasable programmable read-only memory ("EEPROM"), electronically programmable read-only memory ("EPROM"), and magnetic disks, such as internal hard disks and removable disks, flash storage devices, magneto-optical disks, CD-ROM disks, and DVD-ROM disks.

[0081] The instructions 1682 may further be transmitted or received over a communications network 1681 using a transmission medium via a network interface device 1688 utilizing one of a number of transport protocols (e.g., internet protocol ("IP"), user datagram protocol ("UDP"), frame relay, transmission control protocol ("TCP"), hypertext transfer protocol ("HTTP"), etc.). Exemplary communications networks may include a wide area network ("WAN"), a plain old telephone ("POTS") network, a local area network ("LAN"), a packet data network, a cellular phone network, a wireless data network, and a homogeneous (peer-to-peer, "P2P") network. By way of example, the network interface device 1688 may include one or more physical jacks (e.g., Ethernet, coaxial, or phone jacks) or one or more antennas for connecting to the communications network 1681.

[0082] By way of example, network interface device 1688 may include multiple antennas for communicating wirelessly using at least one of a single-input multiple-output ("SIMO") or multiple-input single output ("MISO") methodology. The phrase "transmission medium" includes any intangible medium capable of storing, encoding, or conveying instructions for execution by computer 1600, including analog or digital communications signals or other intangible media to facilitate communication of such software.

[0083] Exemplary methods according to the present disclosure may be at least partially machine or computer-implemented. Some examples may include computer-readable or machine-readable media encoded with instructions operable to configure an electronic device to perform the exemplary methods described herein. Exemplary implementations of such exemplary methods may include code, such as assembly language code, microcode, high-level language code, or other code. Such code may include computer-readable instructions for performing various methods. The code may form part of a computer program product. Further, in one example, the code may be tangibly stored on or in a volatile, non-transitory, or non-volatile tangible computer-readable medium, such as during execution or other times. Examples of these tangible computer-readable media may include, but are not limited to, removable optical disks (e.g., compact disks and digital video disks), hard drives, removable magnetic disks, memory cards or sticks, removable flash storage drives, magnetic cassettes, random access memories (RAMs), read-only memories (ROMS), and other media.

[0084] There are various methods for generating a 3D model from 2D pre- or intraoperative images. As an example, one such method may include receiving a set of 2D radiographic images of the patient's surgical field 170 using a radiographic imaging system, and calculating a first 3D model using epipolar geometry principles with the coordinate system of the radiographic imaging system and projective geometry data from each 2D image (see FIG. 4 and FIG. 5A and FIG. 5B). Such an exemplary method may further include adjusting the initial 3D model by projecting the first 3D model on the 2D radiographic images, and then registering the first and second radiographic images 30, 50 on the first 3D model with an inter-image registration technique. Once the inter-image registration technique is applied, a modified 3D model may be generated. This process may be repeated until a desired clarity is achieved.

[0085] As another example, deep learning networks such as convolutional neural networks ("CNN"), recurrent neural networks ("RNN"), modular neural networks, or sequence to sequence models ("Sequence to Sequence"). A deep learning network (also known as a “deep learning network, DNN”) may be used to generate a 3D model 1100 of a target orthopedic element and / or a 3D model of a patient-specific surgical guide 500 from a set of at least two 2D images of a patient's surgical field 170. The 2D images 30, 50 are preferably tissue-penetrating images such as radiological images (e.g., x-ray or fluoroscopic images). In such a method, the deep learning network may generate the model from projective geometry data (i.e., spatial data 43 or volumetric data 75) from each 2D image. The deep learning network may have the advantage of being able to generate masks of different target orthopedic elements 100 (e.g., bones, soft tissues, etc.) in the surgical field 170, as well as to calculate the volume (see 61 in FIG. 6 ) of one or more imaged orthopedic elements 100.

[0086] FIG. 8 is a schematic diagram of a CNN illustrating how it may be used to identify the surface topography of a target orthopedic element 100. Without being bound by theory, it is contemplated that a CNN may be desirable to reduce the size of the volumetric data 75 without losing features necessary to identify the desired orthopedic element 100 or the desired surface topography. The volumetric data 75 of the multiple backprojected input images 30, 50 is a multi-dimensional array that may be known as an "input tensor." This input tensor contains the input data for the first convolution, which in this example is the volumetric data 75. A filter (also known as a kernel 69) is shown placed on the volumetric data 75. The kernel 69 is a tensor (i.e., a multi-dimensional array) that defines a filter or function (which may be known as a "weight" that is applied to the kernel). In the illustrated embodiment, the kernel tensor 69 is three-dimensional. The filter or function, including the kernel 69, may be manually programmed or learned through a CNN, RNN, or other deep learning network. In the illustrated embodiment, kernel 69 is a 3x3x3 tensor, however, all tensor sizes and dimensions are considered within the scope of this disclosure as long as the kernel tensor size is smaller than the size of the input tensor.

[0087] Each cell or voxel of the kernel 69 has a numerical value. These values ​​define the filter or function of the kernel 69. A convolution or cross-correlation operation is performed between two tensors. In FIG. 8, the convolution is represented by a path 76. The path 76 that the kernel 69 follows is a visualization of the mathematical operation. By following this path 76, the kernel 69 eventually traverses the entire volume 61 of the input tensor (e.g., volumetric data 75) sequentially. The goal of this operation is to extract features from the input tensor.

[0088] The convolutional layers 72 typically include one or more of a convolution stage 67, a detector stage 68, and a pooling stage 58. Although each of these operations is visually represented in the first convolutional layer 72a in FIG. 8, it will be understood that subsequent convolutional layers 72b, 72c, etc. may also include one or more, or all, of the convolutional stage 67, detector stage 68, and pooling layer 58 operations, or combinations or permutations thereof. Additionally, while FIG. 8 shows five convolutional layers 72a, 72b, 72c, 72d, 72e of various resolutions, it will be understood that in other exemplary embodiments, more or fewer convolutional layers may be used.

[0089] In the convolution stage 67, the kernel 69 is multiplied sequentially by multiple patches of pixels in the input data (i.e., in the illustrated example, volumetric data 75). The patches of pixels extracted from the data are known as receptive fields. The multiplication of the kernel 69 with the receptive field involves an element-wise multiplication between each pixel of the receptive field and the kernel 69. After multiplication, the results are summed to form one element of the convolution output. This kernel 69 then shifts to an adjacent receptive field and the element-wise multiplication operation and summation continues until all pixels of the input tensor have been subjected to this operation.

[0090] Up to this stage, the input data in the input tensor (e.g., volumetric data 75) is linear. A nonlinear activation function is then used to introduce nonlinearity into this data. The use of such a nonlinear function marks the beginning of the detector stage 68. A common nonlinear activation function is the Rectified Linear Unit function ("ReLU") given by the function:

[0091]

number

[0092] When used with a bias, the nonlinear activation function serves as a threshold to detect the presence of features extracted by the kernel 69. For example, applying a convolution or cross-correlation operation between the input tensor and the kernel 69 produces a convolution output tensor, where the kernel 69 includes a low-level edge filter in the convolution stage 67. The feature map output tensor is then returned by applying the nonlinear activation function with the bias to the convolution output tensor. The bias is added sequentially to each cell of the convolution output tensor. If for a given cell, the sum is greater than or equal to 0 (assuming that ReLU is used in this example), the sum is returned to the corresponding cell of the feature map output tensor. Similarly, if the sum is less than 0 for a given cell, the corresponding cell of the feature map output tensor is set to 0. Thus, applying the nonlinear activation function to the convolution output behaves like a threshold to determine whether and how closely the convolution output matches a given filter of the kernel 69. In this manner, the non-linear activation function detects the presence of desired features from the input data (eg, volumetric data 75 in this example).

[0093] All non-linear activation functions are considered to be within the scope of this disclosure. Other examples include Sigmoid, TanH, Leaky ReLU, parametric ReLU, Softmax, and Switch activation functions.

[0094] However, a drawback of this approach is that the feature map output of this first convolutional layer 72a records the exact location of the desired feature (edge, in the above example). Thus, small movements of the feature in the input data will generate different feature maps. To address this issue and reduce computational power, downsampling is used to reduce the resolution of the input data while still preserving important structural elements. Downsampling can be achieved by changing the stride of the convolutions along the input tensor. Downsampling is also achieved by using a pooling layer 58.

[0095] Effective padding may be applied to reduce the dimensions of the convolved tensor (see 72b) compared to the input tensor (see 72a). A pooling layer 58 is desirably applied to reduce the spatial size of the convolved data, which reduces the computational power required to process the data. Common pooling techniques may be used, including max pooling and average pooling. Max pooling returns the maximum value of the portion of the input tensor covered by the kernel 69, while average pooling returns the average of all values ​​of the portion of the input tensor covered by the kernel 69. Using max pooling, image noise may be reduced.

[0096] In certain exemplary embodiments, a fully connected layer may be added after the final convolutional layer 72e to learn a nonlinear combination of high-level features (such as the profile of the imaged proximal tibia 110 or the surface topography of the orthopedic element) represented by the output of the convolutional layers.

[0097] The top half of FIG. 8 represents the compression of the input volume data 75, and the bottom half represents the decompression until the original size of the input volume data 75 is reached. The output feature map of each convolutional layer 72a, 72b, 72c, etc. is used as the input of the subsequent convolutional layer 72b, 72c, etc. to enable increasingly complex feature extraction. For example, the first kernel 69 may detect edges, a kernel in the first convolutional layer 72b may detect a set of edges at a desired orientation, a kernel in the third convolutional layer 72c may detect a longer set of edges at a desired orientation, etc. This process may continue until the entire profile of the medial distal femoral condyle is detected by downstream convolutional layers 72.

[0098] The bottom half of FIG. 8 is an upsample (i.e., expanding the spatial support of the lower resolution feature map). A deconvolution operation is performed to increase the size of the input for the next downstream convolutional layer (see 72c, 72d, 72e). For the final convolutional layer 72e, convolution may be employed with a 1×1×1 kernel 69 to generate a multi-channel output volume 59 of the same size as the input volume 61. Each channel of the multi-channel output volume 59 may represent a desired extracted high-level feature. This may be followed by a Softmax activation function to detect the desired orthopedic element 100. For example, the illustrated embodiment may include six output channels numbered 0, 1, 2, 3, 4, and 5, where channel 0 represents the identified background volume, channel 1 represents the identified distal femur 105, channel 2 represents the identified proximal tibia 110, channel 3 represents the identified proximal fibula 111, channel 4 represents the identified patella 901, and channel 5 represents the identified surface topography of the target orthopedic element 100.

[0099] In an exemplary embodiment, a 3D model 1100 of a target orthopedic element may be created using a selected output channel containing output volume data 59 of the desired orthopedic element 100. For example, data from a channel representing the identification surface topography of the target orthopedic element 100 may be mapped and reproduced as one or more mating surfaces (see 40 and 36 in FIG. 11 and 53 and 54 in FIG. 12) on the patient-specific surgical guide 500 to create a patient-specific surgical guide 500 configured to be firmly engaged to the target orthopedic element 100. By generating a physical patient-specific surgical guide 500 via manufacturing techniques and sterilizing the patient-specific surgical guide 500, a surgeon may place and use the patient-specific surgical guide 500 directly in the surgical field 170. In this manner, the patient-specific surgical guide 500 may be said to be "configured to abut" the orthopedic element 100 at the identification surface. Similarly, in this manner, the computer 1600 that uses a deep learning network in this or related methods to isolate an individual orthopedic element 100 or a portion of an orthopedic element (e.g., the surface topography of a target orthopedic element 100) can be said to be configured to "identify" the surface topography on the actual target orthopedic element 100 or on the 3D model 1100 of the target orthopedic element and define an identified surface.

[0100] While the above example describes the use of a three-dimensional tensor kernel 69 to convolve the input volume data 75, it will be appreciated that the general model described above may be used with 2D spatial data 43 from the first calibrated input image 30 and the second calibrated input image 50, respectively. In other exemplary embodiments, a machine learning algorithm (i.e., a deep learning network (e.g., CNN, etc.)) may be used after imager calibration but before 2D to 3D reconstruction. That is, a CNN may be used to detect features (e.g., anatomical landmarks) of the target orthopedic element 100 from the first and second reference frames 30a, 50a of the respective 2D input images 30, 50. In an exemplary embodiment, a CNN may be used to identify high-level orthopedic elements (e.g., the distal femur 105 and a portion of the surface topography of the target orthopedic element 100) from the 2D input images 30, 50. The CNN may then optionally apply a mask or contour to the detected orthopedic element 100 or the surface topography of the target orthopedic element 100. The imager 1800 is calibrated and the CNN finds multiple corresponding image points (e.g., X L , X R ), it is contemplated that a transformation matrix between the reference frames 30a, 50a of the target orthopedic element 100 may be used to align multiple corresponding image points in 3D space.

[0101] In certain exemplary embodiments involving using a deep learning network to add a mask or contour to the 2D orthopedic elements 100 detected from the respective input images 30, 50, the identified orthopedic elements 100 or only the 2D mask or contour of the surface topography of the identified orthopedic elements 100 may be sequentially backprojected in the manner described with reference to Figures 4 and 6 above to define the volume 61 of the identified orthopedic elements 100. In this exemplary method, a 3D model 1100 of the target orthopedic elements may be created.

[0102] In embodiments where the first image 30 and the second image 50 are x-ray images, training a CNN may present some challenges. In comparison, a CT scan typically produces a series of images of a desired volume. Each CT image that comprises a typical CT scan may be thought of as a segment of the imaged volume. From these segments, a 3D model may be relatively easily created by adding the area of ​​the desired element as the element is shown in each successive CT image. The modeled element may then be compared to the data from the CT scan to ensure accuracy.

[0103] In contrast, radiographic imaging systems typically do not generate sequential images that capture different segments of the imaged volume. Rather, all of the information of the image is flattened in a 2D plane. In addition, because a single radiographic image 30 inherently lacks 3D data, it is difficult to check the model generated by the epipolar geometric reconstruction technique described above with the actual geometry of the target orthopedic element 100. To address this issue, the CNN may be trained with CT images, such as digitally reconstructed radiograph ("DRR") images. By training the deep learning network in this manner, the deep learning network may develop unique weights (e.g., filters) for the kernel 69 to identify the surface topography of the desired orthopedic element 100 or the target orthopedic element 100. Because radiographs have a different appearance than DRRs, an image-to-image transformation may be performed to render the input x-ray image to have a DRR-style appearance. An exemplary image-to-image transformation method is the CycleGAN image transformation technique. In embodiments where an image-to-image style transfer method is used, the style transfer method is desirably used before inputting the data into a deep learning network for feature detection.

[0104] The above examples are provided for purposes of aiding understanding and are not intended to limit the scope of the present disclosure. All methods for generating a 3D model 1100 of a target orthopedic element 100 from 2D radiographic images (e.g., 30a, 50a) of the same target orthopedic element 100 taken from at least two lateral positions are considered to be within the scope of the present disclosure.

[0105] 10 is a flow diagram outlining the steps of an exemplary method for using a deep learning network to calculate the dimensions of a patient-specific surgical guide 500 for abutment against an orthopedic element 100 using two flattened input images (30, 50 in FIG. 4 and FIGS. 5A-5B) taken at an offset angle θ. The exemplary method involves calibrating an imager 1800 (FIG. 9) to compute the image points (X L , e L , X R , e R 4) and corresponding spatial coordinates (e.g., Cartesian coordinates on the x,y plane) to define spatial data 43. Imager 1800 is preferably a radiological imager capable of producing X-ray images ("X-ray images" may be understood to include fluoroscopic images), although all medical imaging machines are considered to be within the scope of this disclosure.

[0106] Step 2c includes capturing a first image 30 (FIG. 5A) of the target orthopedic element 100 using an imaging technique (e.g., X-ray, CT, MRI, or ultrasound imaging technique), the first image 30 defining a first reference frame 30a (e.g., a first lateral position). In step 3c, a second image 50 (FIG. 5B) of the target orthopedic element 100 is captured using an imaging technique, the second image 50 defining a second reference frame 50a (e.g., a second lateral position), the first reference frame 30a being offset from the second reference frame 50a by an offset angle θ. The first image 30 and the second image 50 are input images from which data (including spatial data 43) may be extracted. It will be understood that in other exemplary embodiments, more than one image may be used. In such embodiments, each input image is desirably separated from the other input images by an offset angle θ. Step 4c includes projecting spatial data 43 from the first image 30 of the target orthopedic element 100 and spatial data 43 from the second image 50 of the target orthopedic element 100 to define volumetric data 75 (Figure 6) using epipolar geometry.

[0107] Step 5c includes detecting the orthopedic element 100 from the volume data 75 using a deep learning network. Step 6c includes detecting other features (e.g., anatomical landmarks) from the volume data 75 of the target orthopedic element 100 using a deep learning network to define a 3D model 1100 of the target orthopedic element, including the surface topography of the target orthopedic element 100. Step 7c includes calculating dimensions of the patient-specific surgical guide 500. In such an embodiment, the dimensions of the mating surface of the patient-specific surgical guide 500 can be complementary to the surface topography of a portion of the target orthopedic element 100. In this manner, the patient-specific surgical guide 500 can be configured to abut and be firmly engaged with the orthopedic element 100.

[0108] In certain exemplary embodiments, the deep learning network that detects anatomical landmarks of the target orthopedic element 100 from the volumetric data 75 may be the same deep learning network that detects other features from the volumetric data 75 of the target orthopedic element 100, such as the surface topography of the target orthopedic element. In other exemplary embodiments, the deep learning network that detects anatomical landmarks of the target orthopedic element 100 from the volumetric data 75 may be different from the deep learning network that detects other features from the volumetric data 75 of the target orthopedic element 100, such as the surface topography of the target orthopedic element.

[0109] In certain exemplary embodiments, the first image 30 may show the target orthopedic element 100 in a lateral position (i.e., the first image 30 is a side view of the orthopedic element 100). In other exemplary embodiments, the second image 50 may show the orthopedic element 100 in an anterior-posterior ("AP") lateral position (i.e., the second image 50 is an AP view of the orthopedic element 100). In yet other exemplary embodiments, the first image 30 may show the orthopedic element 100 in an AP lateral position. In still other exemplary embodiments, the second image 50 may show the orthopedic element 100 in a lateral position. In still still other exemplary embodiments, neither the first image 30 nor the second image 50 may show the orthopedic element 100 in an AP lateral position or a lateral lateral position, so long as the first image 30 is offset from the second image 50 by an offset angle θ. The calculator 1600 may calculate an offset angle θ from the input images 30, 50 including the calibration fixture (see 973 in FIGS. 5A and 5B). The first image 30 and the second image 50 may be referred to collectively as the “input images” or individually as the “input image.” The input images 30, 50 desirably show the same target orthopedic element 100 from different angles. The input images 30, 50 may be taken along a lateral plane of the target orthopedic element 100.

[0110] Certain exemplary systems or methods may further include using a style transfer deep learning network, such as CycleGAN. A system or method using a style transfer deep learning network may start with a radiology input image (e.g., 30) and use a style transfer deep learning network to convert the style of the input image into a DRR type image. Still further exemplary methods may include using a deep learning network to identify features (e.g., anatomical landmarks) of the target orthopedic elements 100 (which may include a portion of the surface topography of the target orthopedic elements 100) to provide a segmentation mask for each target orthopedic element 100.

[0111] Without being bound by theory, it is contemplated that embodiments utilizing radiological input images may be able to provide smoother surfaces on the 3D model of the orthopedic element compared to 3D models generated from CT or MRI input images. CT scans typically scan the target orthopedic element in 1 mm increments. Surface topography changes between a first CT segment scan and an adjacent CT segment scan may result in loss of information in the output of a conventional CT system because surface topography details spaced less than 1 mm apart are not captured in a CT system that incrementally scans the target orthopedic element in 1 mm increments. As a result, technicians have typically had to manually smooth the surface topography of the CT 3D model to create a surgical guide that can be mated with the actual target orthopedic element during surgery. Because less than 1 mm topography data of the actual target orthopedic element has never been captured, this manual smoothing process tends to be inaccurate and may result in a less than perfect fit. Certain embodiments according to the present disclosure can obviate this problem because radiological x-ray images can be represented as an array of pixel values. Pixel densities vary, but as an example, if the first and second input images have a resolution of 96 dots per inch ("dpi") (a unit of pixel density), then there are 25.4 mm within that inch, or 3.78 pixels per millimeter. In other words, in this example, there are an extra 3.78 pixels of information per millimeter compared to a conventional CT scan. Higher pixel density similarly results in higher resolution of the surface topography, while the use of deep learning network(s) as described herein can reduce the computational burden on the computer compared to systems and methods that do not use deep learning networks.

[0112] It is further contemplated that in certain exemplary embodiments, the exemplary system and / or method may take into account surgeon inputs and settings. For example, if a surgeon desires to orient the distal resection plane of the distal femur in 3 degrees varus, an exemplary patient-specific femoral resection guide mount 500a may be produced in accordance with the present disclosure, and the resection slot 52 may be manufactured relative to the body 42 such that the resection slot 52 is oriented in 3 degrees varus when the patient-specific surgical guide 500 is placed on the distal femur 105. In exemplary embodiments, the orientation of the resection slot 52 may be further altered to accommodate limited access or obstructions to the surgical field 170, which may be common in minimally invasive procedures.

[0113] An exemplary method for generating a patient-specific surgical guide includes calibrating a radiographic machine to determine a mapping relationship between image points and corresponding spatial coordinates to define spatial data; capturing a first image of an orthopedic element using a radiographic technique, the first image defining a first frame of reference; capturing a second image of the orthopedic element using a radiographic technique, the second image defining a second frame of reference, the first frame of reference being offset from the second frame of reference at an offset angle; detecting the orthopedic element using the spatial data using a deep learning network, the spatial data defining anatomical landmarks on or within the orthopedic element; The method includes applying a mask to an orthopedic element defined by anatomical landmarks using a deep learning network; projecting spatial data from a first image of the desired orthopedic element and spatial data from a second image of the desired orthopedic element to define volumetric data, where spatial data including image points located within the mask region of either the first image or the second image has a first value and spatial data including image points located outside the mask region of either the first image or the second image has a second value, the first value being different from the second value; applying the deep learning network to the volumetric data to generate a reconstructed 3D model of the orthopedic element; and calculating dimensions of a patient-specific surgical guide configured to abut the orthopedic element.

[0114] An exemplary method for generating a patient-specific surgical guide includes calibrating a radiographic machine to determine a mapping relationship between image points and corresponding spatial coordinates to define spatial data; capturing a first image of an orthopedic element using a radiographic technique, the first image defining a first frame of reference; capturing a second image of the orthopedic element using a radiographic technique, the second image defining a second frame of reference, the first frame of reference being offset from the second frame of reference at an offset angle; and detecting the orthopedic element using the spatial data using a deep learning network, the spatial data defining anatomical landmarks on or within the orthopedic element. applying a mask to the orthopedic feature defined by the anatomical landmarks using a deep learning network; projecting spatial data from a first image of the desired orthopedic feature and spatial data from a second image of the desired orthopedic feature to define volumetric data, wherein spatial data including image points located within the mask region of either the first image or the second image have positive values ​​and spatial data including image points located outside the mask region of either the first image or the second image have negative values; applying the deep learning network to the volumetric data to generate a 3D model of the orthopedic feature; and calculating dimensions of a patient-specific surgical guide configured to tightly engage the orthopedic feature.

[0115] In an example embodiment, the method further includes performing style transfer on the first image and the second image using a deep learning network.

[0116] In an exemplary embodiment, the style transformation converts spatial data from a radiographic imaging technique into dynamic digital radiography data.

[0117] In an exemplary embodiment, the first value is a positive value.

[0118] In an exemplary embodiment, the second value is a negative value.

[0119] In an exemplary embodiment, the method further includes projecting the reconstructed 3D model onto a display.

[0120] In an exemplary embodiment, the deep learning network includes a deep learning algorithm.

[0121] An exemplary system includes a 3D model of an orthopedic element including a surgical field generated from at least two 2D radiological images, where at least a first radiological image is captured at a first location and at least a second radiological image is captured at a second location, where the first location is different from the second location, and a calculator configured to identify a surface topography on the 3D model of the orthopedic element to define an identification surface, and further configured to calculate dimensions of a patient-specific surgical guide, where the patient-specific surgical guide is configured to abut the orthopedic element at the identification surface.

[0122] The exemplary system may further include a display, and the 3D model of the orthopedic component is displayed on the display. In the exemplary system, the display may be an augmented reality device or a virtual reality device. The exemplary system may further include an X-ray imager.

[0123] The exemplary system may further include a manufacturing device configured to generate a physical model of the patient-specific surgical guide.

[0124] In an exemplary system including a manufacturing device, the manufacturing device can be configured to generate at least a partial physical model of the identification surface of the orthopedic element. The manufacturing device can be an additive manufacturing device.

[0125] In an exemplary system, the physical model of the patient-specific surgical guide can include medical grade polyamide.

[0126] A patient-specific surgical guide generated by an exemplary process includes calibrating a radiographic machine to determine a mapping relationship between radiographic image points and corresponding spatial coordinates to define spatial data; capturing a first radiographic image of a target orthopedic component using a radiographic imaging technique, the first radiographic image defining a first frame of reference; capturing a second radiographic image of the target orthopedic component using a radiographic imaging technique, the second radiographic image defining a second frame of reference, the first frame of reference being offset from the second frame of reference by an offset angle; The method may include projecting spatial data from the image and spatial data from a second radiological image of the target orthopedic element to define volumetric data; detecting the target orthopedic element using the volumetric data using a deep learning network, where the volumetric data defines anatomical landmarks on or within the target orthopedic element; identifying a surface on the orthopedic element using the volumetric data using the deep learning network to define an identified surface; and applying the deep learning network to the volumetric data to calculate dimensions of a patient-specific surgical guide configured to abut the orthopedic element at the identified surface.

[0127] An exemplary product-by-process can further include generating a physical 3D model of the patient-specific surgical guide using a manufacturing technique. In such an embodiment, the physical 3D model of the patient-specific surgical guide can include a mating surface that mates with the identification surface of the orthopedic element.

[0128] In an exemplary product-by-process, the physical 3D model of the patient-specific surgical guide can include a mating surface, which can further include protrusions.

[0129] An exemplary patient-specific surgical guide includes calibrating a radiographic machine to determine a mapping relationship between radiographic image points and corresponding spatial coordinates to define spatial data; capturing a first radiographic image of a target orthopedic component using a radiographic imaging technique, the first radiographic image defining a first frame of reference; capturing a second radiographic image of the target orthopedic component using a radiographic imaging technique, the second radiographic image defining a second frame of reference, the first frame of reference being offset from the second frame of reference at an offset angle; and capturing the spatial data from the first radiographic image of the target orthopedic component using a radiographic imaging technique, the second radiographic image defining a second frame of reference. The orthopedic element may be generated by an exemplary process including projecting the first radiological image onto the first radiological image and spatial data from the second radiological image of the target orthopedic element; detecting the target orthopedic element using the spatial data using a deep learning network, where the spatial data defines anatomical landmarks on or within the target orthopedic element; detecting and identifying a surface on the orthopedic element using the spatial data using a deep learning network to define an identification surface; and applying the deep learning network to the spatial data to calculate dimensions of a patient-specific surgical guide configured to abut the orthopedic element at the identification surface.

[0130] It is to be understood that the invention is not limited to the particular configurations and method steps disclosed herein or illustrated in the drawings, but includes any modifications or equivalents known in the art that are within the scope of the claims. Those skilled in the art will appreciate that the apparatus and methods disclosed herein will find utility.

Claims

A radiation imaging device that is calibrated by determining a mapping relationship between radiation image points and corresponding spatial coordinates to define spatial data, A first radiation image of a target orthopedic element, which is acquired by the radiation imaging device and defines a first reference frame, A second radiation image of the target orthopedic element, which is acquired by the radiation imaging device and defines a second reference frame, and the first reference frame is offset from the second reference frame by a predetermined offset angle, Receiving the first radiation image and the second radiation image; projecting spatial data for forming the first radiation image and the second radiation image along the offset angle to define volume data; using a deep learning network to define a surface of the target orthopedic element using the volume data to define a specific surface; and further outputting a fitting surface of a patient-specific surgical guide using the specific surface, A system comprising the above. The system according to claim 1, further comprising a display, wherein the display displays an image selected from the group consisting of a 3D model of the target orthopedic element, a 3D model of a patient-specific surgical guide, the specific surface, and the fitting surface. The system according to claim 2, wherein the display is an augmented reality device or a virtual reality device. The system according to claim 1, further comprising a manufacturing device, wherein the manufacturing device is configured to generate the patient-specific surgical guide. The system according to claim 4, wherein the manufacturing device is configured to generate at least a partial physical model of the specific surface of the orthopedic element. The system according to claim 4, wherein the manufacturing device is a stereolithography apparatus. The system according to claim 4, wherein the patient-specific surgical guide comprises medical-grade polyamide. The system according to claim 4, wherein the fitting surface of the patient-specific surgical guide further comprises protrusions.

Images