Method for registering monoscopic radiation image and 3D model and system

Abstract

To provide a system and a method including a method for determining an actual position and pose of a known three-dimensional (3D) structure by utilizing at least four individual shapes formed by a reference of a structure shown in a two-dimensional (2D) image of the structure.SOLUTION: A method for determining an actual position and pose of a known 3D construct includes the steps of: identifying at least four reference shadows in a 2D image corresponding to a reference of the 3D construct; correlating the discovered at least four reference shadows with their respective positions on the 3D construct; determining a spatial relation between the 2D image and the 3D construct by determining a focal point of a source of an image relative to the 2D image via predetermined mutual separation distance between the discovered at least four reference shadows and corresponding reference of the 3D construct; and determining the spatial relation between the 2D image and the 3D construct.SELECTED DRAWING: Figure 14

Description

Cross - Reference to Related Applications

[0001] This application claims the benefit of, and incorporates by reference in its entirety, U.S. Provisional Application No. 62 / 817,185, filed on March 12, 2019, entitled "Monoscopic Image Registration Methods and Systems". This application is also related to International PCT Patent Application No. PCT / US2019 / 043326, filed on July 24, 2019, entitled "Methods and Systems of Registering a Radiographic Image and a Three - Dimensional Model of an External Fixation Device", the entire disclosure of which is incorporated by reference herein.

Technical Field

[0002] The present disclosure generally relates to image registration using known three - dimensional (3D) structures depicted in images. More specifically, the present disclosure relates to monoscopic image (e.g., radiographic image) and three - dimensional model registration methods and systems that utilize known physical parameters (e.g., separation distances between at least four shapes or points) of a particular 3D structure depicted in an image.

[0003] The present disclosure also generally relates to systems and methods for deformation analysis using multiple non-orthogonal radiographs. Embodiments of the present disclosure relate to treating musculoskeletal conditions including skeletal fractures. More specifically, methods and systems for fixing and positioning one or more segments of bone in a desired position are disclosed. In some embodiments of the present disclosure, methods and systems are used to generate a three-dimensional computer model of potentially at least one (e.g., at least two) radiographic image representations corresponding to the radiographic images utilized in the creation of a fixation device, bone segments, and a model. Through manipulation on the model, in one embodiment, the desired placement of the bone segments and the operation of an external fixation device are quickly and accurately determined to achieve such a desired placement, regardless of the initial configuration of the fixation device or the orientation of the radiographic images, in relation to the device and / or the bone. Next, the operations necessary to create the desired placement of the bone segments are performed on the corresponding fixation device and bone segments to treat the musculoskeletal condition. However, other devices other than external fixation devices can be utilized with the systems and methods.

Background Art

[0004] In medicine, the correction of orthopedic deformities typically involves at least a pair of X-ray photographs. Usually, these X-ray photographs are taken of the patient along conventional lines in the anterior-to-posterior (AP) direction and the medial-to-lateral (ML) direction, or along other orthogonal or known vantage points (or known differences between vantage points). Conventionally, the AP and ML radiographs are taken or assumed to be orthogonal to each other in the patient space (the patient space is defined such that the X-axis is aligned from right to left, the Y-axis is aligned from front to back, and the Z-axis is aligned from bottom to top). Measurements are made within each pair of radiographs, and the deformation axis and points are annotated. These measurements and annotations are then used to reconstruct a true three-dimensional representation of the deformation, allowing the deformation to be manipulated by some means to correct the condition.

[0005] However, problems often arise due to the uncertainty of radiographs and their mutual spatial relationships. Radiographs are not complete images of the artifacts contained in those images. The relationship between the artifacts shown in the image and the objects actually imaged is a form of perspective in which the magnification of objects closer to the image is smaller than that of objects farther away. Furthermore, due to the uncertainty regarding the orthogonality between pairs of images, it becomes difficult to reconstruct a true representation.

[0006] As a result, these uncertainties of such radiographs require means that can be explained due to their actual viewpoints / vantage points.

[0007] Furthermore, in many research fields, it is often desirable to register 2D images to known 3D objects. "Registration" means constructing a coordinate transformation that can determine the position and pose of a 3D object within a coordinate system that matches the 2D image. For example, if any plane of a 3D coordinate system lies on the same plane as a 2D image, the 3D coordinate system can be considered to match the 2D image. Registering a 3D object within this coordinate system enables the creation of one or more virtual environments in which the viewer's viewpoint can be determined, and the image and object can be appropriately placed within that environment.

[0008] In the field of medicine, this is often an important step in the proper placement or manipulation of implants, surgical instruments, or body tissue structures. In contrast to 3D imaging techniques such as CT and MRI, one of the most common methods of imaging is basic radiographs, which have the advantages of low cost and real-time accessibility in an operating environment. It is desirable to be able to register known 3D objects or body structures with respect to real-time images based on individual images.

[0009] Currently, there is a method of registering using a plurality of images of a specific combination of 3D structures. However, in these cases, it is necessary to know the spatial relationship between the plurality of images with a certain degree of certainty. Some current stereoscopic image guidance systems can achieve this externally and usually rely on the known relationship between the pairs of cameras being used. Some other current methods usually require taking a plurality of images such as anterior-posterior (AP) and medial-lateral (ML) radiographs. The relationship between such images is affected by variables inherent in taking such images, which can lead to errors in 3D registration as described above.

[0010] As a specific example, in orthopedics, it is often necessary to correct bone deformities using a device known as an external fixator. Such external fixators are provided in various configurations, for example, from simple unilateral and pin-to-bar systems to more complex circular constructs. In order to accurately correct such bone deformities, after the construct is attached to the patient, the spatial relationship between the anatomical structure of the bone and the fixator construct must be accurately characterized. This characterization process can start with taking a plurality of images such as two or more 2D radiographs or a 3D image scan of the fixator construct attached to the bone. Since 2D radiographs are simple and low-cost, they are the main means for obtaining such characteristics. Therefore, it is desirable to accurately register each of the plurality of two-dimensional images based on an external fixator or other known 3D entity and accurately position other body structures relative to the known 3D entity.

[0011] Certain aspects of the prior art have been discussed to facilitate the disclosure of the applicant's invention, but the applicant does not in any way disclaim these technical aspects, and it is contemplated that the invention may incorporate one or more of these prior art aspects.

[0012] In this specification, where a document, act, or item of knowledge is referenced or discussed, such reference or discussion is not admitted to constitute prior art based on the fact that the document, act, or item of knowledge, or any combination thereof, was generally available as of the priority date, was generally known, was part of common general knowledge, or was otherwise applicable under the laws in force; or that it was known to be relevant to an attempt to solve the problem to which this specification pertains.

SUMMARY OF THE INVENTION

[0013] The present invention is capable of addressing one or more of the problems and deficiencies in the art discussed above. However, it is contemplated that the present invention may prove useful in addressing other problems and deficiencies in many technical fields. Accordingly, the claimed invention should not necessarily be construed as limited to addressing any of the specific problems or deficiencies discussed herein.

[0014] The present disclosure generally relates to an image registration method and system that utilize known three-dimensional (3D) structures depicted in an image. More specifically, the present disclosure relates to a monoscopic image (e.g., a radiographic image) and a 3D model registration method and system that utilize known physical parameters (e.g., separation distances between at least four shapes or points) of a specific 3D structure depicted in an image.

[0015] The present disclosure also generally relates to a system and method for deformation analysis that use a plurality of radiographs (such as non-orthogonal radiographs) taken from an unknown (or inaccurately or incorrectly identified) vantage point. In some embodiments, the system and method individually register each two-dimensional image (having a known 3D structure and / or reference shape (and size)) and use each registered image to construct a 3D model as part of the deformation and / or for correction analysis and prescription of the deformation.

[0016] Some such embodiments of the present disclosure relate to treating musculoskeletal conditions, including skeletal fractures. More specifically, methods and systems are disclosed for fixing and positioning one or more segments of bone in a desired position. In some embodiments of the present disclosure, methods and systems are used to generate a three-dimensional computer model of potentially at least one (e.g., at least two) radiographic image representations corresponding to the radiographic images utilized in the creation of the fixation device, bone segments, and model. Through manipulation on the model, in one embodiment, the desired placement of the bone segments and the manipulation of the external fixation device are determined quickly and accurately to achieve such desired placement, regardless of the initial configuration of the device and / or the orientation / vantage point of the radiographic image, in relation to the device and / or the bone. Next, the operations necessary to create the desired placement of the bone segments are performed on the corresponding fixation device and bone segments to treat the musculoskeletal condition. However, other devices other than the external fixation device can be utilized with the systems and methods.

[0017] In some embodiments, the present disclosure provides methods and related systems that utilize the planar positions and characteristics of four individual shapes or points included within a particular two-dimensional radiograph to correlate to four individual spatial coordinates included within a fixator construct (or a construct of another known object). Using this information, the methods and related systems obtain an accurate spatial relationship between the fixation construct (or a construct of another known object) and each of the individual radiographs.

[0018] In some embodiments, the radiographic image includes the shadow of a three-dimensional object that was posed and placed on the image (e.g., film) when the image was taken. The apparent source position and orientation of the x-ray source with respect to the image casting the shadow are unknown. In an ideal world, the focus would be a point source placed at an infinite distance from the center of the image itself. In an ideal representation, the shadow would be the true two-dimensional projection of the actual three-dimensional object. Given two ideal representations that are known to be orthogonal to a common axis, two sets of two-dimensional data can be used directly to accurately reconstruct the three-dimensional object and its position and pose in space. However, this is generally not possible considering that current state-of-the-art radiographic techniques, including planar film radiographs, introduce projection distortion. Furthermore, the likelihood that the x-ray source orbits are truly orthogonal and are taken orthogonal to each other about a common axis is negligible considering all the variables required to acquire these images with an actual x-ray apparatus where the actual patient is instructed to lie / pose sideways in the prescribed manner.

[0019] The systems and methods of the present disclosure can draw many conclusions to ultimately correct for an unintentional rotation (e.g., rotation from an orthogonal arrangement) between a pair of radiographic images, utilizing two primary sources of error, focus position and pose, as well as patient orientation, and construct a true three-dimensional model of the object within the radiographic image.

[0020] The system and method can account for projection distortion by determining a radiopaque object and the shadow cast on the radiographic image. As is known, the edges of such artifacts are relatively sharp. The system and method can utilize these relatively sharp edges. The system and method utilize the shape edge and can conclude that the source of the radiographic image (i.e., X-ray) casting the shadow is actually located somewhere above the shadow image and is the posed point. The system and method can also conclude, as shown in FIG. 10, that the object is on the vector that describes the line between the center of the shadow or point of a particular object artifact and the focal point of the X-ray source. The system and method can further conclude that when the shape and its actual size of the artifact are known, the relative distance between the shadow image and the actual artifact, and the distance between the shadow image and the X-ray source can be determined. However, this alone is not sufficient to determine the position and pose of the X-ray source. Therefore, the system and method utilize a number of known objects whose shadows exist as artifacts in the radiographic image and whose relative shape, size, and relationship to other objects are known, so that the apparent focal position of the device relative to the image or the position and pose of the X-ray source can be determined.

[0021] As shown in FIG. 10, the system and method can determine the position and pose of a three-dimensional collection of known objects in the radiation image space using a plurality of closed vector loops passing through the center of the shadow, the center of the object, and the position of the focus. Once the plurality of closed vector loops are determined, the system and method can define a coordinate transformation for a collection of known three-dimensional objects within the shadow image space as shown in FIG. 10 (i.e., determine the row dimension, column dimension, and height dimension). Once the coordinate system is determined, the system and method can use a consistent approach for each image to utilize the collection of known three-dimensional objects in each of the plurality of radiation images to determine the coordinate transformation between any pair of images within the plurality of images. The system and method can correct pairs of images that are rotated non-orthogonally or in other ways when constructing the true three-dimensional position and pose of the three-dimensional object, and thus accurately describe other annotations or measurements made within the radiation image.

[0022] In another aspect, the present disclosure provides a method and system that utilize a known three-dimensional collection of objects, the shadow of which is cast into a two-dimensional X-ray radiation space, to determine the actual position and pose of that collection of known objects in the three-dimensional space of a projected computer model lying above the two-dimensional radiation imaging space.

[0023] In some embodiments, the method and system can utilize projection distortion to determine relative magnification and assist in the reconstruction of the three-dimensional projected space. In some embodiments, the method and system can determine the relationship between multiple radiation images through the analysis of known common objects. In some such embodiments, the method and system can reconstruct an actual three-dimensional conditional model in a modified relative spatial arrangement.

[0024] In some embodiments, the method and system may include a method for determining the actual position and pose of a known three-dimensional (3D) structure using at least four individual shapes formed according to the reference of the structure shown in a two-dimensional (2D) image of the structure. This method includes identifying at least four fiducial shadows in the 2D image corresponding to the reference of the structure, correlating the at least four discovered fiducial shadows with their respective positions on the structure, and determining the spatial relationship between the 2D image and the structure by determining the focus of the source of the image relative to the 2D image through a predetermined mutual separation distance between the at least four discovered fiducial shadows and the corresponding reference of the structure. And determining the spatial relationship between the 2D image and the structure.

[0025] In some embodiments, the step of correlating the at least four discovered fiducial shadows with their respective positions on the structure includes identifying the at least four discovered fiducial shadows as upper or lower fiducial shadows, and based on their respective sizes, determining the foreground or background order of the at least four discovered fiducial shadows, determining the left-to-right or right-to-left order of the at least four discovered fiducials, and annotating the at least four discovered fiducials to correlate with their respective annotated fiducial positions on the structure.

[0026] In some embodiments, the step of determining the spatial relationship between the 2D image and the structure includes identifying the actual fiducial positions along the vectors from the focus to the fiducial shadow positions, converting the actual fiducial positions to three-dimensional (3D) image coordinates, defining the actual fiducial position vectors between the fiducial positions through the 3D image coordinates, constructing a first orthogonal coordinate system of a collection of three discrete fiducials with respect to the 2D image by determining the vector cross product between appropriate pairs of position vectors, and inverting the first constructed orthogonal coordinate system or a second constructed orthogonal coordinate system determined through the first constructed orthogonal coordinate system to develop a coordinate transformation of the 2D image with respect to any coordinate system representing the structure.

[0027] In some embodiments, the step of determining the spatial relationship between a 2D image and a structure by determining the focus of the source of the image with respect to the 2D image through a predetermined mutual separation distance between at least four reference shadows found and the corresponding reference of the structure includes establishing a Cartesian coordinate system that uses the 2D image as one of three planes of the coordinate system, determining positions along the focal rays where each of the at least four references must exist, constraining a model of the at least four references based on known characteristics of the structure via a cost function, where the known characteristics do not include the distance between one ray and the four references, and the constraint forms a tripod model that traces a planar curve in a plane perpendicular to the image plane, reconfiguring the tripod model so that a first plane formed by a first group of three of the at least four references is positioned along the image plane, determining a first equation of a first line representing the intersection of the image plane and the first plane, reconfiguring the tripod model so that a second plane formed by a second group of three of the at least four references is positioned along the image plane, determining a second equation of a first line representing the intersection of the image plane and the second plane, determining the x and y coordinates of the focus via at least the first and second lines, and determining the z coordinate of the focus via the x and y coordinates and the cost function.

[0028] In some embodiments, the method and system may include a method for determining the actual position and pose of a collection of known objects in a projected three-dimensional space above a two-dimensional radiation space, the method including obtaining two or more digital radiographic images of the collection of known objects in the projected three-dimensional space above the two-dimensional radiation space, and using the shadows of the collection of known objects in the two-dimensional radiation space of the two or more digital radiographic images to determine the actual position and pose of the collection of known objects in the projected three-dimensional space above the two-dimensional radiation space.

[0029] In some embodiments, the method further includes constructing a three-dimensional model of the actual positions and poses of a collection of known objects in a projected three-dimensional space. In some such embodiments, determining the actual positions and poses of the collection of known objects using the shadows of the collection of known objects in the two-dimensional radiation space of two or more digital radiographic images includes determining the relative magnification of the images using projection distortion to reconstruct the projected three-dimensional space. In some such embodiments, determining the relationship between two or more digital radiographic images includes comparing common objects of the collection of known objects within the images. In some such embodiments, the two or more digital radiographic images further include at least one anatomical structure that requires correction, and further includes constructing a three-dimensional model of the actual positions and poses of the at least one anatomical structure in the projected three-dimensional space.

[0030] In some embodiments, the present disclosure provides a computer program product including a computer-readable storage medium readable by one or more processing circuits and storing instructions for execution by one or more processors for performing the above method.

[0031] The present disclosure provides a system including a memory, at least one processor in communication with the memory, and program instructions executable by the one or more processors via the memory for performing the above method.

Brief Description of the Drawings

[0032] The present disclosure is described in conjunction with the following drawings, which are not necessarily drawn to scale for ease of understanding, and the same reference numerals retain the same or similar element designations and meanings throughout the various drawings.

[0033]

Figure 1

[0034]

Figure 2

[0035]

Figure 3A

[0036]

Figure 3B

[0037]

Figure 4

[0038]

Figure 5A

[0039]

Figure 5B

[0040]

Figure 6

[0041]

Figure 7

[0042]

Figure 8A

[0043]

Figure 8B

[0044]

Figure 9

[0045]

Figure 10

[0046]

Figure 11

[0047]

Figure 12

[0048]

Figure 13

[0049]

Figure 14

[0050]

Figure 15

[0051]

Figure 16

[0052] Aspects and certain features, advantages, and details of the present invention are described more fully below with reference to the non-limiting embodiments shown in the accompanying drawings. Descriptions of well-known materials, manufacturing tools, processing techniques, etc. are omitted in order not to unnecessarily obscure the present invention in excessive detail. However, it should be understood that the detailed description and specific examples, while indicating embodiments of the present invention, are given by way of illustration only and not as limitations. Various substitutions, modifications, additions, and / or arrangements within the spirit and / or scope of the underlying inventive concept will be apparent to those skilled in the art from this disclosure.

[0053] Methods, systems, and related computer program products for correlating the planar positions and characteristics of four individual shapes included within a particular two-dimensional photograph to four individual spatial coordinates included within a known construct such as a fixator construct are not described in relation to FIGS. 1 through 9. The methods, systems, and related computer program products can utilize such information to obtain and display (to the user) the exact spatial relationship between a construct (e.g., a fixator construct) and each of the individual radiographs. The methods, systems, and related computer program products may be described herein with reference to an external fixation construct (such as a hexapod construct), but it should be particularly noted that the methods, systems, and related computer program products can be agnostic with respect to any 3D construct (another orthopedic construct or a non-orthopedic construct) that includes a known relationship of at least four of its spatial coordinates / shapes (e.g., spheres or spheroids or points) (and at least one 2D image (e.g., a radiograph) having one or more anatomical structures). Further, while spheres are utilized in the description herein to describe the methods, systems, and related computer program products as four individual spatial coordinates and / or shapes, other known shapes such as spheroids or points, but not limited thereto, can equally be used as will be understood by those skilled in the art.

[0054] In three-dimensional spatial relationships, six parameters may be required to relate both the position and pose between any two solid objects in space. The position can usually be considered a translational arrangement within an orthogonal (x, y, z) coordinate system. The pose can be considered a series of rotations about the x, y, and / or z axes of the same position coordinate system. All six of these together are called the degrees of freedom (DOF) within a particular three-dimensional space. In common terms, In-Out, Left-Right, Up-Down (x, y, z), Roll, Pitch, and Yaw (r, p, y).

[0055] The simplest three-dimensional object is a sphere having a central position (x, y, z) and a specified radius (r). Using one sphere (or a rotational ellipsoid, etc.), the position (x, y, z) of such an object in three-dimensional space can be determined. Since both the position and pose of a known structure (e.g., an external fixator structure such as a hexapod) are required, the consideration needed is which type of three-dimensional object can uniquely position and pose in three-dimensional space. Such a relatively simple object is known as a tetrahedron, which is a triangular pyramid having four individual vertices and four triangular faces.

[0056] A common type of circular fixator is that known as a hexapod. This is composed of two planar rings connected by six telescoping struts having spherical joints at both ends. Currently, most hexapod structures on the market are configured in a so-called 6X6 configuration, that is, there are six individual spherical mount positions configured in pairs, usually equally spaced but not necessarily so, about the central axis of each ring. A mathematically simple hexapod structure is that known as a 3X3 configuration, where the pairs of spherical joints coincide with each other and there are three coinciding pairs on each ring. Such a 3X3 hexapod can be divided into 15 individual tetrahedrons, and one of these can be used to describe the position and pose of the hexapod fixator structure in three-dimensional space.

[0057] Note that while three points can be used to define a two-dimensional plane, they cannot be used to define a three-dimensional structure. As shown in FIGS. 11 and 12, multiple foci passing through a triangle (three points) of fixed dimensions can generate the same shadow coordinates of two non-connected triangles, each of which can represent three positions on each of two planar fixed platforms (e.g., rings) of a fixed structure (e.g., a hexapod). By using a fourth point, some embodiments of the methods and systems disclosed herein enable the construction of four connected triangles of known dimensions, which, as will be further explained below, enables the determination of a single focus. However, in some embodiments, three points on a three-dimensional structure can be utilized by the methods and systems disclosed herein, along with multiple views of the same structure and points (and known relationships between the views). However, such three-point embodiments are not as efficient as four-point embodiments.

[0058] Such a structure having four known points / shapes is shown in FIG. 1, which depicts one of 15 possible tetrahedrons that can be constructed. The foregoing description has focused on the use of this method in a hexapod structure having six fiducial markers, but note that if the three-dimensional distances between the markers are known, four or more fiducial markers can be used for any type of 3D structure (e.g., a fixture structure). Specifically, a method is shown for accurately determining the spatial relationship between a fixture structure and a two-dimensional radiographic image using any four of the markers (there are a total of six possibilities).

[0059] An exemplary hexapod structure in which both a 6X6 hexapod mechanical structure and a simpler 3X3 hexapod structure are nested is the AMD T Six Fix system. In the Six Fix system, each pair of strut spherical joints on a ring has an additional spherical whose positional relationship with the pair of spherical joints is known. These spheres are radiopaque and their shadows are known as fiducial markers that become artifacts in 2D radiographic images. Other forms are envisioned but not limited to, such as an ellipsoid of revolution type that can determine both the position and to some extent the pose based on a single form whose shadow image properties can be used.

[0060] To construct a 3X3 hexapod nested within a 6X6 mechanical structure, it is necessary to determine the spatial relationship between each ring. The base ring can be regarded as a reference frame and the platform ring can be regarded as a moving reference. If the position of the spherical joint between the strut and the ring and the length of the strut are known, the spatial relationship between the base and the platform ring can be determined. The spatial relationship between the base and the platform ring can be determined via a forward kinematic solution that returns a transformation matrix that is an extended representation of the position and pose of the platform ring relative to the base ring and the reference frame. Once such a transformation is obtained, the position of the fiducial markers with respect to each ring can be obtained therefrom with respect to the base ring and the reference frame. This enables the construction of a 3X3 hexapod composed of the inner and outer surfaces of 20 triangles. Any triangular face of the 3X3 hexapod structure can be treated as a base reference frame, and any of the remaining triangular faces can be treated as a platform having a deformation relative to the base, and it should be noted that this is also an easily determined problem.

[0061] To properly characterize a particular base reference frame, fiducial shadows found in 2D radiographic images need to be correlated with their respective positions on a 3X3 structure. To facilitate this correlation, a method, system, and associated computer-readable product utilize at least four (and in some cases six) radiopaque fiducial markers, at least one of which is of a different form, which typically results in a smaller diameter in the case of spherical markers, for example, but a single larger fiducial can be used equivalently. The "different" fiducial markers are typically oriented towards the most anterior position when attached to the patient's anatomical structure at known clinically relevant positions such as a base ring or a superior ring. Such a potential preferential orientation can facilitate the identification of fiducial markers, for example, when not all fiducial markers in a 2D radiographic image can be identified. For example, one or more fiducial markers may be hidden by other radiopaque elements of the structure. Note that if all or at least four of the fiducial markers can be identified with the presence of "different" markers, a preferential orientation is not required, and in fact, the correct correlation between the fiducial shadows and their respective positions within the fixture structure can be made regardless of the preferential orientation of the "different" fiducial markers.

[0062] In some embodiments, a method, system, and associated computer-readable product can correlate fiducial shadows found in a 2D radiograph with their respective positions on a 3X3 construct by grouping the fiducial shadows into collections of upper and lower (e.g., superior and inferior) collections of one, two, or three distinguishable artifacts. Next, the shadows may be separated, for example, in foreground / background order based on the respective size (magnification) of the identified fiducial shadows. In the case of spherical fiducial markers, an elliptical minor diameter (or average diameter, or area of the identified shadow, etc.) can be utilized to separate the fiducial shadows in foreground / background order. Next, the shadows can be sorted in left-to-right order (e.g., inner-to-outer, or front-to-back). Next, the absolute magnification of the fiducial shadows can be determined, evaluated, and / or compared to determine outliers or "different" fiducial shadows. For example, the absolute size of a "different" fiducial shadow may not match what is due to differences in foreground / background magnification. Next, the recognized position of a "different" or outlier fiducial shadow compared to other identified fiducial shadows can be used to annotate the fiducial shadow and correlate it with an annotated fiducial position on a construct (such as a fixture construct). If no "different" fiducial shadows are identified, the sort may default back to the assumption that the preferred orientation was used, and the list of possible numbering schemes may be ranked according to compliance with the preferred orientation. Next, such a list of possibilities can then be evaluated based on, for example, how well they match what is known about the construct.

[0063] After correlating the reference image with each 3D position on the structure (e.g., from an absolute perspective or from the perspective of the probability ranking of multiple possibilities), a method, a system, and a related computer-readable product can determine the spatial relationship between the 2D radiographic image and the structure. The method of characterizing the spatial relationship can include determining the focus of the X-ray source relative to the 2D radiographic image. A method, a system, and a related computer-readable product can determine the focus of the X-ray source relative to the 2D radiographic image using any four points whose mutual separation distances are known. For example, a 3X3 hexapod structure (or other structure) can be divided into 15 different collections of four vertices where each collection forms a tetrahedron. Any one of these tetrahedrons is sufficient for a method, a system, and a related computer-readable product to determine the focus of the X-ray source relative to the 2D radiographic image. Thus, the structure may only include four references. In some embodiments, a method, a system, and a related computer-readable product can average multiple tetrahedrons to increase the accuracy of focus determination.

[0064] In some embodiments, a method, a system, and a related computer-readable product can characterize the focus position by establishing a Cartesian coordinate system. The Cartesian coordinate system can be established using the 2D radiographic image as one of the three planes of the coordinate system. Since its origin can be arbitrary, here, for the purpose of explanation / disclosure, it is assumed to be at the center of the image. Although the arrangement of the axes is also arbitrary, here too, for the purpose of explanation / disclosure, the x-axis is positioned along the horizontal line of the 2D radiographic image with the positive direction to the right, the y-axis is positioned along the vertical line of the 2D radiographic image with the positive direction upward, and the z-axis is positioned perpendicular to the plane of the 2D radiographic image with the positive direction facing from the image towards the viewer.

[0065] The orthogonal coordinate system can be established by assuming that the focal position of the X-ray source is above the two-dimensional radiation image in the positive z direction and that the entire structure is between the focal position and the two-dimensional radiation image. It should be noted that if these assumptions do not hold, the complete shadow of the reference markers representing the vertices of the tetrahedron in the two-dimensional radiation image may not be displayed / included / unusable.

[0066] In some embodiments, as shown in FIG. 2, a focal model using a single tetrahedron may include using any tetrahedron consisting of vertices a, b, c, d, where the diameters of the reference markers are known for a, b, c, and d and the corresponding distances between a-b, a-c, a-d, b-c, b-d, and c-d. The focal point FP (x,y,z) is shown by four rays (green columns) emitted from the focal point that intersect the reference points a, b, c, and d of the tetrahedron and cast shadows A, B, C, and D on the image plane. Due to the oblique nature of the rays with respect to the image plane, these shadows are usually essentially elliptical. It should be noted that the minor diameter of the elliptical shadow is a function of the magnification applied to the reference casting the shadow. Based on this fact, along the ray FP-A, the location where the reference marker must be present can be determined. This method can also be used for all other shadows B, C, and D and their associated references b, c, and d. Such determination may include determining the (x,y) centers of each elliptical shadow A, B, C, and D with respect to the image coordinate system, as well as their individual minor diameters. The minor diameters of the shadows divided by the known diameters of the associated reference markers can be utilized as magnifications MA, MB, MC, and MD.

[0067] Once the focus model is constructed, the method, system, and associated computer-readable product can constrain (e.g., algebraically constrain) the model based on known characteristics. For example, the method, system, and associated computer-readable product can construct a cost function that can be used in numerical optimization to return the focus in three dimensions. An example of such a cost function is shown in FIG. 2. As shown in FIG. 2, for example, a tetrahedron cost function can be utilized that includes known relationships between reference positions a, b, c, and d and their associated shadows A, B, C, and D as a function of FPxyz and their respective magnifications MA, MB, MC, and MD. The method, system, and associated computer-readable product may include these relationships with the known separation distances between a, b, c, and d, which are included in Dist = [ab ac ad bc bd cd]. The method, system, and associated computer-readable product can solve such a system of equations so as to avoid ending up with multiple solutions (such as mirror equivalents) and / or non-optimal local minima. For example, in some embodiments, the method, system, and associated computer-readable product can avoid such scenarios by simplifying the search topography in a series of steps, thereby restricting or specifying certain unknowns with respect to certain conditions. For example, solving a system with three unknowns is a volume, two unknowns are surfaces, and one unknown is a one-dimensional curve. Thereby, the method, system, and associated computer-readable product may narrow the search field.

[0068] In some such embodiments, the method, system, and associated computer-readable product may recognize that a particular construct may be fully constrained or else may not move, taking into account its relationships. For example, as shown in FIGS. 1 and 3, the method, system, and associated computer-readable product may recognize that such a construct is fully constrained or else may not move, taking into account the relationships described in FIG. 3A. However, it should be noted that such volume optimization may be affected by one or more of the pitfalls already described. Thereby, the method, system, and associated computer-readable product may be able to utilize fewer constraints (e.g., remove certain constraints), such as modeling / observing the behavior of a simplified structure. For example, the method, system, and associated computer-readable product may remove the single ray FPxyz-D and the four reference spacing constraints and leave only [ab ac] to reach a simplified cost function, as shown, for example, in FIG. 3B. As shown in FIG. 4, the method, system, and associated computer-readable product may thereby utilize or form a tripod structure that traces a planar curve. In some embodiments, the method, system, and associated computer-readable product may show the plane in which the curve exists as the ABC plane perpendicular to the image plane.

[0069] In some embodiments, the method, system, and associated computer-readable product can determine a tripod or the plane of its tetrahedron. For example, to determine the ABC plane, the method, system, and associated computer-readable product can place the tripod horizontally on the image plane or transpose it, as shown in phases or steps 1, 2, 3, 4 shown in FIGS. 5A and 5B. In some embodiments, the method, system, and associated computer-readable product can solve for FPxy for such a construct with the z = 0 condition of plane search rather than volume search, as shown in FIG. 15. In some embodiments, the method, system, and associated computer-readable product can determine or identify two solutions where the curve intersects the image plane, which is utilized to formulate the equation of the line representing the intersection of the image plane and the ABC plane. FIG. 15 shows, for a particular dimensional example, the two intersections of the tripod and the focal point FP with the image plane.

[0070] In some embodiments, a method, system, and associated computer-readable product may construct four separate tripods for a particular tetrahedron (e.g., see FIG. 1) using different base positions such as ABC, ABD, ACD, and BCD. Each tetrahedron behaves in the same way as if it traces a planar curve in each plane when the tripod is placed horizontally or transposed in the image plane, and all of these are perpendicular to the image plane. FIG. 7 shows a second case of such a process for the image plane case ABD. As shown in FIG. 7, a planar curve defining the ABD plane is being traced. As shown in FIGS. 8A and 8B, when taking the normals to the image plane views of both the ABC and ABD planes, the intersection of the two planes may coincide with FP(x,y). It should also be noted that all six combinations of intersecting planes may generate the same FP(x,y). For example, since these values may differ slightly due to small errors in the measurements, the method, system, and associated computer-readable product can reduce such errors by taking the average of all six possible intersections. In some embodiments, the method, system, and associated computer-readable product can deselect intersections that are outliers, for example, by using statistical operations / analysis to average the remaining intersections.

[0071] In some embodiments, a method, system, and associated computer-readable product may determine the z coordinate of the focus FP by utilizing known / determined x and y coordinates, such as via the cost function shown in FIG. 9. Such an approach may involve advantageous optimization since there is only one unknown z.

[0072] Once an optimal focus is determined for a particular two-dimensional radiographic image, a method, system, and associated computer-readable product can determine the spatial relationship between the two-dimensional radiograph and the fixture construct. For example, the actual fiducial positions can be placed along the vector from the focus to the fiducial shadow positions using the solution identified above. These positions can then be transformed into three-dimensional image coordinates such that each fiducial position is represented with respect to the two-dimensional radiographic image.

[0073] In some embodiments, a method, system, and associated computer-readable product can define the actual fiducial position vectors between these fiducial positions using the dimensional image coordinates. In some embodiments, a method, system, and associated computer-readable product can determine the vector cross product between appropriate pairs of these vectors to provide the construction of an orthogonal coordinate system for any collection of three discrete fiducials. Since each of these coordinate systems is with respect to the two-dimensional radiographic image, a method, system, and associated computer-readable product can utilize any of them as a basis for other coordinate systems constructed with any group of fiducial positions. Further, a method, system, and associated computer-readable product can invert any of these resulting coordinate systems to develop a coordinate transformation of the two-dimensional radiographic image with respect to any coordinate system representing the fixture construct.

[0074] In some embodiments, a method, system, and associated computer-readable product can utilize multiple images, if available, to determine the relationship between each image and the construct. Since the construct is a static known entity among the multiple images, a method, system, and associated computer-readable product can determine the spatial relationship between the multiple images such that additional characterization of the artifact common to the multiple images can be accurately characterized in three-dimensional space with respect to the construct.

[0075] Next, with respect to FIG. 10, an additional method, system, and related computer program product for determining the actual position and pose of a collection of known objects in a projected three-dimensional space above a two-dimensional radiation space will be described.

[0076] Referring to FIG. 10, an exemplary external deformation correction device in the general form of a hexapod, consisting of a base and a platform arranged in space, to which six spherical radiation-impermeable reference markers are attached, functioning as known shapes A, B, C, D, E, F, where the code distances AB, BC, CA and DE, EF, and FD are all known, is shown. A, B, and C are further connected to D, E, and F, each having a known length, by a set of six dashed lines. FIG. 10 shows what is known as a 3x3 configuration, pointing to the centers of three matching spheres on the base and the platform. In this example, the reference marker shown as A is selected to be smaller than the remaining reference markers of all the same size. This is done to distinguish the base from the platform and the rotation of the base in the image space.

[0077] In some embodiments, the method and system can utilize typical radiographic images, position the shadow of the reference marker, and evaluate the size and position within the radiographic image. The advantage of using spherical reference markers is that they always cast an elliptical shadow. In some embodiments, the method and system can utilize the minor axis dimension related to the actual diameter, which can be related to the relative distance between the image and the X-ray focus, and the height along the vector where the actual reference marker exists. In some embodiments, the method and system can utilize the image resolution to determine the initial image scale and relative size of the shadow artifact with respect to the actual object. As shown in FIG. 10, the focus FP(xyz)O is defined as an arbitrary point floating in the space above the image. Next, in some embodiments, the method and system can define multiple constraints using closed vector loops such as B0A->A->B->B1A->BOA, P1A->FP(xyz)->P2A->P1A, and B2A->C->E->P1A->B2A as shown in FIG. 10. As shown by the green loop, using multiple closed vector loops passing through the origin sufficiently suppresses the problem, but it should be noted that due to resolution and scattering limitations, measurement of the minor diameter of the shadow with respect to the actual diameter of the spherical object where errors may occur is required. Therefore, more loops can be used to statistically improve the results, and many closed vector loops can be used. It has been determined that it is sufficient to use combinations of four vector loops (red) for each of the base and the platform, four vector loops (blue) between the base and the platform, and three vector loops (green) between the image and the focus using the relative size.

[0078] Another approach is shown in FIG. 13, where the image is displayed with rays emanating from an arbitrarily selected focus FP(x,y,z) to a collection of shadow positions shown on the image / image plane (e.g., formed according to the criteria of a known orthopedic construct such as a hexapod). Two triangles of known dimensions (upper and lower triangles) (e.g., those points corresponding to the positions of the criteria of the known construct) are used / positioned along three rays each (i.e., one ray per point / corner of the triangle). Note that the triangles are shown as two separate articles, but it should be noted that the vertices or edges can be shared if only five or four shadow artifacts can be detected in the image. Also note that any combination of triangles that can be constructed using the available shadow artifacts and their associated rays to any arbitrary focus FP(x,y,z) can be similarly utilized. To determine the appropriate orientation of the triangles, the system and method can evaluate the relative magnification of the shadow artifacts to determine whether outliers lead or lag (i.e., are close to, closest to, far from, or farthest from the focus FP(x,y,z)). For any particular arbitrary focus, the system and method can determine a cost function that is the sum of the errors between the known separation between vertices and the calculated distances between vertices (for a particular arbitrary focus FP(x,y,z)). Such a cost function can be used with a numerical solver to determine the most compatible FP(x,y,z) for a particular construct and the shadow it casts in the projected geometric sense.

[0079] After determining at least four of the node positions A, B, C, D, E, F, and O in the image space, in some embodiments, the method and system can then construct a coordinate transformation suitable for a known three-dimensional object represented by a collection of spherical references. In some such embodiments, the method and system can construct a suitable coordinate transformation by determining the cross product of a pair of suitable vectors, such as, for example, ABxAC yields a vector perpendicular to both AB and AC with the origin at A. Next, the vector can be crossed with either of the previous vectors AB or AC to determine an orthogonal coordinate system representing the basis of the image space defined by ABC in this case. The method and system can use the same cross product method for multiple images, resulting in a plurality of coordinate systems that all describe the same known three-dimensional object within a larger patient space.

[0080] Thereby, the method and system can utilize how the known objects are located in two different spaces to determine their relationships and thus determine the coordinate transformation between those different spaces using matrix operations such as inverse matrices and multiplications. This feature of the method and system eliminates the need to provide orthogonal images rotated about a common axis to determine the true three-dimensional conditions to be corrected.

[0081] As shown in FIG. 14, the method and system of the present disclosure, at 102, digitally acquire a 2D X-ray radiographic image (or its digital version) showing a 3D construct of a known configuration (e.g., shape, size, etc.), as a result of which four distinguishable discrete reference / points of the construct, such as the shadow centers of known spherical references / elements of the 3D construct, are identified (digitally and / or manually) within the image, and the spatial relationships between each point within the construct are known / input, and can execute method 100 including steps. Next, at 104, method 100 may include steps of generating four vectors to an arbitrary focus positioned on the 2D image plane of the image using the four identified discrete points as a basis. Next, method 100 may include steps of digitally establishing a cost function at 106 to evaluate the fitness of an arbitrary focus with respect to a known separation distance of the 3D construct (e.g., via one or more of the vector loops described above, tripod dimensional reduction to 2D space, or the sliding triangle method). Next, at 108, method 100 may include steps of digitally using the cost function as a discriminant in an optimization numerical solver to determine a compatible focus FP(x,y,z). Note that the specific optimization numerical solution steps may vary depending on the approach used. Using the optimized focus FP(x,y,z) and the known discrete points within the image, method 100 may include steps of digitally determining the positions of known elements along vectors (from focus FP(x,y,z)) at 110. A tetrahedron is arranged in the image space by the positions corresponding to the 3D coordinates of the elements of the 3D construct. Next, at 112, method 100 may include steps of digitally establishing two vectors having a common vertex using any two edges of any of the four triangular faces of the tetrahedron. Next, method 100 may include steps of digitally establishing a mutual normal at 114, such as by taking the cross product in a preferred order of the two vectors, and digitally establishing an orthogonal coordinate system in the image space, such as by taking the cross product in a preferred order between the mutual normal and a preferred selection of the original two vectors.In some embodiments, method 100 may then, at 116, include digitally constructing a transformation (e.g., a matrix) between the Cartesian coordinate system from 114 and any other combination of face vertices using known relationships between the faces of a known 3D tetrahedron.

[0082] Next, method 100 may, at 118, include repeating 102 - 114 (and potentially 116) for all combinations of four discrete points within a larger group of discrete points if more than four discrete points are available within the image. Next, at 120, method 100 may include digitally generating a composite coordinate transformation (e.g., a matrix) representing the known 3D structure within the image space, such as by averaging all of the equivalent transformations (e.g., those generated at step 116 and / or 118). Next, method 100 may, at 122, include digitally generating a 2D image transformation with respect to the known 3D structure via the inverse coordinate transformation of 120.

[0083] Next, method 100 may include repeating 102 - 122 for each 2D image obtained from a known 3D structure (e.g., multiple images such as two or more images), and digitally constructing a 3D representation of the multiple images with respect to the 3D structure. Next, method 100 may include digitally establishing the intersection of the closest points between planes and planes, or planes and vectors, or vectors, representing aspects of the anatomical structure emanating from the focus FP(x, y, z) of each corresponding 2D image, to determine the relationship between the known 3D structure and the anatomical structure of interest using the 3D representation of the 2D images with respect to the known 3D structure.

[0084] As will be apparent to those skilled in the art, the invention of the present disclosure provides significant improvements in the fields of external fixation devices and anatomical structure computer modeling, including the field of hexapod and bone segment modeling. Further, the invention of the present disclosure provides important improvements in the field of radiographic imaging, including the field of distortion correction of radiographic images. The invention of the present disclosure also provides important improvements in the field of external fixation device adjustment prescription determination, including the field of hexapod adjustment prescription.

[0085] Those skilled in the art will recognize that aspects of the present invention may be implemented in a system, method, and / or computer program product. In some embodiments, aspects of the present invention may be implemented entirely in hardware, entirely in software (e.g., firmware, resident software, microcode, etc.), or in a combination of software and hardware aspects, all of which are generally referred to herein as a "system" and may include circuits and / or modules.

[0086] FIG. 15 shows an example of a computer system for use incorporating one or more aspects of the present invention. The computer system 500 can be a computer system used to additionally manufacture an article and / or a computer system of an article manufacturing and / or repair facility such as a computer system for generating data used to manufacture an article by an AM device or device. The computer system 500 of FIG. 15 is suitable for storing and / or executing program code such as program code for executing the above process, and includes at least one processor 502 directly or indirectly coupled to the memory 505 via a bus 520. During operation, the processor(s) 502 can obtain instructions from the memory 505 for execution by the processor(s). The memory 505 can include local memory used during actual execution of the program code, bulk storage, and cache memory that provides temporary storage of at least some program code to reduce the number of times code must be retrieved from bulk storage during program code execution. A non-limiting list of examples of the memory 505 includes hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fibers, portable compact disk read only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing. The memory 505 can include an operating system 505 and one or more computer programs 506, for example, one or more programs for executing the aspects described herein, such as performing adjustments to a digital layout of a circuit design.

[0087] Input / output (I / O) devices 512, 515 (such as peripheral devices) can be coupled to the system either directly or via the I / O controller 510. The network adapter 508 can also be coupled to the system to enable the computer system to be coupled to other computer systems via an intervening private or public network. Modems, cable modems, and Ethernet cards are but a few of the currently available types of network adapter 508. In one example, the network adapter 508 facilitates the acquisition of data from a remote source to facilitate aspects of the present invention.

[0088] The computer system 500 can be coupled to storage 516 having one or more databases (e.g., a non-volatile storage area such as a magnetic disk drive, an optical disk drive, a tape drive, etc.). Storage 516 can include an internal storage device or connected or network-accessible storage. Computer programs within storage 516 can be loaded into memory 505 and executed by the processor 502.

[0089] The computer system 500 can include fewer components than shown, additional components not shown herein, or some combination of the illustrated components and additional components. The computer system 500 can include any computing device such as a mainframe, server, personal computer, workstation, laptop, handheld computer, smartphone, tablet, or other mobile device, telephone device, network appliance, virtualization device, storage controller, etc.

[0090] Furthermore, the above process can be executed by a plurality of computer systems 500 operating in cooperation as part of a computing environment.

[0091] In some embodiments, aspects of the invention can take the form of a computer program product embodied on one or more computer-readable media. The one or more computer-readable media can have computer-readable program code embodied thereon. A variety of computer-readable media or combinations thereof can be utilized. For example, the computer-readable media can include computer-readable storage media, examples of which include one or more electronic, magnetic, optical, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing (but not limited thereto). Examples of computer-readable storage media include, for example, electrical connections having one or more wires, portable computer diskettes, hard disks or mass storage devices, random access memory (RAM), read-only memory (ROM), and / or erasable programmable read-only memory such as EPROM or flash memory, optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices (including tape devices), or any suitable combination of the foregoing. A computer-readable storage media is defined to include a tangible medium that can include or store program code for use by or in connection with a device such as an instruction execution system, apparatus, or processor. Thus, program code stored on a computer-readable medium generates a product (such as a "computer program product") that includes the program code.

[0092] Referring now to FIG. 16, in one example, a computer program product 600 includes one or more computer-readable media 602 having computer-readable program code means or logic 604 stored thereon for providing and facilitating one or more aspects of the present invention.

[0093] Program code contained in or stored on a computer-readable medium can be fetched and executed by a computer system (a computer including its components, a computer system, etc.) and / or other devices to cause the computer system, its components, and / or other devices to operate / function in a particular manner. The program code can be transmitted using any suitable medium including, but not limited to, wireless, wired, fiber optic, and / or radio frequency. The program code for performing operations for implementing, achieving, or facilitating aspects of the present invention can be written in one or more programming languages. In some embodiments, the programming languages include object-oriented and / or procedural programming languages such as C, C++, C#, Java, etc. The program code can be executed entirely on the user's computer, entirely away from the user's computer, or in a combination where part is on the user's computer and part is on a remote computer. In some embodiments, the user's computer and the remote computer communicate via a network such as a local area network (LAN) or a wide area network (WAN), and / or via an external computer (e.g., using the Internet via an Internet service provider).

[0094] In one example, the program code includes one or more program instructions obtained for execution by one or more processors. The computer program instructions are provided, for example, to one or more processors of one or more computer systems to manufacture a machine, and as a result, when the program instructions are executed by one or more processors, aspects of the present invention, such as the actions or functions described in the flowcharts and / or block diagrams described herein, can be executed, achieved, or facilitated. Accordingly, each block, or combination of blocks, of the flowchart diagrams and / or block diagrams shown and described herein can, in some embodiments, be implemented by computer program instructions.

[0095] The flowcharts and block diagrams shown and described with reference to the figures illustrate the architecture, functionality, and operation of possible embodiments of a system, method, and / or computer program product according to aspects of the present invention. Accordingly, these flowchart diagrams and / or block diagrams can be those of a method, apparatus (system), and / or computer program product according to aspects of the present invention.

[0096] In some embodiments, as described above, each block of a flowchart or block diagram can represent a module, segment, or portion of code, and can include one or more executable instructions for implementing the specified operations and / or logical functions of the block. Those skilled in the art will appreciate that the operations / functions specified or performed by the blocks can occur in a different order than that shown and / or described, or can occur concurrently with one or more other blocks, or partially / fully concurrently. In fact, two consecutive blocks shown may be executed substantially concurrently in some cases or in reverse order in other cases. Further, each block of the block diagram and / or flowchart diagram, as well as combinations of blocks in the block diagram and / or flowchart diagram, can be implemented entirely by a dedicated hardware-based system that performs the specified operations / functions of the block or block diagram or flowchart as a whole, or in combination with computer instructions.

[0097] It should be understood that the above description is intended to be illustrative and not restrictive. Without departing from the general spirit and scope of the invention as defined by the following claims and their equivalents, many changes and modifications can be made by those skilled in the art in this specification. For example, the above embodiments (and / or aspects thereof) can be used in combination with each other. Further, many modifications can be made to adapt a particular situation or material to the teachings of various embodiments without departing from their scope. The dimensions and types of materials described herein are intended to define parameters of various embodiments, but they are in no way limiting and are merely illustrative. Upon considering the above description, many other embodiments will be apparent to those skilled in the art. Therefore, the scope of the various embodiments should be determined with reference to the appended claims, along with the full scope of equivalents to which such claims are entitled.

[0098] The terms used in this specification are for the purpose of describing particular embodiments only and are not intended to limit the present invention. As used in this specification, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. Further, "comprise" (and any form of comprise such as "comprises" and "comprising"), "have" (and any form of have such as "has" and "having"), "include" (and any form of include such as "includes" and "including"), "contain" (and any form of contain such as "contains" and "containing"), and other grammatical variations are to be understood as free-form conjunctive verbs. As a result, a method or article that "comprises", "has", "includes", or "contains" one or more steps or elements owns those one or more steps or elements but is not limited to owning only those one or more steps or elements. Similarly, a step of a method or an element of an article that "comprises", "has", "includes", or "contains" one or more features owns those one or more features but is not limited to having only those one or more features.

[0099] As used in this specification, the terms "comprising", "has", "including", "containing", and other grammatical variations thereof include the terms "consisting of" and "consisting essentially of".

[0100] As used herein, the phrase "consisting essentially of" or grammatical variations thereof is to be construed as specifying the recited features, integers, steps or components, but not excluding the addition of one or more additional features, integers, steps, components or groups thereof, provided that the additional features, integers, steps, components or groups thereof do not substantially change the basic and novel characteristics of the claimed composition or method.

[0101] All publications cited herein are hereby incorporated by reference as if each individual publication were specifically and individually indicated to be incorporated by reference as though fully set forth herein.

[0102] The subject matter incorporated by reference is not considered an alternative to claim limitations unless specifically stated otherwise.

[0103] Where one or more ranges are referred to herein, each range is intended as a shorthand for presenting information and is to be understood to include each discrete point within the range as if the same were fully set forth herein.

[0104] Although some aspects and embodiments of the invention are described and shown herein, alternative aspects and embodiments may be affected by those skilled in the art to achieve the same purpose. Accordingly, the present disclosure and the appended claims are intended to cover all such further alternative aspects and embodiments as fall within the true spirit and scope of the invention.

Claims

1. A method for determining the actual position and pose of a known three-dimensional structure by using at least four individual shapes formed according to the criteria of the structure shown in a two-dimensional (2D) image of the structure, the method comprising: identifying at least four fiducial shadows in the 2D image corresponding to the criteria of the structure; correlating the at least four discovered fiducial shadows with their respective positions on the structure; determining the spatial relationship between the 2D image and the structure by determining the focus of the source of the image relative to the 2D image via a predetermined mutual separation distance between the at least four discovered fiducial shadows and the corresponding criteria of the structure; determining the spatial relationship between the 2D image and the structure and including.

2. The step of correlating the at least four discovered fiducial shadows with their respective positions on the structure comprises: identifying the at least four discovered fiducial shadows as upper or lower fiducial shadows; determining the foreground or background order of the at least four discovered fiducial shadows based on their respective sizes; determining the left-to-right or right-to-left order of the at least four discovered fiducial shadows; annotating the at least four discovered fiducial shadows and correlating them with their respective annotated fiducial positions on the structure and including. The method according to claim 1.

3. The step of determining the spatial relationship between the 2D image and the structure comprises: identifying the actual fiducial positions along the vectors from the focus to the fiducial shadow positions; converting the actual fiducial positions into three-dimensional (3D) image coordinates; defining the actual fiducial position vectors between the fiducial positions via the 3D image coordinates; constructing a first orthogonal coordinate system of a collection of three discrete fiducials with respect to the 2D image by determining the vector cross product between appropriate pairs of the position vectors; inverting the first constructed orthogonal coordinate system or a second constructed orthogonal coordinate system determined via the first constructed orthogonal coordinate system to develop a coordinate transformation of the 2D image with respect to any coordinate system representing the structure and including. The method according to claim 1.

4. Determining the spatial relationship between the 2D image and the structure by determining the focus of the source of the image with respect to the 2D image via the at least four fiducial shadows found and a predetermined mutual separation distance between the fiducials of the structure corresponding thereto, establishing a Cartesian coordinate system that utilizes the 2D image as one of the three planes of the coordinate system; determining the positions along the focus rays where each of the at least four fiducials must exist; constraining a model of the at least four fiducials based on known characteristics of the structure via a cost function, the known characteristics not including one ray and four fiducial distances, the constraint forming a tripod model that traces a planar curve in a plane perpendicular to the image plane; reconfiguring the tripod model such that a first plane formed by a first group of three of the at least four fiducials is positioned along the image plane; determining a first equation of a first line representing the intersection of the image plane and the first plane; reconfiguring the tripod model such that a second plane formed by a second group of three of the at least four fiducials is positioned along the image plane; determining a second equation of a first line representing the intersection of the image plane and the second plane; determining the x and y coordinates of the focus via at least the first and second lines; determining the z coordinate of the focus via the x and y coordinates and the cost function The method according to claim 1, comprising.

5. A computer-readable storage medium readable by one or more processing circuits and storing instructions for execution by one or more processors for performing the method according to any one of claims 1 to 4 A computer program product comprising.

6. A memory; At least one processor in communication with the memory; Program instructions executable by one or more processors via the memory to perform the method according to any one of claims 1 to 4 A system comprising.

7. A method for determining the actual position and pose of a collection of known objects in a projected three-dimensional space above a two-dimensional radiation space, the method comprising: obtaining two or more digital radiographic images of the collection of known objects in the projected three-dimensional space above the two-dimensional radiation space; using the shadows of the collection of known objects in the two-dimensional radiation space of the two or more digital radiographic images to determine the actual position and pose of the collection of known objects in the projected three-dimensional space above the two-dimensional radiation space; A method comprising:

8. The method according to claim 7, further comprising constructing a three-dimensional model of the actual position and pose of the collection of known objects in the projected three-dimensional space.

9. The method according to claim 7, wherein the step of using the shadows of the collection of known objects in the two-dimensional radiation space of the two or more digital radiographic images to determine the actual position and pose of the collection of known objects includes determining the relative magnification of the images using projection distortion to reconstruct the projected three-dimensional space.

10. The system according to claim 9, further comprising determining the relationship between the two or more digital radiographic images through comparison of common objects in the collection of known objects within the images.

11. The system according to claim 10, wherein the two or more digital radiographic images further include at least one anatomical structure that requires correction, and further comprising constructing a three-dimensional model of the actual position and pose of the at least one anatomical structure in the projected three-dimensional space.

12. A computer-readable storage medium storing instructions executable by one or more processors to perform a method for determining the actual position and pose of a collection of known objects in a projected three-dimensional space above a two-dimensional radiation space, the method being readable by one or more processing circuits. A computer program product comprising: Obtaining two or more digital radiographic images of a collection of the known objects in the projected three-dimensional space above the two-dimensional radiation space; Determining the actual position and pose of the collection of the known objects in the projected three-dimensional space above the two-dimensional radiation space by using the shadows of the collection of the known objects in the two-dimensional radiation space of the two or more digital radiographic images; A computer program product comprising the above. **Claim 13** A memory; At least one processor communicating with the memory; Program instructions executable by one or more processors via the memory to perform a method for determining the actual position and pose of a collection of known objects in a projected three-dimensional space above a two-dimensional radiation space, A system comprising: the method comprising: Obtaining two or more digital radiographic images of a collection of the known objects in the projected three-dimensional space above the two-dimensional radiation space; Determining the actual position and pose of the collection of the known objects in the projected three-dimensional space above the two-dimensional radiation space by using the shadows of the collection of the known objects in the two-dimensional radiation space of the two or more digital radiographic images; A system comprising the above.

Images