Single-field-of-view radiological image and three-dimensional model registration method and system

Abstract

The systems and methods disclosed in this application include a method for determining the actual position and orientation of a known three-dimensional (3D) structure using at least four discrete shapes, the at least four discrete shapes being formed by the reference of the structure shown in a two-dimensional (2D) image of the structure. The method includes identifying at least four reference shadows in the 2D image corresponding to the reference of the structure. The method further includes associating the at least four reference shadows found with their respective positions on the structure. The method further includes determining the spatial relationship between the 2D image and the structure by determining the focus of the image source relative to the 2D image, and determining the focus of the image source relative to the 2D image is carried out by a predetermined mutual separation distance between the at least four reference shadows found and the reference of the corresponding structure. The method further includes determining the spatial relationship between the 2D image and the structure.

Description

[0001] Cross - Reference to Related Applications

[0002] This application claims the benefit of and priority to U.S. Provisional Application No. 62 / 817,185, filed on Mar. 12, 2019, titled “Single - Field - of - View Image Registration Method and System,” the entire content of which is hereby expressly incorporated by reference into this application. This application also relates to International PCT Patent Application No. PCT / US2019 / 043326, filed on Jul. 24, 2019, titled “Methods and Systems for Registering a Three - Dimensional Model of a Radiographic Image and an External Fixation Device,” the entire content of which is hereby expressly incorporated by reference into this application. Technical Field

[0003] This application is mainly related to image registration using known three - dimensional (3D) structures depicted in an image. More specifically, this application relates to methods and systems for registering a single - field - of - view image (e.g., a radiographic image) and a 3D model. The methods and systems utilize known physical parameters of a given 3D structure depicted in the image (e.g., the distance between at least four shapes or points).

[0004] This application is also mainly related to systems and methods for performing deformity analysis using multiple non - orthogonal radiographs. Embodiments of this application relate to treating musculoskeletal conditions, including fractures. More specifically, this application discloses methods and systems for fixing and placing bone segments of one or more bones at a desired location. In some embodiments of this application, these methods and systems are used to generate three - dimensional computer models of a fixation device, bone segments, and possibly at least one (e.g., at least two) radiographic image representations corresponding to a radiographic image used to create the three - dimensional computer model. In one embodiment, regardless of the initial configuration of the fixation device or the orientation of the radiographic image relative to the device and / or the bone, through operations on the model, the desired placement of the bone segments, and operations on the external fixation device, the achievement of such a desired placement can be determined quickly and accurately. Then, the operations required to create the desired placement of the bone segments can be performed on the corresponding fixation device and bone segments to treat musculoskeletal conditions. However, other devices besides the external fixation device can be used with the system and method. Background Art

[0005] In the medical field, the correction of orthopedic deformities typically involves at least a pair of X-ray images. These images of a typical patient are usually taken along a conventional line from anterior to posterior (AP) and from medial to lateral (ML), or along other orthogonal or known advantageous positions (or known differences between advantageous positions). Conventionally, AP and ML X-ray images are taken, or assumed to be orthogonal to each other in the patient space (defining the patient space as aligned from right to left along the X-axis, from anterior to posterior along the Y-axis, and from bottom to top along the Z-axis). Measurements are taken in each pair of images, and the axes and points of the deformity are annotated. These measurements and annotations are then used to reconstruct a true three-dimensional representation of the deformity so that the deformity can be manipulated in certain ways to correct it.

[0006] However, problems often arise due to the uncertainties of the X-ray images and their spatial relationships to each other. X-ray images are not perfect images of the artifacts contained within these images. The relationship between the artifacts shown in the image and the actual object being imaged is a perspective relationship, such that objects closer to the image have a smaller magnification than objects farther from the image. Additionally, the uncertainty of the orthogonality between image pairs makes it difficult to reconstruct a true representation.

[0007] Thus, there is a need for methods to account for these uncertainties of such X-ray images due to their actual perspective / advantageous positions.

[0008] In addition, in many research fields, it is often desirable to register a two-dimensional (2D) image with a known three-dimensional (3D) object. "Registration" refers to constructing a coordinate transformation so that the position and orientation of the 3D object can be determined within a coordinate system consistent with the 2D image. For example, when any plane of the 3D coordinate system is coplanar with the 2D image, the 3D coordinate system can be considered to be in agreement with the 2D image. Registering the 3D object within this coordinate system can create one or more virtual environments in which the viewer's perspective can be determined and the image and object can be correctly placed within the environment.

[0009] In the medical field, this is typically an important step in correctly placing or manipulating implants, surgical instruments, or body tissue structures. Compared with 3D imaging techniques such as computed tomography (CT) and magnetic resonance imaging (MRI), one of the most common imaging methods is basic X-ray radiography, which has the advantages of low cost and real-time accessibility in the operating environment. It is desirable to be able to register a known 3D object or body structure relative to a real-time image on a single-image basis.

[0010] Currently, there are registration methods for multiple images using a given 3D structure combination. However, in these cases, it is necessary to know the spatial relationship between multiple images with a high degree of certainty. Some current stereoscopic image-guided systems can achieve this externally and generally rely on the known relationship between the camera pairs used. Some other current methods generally require taking multiple images, such as anteroposterior (AP) and mediolateral (ML) radiographs. As described above, the relationship between such images is affected by variables inherent in taking such images, which can lead to errors in 3D registration.

[0011] Specifically, in orthopedic surgery, it is often necessary to use a device called an external fixator to correct skeletal deformities. Such external fixators have various configurations, such as from simple unilateral and pin-rod systems to more complex circular structures. To accurately correct such skeletal deformities, when the structure is mounted on a patient, it is necessary to accurately characterize the spatial relationship between the skeletal anatomy and the fixator structure. This characterization process can start with taking multiple images, such as two or more 2D radiographs or 3D image scans of the fixator configuration mounted on the bone. Due to the simplicity and low cost of 2D radiographs, they are the main means of obtaining such a characterization. Therefore, it is desirable to accurately register each of multiple two-dimensional images on an individual basis of an external fixator or other known 3D entity to accurately place other body structures relative to the known 3D entity.

[0012] Although the present application has discussed certain aspects of conventional techniques to facilitate the disclosure of the applicant's invention, the applicant does not deny these technical aspects and believes that its invention may include one or more aspects of conventional techniques.

[0013] In this specification, when a document, act, or knowledge is cited or discussed, such citation or discussion does not admit that the document, act, or knowledge or any combination thereof was publicly available, known to the public as common general knowledge, or formed part of common general knowledge, or was known in relation to an attempt to solve any problem addressed in this specification, as of the priority date. Summary of the Invention

[0014] The present application can solve one or more problems and deficiencies in the art discussed above. However, the present application can solve other problems and deficiencies in many technical fields. Therefore, the invention claimed in the present application should not be construed as limited to solving any specific problem or deficiency discussed herein.

[0015] The present application mainly relates to an image registration method and system that utilize a known three-dimensional (3D) construct shown in an image. More specifically, the present application relates to a registration method and system for a single-field-of-view image (e.g., a radiograph) with known physical parameters and a 3D model (e.g., the distance between at least four shapes or points of a given 3D construct shown in the image).

[0016] This application also generally relates to systems and methods for performing malformation analysis using multiple radiographs taken from unknown (or inaccurate or mis-identified) advantageous positions (such as non-orthogonal radiographs). In some embodiments, the systems and methods each register each two-dimensional image (having a known three-dimensional structure and / or reference shape (and dimensions)), and use the registered images to construct a three-dimensional model as part of malformation and / or malformation correction analysis and prescription determination.

[0017] Some embodiments of this application relate to treating musculoskeletal disorders, including fractures. More specifically, this application discloses methods and systems for fixing and placing bone segments of one or more bones at a desired location. In some embodiments of this application, these methods and systems are used to generate a three-dimensional computer model of a fixation device, bone segments, and possibly at least one (e.g., at least two) radiographic representations corresponding to radiographs for creating the three-dimensional computer model. In one embodiment, regardless of the initial configuration of the fixation device or the orientation / vertices of the radiographs relative to the device and / or the bone, by operating on the model, the desired placement of the bone segments, and the operation of the external fixation device, the achievement of such a desired placement can be determined quickly and accurately. Then, the operations required to create the desired placement of the bone segments can be performed on the corresponding fixation device and bone segments to treat the musculoskeletal disorder. However, other devices in addition to the external fixation device can be used with the systems and methods.

[0018] In some embodiments, this application provides methods and related systems for associating the planar positions and features of four discrete shapes or points (which are included in a given two-dimensional radiograph) with four discrete spatial coordinates included in a fixator construct (or the construct of another known object). Using this information, these methods and related systems obtain an accurate spatial relationship between the fixation construct (or the construct of another known object) and each individual radiograph.

[0019] In some embodiments, the radiograph includes the shadows of three-dimensional objects that are posed and located above the image (e.g., film) at the time of capture. The apparent source position and orientation of the X-ray source relative to the image that projects the shadows are unknown. Ideally, the focal point is a point source placed infinitely far above the image itself and centered on the image itself. The ideal representation would result in the shadows being true two-dimensional projections of the actual three-dimensional objects. If we have two ideal representations and we know that the two ideal representations are orthogonal with respect to a common axis, then we can directly utilize the two sets of two-dimensional data to accurately reconstruct the three-dimensional object and its position and orientation in space. However, this is typically not possible because the current state of radiographic techniques involving plain film radiographs results in perspective distortion. Additionally, considering all of the variables involved in an actual X-ray machine where an actual patient lies / poses in a prescribed manner as directed, the likelihood that the radiographs taken are truly orthogonal to the trajectory of the X-ray source and orthogonal to each other about a common axis is unlikely to occur.

[0020] The systems and methods of the present application can utilize two main sources of error, the focal position and pose, and the patient orientation, to draw multiple conclusions in order to ultimately correct for any unexpected correction (e.g., rotation starting from an orthogonal arrangement) between a pair of radiographs and construct a true three-dimensional model of the objects within the radiographs.

[0021] The systems and methods can account for perspective distortion by identifying radiopaque objects and the shadows they project in the radiograph. It is well known that the edges of such artifacts are relatively sharp. The systems and methods can utilize these relatively sharp edges. The systems and methods can utilize the shape edges and infer that the source of the radiograph (i.e., the x-ray) that projects the shadow is actually located somewhere above the shadow image and forms a point. As Figure 10 shown, the systems and methods can also conclude that the object lies on a vector that describes the line between the center of the shadow or point of a given object artifact and the focal point of the x-ray source. The systems and methods can also conclude that if the shape and actual size of the artifact are known, then the relative distance between the shadow image and the actual artifact as well as the distance between the shadow image and the x-ray source can be determined. However, this alone is not sufficient to determine the position and orientation of the X-ray source. Therefore, the systems and methods can utilize multiple known objects whose shadows exist as artifacts in the radiograph and whose relative shapes, sizes, and relationships to other things are known, to determine the apparent focal position or the X-ray and the orientation of the device relative to the image.

[0022] As Figure 10As shown, the system and method can use multiple closed vector loops through the shadow center, object center, and focal position to determine the position and orientation of a three-dimensional set of known objects in the radiographic space. As Figure 10 As shown, after determining multiple closed-loop vectors, the system and method can define a coordinate transformation for a set of known three-dimensional objects in the shadow image space (i.e., determine the row size, column size, and height size). After determining the coordinate system, the system and method can use the set of known three-dimensional objects in each of the multiple radiographs and use a consistent method in each image to determine the coordinate transformation between any pair of images within the multiple images. When constructing the true three-dimensional position and orientation of the three-dimensional object, the system and method can correct any non-orthogonal or rotated image pairs, thus accurately describing any other annotations or measurements performed within the radiograph.

[0023] On the other hand, the present application provides some methods and systems that use a three-dimensional set of known objects to project their shadows in a two-dimensional X-ray radiographic space to determine the actual position and orientation of the set of known objects in the projection above the two-dimensional radiographic space and in a computer-modeled three-dimensional space.

[0024] In some embodiments, these methods and systems can use perspective distortion to determine the relative magnification to assist in reconstructing the three-dimensional projection space. In some embodiments, these methods and systems can determine the relationship between multiple radiographs through the analysis of known common objects. In some such embodiments, these methods and systems can reconstruct a model of the actual three-dimensional conditions in a corrected relative space setting.

[0025] In some embodiments, these methods and systems can include a method of using at least four discrete shapes to determine the actual position and orientation of a known three-dimensional structure, where the at least four discrete shapes are formed by the reference of the structure, and the structure is shown in a two-dimensional image of the structure. The method includes: identifying at least four reference shadows in the 2D image corresponding to the reference of the structure; associating the at least four found reference shadows with their respective positions on the structure; determining the spatial relationship between the 2D image and the structure by determining the focus of the image source relative to the 2D image, where determining the focus of the image source relative to the 2D image is performed by a predetermined mutual separation distance between the at least four found reference shadows and the reference of the structure corresponding thereto; and determining the spatial relationship between the 2D image and the structure.

[0026] In some embodiments, associating at least four fiducial shadows found therewith to their respective locations in the structure includes: identifying the at least four fiducial shadows found as upper fiducial shadows or lower fiducial shadows; determining the foreground or background order based on the respective sizes of the at least four fiducial shadows found; determining the order of the at least four fiducial shadows found from left to right or from right to left; and annotating the at least four fiducial shadows found to be associated with the locations of the fiducial shadows respectively annotated in the structure.

[0027] In some embodiments, determining the spatial relationship between the 2D image and the structure includes: locating the actual fiducial positions along the vectors from the focus to the fiducial shadow positions; converting the actual fiducial positions to 3D image coordinates; defining the actual fiducial position vectors between the fiducial positions through the 3D image coordinates; constructing a first orthogonal coordinate system for the 2D image for a set of three discrete fiducials by determining the vector cross product between appropriate position vector pairs; and reversing the first constructed orthogonal coordinate system or reversing the second constructed orthogonal coordinate system by the first constructed orthogonal coordinate system to develop a coordinate transformation for the 2D image with respect to any coordinate system representing the structure.

[0028] In some embodiments, determining the spatial relationship between the 2D image and the structure by determining the image source relative to the focus of the 2D image, wherein determining the image source relative to the focus of the 2D image is carried out by a predetermined mutual separation distance between at least four fiducial shadows found and the fiducials of the corresponding structure, includes: establishing an orthogonal coordinate system using the 2D image as one of the three planes of the coordinate system; determining that each of the at least four fiducial shadows must be located along the focus ray; constraining the model of the at least four fiducials by a cost function based on known features of the structure, the known features not including a ray and four fiducial-fiducial distances, wherein the constraint forms a tripod model that outlines a planar curve, the planar curve being located in a plane perpendicular to the image plane; reconfiguring the tripod model such that a first plane formed by three fiducials of a first group of the at least four fiducials lies on the image plane; determining a first equation for a first line, wherein the first line depicts the intersection of the image plane and the first plane; reconfiguring the tripod model such that a second plane is distributed along the image plane, wherein the second plane is formed by three fiducials of a second group of the at least four fiducials; determining a second equation for the first line depicting the intersection of the image plane and the second plane; determining the x and y coordinates of the focus by at least the first line and the second line; and determining the z coordinate of the focus by the x and y coordinates and the cost function.

[0029] In some embodiments, these methods and systems may include a method for determining the actual position and orientation of a set of known objects in a projected three-dimensional space (the projected three-dimensional space is located above a two-dimensional radiographic space), including: obtaining two or more digital radiographic images of the set of known objects in the projected three-dimensional space above the two-dimensional radiographic space; and using the shadows of the set of known objects in the two-dimensional radiographic space in the two or more digital radiographic images to determine the actual position and orientation of the set of known objects in the projected three-dimensional space above the two-dimensional radiographic space.

[0030] In some embodiments, the method further includes constructing a three-dimensional model of the actual position and orientation of the set of known objects in the projected three-dimensional space. In some embodiments, the above-mentioned use of the shadows of the set of known objects in the two-dimensional radiographic space in the two or more digital radiographic images to determine the actual position and orientation of the set of known objects includes using perspective distortion to determine the relative magnification of the images to reconstruct the projected three-dimensional space. In some embodiments, the relationship between two or more digital radiographic images is determined by comparing the common objects in the images of the set of known objects. In some embodiments, the above-mentioned two or more digital radiographic images further include at least one anatomical structure that needs to be corrected, and further include constructing a three-dimensional model of the actual position and orientation of the at least one anatomical structure in the projected three-dimensional space.

[0031] In some embodiments, the present application also provides a computer program product, including: a computer-readable storage medium that can be read by one or more processing circuits and stores instructions executed by one or more processors for performing the above method.

[0032] The present application also provides a system, including: a memory, at least one processor communicatively coupled to the memory, and program instructions executable by one or more processors through the memory to perform the above method. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The present application will be described in conjunction with the following drawings. These drawings are not necessarily drawn to scale and are only for ease of understanding. Among them, the same reference numerals in each drawing have their designated names and meanings for the same or similar elements.

[0034] Figure 1 Shows an exemplary 3×3 external fixator (e.g., hexapod) configuration according to the present application.

[0035] Figure 2 Shows an exemplary focus model using a single tetrahedron according to the present application.

[0036] Figure 3A Shows an exemplary three-dimensional tetrahedron cost function according to the present application.

[0037] Figure 3B Shows an example two-dimensional three-sided cost function according to the present application.

[0038] Figure 4 Shows an example three-sided simplified form of a tetrahedron according to the present application.

[0039] Figure 5A Shows the case of transposing / placing a three-sided body into the image plane for ABC according to the present application.

[0040] Figure 5B Shows Figure 5A A vertical view of transposing / placing a three-sided body into the image plane for ABC.

[0041] Figure 6 Shows example F1(x, y) and F2(x, y) solutions for FPxy according to the present application.

[0042] Figure 7 Shows the transposing / placing of a three-sided body into the image plane for ABD according to the present application.

[0043] Figure 8A Shows an example intersection of plane ABC and plane ABD in two dimensions (x, y) according to the present application.

[0044] Figure 8B Shows an example intersection of plane ABC and plane ABD in three dimensions (x, y, z) according to the present application.

[0045] Figure 9 Shows an exemplary three-dimensional tetrahedron cost function using coordinates according to the present application.

[0046] Figure 10 Is a perspective view of constructing a three-dimensional model of an external deformity correction system from corrected radiographs according to the present application.

[0047] Figure 11 Shows a plurality of foci according to the present application whose connecting lines pass through two unconnected triangles generating the same shadow coordinates.

[0048] Figure 12 Shows three identical triangles casting the same shadow from a single focus according to the present application.

[0049] Figure 13 Shows a pair of triangles positioned along rays from an arbitrary focus to a set of shadow positions in the image plane according to the present application.

[0050] Figure 14 Shows a flowchart of an exemplary method according to the present application.

[0051] Figure 15 An exemplary computer system that can be used to implement aspects (e.g., methods) of the present application is shown.

[0052] Figure 16 An embodiment of a computer program product that can be incorporated into the present application is shown. Detailed Description

[0053] The aspects of the present application and certain of its features, advantages, and details are described more fully below with reference to the non-limiting embodiments shown in the accompanying drawings, omitting descriptions of well-known materials, manufacturing tools, processing techniques, etc., so as not to unnecessarily obscure the present application in detail. However, it should be understood that the detailed description and specific examples, although showing embodiments of the present application, are only for illustrative purposes and do not limit the present application. According to the present application, various substitutions, modifications, additions, and / or arrangements within the spirit and / or scope of the concept of the present application are obvious to those skilled in the art.

[0054] The association of the planar positions and features of four discrete shapes (which are included in a given two-dimensional radiograph) with four discrete spatial coordinates included in a known structure (e.g., a fixator structure) involved in the method, system, and related computer program product will not be described in conjunction with Figures 1-9 This method, system, and related computer program product can utilize such information to obtain and (display to the user) the accurate spatial relationship between the structure (e.g., the fixator structure) and each of the individual radiographs. It is noted here that although the method, system, and related computer program product can be described herein with reference to an external fixator structure (e.g., a hexapod structure), these method, system, and related computer program product may not be named after any 3D structure (another orthopedic structure or non-orthopedic structure) that includes at least four of its spatial coordinates / shapes (e.g., spheres or ellipsoids or points) (and the relationship between at least one 2D image (e.g., a radiograph) thereof and one or more anatomical structures). Further, although spheres may be used in the description of the present application to describe the method, system, and related computer program product as four discrete spatial coordinates and / or shapes, other known shapes (e.g., but not limited to spheroids or points) may equally be used as understood by those of ordinary skill in the art.

[0055] Three-dimensional spatial relationships can require six parameters to relate the position and orientation between any two solid objects in space. The position can be considered as a translational configuration within a typical orthogonal (x, y, z) coordinate system. The orientation can be regarded as a series of rotations about the x, y, and / or z axes of the same position coordinate system. In a given three-dimensional space, all six of these are the degrees of freedom (DOF) in the given three-dimensional space, which in layman's terms are called In-Out, Left-Right, Up-Down (x, y, z), Roll, Pitch, and Yaw (r, p, y).

[0056] The simplest three-dimensional object is a sphere with a central position (x, y, z) and a given radius (r). A sphere (e.g., a spherical body) can be used to determine the position (x, y, z) of such an object in three-dimensional space. Since both the position and orientation of a known structure (e.g., an external fixture structure, such as a hexapod) are required, it is necessary to consider what type of three-dimensional object can uniquely exist in three-dimensional space. A tetrahedron is a relatively simple such object, which is a triangular pyramid with four discrete vertices and four triangular faces.

[0057] A common type of circular fixture is the so-called hexapod structure, which includes two planar rings connected by six telescopic struts, each with a spherical joint at both ends. Most hexapod structures on the market are configured in the so-called 6×6 configuration, that is, there are six discrete spherical mounting positions, usually configured in pairs and equidistantly arranged around the central axis of each ring, but this is not necessary. A mathematically simpler hexapod structure is the so-called 3×3 configuration, in which pairs of spherical joints are synchronized with each other, and there are three synchronous pairs on each ring. Such a 3×3 hexapod structure can be decomposed into fifteen discrete tetrahedrons, any one of which can be used to describe the position and orientation of the hexapod fixture structure in three-dimensional space.

[0058] It should be noted that although three points can be used to define a two-dimensional plane, it cannot be used to define a three-dimensional structure. As Figure 11 and 12As shown, multiple foci of a triangle (3 points) of fixed size can produce the same shadow coordinates for two unconnected triangles, with each triangle representing three positions on each of two planar fixed platforms (such as rings) of a fixed configuration (e.g., a hexapod configuration). By using a fourth point, some embodiments of the methods and systems of the present application allow for the construction of four connected triangles of known size, which allows for the determination of a single focus, as further described below. However, in some embodiments, the methods and systems of the present application can utilize three points on a three-dimensional structure and multiple views (and known relationships between the views) of the same structure and points. However, such three-point embodiments are less effective than four-point embodiments.

[0059] Such a structure with four known points / shapes is as Figure 1 shown. Figure 1 One of the 15 possible tetrahedrons that can be constructed is shown. Although the previous discussion focused on the use of this method with a hexapod configuration having six fiducial markers, it should be noted that any four or more fiducial markers can be used for any type of 3D structure (e.g., a fixture structure), provided that the three-dimensional distances between these markers are known. Specifically, how to use any four markers (possibly a total of six) to accurately determine the spatial relationship between a fixture structure and a two-dimensional radiographic image will be shown below.

[0060] For example, an exemplary hexapod configuration has a 6×6 hexapod mechanical structure, and a simpler 3×3 hexapod structure is nested in its AMDT SixFix system. In this SixFix system, pairs of strut spherical joints (pairs on each ring) have additional spheres, and the positional relationship of these spheres with respect to the pairs of spherical joints is known. These spheres are radiopaque and are called fiducial markers, and their shadows become artifacts in a 2D radiograph. Other forms can be envisioned, such as but not limited to sphere types, where both the position and a certain degree of orientation can be determined based on a singular form, and thus the properties of the shadow image can be used.

[0061] To construct a 3×3 hexapod structure nested within a 6×6 mechanical structure, the spatial relationships between each of the rings must be determined. The base ring can be considered as a reference system and the platform ring as a moving reference. Knowing the positions of the spherical joints between the struts and the rings, as well as the strut lengths, the spatial relationship between the base and the platform rings can be determined. The spatial relationship between the base and the platform rings can be determined by returning the forward kinematic solution of the transformation matrix, which is an augmented representation of the position and orientation of the platform ring relative to the base ring and the reference system. Once such a transformation is obtained, the positions of the fiducial markers relative to each ring and in terms of the base ring and the reference system can be obtained from it. This allows the construction of a 3×3 hexapod structure consisting of 20 triangular inner and outer surfaces. It should also be noted that any triangular face of the 3×3 hexapod structure can be considered as a fiducial reference system, and any one of the remaining triangular faces can be considered as a platform with a transformation relative to the base face, which is also a simple problem to be determined.

[0062] To properly characterize any given fiducial reference system, the fiducial shadows found in the 2D radiograph should be associated with their respective positions on the 3×3 structure. To facilitate such an association, the method, system, and associated computer-readable product use at least four (possibly six) radiopaque fiducial markers, at least one of which has a different form than the others, typically the at least one radiopaque fiducial marker has a smaller diameter in the case of a spherical marker, although for example a single larger fiducial could equally be used. This “different” fiducial marker is typically oriented at known clinically relevant positions, such as on the base ring or the upper ring, and is in the most anterior position when mounted on the patient's body. For example, when not all fiducial markers can be discerned in the two-dimensional radiograph, this potential preferential orientation can facilitate the identification of the fiducial markers. For example, one or more fiducial markers can be obscured by other radiopaque elements of the construction. It should be noted that for all or at least four fiducial markers where the “different” marker is identified, no preferential orientation is required, and in fact, regardless of the preferential orientation of the “different” fiducial marker, the correct association of the fiducial shadows and their respective positions can be made within the fixator construction.

[0063] In some embodiments, the method, system, and related computer-readable product can associate fiducial shadows found in a 2D radiograph with their respective positions on a 3×3 construct by grouping the fiducial shadows into upper and lower (e.g., superior and inferior) sets of one, two, or three recognizable artifacts. The shadows can then be sorted into foreground / background order, e.g., based on the respective sizes (magnification factors) of the identified fiducial shadows. In the case of spherical fiducial markers, the small diameter (or average diameter, or e.g., the area identifying the shadow) of the ellipse can be used to sort the fiducial shadows into foreground / background order. The shadows can then be sorted in left-to-right order (e.g., from the inside out or from the front to the back). The absolute magnification of the fiducial shadows can then be determined, evaluated, and / or compared to identify outliers or "different" fiducial shadows. For example, the absolute size of a "different" fiducial shadow may not be consistent with factors attributable to differences in magnification factors for foreground / background. Using the perceived position of a "different" or outlier fiducial shadow relative to other identified fiducial shadows, the fiducial shadow can be annotated to relate to an annotated fiducial position on the construct (e.g., the fixator construct). If no "different" fiducial shadow is identified, the sorting may default to returning an assumption using a preferred orientation, and a list of possible numbering schemes can be sorted based on their compliance with the preferred orientation. For example, such a list of possibilities can be evaluated based on how well it matches a known construct.

[0064] By associating fiducial shadows with their respective three-dimensional positions on a construct (e.g., either in an absolute manner or in terms of the probability levels of multiple possibilities), the method, system, and related computer-readable product can determine the spatial relationship between a 2D radiograph and the construct. The method of characterizing this spatial relationship can involve determining the focus of the X-ray source relative to the 2D radiograph. The method, system, and related computer-readable product can use any four points with known mutual spacing distances to determine the focus of the X-ray source relative to the 2D radiograph. For example, a 3×3 hexapod construct (or other construct) can be divided into fifteen different sets of four vertices, where each set forms a tetrahedron. Any one of these tetrahedrons is sufficient for the method, system, and related computer-readable product to determine the focus of the X-ray source relative to the 2D radiograph. Thus, the construct can include only four fiducials. In some embodiments, the method, system, and related computer-readable product can average multiple tetrahedrons to increase the accuracy of focus determination.

[0065] In some embodiments, the method, system, and associated computer-readable product may characterize the focal position by establishing an orthogonal coordinate system. The orthogonal coordinate system may be established using a two-dimensional radiographic image as one of the three planes of the coordinate system. Its origin may be arbitrary, and for the purposes of discussion / disclosure, it will be assumed herein that the origin is located at the center of the image. The alignment of the axes is also arbitrary, but again for the purposes of discussion / disclosure, it is assumed that the x-axis is along the horizontal direction of the two-dimensional radiograph, with the positive direction to the right, the y-axis is along the vertical direction of the two-dimensional radiograph, with the positive direction upward, and the z-axis is perpendicular to the plane of the two-dimensional radiograph, with the positive direction out of the image and towards the observer.

[0066] The orthogonal coordinate system may be established by assuming that the focal position of the x-ray source is above the two-dimensional radiographic image in the positive z-direction, and that the constructed body is between the focal position and the two-dimensional radiographic image. It should be noted that if these assumptions do not hold, the complete shadows of the fiducial markers representing the vertices of the tetrahedron within the two-dimensional radiographic image may not be displayed / included / available.

[0067] In some embodiments, as Figure 2 shown, using a focal model of a single tetrahedron may include using any tetrahedron composed of vertices a, b, c, d, where the diameters of the fiducial markers located at a, b, c, and d and the corresponding distances between a-b, a-c, a-d, b-c, b-d, and c-d are known. The focal point FP(x, y, z) is shown as four rays (green columns) emanating from the focal point that intersect the fiducial points a, b, c, and d of the tetrahedron and casting shadows A, B, C, and D on the image plane. Due to the inclination characteristics of the rays with respect to the image plane, these shadows are generally elliptical in nature. It should be noted that the minor diameter of the elliptical shadow is a function of the magnification factor of the fiducial to which the shadow is projected. This allows determination of the position along the ray FP-A where the fiducial marker a must be located. This method can also be used for all other shadows B, C, and D and their associated fiducial points b, c, and d. Such determination may include determining the (x, y) center of each elliptical shadow A, B, C, and D with respect to the image coordinate system, and their respective minor diameters. The minor diameter of the shadow divided by the known diameter of the associated fiducial marker can be used as the magnification coefficients MA, MB, MC, and MD.

[0068] Once the focal model is constructed, the method, system, and associated computer-readable product may constrain the model based on known characteristics (e.g., algebraic constraints). For example, the method, system, and associated computer-readable product may construct a cost function that can be used for numerical optimization to return the three-dimensional focal point. An example of such a cost function is shown in Figure 2 as Figure 2As shown, a tetrahedral cost function can be used, which includes known relationships between reference positions a, b, c, and d and their associated shadows A, B, C, and D, and these known relationships are functions of FPxyz and their respective magnifications MA, MB, MC, and MD. The method, system, and associated computer-readable product can include these relationships of the intervening distances between the known a, b, c, and d (contained in Dist = [ab ac ad bc bd cd]). The method, system, and associated computer-readable product can solve such a system of equations to avoid multiple solutions (e.g., mirror equivalence) and / or end up with local minimum solutions that are not optimal solutions. For example, in some embodiments, the method, system, and associated computer-readable product can avoid such scenarios by simplifying the search topography in a series of steps, thereby restricting or specifying certain unknowns for a given condition. For example, solving a system with three unknowns is volumetric, two unknowns are planar, and a single unknown is a one-dimensional curve. Thus, the method, system, and associated computer-readable product can narrow the search domain.

[0069] In some such embodiments, the method, system, and associated computer-readable product can recognize that a particular configuration can be fully constrained or immovable given its relationships. For example, as Figure 1 and shown in FIG. 3, the method, system, and associated computer-readable product can recognize that, given the relationships described in Figure 3A , such a configuration is fully constrained or immovable. However, it should be noted that this volume optimization may be affected by one or more of the pitfalls already mentioned. The method, system, and associated computer-readable product can thus utilize fewer constraints (e.g., remove certain constraints), such as modeling / observing the behavior of a simplified structure. For example, as Figure 3B shown, the method, system, and associated computer-readable product can remove the single ray FPxyz-D and the four reference-reference distance constraints, leaving only [ab ac] to obtain a simplified cost function. As Figure 4 shown, the method, system, and associated computer-readable product can thus utilize or form a tripod, depicting a planar curve. In some embodiments, the method, system, and associated computer-readable product can represent the plane in which the curve lies as the ABC plane, which is perpendicular to the image plane.

[0070] In some embodiments, the method, system, and associated computer-readable product can determine the plane of the tripod or its tetrahedron. For example, to determine the ABC plane, the method, system, and associated computer-readable product can lay the tripod down or transpose the tripod into the image plane, as Figure 5A and 5BThe stages or steps 1, 2, 3, 4 shown. In some embodiments, the method, system, and associated computer-readable product may solve for the FPxy of such a configuration for the z = 0 condition for a planar search, e.g., not a volume search, as Figure 15 shown. In some embodiments, the method, system, and associated computer-readable product may determine or identify two solutions where the curve will intersect the image plane, which will be used to formulate an equation for the line depicting the intersection of the image plane and the image ABC plane. Figure 15 Shows the tripod and the two intersections of the focus FP with the image plane for a specific size example.

[0071] In some embodiments, the method, system, and associated computer-readable product may construct four different tripods for a given tetrahedron using different base positions such as ABC, ABD, ACD, and BCD (see e.g., Figure 1 ). When the tripod is laid down or transposed into the image plane, each tetrahedron can similarly draw a planar curve in its respective plane, all of these planes being perpendicular to the image plane. Figure 7 Shows a second case of such processing for the image plane ABD. As Figure 7 shown, shows the planar curve defining the ABD plane. As Figure 8A and 8B shown, the normals of the image plane views of the planes ABC and ABD, the intersection of the two planes can coincide with FP(x, y). It should also be noted that all six combinations of the intersection planes can also produce the same FP(x, y). For example, small errors in the measurements can make these values slightly different, so the method, system, and associated computer-readable product can utilize the average of all 6 possible intersections to reduce such errors. In some embodiments, the method, system, and associated computer-readable product can utilize statistical operations / analysis to cancel out intersections that are outliers, e.g., the remaining intersections can be averaged.

[0072] In some embodiments, the method, system, and associated computer-readable product may utilize known / determined x and y coordinates (e.g., via a cost function as Figure 9 shown) to determine the z coordinate of the focus FP. This method can include advantageous optimizations since there is only one unknown z.

[0073] Once the optimal focus has been determined for a given two-dimensional radiograph, the method, system, and associated computer-readable product can determine the spatial relationship between the two-dimensional radiograph and the fixture configuration. For example, the actual fiducial positions can be located using the above solution along the vectors from the focus to the fiducial shadow positions. These positions can then be transformed into three-dimensional image coordinates, thereby representing each fiducial position relative to the two-dimensional radiographic image.

[0074] In some embodiments, the method, system, and associated computer-readable product can use dimensional image coordinates to define the actual fiducial position vectors between these fiducial positions. In some embodiments, the method, system, and associated computer-readable product can determine the vector cross product between appropriate pairs of these vectors for the construction of an orthogonal coordinate system for any set of three discrete fiducials. Since each of these coordinate systems is with respect to a two-dimensional radiograph, the method, system, and associated computer-readable product can utilize any one of them as a basis for any other coordinate system constructed using a set of any fiducial positions. Additionally, the method, system, and associated computer-readable product can invert any one of these synthesized coordinate systems to develop a coordinate transformation for the two-dimensional radiograph with respect to any coordinate system representing the fixator construct.

[0075] In some embodiments, the method, system, and associated computer-readable product can utilize multiple images (if available) to determine the relationship of each image to the construct. Since the construct is a static known entity in the multiple images, the method, system, and associated computer-readable product can determine the spatial relationship between the multiple images such that any further characterization of artifacts common to the multiple images can be accurately performed in three-dimensional space with respect to the construct.

[0076] The following will be combined with Figure 10 , to illustrate an additional method, system, and associated computer program product for determining the actual position and orientation of a known set of objects in a projected three-dimensional space, where the projected three-dimensional space is above a two-dimensional radiographic space.

[0077] Referring to Figure 10 , there is shown an exemplary external deformity correction device, called a hexapod construct, consisting of a base and a platform disposed in space, having six attached spherical radiopaque fiducial markers to serve as known shapes A, B, C, D, E, F, and the rope distances AB, BC, CA and DE, EF, and FD are all known. A, B, and C are further connected to D, E, and F by a set of six dashed lines, and the length of each dashed line is known. Figure 10 There is shown a so-called three-by-three (3×3) configuration, referring to three coincident centers of the spheres on the base and the platform. In this example, the fiducial marker represented by A is selected to be smaller than all the rest of the fiducial markers of the same size. This is done to distinguish the base from the platform and the rotation of the base in the image space.

[0078] In some embodiments, the method and system can utilize typical radiographs and can locate and evaluate the size and position of the shadow of fiducial markers within the radiograph. The advantage of using spherical fiducial markers is that spherical fiducial markers will always project an elliptical shadow. In some embodiments, the method and system can utilize the minor axis dimension related to the actual diameter, which can be related to the relative distance between the image and the x-ray focus, and the height along the direction of the vector where the actual fiducial marker is located. In some embodiments, the method and system can utilize the image resolution to determine the initial image scale and relative size of the shadow artifact relative to its actual object. As Figure 10 shown, the focus FP(xyz)O is defined as an arbitrary point floating in the space above the image. Thus, in some embodiments, the method and system can use closed-loop vectors to define multiple constraints, such as B0A->A->B->B1A->BOA, P1A->FP(xyz)->P2A->P1A, B2A->C->E->P1A->B2A, as Figure 10 shown. It should be noted that by using multiple closed-loop vectors passing through the origin (as shown by the green loop), the problem has been fully constrained, but this requires measuring the minor diameter of the shadow relative to the actual diameter of the spherical object. Due to resolution and scattering limitations, this may result in errors. Therefore, more loops can be used to statistically improve the results - and many closed-loop vectors can be used. It has been determined that in addition to the four vector loop combinations (red) for each of the base and the platform, the four vector loops (blue) between the base and the platform, and the three vector loops (green) that utilize the relative size between the image and the focus are sufficient.

[0079] Figure 13Another method is shown, in which the rays of the displayed image are emitted from an arbitrarily selected focus FP(x, y, z) to a set of shadow positions displayed in the image / image plane (e.g., formed by fiducials of an orthopaedic construct of known, e.g., hexapod configuration). Two triangles of known dimensions (upper and lower triangles) (e.g., whose points correspond to the positions of the fiducials of the known construct) are used / located along 3 rays respectively (i.e., each point / corner of the triangle has a ray). The triangles are shown as two discrete items, but it should be noted that these triangles can share a vertex or an edge in cases where only 5 or 4 shadow artefacts can be found in the image. It should also be noted that any combination of triangles to an arbitrary focus FP(x, y, z) can be constructed using the available shadow items and their associated rays and can equally be exploited. To determine the correct orientation of the triangles, the system and method can evaluate the relative magnification of the shadow items to determine whether the outliers are ahead or behind (i.e., closer or nearest, further or furthest from the focus FP(x, y, z)). For a given arbitrary focus, the system and method can determine a cost function which is the sum of the errors between the known spacing of the vertices and the calculated spacing of the vertices (for a given arbitrary focus FP(x, y, z)). Such a cost function can be used by a numerical solver to determine the best compatible FP(x, y, z) for a given construct and the shadow projected by it in terms of projective geometry.

[0080] After determining at least four of the node positions A, B, C, D, E, F and O in the image space, in some embodiments, the method and system can construct a suitable coordinate transformation for a known three-dimensional object depicted by a set of spherical fiducials. In some such embodiments, the method and system can construct a suitable coordinate transformation by determining the cross product of a pair of suitable vectors, e.g., ABxAC will yield a vector perpendicular to both AB and AC, with its origin at A. The resulting vector can then be crossed with one of the previous vectors AB or AC to determine an orthogonal coordinate system. In this case, the coordinate system describes the base in the image space defined by ABC. These methods and systems can use the same cross product method for multiple images, resulting in multiple coordinate systems, all of which describe the same known three-dimensional object in a larger patient space.

[0081] Thus, the method and system can utilize how the known object is located in two completely different spaces to determine their relationship and can thus determine the coordinate transformation between the different spaces using matrix operations (e.g., inversion and multiplication). This ability of the method and system eliminates the need to provide orthogonal images rotated about a common axis to determine the true three-dimensional condition to be corrected.

[0082] As Figure 14As shown, the method and system of the present application can execute method 100, which includes the following steps. At 102, a 2D X-ray image (or its digital version) is obtained digitally, and the 2D X-ray image shows a 3D structure of a known configuration (such as shape, size, etc.), such that four discrete recognizable fiducials / points of the structure are identified (digitally or manually) within the image (such as the shadow centers of known spherical fiducials / elements of the 3D structure), where the spatial relationship between each point in the structure is known / input. Then, at 104, method 100 may include generating four vectors to an arbitrary focus above the 2D image plane of the image using the four discrete points identified as a basis. Then, at 106, method 100 may include establishing a cost function digitally (e.g., by one or more vector loops, three-dimensional reduction to 2D space, or the sliding triangle method as described above), and the cost function evaluates the suitability of the arbitrary focus for the known separation distance of the 3D structure. Then, at 108, method 100 may include using the cost function digitally as a discriminant to determine the compatible focus FP(x, y, z). It should be noted that the specific optimization numerical solution steps may vary depending on the method used. If the optimized focus FP(x, y, z) and the known discrete points are within the image, then at 110, method 100 may include determining digitally the positions of the known elements along the vectors (starting from the focus FP(x, y, z)). This position is / corresponds to the 3D coordinates of the elements of the 3D structure, thereby placing a tetrahedron in the image space. Then, at 112, method 100 may include digitally establishing two vectors with a common vertex using any two sides of any one of the four triangular faces of the tetrahedron. Then, at 114, method 100 may include digitally establishing a mutual normal, such as by taking the cross product in a preferred order of the two vectors, and digitally establishing an orthogonal coordinate system in the image space, such as by taking the cross product in a preferred order between the mutual normal and the preferred choice of the two original vectors. In some embodiments, then at 116, method 100 may include using the known relationships between the faces of the known 3D tetrahedron to digitally construct a transformation (e.g., a matrix) between the orthogonal coordinate system from 114 and any other face vertex combination.

[0083] Then at 118, method 100 may include: if there are more than 4 discrete points available in the image, repeating 102-114 (and possibly 116) for all combinations of 4 discrete points within a larger discrete point grouping. Then, at 120, method 100 may include numerically generating a composite coordinate transformation (e.g., a matrix) that represents a known 3D structure in the image space, for example, by averaging all (e.g., those generated in steps 116 and / or 118) equivalent transformations. Then at 122, method 100 may include numerically generating a 2D image transformation for the known 3D structure by inversion of the coordinate transformation of 120.

[0084] Then, method 100 may include repeating 102-122 for each 2D image obtained from a known 3D structure (e.g., a plurality of images, such as two or more images) and numerically constructing a 3D representation of the plurality of images for that 3D structure. Then, method 100 may include numerically establishing an intersection between planes and planes, or between a plane and a vector, or the closest points between vectors, for the 3D structure using the 3D representation of the 2D images, representing aspects of the anatomical structure emanating from the focal point FP(x, y, z) of each respective 2D image, to determine the relationship between the known 3D structure and the anatomical structure of interest.

[0085] For those of ordinary skill in the art, it is apparent that the present application provides significant improvements in the fields of external fixation devices and computer modeling of anatomical structures (including the fields of hexapod constructs and bone segment modeling). In addition, the present application provides significant improvements in the field of radiographic imaging (including the field of radiographic image distortion correction). The present application also provides significant improvements in the field of external fixation device adjustment prescription determination (including the field of hexapod construct adjustment prescription).

[0086] Those of ordinary skill in the art will recognize that aspects of the present application may be embodied in a system, method, and / or computer program product. In some embodiments, aspects of the present application may be embodied entirely in hardware, entirely in software (e.g., embodied in firmware, resident software, microcode, etc.), or a combination of software and hardware aspects, which may generally be referred to in the present application as a "system" and include circuits and / or modules.

[0087] Figure 15 An example of a computer system incorporating and using one or more aspects of the present application is shown. Computer system 500 may be a computer system of an article manufacturing and / or repair facility, such as a computer system for an additive article, and / or a computer system for generating data (data used by an AM device or equipment to manufacture an article). Figure 15The computer system 500 can be adapted to store and / or execute program code, such as the program code for performing the above processes, and includes at least one processor 502 that is directly or indirectly coupled to a memory 505 via a bus 520. In operation, the processor 502 can obtain instructions to be executed by the processor from the memory 505. The memory 505 can include local memory, mass storage, and cache memory used during the actual execution of the program code. The cache memory provides temporary storage of at least some of the program code to reduce the number of times code must be retrieved from mass storage during program code execution. A non-limiting list of examples of the memory 505 includes hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM, or flash memory), optical fibers, portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing. The memory 505 can include an operating system 505 and one or more computer programs 506, such as one or more programs for performing the various aspects described in the present application, such as implementing adjustments to the digital layout of a circuit design.

[0088] Input / output (I / O) devices 512, 515 (such as peripheral devices) can be directly or coupled to the system via an I / O controller 510. A network adapter 508 can also be coupled to the system to enable the computer system to start coupling with other computer systems via a private or public network. Modems, cable modems, and Ethernet cards are just a few of the currently available types of network adapters 508. In one example, the network adapter 508 helps obtain data from a remote source to facilitate the various aspects of the present application.

[0089] The computer system 500 can be coupled to a memory 516 having one or more databases (for example, a non-volatile storage area, such as a disk drive, an optical disc drive, a tape drive, etc.). The storage device 516 can include an internal storage device or an additional or network-accessible storage device. The computer program in the memory 516 can be loaded into the memory 505 and executed by the processor 502.

[0090] The computer system 500 can include fewer components than shown, additional components not shown in the present application, or some combination of the shown and additional components. The computer system 500 can include any computing device, such as a mainframe, a server, a personal computer, a workstation, a laptop computer, a handheld computer, a smartphone, a desktop computer, or other mobile devices, telephone devices, network devices, virtualization devices, storage controllers, etc.

[0091] In addition, the above process may be performed by multiple computer systems 500 that work together as part of a computing environment.

[0092] In some embodiments, aspects of the present application may take the form of a computer program product embodied in a computer-readable medium. Computer-readable program code may be included on the computer-readable medium. A variety of computer-readable media or combinations thereof may be utilized. For example, the computer-readable medium may include a computer-readable storage medium, examples of which include (but are not limited to) one or more electronic, magnetic, optical, or semiconductor systems, devices, or apparatuses, or any suitable combination of the foregoing. Computer-readable storage media include, for example: an electrical connection having one or more wires, a portable computer floppy disk, a hard disk, or a mass storage device, a random access memory (RAM), a read-only memory (ROM), and / or an erasable programmable read-only memory, such as an EPROM or flash memory, an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device (including a magnetic tape device), or any suitable combination of the foregoing. A computer-readable storage medium is defined to include tangible media that can contain or store program code for use by or in connection with an instruction execution system, apparatus, or device, such as a processor. Thus, program code stored in or on a computer-readable medium results in an article of manufacture that includes the program code (e.g., a "computer program product").

[0093] See Figure 16 , in one example, the computer program product 600 includes, for example, one or more computer-readable media 602 having stored thereon computer-readable program code means or logic 604 to provide and facilitate one or more aspects of the present application.

[0094] Program code embodied on or stored in a computer-readable medium can be obtained and executed by a computer system (a computer, a computer system, etc., including its components) and / or other devices to cause the computer system, its components, and / or other devices to operate / function in a particular manner. The program code can be transmitted using any suitable medium, including (but not limited to) wireless, wired, fiber optic, and / or radio frequency. The program code for performing the operations to execute, implement, or facilitate aspects of the present application can be written in one or more programming languages. In some embodiments, the programming languages include object-oriented and / or procedural programming languages, such as C, C++, C#, Java, etc. The program code can be executed entirely on the user's computer, entirely remote from the user's computer, or partly on the user's computer and partly on a remote computer. In some embodiments, the user's computer and the remote computer communicate via a network (e.g., a Local Area Network (LAN) or a Wide Area Network (WAN)) and / or via an external computer (e.g., using the Internet via an Internet service provider).

[0095] In one example, the program code includes one or more program instructions for execution by one or more processors. The computer program instructions can be provided to one or more processors (e.g., one or more computer systems) to produce a machine such that the program instructions, when executed by the one or more processors, execute, implement, or facilitate aspects of the present application, such as the actions or functions described in the flowcharts and / or block diagrams of the present application. Thus, in some embodiments, each block or combination of blocks of the flowcharts and / or block diagrams illustrated and described in the present application can be implemented by computer program instructions.

[0096] The flowcharts and block diagrams shown and described with reference to the accompanying drawings illustrate the architecture, functionality, and operation of possible embodiments of a system, method, and / or computer program product according to aspects of the present application. Accordingly, these flowcharts and / or block diagrams can be a method, apparatus (system), and / or computer program product according to aspects of the present application.

[0097] In some embodiments, as described above, each block in a flowchart or block diagram can represent a module, segment, or portion of code that includes one or more executable instructions for implementing specific behaviors and / or logical functions of the block. Those of ordinary skill in the art will understand that the behaviors / functions specified or performed by a block can occur in a different order than shown and / or described, or can occur simultaneously with one or more other blocks, or partially / fully concurrently. Two consecutive blocks can actually be executed substantially simultaneously, or sometimes in the reverse order. Additionally, each block of the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, can be implemented entirely by a dedicated hardware-based system, or in combination with computer instructions to perform the behaviors / functions specified by the block or the entire block diagram or flowchart.

[0098] It should be understood that the above is intended to illustrate the present application and not to limit it. Without departing from the general spirit and scope of the present application as defined by the following claims and their equivalents, those of ordinary skill in the art can make various changes and modifications to the present application. For example, the above embodiments (and / or aspects thereof) can be used in combination with each other. Additionally, many modifications can be made to adapt a particular situation or material to the teachings of various embodiments without departing from their scope. Although the dimensions and types of materials described in the present application are intended to define the parameters of different embodiments, they are in no way restrictive but merely exemplary. After reading the above description, many other embodiments will be apparent to those skilled in the art. Therefore, the scope of each embodiment should be determined with reference to the appended claims and the full scope of the equivalents encompassed by those claims.

[0099] The terms used in this application are only for describing specific embodiments and are not intended to limit the application. The singular forms “a,” “an,” and “the” as used herein are also intended to include the plural forms unless the context clearly indicates otherwise. It will be further understood that the terms “comprises” (in any form of “comprises,” such as “comprises” and “comprising”), “has” (in any form of “has,” such as “has” and “having”), “includes” (in any form of “includes,” such as “includes” and “including”), “contains” (in any form of “contains,” such as “contains” and “containing”), and any other grammatical variants thereof are open-ended linking verbs. Thus, a method or thing that “has,” “includes,” or “contains” one or more steps or elements has those one or more steps or elements but is not limited to only having those one or more steps or elements. Similarly, the steps or elements of a method or thing that “includes,” “has,” or “contains” one or more features have those one or more features but are not limited to only having those one or more features.

[0100] As used herein, the terms "comprising," "having," "including," and other grammatical variations thereof include the terms "consisting of" and "consisting essentially of."

[0101] The phrase "consisting essentially of" or its grammatical variations, when used herein, is regarded as specifying a feature, integer, step, or component, but does not exclude the addition of one or more additional features, integers, steps, components, or groups thereof, provided that the additional features, integers, steps, components, or groups thereof do not materially alter the basic and novel characteristics of the claimed group or method.

[0102] All documents cited in this application are hereby incorporated by reference into this application as if each individual document had been specifically and individually indicated to be incorporated by reference into this application as if fully set forth.

[0103] The subject matter incorporated by reference should not be considered as a substitute for any claim limitation unless expressly stated otherwise.

[0104] In the case where one or more ranges are mentioned in this specification, each range is intended as an abridged version for presenting information, where the range is understood to include each discrete point within the range as if fully set forth in this application.

[0105] Although several aspects and embodiments of this application have been described and illustrated, those skilled in the art can effect alternative aspects and embodiments to achieve the same purpose. Accordingly, this application and the appended claims are intended to cover all such further and alternative aspects and embodiments that fall within the spirit and scope of this application.

Claims

1. A method for determining the actual position and orientation of a known three-dimensional (3D) configuration of a shaping fixation device using at least four discrete shapes, the at least four discrete shapes being formed by fiducials of the configuration shown in a two-dimensional (2D) image of the configuration, the method comprising: Identifying at least four fiducial shadows in the 2D image corresponding to the fiducials of the configuration; Associating the identified at least four fiducial shadows with their respective positions in the configuration; And Determining the spatial relationship between the 2D image and the configuration, including: Determining the 2D image source relative to the focus of the 2D image by a predetermined mutual separation distance between the identified at least four fiducial shadows and the corresponding fiducials of the configuration; Locating the actual fiducial positions along vectors from the focus to the fiducial shadow positions; Converting the actual fiducial positions to 3D image coordinates; Defining actual fiducial position vectors between the fiducial positions by the 3D image coordinates; and Constructing a first orthogonal coordinate system for the 2D image for a set of three discrete fiducials by determining the vector cross product between appropriate pairs of position vectors.

2. The method according to claim 1, wherein associating the identified at least four fiducial shadows with their respective positions in the configuration comprises: Identifying the identified at least four fiducial shadows as upper fiducial shadows or lower fiducial shadows; Determining the foreground or background order based on the respective sizes of the identified at least four fiducial shadows; Determining the order of the identified at least four fiducials from left to right or from right to left; And Annotating the identified at least four fiducials to be associated with the positions of the respective annotated fiducials in the configuration.

3. The method according to claim 1, wherein determining the spatial relationship between the 2D image and the configuration further comprises: Inverting the constructed first orthogonal coordinate system or inverting a second constructed orthogonal coordinate system determined by the constructed first orthogonal coordinate system to develop a coordinate transformation for the 2D image relative to any coordinate system representing the configuration.

4. A computer program product comprising: A non-transitory computer-readable storage medium readable by one or more processing circuits and storing instructions executable by one or more processors for performing the method according to any one of claims 1-3.

5. A system for determining the actual position and orientation of a known three-dimensional (3D) configuration of a shaping fixation device using at least four discrete shapes, comprising: A memory; At least one processor communicatively coupled to the memory; And Program instructions executable by one or more processors via the memory to perform the method according to any one of claims 1-3.

6. A method for determining the actual position and orientation of a known set of spherical objects of a shaping fixation device in a projected three-dimensional space located above a two-dimensional radiographic space, the method comprising: Obtaining two or more digital radiographic images of the known set of spherical objects in the projected three-dimensional space located above the two-dimensional radiographic space; And Determining the actual position and orientation of the known set of spherical objects in the projected three-dimensional space above the two-dimensional radiographic space by measuring at least one of the small diameter, average diameter, or area of the elliptical shadow of the spherical objects in the two-dimensional radiographic space of the known set of spherical objects in two or more of the digital radiographs.

7. The method according to claim 6, further comprising constructing a three-dimensional model of the actual position and orientation of the known set of spherical objects in the projected three-dimensional space.

8. The method according to claim 6, wherein determining the actual position and orientation of the known set of spherical objects using the elliptical shadows of the known set of spherical objects in the two-dimensional radiographic space of two or more of the digital radiographs comprises: Using perspective distortion to determine the relative magnification of the images to reconstruct the projected three-dimensional space; Determining the relationship between two or more of the digital radiographs by comparing common spherical objects of the known set of spherical objects in the images; and wherein two or more of the digital radiographs further comprise at least one anatomical structure to be corrected, and constructing a three-dimensional model of the actual position and orientation of the at least one anatomical structure in the projected three-dimensional space.

9. The method according to claim 6, wherein the measurement of at least one of the small diameter, the average diameter, or the area of the elliptical shadow comprises determining the small diameter.

10. A computer program product comprising: A non-transitory computer-readable storage medium readable by one or more processing circuits and storing instructions executable by one or more processors for performing a method of determining the actual position and orientation of a known set of objects of a shaping fixation device using at least four discrete shapes, the at least four discrete shapes formed by the constructed fiducials shown in a constructed two-dimensional 2D image, the method comprising: Identifying at least four fiducial shadows in the 2D image corresponding to the constructed fiducials; Associating the identified at least four fiducial shadows with their respective positions on the construct; and Determining the spatial relationship between the 2D image and the construct, comprising: Determining the focus of the 2D image source relative to the 2D image by a predetermined mutual separation distance between the identified at least four fiducial shadows and the corresponding fiducials of the construct; Establishing the orthogonal coordinate system using the 2D image as one of the three planes of the orthogonal coordinate system; Determining the positions along the focus rays where each of the at least four fiducials must be located; Constraining the model of the at least four fiducials by a cost function based on known features of the construct, the known features not including a ray and four fiducial-fiducial distances, wherein the constraint forms a tripod model that outlines a planar curve in a plane perpendicular to the image plane.

11. The computer program product according to claim 10, wherein determining the focus of the image source relative to the 2D image further comprises: Reconfigure the tripod model such that a first plane formed by three of the at least four fiducials of a first group lies on the image plane; Determine a first equation for a first line that depicts the intersection of the image plane and the first plane; Reconfigure the tripod model such that a second plane lies along the image plane, the second plane being formed by three of the at least four fiducials of a second group; Determine a second equation for a first line that depicts the intersection of the image plane and the second plane; Determine the x and y coordinates of the focus via at least the first line and a second line; and Determine the z coordinate of the focus via the x and y coordinates and a cost function.

12. A system for determining the actual position and orientation of a known three-dimensional (3D) configuration of a shaping fixation device using at least four discrete shapes, the at least four discrete shapes being formed by fiducials of the configuration shown in a two-dimensional (2D) image of the configuration, comprising: A memory; At least one processor in communication with the memory; and Program instructions executable by the one or more processors via the memory to perform a method for determining the actual position and orientation of the known 3D configuration of the shaping fixation device, the method comprising: Obtain the 2D image and identify at least four shadows in the 2D image corresponding to the fiducials of the configuration; Associate the identified at least four shadows with their respective positions in the configuration; and Determine the spatial relationship between the 2D image and the configuration, including: Determine the focus of the 2D image source relative to the 2D image via a predetermined mutual separation distance between the at least four shadows and the corresponding fiducials of the configuration: Locate the actual fiducial positions along vectors from the focus to the shadow positions; Convert the actual fiducial positions to 3D image coordinates; Define actual fiducial position vectors between the fiducial positions via the 3D image coordinates; and Construct a first orthogonal coordinate system for the set of fiducials in the 2D image by determining the vector cross product between appropriate pairs of position vectors.

13. The system according to claim 12, wherein determining the spatial relationship between the 2D image and the configuration further comprises: Invert the constructed first orthogonal coordinate system to develop a coordinate transformation for the 2D image relative to any coordinate system representing the configuration.

Images