Method and system for registering monoscopic radiographic images and 3D models

Abstract

To provide a system and a method including a method for determining an actual position and pose of a known three-dimensional (3D) structure by utilizing at least four individual shapes formed by a reference of a structure shown in a two-dimensional (2D) image of the structure.SOLUTION: A method for determining an actual position and pose of a known 3D construct includes the steps of: identifying at least four reference shadows in a 2D image corresponding to a reference of the 3D construct; correlating the discovered at least four reference shadows with their respective positions on the 3D construct; determining a spatial relation between the 2D image and the 3D construct by determining a focal point of a source of an image relative to the 2D image via predetermined mutual separation distance between the discovered at least four reference shadows and corresponding reference of the 3D construct; and determining the spatial relation between the 2D image and the 3D construct.SELECTED DRAWING: Figure 14

Description

Cross - reference to related applications

[0001] This application claims the benefit of, and incorporates by reference in its entirety, U.S. Provisional Application No. 62 / 817185, filed on March 12, 2019, entitled "Monoscopic Image Registration Methods and Systems". This application is also related to International PCT Patent Application No. PCT / US2019 / 043326, filed on July 24, 2019, entitled "Methods and Systems of Registering a Radiographic Image and a Three - Dimensional Model of an External Fixation Device", the entire disclosure of which is incorporated by reference herein in its entirety.

Technical Field

[0002] The present disclosure generally relates to image registration that utilizes known three - dimensional (3D) constructs depicted in an image. More specifically, the present disclosure relates to monoscopic image (e.g., radiographic image) and three - dimensional model registration methods and systems that utilize known physical parameters (e.g., separation distances between at least four shapes or points) of a particular 3D construct depicted in an image.

[0003] This disclosure also relates, in general, to systems and methods for deformation analysis using multiple non-orthogonal radiographs. Embodiments of this disclosure relate to the treatment of musculoskeletal conditions, including skeletal fractures. More specifically, methods and systems for fixing and positioning one or more bone segments in desired locations are disclosed. In some embodiments of this disclosure, the methods and systems are used to generate a three-dimensional computer model of a fixation device, a bone segment, and potentially at least one (e.g., at least two) radiographic representations corresponding to the radiographs used to create the model. Through operations on the model, in one embodiment, the desired placement of the bone segment and the operation of the external fixation device are determined quickly and accurately to achieve such desired placement, in relation to the device and / or bone, regardless of the initial configuration of the fixation device or the orientation of the radiographs. The musculoskeletal condition can then be treated by performing the operations necessary to create the desired placement of the bone segment on the corresponding fixation device and bone segment. However, other devices other than external fixation devices may be used with the systems and methods. [Background technology]

[0004] In medicine, the correction of orthopedic deformities typically involves at least a pair of radiographs. These radiographs usually photograph the patient along conventional lines in the front-to-back (AP) and medial-to-outer (ML) directions, or along other orthogonal or known bandage points (or known differences between bandage points). By convention, the AP and ML radiographs are taken or assumed to be orthogonal to each other in patient space (patient space is defined as having the X-axis aligned from right to left, the Y-axis aligned from front to back, and the Z-axis aligned from bottom to top). Measurements are taken within each pair of radiographs, and the deformity axes and points are annotated. These measurements and annotations are then used to reconstruct a true three-dimensional representation of the deformity, allowing for the manipulation of the deformity by some means to correct the condition.

[0005] However, a common problem arises from the uncertainty of radiographic images and their spatial relationships. Radiographic images are not complete images of the artifacts they contain. The relationship between artifacts shown in an image and the objects actually being imaged is one of perspective, where objects closer to the image are magnified less than objects further away. Furthermore, uncertainty regarding the orthogonality between pairs of images makes it difficult to reconstruct a true representation.

[0006] Consequently, these uncertainties in such radiographic images require a means to explain them, as they stem from their actual viewpoint / bandage points.

[0007] Furthermore, in many research fields, it is often desirable to register a two-dimensional (2D) image with a known 3D object. "Registration" means constructing a coordinate transformation that allows the position and pose of the 3D object to be determined within a coordinate system that coincides with the 2D image. For example, if any plane in the 3D coordinate system lies coplanar with the 2D image, then the 3D coordinate system can be considered to coincide with the 2D image. Registering a 3D object within this coordinate system allows for the creation of one or more virtual environments from which the viewer's viewpoint can be determined, and the image and object can be appropriately positioned within that environment.

[0008] In the field of medicine, this is often a critical step in the proper placement or manipulation of implants, surgical instruments, or body tissue structures. In contrast to 3D imaging technologies such as CT and MRI, one of the most common imaging methods is basic X-ray, which has the advantages of low cost and real-time accessibility in the operating environment. It is desirable to be able to register known 3D objects or body structures in relation to the real-time images based on individual images.

[0009] Currently, there are methods for registration using multiple images of specific combinations of 3D structures. However, in these cases, it is necessary to know the spatial relationships between the multiple images with a certain degree of certainty. Some current stereoscopic image guidance systems can achieve this externally and typically rely on known relationships between the pairs of cameras used. Other current methods typically require the acquisition of multiple images, such as front-to-back (AP) and inside-to-outside (ML) radiographs. The relationships between such images are affected by variables inherent in the acquisition of such images, which can lead to errors in 3D registration, as described above.

[0010] As a concrete example, in orthopedics, it is often necessary to correct bone deformities using devices known as external fixators. Such external fixators are available in a variety of configurations, from simple monolateral and pin-to-bar systems to more complex circular structures. To accurately correct such bone deformities, the spatial relationship between the anatomical structure of the bone and the fixator structure must be accurately characterized after the structure is attached to the patient. This characterization process can begin with taking multiple images, such as two or more 2D radiographs or 3D image scans of the fixator structure attached to the bone. 2D radiographs are the primary means of obtaining such characteristics because they are simple and inexpensive. Therefore, it is desirable to accurately register each of the multiple 2D images based on the external fixator or other known 3D entities to accurately position other body structures relative to the known 3D entities.

[0011] While certain aspects of the prior art have been discussed to facilitate the disclosure of the applicant's invention, the applicant has not denied these technical aspects and believes that their invention may encompass one or more of these prior art aspects.

[0012] Where this specification references or discusses any document, action, or knowledge item, such reference or discussion does not acknowledge that the document, action, or knowledge item, or any combination thereof, was generally available, generally known, part of common knowledge, or otherwise constitutes prior art under applicable legal provisions as of the priority date; or is known to be related to an attempt to solve a problem to which this specification pertains. [Overview of the project]

[0013] The present invention can address one or more of the problems and defects of the art discussed above. However, it is intended that the present invention may prove useful in addressing other problems and defects in many other arts. Therefore, the claimed invention should not necessarily be construed as being limited to addressing any of the specific problems or defects discussed herein.

[0014] This disclosure generally relates to image registration methods and systems that utilize known three-dimensional (3D) structures depicted in images. More specifically, this disclosure relates to monoscopic images (e.g., radiographic images) and 3D model registration methods and systems that utilize known physical parameters (e.g., separation distances between at least four shapes or points) of a particular 3D structure depicted in an image.

[0015] This disclosure also relates to systems and methods for deformation analysis using multiple radiographs (such as non-orthogonal radiographs) taken from unknown (or inaccurate or misidentified) bandage points in general. In some embodiments, the system and method individually register each 2D image (having a known 3D construct and / or reference shape (and size)) and use each registered image to construct a 3D model as a correction analysis and prescription decision for part of the deformation and / or deformation.

[0016] Some such embodiments of the present disclosure relate to the treatment of musculoskeletal conditions, including skeletal fractures. More specifically, methods and systems for fixing and positioning one or more bone segments in desired locations are disclosed. In some embodiments of the present disclosure, the methods and systems are used to generate a fixation device, a bone segment, and a three-dimensional computer model of potentially at least one (e.g., at least two) radiographic representations corresponding to radiographic images used to create the model. Through operations on the model, in one embodiment, the desired placement of the bone segment and the operation of the external fixation device are determined quickly and accurately to achieve such desired placement, regardless of the initial configuration of the fixation device or the orientation / bandage point of the radiographic image, in relation to the device and / or bone. The musculoskeletal condition can then be treated by performing the operations necessary to create the desired placement of the bone segment on the corresponding fixation device and bone segment. However, other devices other than external fixation devices may be used with the systems and methods.

[0017] In some embodiments, the disclosure provides a method and associated system that utilizes the planar positions and characteristics of four individual shapes or points contained within a particular two-dimensional radiograph to correlate them with four individual spatial coordinates contained within a stationary structure (or structure of another known object). Using this information, the method and associated system obtain precise spatial relationships between the stationary structure (or structure of another known object) and each of the individual radiographs.

[0018] In some embodiments, a radiographic image includes the shadow of a three-dimensional object posed and positioned on the image (e.g., film) at the time of acquisition. The apparent position and orientation of the X-ray source relative to the image casting the shadow are unknown. In an ideal world, its focal point would be a point light source positioned at an infinite distance from the center of the image itself. In an ideal representation, the shadow would be a true two-dimensional projection of the actual three-dimensional object. If we have two ideal representations and know that they are orthogonal to a common axis, then the two sets of two-dimensional data could be directly used to accurately reconstruct the three-dimensional object and its position and pose in space. However, this is generally impossible, given that the current state-of-the-art techniques of radiography, including plain film radiographs, introduce projection distortion. Furthermore, the possibility of images being truly orthogonal to the trajectory of the X-ray source and orthogonal to each other around a common axis is also negligible, considering all the variables required to acquire these images in a real X-ray machine where the actual patient is instructed to lie / pose in a prescribed manner.

[0019] The system and method of this disclosure utilize two main sources of error, focal position and pose, as well as patient orientation, to draw many conclusions for ultimately correcting unintended rotations between a pair of radiographic images (e.g., rotation from orthogonal arrangement) and construct a true three-dimensional model of objects within the radiographic images.

[0020] The system and method can account for projection distortion by determining the radiopaque objects and the shadows they cast on the radiographic image. As is known, the edges of such artifacts are relatively sharp. The system and method can take advantage of these relatively sharp edges. The system and method can take advantage of the shape edges and conclude that the source of the radiographic image (i.e., X-rays) casting the shadow is actually a posed point located somewhere on the shadow image. The system and method can also conclude that the object lies on a vector describing a line between the center of the shadow or point of a particular object artifact and the focal point of the X-ray source, as shown in Figure 10. The system and method can further conclude that, given the shape of the artifact and its actual size, it can determine the relative distance between the shadow image and the actual artifact, and the distance between the shadow image and the X-ray source. However, this alone is not sufficient to determine the position and pose of the X-ray source. Therefore, the system and method can take advantage of a number of known objects whose shadows exist as artifacts in the radiographic image, and because the relative shape, size, and relationship of the objects to other objects are known, it can determine the apparent focal position of the apparatus or the position of the X-ray source relative to the image.

[0021] The system and method can determine the position and pose of a 3D collection of known objects in radiographic image space using multiple closed vector loops passing through the shadow center, object center, and focal point, as shown in Figure 10. Once the multiple closed vector loops are determined, the system and method can define coordinate transformations for the collection of known 3D objects in shadow image space (i.e., determining the row, column, and height dimensions), as shown in Figure 10. Once the coordinate system is determined, the system and method can determine coordinate transformations between any pair of images in the multiple images by utilizing the collection of known 3D objects in each of the multiple radiographic images, using a consistent approach across each image. The system and method can compensate for non-orthogonal or otherwise rotated pairs of images when constructing the true 3D position and pose of the 3D objects, and thus accurately describe any other annotations or measurements made in the radiographic images.

[0022] In another aspect, the disclosure provides a method and system for utilizing a known three-dimensional collection of objects, the shadows of which are cast into a two-dimensional X-ray radiation space, and determining the actual position and pose of the known collection of objects in three-dimensional space of a projected computer model above the two-dimensional radiographic space.

[0023] In some embodiments, the method and system can utilize projection distortion to determine relative magnification and assist in the reconstruction of a three-dimensional projected space. In some embodiments, the method and system can determine the relationships between multiple radiographic images through the analysis of known common objects. In some such embodiments, the method and system can reconstruct a real three-dimensional conditional model with a modified relative spatial arrangement.

[0024] In some embodiments, the method and system may include a method for determining the actual position and pose of a known three-dimensional structure by utilizing at least four distinct shapes formed by references of the structure shown in a two-dimensional (2D) image of the structure, the method including: identifying at least four reference shadows in the 2D image corresponding to references of the structure; correlating the at least four found reference shadows with their respective positions on the structure; determining the spatial relationship between the 2D image and the structure by determining the focus of the image source on the 2D image via predetermined mutual separation distances between the at least four found reference shadows and their corresponding references of the structure; and determining the spatial relationship between the 2D image and the structure.

[0025] In some embodiments, the step of correlating at least four discovered reference shadows with their respective positions on a structure includes: identifying the at least four discovered reference shadows as upper or lower reference shadows; determining the foreground or background order of the at least four discovered reference shadows based on their respective sizes; determining the left-to-right or right-to-left order of the at least four discovered references; and annotating the at least four discovered references and correlating them with their respective annotated reference positions on the structure.

[0026] In some embodiments, the step of determining the spatial relationship between a 2D image and a construct includes: identifying an actual reference position along a vector from the focal point to a reference shadow position; transforming the actual reference position into three-dimensional (3D) image coordinates; defining an actual reference position vector between the reference positions via the 3D image coordinates; constructing a first orthogonal coordinate system of a collection of three discrete references with respect to the 2D image by determining the vector cross product between suitable pairs of position vectors; and developing a coordinate transformation of the 2D image with respect to any coordinate system representing the construct by inverting the first constructed orthogonal coordinate system or a second constructed orthogonal coordinate system determined via the first constructed orthogonal coordinate system.

[0027] In some embodiments, the step of determining the spatial relationship between a 2D image and a structure by determining the focus of the source of the image with respect to the 2D image through a predetermined mutual separation distance between at least four reference shadows found and the corresponding references of the structure includes establishing a Cartesian coordinate system that uses the 2D image as one of the three planes of the coordinate system, determining positions along the focal rays where each of the at least four references must exist, constraining a model of the at least four references based on known characteristics of the structure through a cost function, where the known characteristics do not include the distance between one ray and the four references, and the constraint forms a tripod model that traces a planar curve in a plane perpendicular to the image plane, reconstructing the tripod model such that a first plane formed by a first group of three of the at least four references is positioned along the image plane, determining a first equation of a first line representing the intersection of the image plane and the first plane, reconstructing the tripod model such that a second plane formed by a second group of three of the at least four references is positioned along the image plane, determining a second equation of a first line representing the intersection of the image plane and the second plane, determining the x and y coordinates of the focus through at least the first and second lines, and determining the z coordinate of the focus through the x and y coordinates and the cost function.

[0028] In some embodiments, the method and system may include a method for determining the actual positions and poses of a collection of known objects in a projected three-dimensional space above a two-dimensional radiation space, the method including obtaining two or more digital radiographic images of the collection of known objects in the projected three-dimensional space above the two-dimensional radiation space, and using the shadows of the collection of known objects in the two-dimensional radiation space of the two or more digital radiographic images to determine the actual positions and poses of the collection of known objects in the projected three-dimensional space above the two-dimensional radiation space.

[0029] In some embodiments, the method further includes constructing a three-dimensional model of the actual positions and poses of a collection of known objects in the projected three-dimensional space. In some such embodiments, determining the actual positions and poses of the collection of known objects, using the shadows of the collection of known objects in the two-dimensional radiation space of two or more digital radiographic images, includes determining the relative magnification of the images using projection distortion to reconstruct the projected three-dimensional space. In some such embodiments, determining the relationship between two or more digital radiographic images is included via comparison of common objects of the collection of known objects within the images. In some such embodiments, the two or more digital radiographic images further include at least one anatomical structure that requires correction, and further includes constructing a three-dimensional model of the actual position and pose of the at least one anatomical structure in the projected three-dimensional space.

[0030] In some embodiments, the present disclosure provides a computer program product including a computer-readable storage medium readable by one or more processing circuits and storing instructions for execution by one or more processors for performing the above method.

[0031] The present disclosure provides a system including a memory, at least one processor in communication with the memory, and program instructions executable by the one or more processors via the memory for performing the above method.

Brief Description of the Drawings

[0032] The present disclosure will be described in conjunction with the following drawings, which are not necessarily drawn to scale for ease of understanding, and the same reference numerals retain the same or similar element designations and meanings throughout the various drawings.

[0033] [Figure 1]This figure shows an exemplary 3x3 external fixator (e.g., hexapod) structure according to the present disclosure.

[0034] [Figure 2] This figure shows an exemplary focal model using a single tetrahedron according to the present disclosure.

[0035] [Figure 3A] This figure shows an exemplary 3D tetrahedron cost function as described in this disclosure.

[0036] [Figure 3B] This figure shows an exemplary two-dimensional tripod cost function as described in this disclosure.

[0037] [Figure 4] This figure shows an exemplary simplification of the tripod of a tetrahedron according to the present disclosure.

[0038] [Figure 5A] This figure shows the tripod being transposed / placed horizontally on the image plane in Case ABC according to this disclosure.

[0039] [Figure 5B] Figure 5A is a typical diagram showing the transposition / horizontal placement of a tripod on the image plane for Case ABC.

[0040] [Figure 6] This figure shows exemplary F1(x,y) and F2(x,y) solutions for FPxy according to this disclosure.

[0041] [Figure 7] This figure shows the tripod being placed horizontally on the image plane in case ABD according to this disclosure.

[0042] [Figure 8A] This figure shows an exemplary intersection of plane ABC and plane ABD in two dimensions (x,y) according to the present disclosure.

[0043] [Figure 8B] This figure shows an exemplary intersection of plane ABC and plane ABD in three dimensions (x,y,z) according to the present disclosure.

[0044] [Figure 9] This figure shows an exemplary 3D tetrahedron cost function using the coordinates provided in this disclosure.

[0045] [Figure 10] This is a perspective view of a three-dimensional model of the external deformation correction system constructed using corrected radiographic images as described in this disclosure.

[0046] [Figure 11] This figure shows multiple foci passing through two unconnected triangles that result in the same shadow coordinates according to this disclosure.

[0047] [Figure 12] This figure shows three identical triangles casting the same shadow from a single focal point as described in this disclosure.

[0048] [Figure 13] This figure shows a pair of triangles positioned along rays radiating from an arbitrary focal point to a collection of shadow locations in the image plane, according to the present disclosure.

[0049] [Figure 14] This figure shows a flowchart illustrating the method described herein.

[0050] [Figure 15] This figure shows an exemplary computer system that may be used to carry out an aspect of this disclosure (e.g., a method).

[0051] [Figure 16] This figure shows an embodiment of a computer program product that can incorporate this disclosure. Detailed description of the invention

[0052] Aspects, specific features, advantages, and details of the present invention will be described more fully below with reference to the non-limiting embodiments shown in the accompanying drawings. Descriptions of well-known materials, manufacturing tools, processing techniques, etc., have been omitted in order to avoid unnecessarily obscuring the invention. However, it should be understood that the detailed descriptions and specific examples illustrating embodiments of the present invention are given only as examples and not as limitations. Various substitutions, modifications, additions, and / or arrangements within the spirit and / or scope of the underlying concept of the present invention will be apparent to those skilled in the art from this disclosure.

[0053] Methods, systems, and related computer program products relating to correlating the planar positions and characteristics of four individual shapes contained within a particular two-dimensional photograph with four individual spatial coordinates contained within a known structure, such as a fixation structure, will not be described in relation to Figures 1 to 9. These methods, systems, and related computer program products can utilize such information to obtain and display (to the user) the precise spatial relationship between the structure (e.g., a fixation structure) and each of the individual radiographs. While these methods, systems, and related computer program products may be described herein with reference to an external fixation structure (e.g., a hexapod structure), it should be noted that they may also apply to any 3D structure (another orthopedic or non-orthopedic structure) containing known relationships of its at least four spatial coordinates / shapes (e.g., spheres or spheroids or points) (and its at least one 2D image (e.g., a radiograph) having one or more anatomical structures). Furthermore, while a sphere may be used in this description to describe methods, systems, and related computer program products as four distinct spatial coordinates and / or shapes, other known shapes such as spheroids or points, but not limited thereto, may be used equally, as will be understood by those skilled in the art.

[0054] In three-dimensional spatial relationships, six parameters may be needed to relate both the position and pose between any two solid objects in space. Position can be thought of as a translational arrangement, usually in an orthogonal (x,y,z) coordinate system. Pose can be thought of as a series of rotations around the x,y and / or z axes in the same position coordinate system. All six of these together are called degrees of freedom (DOF) in a given three-dimensional space. In ordinary terms, these are In-Out, Left-Right, Up-Down (x,y,z), Roll, Pitch, and Yaw (r,p,y).

[0055] The simplest three-dimensional object is a sphere with a central position (x,y,z) and a specified radius (r). A single sphere (or ellipsoid, etc.) can be used to determine the position (x,y,z) of such an object in three-dimensional space. Since both the position and pose of known constructs (e.g., external fixator constructs such as hexapods) are needed, the necessary consideration is what type of three-dimensional object can be uniquely positioned and posed in three-dimensional space. Relatively simple such objects are known as tetrahedrons, which are triangular pyramids with four distinct vertices and four triangular faces.

[0056] A common type of circular mount is known as a hexapod. This consists of two planar rings connected by six telescopic struts, each having spherical joints at both ends. Most hexapod constructs currently on the market are of the so-called 6x6 configuration, meaning there are six individual spherical mounting positions, usually equally spaced but not required pairs, around the central axis of each ring. A mathematically simpler hexapod construct is known as a 3x3 configuration, where pairs of spherical joints coincide, with three such pairs in each ring. Such a 3x3 hexapod can be divided into 15 individual tetrahedrons, one of which can be used to describe the position and pose of the hexapod mount construct in three-dimensional space.

[0057] It should be noted that while three points can be used to define a two-dimensional plane, they cannot be used to define a three-dimensional structure. As shown in Figures 11 and 12, multiple foci passing through a fixed-dimension triangle (three points) can generate the same shadow coordinates of two unconnected triangles, each representing three positions on each of two planar fixed platforms (e.g., rings) of a fixed structure (e.g., a hexapod). By using a fourth point, some embodiments of the methods and systems disclosed herein allow for the construction of four connected triangles of known dimensions, which allows for the determination of a single focal point, as will be further discussed below. However, in some embodiments, three points on a three-dimensional structure can be utilized by the methods and systems disclosed herein, along with multiple views (and known relationships between views) of the same structure and points. However, such three-point embodiments are not as efficient as the four-point embodiments.

[0058] A structure having four known points / shapes is shown in Figure 1, which depicts one of the 15 possible tetrahedra that can be constructed. While the above description focuses on the use of this method in a hexapod structure with six reference markers, it should be noted that more than four reference markers can be used for any type of 3D structure (e.g., a stationary structure) given that the three-dimensional distances between the markers are known. Specifically, we will show how to accurately determine the spatial relationship between a stationary structure and a two-dimensional radiographic image using any four markers (a total of six possibilities).

[0059] An exemplary hexapod construct, nested with both a 6x6 hexapod mechanical construct and a simpler 3x3 hexapod construct, is the AMDTSixFix system. In the SixFix system, each pair of strut spherical joints on each ring has an additional sphere whose positional relationship to the pair of spherical joints is known. These spheres are radiopaque and are known as reference markers, whose shadows become artifacts in 2D radiographs. Other forms are conceivable, but are not limited to, those of the ellipsoid type, on which both position and some degree of pose can be determined based on a single form, by utilizing the properties of the shadow image.

[0060] To construct a 3x3 hexapod nested within a 6x6 mechanical structure, the spatial relationships between each ring must be determined. The base ring can be considered a reference frame, and the platform ring can be considered a translation reference. Knowing the position of the spherical joint between the struts and the rings, and the length of the struts, allows us to determine the spatial relationships between the base and the platform ring. These spatial relationships can be determined via a forward kinematic solution that returns a transformation matrix, which is an extended representation of the position and pose of the platform ring relative to the base ring and reference frame. Once such a transformation is obtained, the position of the reference marker for each ring can be derived from it with respect to the base ring and reference frame. This allows for the construction of a 3x3 hexapod composed of 20 triangular inner and outer faces. Note that any triangular face of the 3x3 hexapod structure can be treated as the base reference frame, and any of the remaining triangular faces can be treated as a platform with deformation relative to the base, which is also a relatively straightforward problem to determine.

[0061] To properly characterize a specific base reference frame, reference shadows found in 2D radiographs must correlate with their respective positions on a 3x3 construct. To facilitate this correlation, methods, systems, and associated computer-readable products utilize at least four (possibly six) radiopaque reference markers, at least one of which is of a different form from the others, so that, typically, a single larger reference can be used equally, although this would usually be a smaller diameter in the case of spherical markers. The “different” reference markers are typically oriented to their most anterior positions when attached to the patient’s anatomical structure, such as a base ring or upper ring, at known clinically relevant locations. Such potential preferential orientation can facilitate the identification of reference markers, for example, when not all reference markers in a 2D radiograph are identifiable. For example, one or more reference markers may be obscured by other radiopaque elements of the construct. Note that if all or at least four of the reference markers can be identified in the presence of "different" markers, preferential orientation is not necessary, and in fact, a correct correlation can be created between the reference shadow and their respective positions within the fixture structure, regardless of the preferential orientation of the "different" reference markers.

[0062] In some embodiments, methods, systems, and associated computer-readable products can correlate reference shadows found in 2D radiographs with their respective positions on a 3x3 structure by grouping the reference shadows into upper and lower (e.g., upper and lower) collections of one, two, or three identifiable artifacts. The shadows may then be separated into foreground / background order, for example, based on the size (magnification) of each identified reference shadow. For spherical reference markers, the reference shadows can be separated into foreground / background order by utilizing the elliptical diameter (or average diameter, or area of ​​the identified shadow, etc.). The shadows can then be sorted from left to right (e.g., inside to outside, or front to back). The absolute magnification of the reference shadows can then be determined, evaluated, and / or compared to identify outliers or “different” reference shadows. For example, the absolute size of “different” reference shadows may not match those attributable to differences in foreground / background magnification. Next, the identified locations of "different" or outlier reference shadows compared to other identified reference shadows can be used to annotate the reference shadows and correlate them with annotated reference locations on the structure (e.g., a fixed structure). If no "different" reference shadows are identified, the sorting may revert to the assumption that a preferred orientation was used by default, and a list of possible numbering schemes may be ranked according to adherence to the preferred orientation. Such lists of possibilities may then be evaluated based, for example, how well they match those of known structures.

[0063] After correlating reference shadows with their respective three-dimensional positions on a structure (e.g., from an absolute perspective or from the perspective of a probability ranking of multiple possibilities), the method, system, and associated computer-readable product can determine the spatial relationship between the two-dimensional radiograph and the structure. The method for characterizing the spatial relationship may include determining the focus of an X-ray source on the two-dimensional radiograph. The method, system, and associated computer-readable product can determine the focus of an X-ray source on the two-dimensional radiograph using any four points whose mutual separation distances are known. For example, a 3x3 hexapod structure (or other structure) may be divided into 15 different collections, each collection having four vertices that form a tetrahedron. Any one of these tetrahedrons is sufficient for the method, system, and associated computer-readable product to determine the focus of an X-ray source on the two-dimensional radiograph. Therefore, the structure may contain only four references. In some embodiments, the method, system, and associated computer-readable product can average multiple tetrahedrons to improve the accuracy of focus determination.

[0064] In some embodiments, methods, systems, and associated computer-readable products can characterize the focal position by establishing a Cartesian coordinate system. The Cartesian coordinate system can be established using a two-dimensional radiographic image as one of the three planes of the coordinate system. Its origin can be arbitrary, so here, for explanatory / disclosure purposes, we assume it is at the center of the image. The arrangement of the axes is also arbitrary, but here again, for explanatory / disclosure purposes, the x-axis is positioned along the horizontal line of the two-dimensional radiographic image, with the positive direction to the right; the y-axis is positioned along the vertical line of the two-dimensional radiographic image, with the positive direction pointing upwards; and the z-axis is positioned perpendicular to the plane of the two-dimensional radiographic image, with the positive direction pointing from the image towards the viewer.

[0065] The Cartesian coordinate system can be established by assuming that the focal point of the X-ray source lies on the 2D radiation image in the positive z direction, and that the entire structure lies between the focal point and the 2D radiation image. Note that if these assumptions do not apply, the complete shadow of the reference markers representing the vertices of the tetrahedron in the 2D radiation image may not be displayed / included / available.

[0066] In some embodiments, as shown in Figure 2, a focal model using a single tetrahedron may involve utilizing an arbitrary tetrahedron consisting of vertices a, b, c, and d, where the diameter of the reference marker is known, and the corresponding distances between a, b, c, and d and between ab, ac, ad, bc, bd, and cd are known. Focal FP (x,y,z) The image is shown by four rays (in green) emanating from a focal point that intersects the tetrahedron references a, b, c, and d, casting shadows A, B, C, and D on the image plane. Due to the oblique nature of the rays relative to the image plane, these shadows are typically elliptical in nature. Note that the diameter of the elliptical shadow is a function of the magnification applied to the reference from which it is cast. This fact allows us to determine where the reference marker must be located along ray FP-A. This method can also be used for all the other shadows B, C, and D, and their associated references b, c, and d. Such a determination may involve determining the (x,y) centers of each elliptical shadow A, B, C, and D relative to the image coordinate system, as well as their individual diameters. The diameters of the shadows, divided by the known diameters of the associated reference markers, can be used as the MA, MB, MC, and MD magnifications.

[0067] Once a focus model is constructed, the method, system, and associated computer-readable products can constrain the model based on known properties (e.g., algebraically). For example, the method, system, and associated computer-readable products can construct a cost function that can be used in numerical optimization to return the focus in three dimensions. An example of such a cost function is shown in Figure 2. As shown in Figure 2, for example, a tetrahedral cost function can be utilized that includes known relationships between reference positions a, b, c, and d and their associated shadows A, B, C, and D as functions of FPxyz and their respective magnifications MA, MB, MC, and MD. The method, system, and associated computer-readable products may include these relationships with known separation distances between a, b, c, and d, which are contained in Dist=[ab ac ad bc bd cd]. The method, system, and associated computer-readable products can solve such a system of equations so as to avoid ending with multiple solutions (such as mirror equivalents) and / or suboptimal local minima. For example, in some embodiments, methods, systems, and associated computer-readable products can avoid such scenarios by simplifying the search topography in a series of steps, thereby restricting or specifying certain unknowns for certain conditions. For instance, a system with three unknowns might be solved for volume, two for surface, and one for a one-dimensional curve. Thus, methods, systems, and associated computer-readable products may narrow the search field.

[0068] In some such embodiments, the method, system, and associated computer-readable product may recognize that a particular structure may be completely constrained or otherwise unable to function, taking its relationships into account. For example, as shown in Figures 1 and 3, the method, system, and associated computer-readable product may recognize that such a structure may be completely constrained or otherwise unable to function, taking into account the relationships described in Figure 3A. However, it should be noted that such volumetric optimization may be affected by one or more pitfalls already mentioned. Thus, the method, system, and associated computer-readable product may utilize fewer constraints (e.g., remove certain constraints), such as modeling / observing the behavior of a simplified structure. For example, the method, system, and associated computer-readable product may remove the single ray FPxyz-D and the four inter-reference distance constraints, leaving only [ab ac] to arrive at a simplified cost function, for example, as shown in Figure 3B. As shown in Figure 4, the method, system, and associated computer-readable product may thereby utilize or form a tripod structure that traces a planar curve. In some embodiments, methods, systems, and associated computer-readable products can represent the plane on which the curve resides as an ABC plane perpendicular to the image plane.

[0069] In some embodiments, the method, system, and associated computer-readable product can determine the plane of a tripod or its tetrahedron. For example, to determine the ABC plane, the method, system, and associated computer-readable product can place the tripod laterally or transpose it to the image plane, as shown in phases or steps 1, 2, 3, and 4 shown in Figures 5A and 5B. In some embodiments, the method, system, and associated computer-readable product can solve FPxy for such a construct of a plane search under the z=0 condition, rather than a volume search, as shown in Figure 15. In some embodiments, the method, system, and associated computer-readable product can determine or identify two solutions where a curve intersects the image plane, which is used to formulate the equation of a line representing the intersection of the image plane and the ABC plane. Figure 15 shows two intersections of the tripod and focal point FP with the image plane for a specific dimensional example.

[0070] In some embodiments, the method, system, and associated computer-readable product may construct four separate tripods for a particular tetrahedron (see, for example, Figure 1) using different base positions such as ABC, ABD, ACD, and BCD. Each tetrahedron behaves in the same way as tracing a plane curve on its respective plane, when the tripod is placed laterally or transposed on the image plane, and these are all perpendicular to the image plane. Figure 7 shows a second case of such a process for the image plane case ABD. As shown in Figure 7, a plane curve defining the ABD plane is traced. As shown in Figures 8A and 8B, if normals are taken to both the image planes ABC and ABD, the intersection of the two planes may coincide with FP(x,y). It should also be noted that all six combinations of intersecting planes can produce the same FP(x,y). For example, these values ​​may differ slightly due to small errors in measurements, so the method, system, and associated computer-readable product may reduce such errors by taking the average of all six possible intersections. In some embodiments, methods, systems, and associated computer-readable products utilize statistical operations / analysis to deselect crossovers that are outliers, for example, that can be averaged out of the remaining crossovers.

[0071] In some embodiments, methods, systems, and associated computer-readable products may determine the z-coordinate of the focal point FP by utilizing known / determined x and y coordinates, such as via a cost function shown in Figure 9. Such an approach may involve favorable optimizations because there is only one unknown z.

[0072] Once the optimal focus is determined for a particular two-dimensional radiographic image, the method, system, and associated computer-readable product can determine the spatial relationship between the two-dimensional radiograph and the stationary structure. For example, actual reference positions can be positioned along a vector from the focus to the reference shadow position using the solution identified above. These positions can then be transformed into three-dimensional image coordinates so that each reference position is represented relative to the two-dimensional radiographic image.

[0073] In some embodiments, the method, system, and associated computer-readable product can use dimensional image coordinates to define actual reference position vectors between these reference positions. In some embodiments, the method, system, and associated computer-readable product can determine the vector cross product between suitable pairs of these vectors to provide the construction of an orthogonal coordinate system for any collection of three discrete references. Since each of these coordinate systems relates to a two-dimensional radiographic image, the method, system, and associated computer-readable product can utilize any of them as the basis for other coordinate systems constructed with any group of reference positions. Furthermore, the method, system, and associated computer-readable product can invert any of these resulting coordinate systems to develop coordinate transformations of the two-dimensional radiographic image with respect to any coordinate system representing a stationary construct.

[0074] In some embodiments, the method, system, and associated computer-readable product can utilize multiple images, where available, to determine the relationship between each image and the construct. Since the construct is a static, known entity within the multiple images, the method, system, and associated computer-readable product can determine the spatial relationships between the multiple images so that further characterization of artifacts common to the multiple images can be accurately characterized in three-dimensional space relative to the construct.

[0075] Next, with respect to Figure 10, we will describe additional methods, systems, and related computer program products for determining the actual positions and poses of a collection of known objects in a projected three-dimensional space above a two-dimensional radial space.

[0076] Referring to Figure 10, an exemplary external deformation correction device in a schematic form known as a hexapod is shown, consisting of a base and platform positioned in space, to which six spherical radiopaque reference markers, functioning as known shapes A, B, C, D, E, and F, are attached, with code distances AB, BC, CA and DE, EF, and FD all known. A, B, and C are further connected by a set of six dashed lines to D, E, and F, each having a known length. Figure 10 shows what is known as a 3x3 configuration, pointing to the centers of three matching spheres on the base and platform. In this example, the reference marker indicated by A is selected to be smaller than the remaining reference markers, all of the same size. This is done to distinguish between the base and platform, and between the rotation of the base in image space.

[0077] In some embodiments, the method and system can utilize a typical radiographic image to position the shadow of a reference marker and evaluate its size and position within the radiographic image. An advantage of using spherical reference markers is that they always cast elliptical shadows. In some embodiments, the method and system can utilize the minor axis dimension, which may relate to the relative distance between the image and the X-ray focus, and the height along the vector where the actual reference marker resides. In some embodiments, the method and system can utilize image resolution to determine the initial image scale and relative size of the shadow artifact relative to the actual object. As shown in Figure 10, the focus FP(xyz) )teeth, is defined as any point floating in space above the image. Next, in some embodiments, the method and system can define multiple constraints using closed vector loops such as B0A->A->B->B1A->BOA, P1A->FP(xyz)->P2A->P1A, and B2A->C->E->P1A->B2A, as shown in Figure 10. The problem is well suppressed by using multiple closed vector loops passing through the origin, as shown by the green loops, but it should be noted that this requires measuring the small diameter of the shadow relative to the actual diameter of a spherical object, which can be error-prone due to the limitations of resolution and scattering. Therefore, the results can be statistically improved by using more loops, and many closed vector loops can be used. It was determined that it is sufficient to use more than four vector loop combinations for the base and platform respectively (red), four vector loops between the base and platform (blue), and three vector loops between the image and the focus (green) by utilizing relative size.

[0078] Another approach is shown in Figure 13, where the image is represented by rays emanating from an arbitrarily selected focal point FP(x,y,z) to a collection of shadow locations shown on the image / image plane (e.g., formed by references of known orthopedic structures such as hexapods). Two triangles of known dimensions (upper and lower triangles) (e.g., their points corresponding to reference locations of known structures) are each used / positioned along three rays (i.e., one ray per point / corner of the triangle). Although the triangles are shown as two separate articles, note that they can share vertices or edges if only five or four shadow artifacts can be detected in the image. Note that any combination of triangles that can be constructed using the available shadow articles and their associated rays to any focal point FP(x,y,z) is also available. To determine the appropriate orientation of a triangle, the system and method can evaluate the relative magnification of the shadowed article to determine whether outliers lead or lag (i.e., are close to, near, far from, or furthest from the focal point FP(x,y,z)). For any particular focal point, the system and method can determine a cost function which is the sum of the errors between the known separation between vertices and the calculated distances between vertices (for any particular focal point FP(x,y,z)). Such a cost function can be used in a numerical solver to determine the best compatible FP(x,y,z) for a given construct and the shadow it casts in a projected geometric sense.

[0079] After determining at least four of the node positions A, B, C, D, E, F, and O in image space, in some embodiments, the method and system can then construct a coordinate transformation suitable for a known three-dimensional object represented by a collection of spherical references. In some such embodiments, the method and system can construct a suitable coordinate transformation by determining the cross product of a suitable pair of vectors, for example, ABxAC yields a vector perpendicular to both AB and AC, with the origin at A. The vector can then be intersected with either the previous vector AB or AC to determine a Cartesian coordinate system representing the base of the image space defined in this case by ABC. The method and system can use the same cross product method for multiple images, resulting in multiple coordinate systems describing all of the same known three-dimensional objects in a larger patient space.

[0080] This allows the method and system to determine the relationships between known objects by utilizing how they exist in two different spaces, and thus determine coordinate transformations between those different spaces using matrix operations such as inverse and multiplication. This capability of the method and system eliminates the need to provide orthogonal images rotated around a common axis to determine the true three-dimensional conditions to modify.

[0081] As shown in Figure 14, the methods and systems of the present disclosure can perform Method 100, which includes the step in 102 digitally acquiring a 2D radiographic image (or a digital version thereof) showing a 3D structure of known configuration (e.g., shape, size, etc.), such that four identifiable discrete criteria / points of the structure, such as shadow centers of known spherical criteria / elements of the 3D structure, are identified (digitally and / or manually) in the image, and the spatial relationships between each point in the structure are known / inputted. Next, in 104, Method 100 may include the step of generating four vectors to arbitrary focal points positioned on the 2D image plane of the image, using the four identified discrete points as a basis. Next, Method 100 may include the step in 106 digitally establishing a cost function that evaluates the suitability of the arbitrary focal points with respect to known separation distances of the 3D structure (e.g., via one or more vector loops, tripod dimensionality reduction to 2D space, or the sliding triangle method as described above). Next, in 108, method 100 may include the step of digitally using a cost function as a discriminant in an optimization numerical solver to determine compatible focal FP(x,y,z). Note that specific optimization numerical solution steps may differ depending on the approach used. Using the optimized focal FP(x,y,z) and known discrete points in the image, method 100 may include the step in 110 of digitally determining the positions of known elements along a vector (from focal FP(x,y,z)). The tetrahedron is positioned in image space by the positions corresponding to the / 3D coordinates, which are the 3D coordinates of the elements of the 3D construct. Next, in 112, method 100 may include the step of digitally establishing two vectors having a common vertex using any two edges of any two of the four triangular faces of the tetrahedron. Next, method 100 may include the steps of digitally establishing mutual normals in 114, such as by taking the cross product of two vectors in a preferred order, and digitally establishing a Cartesian coordinate system in image space, such as by taking the cross product of the mutual normals in a preferred order between the preferred selection of the original two vectors.In some embodiments, method 100 may then include the step of digitally constructing a transformation (e.g., a matrix) between the Cartesian coordinate system from 114 and any other combination of face vertices, using known relationships between the faces of a known 3D tetrahedron, in 116.

[0082] Next, method 100 may include a step in 118 to repeat steps 102-114 (and potentially 116) for all combinations of four discrete points in a larger group of discrete points, if more than four discrete points are available in the image. Next, in 120, method 100 may include a step to digitally generate a composite coordinate transformation (e.g., a matrix) representing a known 3D construct in image space, for example, by averaging all equivalent transformations (e.g., those generated in step 116 and / or 118). Next, method 100 may include a step in 122 to digitally generate a 2D image transformation with respect to the known 3D construct via the inverse coordinate transformation of 120.

[0083] Next, Method 100 may include the step of repeating steps 102-122 for each 2D image obtained from a known 3D construct (e.g., multiple images such as two or more images), and the step of digitally constructing a 3D representation of the multiple images with respect to the 3D construct. Next, Method 100 may include the step of digitally establishing plane-to-plane, or plane-to-vector, or the intersection of the nearest points between vectors, representing the anatomical structure originating from the focal point FP(x,y,z) of each corresponding 2D image, in order to determine the relationship between the known 3D construct and the anatomical structure of the subject, using the 3D representation of the 2D images with respect to the known 3D construct.

[0084] As will be apparent to those skilled in the art, the inventions of this disclosure provide significant improvements in the fields of external fixation devices and anatomical structure computer modeling, including the fields of hexapod and bone segment modeling. Furthermore, the inventions of this disclosure provide significant improvements in the fields of radiography, including the fields of distortion correction of radiographic images. The inventions of this disclosure also provide significant improvements in the fields of external fixation device adjustment prescription determination, including the fields of hexapod adjustment prescriptions.

[0085] Those skilled in the art will recognize that aspects of the present invention can be implemented in systems, methods, and / or computer program products. In some embodiments, aspects of the present invention can be implemented entirely in hardware, entirely in software (e.g., firmware, resident software, microcode, etc.), or in a combination of software and hardware aspects, all of which are generally referred to herein as “systems” and may include circuits and / or modules.

[0086] Figure 15 shows an example of a computer system for use incorporating one or more aspects of the present invention. The computer system 500 may be a computer system for an article manufacturing and / or repair facility, such as a computer system used for additively manufacturing articles and / or a computer system for generating data used to manufacture articles by AM equipment or devices. The computer system 500 in Figure 15 is suitable for storing and / or executing program code, such as program code for performing the above processes, via the bus 520 to memory 504 It includes at least one processor 502 directly or indirectly coupled to the memory. During operation, the processor(s) 502 504 From this, instructions for execution by the processor(s) can be obtained. 504This may include local memory used during the actual execution of program code, bulk storage, and cache memory that provides temporary storage for at least some program code to reduce the number of times code must be retrieved from bulk storage during program code execution. 504 A non-exclusive list of examples includes hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. 504 This may include an operating system 505 and one or more computer programs 506 for performing the embodiments described herein, such as making adjustments to the digital layout of a circuit design.

[0087] Input / Output (I / O) device 512, 514 Peripheral devices (such as) can be coupled to the system directly or via the I / O controller 510. The network adapter 508 can also be coupled to the system to enable the computer system to be coupled to other computer systems via an intervening private or public network. Modems, cable modems, and Ethernet cards are just a few of the types of network adapters 508 currently available. In one example, the network adapter 508 facilitates the acquisition of data from a remote source to facilitate aspects of the present invention.

[0088] The computer system 500 may be coupled to storage 516 (e.g., non-volatile storage areas such as magnetic disk drives, optical disk drives, tape drives, etc.) having one or more databases. Storage 516 may include internal storage devices or connected or network-accessible storage. Computer programs in storage 516 are stored in memory504 It can be loaded and executed by processor 502.

[0089] The computer system 500 may include fewer components than those shown, additional components not shown herein, or several combinations of the illustrated components and additional components. The computer system 500 may include any computing devices such as mainframes, servers, personal computers, workstations, laptops, handheld computers, smartphones, tables, or other mobile devices, telephony devices, network appliances, virtualization devices, and storage controllers.

[0090] Furthermore, the above process may be executed by multiple computer systems 500 working together as part of a computing environment.

[0091] In some embodiments, aspects of the present invention may take the form of a computer program product implemented on a computer-readable medium(s). A computer-readable medium(s) can implement computer-readable program code thereon. Various computer-readable mediums(s) or combinations thereof can be used. For example, a computer-readable medium may include (but is not limited to) computer-readable storage media(s), examples of which include one or more electronic, magnetic, optical, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Examples of computer-readable storage media(s) include, for example, electrical connections having one or more wires, portable computer diskettes, hard disks or mass storage devices, random access memory (RAM), read-only memory (ROM), and / or erasable and programmable read-only memory such as EPROM or flash memory, optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices (including tape devices), or any suitable combination thereof. Computer-readable storage media are defined as tangible media that contain or can store program code for use by or in connection with devices such as instruction execution systems, apparatuses, or processors. Therefore, program code stored on computer-readable media generates a product containing the program code (such as a "computer program product").

[0092] Referring now to Figure 16, in one example, the computer program product 600 includes, for example, one or more computer-readable media 602 on which computer-readable program code means or logic 604 are stored in order to provide and promote one or more aspects of the present invention.

[0093] Program code contained in or stored on a computer-readable medium can be retrieved and executed by a computer system (such as a computer including its components) and / or other devices to cause the computer system, its components, and / or other devices to operate / function in a particular manner. The program code can be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, and / or radio frequencies. The program code for performing operations to execute, achieve, or facilitate aspects of the present invention can be written in one or more programming languages. In some embodiments, the programming languages ​​include object-oriented and / or procedural programming languages ​​such as C, C++, C#, and Java. The program code can be executed entirely on the user's computer, entirely off the user's computer, or a combination of part on the user's computer and part on a remote computer. In some embodiments, the user's computer and the remote computer communicate over a network such as a local area network (LAN) or wide area network (WAN) and / or over an external computer (e.g., using the Internet with an Internet Service Provider).

[0094] In one example, program code includes one or more program instructions obtained for execution by one or more processors. Computer program instructions are provided, for example, to one or more processors of one or more computer systems to manufacture a machine, and as a result, when the program instructions are executed by one or more processors, embodiments of the invention, such as actions or functions described in the flowcharts and / or block diagrams described herein, can be performed, achieved, or facilitated. Therefore, each block, or combination of blocks, in the flowcharts and / or block diagrams shown and described herein can be implemented by computer program instructions in some embodiments.

[0095] The flowcharts and block diagrams shown and described with reference to the figures illustrate the architecture, function, and operation of possible embodiments of the system, method, and / or computer program product according to aspects of the present invention. Therefore, these flowcharts and / or block diagrams may be of the method, apparatus (system), and / or computer program product according to aspects of the present invention.

[0096] In some embodiments, as described above, each block in a flowchart or block diagram may represent a module, segment, or portion of code and may include one or more executable instructions for implementing the specified operation and / or logical function of the block. Those skilled in the art will understand that the operations / functions specified or performed by a block may occur in a different order than shown and / or described, or may occur simultaneously with, or partially / fully simultaneously with, one or more other blocks. In fact, two blocks shown consecutively may be executed substantially simultaneously or in reverse order. Furthermore, each block in a block diagram and / or flowchart diagram, as well as any combination of blocks in a block diagram and / or flowchart diagram, may be fully implemented by a dedicated hardware-based system that performs the operations / functions specified in the block or the block diagram or flowchart as a whole, or in combination with computer instructions.

[0097] It should be understood that the above description is illustrative and not limiting. A number of changes and modifications can be made herein by those skilled in the art without departing from the general spirit and scope of the invention as defined by the following claims and their equivalents. For example, the above embodiments (and / or aspects thereof) can be used in combination with one another. Furthermore, many modifications can be made to adapt specific situations or materials to the teachings of the various embodiments without departing from their scope. The dimensions and types of materials described herein are intended to define parameters of the various embodiments, but they are not limiting and are merely illustrative. Considering the above description, many other embodiments will become apparent to those skilled in the art. Therefore, the scope of the various embodiments should be determined by reference to the appended claims, along with the entire scope of the equivalents to which such claims are granted.

[0098] The terms used herein are intended solely to describe specific embodiments and are not intended to limit the invention. As used herein, the singular forms "a," "an," and "the" are intended to include the plural form unless the context clearly indicates otherwise. Furthermore, "comprise" (and any form such as "comprises" and "comprising"), "have" (and any form such as "has" and "having"), "include" (and any form such as "includes" and "including"), "contain" (and any form such as "contains" and "containing"), and other grammatical variations will be understood to be free-form linking verbs. As a result, a method or article that “comprises,” “has,” “includes,” or “contains” one or more steps or elements possesses, but is not limited to possessing only, those one or more steps or elements. Similarly, an element of a step or article of a method that “comprises,” “has,” “includes,” or “contains” one or more features possesses, but is not limited to possessing only, those one or more features.

[0099] As used herein, the terms “comprising,” “has,” “including,” “containing,” and their other grammatical variations encompass the terms “consisting of” and “consisting essentially of.”

[0100] As used herein, the phrase "consisting essentially of" or its grammatical variations thereof should be interpreted as specifying a described feature, integer, step, or component, but not precluding the addition of one or more additional features, integers, steps, components, or groups thereof, provided that the additional features, integers, steps, components, or groups thereof do not substantially alter the basic and novel characteristics of the claimed component or method.

[0101] All publications cited herein are incorporated herein by reference in such a way that they are specifically and individually indicated as if each individual publication were a complete reference.

[0102] Subject matter incorporated by reference is not considered a substitute for claim limitations unless otherwise specified.

[0103] Wherever one or more scopes are referenced throughout this specification, each scope is intended to be an abbreviation for informational purposes, and the scope is understood to encompass each discrete point within that scope as if it were fully described herein.

[0104] While several aspects and embodiments of the present invention are described and shown herein, alternative aspects and embodiments may be influenced by those skilled in the art to achieve the same objective. Therefore, the present disclosure and the accompanying claims are intended to cover all such further alternative aspects and embodiments that fall within the true spirit and scope of the present invention.

Claims

1. A method for determining the actual position and pose of a known three-dimensional structure of an orthopedic fixation device, using at least four individual shapes formed by a reference of the structure shown in a two-dimensional (2D) image of the structure, the method comprising: The steps include identifying at least four reference shadows in the 2D image corresponding to the criteria of the structure, The steps include correlating at least four identified reference shadows with the respective reference shadow locations on the structure, The step of determining the spatial relationship between the 2D image and the structure. Includes, The step of determining the spatial relationship between the 2D image and the structure is, The steps include determining the focus of the source of the 2D image via a predetermined separation distance between the at least four identified reference shadows and the corresponding references of the structure, A step of identifying a reference position along the vector from the focal point to the reference shadow position, The steps include converting the aforementioned reference position into three-dimensional (3D) image coordinates, The steps include defining a reference position vector between the reference positions via the 3D image coordinates, The steps include: constructing a first orthogonal coordinate system of a collection of three discrete criteria formed by the criteria of the construct with respect to the 2D image by determining the vector cross product between appropriate pairs of the position vectors; Methods that include...

2. The step of correlating the identified at least four reference shadows with the respective reference shadow positions on the structure is: The steps include identifying the at least four identified reference shadows as upper or lower reference shadows, The steps include determining the order of the foreground or background of the identified at least four reference shadows based on their respective sizes, The steps include determining the order of the identified at least four reference shadows from left to right or right to left, The steps include annotating the four identified reference shadows and correlating them with the respective annotated reference shadow locations on the structure. The method according to claim 1, including the method described in claim 1.

3. The step of determining the spatial relationship between the 2D image and the structure is, Steps to perform a coordinate transformation of the 2D image with respect to any coordinate system representing the structure by inverting the first constructed orthogonal coordinate system or the second constructed orthogonal coordinate system determined through the first constructed orthogonal coordinate system. The method according to claim 1 or 2, further comprising:

4. The step of determining the spatial relationship between the 2D image and the structure is: The steps include: reconstructing the tripod model such that a first plane formed by a first group of three of the four criteria is located along the image plane; The steps include determining a first equation for a first line representing the intersection of the image plane and the first plane, The steps include: reconfiguring the tripod model such that a second plane formed by a second group of three of the four criteria is located along the image plane; The steps include determining a second equation for a second line representing the intersection of the image plane and the second plane, The steps include determining the x and y coordinates of the focus via at least the first and second lines, The steps include determining the z coordinate of the focus via the x and y coordinates and the cost function, and The method according to any one of claims 1 to 3, including

5. A computer-readable storage medium that is readable by one or more processing circuits and stores instructions for causing one or more processors to perform the method according to any one of claims 1 to 4. A computer program product that includes the following features.

6. Memory and At least one processor communicating with the memory, A program instruction via the memory causing the at least one processor to execute the method according to any one of claims 1 to 4 A system equipped with these features.

7. A method for determining the actual position and pose of a collection of known spherical objects originating from orthopedic fixation devices in a projected three-dimensional space above a two-dimensional radiation space, wherein the method is: The steps include obtaining two or more digital radiation images of the collection of known spherical objects in the projected three-dimensional space above the two-dimensional radiation space, A step of determining the actual position and pose of the collection of known spherical objects in the projected three-dimensional space above the two-dimensional radiation space by using a measurement of at least one of the average diameter or area of ​​the elliptical shadows obtained from the spherical objects of the collection of known spherical objects in the two-dimensional radiation space of the two or more digital radiation images. Methods that include...

8. The method according to claim 7, further comprising the step of constructing a three-dimensional model of the actual positions and poses of the collection of known spherical objects in the projected three-dimensional space.

9. The method according to claim 7 or 8, wherein the step of determining the actual position and pose of the collection of known spherical objects includes the step of determining the relative magnification of the two or more digital radiographic images by utilizing projection distortion to reconstruct the projected three-dimensional space.

10. The method according to claim 9, further comprising the step of determining a relationship between the two or more digital radiographic images by comparing common spherical objects in a collection of known spherical objects in the two or more digital radiographic images.

11. The method according to claim 10, further comprising the steps of: the two or more digital radiographic images further include at least one anatomical structure requiring correction; and constructing a three-dimensional model of the actual position and pose of the at least one anatomical structure in the projected three-dimensional space.

12. A computer-readable storage medium that is readable by one or more processing circuits and stores instructions for one or more processors to perform a method of determining the actual position and pose of a known collection of objects of an orthopedic fixation device using at least four individual shapes formed by a reference of the structure shown in a two-dimensional (2D) image of the structure. A computer program product comprising the method, The steps include identifying at least four reference shadows in the 2D image corresponding to the criteria of the structure, The steps include correlating at least four identified reference shadows with the location of each reference shadow on the structure, The step of determining the spatial relationship between the 2D image and the structure. Includes, The step of determining the spatial relationship between the 2D image and the structure is, The steps include determining the focus of the source of the 2D image via a predetermined separation distance between the at least four identified reference shadows and the corresponding references of the structure, The steps include establishing a Cartesian coordinate system that uses the 2D image as one of the three planes of the coordinate system, A step of determining the position along the focal ray where each of the four criteria mentioned above must exist, A step of constraining a model of at least four criteria based on known properties of the structure via a cost function, wherein the known properties do not include a predetermined separation distance between the focal ray and the criteria, and the constraints form a tripod model that traces a planar curve in a plane perpendicular to the image plane of a 2D image. Computer program products, including [this].

13. Memory and At least one processor communicating with the memory, Program instructions via the memory causing at least one processor to perform a method for determining the actual position and pose of a known collection of objects of orthopedic fixation devices in a projected three-dimensional space above a two-dimensional radiation space, A system comprising the method, The steps include obtaining a digital radiographic image of the collection of known objects in the projected three-dimensional space above the two-dimensional radiographic space, The step of determining the actual position and pose of the collection of known objects in the projected three-dimensional space above the two-dimensional radiation space by utilizing the shadows of the collection of known objects in the two-dimensional radiation space of the digital radiation image, wherein the determining step is A step of determining the focus and identifying the position of the known collection of objects along the vector from the focus to the position of the shadow, The steps include converting the positions of the collection of known objects into three-dimensional image coordinates, The steps include defining position vectors between the positions of the known collection of objects via the three-dimensional image coordinates, A system equipped with these features.

14. The step of determining the actual position and pose of the collection of known objects in the projected three-dimensional space above the two-dimensional radiation space is: The steps include: determining the vector cross product between appropriate pairs of the position vectors to construct a first orthogonal coordinate system for three collections of known objects with respect to the digital radiation image; The steps include: inverting the first constructed orthogonal coordinate system and unfolding the coordinate transformation of the digital radiographic image with respect to any coordinate system representing the three collections of known objects; The system according to claim 13, further comprising:

Images