Orthopedic Fixation Control and Visualization

Through guided stent matching technology and a three-dimensional overview, the problem of difficulty in identifying the position of the fixator element is solved, improving the accuracy and reliability of fracture and malformation treatment, ensuring correct alignment and healing of anatomical segments.

CN113841183BActive Publication Date: 2024-11-26SYNTHES GMBH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202080039246.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2019-03-28
Filing Date
2020-03-25
Publication Date
2024-11-26
Estimated Expiration
2040-03-25

AI Technical Summary

Technical Problem

Prior Art When using external fixators for fracture and malformation treatment, it is difficult for users to identify and mark the location of fixator elements, resulting in increased recognition time, increased probability of error and reduced reliability of treatment regimens, especially when images are not orthogonal. Inadequate feedback affects the correct alignment and healing of anatomical segments.

Method used

Using guide bracket matching technology, helps users identify component locations by identifying the position of fixer components in an easily recognizable image, generating a graphic projection and superimposing it onto a less recognizable image, helping users identify component locations while providing a three-dimensional overview to ensure Correctness of position and orientation.

Benefits of technology

Improves identification accuracy and computational reliability of fixator elements, ensures correct alignment and healing of anatomical segments, and reduces additional radiographic and surgical needs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113841183B_ABST
    Figure CN113841183B_ABST
Patent Text Reader

Abstract

A guided fixture matching technique may be employed, wherein a graphic projection of the fixture is used to guide and assist the user in identifying the fixture elements within the image. When superimposed on the image, the graphic projection of the fixture may be manipulated to align and superimpose with the fixture elements displayed within the image. In addition, a three-dimensional overview of the imaging scene of the fixture may be generated. The three-dimensional overview may include a three-dimensional graphical model that displays an image representation of the fixture attached to an anatomical structure segment and a representation of the imaging source, reference position, fixture elements, and anatomical structure segment. When the images are not truly orthogonal, a modified second image representation may be displayed to represent a second image that is truly orthogonal to the first image.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of U.S. Patent Application No. 16 / 367,526, filed on March 28, 2019, which is hereby incorporated by reference in its entirety. Background Art

[0003] Techniques for treating fractures and / or deformities of anatomical structures such as bones may include the use of external fixators, such as hexagonal brackets and other fixation mounts, which are surgically mounted to segments of the anatomical structure on opposite sides of the fracture site. A pair of radiographic images of the fixator and the segment of the anatomical structure at the fracture site are taken. Data from the images are then manipulated to construct a three-dimensional representation of the fixator and the segment of the anatomical structure that can be used to formulate a treatment plan, which may, for example, include realigning the segment of the anatomical structure by adjusting the fixator.

[0004] However, existing techniques for controlling fixator manipulation may involve many limitations that introduce inefficiencies, complications, and unreliability. For example, some conventional techniques may rely on surgeons or other users to indicate the positions of certain fixator elements, such as struts, within an image displayed in a graphical user interface of a computer. However, it may be difficult for users to identify and mark the positions of struts and other fixator elements within the image. Specifically, depending on the position and orientation of the captured image, it may not be easy to identify struts and other fixator elements, such as because they may overlap each other completely or partially, or may be otherwise blocked within the image. This may make it difficult for users to identify fixator elements, thereby increasing the time required to identify elements, increasing the probability of errors, and reducing the reliability of calculations. For another example, some conventional techniques may be limited in their ability to visualize a complete three-dimensional fixator imaging scene for users. In addition, when the image is not orthogonal, the user may not be able to effectively view the representation of various corrections (such as corrections to anatomical structure angulation) calculated by the software corresponding to the truly orthogonal image. Therefore, the feedback provided to the user may be insufficient, resulting in additional errors. This can reduce the reliability of the treatment plan, potentially leading to improper alignment of the anatomical segments during recovery, poor healing between the anatomical segments, the need for additional rounds of radiographic imaging to facilitate alignment correction, or even the need for additional surgical procedures. Summary of the invention

[0005] Techniques for orthopedic fixation control and visualization are described herein, for example, for correcting deformities of anatomical structures such as bones. Specifically, in some examples, a fixation device may be attached to a first anatomical structure segment and a second anatomical structure segment. Images of the fixation device and the attached anatomical structure segments, such as x-rays, may then be taken from different orientations relative to the fixation device.

[0006] In some examples, various manipulations of the fixation device for correcting anatomical deformities may be determined based on the position and orientation of the anatomical segment in three-dimensional space. In addition, in some examples, the position and orientation of the anatomical segment in three-dimensional space may be determined based on the image. Specifically, in some cases, the position and orientation of the anatomical segment in three-dimensional space may be determined by having a surgeon or other user indicate the position of various fixator elements and anatomical structures within the image. However, as described above, it may be difficult for the user to identify and mark the position of certain fixator elements (such as struts) within the image. Specifically, depending on the position and orientation of the image taken, it may not be easy to identify the struts and other fixator elements, such as because they may overlap each other completely or partially, or may be otherwise obscured within the image. For example, in some cases, it may be more difficult to identify the struts within the lateral image than to identify the struts within the front image, such as because the struts may generally overlap each other when viewed within the lateral image. This may make it difficult for the user to identify the fixator elements, thereby increasing the time required to identify the elements, increasing the probability of errors, and reducing the reliability of the calculation.

[0007] In some examples, to alleviate the above and other problems, a guided bracket matching technique may be employed, wherein a graphic projection of a fixture is used to guide and assist a user in identifying a fixture element within an image. Specifically, the position of a fixture element (such as a ring and / or a strut of a fixture) may be identified by a user within a first image (e.g., a front image or other image in which the fixture element is easily identified). A graphic projection of the fixture, such as a graphic representation including the ring and / or the strut, may then be generated based at least in part on the identified position of the fixture element in the first image. The graphic projection of the fixture may then be superimposed on a second image, such as a lateral image or other image in which the fixture element is not easily identified. The graphic projection of the fixture may then be used within the second image as a guide to help the user identify the position of the fixture element within the second image. In some examples, the graphic projection of the fixture may be rotated relative to the position of the fixture element in the first image. Specifically, the graphic projection of the fixture may be rotated based at least in part on the angle of the image planes of the first image and the second image relative to each other. For example, if the first image is a front image and the second image is a side image at a ninety degree angle to the front image, the graphic projection of the fixture may be rotated ninety degrees relative to the identified position of the fixture element in the first image when superimposed on the second image. The graphic projection of the fixture may also be modified by the user within the second image, for example by moving, resizing and / or further rotating the graphic projection of the fixture, such as to better align with the position of the fixture element in the second image. When superimposed on the second image, the graphic projection of the fixture may be used as a guide to help the user identify the fixture element within the second image, such as by manipulating the graphic projection of the fixture to align and superimpose with the fixture element displayed within the second image.

[0008] Another technique for improving the accuracy and reliability of input values ​​and resulting calculations is disclosed herein, which provides a three-dimensional overview of the imaging scene of the fixator. The three-dimensional overview can be used to provide feedback and visual confirmation to help ensure that the position and orientation of the calculated anatomical structure are reliable and correct. Specifically, the three-dimensional overview can include a three-dimensional graphical model displayed in a graphical user interface of a computing system. The three-dimensional graphical model can include a first image representation representing a first image, and can include a second image representation representing a second image. The first image representation and the second image representation can be displayed at the same angle as the angle between the image planes of the first image and the second image. The three-dimensional model can also include a graphical representation of the virtual positions of the corresponding first imaging source and the second imaging source corresponding to the first image and the second image. The first virtual line can connect the first virtual position corresponding to the first imaging source to a reference position on the first image representation. The second virtual line can connect the second virtual position corresponding to the second imaging source to the same reference position on the second image representation. In some examples, the reference position can be a reference point of the first anatomical structure segment or the second anatomical structure segment, such as an end point of the anatomical structure segment. The three-dimensional graphical model may display a first graphical representation associated with the shortest distance (e.g., the intersection) between the first virtual line and the second virtual line, and the first graphical representation may represent the physical location of the reference position in the three-dimensional space. The three-dimensional graphical model may also display a graphical representation of a fixator element (ring, strut, etc.) at a virtual location that represents the physical location of the fixator element in the three-dimensional space. The three-dimensional graphical model may also display a graphical representation of an anatomical structure segment at a virtual location that represents the physical location of the anatomical structure segment in the three-dimensional space. The three-dimensional graphical model may be scaled, translated, and rotated by the user in any combination of one or more directions.

[0009] In some examples, when the first image plane and the second image plane are not orthogonal, the three-dimensional graphical model can display a modified second image representation that is orthogonal to the first image representation. The modified second image representation can represent a modified second image having an image plane that is truly orthogonal to the image plane of the first image. In some examples, if the second image is truly orthogonal to the first image, the software can calculate the angle of the anatomical structure segment to be displayed in the second image. The software can then display the modified second image in the modified second image representation, where the anatomical structure segments are displayed at the calculated angles relative to each other. In this way, the software can show the user how to adjust the measured anatomical structure deformity values ​​from the second image that is not truly orthogonal to the first image to the corrected anatomical structure deformity values ​​of the truly orthogonal image, and the software provides guidance for verifying the correction values ​​used in the deformity calculation. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The above summary of the invention and the following detailed description of the preferred embodiments of the present application will be better understood when read in conjunction with the accompanying drawings. The accompanying drawings show preferred embodiments for the purpose of illustrating the method and / or technique of performing plastic surgery fixation using image analysis. However, it should be understood that the present application is not limited to the precise arrangements and / or instruments shown in the accompanying drawings, wherein:

[0011] Figure 1 is a perspective view of a fixation assembly positioned for imaging according to one embodiment;

[0012] Figure 2 yes Figure 1 A perspective view of an exemplary imaging process of the fixed assembly is shown;

[0013] Figure 3A and Figure 3B is a flow chart illustrating an exemplary process for controlling manipulation of a fixation device to correct an anatomical deformity;

[0014] Figure 4 is a screenshot of an exemplary interface for selecting the perspective frame matching (PFM) technique;

[0015] Figure 5 is a screenshot of an exemplary configuration information input interface of the PFM technology;

[0016] Figure 6 is a screenshot of an exemplary first image information input interface of the PFM technology;

[0017] Figure 7 is a screenshot of an exemplary proximity assistance interface of the PFM technology;

[0018] FIG. 8A to FIG. 8H is a screenshot of an exemplary second image information input interface of the PFM technology;

[0019] Fig. 9 is a screenshot of an exemplary deformity parameter interface of the PFM technique;

[0020] Fig.10 is a screenshot of an exemplary installation parameter interface of the PFM technology;

[0021] Fig.11 is a screen shot of a first exemplary treatment plan interface of the PFM technology;

[0022] Fig.12 is a screen shot of a second exemplary treatment plan interface of the PFM technology;

[0023] Fig.13 is a screenshot of a third exemplary treatment plan interface of the PFM technology;

[0024] Fig.14 is a flow chart illustrating an exemplary process for providing a graphical projection of a fixture using a guide bracket matching technique;

[0025] Fig.15A is a diagram showing exemplary images of a first anatomical structure segment and a second anatomical structure segment and a fixator attached to the first anatomical structure segment and the second anatomical structure segment;

[0026] Fig. 15B is a diagram showing an exemplary first image and a second image of a fixture, wherein the location of a fixture element is indicated in the first image but not in the second image;

[0027] Fig.16 is a diagram showing an exemplary graphic projection of a fixture superimposed on an image;

[0028] Fig.17A is a diagram showing an exemplary graphical projection of a fixture manipulated by a user;

[0029] Fig. 17B is a diagram showing an exemplary first image and a second image of a fixture, wherein positions of fixture elements are indicated in both the first image and the second image;

[0030] Fig.18 is a flow chart illustrating an exemplary process for generating a three-dimensional overview of an imaging scene of a fixture;

[0031] Fig.19 is a diagram showing an exemplary three-dimensional graphical model of an imaging scene of a fixture;

[0032] Fig. 20 is a diagram illustrating an exemplary three-dimensional graphical model including a graphical representation of a retaining ring;

[0033] Fig.21 is a diagram showing an exemplary three-dimensional graphical model including a graphical representation of fixator struts and anatomical structure segments;

[0034] Fig. 22 is a diagram showing an exemplary three-dimensional graphical model from front and back perspectives;

[0035] Fig.23 is a diagram showing an exemplary three-dimensional graphical model from a side perspective;

[0036] Fig.24 is a diagram showing an exemplary rotating three-dimensional graphics model;

[0037] Fig.25 is a diagram showing an exemplary modified second image representation; and

[0038] Fig.26is a block diagram of an exemplary computing device for use in accordance with the present disclosure. DETAILED DESCRIPTION

[0039] For convenience, identical or equivalent elements in each embodiment shown in the drawings are marked with the same figure numerals. Certain terms used in the following description are for convenience only and are not restrictive. The words "right", "left", "top" and "bottom" specify directions in the drawings for reference. The words "inward", "inwardly", "outwardly" and "outwardly" refer to directions toward and away from the geometric center of the device and / or its designated parts, respectively. The terms are intended to include the above-mentioned words, derivatives of the above-mentioned words, and words with similar meanings in a non-limiting manner.

[0040] See first Figure 1 , body tissues, such as first anatomical structure segments and second anatomical structure segments 102, 104, can be aligned and / or oriented to promote healing or healing between body tissues. The anatomical structures may include, for example, anatomical tissues and artificial anatomical implants. The anatomical tissues may include, for example, bones or other tissues in the body. The alignment and / or orientation of body tissues can be achieved by connecting the body tissues to an adjustable fixation device, such as an orthopedic fixator 100. The orthopedic fixator may include an external fixation device, which includes a plurality of discrete fixation device members that are always outside the patient's body but are attached to corresponding discrete body tissues (e.g., with minimally invasive attachment members). The fixation device may include, for example, a distraction osteogenesis ring system, a hexagonal frame, or a Taylor space bracket. By adjusting the spatial positioning of the fixation members relative to each other, the attached corresponding body tissues can be reoriented and / or otherwise aligned with each other, for example, to promote healing between body tissues during recovery. The use of external orthopedic fixators in combination with the image analysis and localization techniques described herein is advantageous in the following application scenarios: measurement and manipulation of body tissue cannot be performed directly, limited or minimal trauma to body tissue is desired, etc. Some examples of orthopedic fixators and their use for correcting deformities of anatomical segments and techniques for performing imaging analysis of fixators and anatomical segments are described in U.S. Patent No. 9,642,649, entitled "ORTHOPEDIC FIXATION WITH IMAGERY ANALYSIS," issued on May 9, 2017, which is hereby incorporated by reference in its entirety.

[0041] The fixator members can be connected to each other by adjusting members, which are configured to facilitate the spatial repositioning of the fixator members relative to each other. For example, in the illustrated embodiment, the plastic surgery fixator 100 includes a pair of fixator members in the form of an upper fixing ring 106 and a lower fixing ring 108. The construction mode of the fixing rings 106 and 108 can be the same or different. For example, the diameters of the fixing rings 106 and 108 can be the same or different. Similarly, the fixing rings 106 and 108 can be constructed according to different cross-sectional diameters, thicknesses, etc. It should be understood that the fixator members of the plastic surgery fixator 100 are not limited to the upper and lower fixing rings 106 and 108 shown, and the plastic surgery fixator 100 can have other configurations. For example, additional fixing rings can be provided and interconnected with the fixing rings 106 and / or 108. It should also be understood that the geometry of the fixator member is not limited to the ring, and any other suitable geometry can be used alternatively to construct at least one fixator member, such as all fixator members.

[0042] The first and second anatomical structure sections 102, 104 may be rigidly attached to upper and lower fixation rings 106, 108, respectively, wherein attachment members may be mounted to the upper and lower fixation rings 106, 108. For example, in the illustrated embodiment, the attachment members are provided in the form of attachment rods 110 and attachment lines 112.

[0043] The rod 110 and the wire 112 extend between the proximal end attached to the mounting member 114 mounted to the fixing ring 106, 108 and the opposite distal end inserted or otherwise fixed to the anatomical structure segment 102, 104. For example, by placing the mounting member 114 into the threaded hole defined by the fixing ring, the mounting member can be removably mounted to the fixing ring 106, 108 at a predetermined position along the edge of the fixing ring 106, 108. For each fixing ring 106, 108, the mounting member 114 can be mounted to the upper surface of the ring, the lower surface of the ring, or any combination thereof. It should be understood that the attachment member is not limited to the configuration of the illustrated embodiment. For example, any number of attachment members (such as the rod 110 and wire 112 shown and any other members) can be used to fix the anatomical structure segment to the corresponding fixator member as needed. It should also be appreciated that one or more of the attachment members (eg, rod 110 and / or wire 112 ) may alternatively be configured to mount directly to the retaining rings 106 , 108 without the use of mounting members 114 .

[0044] The upper and lower fixing rings 106, 108 can be connected to each other by at least one adjustment member, such as a plurality of adjustment members. At least one (such as, all) adjustment members can be configured to allow the spatial positioning of the fixing rings to be adjusted relative to each other. For example, in the illustrated embodiment, the upper and lower fixing rings 106, 108 are connected to each other with a plurality of adjustment members provided in the form of length-adjustable stays 116. It should be understood that the construction of the orthopedic fixator 100 is not limited to the six stays 116 of the illustrated embodiment, and more or fewer stays can be used as needed.

[0045] Each adjustable-length stay 116 may include opposing upper and lower stay arms 118, 120. Each of the upper and lower stay arms 118, 120 has a proximal end disposed in a coupling member or sleeve 122; and an opposing distal end coupled to a universal joint 124, which is mounted to the upper and lower fixing rings 106, 108, respectively. The universal joints of the illustrated embodiment are disposed in pairs at even intervals on the edges of the upper and lower fixing rings 106, 108, but may alternatively be placed in any other position of the fixing ring as desired.

[0046] The proximal ends of the upper and lower strut arms 118, 120 of each strut 116 may be defined with threads, which are configured to be received by complementary threads defined in the sleeve 122, so that when the proximal ends of the upper and lower strut arms 118, 120 of the strut 116 are received in the corresponding sleeve 122, rotating the sleeve 122 will cause the upper and lower strut arms 118, 120 to translate within the sleeve 122, thereby causing the strut 116 to lengthen or shorten with the direction of rotation. Therefore, the length of each strut 116 can be adjusted independently relative to the remaining struts. It should be understood that the adjustment member is not limited to the length-adjustable struts 116 of the illustrated embodiment, and the adjustment member can be alternatively constructed as desired, such as by one or more alternative geometric shapes, alternative length adjustment mechanisms, etc.

[0047] The length-adjustable struts 116 and the universal joints 124 (the length-adjustable struts are mounted to the upper and lower fixing rings 106, 108 via universal joints) allow the orthopaedic fixator 100 to function very similarly to a Stewart platform, and more specifically, to a distraction osteogenesis ring system, a hexagonal rack, or a Taylor space frame. That is, by adjusting the length of the struts 116, the spatial positioning of the upper and lower fixing rings 106, 108 can be changed, thereby changing the spatial positioning of the anatomical structure segments 102, 104. For example, in the illustrated embodiment, the first anatomical structure segment 102 is attached to the upper fixing ring 106, and the second anatomical structure segment 104 is attached to the lower fixing ring 108. It should be understood that the attachment of the first and second anatomical structure segments 102, 104 to the upper and lower fixation rings 106, 108 is not limited to the illustrated embodiment (e.g., wherein the central longitudinal axes L1, L2 of the first and second anatomical structure segments 102, 104 are substantially perpendicular to the respective planes of the upper and lower fixation rings 106, 108), and that the surgeon has complete flexibility in aligning the first and second anatomical structure segments 102, 104 within the upper and lower fixation rings 106, 108 when constructing the orthopaedic fixator 100.

[0048] By changing the length of one or more struts 116, the upper and lower fixation rings 106, 108 and thus the anatomical structure segments 102, 104 can be repositioned relative to each other so that their respective longitudinal axes L1, L2 are substantially aligned with each other and so that their corresponding fracture ends 103, 105 are adjacent to each other to promote healing during the recovery process. It should be understood that the adjustment of the struts 116 is not limited to the length adjustment described herein, and the struts 116 can be adjusted in different ways as needed. It should also be understood that adjusting the position of the fixator member is not limited to adjusting the length of the length-adjustable struts 116, and the positioning of the fixator members relative to each other can be alternatively adjusted, for example, according to the type and / or number of adjustment members connected to the fixation device.

[0049] Repositioning of the fixture member of a plastic surgery fixation device (such as, plastic surgery fixator 100) can be used to correct displacement of angulation, translation, rotation or any combination thereof within body tissue. Fixation devices used with the technology described herein, such as, plastic surgery fixator 100, can correct multiple such displacement defects individually or simultaneously. However, it should be understood that the fixation device is not limited to the plastic surgery fixator 100 shown, and the fixation device may have other structures as needed. For example, the fixation device may include additional fixing members, may include fixing members with alternative geometric shapes, may include more or fewer adjustment members, may include adjustment members of alternative structures, or any combination thereof.

[0050] See now Figure 2, exemplary imaging of the fixation device will now be described in detail. The images may be taken by the same or different imaging techniques. For example, the images may be acquired by the following techniques: x-ray imaging, computed tomography, magnetic resonance imaging, ultrasound imaging, infrared imaging, photography, fluoroscopic imaging, visible spectrum imaging, or any combination thereof.

[0051] The images may be captured at any position and / or orientation relative to each other and relative to the fixator 100 and the anatomical structure segments 102, 104. In other words, there is no requirement that the images be captured orthogonal to each other or aligned with the patient's anatomical axis, thereby providing the surgeon with nearly complete flexibility in positioning the imager 130. Preferably, the images 126, 128 are captured from different directions or orientations such that the images do not overlap. For example, in the illustrated embodiment, the image planes of the image pairs 126, 128 are not perpendicular to each other. In other words, the angle α between the image planes of the images 126, 128 is not equal to 90 degrees, such that the images 126, 128 are not orthogonal to each other. Preferably, at least two images are acquired, but capturing additional images may improve the accuracy of the method.

[0052] The images 126, 128 may be captured by one or more imaging sources or imagers, such as an x-ray imager 130 and / or corresponding image capture devices 127, 129. The images 126, 128 may be x-ray images captured by a single repositionable x-ray imager 130, or may be captured by separately positioned imagers 130. Preferably, the position of the image capture devices 127, 129 and / or imagers 130 relative to a spatial origin 135 in three-dimensional space (as described in more detail below) is known. The imager 130 may be positioned and / or oriented manually under the control of a surgeon, or may be positioned automatically, such as with the assistance of software, or any combination thereof. The fixture 100 may also have a corresponding fixture origin 145.

[0053] Reference now Figure 3A and Figure 3B , an exemplary process for controlling manipulation of a fixation device including a ring and a strut to correct an anatomical deformity of a first anatomical segment and a second anatomical segment will now be described in detail. Specifically, at operation 310, the first anatomical segment and the second anatomical segment are attached to the fixation device, for example, Figure 1 In operation 312, first and second images of the fixation device and the attached first and second anatomical structure segments are captured, for example, Figure 2 As shown and described in detail above.

[0054] We will now introduce a treatment technique called fluoroscopic stent matching. Figure 3A and Figure 3B Other operations (eg, operations 314 to 342) in the process of wherein images (eg, post-operative x-ray images) can be used together with the brace to generate deformity parameters and installation parameters for a brace adjustment plan. For example, referring now to Figure 4 , an exemplary treatment planning technique selection interface 400-A is shown. Figure 4 In the example of FIG. 4 , the user has selected option 401 to use perspective frame matching (PFM) technology and will now refer to Figure 5-13 Provide a detailed introduction.

[0055] See again Figure 3A At operation 314, configuration information associated with the fixture is received, such as via one or more graphical user interfaces of the computing system. In some examples, the configuration information may include one or more geometric features (e.g., size, length, diameter, etc.) of one or more elements of the fixture (e.g., stays, hinges, loops, etc.). In some examples, the configuration information may include information such as loop type (e.g., full loop, foot plate, etc.), indications of mounting points (e.g., loop holes) for mounting the stays, and other information. In some examples, the configuration information may also include marking element information, such as marking elements mounted to fixture components (such as stays, hinges, and loops). Referring now to FIG. Figure 5 , showing an exemplary configuration information input interface 500. As shown, the interface 500 includes ring type indicators 501 and 502, which in this example are drop-down menus that can be used to select the ring type of the proximal ring and the distal ring, respectively. When indicators 501 and 502 are set to the "Full" option, it indicates that the proximal ring and the distal ring are both full rings. The interface 500 also includes diameter indicators 503 and 504, which in this example are drop-down menus that can be used to select the diameter or length of the proximal ring and the distal ring, respectively.

[0056] The interface 500 also includes controls for inputting brace information. Specifically, the interface 500 includes six drop-down menus 512, each of which can be used to indicate the size of the corresponding brace. A global brace size indicator 511 can also be used to globally select a size for all six braces. A length selector 513 can be used to select the length of the corresponding brace. A length indicator 514 can be used to provide a visual illustration of the length of the corresponding brace. It should be noted that the length indicator 514 may not necessarily show the actual accurate length of each brace, but rather represents the comparative length of the braces relative to each other.

[0057] A "Save and Update" button 516 may be selected to save and update the configuration information values ​​shown in the interface 500. In some examples, selecting button 516 may cause the interface 500 to display and / or update a graphical representation 520 of a fixture that is generated at least in part based on the input configuration information. The graphical representation 520 may be displayed by one or more graphical user interfaces of a computing system. As shown, the graphical representation 520 includes six braces that may be color-coded with a plurality of colors for ease of distinction. For example, in some cases, each of the braces (or at least two of the braces) is displayed in a color different from one another. The braces in the graphical representation 520 may have dimensions, lengths, mounting points, and other features corresponding to the input configuration information. The graphical representation 520 also shows retaining rings that may have diameters / lengths, ring types, and other features corresponding to the input configuration information. For example, the graphical representation 520 may improve efficiency and reliability, for example, by providing a user with visual confirmation of information input into the interface 500, for example, to enable quick and easy identification of errors or other problems.

[0058] In operation 316, images of the fixation device and the first anatomical structure segment and the second anatomical structure segment attached to the fixation device are displayed, for example, through one or more graphical user interfaces of the computing system. The displayed images may include the images captured in operation 312, for example, images captured by the following techniques: x-ray imaging, computer tomography, magnetic resonance imaging, ultrasound imaging, infrared imaging, photography, fluoroscopic imaging, visible spectrum imaging, or any combination thereof. The techniques for acquiring images of the fixation device and the first anatomical structure segment and the second anatomical structure segment are described in detail above and will not be repeated here. As described above, the images obtained and displayed do not have to be orthogonal to each other. Now referring to Figure 6 , shows an exemplary first image information input interface 600. As shown in the figure, the interface 600 includes images 601-A and 601-B, which show the fixation device and the first anatomical structure segment and the second anatomical structure segment at different angles. Figure 6 In the example of , image 601-A corresponds to an anteroposterior (AP) view, and image 601-B corresponds to a lateral (LAT) view. In some examples, displayed image 601-AB can be loaded and saved in a computer memory, such as an image library, database, or other local collection storing images. Displayed image 601-AB can then be selected and retrieved, acquired, and / or received from the memory for display.

[0059] In operation 318, first image information is received, for example, through one or more graphical user interfaces of a computing system. The first image information may include an indication of one or more locations of at least a portion of one or more elements of a fixture within an image. For example, the first image information may include one or more indications of the location of a stay, hinge, ring, and other fixture elements. In some examples, the first image information may also include location information of a marker element within the image, for example, a marker element mounted to a fixture component such as a stay, hinge, and ring. In some cases, the first image information may include a point representing a hinge location and / or a line or vector representing a stay location. In some examples, the first image information may be input into the computing system by, for example, selecting or indicating one or more locations within a displayed image through a mouse, keyboard, touch screen, or other user input device. Specifically, a user may use one or more input devices to select a point or other location in an image, draw a line, a circle, and generate other graphical indications within the image. For example, in some cases, a user may generate a point or a small circle at a specific location of the image to indicate the location of the hinge within the image (e.g., a center point). As another example, in some cases, a user may generate lines and / or vectors within an image to indicate the location and / or length of a strut within the image.

[0060] For example, Figure 6As shown, interface 600 includes six "AP View" strut indicator buttons 611-A corresponding to each of the six struts of the fixture shown in "AP View" image 601-A. Each button 611-A includes text indicating the corresponding strut number (i.e., strut 1, strut 2, strut 3, strut 4, strut 5, strut 6). The user can select button 611-A to indicate a strut for which the user will provide first image information (e.g., hinge location, strut location, etc.) in "AP View" image 601-A. For example, in some cases, to provide first image information for strut 1 in "AP View" image 601-A, the user can first select the topmost strut indicator button 611-A (labeled with text as "Strut 1") to prompt the software user that first image information for strut 1 will be provided within "AP View" image 601-A. In some cases, the strut indicator button 611-A for strut 1 can be automatically pre-selected for the user. After selecting (or automatically pre-selecting) the strut indicator button 611-A for strut 1, the user may proceed to draw (or otherwise indicate) a diagram of strut 1 within the "AP View" image 601-A. For example, in some cases, the user may select a location 621 (e.g., a center point) of the proximal hinge of strut 1 within image 601-A using a mouse or other input device. In some examples, the user may then select a location 622 (e.g., a center point) of the distal hinge of strut 1 within image 601-A using a mouse or other input device. In some examples, the user may indicate the location and / or length of strut 1 by selecting the location of the proximal and distal hinges and / or the endpoints of a line or vector representing the location and / or length of strut 1. For example, as Figure 6 As shown, the software can generate points or circles at the locations 621 and 622 of the proximal and distal hinges selected by the user in image 601-A. In addition, the software can generate a straight line 623, which represents the position and / or length of the strut 1 connecting the points or circles at the locations 621 and 622 and the positions and / or lengths of the proximal and distal hinges selected by the user in image 601-A. The user can also use any other suitable input technology to indicate the position and / or length of the strut 1 in image 610-A, such as, by dragging a mouse, using a finger and / or a pen on a touch screen, keypad or other device to generate the straight line 623. In some examples, the above process can be repeated to draw points representing the proximal and distal hinges and straight lines representing the position and / or length of each of the six struts in the "AP view" image 601-A. Additionally, the above process may be repeated via the "LAT View" strut indicator button 611-B to draw points representing the proximal and distal hinges and lines representing the location and / or length of each of the six struts in the "LAT View" image 601-B.

[0061] In some examples, the first image information generated within images 601-A and 601-B may include color-coded graphical representations of the struts, for example, so that the graphical representations can be more clearly associated with their corresponding struts. Figure 6 In the example, the image illustration (e.g., dot, circle, and / or line) of brace 1 in images 601-A and 601-B may be colored red. This may match the brace icon (which may also be colored red) displayed in brace indicator buttons 611-A and 611-B for brace 1 (displayed to the right of the text "Brace 1" in buttons 611-A and 611-B). For another example, in Figure 6 In the example, the graphical representation (e.g., dot, circle, and / or line) of brace 3 in images 601-A and 601-B may be colored yellow. This may match the brace icon (which may also be colored yellow) displayed in brace indicator buttons 611-A and 611-B for brace 3 (displayed to the right of the text "Brace 3" in buttons 611-A and 611-B).

[0062] Figure 6 An "AP view" close-up assistance checkbox 616-A and a "LAT view" close-up assistance checkbox 616-B are included, for example, provided via one or more graphical interfaces of the computing system. Selecting checkboxes 616-A and 616-B may enable a close-up view of multiple areas of images 601-A and 601-B that are adjacent to the proximal and distal hinges of the strut that the user is currently drawing. This may enable a more accurate indication of the location of the hinges (e.g., the center point). Referring now to Figure 7 , the close-up assistance interface 700 shows another "AP view" image 701, where the close-up assistance is selected to provide a proximal hinge close-up assistance view 702 and a distal hinge close-up assistance view 703. As shown, the proximal hinge close-up assistance view 702 provides an enlarged view of an area of ​​the "AP view" image 701 associated with the proximal hinge, while the distal hinge close-up assistance view 703 provides an enlarged view of an area of ​​the "AP view" image 701 associated with the distal hinge. In the proximal hinge close-up assistance view 702, the user can manipulate (e.g., drag) the position of the point / circle 721 to more accurately depict the center point of the proximal hinge. In the distal hinge close-up assistance view 703, the user can also manipulate (e.g., drag) the position of the point / circle 722 to more accurately depict the center point of the distal hinge. It should be understood that corresponding close-up assistance views similar to views 702 and 703 can also be provided for the corresponding "LAT view" image, such as through one or more graphical interfaces of the computing system.

[0063] See again Figure 6, to the right of button 611-A are six proximal hinge selector buttons 612-A. Additionally, to the right of button 612-A are six distal hinge selector buttons 613-A. Additionally, to the right of button 613-A are six strut line selector buttons 614-A. In some examples, buttons 612-A and / or 613-A may be selected to calculate the position and orientation of the first anatomical structure segment and the second anatomical structure segment and the ring of the fixation device in three-dimensional space using the position (e.g., center point) of the proximal and / or distal hinges indicated in the “AP View” image 601-A (see operation 322). Additionally, in some examples, buttons 612-A and / or 613-A may be selected to calculate the position and orientation of the first anatomical structure segment and the second anatomical structure segment in three-dimensional space using the lines or vectors indicating the position and / or length of the struts indicated in the “AP View” image 601-A. Similarly, buttons 612-B, 613-B, and 614-B may be used to select: using the positions of the proximal and / or distal hinge or strut lines indicated in the "LAT View" image 601-B to calculate the position and orientation of the first anatomical structure segment and the second anatomical structure segment in three-dimensional space.

[0064] See again Figure 3A In operation 320, second image information is received, for example, through one or more graphical user interfaces of a computing system. The second image information may include an indication of one or more locations of at least a portion of the first anatomical structure segment and the second anatomical structure segment within the image. In some examples, the second image information may include an indication of a centerline of the first anatomical structure segment and the second anatomical structure segment and / or one or more reference points (e.g., endpoints) of the first anatomical structure segment and the second anatomical structure segment. In some examples, the second image information may also include an indication of the location of a marker element, for example, implanted or otherwise associated with the first anatomical structure segment and the second anatomical structure segment. In some examples, the second image information may be input into the computing system by, for example, selecting or indicating one or more locations within the displayed image through a mouse, keyboard, touch screen, or other user input device. Specifically, a user may use one or more input devices to select a point or other location in the image, draw a line, a circle, and generate other graphical indications within the image. For example, in some cases, a user may generate a point or a small circle at a specific location of the image to indicate one or more reference points (e.g., endpoints) of the first anatomical structure segment and the second anatomical structure segment within the image. As another example, in some cases, the user may generate a straight line within the image to indicate the center lines of the first anatomical structure segment and the second anatomical structure segment within the image.

[0065] See now Fig. 8A, an exemplary second image information input interface 800 is shown. As shown, interface 800 includes an "AP view" image 601-A and a "LAT view" image 601-B. In addition, interface 800 includes buttons 801-808, which can be used to assist in indicating anatomical structure centerlines and reference points, as described below. Specifically, buttons 801 and 805 can be selected to indicate proximal anatomical structure reference points in the "AP view" and "LAT view", respectively. Buttons 802 and 806 can be selected to indicate distal anatomical structure reference points in the "AP view" and "LAT view", respectively. Buttons 803 and 807 can be selected to indicate the proximal anatomical structure centerline in the "AP view" and "LAT view", respectively. Buttons 804 and 808 can be selected to indicate the distal anatomical structure centerline in the "AP view" and "LAT view", respectively. For example, as Fig. 8A As shown, the user can select button 807 and then draw a proximal anatomical structure centerline 831 in the "LAT view" image 601-B through one or more input devices. In some examples, the centerline 831 can be painted red. In addition, the software generates and displays two wires 832 on both sides of the red centerline. In some examples, the wires 832 can be painted green. These wires 832 can be displayed when the user draws the centerline 831 to assist the user in locating the center of the anatomical structure segment. The wires 831 can be generated at locations equidistant from each side of the centerline 832, and can assist the user by, for example, potentially allowing the user to match (or nearly match) the wires 832 to both sides of the anatomical structure segment. As shown in FIG. Figure 8B As shown, the user may select button 808 and then, through one or more input devices, draw a centerline 841 of the distal anatomical structure in the "LAT view" image 601-B. Figure 8C As shown, the user may select button 803 and then, through one or more input devices, draw a centerline 851 of the proximal anatomical structure within the “AP view” image 601-A. Fig.8D As shown, the user may select button 804 and then, through one or more input devices, draw a centerline 861 of the distal anatomical structure within the “AP view” image 601-A. Figures 8B-8D As shown, guide wire 832 may also be displayed to assist in drawing center lines 841 , 851 , and 861 .

[0066] like Fig. 8EAs shown, the user can select button 805 and then, through one or more input devices, indicate a reference point (e.g., an endpoint) of the proximal anatomical structure within the "LAT View" image 601-B. As shown, in the "LAT View" image 601-B, the user has indicated a reference point 811 at the endpoint of the proximal anatomical structure segment. In addition, after indicating the reference point 811, the software can generate and display a corresponding dashed reference line 812 in the "AP View" image 601-A. Reference line 812 is a line drawn across the "AP View" image 601-A that passes through the location of the "LAT View" proximal reference point 811 within the "AP View" image 601-A. Therefore, reference line 812 can assist the user in determining the location of the corresponding "AP View" proximal reference point, which is typically at the intersection of reference line 812 and "AP View" proximal centerline 851 within the "AP View" image 601-A. As shown Figure 8F As shown, the user may select button 801 and then, through one or more input devices, indicate a reference point (e.g., an endpoint) of the proximal anatomical structure within the “AP View” image 601-A. In this example, the “AP View” proximal anatomical structure reference point 814 is shown within the “AP View” image 601-A at the intersection of reference line 812 and the “AP View” proximal centerline 851. The software may then generate and display a corresponding dashed reference line 813 in the “LAT View” image 601-B. Reference line 813 is a line drawn across the “LAT View” image 601-B that passes through the location of the “AP View” proximal reference point 814 within the “LAT View” image 601-B. Reference line 813 may assist the user in confirming that the “AP View” reference point 811 has been correctly placed by displaying its alignment relative to the “LAT View” reference point 814.

[0067] like Figure 8G As shown, the user can select button 806 and then, through one or more input devices, indicate a reference point (e.g., an endpoint) of the distal anatomical structure within the "LAT View" image 601-B. As shown, in the "LAT View" image 601-B, the user has indicated a reference point 815 at the endpoint of the distal anatomical structure segment. In addition, after indicating the reference point 815, the software can generate and display a corresponding dashed reference line 816 in the "AP View" image 601-A. Reference line 816 is a line drawn across the "AP View" image 601-A that passes through the location of the "LAT View" distal reference point 815 within the "AP View" image 601-A. Therefore, reference line 816 can assist the user in determining the location of the corresponding "AP View" distal reference point, which is generally at the intersection of reference line 816 and the "AP View" distal centerline within the "AP View" image 601-A. As shown Figure 8HAs shown, the user may select button 802 and then, through one or more input devices, indicate a reference point (e.g., an endpoint) of the distal anatomical structure within the “AP View” image 601-A. In this example, the “AP View” distal anatomical structure reference point 817 is shown within the “AP View” image 601-A at the intersection of reference line 816 and the “AP View” distal centerline. The software may then generate and display a corresponding dashed reference line 818 in the “LAT View” image 601-B. Reference line 818 is a line drawn across the “LAT View” image 601-B that passes through the location of the “AP View” distal reference point 817 within the “LAT View” image 601-B. Reference line 818 may assist the user in confirming that the “AP View” reference point 817 has been correctly placed by showing its alignment relative to the “LAT View” reference point 815.

[0068] See again Figure 3A , in operation 322, the position and orientation of the first anatomical structure segment and the second anatomical structure segment and the ring of the fixation device in three-dimensional space are determined. For example, in some cases, imaging scene parameters related to the fixator 100, the anatomical structure segments 102, 104, one or more imagers 130 and the image capture devices 127, 129 are obtained. The imaging scene parameters can be used to construct a three-dimensional illustration of the positioning of the anatomical structure segments 102, 104 in the fixator 100, as described in more detail below. One or more of the imaging scene parameters may be known. The unknown imaging scene parameters can be obtained, for example, by mathematically comparing the following items: the position of the fixture element illustration in the two-dimensional space of the x-ray images 126, 128; and the three-dimensional position of these elements in the geometry of the fixator 100. In a preferred embodiment, the imaging scene parameters can be calculated by a pinhole camera or a perspective camera model of the computational model. For example, the imaging scene parameters can be determined digitally by matrix algebra, as described in more detail below.

[0069] Imaging scene parameters may include, but are not limited to: image pixel scaling factor, image pixel aspect ratio, image sensor skew factor, image size, focal length, position and orientation of the imaging source, position of the principal point (defined as the point on the plane of the respective image 126, 128 closest to the respective imager 130), position and orientation of the various elements of the fixture 100, position and orientation of the respective image receptors, and position and orientation of the imaging source lens.

[0070] In a preferred embodiment, at least some (such as, all) imaging scene parameters can be obtained by comparing the following items: the illustrated positions of specific components or the illustrated positions of the fixture elements of the fixture 100 in the two-dimensional space of the images 126, 128; the corresponding positions of these same fixture elements in the actual three-dimensional space. The fixture elements include components of the plastic surgery fixture 100, and are preferably components that are easily identified in the images 126, 128. Points, lines, cones, etc. or any combination thereof can be used to describe the corresponding geometric shapes of the fixture elements. For example, the illustrations of the fixture elements used in the comparison process can include the center line of one or more of the adjustable length struts 116, the center point of the universal joint 124, the center point of the mounting member 114, etc.

[0071] The fixture element may also include a marking element that is different from the above-mentioned components of the fixture 100. The marking element can be used in the comparison process as a supplement or alternative to the components of the fixture 100. Before imaging, the marking element can be installed at a specific component position of the fixture 100, can be embedded in the component of the fixture 100, or any combination thereof. The marking element can be configured to provide higher visibility of the images 126 and 128 when compared with the visibility of other components of the fixture 100. For example, the marking element can be constructed with different materials, such as, a radiopaque material, or can be constructed with a geometric figure that is easily distinguishable from other components of the fixture 100 in the images 126 and 128. In an exemplary embodiment, the marking element may have a specified geometric shape that corresponds to its corresponding position on the fixture 100.

[0072] The fixture element can be identified for use in the comparison process. For example, the first image information received in operation 318 and described in detail above can be used to indicate the position of the fixture element within the images 126, 128. In some examples, the position of the fixture element in the two-dimensional space of the images 126, 128 can be determined relative to the following items: the local origin 125 defined in the imaging plane of the images 126, 128. The local origin 125 is used as a "zero point" to determine the position of the fixture element in the images 126, 128. The position of the fixture element can be defined by their corresponding x-coordinates and y-coordinates relative to the corresponding local origin 125. The position of the local origin 125 within the corresponding image can be arbitrary as long as it is in the image plane. Typically, the origin is located at the center of the image or at the corner of the image, such as, the lower left corner. It should be understood that the position of the local origin is not limited to the local origin 125 shown, and the local origin 125 can be additionally defined at any other position.

[0073] In some examples, a transformation matrix P corresponding to each of the images 126, 128 may then be calculated. The transformation matrix may be used to map the position coordinates of one or more corresponding fixture elements in the actual three-dimensional space to the corresponding position coordinates of the fixture elements in the two-dimensional space of the corresponding images 126, 128. It should be understood that the same one or more fixture elements need not be used when comparing two images 126, 128. For example, the fixture elements used to construct the transformation matrix associated with image 126 may be the same or different than the fixture elements used to construct the transformation matrix associated with image 128. It should also be understood that the greater the number of fixture elements used to calculate the transformation matrix, the more accurate the method. The following formula may represent this operation:

[0074] (1).

[0075] symbol x and y Denotes the position coordinates of the fixture element point in the two-dimensional space of the images 126, 128 relative to the local origin 125. Symbol X , Y and Z represents the corresponding position coordinates of the fixture element points in the actual three-dimensional space relative to the spatial origin 135. In the illustrated embodiment, the following point has been designated as the spatial origin 135: the point corresponding to the center of the plane defined by the upper surface of the upper fixing ring 106. The illustrated matrix P may be at least four elements wide and three elements high. In a preferred embodiment, the elements of the matrix P may be calculated by solving the following matrix formula:

[0076] (2).

[0077] vector p can contain eleven elements representing the values ​​of the matrix P. The following formula gives the vector p And the arrangement of elements in the matrix P:

[0078] (3)

[0079] (4).

[0080] In this preferred embodiment, the twelfth element of the matrix P can be p 12 Set to a value of 1. The matrices A and B can be combined using the two-dimensional and three-dimensional information of the fixture element. For each point representing a corresponding fixture element, two rows of matrices A and B can be constructed. The following formula gives the values ​​of the two rows added to the matrices A and B for each point of the fixture element (e.g., the center point of the corresponding universal joint 124):

[0081] .

[0082] symbol X , Y and Z Indicates the position coordinate value of the fixture element point in the actual three-dimensional space relative to the space origin 135, symbol x and y It represents the position coordinate value of the corresponding fixture element point relative to the local origin 125 in the two-dimensional space of the corresponding image 126, 128.

[0083] For each line representing a corresponding fixture element, two rows of matrices A and B may be constructed. The following formula gives the values ​​added to the two rows of matrices A and B for each line of fixture elements (eg, the centerline of a corresponding adjustable length stay 116):

[0084] .

[0085] symbol X , Y and Z Indicates the position coordinate value of a point on the fixture element line in the actual three-dimensional space relative to the space origin 135. Symbol dX , dY and Z Indicates the gradient value of this line in the actual three-dimensional space. Symbol a , b and c represents a constant that defines a line in the two-dimensional space of the corresponding images 126, 128. For example, a , b and c It can be calculated using two points belonging to a line in the corresponding images 126, 128. In a preferred embodiment, unless the line is a vertical line (in which case, b is zero), otherwise b is assumed to be 1. a , b and c With the corresponding image coordinates x and y The correlation is given by the following formula:

[0086] .

[0087] Equation (2) may be over-constrained by six or more fixture elements (e.g., adjustable length struts 116). It should be appreciated that it is not necessary to have all fixture elements visible in a single image of images 126, 128 in order to obtain the matrix P. It should also be appreciated that if one or more of the above-mentioned imaging scene parameters are known, these known parameters may be used to reduce the minimum number of fixture elements required to constrain equation (2). For example, such information may be obtained from the DICOM image header of a modern imaging system. Preferably, a singular value decomposition method or a least squares method may be used to solve equation (2) to obtain the vector p The value of .

[0088] In some examples, the transformation matrix can then be decomposed into imaging scene parameters. The following formula can be used to relate the matrix P to the matrices E and I:

[0089] .

[0090] It should be understood that additional terms may be introduced when decomposing the matrix P. For example, the method proposed by Tsai (see "AVersatile Camera Calibration Technique for High-Accuracy 3D Machine Vision Metrology Using of-the-shelf TV Cameras and Lenses", IEEE Journal of Robotics & Automation, RA-3, No. 4, 323-344, August 1987, which is incorporated herein by reference in its entirety) can be used to correct radially distorted images 126, 128.

[0091] The matrices E and I contain the imaging scene parameters. The following formula represents the composition of matrix I:

[0092] .

[0093] symbol sx and sy A value representing an image coordinate scaling factor (for example, a pixel scaling factor). Symbol f represents the focal length, which corresponds to the value of the shortest distance between the corresponding imaging source 130 and the corresponding image 126, 128 plane. tx and ty represents the coordinates of the principal point relative to the local origin 125 of the corresponding image 126, 128. The following formula represents the composition of the matrix E:

[0094] .

[0095] symbol ox , o y and o z A value indicating the position of the fixture 100 in the actual three-dimensional space. Symbol r 1 - r 9 The orientation of the fixture 100 is described. These values ​​can be composed into a three-dimensional rotation matrix R represented by the following formula:

[0096] .

[0097] The following methods can be used to obtain the values ​​of the matrices E and / or I: the method of Trucco and Verri, as described in "Introductory Techniques of 3-D Computer Vision" (Prentice Hall, 1998); or the method of Hartley, as described in "Euclidian Reconstruction from Uncalibrated Views" (Applications of Invariance in Computer Vision, pp. 237-256, Springer Verlag, Berlin Heidelberg, 1994), which are incorporated herein in their entirety. Using the obtained values ​​of the matrices E and I, the complete three-dimensional imaging scene of the fixator 100 and the anatomical structure segments 102, 104 can be reconstructed.

[0098] For example, Figure 2 An exemplary three-dimensional imaging scene reconstructed by x-ray images 126, 128 is shown. In the illustrated embodiment, x-rays are emitted from an x-ray imager 130. It should be understood that the x-ray imager 130 can be the same or different imagers, as described above. The x-rays emitted from the imager 130 are received by the corresponding imaging device, thereby capturing images 126, 128. Preferably, the position of the imager 130 relative to the local origin 125 is known.

[0099] In some examples, the images 126, 128 and the imaging scene parameters can then be used to obtain the position and / or orientation of the anatomical structure segments 102, 104 in three-dimensional space. The position and / or orientation data obtained can be used to formulate a treatment plan for the patient, for example, to change the orientation and / or position of the fractured first anatomical structure segment and the second anatomical structure segment 102, 104, so as to promote healing between the anatomical structure segments 102, 104, as described in more detail below. It should be understood that the application scenarios of the methods and techniques described herein are not limited to repositioning fractured anatomical structures, and the surgical plastic fixation process using imaging analysis can be used in any other type of fixation as needed, for example, lengthening anatomical structures, correction of anatomical defects, etc.

[0100] In some examples, anatomical elements of the illustration including a particular portion (e.g., an anatomical feature) of the anatomical segments 102, 104 may then be identified and their locations within the images 126, 128 determined. For example, the locations of the first anatomical segment and the second anatomical segment within the images 126, 128 may be indicated by the second image information received at operation 320 and described in detail above. In some examples, the locations of the anatomical elements may be determined relative to the respective local origins 125 of the images 126, 128.

[0101] The anatomical elements can be used to construct a three-dimensional illustration of the position and / or orientation of the anatomical segments 102, 104. Preferably, the anatomical elements are easily identifiable in the images 126, 128. Points, lines, cones, etc., or any combination thereof, can be used to describe the corresponding geometric shapes of the anatomical segments. For example, in the illustrated embodiment, points 134 and 136 representing the fracture ends 103, 105 of the anatomical segments 102, 104, respectively, are identified as anatomical elements in the images 126, 128.

[0102] The anatomical elements may also include marker elements that are implanted into the anatomical sections 102, 104 prior to imaging. The marker elements may be used in addition to or in place of the anatomical elements identified in the images 126, 128 described above. The marker elements may be configured to provide greater visibility of the images 126, 128 than the visibility of anatomical features of the anatomical sections 102, 104. For example, the marker elements may be constructed of a radiopaque material or may be constructed of an easily discernible geometric shape.

[0103] A three-dimensional representation 200 of the anatomical structure segments 102, 104 can be reconstructed. The three-dimensional representation can be constructed with or without a corresponding representation of the fixture 100. In the illustrated embodiment, pairs of rays, such as rays 138, 140 and 142, 144, can be constructed for the anatomical structure element points 134, 136, respectively. Each ray connects the anatomical structure element in one of the images 126, 128 to the corresponding imager 130. Each pair of rays can be analyzed for a common intersection, such as points 146, 148. The common intersections 146, 148 represent the corresponding positions of the anatomical structure element points 134, 136 in the three-dimensional representation of the anatomical structure segments 102, 104. Of course, more than one pair (such as, multiple pairs) of rays can be constructed, for example, if more than two images are taken. If the rays of a particular set do not intersect, the point closest to all the rays in the set can be used as the common intersection.

[0104] The position and / or orientation of the anatomical segments 102, 104 can be quantified or measured through the common intersection point (e.g., point 146, 148). For example, a straight line representing the centerline of the anatomical segments 102, 104 can be constructed and compared to the patient's anatomical axis. In addition, the spacing of the fracture ends 103, 105 of the anatomical segments 102, 104 can be quantified. Through these or similar techniques, the position and / or orientation of the anatomical segments 102, 104 can be determined. It should also be noted that in some examples, in addition to the position and orientation of the first anatomical segment and the second anatomical segment, the position and orientation of multiple rings (and / or other elements of the fixation device) in three-dimensional space can also be determined, for example, through any of the described techniques. For example, in some cases, based on the first image information and / or other information provided, the position of the ring within the image 126, 128 can be determined. In some examples, these positions can then be used to determine the position and orientation of the ring in three-dimensional space. Additionally, in some examples, configuration information of the fixture (such as ring diameter, strut length, and mounting information) may also be used to determine the position and orientation of the ring in three-dimensional space.

[0105] See now Figure 3B, in operation 324, one or more deformity parameters are calculated. The deformity parameters may include parameters related to the following deformities: deformities associated with the first anatomical structure segment and the second anatomical structure segment. For example, in some cases, the deformity parameters may include the amount of translation (e.g., lateral displacement, medial displacement, anterior displacement and / or posterior displacement), the degree of coronal angulation (e.g., valgus and / or varus), the degree of sagittal angulation, the amount of anatomical length that is too short and / or too long, the degree of clinical rotation deformity (e.g., internal rotation and / or external rotation), etc. In some examples, such as by the techniques described above in conjunction with operation 422, the deformity parameters can be calculated as part of the following process: determining the position and orientation of the first anatomical structure segment and the second anatomical structure segment, as described above in operation 422.

[0106] In operation 326, the deformity parameters calculated in operation 424 are displayed, for example, via one or more graphical user interfaces of the computing system. Fig. 9 , a deformity parameter interface 900 is shown. As shown, the interface 900 includes a plurality of fields 901-906 for displaying calculated values ​​of a plurality of exemplary deformity parameters, including "AP view" translation and coronal angulation, "LAT view" translation and sagittal angulation, the amount of anatomical structure length that is too short or too long, and the degree of clinical rotational deformity. Fig. 9 In the example of FIG. 9 , fields 901-905 each have a corresponding PFM badge 915 (including the text “PFM”) displayed to the left of each field 901-905. Each PFM badge 915 indicates that the value shown in the corresponding field 901-905 has been calculated by the software. The interface 900 allows a user to edit the deformity parameter value displayed in each field 901-906, for example, by entering a number in the field 901-906 and / or by using a numeric increment control 916 displayed to the right of each field 901-906. When the user edits the value calculated by the software, the PFM badge 915 adjacent to the corresponding field can be removed to indicate that the value of the field has been edited by the user. In some examples, after editing the value in one or more fields, the user can select a “Refresh Perspective Frame Matching Data” button 920 to restore each field to the software calculated value. Additionally, in some examples, after editing the values ​​in one or more fields, the user may select a “Save and Update” button 921 to recalculate the deformity parameters based on the edited values ​​provided by the user, for example, by repeating all or part of the calculations performed in operation 322 .

[0107] In operation 328, a graphical representation of the position and orientation of the first anatomical structure segment and the second anatomical structure segment is generated and displayed. The graphical representation of the position and orientation of the first anatomical structure segment and the second anatomical structure segment may be displayed via one or more graphical user interfaces of the computing system. Fig. 9 As shown, interface 900 includes a graphical representation 950 of the position and orientation of the first anatomical structure segment and the second anatomical structure segment. Graphical representation 950 includes: an illustration 931 of the proximal anatomical structure segment and an illustration 932 of the distal anatomical structure segment. In some examples, graphical representation 950 can be generated based at least in part on the position and orientation of the first anatomical structure segment and the second anatomical structure segment determined in operation 322. In some examples, when the user edits one or more deformity parameters and selects the "Save and Update" button 921, graphical representation 950 can also be adjusted to reflect the saved edits to the deformity parameters. For example, graphical representation 950 can improve efficiency and reliability, for example, by providing the user with visual confirmation information of information entered into interface 900, for example, to enable quick and easy identification of errors or other problems.

[0108] In operation 330, one or more installation parameters are calculated. The installation parameters may include parameters related to the installation of the reference ring of the fixator to the corresponding anatomical structure segment. For example, in some cases, the installation parameters may include an offset (e.g., lateral, medial, anterior, and / or posterior) (such as an offset of the center of the reference ring relative to the reference point), a tilt (e.g., proximally and / or distally), an axial offset, a main bayonet rotation, etc. In some examples, the installation parameters may be calculated as part of the following process: determining the position and orientation of the first anatomical structure segment and the second anatomical structure segment, such as described above in operation 322, such as by the techniques described above in conjunction with operation 322. It should be noted that for the process of FIG. 3, the reference ring does not have to be orthogonal relative to the corresponding anatomical structure segment to which the ring is installed. Therefore, in some examples, the reference ring may not be orthogonal relative to the corresponding anatomical structure segment to which the ring is installed.

[0109] In operation 432, the installation parameters calculated in operation 430 are displayed, for example, via one or more graphical user interfaces of the computing system. Fig.10 , an installation parameter interface 1000 is shown. As shown, the interface 1000 includes a plurality of fields 1001-1006 for displaying calculated values ​​of a plurality of exemplary installation parameters, including "AP view" offset and tilt, "LAT view" offset and tilt, axial offset, and primary bayonet rotation. Fig.10In the example of FIG. 1 , fields 1001-1006 each have a corresponding PFM badge 1015 displayed to the left of each field 1001-1006. Each PFM badge 1015 indicates that the value shown in the corresponding field 1001-1006 has been calculated by the software. Interface 1000 allows a user to edit the installation parameter value displayed in each field 1001-1006, for example, by entering a number in the field 1001-1006 and / or by using a numeric increment control 1016 displayed to the right of each field 1001-1006. When the user edits the value calculated by the software, the PFM badge 1015 adjacent to the corresponding field can be removed to indicate that the value of the field has been edited by the user. In some examples, after editing the value in one or more fields, the user can select a "Refresh Perspective Bracket Matching Data" button 1020 to restore each field to the software calculated value. Additionally, in some examples, after editing the values ​​in one or more fields, the user may select a “Save and Update” button 1021 to recalculate the deformity parameters based on the edited values ​​provided by the user, for example, by repeating all or part of the calculations performed in operation 322 .

[0110] In operation 334, a graphical representation of the position and orientation of the reference ring and the corresponding anatomical structure segment to which the ring is mounted is generated and displayed. The graphical representation of the position and orientation of the reference ring and the corresponding anatomical structure segment can be displayed via one or more graphical user interfaces of the computing system. For example, Fig.10 As shown, the interface 1000 includes a graphical representation 1050 of the position and orientation of the reference ring and the corresponding anatomical structure segment. The graphical representation 1050 includes: an illustration 1031 of the proximal anatomical structure segment, an illustration 1033 of the proximal (reference) ring, and an illustration 1032 of the distal anatomical structure segment. In some examples, the graphical representation 1050 can be generated based at least in part on the position and orientation of the reference ring and the corresponding anatomical structure segment determined in operation 322. Therefore, the graphical representation of the reference ring and the corresponding anatomical structure segment can reflect and / or indicate the position and orientation of the reference ring and the corresponding anatomical structure segment determined in operation 322. In some examples, when the user edits one or more installation parameters and selects the "Save and Update" button 1021, the graphical representation 1050 can also be adjusted to reflect the saved edits to the installation parameters. For example, the graphical representation 1050 can improve efficiency and reliability, for example, by providing the user with visual confirmation information of the information entered into the interface 1000, for example, to enable quick and easy identification of errors or other problems.

[0111] In operation 336, one or more treatment plan options are received, such as via one or more graphical user interfaces of a computing system. A treatment plan is a plan for manipulating the fixation device, for example, to correct a deformity of the first anatomical structure segment and the second anatomical structure segment. The treatment plan may include, for example, a plan for incrementally adjusting the position and orientation of the fixation rings relative to each other by, for example, changing the length of the struts of the fixation device. Referring now to Fig.11 , an exemplary treatment plan interface 1100A is shown. Interface 1100A includes a plurality of controls for selecting various treatment plan options by a user. Specifically, controls 1101 and / or 1102 allow selection of a treatment plan start date, control 1103 allows selection of an option to perform axial movement first (e.g., in an initial portion of treatment, such as before rotational movement), control 1104 allows selection of an option to indicate a final distance between reference points, control 1105 allows selection of an option to calculate a treatment plan based on a specified duration (e.g., multiple days) of axial movement, control 1106 allows selection of an option to calculate a treatment plan based on a distraction rate (e.g., millimeters (mm) / day) at a reference point of axial movement, control 1108 allows selection of an option to calculate a treatment plan based on a specified duration (e.g., multiple days) of deformity correction, control 1109 allows selection of an option to calculate a treatment plan for deformity correction based on a distraction rate (e.g., millimeters (mm) / day) at a reference point, and control 1107 allows selection of an option to perform two adjustments per day. In some examples, when control 1007 is not selected, a default option of adjusting once a day may be used. In some examples, after selecting the desired treatment plan options, the user may select an "update adjustment plan" button 1110 to trigger the generation of the treatment plan. In addition, after initially generating the treatment plan, the user may also be allowed to adjust the treatment plan options by reselecting the "update adjustment plan" button 1110 and regenerate the treatment plan using the adjusted options.

[0112] In operation 338, manipulation of the fixation device to correct the anatomical deformity (i.e., a treatment plan) is determined. Manipulation of the fixation device may include adjustment of the struts of the fixation device, such as adjusting the size and / or length of the struts. In some examples, operation 338 may be performed based at least in part on the treatment plan options received in operation 336. For example, operation 338 may be based at least in part on a specified start date, instructions to perform axial movement first (e.g., in an initial portion of treatment, such as before rotational movement), a specified final distance between reference points, instructions to perform a specified amount of additional extension, instructions to generate an axial gap to ensure anatomical gap, a specified treatment duration (e.g., multiple days), a specified distraction rate, and / or instructions to perform a specified amount of adjustment per day (e.g., once, twice, etc.).

[0113] In some examples, the treatment plan can also be determined based at least in part on determining whether the position and / or orientation of the anatomical segments 102, 104 needs to be changed, for example, how to reposition the anatomical segments 102, 104 relative to each other in order to promote healing between the anatomical segments 102, 104. For example, in some cases, it may be advantageous to: change the angulation of the second anatomical segment 104 so that the axes L1 and L2 are aligned; change the position of the second anatomical segment so that the fracture ends 103, 105 of the anatomical segments 102, 104 are adjacent to each other. Once it is determined that the position and / or orientation of the anatomical segments 102, 104 needs to be changed, a treatment plan for achieving the position and / or orientation can be determined. In a preferred embodiment, the desired change in the position and / or orientation of the anatomical segments 102, 104 can be gradually achieved through a series of small changes. The position and / or orientation of the anatomical segments 102 , 104 may be changed by changing the position and / or orientation of the upper and lower fixation rings 106 , 108 relative to each other, for example, by lengthening or shortening one or more adjustable length struts 116 .

[0114] By means of the matrix algebra described above, it is possible to calculate the changes that need to be made to the geometry of the fixture 100 (i.e., the position and / or orientation of the fixture 100) that can achieve the changes that need to be made to the position and / or orientation of the anatomical structure segments 102, 104. For example, the need to reposition and / or reorient the second anatomical structure segment 104 relative to the first anatomical structure segment 102 can be translated into: changing the position and / or orientation of the lower fixation ring 108 relative to the upper fixation ring 106.

[0115] At operation 340, an indication of the determined manipulation of the stationary device is provided to one or more users. For example, in some cases, the indication of the determined manipulation of the stationary device may be provided using one or more graphical user interfaces of a computing system, using printed hard copy, using audio feedback, and / or using other techniques. Specifically, referring now to Fig.12, it can be seen that an indication of the determined manipulation of the fixation device can be provided within the interface 1100B. Specifically, selecting the strut adjustment regimen tab 1122 causes the treatment regimen interface 1100B to provide a chart 1130 including daily manipulation information for each strut within the fixation device. In this example, chart 1130 shows the length of each strut on each treatment day. In some examples, one or more alerts can be generated for one or more fixation device manipulations that result in the following result: the movement of at least one strut exceeds a threshold amount. For example, in some cases, a strut movement that exceeds a certain threshold amount (e.g., 3 mm per day) (which may be referred to as a rapid strut movement) may be indicated by displaying a red triangle icon next to the indication of the strut movement in chart 1130. Also as shown Fig.12 As shown, by selecting the “View Draft PDF” button 1131, a PDF version of the diagram 1130 may be generated. In some examples, the generated PDF may be printed out to form a hard copy version of the diagram 1130.

[0116] exist Fig.12 In the example of FIG. 1130 , chart 1130 includes boxes 1132-A and 1132-B that indicate date ranges during which a brace size change (referred to as a brace swap) may be performed. Specifically, box 1132-A indicates that a brace swap may be performed on brace 4 on days 0 through 2, while box 1132-B indicates that a brace swap may be performed on brace 4 on days 3 through 14 (and subsequent days). In some examples, boxes 1132-A and 1132-B may be color coded to match the color assigned to the corresponding brace. For example, boxes 1132-A and 1132-B may be colored green to match the green color that may be assigned to brace 4. Referring now to FIG. 1130 , box 1132-A and 1132-B may be colored green to match the green color that may be assigned to brace 4. Fig.13 , a "Strut Replacement Calendar" tab 1123 in the treatment plan interface 1100-C may be selected to generate a calendar 1140 indicating a range of dates when strut replacements may be performed.

[0117] In some examples, the stays of the fixture attached to the patient can be color-coded, for example, by color-coded caps, markings, or other color-coded materials contained or attached to the stays. In some examples, the actual color coding of the stays of the fixture attached to the patient can match the color coding of the stays used by the software. For example, the actual color coding of the stays in the fixture can match the color coding of the stays that can be used to color-code the following items: boxes 1132-A and 1132-B of chart 1130, graphical representations 520, and other color-coded illustrations of the stays displayed by the software. In some examples, this can make it easier for the physician and / or patient to confirm that when the physician and / or patient actually adjusts the stays of the fixture, they adjust the correct stays and the adjustment amount is correct.

[0118] In operation 342, one or more graphical representations of the position and orientation of the first anatomical structure segment and the second anatomical structure segment and the ring of the fixation device are generated and displayed. The graphical representations of the position and orientation of the first anatomical structure segment and the second anatomical structure segment and the ring of the fixation device can be displayed through one or more graphical user interfaces of the computing system. For example, again referring to Fig.11 , selecting the “Treatment Simulation” tab 1121 may cause the interface 1100 to display a graphical representation 1150 of the position and orientation of the first anatomical segment and the second anatomical segment and the ring of the fixation device. The graphical representation 1150 includes: an illustration 1031 of the proximal anatomical segment, an illustration 1033 of the proximal (reference) ring, an illustration 1032 of the distal anatomical segment, and an illustration 1034 of the distal ring. In some examples, the one or more graphical representations of the position and orientation of the first anatomical segment and the second anatomical segment and the ring of the fixation device may include: a daily graphical representation of the position and orientation of the first anatomical segment and the second anatomical segment and the ring of the fixation device throughout the treatment of the anatomical deformity. For example, as Fig.11 As shown, through controls 1151, 1152, 1153 and / or 1154, the user can select a specific treatment date for which graphical representation 1150 is to be generated and displayed. For example, control 1151 can be selected to allow the selected date to be incremented, control 1154 can be selected to allow the selected date to be decremented, and slider 1152 can be slid along bar 1153 to increment and / or decrement the selected date. It should also be noted that slider 1152 displays an indication of the currently selected day, and in Fig.11 In the example of , the currently selected day is treatment day zero. Fig.11, graphical representation 1150 shows the position and orientation of the first anatomical structure segment and the second anatomical structure segment and the ring of the fixation device on treatment day zero. Selection of different treatment days via controls 1151-1154 triggers adjustment of graphical representation 1150 to display the position and orientation of the first anatomical structure segment and the second anatomical structure segment and the ring of the fixation device on the different selected days. It should be understood that a graphical representation that allows the surgeon and / or patient to see the position and orientation of the first anatomical structure segment and the second anatomical structure segment and the ring of the fixation device throughout the treatment process can be beneficial by, for example, providing additional visual tools to improve the accuracy of treatment and assist in planning treatment. In addition, graphical representation 1150 (and graphical representations described herein) can improve efficiency and reliability, for example, by providing a user with visual confirmation information of information entered into interface 1100, for example, to enable quick and easy identification of errors or other problems. It should also be noted that the view of the graphical representation 1150 (and other graphical representations described herein) can be rotated (e.g., a full 360 degree rotation), zoomed in and out, moved in direction, and otherwise manipulated, for example, via controls 1181-1184 adjacent to the upper right side of the graphical representation 1150. This can allow the first anatomical structure segment and the second anatomical structure segment and / or the ring of the fixation device to be viewed from various orientations that may not be available or may be difficult to obtain using x-ray and other imaging techniques, thereby also improving reliability and accuracy and providing additional visual confirmation of the calculated values. Specifically, the view of the graphical representation 1150 can be rotated using control 1181, zoomed in using control 1182, zoomed out using control 1183, and translated using control 1184. In addition, in some examples, other controls (such as a mouse and touch screen) can also be used to rotate, zoom, pan, and otherwise manipulate the graphical representation 1150. Additionally, in some examples, control 1185 may be used to select an anteroposterior (AP) view, control 1186 may be used to select a lateral view, and control 1187 may be used to select a proximal view.

[0119] In operation 344 , a treatment plan may be implemented, ie, the geometry of the fixation device may be changed, such as based on the manipulation determined in operation 338 , in order to change the position and orientation of the anatomical structure segment.

[0120] Matches the guide bracket projected by the graphic holder

[0121] As described above, a stent-matching process may be employed to determine the position and orientation of an anatomical structure segment in three-dimensional space, such as for use in generating a treatment plan for correcting an anatomical deformity. Also as described above, in some examples, as part of the stent-matching process, a surgeon or other user may identify the position of a fixator element (e.g., ring, strut, etc.) within a displayed image (e.g., x-ray) showing the fixator attached to the anatomical structure segment. Figure 3A and Figure 6 Operation 318 of the embodiment describes some examples of this process. Figure 6 As shown and described above, as part of the stent matching process, a user may identify the location of struts within the “AP view” image 601-A and the “LAT view” image 601-B. However, it may often be difficult for a user to identify and mark the location of certain fixator elements, such as struts, within the images. In particular, depending on the location and orientation at which the images were taken, it may not be easy to identify the struts and other fixator elements, such as because they may completely or partially overlap one another, or may otherwise be obscured within the image. For example, in some cases, it may often be more difficult to identify struts within a lateral image than to identify struts within a front image, such as because the struts may often overlap one another when viewed within a lateral image. For example, as shown in FIG. Figure 6 As shown, several of the struts are positioned within the "LAT view" image 601-B so that they are displayed very close together. Specifically, struts 1 and 2 are displayed very close together on the left side of image 601-B, struts 3 and 6 are displayed very close together in the middle of image 601-B, and struts 4 and 5 are displayed very close together on the right side of image 601-B. This can make it difficult for a user to identify, for example, strut 1 opposite strut 2, strut 3 opposite strut 6, or strut 4 opposite strut 5.

[0122] In some examples, to alleviate the above and other problems, guide bracket matching techniques can be used. Figure 14 to Figure 1 7 to describe some examples of guide bracket matching technology. Specifically, now refer to Fig.14 , an exemplary process for providing a graphical projection of a fixator using a guided stent matching technique will now be described in detail. As described above, a fixator may include fixator elements, such as rings and struts, and may be used to correct a deformity of a first anatomical segment and a second anatomical segment. Fig.14 The process begins at operation 1410, where first and second images of a first anatomical structure segment and a second anatomical structure segment and a fixture attached to the first anatomical structure segment and the second anatomical structure segment are displayed. The first image and the second image may have corresponding image planes. Figure 2As shown and described above, there is an angle α between the image planes of images 126 , 128 .

[0123] exist Figure 6 4 shows a first example of displaying the first image and the second image in operation 1410 , which includes the “AP view” image 601 -A and the “LAT view” image 601 -B as described above. Fig.15A An additional example of displaying the first image and the second image at operation 1410 is shown in FIG, which will now be described in detail. Specifically, Fig.15A An “AP view” image 1501-A and a “LAT view” image 1501-B are displayed, which are images of a fixator 1510 including a proximal fixation ring 1511, a distal fixation ring 1512, and fixator struts 1513. Images 1501-A and 1501-B show the fixator 1510 attached to a first anatomical structure segment 1521 and a second anatomical structure segment 1522. In operation 1410, one or more graphical user interfaces of a computing system may be used to display first and second images of the first anatomical structure segment and the second anatomical structure segment and the fixator attached to the first anatomical structure segment and the second anatomical structure segment. For example, one or more graphical user interfaces of a computing system may be used to display FIG. 15A to FIG. 17B It should be noted that one or more graphical user interfaces of the computing system may be used to display the FIG. 15A to FIG. 17B Any or all of the contents shown in each of the Figures.

[0124] It should be noted that FIG. 15A to FIG. 17B In the example of FIG. 1501 , images 1501 -A and 1501 -B are simulated images—unlike actual x-rays (such as Figure 6 15 to 17 are provided merely for the purpose of illustrating the concepts described herein. In implementation, images 1501-A and 1501-B may be non-simulated images, such as x-rays, captured using an imager, imaging source, x-ray imager, camera, or other image capturing device, and may show an actual fixator (such as a Figure 6 ). Therefore, even though images 1501-A and 1501-B are shown as simulations, the concepts described herein should be understood to be applicable to similar Figure 6 Non-simulated images (ie, images captured using an imager, imaging source, x-ray imager, camera, or other image capturing device) of images 601-A and 601-B.

[0125] At operation 1412, a first indication of a first position of a plurality of elements of a fixture (such as a brace of the fixture) within a first image is received, for example using one or more graphical user interfaces of a computing system. Figure 6 As described, a user may indicate the location of a strut (e.g., hinge) within the "AP View" image 601-A, such as by clicking on an endpoint of a strut (e.g., hinge) using an attached mouse or other input device. As described above, a strut indicator button 611-A for strut 1 may be automatically pre-selected for the user. After selecting (or automatically pre-selecting) the strut indicator button 611-A for strut 1, the user may proceed to draw (or otherwise indicate) a diagram of strut 1 within the "AP View" image 601-A. For example, in some cases, a user may select a location 621 (e.g., a center point) of a proximal hinge of strut 1 within image 601-A using a mouse or other input device. In some examples, the user may then select a location 622 (e.g., a center point) of a distal hinge of strut 1 within image 601-A using a mouse or other input device. In some examples, a user may indicate the location and / or length of strut 1 by selecting the locations of the proximal and distal hinges and / or the endpoints of a line or vector representing the location and / or length of strut 1. For example, as Figure 6 As shown, the software can generate points or circles at the locations 621 and 622 of the proximal and distal hinges selected by the user in image 601-A. In addition, the software can generate a straight line 623, which represents the position and / or length of the strut 1 connecting the points or circles at the locations 621 and 622 and the position and / or length of the proximal and distal hinges selected by the user in image 601-A. The user can also use any other suitable input technique to indicate the position and / or length of the strut 1 in image 610-A, such as, by dragging a mouse, using a finger and / or a pen on a touch screen, keypad, or other device to generate the straight line 623. In some examples, the above process can be repeated to draw points representing the proximal hinge and the distal hinge and lines representing the position and / or length of each of the six struts in the "AP view" image 601-A. Similar techniques can also be used to indicate the position and / or length of each of the six fixator struts 1513 in the image 610-A. Fig.15A The location in the “AP view” image 1501-A.

[0126] In some examples, after the user indicates the location of the strut 1513 within the "AP view" image 1501-A, the software may use the indicated strut location to determine the location of the fixation rings 1511 and 1512 within the "AP view" image 1501-A. The software may then generate ring graphical representations 1531 and 1532 corresponding to the fixation rings 1511 and 1512, respectively, and display the ring graphical representations 1531 and 1532 at the determined locations of the fixation rings 1511 and 1512 within the "AP view" image 1501-A. Referring now to FIG. Fig. 15B , it can be seen that the ring graphic representations 1531 and 1532 are generated by the software and displayed at the corresponding positions of the corresponding fixed rings 1511 and 1512 within the "AP view" image 1501-A. It should be noted that the fixed ring graphic representations 1531 and 1532 are Fig. 15B 1501 -A to indicate that the fixing ring graphical representations 1531 and 1532 are generated by software and are not included in the actual underlying AP view image 1501 -A. In some examples, the fixing ring graphical representations 1531 and 1532 may be shown in blue / shaded color while the fixing rings 1511 and 1512 may be shown in black / shaded color. Fig. 15B of the AP View image 1501-A (but not in Fig. 15B In the "LAT view" image 1501-B) there are fixed ring graphical representations 1531 and 1532 indicating that the received Fig. 15B The fixer element position information of the "AP view" image 1501-A of the Fig. 15B The fixture element position information of the "LAT View" image 1501-B.

[0127] At operation 1414, a graphical projection of the fixator is superimposed on the second image, for example using one or more graphical user interfaces of the computing system. The graphical projection of the fixator may include, for example, graphical representations of the rings and / or struts of the fixator. For example, referring now to Fig.16 , it can be seen that a graphical projection 1600 of the fixator is displayed, which includes a graphical representation 1611 of the proximal ring and a graphical representation 1612 of the distal ring. As shown, the graphical projection 1600 including the graphical representations 1611 and 1612 is superimposed on a second image, which in this example is the "LAT view" image 1501-B. Although Fig.1616, but graphic projection 1600 may include graphic representations of other fixture elements, such as fixture stays, in addition to or in lieu of graphic representations of fixture rings 1611 and 1612. Furthermore, in some examples, the graphic representations of fixture stays included in the graphic projection may be color coded such that different stays are shown in different colors, for example to match the different colors of stay indicator buttons 611-A and 611-B.

[0128] The graphical projection 1600 of the fixture may be rotated relative to the first position of the plurality of fixture elements identified in the first image. Specifically, the graphical projection 1600 of the fixture may be rotated relative to the first position based at least in part on the angle of the image planes of the first image and the second image relative to each other (such as at an exact angle or at an approximation of the angle). Figure 2 As shown and described above, there is an angle α between the image planes of images 126, 128. Fig.16 In the example of , the “AP view” image 1501-A may have a corresponding “AP view” image plane, and the “LAT view” image 1501-B may have a corresponding “LAT view” image plane that is at a ninety degree angle relative to the “AP view” image plane. Fig.16 In the example of FIG. 1 , the graphical projection 1600 of the fixture is rotated ninety degrees relative to the first position of the plurality of fixture elements identified in the first image. For example, Fig.16 Both the proximal ring graphical representation 1611 and the distal ring graphical representation 1612 are rotated ninety degrees relative to the proximal fixation ring 1511 (and / or corresponding ring representation 1531 ) and the distal fixation ring 1512 (and / or corresponding ring representation 1532 ) in the “AP view” image 1501 -A.

[0129] The graphical projection 1600 of the fixture may be rotated based at least in part on the angle between the image planes of the images, as the rotation may correspond to the expected position of the fixture in the second image. For example, if the image plane of the "LAT view" image 1501-B is at a ninety degree angle to the image plane of the "AP view" image 1501-A, it may be expected that the position of the fixture ring in the "LAT view" image 1501-B will be rotated ninety degrees relative to the position of the fixture ring in the "AP view" image 1501-A. In this way, the superposition of the graphical projection 1600 on the second image may help the user identify the position of the plurality of fixture elements in the second image. In some examples, the user may provide a numerical value, such as a number of degrees (e.g., ninety degrees), which explicitly indicates to the software the value of the angle between the image planes of the images. In other examples, the value of the angle may be inferred by the software based on a description of the image (e.g., anterior-posterior, anterior, posterior, lateral, medial, etc.) or using other techniques. In the examples of FIGS. 15 to 17 , image 1501-A is an AP view image, and image 1501-B is a lateral image. However, it should be noted that the guide support matching techniques described herein can be used between any different combinations of images taken from any direction and orientation and having image planes at any angles relative to each other.

[0130] Additionally, it should be noted that the software may also manipulate other features of the graphic projection 1600 (e.g., size, position, orientation, etc.) in order to correct for other differences between the first image and the second image (e.g., position, orientation, zoom level, etc.). For example, in some cases, if the second image is taken from a position closer to the fixture and / or is more zoomed in than the first image, the software may correct for this by enlarging the size of the graphic projection 1600 relative to the size of the fixture element in the first image. In contrast, in some cases, if the second image is taken from another position of the fixture and / or is more zoomed in than the first image, the software may correct for this by reducing the size of the graphic projection 1600 relative to the size of the fixture element in the first image.

[0131] Thus, in some examples, the graphical projection 1600 of the fixture may be generated based at least in part on the first positions of the plurality of fixture elements in the first image indicated at operation 1412. Additionally or alternatively, in some examples, the graphical projection 1600 of the fixture may be generated based at least in part on configuration information of the fixture provided to the software by a user, such as an indication of the type of ring (e.g., full ring, foot plate, etc.), ring size, brace length, mounting point (e.g., ring hole), and other information. Figure 5 and Figure 3A Operation 314 describes in detail various types of configuration information and techniques for providing such information to the software and will not be repeated here.

[0132] At operation 1416, the software may allow the user to manipulate (e.g., resize, rotate, move, etc.) the graphic projection and / or the second image. For example, the user may manipulate the graphic projection to more precisely align it with the position of the fixture element in the second image. For example, when superimposed on the second image at operation 1414, the software may provide controls that allow the graphic projection to be resized (made larger or smaller) or rotated relative to the initial placement made by the software. For example, in some cases, it may be necessary to resize and / or rotate the graphic projection to correct for slight differences in the actual angle between the first image and the second image relative to the expected angle (e.g., if the image is actually ninety-two degrees instead of ninety degrees, etc.), to correct for differences in the distance, position, or orientation of the first image and the second image relative to an object included in the image, or for other reasons. In some examples, the software may provide various controls such as buttons that allow selection of operations such as moving, resizing, and rotating, and the software may be configured to receive input from an input device such as a mouse or keyboard to implement those manipulations, such as via drag and drop, button clicks, keystrokes, etc.

[0133] In some examples, in addition to or as an alternative to allowing the user to manipulate the graphic projection, the software may allow the user to manipulate the second image (e.g., "LAT view" image 1501-B) on which the graphic projection is superimposed. For example, in some cases, the software may allow the user to resize the second image, rotate and / or move the second image and / or elements shown in the second image, such as to help align a fixture element shown in the second image with a corresponding element of the graphic projection. See now Fig.17A , it can be seen that the user has removed the “LAT view” image 1501-B from Fig.16 The second image is manipulated as "LAT view" image 1501-B by moving its previous screen / interface position downward and rightward as shown. By moving "LAT view" image 1501-B in this manner (without moving graphic projection 1600), this allows the fixture elements in "LAT view" image 1501-B to move downward and rightward so that they align with corresponding elements of graphic projection 1600. For example, as shown Fig.17A As shown, the graphical representations 1611 and 1612 of the retaining rings are substantially aligned with the corresponding retaining rings 1511 and 1512. Therefore, only a small portion of the retaining rings 1511 and 1512 is visible in FIG. 17 because they are almost completely superimposed by the corresponding graphical representations 1611 and 1612 of the retaining rings. Specifically, in FIG. Fig.17A, the proximal ring graphical representation 1611 is substantially aligned with (and almost completely superimposed on) the proximal retaining ring 1511 , and the distal ring graphical representation 1612 is substantially aligned with (and almost completely superimposed on) the distal retaining ring 1512 .

[0134] At operation 1418, a second indication of a second position of the plurality of elements of the fixture within the second image is received, for example using one or more graphical user interfaces of the computing system. For example, in some cases, once the graphical projection 1600 of the fixture is satisfactorily aligned with the position of the fixture in the second image (e.g., in the "LAT view" image 1501-B), the user may use the graphical projection 1600 as a guide to help identify the position of the plurality of elements of the fixture in the second image (e.g., in the "LAT view" image 1501-B). For example, while not being in the second image, the user may be able to identify the second position of the plurality of elements of the fixture within the second image (e.g., in the "LAT view" image 1501-B). Fig.16 and Fig.17A , but the graphical projection 1600 of the fixture may include graphical representations of the fixture braces. In these examples, when the user aligns the graphical projection of the fixture with the position of the fixture in the second image, the graphical representations of the fixture braces in the graphical projection will align with (and overlay) the corresponding positions of the fixture braces within the second image. Also as described above, the graphical representations of the fixture braces in the graphical projection may be color coded such that different braces are shown in different colors, such as to match the different colors of the brace indicator buttons 611-A and 611-B and the different colors of the braces in the first image. Thus, the color coding of the graphical representations of the braces in the graphical projection may help the user distinguish between corresponding braces that are closely positioned together in the second image. For example, as Fig.17A As shown, the first brace 1513-A and the second brace 1513-B are positioned closely together on the right side of the "LAT view" image 1501-B. If graphical representations of these braces are included in the graphic projection 1600 and aligned with (and superimposed on) the braces 1513-A and 1513-B, this can help the user distinguish between the braces 1513-A and 1513-B in the "LAT view" image, especially if the graphical representations of the braces in the graphic projection 1600 are color-coded. For example, if the graphical representation of the first brace 1513-A is shown in red, the user can immediately understand that the red brace representation is aligned with and superimposed on the first brace 1513-A. In addition, if the graphical representation of the second brace 1513-B is shown in orange, the user can immediately understand that the orange brace representation is aligned with and superimposed on the second brace 1513-B. In this way, the user may use the graphical representations of the struts (and / or other fixture elements) in the graphical projection as a guide to identify corresponding ones of the plurality of fixture elements in the second image.It should be noted that in addition to the struts, the graphical representations of the rings 1611 and 1612 may also be color coded.

[0135] Thus, by using the graphical projection 1600 of the fixture as a guide, the user may identify the location of the plurality of fixture elements in the second image. The user may then indicate the location of these fixture elements to the software, such as by using the same or similar process as used to identify the plurality of fixture elements in the first image in operation 1412. For example, the techniques described above in operation 1412 for the first image ("AP view" images 601-A and 1501-A) (e.g., identifying the proximal and distal hinges or endpoints of each strut) may be repeated for the second image ("LAT view" images 601-B and 1501-B), such as by using the "LAT view" strut indicator button 611-B to draw points representing the proximal and distal hinges and lines representing the location and / or length of each of the six struts in the "LAT view" images 601-B and 1501-B.

[0136] In some examples, at operation 1418, after the user has indicated the locations of the stays 1513 and / or other fixture elements within the "LAT view" image 1501-A (e.g., using the graphical projection 1600 of the fixture as a guide), the software may use the indicated fixture element locations to determine the locations of the fixture rings 1511 and 1512 within the "LAT view" image 1501-B. The software may then generate graphical representations of the rings 1731 and 1732 corresponding to the fixture rings 1511 and 1512, respectively, and display the graphical representations of the rings 1731 and 1732 at the determined locations of the fixture rings 1511 and 1512 within the "LAT view" image 1501-B. Referring now to FIG. Fig. 17B As can be seen, the ring graphic representations 1731 and 1732 are generated by the software and displayed at corresponding positions of the respective fixed rings 1511 and 1512 within the "LAT View" image 1501-B.

[0137] At operation 1420, the position and orientation of the first anatomical structure segment and the second anatomical structure segment in three-dimensional space are determined using the first position of the plurality of fixture elements in the first image (indicated at operation 1412) and the second position of the plurality of fixture elements in the second image (indicated at operation 1418 using the graphic projection as a guide). For example, as described above with respect to Figure 3A As described in detail in operation 322 of the present invention, the imaging scene parameters may be used to determine the position and orientation of the first anatomical structure segment and the second anatomical structure segment in three-dimensional space. As also described above, the imaging scene parameters may be obtained by comparing the illustrated positions of specific components or the illustrated positions of fixture elements of the fixture in the two-dimensional space of the first image and the second image with the corresponding positions of these same fixture elements in the actual three-dimensional space. As also described above, such as relative to Figure 3BIn operation 338, the position and orientation of the first anatomical structure segment and the second anatomical structure segment in three-dimensional space may be used to determine a manipulation of the fixation device (i.e., a treatment plan) for correcting the anatomical deformity. Specifically, the treatment plan may be determined based at least in part on determining whether the position and / or orientation of the anatomical structure segments need to be changed, for example, how to reposition the anatomical structure segments relative to each other in order to promote healing between the anatomical structure segments.

[0138] 3D overview of the imaging scene of the fixture

[0139] Another technique for improving the accuracy and reliability of input values ​​and resulting calculations is disclosed herein, which provides a three-dimensional overview of the imaging scene of the fixator. This three-dimensional overview can be used to provide feedback and visual confirmation to help ensure that the calculated position and orientation of the anatomical structure is reliable and correct. Reference will now be made to Figures 18 to 25 Various techniques for generating a three-dimensional overview are described in detail. In particular, see now Fig.18 , an exemplary method for generating a three-dimensional overview of an imaging scene of a fixator including rings and struts to correct a deformity of a first anatomical segment and a second anatomical segment will now be described in detail. Fig.18 The process begins at operation 1810, where first and second images, such as x-rays, of a fixture and first and second anatomical structure segments attached to the fixture are received, for example, by a computing system. As described above, the images may be captured using one or more imaging sources such as an x-ray imager and / or an image capture device. Also as described above, the first and second images have respective first and second image planes that are at an angle relative to each other. The images may be received by a computing system, such as by scanning or otherwise electronically loading or transmitting image data of the images to the computing system.

[0140] As described above, when the first image and the second image are received, a bracket matching technique may be used in association with the first image and the second image, for example, as described above with reference to FIG. 3A to FIG. 3B The operations 314 to 334 of FIG. 3 are described in order to determine the position and orientation of the anatomical structure segment, the fixator, the imaging source, and other elements of the fixator imaging scene in three-dimensional space. These techniques are described in detail above and will not be repeated here. In some examples, when the bracket matching process is completed, for example, including operations 314 to 334 (and optionally Fig.14 A three-dimensional overview can be displayed to the user, such as will now be described in detail. Fig.18At operation 1812, the three-dimensional graphical model is displayed, for example, using one or more graphical user interfaces of the computing system. The three-dimensional graphical model may include a first image representation representing the first image and a second image representation representing the second image. For example, referring now to Fig.19 It can be seen that the model 1900 (which is a three-dimensional graphic model) includes an "AP view" image representation 1901-A representing an "AP view" image and a "LAT view" image representation 1901-B representing a "LAT view" image. Fig.19 In the example of , image representations 1901-A and 1901-B are graphical simulations of the first image and the second image, which may be x-ray images. However, in some examples, image representations 1901-A and 1901-B may include actual x-ray images or other actual images taken from an imaging source. It should be noted that one or more graphical user interfaces of a computing system may be used to display Figures 19 to 25 Any or all of the contents shown in each of the Figures.

[0141] In model 1900, image representations 1901-A and 1901-B may be displayed relative to each other at the same angle as the angle between their corresponding image planes. For example, the image planes of the "AP view" image and the "LAT view" image may have an angle of approximately (but in some cases not exactly) ninety degrees relative to each other. Thus, in model 1900, the "AP view" image representation 1901-A and the "LAT view" image representation 1901-B are positioned relative to each other at the same angle (approximately ninety degrees) as their corresponding image planes. In some examples, the software may calculate and display the actual angle between the image planes of the first image and the second image as a numerical value.

[0142] In some examples, model 1900 may display a graphical representation of the respective locations of the imaging sources of the first image and the second image. Fig.19 In the example of FIG. 1 , the AP imaging source representation 1911-A represents the location of the imaging source (e.g., an x-ray imager, an image capture device, etc.) of the “AP view” image, and the LAT imaging source representation 1911-B represents the location of the imaging source (e.g., an x-ray imager, an image capture device, etc.) of the “LAT view” image. The imaging source representations 1911-A and 1911-B may indicate respective virtual locations corresponding to the first imaging source and the second imaging source. For example, when performing a support matching process, such as described above with reference to FIG. FIG. 3A to FIG. 3BAs described in operations 314 to 334 of the present invention, the software may be able to calculate the position and orientation of the entire three-dimensional scene, including the position and orientation of the anatomical structure segment, the fixator, and the imaging source relative to each other. Therefore, based on the bracket matching process, the software can determine corresponding virtual positions corresponding to the first imaging source and the second imaging source. These virtual positions can reflect the distance, position, and / or orientation of the imaging source relative to the anatomical structure segment and / or the fixator.

[0143] exist Fig.19 In the example of FIG. 1 , reference position 1913 is shown in both the “AP view” image representation 1901 -A and the “LAT view” image representation 1901 -B. In this specific example, reference position 1913 is a reference point on a proximal anatomical structure segment 1915, such as an endpoint.

[0144] In some examples, the first virtual line may connect the first virtual location corresponding to the first imaging source to the reference location in the first image representation. Fig.19 , beam 1912-A is shown as a graphical representation of this first virtual line displayed in model 1900. As shown, beam 1912-A connects a first virtual location corresponding to a first imaging source (e.g., the location indicated by AP imaging source representation 1911-A) to a reference location in the first image representation (e.g., reference location 1913 in "AP view" image representation 1901-A).

[0145] Additionally, in some examples, a second virtual line may connect a second virtual location corresponding to the second imaging source to a reference location in the second image representation. Fig.19 1-B, beam 1912-B is shown as a graphical representation of the second virtual line displayed in model 1900. As shown, beam 1912-B connects a second virtual position corresponding to a second imaging source (e.g., the position indicated by LAT imaging source representation 1911-B) to a reference position in the second image representation (e.g., reference position 1913 in "LAT view" image representation 1901-B). Although beams 1912-A and 1912-B are shown in model 1900, it is not required that the first virtual line and the second virtual line must be displayed in the model, and some models may not display beams 1912-A and 1912-B (or may only display portions thereof).

[0146] At operation 1814, a first graphical representation associated with the shortest distance (eg, the intersection point) between the first virtual line and the second virtual line is displayed in the three-dimensional graphical model. Fig.19In the example of , beams 912-A and 1912-B intersect, and point 1914 is shown at the intersection of beams 1912-A and 1912-B, which is the shortest distance between beams 1912-A and 1912-B. Thus, in this example, point 1914 is a first graphical representation of the intersection between a first virtual line (represented by beam 1912-A) and a second virtual line (represented by beam 1912-B). In some examples, unlike a point, the first graphical representation may simply be the intersection between two graphical lines or beams, or may be any other type of graphical representation. The first graphical representation (e.g., point 1914) represents the physical location of a reference location in three-dimensional space. Thus, in Fig.19 In the example of , point 1914 represents the physical location of an endpoint of a proximal anatomical structure segment 1915 (the endpoint being reference location 1913 ) in three-dimensional space.

[0147] In the above example, beams 1912-A (representing a first virtual line) and 1912-B (representing a second virtual line) intersect each other at point 1914. However, in some examples, the above first virtual line and second virtual line may not actually intersect each other. In some examples, in these scenarios, instead of being displayed at the intersection between the lines (because there is no intersection), the first graphical representation can be displayed at a point that bisects (i.e., divides into two equal halves) the shortest distance between the first virtual line and the second virtual line. Specifically, a vector can connect corresponding points of the shortest distance on the first virtual line and the second virtual line, and the first graphical representation can be displayed at the midpoint of the vector.

[0148] Operation 1814 is described above with respect to reference location 1913 on proximal anatomical structure segment 1915. However, it should be noted that operation 1814 may be repeated for any number of other reference locations. For example, Fig.19Also shown is a reference location 1923 in both the "AP view" image representation 1901-A and the "LAT view" image representation 1901-B. In this particular example, the reference location 1923 is a reference point, such as an endpoint, on the distal anatomical structure segment 1925. As shown, a beam 1922-A connects a first virtual location corresponding to a first imaging source (e.g., the location indicated by the AP imaging source representation 1911-A) to the reference location 1923 in the AP view image representation 1901-A. Additionally, a beam 1922-B connects a second virtual location corresponding to a second imaging source (e.g., the location indicated by the LAT imaging source representation 1911-B) to the reference location 1923 in the "LAT view" image representation 1901-B. Furthermore, a point 1924 is shown at the intersection of the beams 1922-A and 1922-B. Thus, in this example, the point 1924 is a first graphical representation of the intersection between the beams 1922-A and the beams 1922-B. The first graphical representation (eg, point 1924) represents the physical location of the reference location in three-dimensional space. Thus, in this example, point 1924 represents the physical location of the endpoint of distal anatomical structure segment 1925 (which is reference location 1923) in three-dimensional space.

[0149] At operation 1816, graphical representations of elements of the fixture (e.g., rings, braces, etc.) may be displayed in the three-dimensional graphical model at virtual locations that represent the physical locations of the elements of the fixture in three-dimensional space. Fig. 20 As can be seen, the user can activate a control of the software, such as the hexagonal visual control 2001, which causes the model 1900 to display a graphical representation of the retaining ring. The user can also deactivate the hexagonal visual control 2001, which causes the model 1900 to stop displaying the graphical representation of the retaining ring. Fig. 20 As shown, activation of the hexagonal visual control 2001 causes the proximal ring graphical representation 2011 and the distal ring graphical representation 2012 to be displayed in the model 1900. In addition, reference is now made to Fig.21 , it can be seen that the user can activate a control of the software, such as a brace visual control 2101, which causes the model 1900 to display a graphical representation 2110 of the anchor brace. The user can also deactivate the brace visual control 2101, which causes the model 1900 to stop displaying the graphical representation 2110 of the anchor brace. As shown, the graphical representation 2110 of the anchor brace can be color coded so that different braces are shown in different colors, for example to match the color of the anchor brace. Figure 6 In some examples, the physical position of the fixture element in three-dimensional space can be determined by the software by performing the above-mentioned bracket matching technique (e.g., FIG. 3A to FIG. 3BThe software may then determine virtual locations within model 1900 that correspond to these physical locations, and the software may display a graphical representation of the fixture element at the determined virtual locations.

[0150] At operation 1818, graphical representations of the first anatomical structure segment and the second anatomical structure segment may be displayed in the three-dimensional graphical model at virtual locations that represent the physical locations of the first anatomical structure segment and the second anatomical structure segment in the three-dimensional space. Fig.21 , it can be seen that the user can activate a control of the software, such as the proximal rod visual control 2102, which causes the model 1900 to display a graphical representation 2112 of the proximal anatomical structure section 1915. The user can also deactivate the proximal rod visual control 2102, which causes the model 1900 to stop displaying the graphical representation 2112 of the proximal anatomical structure section 1915. Fig.21 As shown, the user can activate a control of the software, such as the distal rod visual control 2103, which causes the model 1900 to display a graphical representation 2113 of the distal anatomical structure segment 1925. The user can also deactivate the distal rod visual control 2103, which causes the model 1900 to stop displaying the graphical representation 2113 of the distal anatomical structure segment 1925. In some examples, the physical position of the anatomical structure segment in three-dimensional space can be determined by the software by performing the above-mentioned bracket matching techniques (e.g., FIG. 3A to FIG. 3B The software may then determine virtual locations within the model 1900 that correspond to these physical locations, and the software may display a graphical representation of the anatomical structure segment at the determined virtual locations.

[0151] Thus, as described above, the model 1900 may provide a three-dimensional graphical model that displays the first image and the second image (or representation thereof) in combination with a graphical representation of an imaging source, a graphical representation of a reference position (e.g., an endpoint of a first anatomical structure segment and a second anatomical structure segment), a graphical representation of a virtual line connecting a virtual position corresponding to the imaging source to the reference position and its intersection, a graphical representation of a fixator element (e.g., a ring, a strut, etc.), and a graphical representation of an anatomical structure segment, thereby indicating the spatial relationship between these objects. In addition, each of the graphical representations may be displayed at a virtual position representing a corresponding physical position in three-dimensional space. It should be understood that the above-described three-dimensional graphical model may thus provide visual feedback that allows a user to locate points and other positions in three-dimensional space and to view and confirm the correctness of the values ​​calculated during the stent matching process. For example, a user may confirm that the reference positions 1913 and 1923 (such as the endpoints of the anatomical structure segment) are in the correct position relative to other objects (such as fixation rings, fixator struts, and other positions on the anatomical structure segment). For example, in some cases, if points 1914 and / or 1924 are positioned in an incorrect position relative to the graphical representations 2011 and 2012 of the retainer rings and / or the graphical representations 2110 of the retainer struts, then this will indicate to the user that one or more calculations have not been correctly performed, for example due to an error in the user's identification of input values ​​during the stent matching process. For example, if points 1914 and / or 1924 are positioned in a non-sensitive position (e.g., a position that collides with the retainer ring or strut, a position that is outward from the strut, a position above the upper ring or below the lower ring, etc.), then this may clearly indicate an error in the stent matching calculation. Upon determining such an error, the user may choose to review and resubmit any or all input values ​​provided during the stent matching process and then re-perform the calculation.

[0152] In some examples, model 1900 can be scalable, resizable, and rotatable, and can be manipulated by the user in other ways. For example, the user can zoom in to enlarge a portion of model 1900 and / or zoom out to increase the field of view. As another example, the user can translate and / or rotate the model, such as in any combination of directions (e.g., up, down, left, right, pitch, yaw, etc.). In some examples, model 1900 can be manipulated to be shown from a variety of different perspectives, such as corresponding perspectives of the first image and the second image. For example, referring now to Fig. 22 , it can be seen that the user can select "Show" from the AP button 2200 to display the model 1900 from the "AP view" perspective corresponding to the "AP view" image. In addition, now see Fig.23 , it can be seen that the user can select "Show" from the LAT button 2300 to display the model 1900 from the "LAT view" perspective corresponding to the "LAT view" image. Fig.24, we can see that the model 1900 can be compared to Figures 19 to 23 The perspective view shown in is rotated vertically.

[0153] As described above, the first image and the second image corresponding to image representations 1901-A and 1901-B have image planes that are angled relative to each other. For example, the "AP view" and "LAT view" images corresponding to image representations 1901-A and 1901-B may have image planes that are substantially orthogonal to each other. However, in some cases, these image planes may not be completely orthogonal to each other. In some examples, when the image planes are not completely orthogonal, the software may display a modified second image representation that is truly orthogonal to the first image representation. The modified second image representation may represent a modified second image having image planes that are truly orthogonal to the image planes of the first image. For example, now referring to Fig.25 , a modified “LAT view” image representation 2500 is shown in model 1900. The modified “LAT view” image representation 2500 is a modification of the LAT view image representation 1901-B. The modified “LAT view” image representation 2500 is truly orthogonal to the “AP view” image representation 1901-A within model 1900. The modified “LAT view” image representation 2500 may represent a modified “LAT view” image having an image plane that is truly orthogonal to the image plane of the “AP view” image. In some examples, the user may activate an ideal plane visual control 2510 to cause the modified “LAT view” image representation 2500 to be shown in model 1900. The user may also deactivate the ideal plane visual control 2510 to stop the modified “LAT view” image representation 2500 from being shown in model 1900.

[0154] In some examples, if the second image is truly orthogonal to the first image, the software may calculate the angles of the anatomical segments that would be displayed in the second image. The software may then display the modified second image in a modified second image representation in which the anatomical segments are displayed at calculated angles relative to each other. For example, if the “LAT view” image is truly orthogonal to the “AP view” image, the software may calculate the angles of the anatomical segments 1915 and 1925 that would be displayed in the “LAT view” image. The software may then display the modified “LAT view” image in a modified “LAT view” image representation 2500 in which the anatomical segments 1915 and 1925 are displayed at calculated angles relative to each other. In some examples, the angles of the anatomical segments in the modified “LAT view” image representation 2500 may be determined based on knowledge of the position and orientation of the anatomical segments in physical three-dimensional space, such as may be determined by performing the above-described scaffold matching technique (e.g., FIG. 3A to FIG. 3B Operations 314 to 334).

[0155] It should be appreciated that the ability to generate and display a modified second image representation (e.g., a modified "LAT view" image representation 2500) can provide a number of benefits. For example, when a surgeon measures an anatomical deformity on a two-dimensional image, it is assumed that the images are orthogonal. Any deviation from the orthogonality of these images can result in inaccurate measurements, such as associated with angles in the front view (varus / valgus) and the side view (anterior apex / posterior apex) that begin to blend. Such deviations can result in residual deformity after correction. Using the above-described support matching techniques, the software understands the position and orientation of the anatomical segments in physical three-dimensional space. Therefore, if the second image is truly orthogonal to the first image, the software can calculate the angle of the anatomical segment that would be displayed in the second image. In this way, the software can show the user how to adjust the measured anatomical deformity values ​​from the second image that is not truly orthogonal to the first image to the corrected anatomical deformity values ​​corresponding to the modified second image that will be truly orthogonal to the first image, and provide guidance for verifying the correction values ​​used in the above-described calculations. In some examples, if the user is not confident in the correction value, the user may be provided with the option to override the correction value (such as using the measured value).

[0156] It should be noted that while the above-described model 1900 shows a graphical representation of a first image and a second image, a three-dimensional graphical model showing only a single image representation of a single image may also be provided. Such a single image model may be used to provide feedback to the user at an earlier stage of the input process, such as when the brace matching is performed only on a single image, for example, when the user has indicated the position of the strut on the first image but not yet on the second image. In addition, while the above-described model 1900 shows points 1914 and 1924 representing the reference position of the anatomical structure, a three-dimensional graphical model may also be provided that does not necessarily include anatomical structure information. This type of model may also be used to provide feedback to the user at an earlier stage of the input process, such as when the user has indicated the position of the fixator element in the first image and / or the second image, but has not yet indicated the position of the anatomical structure in the first image and / or the second image.

[0157] For example, for a situation where the user has indicated a strut location on a first image (but has not yet indicated a strut location on a second image), the single image model may depict an image representation of the first image, an imaging source representation of the first image, and a graphical representation of the stent (e.g., rings and struts). Additionally, if the user subsequently performs deformity planning and indicates the location of the anatomical structure segment on the first image, the single image model may be updated to display planning elements (e.g., anatomical structure reference points, anatomical structure centerline) on the first image representation, and include beams connecting the imaging source representation of the first image to corresponding points on the first image illustration, in such a way as to show the user where these beams pass through the fixator stent. Additionally, for a situation where fixator element (e.g., strut) location identification has been performed for both images, but deformity planning (e.g., indication of anatomical structure location) has not yet been performed, a model may be generated showing an image representation of the two images, an imaging source representation of the two images, and a graphical representation of the stent (e.g., rings and struts), thereby allowing the user to view the relative orientation (i.e., angulation and rotation) between the images. Any or all of the above models may be rotated, scaled, and translated by the user similar to model 1900.

[0158] Changes in stretch rate

[0159] In some examples, during the course of computer-assisted ring fixator treatment, the surgeon may desire a change in the patient's distraction rate. This may be due to premature consolidation (correction is too slow), poor regeneration (correction is too fast), a clinically required re-arranged strut exchange, too severe pain, or any other reason. In some examples, in order to allow the distraction rate to be changed in an effective and reliable manner, an option may be provided within the software for the surgeon to select a "change distraction rate" button or other control, which will allow the distraction rate to be changed to a new distraction rate from a given day of the treatment plan, and the new distraction rate is used for the remaining time of the treatment plan. In some examples, this may allow the distraction rate to be changed using only one screen, one field, and one click during the treatment process (e.g., within a day in the middle of the treatment plan) to produce clinically relevant changes in patient treatment and potentially reduce the patient's time in the brace. In some examples, when the change distraction rate control is selected, the plan can be reopened by the software in the plan state, such as on the treatment plan tab. In some examples, all other tabs (except the treatment plan tab) may be inactive. The user can then edit the "Days" field (or other field representing the duration of the treatment plan) or the "Distraction at Reference Point" field. The user can then select the "Update Adjustment Plan" control, and the software can generate a new treatment plan starting from the day the change distraction rate control was selected. The user can then deliver the new plan to the patient.

[0160] Exemplary Computing Devices

[0161] See also Fig.26 A suitable computing device, such as the exemplary computing device 78, may be configured to perform any or all of the techniques described above. It should be understood that the computing device 78 may include any suitable device, examples of which include a desktop computing device, a server computing device, or a portable computing device, such as a laptop, a tablet computer, or a smartphone.

[0162] In an exemplary configuration, the computing device 78 includes a processing portion 80, a memory portion 82, an input / output portion 84, and a user interface (UI) portion 86. It should be emphasized that the block diagram description of the computing device 78 is exemplary and is not intended to imply a specific implementation and / or configuration. The processing portion 80, the memory portion 82, the input / output portion 84, and the user interface portion 86 can be coupled together to enable communication therebetween. It should be understood that any of the above components can be distributed in one or more independent devices and / or locations.

[0163] In various embodiments, the input / output portion 84 includes a receiver of the computing device 78, a transmitter of the computing device 78, or a combination thereof. The input / output portion 84 is capable of receiving and / or providing information related to communications with a network such as the Internet, for example. It should be understood that the sending and receiving functions may also be provided by one or more devices external to the computing device 78.

[0164] Processing portion 80 may include one or more processors. Depending on the exact configuration and type of processor, memory portion 82 may be volatile (such as certain types of RAM), non-volatile (such as ROM, flash memory, etc.), or a combination thereof. Computing device 78 may include additional storage devices (e.g., removable storage devices and / or non-removable storage devices) including, but not limited to, tape, flash memory, smart cards, CD-ROMs, digital versatile disks (DVDs) or other optical storage devices, cassettes, magnetic tape, magnetic disk storage devices or other magnetic storage devices, universal serial bus (USB) compatible memory, or any other medium that can be used to store information and can be accessed by computing device 78.

[0165] The computing device 78 may also include a user interface portion 86 that allows a user to communicate with the computing device 78. The user interface 86 may include an input that provides the ability to control the computing device 78 via, for example, buttons, soft keys, a mouse, a voice-activated control, a touch screen, movement of the computing device 78, visual cues (e.g., moving a hand in front of a camera on the computing device 78), etc. The user interface portion 86 may provide outputs, including visual information (e.g., via a display), audio information (e.g., via a speaker), mechanical forms (e.g., via a vibration mechanism), or a combination thereof. In various configurations, the user interface portion 86 may include a display, one or more graphical user interfaces, a touch screen, a keyboard, a mouse, an accelerometer, a motion detector, a speaker, a microphone, a camera, a tilt sensor, or any combination thereof. Thus, a computing system including, for example, one or more computing devices 78 may include a processor, a display coupled to the processor, a memory communicating with the processor, one or more graphical user interfaces, and a variety of other components. The memory may store instructions, and after the processor executes the instructions, the computer system will perform operations, such as the operations described above. As used herein, the term computing system may refer to a system including one or more computing devices 78. For example, a computing system may include one or more server computing devices in communication with one or more client computing devices.

[0166] Although exemplary embodiments of devices for performing the disclosed techniques are described herein, the basic concepts can be applied to any computing device, processor, or system capable of communicating and presenting information as described herein. The various techniques described herein can be implemented in combination with hardware or software, or in combination with both where appropriate. Therefore, the methods and devices described herein can be implemented, or some aspects or parts thereof can be in the form of program code (i.e., instructions) embodied in a tangible, non-transitory storage medium, such as a floppy disk, a CD-ROM, a hard drive, or any other machine-readable storage medium (computer-readable storage medium), wherein when the program code is loaded into a machine (such as a computer) and executed by the machine, the machine becomes a device for performing the techniques described herein. In the case of executing program code on a programmable computer, the computing device will generally include a processor, a storage medium (including volatile and non-volatile memory and / or storage elements) readable by the processor, at least one input device, and at least one output device, such as a display. The display can be configured to display visual information. One or more programs can be implemented in component or machine language as needed. The language can be a compiled or interpreted language and combined with a hardware specific implementation.

[0167] It should be understood that the plastic surgery fixation using imaging analysis described in this article not only provides for the use of non-orthogonal images, but also enables the use of overlapping images, images taken by different imaging techniques, images taken under different settings, etc., thereby providing surgeons with greater flexibility compared to existing fixation and imaging techniques.

[0168] The techniques described herein may also be practiced via communications embodied in the form of program code, which is transmitted over some transmission medium, such as by electrical wiring or cabling, by optical fiber, or via any other form of transmission. When implemented on a general-purpose processor, the program code combines with the processor to provide a unique means for invoking the functions described herein. In addition, any storage technology used in conjunction with the techniques described herein may always be a combination of hardware and software.

[0169] The above can be better understood in terms of the following:

[0170] 1. A computer-implemented method for providing a graphical projection of a fixation device comprising rings and struts to correct a deformity of a first anatomical segment and a second anatomical segment, the method comprising:

[0171] displaying, using one or more graphical user interfaces of a computing system, first and second images of the first and second anatomical segments and the fixture attached to the first and second anatomical segments, the first and second images having image planes at an angle relative to each other;

[0172] receiving, using the one or more graphical user interfaces, a first indication of a first position of a plurality of elements of the fixture within the first image; and

[0173] The graphical projection of the fixture is superimposed on the second image using the one or more graphical user interfaces, wherein the graphical projection of the fixture is rotated relative to the first position based at least in part on the angle.

[0174] 2. The computer-implemented method of claim 1, further comprising receiving, using the one or more graphical user interfaces, a second indication of a second position of the plurality of elements of the fixture within the second image.

[0175] 3. The computer-implemented method of claim 2, wherein superimposing the graphical projection of the fixture on the second image assists a user in identifying the second position.

[0176] 4. The computer-implemented method of claim 1, wherein the graphical projection of the fixture is movable, resizable, and rotated by a user.

[0177] 5. The computer-implemented method of claim 1, wherein the deformity is corrected using a treatment regimen in which a rate of distraction is altered within an intermediate day within the treatment regimen.

[0178] 6. The computer-implemented method of claim 1, wherein the plurality of elements of the fixture include the braces.

[0179] 7. The computer-implemented method of claim 1, wherein the graphical projection of the fixture includes a graphical representation of the brace.

[0180] 8. The computer-implemented method of claim 1, wherein the graphical projection of the fixture comprises a graphical representation of the ring.

[0181] 9. A computer-implemented method for generating a three-dimensional overview of an imaging scene of a fixator comprising rings and struts to correct a deformity of a first anatomical segment and a second anatomical segment, the method comprising:

[0182] receiving, by a computing system, first and second images of the fixture and the first and second anatomical segments attached to the fixture, the first image taken from a first imaging source and the second image taken from a second imaging source, the first and second images having respective first and second image planes at an angle relative to each other;

[0183] displaying, using one or more graphical user interfaces of the computing system, a three-dimensional graphical model, the three-dimensional graphical model comprising a first image representation representing the first image and a second image representation representing the second image, wherein a first virtual line connects a first virtual position corresponding to the first imaging source to a reference position in the first image representation, and wherein a second virtual line connects a second virtual position corresponding to the second imaging source to the reference position in the second image representation; and

[0184] A first graphical representation associated with the shortest distance between the first virtual line and the second virtual line is displayed in the three-dimensional graphical model, wherein the first graphical representation represents a physical location of the reference location in three-dimensional space.

[0185] 10. The computer-implemented method of claim 9, wherein the shortest distance between the first virtual line and the second virtual line is an intersection point between the first virtual line and the second virtual line, and wherein the first graphical representation is displayed at the intersection point.

[0186] 11. The computer-implemented method of claim 9, wherein the first virtual line and the second virtual line do not intersect, and wherein the first graphical representation is displayed at a point that bisects the shortest distance.

[0187] 12. The computer-implemented method of claim 9, wherein the reference location is a reference point of the first anatomical segment or the second anatomical segment.

[0188] 13. The computer-implemented method of claim 9, further comprising displaying graphical representations of the first virtual line and the second virtual line in the three-dimensional graphical model.

[0189] 14. The computer-implemented method of claim 9, further comprising displaying graphical representations of the rings and the braces of the binder in the three-dimensional graphical model at virtual locations, the virtual locations representing physical locations of the rings and the braces of the binder in the three-dimensional space.

[0190] 15. The computer-implemented method of claim 9, further comprising displaying graphical representations of the first anatomical structure segment and the second anatomical structure segment at virtual locations in the three-dimensional graphical model, the virtual locations representing physical locations of the first anatomical structure segment and the second anatomical structure segment in the three-dimensional space.

[0191] 16. The computer-implemented method of claim 9, wherein the three-dimensional graphics model displays the first image representation and the second image representation at the angle relative to each other.

[0192] 17. The computer-implemented method of claim 9, further comprising displaying a modified second image representation that is orthogonal to the first image representation when the first image plane and the second image plane are not orthogonal.

[0193] 18. The computer-implemented method of claim 9, wherein the three-dimensional graphical model can be scaled and rotated by a user.

[0194] 19. The computer-implemented method of claim 9, further comprising displaying, in the three-dimensional graphical model, a graphical representation of the first virtual position corresponding to the first imaging source and the second virtual position corresponding to the second imaging source.

[0195] 20. One or more non-transitory computer-readable storage media having stored thereon instructions that, when executed by one or more computing devices, cause the one or more computing devices to perform operations for generating a three-dimensional overview of an imaging scene of a fixator including rings and struts to correct a deformity of a first anatomical segment and a second anatomical segment, the operations comprising:

[0196] receiving, by a computing system, first and second images of the fixture and the first and second anatomical segments attached to the fixture, the first image taken from a first imaging source and the second image taken from a second imaging source, the first and second images having respective first and second image planes at an angle relative to each other;

[0197] displaying, using one or more graphical user interfaces of the computing system, a three-dimensional graphical model, the three-dimensional graphical model comprising a first image representation representing the first image and a second image representation representing the second image, wherein a first virtual line connects a first virtual position corresponding to the first imaging source to a reference position in the first image representation, and wherein a second virtual line connects a second virtual position corresponding to the second imaging source to the reference position in the second image representation; and

[0198] A first graphical representation associated with the shortest distance between the first virtual line and the second virtual line is displayed in the three-dimensional graphical model, wherein the first graphical representation represents a physical location of the reference location in three-dimensional space.

[0199] Although the technology described herein can be implemented in conjunction with the various embodiments of the various figures and has been described in conjunction with the various embodiments of the various figures, it should be understood that other similar embodiments may be used, or the embodiments may be modified or added without departing from the embodiments. For example, it should be understood that the steps disclosed above can be performed in the order described above or in any other order as needed. In addition, those skilled in the art will recognize that the technology described in this patent application can be applied to any environment, whether wired or wireless, and can be applied to any number of such devices connected via a communication network and interacting over the entire network. Therefore, the technology described herein should not be limited to any single embodiment, but should be understood in terms of breadth and scope based on the appended claims.

Claims

1. A computer-implemented method for providing a graphical projection of a fixation device comprising rings and struts to correct a deformity of a first anatomical segment and a second anatomical segment, the method comprising: displaying, using one or more graphical user interfaces of a computing system, first and second images of the first and second anatomical segments and the fixture attached to the first and second anatomical segments, the first and second images having image planes at an angle relative to each other; receiving, using the one or more graphical user interfaces, a first indication of a first position of a plurality of elements of the fixture within the first image; generating the graphical projection of the fixture including graphical representations of the plurality of elements of the fixture based at least in part on the first indication of the first positions of the plurality of elements of the fixture within the first image; as well as The graphical projection of the fixture is superimposed on the second image using the one or more graphical user interfaces, wherein the graphical projection of the fixture is rotated relative to the first position based at least in part on the angle.

2. The computer-implemented method of claim 1 , further comprising receiving, using the one or more graphical user interfaces, a second indication of a second position of the plurality of elements of the fixture within the second image.

3. The computer-implemented method of claim 2, wherein superimposing the graphical projection of the fixture on the second image assists a user in identifying the second position.

4. The computer-implemented method of claim 1, wherein the graphical projection of the fixture is movable, resizable, and rotated by a user.

5. The computer-implemented method of claim 1, wherein the deformity is corrected using a treatment regimen in which a rate of distraction is altered within an intermediate day within the treatment regimen. The computer-implemented method of claim 1 , wherein the plurality of elements of the fixture include the braces.

7. The computer-implemented method of claim 1, wherein the graphical projection of the fixture includes a graphical representation of the brace.

8. The computer-implemented method of claim 1, wherein the graphical projection of the fixture comprises a graphical representation of the ring.

9. A computer-implemented method for generating a three-dimensional overview of an imaging scene of a fixator comprising rings and struts to correct a deformity of a first anatomical segment and a second anatomical segment, the method comprising: receiving, by a computing system, first and second images of the fixture and the first and second anatomical segments attached to the fixture, the first image taken from a first imaging source and the second image taken from a second imaging source, the first and second images having respective first and second image planes at an angle relative to each other; displaying, using one or more graphical user interfaces of the computing system, a three-dimensional graphical model, the three-dimensional graphical model comprising a first image representation representing the first image and a second image representation representing the second image, wherein the one or more graphical user interfaces of the computing system display a first virtual line connecting a first virtual position corresponding to the first imaging source to a reference position in the first image representation, and wherein the one or more graphical user interfaces of the computing system display a second virtual line connecting a second virtual position corresponding to the second imaging source to the reference position in the second image representation; as well as A first graphical representation associated with the shortest distance between the first virtual line and the second virtual line is displayed in the three-dimensional graphical model displayed by using the one or more graphical user interfaces of the computing system, wherein the first graphical representation represents a physical location of the reference location in three-dimensional space.

10. The computer-implemented method of claim 9, wherein the shortest distance between the first virtual line and the second virtual line is an intersection point between the first virtual line and the second virtual line, and wherein the first graphical representation is displayed at the intersection point.

11. The computer-implemented method of claim 9, wherein the first virtual line and the second virtual line do not intersect, and wherein the first graphical representation is displayed at a point that bisects the shortest distance.

12. The computer-implemented method of claim 9, wherein the reference location is a reference point of the first anatomical segment or the second anatomical segment.

13. The computer-implemented method of claim 9, further comprising displaying graphical representations of the first virtual line and the second virtual line in the three-dimensional graphical model.

14. The computer-implemented method of claim 9, further comprising displaying graphical representations of the rings and the braces of the binder in the three-dimensional graphical model at virtual locations that represent physical locations of the rings and the braces of the binder in the three-dimensional space.

15. A computer-implemented method according to claim 9, the method also comprising displaying graphical representations of the first anatomical structure segment and the second anatomical structure segment at a virtual position in the three-dimensional graphical model, the virtual position representing the physical position of the first anatomical structure segment and the second anatomical structure segment in the three-dimensional space.

16. The computer-implemented method of claim 9, wherein the three-dimensional graphics model displays the first image representation and the second image representation at the angle relative to each other.

17. The computer-implemented method of claim 9, further comprising displaying a modified second image representation that is orthogonal to the first image representation when the first image plane and the second image plane are not orthogonal.

18. The computer-implemented method of claim 9, wherein the three-dimensional graphical model is scalable and rotatable by a user.

19. The computer-implemented method of claim 9, further comprising displaying in the three-dimensional graphical model a graphical representation of the first virtual position corresponding to the first imaging source and the second virtual position corresponding to the second imaging source.

20. One or more non-transitory computer-readable storage media having stored thereon instructions that, when executed by one or more computing devices, cause the one or more computing devices to perform operations for generating a three-dimensional overview of an imaging scene of a fixator including rings and struts to correct a deformity of a first anatomical segment and a second anatomical segment, the operations comprising: receiving, by a computing system, first and second images of the fixture and the first and second anatomical segments attached to the fixture, the first image taken from a first imaging source and the second image taken from a second imaging source, the first and second images having respective first and second image planes at an angle relative to each other; displaying, using one or more graphical user interfaces of the computing system, a three-dimensional graphical model, the three-dimensional graphical model comprising a first image representation representing the first image and a second image representation representing the second image, wherein the one or more graphical user interfaces of the computing system display a first virtual line connecting a first virtual position corresponding to the first imaging source to a reference position in the first image representation, and wherein the one or more graphical user interfaces of the computing system display a second virtual line connecting a second virtual position corresponding to the second imaging source to the reference position in the second image representation; as well as A first graphical representation associated with the shortest distance between the first virtual line and the second virtual line is displayed in the three-dimensional graphical model displayed by using the one or more graphical user interfaces of the computing system, wherein the first graphical representation represents a physical location of the reference location in three-dimensional space.