Hinge detection for orthopedic fixation

Abstract

The invention name of the present disclosure is "Hinge detection for orthopedic fixation." A first image and a second image of an anatomical segment with an attached fixator are displayed. An indication of first image hinge locations of a plurality of hinges of the fixator in the first image can be received. Projected second image hinge locations can be determined based at least in part on the first image hinge locations. Hinge candidates having shapes associated with the plurality of hinges can be detected in the second image. The hinge candidates can be detected by computer software using image analysis techniques based on automated software. Adjusted second image hinge locations can then be calculated based at least in part on the projected second image hinge locations and candidate second image hinge locations. The adjusted second image hinge locations can be used to determine physical locations of the fixator and anatomical segment in three-dimensional space, which can be used to determine manipulations of the fixator for deformity correction.

Description

BACKGROUND

[0001] Techniques for treating fractures and / or deformities of anatomical structures, such as bones, can include the use of external fixators, such as hexapods and other fixation brackets, which are surgically installed to segments of the anatomical structure on opposite sides of the fracture site. A pair of radiographic images of the fixator and the anatomical structure segments at the fracture site are taken. Data from the images are then manipulated to construct a three-dimensional illustration of the fixator and the anatomical structure segments that can be used to develop a treatment plan, which can include, for example, realigning the anatomical structure segments by adjusting the fixator.

[0002] However, existing techniques for controlling fixator manipulation can involve a number of limitations that introduce inefficiencies, complications, and unreliability issues. For example, some conventional techniques can rely on a surgeon or other user to indicate the positions of certain fixator elements, such as hinges, within images displayed in a graphical user interface of a computer. However, it can often be difficult for the user to identify and mark the positions of the hinges and other fixator elements within the images. In particular, depending on the positions and orientations at which the images are captured, it can not be easy to identify the hinges and other fixator elements, such as because they can be completely or partially overlapped with one another, or can otherwise be occluded within the images. This can make it difficult for the user to identify the fixator elements, increasing the time required to identify the elements, increasing the probability of errors, and reducing the reliability of the computations. This can reduce the reliability of the treatment plan, potentially resulting in improper alignment of the anatomical structure segments during the recovery process, poor healing between the anatomical structure segments, the need for additional rounds of radiographic imaging to facilitate alignment correction, or even the need for additional surgical procedures. SUMMARY

[0003] Described herein are techniques for hinge detection for orthopedic fixation, such as for correcting deformities of anatomical structures, such as bones. In particular, in some examples, a fixation device can be attached to a first anatomical structure segment and a second anatomical structure segment. Images, such as x-rays, of the fixation device and the attached anatomical structure segments can then be taken from different orientations with respect to the fixation device.

[0004] In some examples, various manipulations of the fixation device for correcting a deformity of the anatomical structure can be determined based on the position and orientation of the anatomical structure segment in three-dimensional space. Additionally, in some examples, the position and orientation of the anatomical structure segment in three-dimensional space can be determined based on the images. In particular, in some cases, the position and orientation of the anatomical structure segment in three-dimensional space can be determined by having a surgeon or other user indicate the positions of various fixator elements and the anatomical structure within the images. However, as noted above, it can often be difficult for a user to identify and mark the positions of certain fixator elements, such as hinges, within the images. In particular, depending on the position and orientation from which the images are captured, it can not be easy to identify hinges and other fixator elements, such as because they can be completely or partially overlapping with one another, or can otherwise be occluded within the images. This can make it difficult for the user to identify the fixator elements, increasing the time required to identify the elements, increasing the probability of errors, and decreasing the reliability of the calculations.

[0005] To mitigate the above problems and other problems, an automated or semi-automated hinge detection process can be employed. In particular, in some examples, a first image and a second image of first and second anatomical structure segments to which a fixation device is attached can be displayed. An indication of first image hinge locations associated with a plurality of hinges in the first image can be received. Projected second image hinge locations associated with the plurality of hinges in the second image can then be determined based at least in part on the indication of the first image hinge locations. Hinge candidates having shapes associated with the plurality of hinges can be detected in the second image. The hinge candidates can be detected by computer software using image analysis techniques based on automated software. For example, the hinges can have a circular shape, and the computer software can employ a circular detection algorithm, such as a Hough transform, to identify circular shapes in the second image as hinge candidates. Candidate second image hinge locations of the hinge candidates within the second image can then be identified.

[0006] Adjusted second image hinge locations associated with the plurality of hinges can then be computed within the second image based at least in part on the projected second image hinge locations and the candidate second image hinge locations. In some examples, to compute the adjusted second image hinge locations, hinge candidates can be grouped into a set of hinge candidate groups, e.g., based on similarity of their locations and dimensional characteristics (e.g., radius length). The set of hinge candidate groups can then be weighted based at least in part on a number of hinge candidates within each hinge candidate group of the set of hinge candidate groups. A highest weighted subset of hinge candidate groups can then be selected from the set of hinge candidate groups. A plurality of average group locations can then be computed, where each average group location of the average group locations is associated with a respective hinge candidate group of the highest weighted subset of hinge candidate groups. A transformation matrix describing a spatial relationship between the projected second image hinge locations and the plurality of average group locations can then be constructed. The transformation matrix can then be used to adjust the projected second image hinge locations to the adjusted second image hinge locations. The adjusted second image hinge locations can then be used to determine physical positions of the physical fixation device and the first and second anatomical structure segments in three-dimensional space. Manipulation of the fixation device for correction of the deformity can then be determined using the physical positions of the fixation device and the first and second anatomical structure segments. BRIEF DESCRIPTION OF DRAWINGS

[0007] The patent or application file contains at least one drawing executed in color. Copies of this patent or patent application publication with color drawing(s) will be provided by the Office upon request and payment of the necessary fee.

[0008] The foregoing summary, as well as the following detailed description of preferred embodiments of the application, will be better understood when read in conjunction with the accompanying drawings, in which:

[0009] Figure 1 is a perspective view of a fixation assembly positioned for imaging in accordance with an embodiment;

[0010] Figure 2 is Figure 1 is a perspective view of an exemplary imaging process of the fixation assembly shown;

[0011] Figure 3A and Figure 3B is a flowchart illustrating an exemplary process for controlling manipulation of a fixation device to correct a deformity of an anatomical structure;

[0012] Figure 4is a screenshot of an exemplary interface for selecting a perspective frame match (PFM) technique;

[0013] Figure 5 is a screenshot of an exemplary configuration information input interface for a PFM technique;

[0014] Figure 6 is a screenshot of an exemplary first image information input interface for a PFM technique;

[0015] Figure 7 is a screenshot of an exemplary close-up assist interface for a PFM technique;

[0016] Figures 8A-8H is a screenshot of an exemplary second image information input interface for a PFM technique;

[0017] Figure 9 is a screenshot of an exemplary deformity parameter interface for a PFM technique;

[0018] Figure 10 is a screenshot of an exemplary mounting parameter interface for a PFM technique;

[0019] Figure 11 is a screenshot of a first exemplary treatment plan interface for a PFM technique;

[0020] Figure 12 is a screenshot of a second exemplary treatment plan interface for a PFM technique;

[0021] Figure 13 is a screenshot of a third exemplary treatment plan interface for a PFM technique;

[0022] Figure 14A is a flowchart showing an exemplary process for hinge detection for orthopedic fixation;

[0023] Figure 14B is a flowchart showing an exemplary process for calculating an adjusted second image hinge position;

[0024] Figure 15A is a diagram showing an exemplary image of a first anatomical structure segment and a second anatomical structure segment and a fixator attached to the first and second anatomical structure segments;

[0025] Figure 15B is a diagram showing exemplary first and second images of a fixator, where a hinge position is indicated in the first image but not in the second image;

[0026] Figure 16 is a diagram showing an exemplary graphical projection of a fixator superimposed on an image;

[0027] Figure 17 This is a diagram illustrating an exemplary graphical projection of a fastener manipulated by a user;

[0028] Figure 18 This is a diagram illustrating the hinge position of the second image in an exemplary projection;

[0029] Figure 19 This is a diagram showing an exemplary list of hinge candidates;

[0030] Figure 20 This is a diagram showing a list of exemplary hinge candidate groups;

[0031] Figure 21 This is a diagram illustrating an exemplary set of hinge candidates;

[0032] Figure 22 This is a diagram illustrating an example representation of the transformation matrix used to calculate the adjusted hinge position of the second image;

[0033] Figure 23 This is a diagram illustrating an exemplary adjustment of the hinge position of the projected second image;

[0034] Figure 24 This is a diagram illustrating an exemplary adjusted position of the second image hinge;

[0035] Figure 25 This is a diagram showing an exemplary first and second image of the fastener, wherein the hinge position is indicated in the first image, and the adjusted second image hinge position is calculated in the second image; and

[0036] Figure 26 This is a block diagram of an exemplary computing device used in accordance with this disclosure. Detailed Implementation

[0037] For convenience, the same or equivalent elements in the embodiments shown in the accompanying drawings are identified by the same reference numerals. Some terms used in the following description are for convenience only and are not restrictive. The terms “right,” “left,” “top,” and “bottom” specify directions in the drawings for reference. The terms “inward,” “inner,” “outward,” and “outer” refer to directions toward and away from the geometric center of the device and / or its designated components, respectively. The terms are intended to include, in a non-limiting manner, the above terms, derivatives of the above terms, and terms with similar meanings.

[0038] First see Figure 1Body tissues, such as first and second anatomical segments 102, 104, can be aligned and / or oriented to promote healing or cure between body tissues. Anatomical structures may include, for example, anatomical tissues and artificial anatomical implants. Anatomical tissues may include, for example, bone or other tissues within the body. Alignment and / or orientation of body tissues can be achieved by attaching the body tissues to an adjustable fixation device, such as orthopedic fixator 100. The orthopedic fixator may include an external fixation device comprising multiple discrete fixation components that are always outside the patient's body but (e.g., with minimally invasive attachment components) attached to corresponding discrete body tissues. The fixation device may include, for example, a distraction osteogenic ring system, a hexagonal frame, or a Taylor space scaffold. By adjusting the spatial positioning of the fixation components relative to each other, the attached corresponding body tissues can be reoriented and / or otherwise aligned with each other, for example, to promote healing between body tissues during recovery. The use of external orthopedic fixators in conjunction with the image analysis and localization techniques described herein is advantageous in applications where direct measurement and manipulation of body tissue is not possible, or where limited or minor wounds to body tissue are desired. Examples of orthopedic fixators and their use in correcting deformities of anatomical segments, as well as techniques for performing imaging analysis on fixators and anatomical segments, are described in U.S. Patent No. 9,642,649, entitled “ORTHOPEDIC FIXATION WITH IMAGERY ANALYSIS,” published May 9, 2017, the entire contents of which are incorporated herein by reference.

[0039] The fixator components can be connected to each other by an adjusting member configured to facilitate spatial repositioning of the fixator components relative to each other. For example, in the illustrated embodiment, the orthopedic fixator 100 includes a pair of fixator components in the form of an upper fixation ring 106 and a lower fixation ring 108. The fixation rings 106 and 108 can be constructed in the same or different ways. For example, the diameters of the fixation rings 106 and 108 can be the same or different. Similarly, the fixation rings 106 and 108 can be constructed with different cross-sectional diameters, thicknesses, etc. It should be understood that the fixator components of the orthopedic fixator 100 are not limited to the illustrated upper and lower fixation rings 106 and 108, and the orthopedic fixator 100 can have other configurations. For example, additional fixation rings can be provided and interconnected with fixation rings 106 and / or 108. It should also be understood that the geometry of the fixator components is not limited to rings, and at least one fixator component, such as all fixator components, can be constructed using any other suitable geometry.

[0040] The first anatomical segment and the second anatomical segment 102, 104 can be rigidly attached to the upper and lower fixing rings 106, 108, respectively, wherein the attachment members can be installed to the upper and lower fixing rings 106, 108. For example, in the illustrated embodiment, the attachment members are provided in the form of an attachment rod 110 and an attachment line 112.

[0041] Rod 110 and wire 112 extend between the proximal end of the mounting member 114, which is attached to and mounted to retaining rings 106, 108, and the opposite distal end, which is inserted into or otherwise secured to the anatomical segment 102, 104. The mounting member can be detachably mounted to a predetermined position along the edge of retaining rings 106, 108, for example, by inserting the mounting member 114 into a threaded hole defined by the retaining ring. For each retaining ring 106, 108, the mounting member 114 can be mounted to the upper surface of the ring, the lower surface of the ring, or any combination thereof. It should be understood that the attachment members are not limited to the configuration of the illustrated embodiment. For example, any number of attachment members (such as the rod 110 and wire 112 shown, and any other members) can be used as needed to secure the anatomical segment to the corresponding retainer member. It should also be understood that one or more of the attachment members (e.g., rod 110 and / or line 112) may alternatively be configured to be directly mounted to retaining rings 106, 108 without using mounting member 114.

[0042] The upper and lower retaining rings 106, 108 can be connected to each other by at least one adjusting member, such as multiple adjusting members. At least one (such as all) adjusting member can be configured to allow adjustment of the spatial positioning of the retaining rings relative to each other. For example, in the illustrated embodiment, the upper and lower retaining rings 106, 108 are connected to each other by multiple adjusting members provided in the form of length-adjustable support bars 116. It should be understood that the construction of the orthopedic fixator 100 is not limited to the six support bars 116 of the illustrated embodiment, and more or fewer support bars may be used as needed.

[0043] Each length-adjustable support 116 may include opposing upper and lower support arms 118, 120. Each of the upper and lower support arms 118, 120 has: a proximal end disposed in the connecting member or sleeve 122; and an opposing distal end connected to a universal joint 124, which is respectively mounted to upper and lower retaining rings 106, 108. The universal joints of the illustrated embodiment are arranged in pairs at even intervals along the edges of the upper and lower retaining rings 106, 108, but may alternatively be placed in any other location on the retaining rings as needed.

[0044] Each support 116 may have a threaded proximal end of its upper and lower support arms 118, 120, configured to be received by a complementary thread defined in the sleeve 122, such that when the proximal ends of the upper and lower support arms 118, 120 of the support 116 are received in the respective sleeve 122, rotating the sleeve 122 will cause the upper and lower support arms 118, 120 to translate within the sleeve 122, thereby causing the support 116 to lengthen or shorten in the direction of rotation. Therefore, the length of each support 116 can be adjusted individually relative to the remaining supports. It should be understood that the adjusting member is not limited to the length-adjustable support 116 of the illustrated embodiment, and the adjusting member may alternatively be constructed as needed, for example by one or more alternative geometries, alternative length adjustment mechanisms, etc.

[0045] The adjustable-length support bar 116 and the universal joint 124 (the adjustable-length support bar is attached to the upper fixation ring 106 and the lower fixation ring 108 via the universal joint) allow the orthopedic fixator 100 to function very similarly to a Stewart platform, and more specifically, to a distraction osteogenic ring system, a hexagonal frame, or a Taylor space scaffold. That is, by adjusting the length of the support bar 116, the spatial positioning of the upper and lower fixation rings 106, 108 can be altered, thereby changing the spatial positioning of the anatomical segments 102, 104. For example, in the illustrated embodiment, the first anatomical segment 102 is attached to the upper fixation ring 106, and the second anatomical segment 104 is attached to the lower fixation ring 108. It should be understood that the attachment of the first and second anatomical segments 102, 104 to the upper and lower fixation rings 106, 108 is not limited to the illustrated embodiment (e.g., where the central longitudinal axes L1, L2 of the first and second anatomical segments 102, 104 are substantially perpendicular to the corresponding planes of the upper and lower fixation rings 106, 108), and when constructing the orthopedic fixator 100, the surgeon has complete flexibility in aligning the first and second anatomical segments 102, 104 within the upper and lower fixation rings 106, 108.

[0046] By changing the length of one or more support bars 116, the upper and lower fixation rings 106, 108, and therefore the anatomical segments 102, 104, can be repositioned relative to each other so that their respective longitudinal axes L1, L2 are substantially aligned with each other, and their corresponding fracture ends 103, 105 are adjacent to each other, in order to promote healing during the recovery process. It should be understood that the adjustment of the support bars 116 is not limited to the length adjustment described herein, and the support bars 116 can be adjusted in different ways as needed. It should also be understood that adjusting the position of the fixation components is not limited to adjusting the length of the length-adjustable support bars 116, and the positioning of the fixation components relative to each other can alternatively be adjusted, for example, according to the type and / or number of adjusting components connected to the fixation device.

[0047] Repositioning of the fixator components of an orthopedic fixation device (such as orthopedic fixator 100) can be used to correct angular, translational, rotational, or any combination thereof displacements within body tissues. Fixation devices used with the techniques described herein, such as orthopedic fixator 100, can correct multiple such displacement defects individually or simultaneously. However, it should be understood that fixation devices are not limited to the orthopedic fixator 100 shown, and fixation devices may have other configurations as needed. For example, a fixation device may include additional fixation components, may include fixation components with alternative geometries, may include more or fewer adjustment components, may include adjustment components with alternative configurations, or any combination thereof.

[0048] See now Figure 2 Exemplary imaging techniques for fixed devices will now be described in detail. Images can be captured using the same or different imaging techniques. For example, images can be acquired using techniques such as X-ray imaging, computed tomography, magnetic resonance imaging, ultrasound imaging, infrared imaging, photography, fluorescence imaging, visible spectral imaging, or any combination thereof.

[0049] Images can be captured at any position and / or orientation relative to each other and relative to the fixator 100 and anatomical segments 102, 104. In other words, it is not required that the captured images be orthogonal to each other or aligned with the patient's anatomical axes, thus providing the surgeon with near-complete flexibility in the positioning of the imager 130. Preferably, images 126, 128 are captured from different directions or orientations such that the images do not overlap. For example, in the illustrated embodiment, the image planes of image pair 126, 128 are not perpendicular to each other. In other words, the angle α between the image planes of images 126, 128 is not equal to 90 degrees, such that images 126, 128 are not orthogonal to each other. Preferably, at least two images are acquired, but capturing additional images can improve the accuracy of the method.

[0050] Images 126, 128 can be captured by one or more imaging sources or imagers, such as X-ray imager 130 and / or corresponding image capturing devices 127, 129. Images 126, 128 can be X-ray images captured by a single repositionable X-ray imager 130, or can be captured by an imager 130 positioned individually. Preferably, the positions of the image capturing devices 127, 129 and / or imager 130 relative to a spatial origin 135 in three-dimensional space (described in more detail below) are known. Imager 130 can be manually positioned and / or oriented under the control of a surgeon, or automatically positioned, for example, by software-assisted imager positioning, or any combination thereof. Fixator 100 may also have a corresponding fixator origin 145.

[0051] Now for reference Figure 3Aand Figure 3B An exemplary process will now be described in detail for controlling the manipulation of a fixation device comprising rings and struts to correct anatomical deformities in a first anatomical segment and a second anatomical segment. Specifically, at operation 310, the first and second anatomical segments are attached to the fixation device, for example, as... Figure 1 As shown and detailed above. In operation 312, first and second images of the fixed device and the attached first and second anatomical segment are captured, for example, as... Figure 2 As shown and detailed above.

[0052] We will now introduce the treatment technique, referred to below as fluoroscopic stent matching. Figure 3A and Figure 3B Other procedures during the process (e.g., procedures 314 to 342) include images (such as postoperative X-ray images) that can be used in conjunction with the stent to generate deformity and installation parameters for the strut adjustment scheme. For example, see now. Figure 4 An exemplary treatment planning technology selection interface 400-A is shown. Figure 4 In the example, the user has selected option 401 to use the fluoroscopic support fitting (PFM) technique, which will now be seen... Figures 5-13 A detailed introduction will be provided.

[0053] See again Figure 3A In operation 314, configuration information associated with the fixed device is received, for example, through one or more graphical user interfaces of the computing system. In some examples, the configuration information may include one or more geometric features (e.g., dimensions, length, diameter, etc.) of one or more components of the fixed device (e.g., struts, hinges, rings, etc.). In some examples, the configuration information may include information such as ring type (e.g., full ring, foot plate, etc.), indication of mounting points (e.g., ring holes) for strut mounting, and other information. In some examples, the configuration information may also include marking element information, such as marking elements mounted on fixed device components (e.g., struts, hinges, and rings). See now. Figure 5 An exemplary configuration information input interface 500 is shown. As illustrated, interface 500 includes ring type indicators 501 and 502, which in this example are drop-down menus for selecting the ring type of the proximal and distal rings, respectively. When indicators 501 and 502 are set to the "Full" option, it indicates that both the proximal and distal rings are full rings. Interface 500 also includes diameter indicators 503 and 504, which in this example are drop-down menus for selecting the diameter or length of the proximal and distal rings, respectively.

[0054] Interface 500 also includes controls for inputting support bar information. Specifically, interface 500 includes six drop-down menus 512, each of which can be used to indicate the size of the corresponding support bar. A global support bar size indicator 511 can also be used to globally select a size for all six support bars. Length selectors 513 can each be used to select the length of the corresponding support bar. Length indicators 514 can each be used to provide a visual representation of the length of the corresponding support bar. It should be noted that length indicators 514 do not necessarily show the actual exact length of each support bar, but rather represent the comparative length of the support bars relative to each other.

[0055] The “Save and Update” button 516 can be selected to save and update the configuration information values ​​shown in the interface 500. In some examples, selecting button 516 can cause the interface 500 to display and / or update a graphical representation 520 of the fixed device, which is generated at least in part based on the input configuration information. The graphical representation 520 can be displayed through one or more graphical user interfaces of the computing system. As shown, the graphical representation 520 includes six support bars, which can be color-coded in multiple colors for easy differentiation. For example, in some cases, each support bar (or at least two support bars) is displayed in a different color than each other. The support bars in the graphical representation 520 may have dimensions, lengths, mounting points, and other characteristics corresponding to the input configuration information. The graphical representation 520 also shows fixing rings, which may have diameters / lengths, ring types, and other characteristics corresponding to the input configuration information. For example, the graphical representation 520 can improve efficiency and reliability by providing the user with visual confirmation of the information input into the interface 500, for example, to enable quick and easy identification of errors or other problems.

[0056] In operation 316, images of the fixation device and the first and second anatomical segments attached to the fixation device are displayed, for example, via one or more graphical user interfaces of a computing system. The displayed images may include images captured in operation 312, such as images captured using techniques including X-ray imaging, computed tomography, magnetic resonance imaging, ultrasound imaging, infrared imaging, photography, fluoroscopy, visible spectroscopy, or any combination thereof. The techniques for acquiring images of the fixation device and the first and second anatomical segments have been detailed above and will not be repeated here. As mentioned above, the acquired and displayed images need not be orthogonal to each other. See now. Figure 6 An exemplary first image information input interface 600 is shown. As shown, interface 600 includes images 601-A and 601-B, which illustrate the fixation device and the first and second anatomical structure segments from different angles. Figure 6In the example, image 601-A corresponds to the front-to-back (AP) view, while image 601-B corresponds to the side-to-side (LAT) view. In some examples, the displayed image 601-AB can be loaded and stored in computer memory, for example, in an image library, database, or other local collection where images are stored. The displayed image 601-AB can then be selected, retrieved, and / or received from memory for display.

[0057] In operation 318, the first image information is received, for example, through one or more graphical user interfaces of a computing system. The first image information may include indications of one or more locations within the image of at least a portion of one or more components of the fixing device. For example, the first image information may include one or more indications of the location of struts, hinges, rings, and other fixing components. In some examples, the first image information may also include location information of marking elements within the image, such as marking elements mounted on fixing device components (such as struts, hinges, and rings). In some cases, the first image information may include points indicating hinge locations and / or lines or vectors indicating strut locations. In some examples, the first image information may be input into the computing system by, for example, by selecting or indicating one or more locations within the displayed image using a mouse, keyboard, touchscreen, or other user input device. Specifically, the user may use one or more input devices to select points or other locations in the image, draw lines, circles, and generate other graphical indications within the image. For example, in some cases, the user may generate points or small circles at specific locations in the image to indicate the location of a hinge within the image (e.g., a center point). For example, in some cases, users can generate lines and / or vectors within an image to indicate the position and / or length of the struts within the image.

[0058] For example, such as Figure 6As shown, interface 600 includes six "AP View" support bar indicator buttons 611-A, corresponding to each of the six supports of the fixed device shown in "AP View" image 601-A. Each button 611-A includes text indicating the corresponding support bar number (i.e., support bar 1, support bar 2, support bar 3, support bar 4, support bar 5, support bar 6). A user can select a button 611-A to indicate a support bar, providing initial image information (e.g., hinge position, support bar position, etc.) for that support bar in "AP View" image 601-A. For example, in some cases, to provide initial image information for support bar 1 in "AP View" image 601-A, the user can first select the topmost support bar indicator button 611-A (labeled "Support Bar 1") to prompt the software user that initial image information for support bar 1 will soon be provided in "AP View" image 601-A. In some cases, the support bar indicator button 611-A for support bar 1 can be automatically pre-selected for the user. After selecting (or automatically pre-selecting) the support indicator button 611-A for support bar 1, the user can continue to draw (or otherwise indicate) a diagram of support bar 1 within the "AP View" image 601-A. For example, in some cases, the user can use a mouse or other input device to select the position 621 of the proximal hinge of support bar 1 (e.g., the center point) within image 601-A. In some examples, the user can then use a mouse or other input device to select the position 622 of the distal hinge of support bar 1 (e.g., the center point) within image 601-A. In some examples, the user can indicate the position and / or length of support bar 1 by selecting the positions of the proximal and distal hinges and / or the endpoints of a straight line or vector representing the position and / or length of support bar 1. For example, as... Figure 6 As shown, the software can generate points or circles at positions 621 and 622 of the proximal and distal hinges selected by the user within image 601-A. Additionally, the software can generate a straight line 623 representing the position and / or length of the strut 1 connecting the points or circles at positions 621 and 622, as well as the position and / or length of the proximal and distal hinges selected by the user within image 601-A. The user can also use any other suitable input technique to indicate the position and / or length of the strut 1 within image 610-A, such as by dragging a mouse, using a touchscreen, keypad, or other device with a finger and / or pen to generate the straight line 623. In some examples, the above process can be repeated to draw points representing the proximal and distal hinges and straight lines representing the position and / or length of each of the six struts in the "AP view" image 601-A. In addition, the above process can be repeated via the "LAT View" strut indicator button 611-B to draw points representing the proximal and distal hinges and straight lines representing the position and / or length of each of the six struts in the "LAT View" image 601-B.

[0059] In some examples, the first image information generated within images 601-A and 601-B may include a color-coded graphic representation of the strut, for example, to make the graphic representation more explicitly associated with its corresponding strut. For example, in Figure 6 In images 601-A and 601-B, the image representation of support bar 1 (e.g., a dot, circle, and / or line) can be highlighted in red. This matches the support bar icon displayed in the support bar indicator buttons 611-A and 611-B (which can also be highlighted in red) (displayed to the right of the text "Support Bar 1" in buttons 611-A and 611-B). For example, in... Figure 6 In the images 601-A and 601-B, the graphical representation of support bar 3 (e.g., dots, circles, and / or lines) can be highlighted in yellow. This can match the support bar icon displayed in the support bar indicator buttons 611-A and 611-B (which can also be highlighted in yellow) (displayed to the right of the text "Support Bar 3" in buttons 611-A and 611-B).

[0060] Figure 6 Includes checkboxes 616-A ("AP View") and 616-B ("LAT View"), which are provided, for example, through one or more graphical interfaces of the computing system. Selecting checkboxes 616-A and 616-B enables close-up views of multiple areas of images 601-A and 601-B, adjacent to the proximal and distal hinges of the strut the user is currently drawing. This allows for more accurate indication of the hinge's location (e.g., center point). See now. Figure 7 The close-up auxiliary interface 700 displays another "AP view" image 701, where the close-up auxiliary is selected to provide a close-up auxiliary view 702 for the proximal hinge and a close-up auxiliary view 703 for the distal hinge. As shown, the close-up auxiliary view 702 for the proximal hinge provides a magnified view of a region of the "AP view" image 701 associated with the proximal hinge, while the close-up auxiliary view 703 for the distal hinge provides a magnified view of a region of the "AP view" image 701 associated with the distal hinge. In the close-up auxiliary view 702 for the proximal hinge, the user can manipulate (e.g., drag) the position of point / circle 721 to more accurately depict the center point of the proximal hinge. In the close-up auxiliary view 703 for the distal hinge, the user can also manipulate (e.g., drag) the position of point / circle 722 to more accurately depict the center point of the distal hinge. It should be understood that, for example, through one or more graphical interfaces of the computing system, corresponding close-up auxiliary views similar to views 702 and 703 can also be provided for the corresponding "LAT view" images.

[0061] See you again Figure 6To the right of button 611-A are six proximal hinge selector buttons 612-A. Additionally, to the right of button 612-A are six distal hinge selector buttons 613-A. Furthermore, to the right of button 613-A are six strut line selector buttons 614-A. In some examples, buttons 612-A and / or 613-A can be selected to calculate the position and orientation in three-dimensional space of the first and second anatomical segments and the ring of the fixation device using the positions (e.g., center points) of the proximal and / or distal hinges indicated in “AP view” image 601-A (see Operation 322). Additionally, in some examples, buttons 612-A and / or 613-A can be selected to calculate the position and orientation in three-dimensional space of the first and second anatomical segments using the lines or vectors indicating the strut positions and / or lengths indicated in “AP view” image 601-A. Similarly, buttons 612-B, 613-B, and 614-B can be used to select: to calculate the position and orientation of the first and second anatomical segments in three-dimensional space using the positions of the proximal and / or distal hinge or strut lines indicated in the LAT view image 601-B.

[0062] See you again Figure 3A In operation 320, second image information is received, for example, through one or more graphical user interfaces of a computing system. The second image information may include indications of one or more locations within the image of at least a portion of the first and second anatomical segments. In some examples, the second image information may include indications of the centerlines of the first and second anatomical segments and / or one or more reference points (e.g., endpoints) of the first and second anatomical segments. In some examples, the second image information may also include indications of the location of, for example, implanted or otherwise associated with the first and second anatomical segments. In some examples, the second image information may be input into the computing system by, for example, by selecting or indicating one or more locations within a displayed image using a mouse, keyboard, touchscreen, or other user input device. Specifically, a user may use one or more input devices to select points or other locations in the image, draw lines, circles, and generate other graphical indications within the image. For example, in some cases, a user may generate points or small circles at specific locations in the image to indicate one or more reference points (e.g., endpoints) of the first and second anatomical segments within the image. For example, in some cases, users can generate a straight line within the image to indicate the center line of the first and second anatomical segments within the image.

[0063] See now Figure 8AAn exemplary second image information input interface 800 is shown. As shown, interface 800 includes “AP view” images 601-A and “LAT view” images 601-B. Additionally, interface 800 includes buttons 801-808, which can be used to assist in indicating the center lines and reference points of anatomical structures, as described below. Specifically, buttons 801 and 805 can be selected to indicate proximal anatomical structure reference points in the “AP view” and “LAT view,” respectively. Buttons 802 and 806 can be selected to indicate distal anatomical structure reference points in the “AP view” and “LAT view,” respectively. Buttons 803 and 807 can be selected to indicate the proximal anatomical structure center lines in the “AP view” and “LAT view,” respectively. Buttons 804 and 808 can be selected to indicate the distal anatomical structure center lines in the “AP view” and “LAT view,” respectively. For example, as... Figure 8A As shown, the user can select button 807 and then draw a proximal anatomical centerline 831 within the "LAT View" image 601-B using one or more input devices. In some examples, the centerline 831 can be colored red. Additionally, the software generates and displays two guide lines 832 on either side of the red centerline. In some examples, the guide lines 832 can be colored green. These guide lines 832 can be displayed while the user draws the centerline 831 to assist the user in locating the center of the anatomical segment. The guide lines 831 can be generated at equidistant positions from each side of the centerline 832, and can assist the user by, for example, potentially allowing the user to match (or nearly match) the guide lines 832 to the sides of the anatomical segment. Figure 8B As shown, the user can select button 808 and then, through one or more input devices, draw the centerline 841 of the distal anatomical structure within the "LAT view" image 601-B. Figure 8C As shown, the user can select button 803 and then draw the centerline 851 of the proximal anatomical structure within the "AP view" image 601-A using one or more input devices. Figure 8D As shown, the user can select button 804 and then draw the centerline 861 of the distal anatomical structure within the "AP view" image 601-A via one or more input devices. Figures 8B-8D As shown, conductor 832 can also be displayed to assist in drawing center lines 841, 851 and 861.

[0064] like Figure 8EAs shown, the user can select button 805 and then, through one or more input devices, indicate a reference point (e.g., an endpoint) of the proximal anatomical structure within the "LAT View" image 601-B. As shown, in the "LAT View" image 601-B, the user has indicated reference point 811 at the endpoint of the proximal anatomical structure segment. Furthermore, after indicating reference point 811, the software can generate and display the corresponding dashed reference line 812 in the "AP View" image 601-A. Reference line 812 is a line drawn across the "AP View" image 601-A, passing through the location of the proximal reference point 811 in the "LAT View" within the "AP View" image 601-A. Therefore, reference line 812 helps the user determine the location of the corresponding proximal reference point in the "AP View," which is typically at the intersection of reference line 812 and the proximal centerline 851 in the "AP View" image 601-A. Figure 8F As shown, the user can select button 801 and then, through one or more input devices, indicate a reference point (e.g., an endpoint) for the proximal anatomical structure within the "AP View" image 601-A. In this example, the "AP View" proximal anatomical structure reference point 814 is shown at the intersection of reference line 812 and the "AP View" proximal centerline 851 within the "AP View" image 601-A. The software can then generate and display the corresponding dashed reference line 813 in the "LAT View" image 601-B. Reference line 813 is a line drawn across the "LAT View" image 601-B, passing through the position of the "AP View" proximal reference point 814 within the "LAT View" image 601-B. Reference line 813 assists the user in confirming that the "AP View" reference point 811 is correctly placed by displaying its alignment relative to the "LAT View" reference point 814.

[0065] like Figure 8G As shown, the user can select button 806 and then, through one or more input devices, indicate a reference point (e.g., an endpoint) of the distal anatomical structure within the "LAT View" image 601-B. As shown, in the "LAT View" image 601-B, the user has indicated reference point 815 at the endpoint of the distal anatomical structure segment. Furthermore, after indicating reference point 815, the software can generate and display the corresponding dashed reference line 816 in the "AP View" image 601-A. Reference line 816 is a line drawn across the "AP View" image 601-A, passing through the location of the distal reference point 815 in the "LAT View" within the "AP View" image 601-A. Therefore, reference line 816 helps the user determine the location of the corresponding distal reference point in the "AP View," which is typically at the intersection of reference line 816 and the distal centerline of the "AP View" within the "AP View" image 601-A. Figure 8HAs shown, the user can select button 802 and then, through one or more input devices, indicate a reference point (e.g., an endpoint) for the distal anatomical structure within the "AP View" image 601-A. In this example, the distal anatomical structure reference point 817 in the "AP View" is shown at the intersection of reference line 816 and the distal center line of the "AP View" within the "AP View" image 601-A. The software can then generate and display the corresponding dashed reference line 818 in the "LAT View" image 601-B. Reference line 818 is a line drawn across the "LAT View" image 601-B, passing through the position of the distal reference point 817 in the "AP View" within the "LAT View" image 601-B. Reference line 818 assists the user in confirming that the "AP View" reference point 817 is correctly positioned by displaying its alignment relative to the "LAT View" reference point 815.

[0066] See you again Figure 3A In operation 322, the positions and orientations of the first and second anatomical segments and the rings of the fixation device in three-dimensional space are determined. For example, in some cases, imaging scene parameters associated with the fixator 100, anatomical segments 102, 104, one or more imagers 130, and image capturing devices 127, 129 are obtained. The imaging scene parameters can be used to construct a three-dimensional representation of the positioning of the anatomical segments 102, 104 within the fixator 100, as described in more detail below. One or more of the imaging scene parameters may be known. Unknown imaging scene parameters can be obtained, for example, by mathematically comparing the positions of the fixator elements in the two-dimensional space of the x-ray images 126, 128; and the three-dimensional positions of these elements within the geometry of the fixator 100. In a preferred embodiment, the imaging scene parameters can be calculated using a pinhole camera or fluoroscopic camera model. For example, the imaging scene parameters can be determined numerically using matrix algebra, as described in more detail below.

[0067] The imaging scene parameters may include, but are not limited to: image pixel scaling factor, image pixel aspect ratio, image sensor skew factor, image size, focal length, position and orientation of the imaging source, position of the principal point (defined as the point on the plane of the corresponding image 126, 128 that is closest to the corresponding imager 130), position and orientation of multiple elements of the fixture 100, position and orientation of the corresponding image receiver, and position and orientation of the imaging source lens.

[0068] In a preferred embodiment, at least some (such as all) imaging scene parameters can be obtained by comparing: the illustrated position of a particular component or the illustrated position of the fixation element of fixator 100 in the two-dimensional space of images 126, 128; and the corresponding position of these identical fixation elements in actual three-dimensional space. Fixation elements include components of the orthopedic fixator 100, and are preferably components easily identifiable in images 126, 128. Points, lines, cones, etc., or any combination thereof, can be used to describe the corresponding geometry of the fixation element. For example, the illustration of the fixation element used in this comparison process may include the centerline of one or more of the length-adjustable struts 116, the center point of the universal joint 124, the center point of the mounting member 114, etc.

[0069] The fixation element may also include a marking element, which is different from the aforementioned components of the fixation 100. The marking element can be used in the comparison process as a supplement or replacement to the components using the fixation 100. Prior to imaging, the marking element can be mounted to specific component locations on the fixation 100, embedded within components of the fixation 100, or any combination thereof. The marking element can be configured to provide higher visibility of images 126, 128 compared to the visibility of other components of the fixation 100. For example, the marking element can be constructed from different materials, such as radiopaque materials, or with a geometry that is easily distinguishable from other components of the fixation 100 in images 126, 128. In an exemplary embodiment, the marking element may have a specified geometry corresponding to its respective location on the fixation 100.

[0070] Fixator elements can be identified for use during the comparison process. For example, the position of the fixator elements within images 126, 128 can be indicated using the first image information received in operation 318 and as detailed above. In some examples, the position of the fixator elements in the two-dimensional space of images 126, 128 can be determined relative to a local origin 125 defined in the imaging plane of images 126, 128. The local origin 125 serves as a “zero point” for determining the position of the fixator elements in images 126, 128. The position of the fixator elements can be defined by their respective x and y coordinates relative to the corresponding local origin 125. The position of the local origin 125 within the corresponding image can be arbitrary, as long as it is in the image plane. Typically, the origin is located at the center of the image or at an image corner, such as the lower left corner. It should be understood that the position of the local origin is not limited to the local origin 125 shown, and the local origin 125 can be defined at any other location.

[0071] In some examples, the transformation matrix P corresponding to each of images 126 and 128 can then be calculated. The transformation matrix can be used to map the position coordinates of one or more corresponding fixator elements in actual three-dimensional space to their corresponding position coordinates in the two-dimensional space of the corresponding images 126 and 128. It should be understood that when comparing two images 126 and 128, it is not necessary to use the same one or more fixator elements. For example, the fixator elements used to construct the transformation matrix associated with image 126 can be the same as or different from the fixator elements used to construct the transformation matrix associated with image 128. It should also be understood that the greater the number of fixator elements used to calculate the transformation matrix, the higher the accuracy of the method. The following formula can represent this calculation process:

[0072]

[0073] The symbols x and y represent the position coordinates of the fixator element point in the two-dimensional space of images 126 and 128 relative to the local origin 125. The symbols X, Y, and Z represent the corresponding position coordinates of the fixator element point in the actual three-dimensional space relative to the spatial origin 135. In the illustrated embodiment, the spatial origin 135 has been designated as the point corresponding to the center of the plane defined by the upper surface of the upper fixing ring 106. The matrix P shown may have at least four elements wide and three elements high. In a preferred embodiment, the elements of matrix P can be calculated by solving the following matrix formula:

[0074] A·p=B (2)

[0075] A vector p can contain eleven elements representing the values ​​of a matrix P. The following equations illustrate the arrangement of elements in vector p and matrix P:

[0076] p=[p1 p2 p3 p4 p5 p6 p7 p8 p9 p 10 p 11 ] T (3)

[0077]

[0078] In this preferred embodiment, the twelfth element p of matrix P can be... 12 Set to a value of one. Matrices A and B can be combined using two-dimensional and three-dimensional information of the fastener elements. For each point representing the corresponding fastener element, two rows of matrices A and B can be constructed. The following formulas give the values ​​added to the two rows of matrices A and B for each point of the fastener element (e.g., the center point of the corresponding universal joint 124):

[0079]

[0080] The symbols X, Y, and Z represent the position coordinates of the fixation element points in the actual three-dimensional space relative to the spatial origin 135, and the symbols x and y represent the position coordinates of the corresponding fixation element points in the two-dimensional space of the corresponding images 126 and 128 relative to the local origin 125.

[0081] For each line representing a corresponding fastener element, two rows of matrices A and B can be constructed. The following formulas give the values ​​added to the two rows of matrices A and B for each line of the fastener element (e.g., the center line of the corresponding length adjustable strut 116):

[0082]

[0083] Symbols X, Y, and Z represent the position coordinates of a point belonging to the fixture element line in actual three-dimensional space relative to the spatial origin 135. Symbols dX, dY, and dZ represent the gradient values ​​of the line in actual three-dimensional space. Symbols a, b, and c represent constants that define a line in the two-dimensional space of the corresponding images 126 and 128. For example, a, b, and c can be calculated using two points belonging to a line in the corresponding images 126 and 128. In a preferred embodiment, b is assumed to be 1 unless the line is vertical (in which case the value of b is zero). The correlation of constants a, b, and c, and the corresponding image coordinates x and y, are given in the following formula:

[0084] a·x+b·y+c=0 (7)

[0085] Equation (2) can be over-constrained by six or more fixture elements (e.g., length-adjustable struts 116). It should be understood that it is not necessary for all fixture elements to be visible in a single image of images 126 and 128 in order to obtain matrix P. It should also be understood that if one or more of the aforementioned imaging scene parameters are known, these known parameters can be used to reduce the minimum number of fixture elements required to constrain equation (2). For example, such information can be obtained from the DICOM image header of a modern imaging system. Preferably, singular value decomposition or least squares methods can be used to solve equation (2) to obtain the value of vector p.

[0086] In some examples, the transformation matrix can then be decomposed into imaging scene parameters. The following formula can be used to decompose the correlation matrix P into matrices E and I:

[0087] P = I·E (8)

[0088] It should be understood that additional terms can be introduced when decomposing matrix P. For example, the method proposed by Tsai (see “AVersatile Camera Calibration Technique for High-Accuracy 3D Machine VisionMetrology Using of-the-shelfTV Cameras and Lenses”, IEEE Journal of Robotics & Automation, RA-3, No.4, 323-344, August 1987, the full text of which is incorporated herein by reference) can be used to correct radial distortion in images 126 and 128.

[0089] The basis matrices E and I contain imaging scene parameters. The following formula represents the composition of matrix I:

[0090]

[0091] The symbols sx and sy represent the values ​​of the image coordinate scale factor (e.g., pixel scale factor). The symbol f represents the focal length, corresponding to the value of the shortest distance between the corresponding imaging source 130 and the corresponding image planes 126 and 128. The symbols tx and ty represent the coordinates of the principal point relative to the local origin 125 of the corresponding image planes 126 and 128. The following formula represents the composition of matrix E:

[0092]

[0093] symbol o x o y and o z The values ​​represent the position of the fixation device 100 in actual three-dimensional space. Symbols r1 to r9 describe the orientation of the fixation device 100. These values ​​can form the three-dimensional rotation matrix R represented by the following formula:

[0094]

[0095] The methods of Trucco and Verri (see “Introductory Techniques of 3-D Computer Vision”, Prentice Hall, 1998, the full text of which is incorporated herein by reference) or Hartley’s methods (see “Euclidian Reconstruction from Uncalibrated Views”, Applications of Invariance in Computer Vision, pp. 237-256, Springer Verlag, Berlin Heidelberg, 1994, the full text of which is incorporated herein by reference) can be used to obtain the values ​​of matrices E and / or I. By utilizing the values ​​obtained from matrices E and I, the complete three-dimensional imaging scene of the fixator 100 and anatomical segments 102 and 104 can be reconstructed.

[0096] For example, Figure 2 An exemplary three-dimensional imaging scene reconstructed from x-ray images 126 and 128 is shown. In the illustrated embodiment, x-rays are emitted from an x-ray imager 130. It should be understood that the x-ray imager 130 can be the same or different imagers, as described above. The x-rays emitted from the imager 130 are received by a corresponding imaging device, thereby capturing images 126 and 128. Preferably, the positioning of the imager 130 relative to the local origin 125 is known.

[0097] In some examples, images 126, 128 and imaging scene parameters can then be used to obtain the location and / or orientation of anatomical segments 102, 104 in three-dimensional space. The obtained location and / or orientation data can be used to develop treatment plans for patients, for example, to alter the orientation and / or location of the first and second anatomical segments 102, 104 of the fracture to promote healing between anatomical segments 102, 104, as described in more detail below. It should be understood that the applications of the methods and techniques described herein are not limited to repositioning fractured anatomical structures; surgical orthopedic fixation procedures employing imaging analysis can be used as needed in any other type of fixation procedure, such as elongating anatomical structures, correcting anatomical defects, etc.

[0098] In some examples, anatomical structural elements, including specific portions (e.g., anatomical features) of anatomical structural segments 102, 104, can then be identified, and their positions within images 126, 128 can be determined. For example, the positions of the first and second anatomical structural segments within images 126, 128 can be indicated by the second image information received during operation 320 and detailed above. In some examples, the positions of the anatomical structural elements can be determined relative to the respective local origin 125 of images 126, 128.

[0099] Anatomical structural elements can be used to construct a three-dimensional illustration of the location and / or orientation of anatomical structural segments 102, 104. Preferably, the anatomical structural elements are readily identifiable in images 126, 128. Points, lines, cones, etc., or any combination thereof, can be used to describe the corresponding geometry of the anatomical structural segments. For example, in the illustrated embodiment, points 134 and 136, representing the fracture ends 103 and 105 of anatomical structural segments 102, 104, respectively, are identified as anatomical structural elements in images 126, 128.

[0100] The anatomical structural elements may also include marker elements, which are implanted into the anatomical structural segments 102 and 104 prior to imaging. The marker elements can serve as a supplement to or replacement for the anatomical structural elements identified in images 126 and 128. The marker elements can be configured to provide higher visibility in images 126 and 128 compared to the visibility of anatomical features in the anatomical structural segments 102 and 104. For example, the marker elements can be constructed using a radiopaque material or using easily distinguishable geometry.

[0101] A three-dimensional representation 200 of anatomical structural segments 102 and 104 can be reconstructed. This three-dimensional representation can be constructed with or without a corresponding representation of the fixator 100. In the illustrated embodiment, pairs of rays, such as rays 138, 140 and 142, 144, can be constructed for anatomical structural element points 134 and 136, respectively. Each ray connects an anatomical structural element in one of images 126 and 128 to the corresponding imager 130. Each pair of rays, such as points 146 and 148, can be analyzed for common intersections. Common intersections 146 and 148 represent the corresponding positions of anatomical structural element points 134 and 136 in the three-dimensional representation of anatomical structural segments 102 and 104. Of course, for example, if more than two images are taken, more than one pair (e.g., multiple pairs) of rays can be constructed. If rays of a particular set do not intersect, the point closest to all rays in the set can be used as the common intersection.

[0102] The position and / or orientation of anatomical segments 102, 104 can be quantified or measured through common intersections (e.g., points 146, 148). For example, a straight line representing the centerline of anatomical segments 102, 104 can be constructed and compared with the patient's anatomical axis. Additionally, the spacing of the fracture ends 103, 105 of anatomical segments 102, 104 can be quantified. The position and / or orientation of anatomical segments 102, 104 can be determined using these or similar techniques. It should also be noted that in some examples, in addition to the position and orientation of the first and second anatomical segments, the position and orientation of multiple rings (and / or other elements of the fixation device) in three-dimensional space can also be determined, for example, using any of the aforementioned techniques. For example, in some cases, the position of the rings within images 126, 128 can be determined based on first image information and / or other information provided. In some examples, these positions can then be used to determine the position and orientation of the rings in three-dimensional space. In addition, in some examples, the configuration information of the fixed device (such as ring diameter, strut length, and installation information) can also be used to determine the position and orientation of the ring in three-dimensional space.

[0103] See now Figure 3B In operation 324, one or more deformity parameters are calculated. Deformity parameters may include parameters associated with deformities related to the first and second anatomical segments. For example, in some cases, deformity parameters may include the amount of translation (e.g., lateral, medial, anterior, and / or posterior displacement), the degree of coronal angulation (e.g., eversion and / or inversion), the degree of sagittal angulation, the amount of anatomical length that is too short and / or too long, the degree of clinical rotational deformity (e.g., internal and / or external rotation), etc. In some examples, such as by means of the technique described above in conjunction with operation 422, the deformity parameters may be calculated as part of a process that determines the position and orientation of the first and second anatomical segments, as described above in operation 422.

[0104] In operation 326, the malformation parameters calculated in operation 424 are displayed, for example, through one or more graphical user interfaces of the computing system. See now. Figure 9 The diagram illustrates the deformity parameter interface 900. As shown, interface 900 includes multiple fields 901-906 for displaying calculated values ​​of several exemplary deformity parameters, including translation and coronal angulation in the "AP view," translation and sagittal angulation in the "LAT view," the amount of anatomical structure length that is too short or too long, and the degree of clinical rotational deformity. Figure 9In the example, fields 901-905 each have a corresponding PFM badge 915 (including the text "PFM"), which is displayed to the left of each field 901-905. Each PFM badge 915 indicates that the value shown in the corresponding field 901-905 has been calculated by the software. Interface 900 allows the user to edit distorted parameter values ​​displayed in each field 901-906, for example, by entering numbers in fields 901-906 and / or by using the numeric increment control 916 displayed to the right of each field 901-906. When the user edits a value calculated by the software, the PFM badge 915 adjacent to the corresponding field can be removed to indicate that the value of that field has been edited by the user. In some examples, after editing the value in one or more fields, the user can select the "Refresh Physiological Stent Matching Data" button 920 to restore each field to the software-calculated value. Additionally, in some examples, after editing the values ​​in one or more fields, the user can select the "Save and Update" button 921 to recalculate the malformed parameters based on the edited values ​​provided by the user, for example, by repeating all or part of the calculation performed in operation 322.

[0105] In operation 328, a graphical representation of the position and orientation of the first and second anatomical segments is generated and displayed. This graphical representation of the position and orientation of the first and second anatomical segments can be displayed through one or more graphical user interfaces of the computing system. For example, as... Figure 9 As shown, interface 900 includes a graphical representation 950 of the position and orientation of a first anatomical segment and a second anatomical segment. Graphical representation 950 includes an illustration 931 of the proximal anatomical segment and an illustration 932 of the distal anatomical segment. In some examples, graphical representation 950 may be generated at least in part based on the position and orientation of the first and second anatomical segments determined in operation 322. In some examples, graphical representation 950 may also be adjusted to reflect the saved edits to the anatomical parameters when the user edits one or more deformity parameters and selects the "Save and Update" button 921. For example, graphical representation 950 may improve efficiency and reliability by providing the user with visual confirmation of the information entered into interface 900, for example, to enable quick and easy identification of errors or other problems.

[0106] In operation 330, one or more installation parameters are calculated. Installation parameters may include parameters related to mounting the fixator's reference ring onto the corresponding anatomical segment. For example, in some cases, installation parameters may include offsets (e.g., lateral, medial, anterior, and / or posterior displacements) (such as the offset of the reference ring center relative to a reference point), tilt (e.g., proximal and / or distal), axial offset, master bayonet rotation, etc. In some examples, such as by means of the technique described above in conjunction with operation 322, the installation parameters may be calculated as part of a process that determines the position and orientation of the first and second anatomical segments, as described above in operation 322. It should be noted that for the process of Figure 3, the reference ring need not be orthogonal to the corresponding anatomical segment on which it is mounted. Therefore, in some examples, the reference ring may not be orthogonal to the corresponding anatomical segment on which it is mounted.

[0107] In operation 432, the installation parameters calculated in operation 430 are displayed, for example, through one or more graphical user interfaces of the calculation system. See now. Figure 10 The installation parameter interface 1000 is shown. As illustrated, interface 1000 includes multiple fields 1001-1006 for displaying calculated values ​​of several exemplary installation parameters, including "AP view" offset and tilt, "LAT view" offset and tilt, axial offset, and main bayonet rotation. Figure 10 In the example, fields 1001-1006 each have a corresponding PFM label 1015, which is displayed to the left of each field 1001-1006. Each PFM label 1015 indicates that the value shown in the corresponding field 1001-1006 has been calculated by the software. Interface 1000 allows the user to edit the installation parameter values ​​displayed in each field 1001-1006, for example, by entering numbers in fields 1001-1006 and / or by using the numeric increment control 1016 displayed to the right of each field 1001-1006. When the user edits a value calculated by the software, the PFM label 1015 adjacent to the corresponding field can be removed to indicate that the value of that field has been edited by the user. In some examples, after editing the value in one or more fields, the user can select the "Refresh Transillumination Brake Matching Data" button 1020 to restore each field to the software-calculated value. Additionally, in some examples, after editing the values ​​in one or more fields, the user can select the "Save and Update" button 1021 to recalculate the malformed parameters based on the edited values ​​provided by the user, for example, by repeating all or part of the calculation performed in operation 322.

[0108] In operation 334, a graphical representation of the position and orientation of the reference ring and the corresponding anatomical segment on which it is mounted is generated and displayed. This graphical representation of the position and orientation of the reference ring and the corresponding anatomical segment can be displayed through one or more graphical user interfaces of the computing system. For example, as... Figure 10 As shown, interface 1000 includes a graphical representation 1050 of the position and orientation of a reference ring and corresponding anatomical segments. Graphical representation 1050 includes: an illustration 1031 of the proximal anatomical segment, an illustration 1033 of the proximal (reference) ring, and an illustration 1032 of the distal anatomical segment. In some examples, graphical representation 1050 may be generated at least in part based on the position and orientation of the reference ring and corresponding anatomical segments determined in operation 322. Therefore, the graphical representation of the reference ring and corresponding anatomical segments may reflect and / or indicate the position and orientation of the reference ring and corresponding anatomical segments determined in operation 322. In some examples, graphical representation 1050 may also be adjusted to reflect the saved edits to the installation parameters when the user edits one or more installation parameters and selects the "Save and Update" button 1021. For example, graphical representation 1050 may improve efficiency and reliability by providing the user with visual confirmation information of the information entered into interface 1000, for example, to enable quick and easy identification of errors or other problems.

[0109] In operation 336, one or more treatment plan options are received, for example, through one or more graphical user interfaces of a computing system. The treatment plan is a scheme for manipulating the fixation device, for example, to correct deformities in a first anatomical segment and a second anatomical segment. The treatment plan may include, for example, the following: progressively adjusting the position and orientation of the fixation rings relative to each other by, for example, changing the length of the support bars of the fixation device. See now. Figure 11An exemplary treatment protocol interface 1100A is shown. Interface 1100A includes multiple controls for users to select various treatment protocol options. Specifically, controls 1101 and / or 1102 allow selection of the treatment protocol start date, control 1103 allows selection of the option to perform axial movement first (e.g., in the initial portion of treatment, such as before rotational movements), control 1104 allows selection of an option indicating the final distance between reference points, control 1105 allows selection of an option to calculate the treatment protocol based on a specified duration of axial movement (e.g., multiple days), control 1106 allows selection of an option to calculate the treatment protocol based on the traction rate at the reference point of axial movement (e.g., millimeters (mm) / day), control 1108 allows selection of an option to calculate the treatment protocol based on a specified duration of deformity correction (e.g., multiple days), control 1109 allows selection of an option to calculate the treatment protocol for deformity correction based on the traction rate at the reference point (e.g., millimeters (mm) / day), and control 1107 allows selection of an option to perform adjustments twice daily. In some examples, when control 1007 is not selected, the default option of adjusting once a day can be used. In some examples, after selecting the desired treatment plan option, the user can select the "Update Adjustment Plan" button 1110 to trigger the generation of the treatment plan. Alternatively, after the initial generation of the treatment plan, the user can also be allowed to adjust the treatment plan options by reselecting the "Update Adjustment Plan" button 1110 and regenerate the treatment plan using the adjusted options.

[0110] In some examples, the software can allow treatment plans to be divided into multiple treatment phases. This allows for greater control over deformity correction, such as by allowing surgeons to define the starting and target poses for each treatment phase, controlling the options for each phase, and controlling the type of movement within each phase. For example, in some cases, users may be allowed to create multiple treatment phases. Each of the multiple treatment phases can be defined by an assigned starting pose and an assigned target (i.e., ending) pose. The starting pose of the first (initial) treatment phase can be the initial anatomical deformity position at the start of treatment on the first day. The target pose of the final treatment phase can be the desired position of the anatomical segment at the conclusion of treatment. The starting pose of each subsequent treatment phase (after the initial treatment phase) can be the same as the target pose of each previous treatment phase. For example, the starting pose of the second treatment phase can be the same as the target pose of the first treatment phase, and so on. Multiple phases (1 to N) can be combined in a list, and a plan with N separate phases can be combined into a treatment plan.

[0111] The software provides an interface that allows users to select the desired number of treatment phases, the initial and target poses for each phase, and individual options for each phase. For example, for each treatment phase, the software may allow users to select the corresponding duration (e.g., number of days), traction rate, amount of adjustment per day, and number of adjustments per day (e.g., strut movement). The options for each treatment phase can differ from each other. For example, the duration, traction rate, amount of adjustment per day, and / or number of adjustments per day for the first treatment phase may differ from those for the second treatment phase. In some examples, the input parameters for each treatment phase may include the initial pose of the distal segment, the target pose of the distal segment, and the scheme options for that phase (duration / traction rate at a given point / degree per day, etc.). In some examples, multiple treatment phases may allow overcorrection of the deformity. For example, multiple treatment phases may allow compression, which can be calculated by using overcorrection with a negative value for axial traction. Multiple treatment phases may also provide a simple and intuitive mechanism to allow axial movement to be performed in the initial phase of treatment and to allow for additional axial elongation / traction. In some examples, users can describe treatment plans using clinical terminology (e.g., residual deformities due to overcorrection, number of stages, initial traction, etc.). The software layer can then interpret the clinical language based on the stage definitions and create the corresponding treatment stages.

[0112] In operation 338, manipulation of the fixation device for correcting anatomical deformities (i.e., treatment plan) is determined. Manipulation of the fixation device may include adjustment of the fixation device struts, such as adjusting the size and / or length of the struts. In some examples, operation 338 may be performed at least in part based on the treatment plan options received in operation 336. For example, operation 338 may be based at least in part on a specified start date, instructions to perform axial movement first (e.g., in the initial portion of treatment, such as before rotational movements), a specified final distance between reference points, instructions to perform a specified amount of additional extension, instructions to generate axial clearance to ensure anatomical clearance, a specified treatment duration (e.g., multiple days), a specified stretching rate, and / or instructions to perform a specified amount of adjustment daily (e.g., once, twice, etc.).

[0113] In some examples, the treatment plan may also be determined at least in part based on: determining whether the position and / or orientation of anatomical segments 102, 104 need to be altered, for example, how to reposition anatomical segments 102, 104 relative to each other to promote healing between anatomical segments 102, 104. For example, in some cases, it may be advantageous to: change the angularity of the second anatomical segment 104 so that axes L1 and L2 are aligned; change the position of the second anatomical segment so that the fracture ends 103, 105 of anatomical segments 102, 104 are adjacent to each other. Once the need to alter the position and / or orientation of anatomical segments 102, 104 is determined, a treatment plan for achieving that position and / or orientation can be determined. In a preferred embodiment, the desired alteration to the position and / or orientation of anatomical segments 102, 104 can be achieved gradually through a series of small changes. The position and / or orientation of the anatomical segments 102 and 104 can be changed by altering the position and / or orientation of the upper and lower fixing rings 106 and 108 relative to each other, for example, by lengthening or shortening one or more length-adjustable support bars 116.

[0114] Using the matrix algebra method described above, the necessary changes to the geometry of the fixator 100 (i.e., the position and / or orientation of the fixator 100) can be calculated, which can achieve the necessary changes to the position and / or orientation of the anatomical structural segments 102 and 104. For example, if it is necessary to reposition and / or reorient the second anatomical structural segment 104 relative to the first anatomical structural segment 102, this can be converted into changing the position and / or orientation of the lower fixation ring 108 relative to the upper fixation ring 106.

[0115] At operation 340, instructions for the determined manipulation of the fixed device are provided to one or more users. For example, in some cases, instructions for the determined manipulation of the fixed device may be provided using one or more graphical user interfaces of a computing system, using printed hard copies, using audio feedback, and / or using other technologies. Specifically, see now. Figure 12As can be seen, instructions for the defined manipulations of the fixation device can be provided within interface 1100B. Specifically, selecting the support bar adjustment scheme tab 1122 causes the treatment scheme interface 1100B to provide chart 1130, which includes daily manipulation information for each support bar within the fixation device. In this example, chart 1130 shows the length of each support bar on each treatment day. In some examples, one or more alerts can be generated for one or more fixation device manipulations that result in at least one support bar moving beyond a threshold amount. For example, in some cases, support bar movement exceeding a specific threshold amount (e.g., 3 mm per day) (this may be referred to as rapid support bar movement) can be indicated by displaying a red triangle icon next to the support bar movement indication in chart 1130. Also, Figure 12 As shown, a PDF version of Figure 1130 can be generated by selecting the "View Draft PDF" button 1131. In some examples, the generated PDF can be printed out to create a hard copy of Figure 1130.

[0116] exist Figure 12 In the example, diagram 1130 includes boxes 1132-A and 1132-B, which indicate the date range for which the support bar size can be changed (referred to as support bar replacement). Specifically, box 1132-A indicates that support bar replacement can be performed on day 0 through day 2, while box 1132-B indicates that support bar replacement can be performed on day 3 through day 14 (and subsequent days). In some examples, boxes 1132-A and 1132-B can be color-coded to match the color assigned to the respective support bar. For example, boxes 1132-A and 1132-B can be painted green to match the green that can be assigned to support bar 4. See now. Figure 13 You can select the “Stretcher Replacement Calendar” tab 1123 in the treatment plan interface 1100-C to generate a calendar 1140 that indicates the date range in which stretcher replacements can be performed.

[0117] In some examples, the fixation device support strips attached to the patient may be color-coded, for example, by color-coded caps, markings, or other color-coded materials included or attached to the support strips. In some examples, the actual color code of the fixation device support strips attached to the patient may match the color code of the support strips used by the software. For example, the actual color code of the support strips in the fixation device may match the support strip color code that can be used to color-code the following: boxes 1132-A and 1132-B in Figure 1130, graphic representation 520, and other color-coded illustrations of the support strips displayed by the software. In some examples, this makes it easier for physicians and / or patients to confirm that when they actually adjust the fixation device support strips, they are adjusting the correct support strips and by the correct amount.

[0118] In operation 342, one or more graphical representations of the position and orientation of the first anatomical segment, the second anatomical segment, and the ring of the fixation device are generated and displayed. These graphical representations can be displayed through one or more graphical user interfaces of the computing system. For example, see again... Figure 11 Selecting the "Treatment Simulation" tab 1121 allows the interface 1100 to display a graphical representation 1150 of the position and orientation of the first anatomical segment, the second anatomical segment, and the ring of the fixation device. The graphical representation 1150 includes: an illustration 1031 of the proximal anatomical segment, an illustration 1033 of the proximal (reference) ring, an illustration 1032 of the distal anatomical segment, and an illustration 1034 of the distal ring. In some examples, one or more graphical representations of the position and orientation of the first anatomical segment, the second anatomical segment, and the ring of the fixation device may include: a daily graphical representation of the position and orientation of the first anatomical segment, the second anatomical segment, and the ring of the fixation device throughout the treatment of the anatomical deformity. For example, as... Figure 11 As shown, through controls 1151, 1152, 1153, and / or 1154, the user can select a specific treatment date to be generated and displayed as a graphical representation 1150. For example, control 1151 can be selected to allow incrementing the selected date, control 1154 can be selected to allow decrementing the selected date, and slider 1152 can be slid along bar 1153 to increment and / or decrement the selected date. It should also be noted that slider 1152 displays an indication of the currently selected day. Figure 11 In the example, the currently selected day is treatment day zero. Therefore, in Figure 11In the diagram, graphical representation 1150 shows the position and orientation of the first and second anatomical segments and the ring of the fixation device on the zeroth treatment day. Selecting different treatment days via controls 1151-1154 triggers adjustments to the graphical representation 1150 to display the position and orientation of the first and second anatomical segments and the ring of the fixation device on the selected dates. It should be understood that allowing surgeons and / or patients to see the position and orientation of the first and second anatomical segments and the ring of the fixation device throughout treatment can be beneficial, for example, by providing additional visual tools to improve the accuracy of treatment and aid in treatment planning. Furthermore, graphical representation 1150 (and the graphical representations described herein) can improve efficiency and reliability, for example, by providing users with visual confirmation information of the information entered into interface 1100, for example, to enable quick and easy identification of errors or other problems. It should also be noted that, for example, the view of the graphic representation 1150 (and other graphic representations described herein) can be rotated (e.g., rotated a full 360 degrees), zoomed in and out, moved directionally, and otherwise manipulated using controls 1181-1184 adjacent to the upper right side of the graphic representation 1150. This allows the first and second anatomical segments and / or the ring of the fixation device to be observed from various orientations that may be unavailable or difficult to obtain using X-rays and other imaging techniques, thereby also improving reliability and accuracy and providing additional visual confirmation of the calculated values. Specifically, the view of the graphic representation 1150 can be rotated using control 1181, zoomed in using control 1182, zoomed out using control 1183, and panned using control 1184. Additionally, in some examples, other controls (such as a mouse and touchscreen) can be used to rotate, zoom, pan, and otherwise manipulate the graphic representation 1150. In addition, in some examples, control 1185 can be used to select the front-to-back (AP) view, control 1186 can be used to select the side view, and control 1187 can be used to select the near-side view.

[0119] In operation 344, a treatment plan can be implemented, i.e., based on the manipulation determined in operation 338, the geometry of the fixation device can be changed in order to change the position and orientation of the anatomical segment.

[0120] Hinge detection for orthopedic fixation

[0121] As described above, a scaffold matching process can be used to determine the location and orientation of anatomical segments in three-dimensional space, such as for generating treatment plans to correct anatomical deformities. Also as described above, in some examples, as part of the scaffold matching process, the surgeon or other user can identify the location of fixation elements (e.g., hinges, struts, etc.) within a display image (e.g., an X-ray) showing the fixation attached to the anatomical segment. (See above reference) Figure 3A and Figure 6 Operation 318 describes some examples of this process. For example, such as... Figure 6 As shown and as described above, as part of the bracket matching process, the user can identify the position of the hinge within the "AP view" image 601-A and the "LAT view" image 601-B. However, users may often find it difficult to identify and mark the position of certain fastener elements (such as hinges) within the image. Specifically, depending on the position and orientation of the captured image, hinges and other fastener elements may not be easily identified, as they may completely or partially overlap each other, or may otherwise be occluded within the image.

[0122] In some examples, automated or semi-automated hinge detection techniques can be employed to mitigate the above and other issues. References will now be made. Figures 14A-25 To illustrate some examples of these hinge detection techniques, see now for reference. Figure 14A An exemplary process for hinge testing for orthopedic fixation will now be described in detail. Figure 14A -B Any or all operations and sub-operations described herein can be performed by one or more computing devices, such as by computer software executed on one or more computing devices. As described above, the fixator may include fixator elements such as rings, struts, and multiple hinges, and can be used to correct deformities of the first and second anatomical segments to which the fixator is attached. The process in Figure 14 begins at operation 1410, where a first image and a second image of the first and second anatomical segments and the fixator attached to the first and second anatomical segments are shown. The first and second images may have corresponding image planes. Figure 2 As shown and as described above, there is an angle α between the image planes of images 126 and 128.

[0123] exist Figure 6 The diagram shows a first example of displaying a first image and a second image in operation 1410, which includes the “AP view” image 601-A and the “LAT view” image 601-B as described above. Figure 15A An additional example of displaying the first and second images at operation 1410 is shown, and it will now be described in detail. Specifically, Figure 15AImages 1501-A (AP view) and 1501-B (LAT view) show images of a fixator 1510 including a proximal fixation ring 1511, a distal fixation ring 1512, a fixator strut 1513, and twelve hinges 1541 (shown as black circles at the ends of strut 1513). The hinges 1541 comprise six proximal hinges (adjacent to the proximal fixation ring 1511) and six distal hinges (adjacent to the distal fixation ring 1512). Images 1501-A and 1501-B show the fixator 1510 attached to the first anatomical segment 1521 and the second anatomical segment 1522. In operation 1410, one or more graphical user interfaces of a computing system can be used to display first and second images of the first and second anatomical segments and the fixator attached to them. For example, one or more graphical user interfaces of a computing system can be used to display... Figures 15A-25 Images 1501-A and 1501-B are shown. It should be noted that in some examples, one or more graphical user interfaces of the computing system may be used to display these images. Figures 15A-25 Any or all of the content shown in each of the diagrams.

[0124] It should be noted that, Figures 15A-25 In the examples, images 1501-A and 1501-B are simulated images—compared to actual X-rays (such as...). Figure 6 (As shown) or other images captured from the imager or imaging source. It should be noted that the provided... Figures 15A-25 The simulated images are provided merely for illustrative purposes of the concepts described herein. In practice, images 1501-A and 1501-B may be non-simulated images, such as X-rays, taken using an imager, imaging source, X-ray imager, camera, or other imaging device, and show actual fixators (such as...) physically attached to actual anatomical structures. Figure 6 (As shown). Therefore, even though images 1501-A and 1501-B are displayed as analog, the concepts described herein should be understood to apply to similar... Figure 6 The images 601-A and 601-B are non-analog images (i.e., images taken using an imager, imaging source, X-ray imager, camera or other imaging device).

[0125] At operation 1412, for example, by using one or more graphical user interfaces of a computing system, an indication of the position of a first image hinge associated with multiple hinges in the first image is received. For example, as described above relative to... Figure 6As described above, the user can indicate the position of the hinge within the "AP view" image 601-A by clicking the hinge using an attached mouse or other input device. As mentioned above, the support bar indicator button 611-A for support bar 1 can be automatically pre-selected for the user. While selecting (or automatically pre-selecting) the support bar indicator button 611-A for support bar 1, the user can continue to draw (or otherwise indicate) a representation of the hinge at the endpoints of support bar 1 within the "AP view" image 601-A. For example, in some cases, the user can use a mouse or other input device to select the position 621 (e.g., the center point) of the proximal hinge of support bar 1 within image 601-A. In some examples, the user can then use a mouse or other input device to select the position 622 (e.g., the center point) of the distal hinge of support bar 1 within image 601-A. Figure 6 As shown, the software can generate points or circles at positions 621 and 622 of the proximal and distal hinges selected by the user within image 601-A. Additionally, the software can generate a line 623 representing the position and / or length of strut 1, connecting the points or circles at positions 621 and 622, as well as the proximal and distal hinges selected by the user in image 601-A. In some examples, the above process can be repeated to draw points representing the proximal and distal hinges at the endpoints of each of the six struts in the "AP view" image 601-A. Similar techniques can also be used to indicate... Figure 15A The position of each of the twelve hinges in the "AP view" image 1501-A.

[0126] In some examples, after the user indicates the position of hinge 1541 within the "AP view" image 1501-A, the software can use the indicated hinge position to determine the positions of retaining rings 1511 and 1512 within the "AP view" image 1501-A. The software can then generate ring graphic representations 1531 and 1532 corresponding to retaining rings 1511 and 1512, respectively, and display these ring graphic representations 1531 and 1532 at the determined positions of retaining rings 1511 and 1512 within the "AP view" image 1501-A. Now refer to... Figure 15B As can be seen, the ring graphics representations 1531 and 1532 are generated by software and displayed at the corresponding positions of the fixed rings 1511 and 1512 within the "AP View" image 1501-A. It should be noted that the fixed ring graphics representations 1531 and 1532 are... Figure 15B The images shown are in a different hue / color from the actual retaining rings 1511 and 1512 to indicate that the retaining ring graphic representations 1531 and 1532 were generated by software and are not included in the actual underlying "AP view" images 1501-A. Specifically, retaining ring graphic representations 1531 and 1532 are shown in blue / shading, while retaining rings 1511 and 1512 are shown in black / shading.

[0127] At operation 1414, for example using one or more graphical user interfaces of a computing system, the graphic projection of the fixator is superimposed on the second image. For example, see now. Figure 16 As can be seen, the graphic projection 1600 of the display fixture includes a graphic representation 1611 of the proximal ring and a graphic representation 1612 of the distal ring. As shown, the graphic projection 1600 including graphic representations 1611 and 1612 is overlaid on a second image, which in this example is the "LAT view" image 1501-B.

[0128] The graphic projection 1600 of the fixator can be rotated relative to the fixator element in the first image, at least in part, based on the angle (e.g., by an exact angle or an approximation of an angle) between the image planes of the first and second images and each other. Figure 2 As shown and as described above, there exists an angle α between the image planes of images 126 and 128. Therefore, in Figure 16 In the example, the "AP view" image 1501-A may have a corresponding "AP view" image plane, and the "LAT view" image 1501-B may have a corresponding "LAT view" image plane at a 90-degree angle relative to the "AP view" image plane. Therefore, in Figure 16 In the example, the graphic projection 1600 of the fastener is rotated ninety degrees relative to a first position of the plurality of fastener elements identified in the first image. For example, Figure 16 The proximal ring graphic representation 1611 and the distal ring graphic representation 1612 are both rotated ninety degrees relative to the proximal fixation ring 1511 (and / or the corresponding ring representation 1531) and the distal fixation ring 1512 (and / or the corresponding ring representation 1532) in the “AP view” image 1501-A.

[0129] The graphic projection 1600 of the fastener can be rotated at least in part based on the angle between the image planes of the images, because this rotation can correspond to the expected position of the fastener in the second image. For example, if the image plane of the "LAT view" image 1501-B is at a 90-degree angle to the image plane of the "AP view" image 1501-A, it can be expected that the position of the fastener ring in the "LAT view" image 1501-B will be rotated 90 degrees relative to the position of the fastener ring in the "AP view" image 1501-A. Thus, the superposition of the graphic projection 1600 on the second image helps the user identify the positions of multiple fastener elements in the second image. In some examples, the user can provide a numerical value, such as degrees (e.g., 90 degrees), which explicitly indicates to the software the angle value between the image planes of the images. In other examples, the angle value can be inferred by the software based on the description of the image (e.g., front-back, front, back, outside, inside, etc.) or using other techniques. Figures 15A-17In the example, image 1501-A is an AP view image, and image 1501-B is a side view image. However, it should be noted that the techniques described herein can be used between any different combinations of images captured from any direction and orientation and having image planes at any angle relative to each other.

[0130] Additionally, it should be noted that the software can also manipulate other features of the graphic projection 1600 (e.g., size, position, orientation, etc.) to correct other differences between the first and second images (e.g., position, orientation, scaling level, etc.). For example, in some cases, if the second image is taken from a position closer to the fixture and / or is magnified to a greater extent than the first image, the software can correct this by enlarging the size of the graphic projection 1600 relative to the size of the fixture element in the first image. Conversely, in some cases, if the second image is taken from a different position on the fixture and / or is magnified to a greater extent than the first image, the software can correct this by decreasing the size of the graphic projection 1600 relative to the size of the fixture element in the first image.

[0131] Therefore, in some examples, the graphic projection 1600 of the fastener may be generated at least in part based on the position of the fastener element in the first image. Alternatively, in some examples, the graphic projection 1600 of the fastener may be generated at least in part based on configuration information of the fastener provided to the software by the user, such as ring type (e.g., full ring, footplate, etc.), ring size, strut length, indication of mounting points (e.g., ring holes), and other information. The above, such as relative to... Figure 5 and Figure 3A Operation 314 describes in detail the various types of configuration information and techniques used to provide such information to the software, which will not be repeated here.

[0132] At operation 1416, the software may allow the user to manipulate (e.g., resize, rotate, move, etc.) the graphic projection and / or the second image. For example, the user may manipulate the graphic projection to more precisely align it with the position of the fixture element in the second image. For example, when superimposed on the second image at operation 1414, the software may provide controls that allow the graphic projection to be resized (made larger or smaller) or rotated relative to the initial placement made by the software. For example, in some cases, it may be necessary to adjust the size and / or rotate the graphic projection to correct subtle differences in the actual angle between the first and second images relative to the expected angle (e.g., if the images are actually ninety-two degrees instead of ninety degrees, etc.), to correct differences in distance, position, or orientation of the first and second images relative to objects included in the images, or for other reasons. In some examples, the software may provide various controls such as buttons that allow selection of operations such as moving, resizing, and rotating, and the software may be configured to receive input from input devices such as a mouse or keyboard to perform those manipulations, for example, via drag and drop, button clicks, keystrokes, etc.

[0133] In some examples, in addition to allowing the user to manipulate the graphic projection or, alternatively, the software may allow the user to manipulate a second image overlaid with the graphic projection (e.g., "LAT View" image 1501-B). For example, in some cases, the software may allow the user to adjust the size, rotate, and / or move the second image and / or the elements shown in the second image, such as to help align the fixture elements shown in the second image with the corresponding elements in the graphic projection. See now. Figure 17 As can be seen, the user has accessed the "LAT View" image 1501-B from... Figure 16 The previous screen / interface position is moved down and to the right to manipulate the second image as "LAT view" image 1501-B. By moving the "LAT view" image 1501-B in this way (without moving the graphic projection 1600), this allows the fixture elements in the "LAT view" image 1501-B to move down and to the right, so that they are aligned with the corresponding elements in the graphic projection 1600. For example, as Figure 17 As shown, the graphic representations of retaining rings 1611 and 1612 are substantially aligned with the corresponding retaining rings 1511 and 1512. Therefore, Figure 17 Only a small portion of the retaining rings 1511 and 1512 are visible, as they are almost entirely superimposed by the corresponding graphic representations of the retaining rings 1611 and 1612. Specifically, in Figure 17In the diagram, the proximal ring 1611 is substantially aligned (and almost completely overlaps) with the proximal fixation ring 1511, and the distal ring 1612 is substantially aligned (and almost completely overlaps) with the distal fixation ring 1512. It should be noted that although... Figure 16 and Figure 17 The use of graphic overlay shown can sometimes be helpful in correlating the first image with the second image, but it is not necessary for the hinge detection technique described herein to include the use of graphic overlay.

[0134] At operation 1418, the projected second image hinge position associated with multiple hinges in the second image is determined. In some examples, the software may determine the projected second image hinge position based at least in part on the indication of the first image hinge position received at operation 1412. The projected second hinge position is an estimated position by the software, where the software anticipates that the hinges will be located within the second image based on the user's indication of the hinge positions in the first image. For example, because the software knows the spatial relationship (e.g., angle) between the first and second images, the software can use the hinge positions in the first image to project / estimate the expected position of the hinges in the second image. In some examples, the projected second image hinge position may be expressed by the software via X and Y coordinate values ​​within the second image. Figure 18 A diagram showing a second image (e.g., “LAT View” image 1501-B) illustrates the fixator strut 1513 and twelve hinges 1541, all of which are included in the second image (e.g., “LAT View” image 1501-B). It should be understood that although the second image (e.g., “LAT View” image 1501-B) will also include the fixation ring and anatomical segments to which the fixator is attached (e.g., ... Figures 15A-17 (as shown), but the fixation ring and anatomical structure section are not in Figure 18 and Figures 22-24 This is shown in the diagram to reduce clutter in the image. For example... Figure 18 As shown, the software can determine the second image hinge position 1841 corresponding to the projection of hinge 1541 in the "LAT view" image 1501-B. It should be understood that the actual position of hinge 1541, which is part of the "LAT view" image 1501-B, is... Figure 18 The image shown consists of twelve circles with normal (thin) contours. In contrast, the hinge position of the second projected image is 1841. Figure 18 The image shows twelve circles with a heavier (thicker) profile. It should be noted that the projected second image hinge position 1841 is very close to (i.e., adjacent but not identical to) the actual position of hinge 1541 in the LAT view image 1501-B. Specifically, in this example, the projected second image hinge position 1841 is located above and to the right of the corresponding actual position of the hinge within the LAT view image 1501-B.

[0135] In some examples, the software can determine the position of the projected second image hinge by rotating the first image hinge position in the first image, at least in part, based on the angle between the image planes of the first and second images relative to each other (e.g., by an exact angle or an approximation of an angle). For example... Figure 2 As shown and as described above, there is an angle α between the image planes of images 126 and 128. Therefore, as described above, the "AP view" image 1501-A may have a corresponding "AP view" image plane, and the "LAT view" image 1501-B may have a corresponding "LAT view" image plane at a 90-degree angle relative to the "AP view" image plane. Therefore, in Figure 18 In the example, the second image hinge position 1841 of the projection is rotated ninety degrees relative to the first image hinge position of the hinge 1541 identified in the first image.

[0136] The second image hinge position 1841 of the projected image can be rotated at least in part based on the angle between the image planes of the images, because this rotation corresponds to the expected position of the fixture in the second image. For example, if the image plane of the "LAT view" image 1501-B is at a 90-degree angle to the image plane of the "AP view" image 1501-A, it can be expected that the position of hinge 1541 in the "LAT view" image 1501-B will be rotated 90 degrees relative to the position of hinge 1541 in the "AP view" image 1501-A. In some examples, the user can provide a numerical value, such as an angle quantity (e.g., 90 degrees), which explicitly indicates to the software the angle value between the image planes of the images. In other examples, the value of the angle can be inferred by the software based on the description of the image (e.g., front-back, front, back, outside, inside, etc.) or using other techniques.

[0137] At operation 1420, hinge candidates are detected in the second image. Hinge candidates have shapes associated with multiple hinges. A hinge candidate is a region in the second image that has a shape associated with (e.g., similar to) a hinge. For example, a hinge candidate may be a region in the second image defined by the same or similar visual features (e.g., white, black, or gray, or the same or similar shade of another color) and has a shape that matches or corresponds to the shape of one of the hinges (e.g., a generally circular shape). Hinge candidates can be detected by computer software using image analysis techniques based on automated software performed on the second image. For example, a hinge may have a circular shape, and hinge detection performed by computer software may include applying a circular detection algorithm, such as the Hough transform, to the second image to identify circular shapes in the second image as hinge candidates. It should be noted that the identification of circular shapes for hinge detection purposes (as used herein) is intended to encompass the identification of precisely circular shapes and substantially circular shapes, such as possibly partially blurred circles or shapes with elliptical shapes. In many cases, the number of hinge candidates detected in the second image may be greater than the number of actual fastener hinges. This is likely because, in addition to detecting the actual hinge, the software may also detect multiple false hinge candidates, such as other circular shapes in the second image (e.g., wires, other objects, etc.). Furthermore, in some examples, even a single hinge may sometimes be detected as multiple hinge candidates—different circles in similar or adjacent positions but with different dimensional characteristics (e.g., radius).

[0138] In some examples, to improve hinge candidate detection results, the software can use prior knowledge to detect hinge candidates. In some cases, the software can determine a range of expected dimensional characteristics (e.g., radius length) for multiple hinges, and the software can restrict hinge candidates to circular shapes having defined dimensional characteristics within the expected dimensional characteristic range. For example, the software can determine a range of expected radius lengths for hinges, and the software can restrict hinge candidates to circular shapes with lengths within the expected radius length range. For example, the range of expected radius lengths can include a minimum expected radius length and a maximum expected radius length, as well as all lengths between the minimum and maximum expected radius lengths. In some examples, the minimum expected radius length can be based on the minimum hinge radius detected in the first image (e.g., “AP View” image 1501-A). For example, the minimum expected radius length can be equal to the minimum hinge radius detected in the first image (e.g., “AP View” image 1501-A) minus a selected offset value. Moreover, in some examples, the maximum expected radius length can be based on the maximum hinge radius detected in the first image (e.g., “AP View” image 1501-A). For example, the maximum expected radius length can be equal to the maximum hinge radius detected in the first image (e.g., “AP View” image 1501-A) plus a selected offset value. In some cases, the radius length or other dimensional characteristics of the hinge in the first image can be determined by the software by performing automated image analysis (e.g., using a Hough transform) on the first image to detect the dimensional characteristics of the circle at the first image location indicated by the user for the hinge in the first image. In some examples, since the second and first images can be captured from the same or similar distances from the fixture, the software can reasonably assume that the dimensional characteristics of the hinge (e.g., radius length) should be the same or similar in the first and second images. Therefore, the dimensional characteristics of the hinge in the first image can be used as prior knowledge to more accurately identify hinge candidates, such as by eliminating certain false alarms, for example, shapes or objects with dimensional characteristics (e.g., radius length) that are too large or too small and are not actually hinges. It should be noted that, in addition to or as an alternative to the radius length, other dimensional characteristics (e.g., circumference, diameter, etc.) can also be used in a manner corresponding to the aforementioned radius length characteristics to limit the range of hinges detected.

[0139] Additionally, in some examples, prior knowledge used to improve hinge detection results may include hinge orientation. For example, in some cases, the software may expect one or more loops of the fixator to appear at an angle within the image, such as substantially perpendicular to the bone segment, which could result in the loops appearing substantially horizontal in the second image. Furthermore, the software may also expect hinges corresponding to specific loops to be aligned with each other in a straight line. For example, the software may expect proximal hinges adjacent to proximal loops to be aligned with each other in a straight line. The software may also expect distal hinges adjacent to distal loops to be aligned with each other in a straight line. The software may also expect lines to have the same or similar angles as the corresponding loops. The software can use this prior knowledge to more accurately identify hinge candidates, such as by eliminating certain false positives. For example, in some cases, if the software identifies an irrelevant circular shape that is not aligned with any other detected circular shape, the software may consider this irrelevant circular shape a false positive and may not include it in the detected set of hinge candidates. Therefore, in some examples, hinge candidate detection may be performed at least in part based on the orientation of the detected shape within the second image.

[0140] At operation 1422, candidate hinge positions in the second image are identified. A candidate hinge position in the second image is the location of a hinge candidate in the second image. In some examples, the software can identify the candidate hinge position in the second image by determining the coordinate values ​​(e.g., X and Y coordinate values) of each of the detected hinge candidates within the second image. Now refer to... Figure 19 An example of a hinge candidate list 1900 is shown, which includes information corresponding to exemplary hinge candidates that can be detected by software at operation 1420. Specifically, each row of the hinge candidate list 1900 includes information for the corresponding hinge candidate. The first (i.e., leftmost) value listed in each row is the corresponding X-coordinate value of the hinge candidate's position (e.g., center point) within the second image. The second (i.e., center) value listed in each row is the corresponding Y-coordinate value of the hinge candidate's position (e.g., center point) within the second image. The third (i.e., rightmost) value listed in each row is the corresponding determined radius length of the hinge candidate.

[0141] At operation 1424, adjusted second image hinge positions associated with multiple hinges in the second image are calculated. The adjusted second image hinge positions can be calculated at least in part based on the projected second image hinge positions (determined at operation 1418) and candidate second image hinge positions (determined at operation 1422). For example, Figure 14BAn example is shown of a set of sub-operations 1424A-F that may be included in operation 1424 to calculate the adjusted second image hinge position. Specifically, at sub-operation 1424A, hinge candidates are grouped into a set of hinge candidate groups. In some examples, hinge candidates may be grouped at least in part based on size characteristics (e.g., radius length) and / or positional similarity. For example, in some cases, two or more hinge candidates may be grouped together if their respective size characteristics (e.g., radius length) are within a selected allowable threshold size / length of each other. Moreover, in some cases, two or more hinge candidates may be grouped together if their respective positions are within a selected allowable threshold distance of each other. For example, two or more hinge candidates may be grouped together if their respective X-coordinate position (e.g., center point) values ​​are within a selected allowable threshold distance of each other and their respective Y-coordinate position (e.g., center point) values ​​are within a selected allowable threshold distance of each other. For example, as Figure 19 As shown, four underlined rows in the hinge candidate list 1900 indicate examples of four hinge candidates that can be grouped into a hinge candidate group. Specifically, it can be seen that the four underlined rows include X-coordinate values ​​that are adjacent to each other (X:134, X:134, X:136, and X:138), for example, such that they are within a selected allowed X-coordinate threshold distance. It can also be seen that the four underlined rows include Y-coordinate values ​​that are adjacent to each other (Y:376, Y:378, Y:378, and Y:378), for example, such that they are within a selected allowed Y-coordinate threshold distance. In some examples, the radius lengths of the four underlined rows (11, 12, 12, and 14) are close to each other, for example, such that they are within a selected allowed radius length threshold. Therefore, corresponding to Figure 19 The four hinge candidates in the four underlined rows can be grouped into corresponding hinge candidate groups. It should be understood that, although... Figure 19 Not shown in the image, but corresponding to... Figure 19 Other hinge candidates in other rows can also be grouped into other hinge candidate groups.

[0142] Now for reference Figure 20 Example of a hinge candidate group list 2000 is shown, which includes hinge candidate groups corresponding to those that can be made by Figure 19The hinge candidate list 1900 contains information about exemplary hinge candidate groups formed by the identified hinge candidates. Specifically, each row of the hinge candidate group list 2000 includes information for the corresponding hinge candidate group. The first (i.e., leftmost) value listed in each row is the number of hinge candidates included in the corresponding group. The second (i.e., center) value listed in each row is the average of the X-coordinate position values ​​of the hinge candidates within the corresponding group. The third (i.e., rightmost) value listed in each row is the average of the Y-coordinate position values ​​of the hinge candidates within the corresponding group. The underlined rows in the hinge candidate group list 2000 include information about hinge candidate groups formed by the four hinge candidates corresponding to the four underlined rows in the hinge candidate list 1900.

[0143] At sub-operation 1424B, the set of hinge candidate groups is weighted. In some examples, the set of hinge candidate groups may be weighted at least in part based on the number of hinge candidates within each hinge candidate group in the set. In some examples, hinge candidate groups with more included hinge candidates may be assigned higher priority weights, while hinge candidate groups with fewer included hinge candidates may be assigned lower priority weights. In some examples, the number of hinge candidates in each group may correspond to the exact weight assigned to that group. As shown in hinge candidate group list 2000, the weight assigned to each hinge candidate group may be the first value shown in each row (i.e., the leftmost), which indicates the number of hinge candidates included in the corresponding group. For example, the underlined row of hinge candidate group list 2000 indicates that the corresponding hinge candidate group includes four hinge candidates, and this group may therefore receive a weight of four. In contrast, the top row of hinge candidate group list 2000 indicates that the corresponding hinge candidate group includes five hinge candidates, and this group may therefore receive a weight of five.

[0144] At sub-operation 1424C, the highest-weighted subset of hinge candidate groups can be selected from the hinge candidate group set. For example, if a subset of four highest-weighted hinge candidate groups is selected from hinge candidate group list 2000, this subset will include the hinge candidate groups represented by the first / top four rows of hinge candidate group list 2000 (e.g., with corresponding weights of five or four). As another example, if a subset of seven highest-weighted hinge candidate groups is selected from hinge candidate group list 2000, this subset will include the hinge candidate groups represented by the first / top seven rows of hinge candidate group list 2000 (e.g., with corresponding weights of five, four, or three).

[0145] At sub-operation 1424D, the software calculates multiple average group positions for a subset of the highest-weighted hinge candidate groups. Each of these average group positions can be associated with a corresponding hinge candidate group in the highest-weighted subset of the hinge candidate groups. For example, the average group position of a group can include the average of the X-coordinate values ​​(i.e., the second / center values ​​shown in each row of the hinge candidate group list 2000) of all hinge candidates in that group and the average of the Y-coordinate values ​​(i.e., the third / rightmost values ​​shown in each row of the hinge candidate group list 2000). Now refer to Figure 21 An example of averaging group position 2120 is shown, calculated for a hinge candidate group 2100 comprising four hinge candidates 2111, 2112, 2113, and 2114. Hinge candidates 2111, 2112, 2113, and 2114 are labeled with the letter HC and include solid outlines, while average group position 2120 is labeled with the letter A and includes dashed outlines. As shown, average group position 2120 is located at... Figure 21 A certain horizontal (X-axis) position is defined within hinge candidate group 2100, which is the average of the horizontal positions of the four hinge candidates 2111, 2112, 2113, and 2114. Additionally, the average group position 2120 is located at... Figure 21 A certain vertical (Y-axis) position in the hinge candidate group 2100, which is the average of the vertical positions of the four hinge candidates 2111, 2112, 2113 and 2114.

[0146] At sub-operation 1424E, a transformation matrix is ​​constructed describing the spatial relationship between the second image hinge position of the projection (determined at operation 1418) and multiple average group positions (determined at sub-operation 1424D). Now refer to Figure 22 This illustrates an example where the subset with the highest weighted hinge candidate group (selected at sub-operation 1424C) comprises six hinge candidate groups. For example... Figure 22 As shown, six average group positions 2200 are calculated, including an average group position for each of the six hinge candidate groups in the highest weighted subset of the hinge candidate groups. Figure 22 In the diagram, the six average group positions 2200 are each marked with the letter A and include a dashed outline. Figure 22 In this specific example, each of the six average group positions 2200 is directly aligned with the corresponding actual hinge position. This is merely a simplified example chosen for clarity and ease of illustration. In practice, the average group positions 2200 do not need to be directly aligned with the corresponding actual hinge positions, but can generally be close to (but not exactly the same as) the corresponding actual hinge positions. Figure 22As shown, six arrows (diagonally pointing to the upper right) are drawn between the average group position 2200 and the second image hinge position 1841 (shown as circles with bold / heavy outlines) of the six closest corresponding projections. Figure 22 The six diagonal arrows in the diagram represent the transformation matrix describing the spatial relationship between the second image hinge position 1841 and multiple average group positions 2200 of the projection. It should be understood that in... Figure 22 In the example, because only a subset of the six highest-weighted hinge candidates is selected, the six average group positions 2200 correspond to the second image hinge positions 1841 for only six (instead of all twelve) projections. In some examples, the software may use one or more related algorithms to compute the transformation matrix, such as the Iterative Point Cloud or Iterative Closest Point (ICP) algorithm and / or the Coherent Point Shift (CPD) algorithm.

[0147] At suboperation 1424F, a transformation matrix is ​​used to adjust the projected second image hinge position to the adjusted second image hinge position. This may include, for example, determining a spatial relationship that associates the average group position with the projected second image hinge position, and then using these spatial relationships (e.g., by inverting the rotation spatial relationship) to adjust (e.g., transform) the projected second image hinge position. For example, now referring to... Figure 23 The source used Figure 22 The six diagonal arrows (diagonally pointing to the upper right) from the average group position 2200 to the second image hinge position 1841 of its corresponding nearest projection (and representing the transformation matrix) are now inverted (diagonally pointing to the lower left) to represent the adjustments that can be made to the second image hinge position 1841 of the projection using the transformation matrix. Additionally, it should be noted that instead of just six arrows, Figure 23 Includes twelve diagonal arrows to indicate the hinge positions of the second image for all twelve projections that can be converted in this manner 1841. Now refer to Figure 24 This shows the output of the adjustment (e.g., transformation) performed at sub-operation 1424F. Specifically, each of the twelve projected second image hinge positions 1841 is diagonally offset downwards and to the left (e.g., Figure 23 (As shown by the twelve arrows in the image), to form twelve adjusted second image hinge positions 2400 (represented by circles with bold outlines). Figure 24 In this specific example, each of the twelve adjusted second image hinge positions 2400 is directly aligned with the corresponding actual hinge position. This is merely a simple example chosen for clarity and ease of illustration. In practice, it is not necessary for any or all of the adjusted second image hinge positions 2400 to be directly aligned with the corresponding actual hinge positions; rather, they can typically be close to (but not exactly the same as) the corresponding actual hinge positions.

[0148] In some examples, the software can use the adjusted second image hinge position 2400 to determine the positions of retaining rings 1511 and 1512 within the "LAT view" image 1501-B. The software can then generate ring graphic representations corresponding to retaining rings 1511 and 1512 respectively, and display these ring graphic representations at the determined positions of retaining rings 1511 and 1512 within the "LAT view" image 1501-B. Now refer to... Figure 25 As can be seen, the ring graphics representations 1731 and 1732 are generated by software and displayed at the corresponding positions of the fixed rings 1511 and 1512 within the "LAT View" image 1501-B.

[0149] Return to reference Figure 14A At operation 1426, the adjusted second image hinge position 2400 is used to determine the position and orientation of the first and second anatomical segment in three-dimensional space. For example, as described above relative to... Figure 3A As detailed in operation 322, imaging scene parameters can be used to determine the position and orientation of the first and second anatomical segments in three-dimensional space. Also as described above, imaging scene parameters can be obtained by comparing the illustrated position of a specific component or the illustrated position of the fixation element of the fixator in the two-dimensional space of the first and second images with the corresponding positions of these same fixation elements in actual three-dimensional space. At operation 1428, the fixation device and the physical positions of the first and second anatomical structures in three-dimensional space can then be used to determine manipulation of the fixation device for correcting deformities. For example, as also described above, such as relative to... Figure 3B Operation 338 can use the position and orientation of the first and second anatomical segments in three-dimensional space to determine the manipulation of the fixation device for correcting anatomical deformities (i.e., the treatment plan). Specifically, the treatment plan can be determined at least in part based on: determining whether it is necessary to change the position and / or orientation of the anatomical segments, for example, how to reposition the anatomical segments relative to each other to promote healing between the anatomical segments.

[0150] It should be noted that the above description of hinge detection techniques includes examples where prior knowledge from the first image is used for various purposes with respect to the second image, such as determining the hinge position of the second image projection in the second image and aiding in the identification of hinge candidates in the second image. However, it should be noted that the techniques described herein do not necessarily require prior knowledge from the first image to perform hinge detection in the second image (or vice versa). For example, in some cases, hinge candidates can be detected in an image by performing image analysis techniques based on automated software. Image analysis techniques may include performing a Hough transform to detect circular shapes within the image. The hinge position within the image can then be determined at least in part based on the detected hinge candidates, in some examples without using any prior knowledge from the other image. In some examples, various techniques described above, such as grouping, weighting, position averaging, and / or other techniques, may also be optionally employed. For example, in some cases, the detected hinge candidates may be grouped using the grouping techniques described above. In some examples, only a selected subset of the highest-weighted hinge groups may be used. In some examples, the average group position of the hinge groups may be calculated using the techniques described above. In some examples, these average group positions can be used as determined hinge positions, or determined hinge positions can be calculated at least in part based on these average group positions. The determined hinge positions can then be used to determine the physical positions of the fixation device, the first anatomical segment, and the second anatomical segment in three-dimensional space. The physical positions of the fixation device, the first anatomical segment, and the second anatomical segment can then be used to determine the manipulation of the fixation device for correcting deformities.

[0151] Exemplary computing device

[0152] See Figure 26 Suitable computing devices (such as exemplary computing device 78) may be configured to perform any or all of the techniques described above. It should be understood that computing device 78 may include any suitable device, examples of which include: desktop computing devices, server computing devices, or portable computing devices (such as laptops, tablets, or smartphones).

[0153] In an exemplary configuration, computing device 78 includes a processing section 80, a memory section 82, an input / output section 84, and a user interface (UI) section 86. It should be emphasized that the block diagram description of computing device 78 is exemplary and not intended to indicate a specific implementation and / or configuration. The processing section 80, memory section 82, input / output section 84, and user interface section 86 may be coupled together to enable communication between them. It should be understood that any of the above components may be distributed across one or more separate devices and / or locations.

[0154] In various embodiments, the input / output section 84 includes a receiver of the computing device 78, a transmitter of the computing device 78, or a combination thereof. The input / output section 84 is capable of receiving and / or providing information, for example, regarding communication with a network such as the Internet. It should be understood that the sending and receiving functions may also be provided by one or more devices external to the computing device 78.

[0155] Processing section 80 may include one or more processors. Depending on the exact configuration and type of the processor, memory section 82 may be volatile (such as certain types of RAM), non-volatile (such as ROM, flash memory, etc.), or a combination thereof. Computing device 78 may include additional storage devices (e.g., removable and / or non-removable storage devices), including but not limited to tape, flash memory, smart cards, CD-ROMs, digital versatile optical discs (DVDs) or other optical storage devices, magnetic tape cassettes, magnetic tape, disk storage devices or other magnetic storage devices, Universal Serial Bus (USB) compatible memory, or any other medium that can be used to store information and is accessible by computing device 78.

[0156] The computing device 78 may also include a user interface portion 86 that allows users to communicate with the computing device 78. The user interface 86 may include input elements that provide the ability to control the computing device 78 via, for example, buttons, soft keys, a mouse, voice-actuated controls, a touchscreen, movement of the computing device 78, visual cues (e.g., moving a hand in front of a camera on the computing device 78), etc. The user interface portion 86 may provide output, including visual information (e.g., via a display), audio information (e.g., via a speaker), mechanical information (e.g., via a vibration mechanism), or combinations thereof. In various configurations, the user interface portion 86 may include a display, one or more graphical user interfaces, a touchscreen, a keyboard, a mouse, an accelerometer, a motion detector, a speaker, a microphone, a camera, a tilt sensor, or any combination thereof. Therefore, a computing system including, for example, one or more computing devices 78 may include a processor, a display coupled to the processor, memory communicating with the processor, one or more graphical user interfaces, and a variety of other components. The memory may store instructions that, when executed by the processor, cause the computer system to perform operations, such as those described above. As used herein, the term computing system may refer to a system that includes one or more computing devices 78. For example, a computing system may include one or more server computing devices that communicate with one or more client computing devices.

[0157] While this document describes exemplary embodiments of apparatuses for implementing the disclosed techniques, the basic concepts can be applied to any computing device, processor, or system capable of conveying and presenting information as described herein. The various techniques described herein can be implemented in combination with hardware or software, or both where appropriate. Therefore, the methods and apparatuses described herein can be implemented, or certain aspects or portions thereof, in the form of program code (i.e., instructions) embodied in a tangible, non-transitory storage medium such as a floppy disk, CD-ROM, hard disk drive, or any other machine-readable storage medium (computer-readable storage medium), wherein the machine becomes an apparatus for implementing the techniques described herein when the program code is loaded into and executed by the machine. In the case of executing program code on a programmable computer, the computing device will generally include a processor, a storage medium readable by the processor (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device, such as a display. The display may be configured to display visual information. One or more programs may be implemented as components or in machine language as needed. The language may be a compiled or interpreted language and may be combined with a hardware-specific implementation.

[0158] It should be understood that the use of imaging analysis for plastic surgery fixation described in this article not only provides the use of non-orthogonal images, but also enables the use of overlapping images, images taken through different imaging techniques, and images taken under different settings, thus providing surgeons with greater flexibility compared to existing fixation and imaging techniques.

[0159] The techniques described herein can also be practiced via communication embodied in program code, transmitted through a transmission medium such as electrical wiring or cabling, fiber optic cable, or any other form of transmission. When implemented on a general-purpose processor, the program code, in conjunction with the processor, provides the sole means for invoking the functions described herein. Furthermore, any storage technology used in conjunction with the techniques described herein can always be a combination of hardware and software.

[0160] While the techniques described herein can be implemented and have been described in conjunction with various embodiments of the figures, it should be understood that other similar embodiments may be used, or modifications or additions may be made to the embodiments without departing from them. For example, it should be understood that the steps disclosed above can be performed in the order described above or in any other order as needed. Furthermore, those skilled in the art will recognize that the techniques described in this patent application are applicable to any environment, whether wired or wireless, and to any number of such devices connected via a communication network and interacting across the network. Therefore, the techniques described herein should not be limited to any single embodiment, but should be understood in breadth and scope according to the appended claims.

Claims

1. A computer-implemented method, comprising: The images show a first image and a second image of a first anatomical segment and a second anatomical segment, with fixation devices attached to the first anatomical segment and the second anatomical segment for correcting deformities of the first anatomical segment and the second anatomical segment. The first image and the second image are captured from different positions and orientations relative to each other. The fixation device includes a ring, a support bar, and a plurality of hinges. Receive an indication of the position of the first image hinge associated with the plurality of hinges in the first image; The second image hinge position of the projection associated with the plurality of hinges in the second image is determined at least in part based on the indication of the first image hinge position; In the second image, hinge candidates with shapes associated with the plurality of hinges are detected; Identify the candidate hinge position in the second image; The adjusted second image hinge position associated with the plurality of hinges in the second image is calculated at least in part based on the projected second image hinge position and the candidate second image hinge position; The adjusted second image hinge position is used to determine the physical positions of the fixation device, the first anatomical segment, and the second anatomical segment in three-dimensional space; as well as The fixation device and the physical locations of the first and second anatomical segments are used to determine the manipulation of the fixation device for correcting the deformity.

2. The computer-implemented method of claim 1, further comprising determining a range of expected dimensional characteristics for the plurality of hinges, wherein the hinge candidates are limited to shapes having determined dimensional characteristics within the range of expected dimensional characteristics.

3. The computer-implemented method according to claim 1, wherein, The detection of the hinge candidate is performed at least in part based on the orientation of the detected shape within the second image.

4. The computer-implemented method according to claim 1, wherein, The plurality of hinges have a circular shape, and the detection of the hinge candidates includes performing a Hough transform to detect the circular shape in the second image.

5. The computer-implemented method according to claim 1, wherein, Calculating the adjusted second image hinge position includes grouping the hinge candidates into a hinge candidate set.

6. The computer-implemented method according to claim 5, wherein, The hinge candidates are grouped at least in part based on the similarity of at least one of the size characteristics or the location.

7. The computer-implemented method according to claim 5, wherein, Calculating the adjusted second image hinge position also includes weighting the hinge candidate set at least in part based on the number of hinge candidates within each hinge candidate set.

8. The computer-implemented method according to claim 7, wherein, Calculating the adjusted second image hinge position also includes selecting the highest weighted subset of the hinge candidate groups from the set of hinge candidate groups.

9. The computer-implemented method according to claim 8, wherein, Calculating the adjusted second image hinge position also includes calculating a plurality of average group positions, each of which is associated with a corresponding hinge candidate group of the highest weighted subset of the hinge candidate group.

10. The computer-implemented method according to claim 9, wherein, Calculating the adjusted second image hinge position further includes: Construct a transformation matrix describing the spatial relationship between the hinge position of the projected second image and the plurality of average group positions; and The transformation matrix is ​​used to adjust the hinge position of the projected second image to the adjusted hinge position.

11. A non-transitory computer-readable storage medium storing instructions thereon, the instructions, when executed by one or more computing devices, causing the one or more computing devices to perform operations including: The images show a first image and a second image of a first anatomical segment and a second anatomical segment, with fixation devices attached to the first anatomical segment and the second anatomical segment for correcting deformities of the first anatomical segment and the second anatomical segment. The first image and the second image are captured from different positions and orientations relative to each other. The fixation device includes a ring, a support bar, and a plurality of hinges. Receive an indication of the position of the first image hinge associated with the plurality of hinges in the first image; The second image hinge position of the projection associated with the plurality of hinges in the second image is determined at least in part based on the indication of the first image hinge position; In the second image, hinge candidates with shapes associated with the plurality of hinges are detected; Identify the candidate hinge position in the second image; The adjusted second image hinge position associated with the plurality of hinges in the second image is calculated at least in part based on the projected second image hinge position and the candidate second image hinge position; The adjusted second image hinge position is used to determine the physical positions of the fixation device, the first anatomical segment, and the second anatomical segment in three-dimensional space; as well as The fixation device and the physical locations of the first and second anatomical segments are used to determine the manipulation of the fixation device for correcting the deformity.

12. The non-transitory computer-readable storage medium according to claim 11, wherein, The operation also includes determining a range of expected dimensional characteristics for the plurality of hinges, wherein the hinge candidates are limited to shapes having determined dimensional characteristics within the range of expected dimensional characteristics.

13. The non-transitory computer-readable storage medium according to claim 11, wherein, The plurality of hinges have a circular shape, and the detection of the hinge candidates includes performing a Hough transform to detect the circular shape in the second image.

14. The non-transitory computer-readable storage medium according to claim 11, wherein, Calculating the adjusted second image hinge position includes grouping the hinge candidates into a hinge candidate set.

15. The non-transitory computer-readable storage medium according to claim 13, wherein, The hinge candidates are grouped at least in part based on the similarity of at least one of the size characteristics or the location.

16. The non-transitory computer-readable storage medium according to claim 14, wherein, Calculating the adjusted second image hinge position also includes weighting the hinge candidate set at least in part based on the number of hinge candidates within each hinge candidate set.

17. The non-transitory computer-readable storage medium according to claim 16, wherein, Calculating the adjusted second image hinge position also includes selecting the highest weighted subset of the hinge candidate groups from the set of hinge candidate groups.

18. The non-transitory computer-readable storage medium according to claim 17, wherein, Calculating the adjusted second image hinge position also includes calculating a plurality of average group positions, each of which is associated with a corresponding hinge candidate group of the highest weighted subset of the hinge candidate group.

19. The non-transitory computer-readable storage medium according to claim 18, wherein, Calculating the adjusted second image hinge position further includes: Construct a transformation matrix describing the spatial relationship between the hinge position of the projected second image and the plurality of average group positions; and The transformation matrix is ​​used to adjust the hinge position of the projected second image to the adjusted hinge position.

20. A computing system, comprising: One or more computer processors; as well as One or more memories storing instructions that, when executed by the one or more computer processors, cause the one or more computer processors to perform operations including: The images show a first image and a second image of a first anatomical segment and a second anatomical segment, with fixation devices attached to the first anatomical segment and the second anatomical segment for correcting deformities of the first anatomical segment and the second anatomical segment. The first image and the second image are captured from different positions and orientations relative to each other. The fixation device includes a ring, a support bar, and a plurality of hinges. Receive an indication of the position of the first image hinge associated with the plurality of hinges in the first image; The second image hinge position of the projection associated with the plurality of hinges in the second image is determined at least in part based on the indication of the first image hinge position; In the second image, hinge candidates with shapes associated with the plurality of hinges are detected; Identify the candidate hinge position in the second image; The adjusted second image hinge position associated with the plurality of hinges in the second image is calculated at least in part based on the projected second image hinge position and the candidate second image hinge position; The adjusted second image hinge position is used to determine the physical positions of the fixation device, the first anatomical segment, and the second anatomical segment in three-dimensional space; as well as The fixation device and the physical locations of the first and second anatomical segments are used to determine the manipulation of the fixation device for correcting the deformity.

Images