Pre-operative planning of surgical revision procedures for orthopedic joints

Abstract

To provide techniques for determining a morbid approximation before implantation of a bone of an orthopedic joint in which one or more implant components have been placed.SOLUTION: A system is configured to: obtain image data of a joint of a patient; determine that the joint includes an existing implant; and generate an identification of a type for the existing implant and / or generate a pre-implantation, morbid approximation of the joint.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001]

[0001] This application claims the benefit of U.S. Provisional Patent Application No. 62 / 887,838, filed August 16, 2019, the entire contents of which are incorporated herein by reference. [Background technology]

[0002]

[0002] Surgical joint repair procedures involve the repair and / or replacement of damaged or diseased joints. As an example, a surgical joint repair procedure, such as joint replacement, may involve replacing a damaged joint or a damaged implant with an artificial joint that is implanted into the patient's bone. Proper selection or design of an appropriately sized and shaped artificial joint and proper positioning of the artificial joint are important to ensure an optimal surgical outcome. A surgeon may analyze the damaged bone to aid in artificial joint selection, design, and / or positioning, as well as the surgical steps of preparing the bone or tissue to receive or interact with the artificial joint. Summary of the Invention

[0003]

[0003] This disclosure describes techniques for determining a pre-implant, morbid approximation of bones of an orthopedic joint in which one or more implant components have been placed. This disclosure also describes techniques for determining the type of implant to be implanted in the orthopedic joint.

[0004]

[0004] According to one example, a method includes determining a shape model for an affected anatomical object; acquiring, by a computing system, image data of a patient's joint including an implant, wherein the image data comprises a 3D model of the joint with the implant; identifying voxels corresponding to bone in the 3D model of the joint with the implant; determining an initial shape estimate for the bone based on the voxels identified as corresponding to bone; aligning the initial shape estimate to the shape model to form an aligned initial shape estimate; and deforming the shape model based on the aligned initial shape estimate to generate an affected approximation of the bone before implantation.

[0005]

[0005] According to another example, a method includes acquiring, by a computing system, image data of a patient's joint including an implant, wherein the image data comprises a 3D model of the joint with the implant, identifying a first object within the 3D model, wherein the first object corresponds to a first component of the implant, determining a first vector based on the first object, identifying a second object within the 3D model, wherein the second object corresponds to a second component of the implant, determining a second vector based on the second object, determining a third vector that is perpendicular to a plane defined by the first vector and the second vector, determining a patient coordinate system based on the first vector, the second vector, and the third vector, identifying an anatomical object within the 3D model, generating an initial aligned shape that initially aligns the anatomical object from the image data to a shape model based on the determined patient coordinate system, and generating information indicative of the affected pre-implant shape of the anatomical object based on the initial aligned shape.

[0006]

[0006] According to another example, a method includes acquiring image data of a patient's joint by a computing system, determining by the computing system that the joint includes an existing implant, segmenting the image data of the joint, and generating a type identification for the existing implant.

[0007]

[0007] According to another example, a method includes obtaining, by a computing system, image data of a joint of a patient with an existing implant; identifying, by the computing system, an image of the existing implant within the image data of the joint of the patient with the existing implant; accessing, in a memory device, a database associating a plurality of implant products with images for the plurality of implant products; and identifying at least one implant product corresponding to the existing implant based on a comparison of the image of the existing implant to the images for the plurality of implant products, each implant product of the plurality of implant products being associated with at least one image.

[0008]

[0008] According to another example, a method includes acquiring, by a computing system, image data of a joint of a patient having an existing implant, identifying, by the computing system, a portion of the image corresponding to cement that secures a portion of the existing implant to bone, determining a cement strength of the joint of the patient having the existing implant based on the portion of the image corresponding to the cement, and generating an output based on the determined cement strength.

[0009]

[0009] According to another example, a device includes a memory and one or more processors implemented in a circuit and configured to determine a shape model for an affected anatomical object, acquire image data of a patient's joint including an implant, wherein the image data comprises a 3D model of the joint with the implant, identify voxels corresponding to bone in the 3D model of the joint with the implant, determine an initial shape estimate for the bone based on the voxels identified as corresponding to bone, align the initial shape estimate to the shape model to form an aligned initial shape estimate, and deform the shape model based on the aligned initial shape estimate to generate an affected approximation of the bone before implantation.

[0010]

[0010] According to another example, a device includes a memory and one or more processors implemented in a circuit and configured to acquire image data of a patient's joint including an implant, wherein the image data comprises a 3D model of the joint with the implant, identify a first object in the 3D model, wherein the first object corresponds to a first component of the implant, determine a first vector based on the first object, identify a second object in the 3D model, wherein the second object corresponds to a second component of the implant, determine a second vector based on the second object, determine a third vector that is perpendicular to a plane defined by the first vector and the second vector, determine a patient coordinate system based on the first vector, the second vector, and the third vector, identify anatomical objects in the 3D model, generate an initial aligned shape that initially aligns the anatomical objects from the image data to a shape model based on the determined patient coordinate system, and generate information indicative of the affected pre-implantation shape of the anatomical objects based on the initial aligned shape.

[0011]

[0011] According to another example, the device includes a memory and one or more processors implemented in circuitry and configured to acquire image data of a patient's joint, determine that the joint includes an existing implant, segment the image data of the joint, and generate a type identification for the existing implant.

[0012]

[0012] According to another example, a device includes a memory and one or more processors implemented in circuitry and configured to acquire image data of a joint of a patient with an existing implant, identify images of the existing implant within the image data of the joint of the patient with the existing implant, access a database that associates a plurality of implant products with images for the plurality of implant products, each implant product of the plurality of implant products being associated with at least one image, and identify at least one implant product that corresponds to the existing implant based on a comparison of the image of the existing implant to the images for the plurality of implant products.

[0013]

[0013] According to another example, a device includes a memory and one or more processors implemented in circuitry and configured to acquire image data of a joint of a patient with an existing implant, identify portions of the image corresponding to cement that secures a portion of the existing implant to the bone, determine a cement strength of the joint of the patient with the existing implant based on the portions of the image corresponding to the cement, and generate an output based on the determined cement strength.

[0014]

[0014] According to another example, a system includes means for determining a shape model for an affected anatomical object; means for acquiring, by a computing system, image data of a patient's joint including an implant, wherein the image data comprises a 3D model of the joint with the implant; means for identifying voxels corresponding to bone in the 3D model of the joint with the implant; means for determining an initial shape estimate for the bone based on the voxels identified as corresponding to bone; means for aligning the initial shape estimate to the shape model to form an aligned initial shape estimate; and means for deforming the shape model based on the aligned initial shape estimate to generate an affected approximation of the bone before implantation.

[0015]

[0015] According to another example, a system includes: means for acquiring, by a computing system, image data of a patient's joint including an implant; wherein the image data comprises a 3D model of the joint with the implant; means for identifying a first object within the 3D model; wherein the first object corresponds to a first component of the implant; means for determining a first vector based on the first object; means for identifying a second object within the 3D model; wherein the second object corresponds to a second component of the implant; means for determining a second vector based on the second object, the second vector being perpendicular to a plane defined by the first vector and the second vector; means for determining a patient coordinate system based on the first vector, the second vector, and the third vector; means for identifying an anatomical object within the 3D model; means for generating an initial aligned shape that initially aligns the anatomical object from the image data to a shape model based on the determined patient coordinate system; and means for generating information indicative of the affected pre-implantation shape of the anatomical object based on the initial aligned shape.

[0016]

[0016] According to another example, the system includes means for acquiring, by a computing system, image data of a patient's joint, means for determining, by the computing system, that the joint includes an existing implant, means for segmenting the image data of the joint, and means for generating a type identification for the existing implant.

[0017]

[0017] According to another example, a system includes means for obtaining, by a computing system, image data of a joint of a patient with an existing implant; means for identifying, by the computing system, an image of the existing implant within the image data of the joint of the patient with the existing implant; means for accessing, in a memory device, a database that associates a plurality of implant products with images for the plurality of implant products; and means for identifying at least one implant product that corresponds to the existing implant based on a comparison of the image of the existing implant to the images for the plurality of implant products, each implant product of the plurality of implant products being associated with at least one image.

[0018]

[0018] According to another example, the system includes means for acquiring, by a computing system, image data of a joint of a patient with an existing implant, means for identifying, by the computing system, a portion of the image corresponding to cement that secures a portion of the existing implant to the bone, means for determining a cement strength of the joint of the patient with the existing implant based on the portion of the image corresponding to the cement, and means for generating an output based on the determined cement strength.

[0019]

[0019] According to another example, a computer-readable storage medium stores instructions that, when executed by one or more processors, cause the one or more processors to determine a shape model for an affected anatomical object; acquire image data of a patient's joint including an implant, wherein the image data comprises a 3D model of the joint with the implant; identify voxels corresponding to bone in the 3D model of the joint with the implant; determine an initial shape estimate for the bone based on the voxels identified as corresponding to bone; align the initial shape estimate to the shape model to form an aligned initial shape estimate; and deform the shape model based on the aligned initial shape estimate to generate an affected approximation of the bone before implantation.

[0020]

[0020] According to another example, a computer-readable storage medium stores instructions that, when executed by one or more processors, cause the one or more processors to acquire image data of a patient's joint including an implant, wherein the image data comprises a 3D model of the joint with the implant, identify a first object within the 3D model, wherein the first object corresponds to a first component of the implant, determine a first vector based on the first object, identify a second object within the 3D model, wherein the second object corresponds to a second component of the implant, determine a second vector based on the second object, determine a third vector that is perpendicular to a plane defined by the first vector and the second vector, determine a patient coordinate system based on the first vector, the second vector, and the third vector, identify an anatomical object within the 3D model, generate an initial aligned shape that initially aligns the anatomical object from the image data to a shape model based on the determined patient coordinate system, and generate information indicative of the affected pre-implantation shape of the anatomical object based on the initial aligned shape.

[0021]

[0021] According to another example, a computer-readable storage medium stores instructions that, when executed by one or more processors, cause the one or more processors to acquire image data of a patient's joint, determine that the joint includes an existing implant, segment the image data of the joint, and generate a type identification for the existing implant.

[0022]

[0022] According to another example, a computer-readable storage medium stores instructions that, when executed by one or more processors, cause the one or more processors to acquire image data of a joint of a patient with an existing implant, identify an image of the existing implant within the image data of the joint of the patient with the existing implant, access a database that associates a plurality of implant products with images for a plurality of implant products, and identify at least one implant product that corresponds to the existing implant based on a comparison of the image of the existing implant to the images for the plurality of implant products, each implant product of the plurality of implant products being associated with at least one image.

[0023]

[0023] According to another example, a computer-readable storage medium stores instructions that, when executed by one or more processors, cause the one or more processors to acquire image data of a joint of a patient with an existing implant, identify a portion of the image corresponding to cement that secures a portion of the existing implant to the bone, determine a cement strength of the joint of the patient with the existing implant based on the portion of the image corresponding to the cement, and generate an output based on the determined cement strength.

[0024]

[0024] The details of various examples of the disclosure are set forth in the accompanying drawings and the description below. Various features, objects, and advantages will be apparent from the description, drawings, and claims. [Brief explanation of the drawings]

[0025] [Figure 1]

[0025] A block diagram illustrating an example computing device that may be used to implement the techniques of this disclosure. [Figure 2]

[0026] FIG. 1 illustrates an exemplary implementation of a modification unit. [Figure 3A]

[0027] FIG. 1 shows an axial slice of a computed tomography (CT) scan of a shoulder joint including a reverse implant. [Figure 3B]

[0028] FIG. 3B shows the implant of FIG. 3A after segmentation to isolate the implant components. [Figure 4]

[0029] FIG. 10 is a diagram showing an example of an implant analysis unit. [Figure 5]

[0030] Diagram showing an axial slice of a CT scan of a shoulder joint including an anatomical implant. [Figure 6]

[0031] FIG. 3C illustrates how feature extraction can be performed on the segmented image of FIG. 3B. [Figure 7A]

[0032] 10A-10C illustrate how feature extraction can be performed on segmented images that isolate implant components for an anatomical implant. [Figure 7B] 10A-10C illustrate how feature extraction can be performed on segmented images that isolate implant components for an anatomical implant. [Figure 8]

[0033] FIG. 10 is a diagram showing an example of bone pseudo-segmentation. [Figure 9A]

[0034] FIG. 1 illustrates an example of patient anatomy that is first aligned to a statistical shape model (SSM) for use with implant features. [Figure 9B]

[0035] FIG. 1 illustrates an example of an SSM registered to patient anatomy using iterative closest point (ICP) registration. [Figure 10]

[0036] FIG. 10 illustrates an example of SSM on a patient projection to generate a diseased pre-implant approximation of the patient anatomy. [Figure 11]

[0037] An example of a CT scan with overlaid scapula pseudo-segmentation and scapula SSM predictions. [Figure 12]

[0038] Figure 1 shows an example CT scan with overlaid scapula pseudo-segmentation, scapula SSM predictions, and ML probability maps. [Figure 13]

[0039] FIG. 10 is a diagram showing an example of a machine learning unit. [Figure 14]

[0040] 1 is an illustration of a scapula with points of interest used to determine the patient coordinate system. [Figure 15A]

[0041] 12 is an illustration of a planar cut through the scapula to determine the patient's coordinate system. [Figure 15B] 12 is an illustration of a planar cut through the scapula to determine the patient's coordinate system. [Figure 16]

[0042] Conceptual diagram of a perspective view of a line normal to a plane through the scapula for determining the patient's coordinate system. [Figure 17]

[0043] FIG. 10 is a conceptual diagram of another perspective view of the normal to the plane through the scapula for determining the patient's coordinate system. [Figure 18]

[0044] Schematic diagram illustrating the transverse axis through the scapula for determining the patient's coordinate system. [Figure 19A]

[0045] 19 is a conceptual diagram illustrating an example of a sagittal cut through the scapula for determining the transverse axis of FIG. 18. [Figure 19B] 19 is a conceptual diagram illustrating an example of a sagittal cut through the scapula for determining the transverse axis of FIG. 18. [Figure 20A]

[0046] 19 is a conceptual diagram illustrating the results of a sagittal cut through the scapula to determine the transverse axis of FIG. [Figure 20B] 19 is a conceptual diagram illustrating the results of a sagittal cut through the scapula to determine the transverse axis of FIG. [Figure 20C] 19 is a conceptual diagram illustrating the results of a sagittal cut through the scapula to determine the transverse axis of FIG. [Figure 21A]

[0047] 19 is a conceptual diagram illustrating another example of a sagittal cut through the scapula to determine the transverse axis of FIG. 18. [Figure 21B] 19 is a conceptual diagram illustrating another example of a sagittal cut through the scapula to determine the transverse axis of FIG. 18. [Figure 22]

[0048] 1 is a conceptual diagram illustrating the transverse axis through the scapula and the normal to the plane through the scapula for determining the patient's coordinate system. [Figure 23]

[0049] 10 is a conceptual diagram illustrating the movement of the transverse axis through the scapula and the normal to the plane through the scapula relative to the location of the center of the glenoid cavity to determine the patient's coordinate system. [Figure 24]

[0050] 1 is a conceptual diagram illustrating an example of an initial alignment of a segmentation object to a shape model. [Figure 25]

[0051] 1 is a conceptual diagram illustrating an example of intermediate alignment of a segmentation object to a shape model. [Figure 26]

[0052] 1 is a conceptual diagram illustrating an example for determining a difference value for an ICP algorithm. [Figure 27A]

[0053] 1 is a conceptual diagram illustrating portions of a glenoid for determining parameters of a cost function used to determine the pre-morbid shape of a patient's anatomy. [Figure 27B] 1 is a conceptual diagram illustrating portions of a glenoid for determining parameters of a cost function used to determine the pre-morbid shape of a patient's anatomy. [Figure 28]

[0054] 10 is a flowchart illustrating an example method of operation of a computing device in accordance with one or more example techniques described in this disclosure. [Figure 29]

[0055] 10 is a flowchart illustrating an example method of operation of a computing device in accordance with one or more example techniques described in this disclosure. [Figure 30]

[0056] 10 is a flowchart illustrating an example method of operation of a computing device in accordance with one or more example techniques described in this disclosure. [Figure 31]

[0057] 10 is a flowchart illustrating an example method of operation of a computing device in accordance with one or more example techniques described in this disclosure. [Figure 32]

[0058] 10 is a flowchart illustrating an example method of operation of a computing device in accordance with one or more example techniques described in this disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0026]

[0059] A patient may suffer from an injury or condition (e.g., disease) that causes damage to the patient anatomy. With respect to the shoulder, as an example of a patient anatomy, a patient may suffer from primary glenohumeral osteoarthritis (PGHOA), rotator cuff tear arthropathy (RCTA), instability, massive rotator cuff tear (MRCT), rheumatoid arthritis (RA), post-traumatic arthritis (PTA), osteoarthritis (OA), or acute fracture, to name a few.

[0027]

[0060] To address a disease or injury, a surgeon may perform a surgical procedure such as a reverse total arthroplasty (RA), an extended reverse total arthroplasty (RA), a standard total shoulder arthroplasty (TA), an extended total shoulder arthroplasty (TA), or hemispherical shoulder surgery, to name a few. It may be beneficial for the surgeon to determine characteristics (e.g., size, shape, and / or location) of the patient's anatomy prior to surgery. For example, determining characteristics of the patient's anatomy may aid in planning prosthesis selection, design, and / or positioning, as well as surgical steps to prepare the damaged bone surface to accept or interact with the prosthesis. Advance planning allows the surgeon to determine bone or tissue preparation steps, tools needed during surgery, the size and shape of the tools, the size, shape, or other characteristics of one or more prosthetic joints to be implanted, etc., prior to surgery rather than during surgery.

[0028]

[0061] Placing an additional or replacement implant component into a joint already mated with an existing implant component is sometimes referred to as revision surgery or surgical revision. Surgical revision procedures for orthopedic joints are fairly common. For example, approximately 30% of total shoulder arthroplasties are surgical revision procedures. A surgical revision procedure may be required when a previous surgical procedure or component fails. Examples of how a procedure or component fails can include one or more implant components experiencing other complications, such as trauma or infection, a disease state that further progresses to the point of implant failure, or a patient with an existing implant component that otherwise fails or ceases to function properly. The presence of an existing implant component may correlate with a patient also having a fracture or fragment or otherwise deteriorated bone condition. Obtaining good images, segmentation, and modeling of the joint can be particularly important in preoperative and intraoperative planning for surgical revision procedures.

[0029]

[0062] The computing device may segment joint image data (e.g., CT image data) to isolate individual anatomical objects so that their shapes and sizes are visualized. The computing device may present a 3D image of the patient anatomy to a physician or other user, including specific anatomical structures such as the scapula, clavicle, glenoid cavity, and humerus. When acquiring and segmenting the image data, the presence of existing implants, particularly metal implants, can potentially complicate the segmentation process. The implants may, for example, introduce noise or artifacts in the CT image that make the segmentation difficult to perform or that make the resulting segmentation unreliable.

[0030]

[0063] This disclosure describes techniques for segmenting image data representing an orthopedic joint in which one or more implant components have been placed (e.g., to perform a revision procedure). The techniques of this disclosure may help overcome or avoid complications caused by noise and artifacts caused by the implants, enabling a computing device to produce a useful segmentation of the joint, even when the joint includes an implant component. Moreover, this disclosure describes techniques that may enable a device to obtain information about one or more existing implant components and the joint from the acquired image data. The device may alternatively or additionally prepare a revision surgical treatment plan to replace the existing implant components with one or more new implant components.

[0031]

[0064] In some examples, the techniques may enable a computing device to provide intraoperative surgical guidance to a surgeon or surgical team for performing an informed surgical revision procedure. Thus, the techniques of the present disclosure may improve existing computing devices by enabling the computing device to better support preoperative and / or intraoperative surgical planning and guidance for surgical revision procedures. This better surgical planning and / or guidance may take the form of improved segmentation of joints with existing implants, but may also take the form of more reliable surgical classification and / or recommendations, such as automated classification of existing implant components and automated recommendations to the surgeon related to procedure type and implant type for surgical revisions to replace such existing implant components.

[0032]

[0065] The techniques of the present disclosure may provide better pre- and intra-operative surgical guidance for revision surgery by enabling a computing device to present various types of 2D and / or 3D images to a physician, nurse, patient, or other user via a display that can aid in pre-operative planning and intra-operative decision-making. In some examples, the computing device may be configured to generate images that either show multiple portions of a joint and / or images that isolate various portions of a joint. Despite the presence of an existing implant, the computing device may be configured to generate images that show the joint (in either a post-diseased state or a pre-diseased approximation) as if the implant were not present. In other examples, the computing device may be configured to generate images that either show the existing implant or annotate the image to indicate the presence of the implant.

[0033]

[0066] Pre-morbid anatomy, also called native anatomy, refers to the anatomy before the onset of disease or the occurrence of injury. Even after disease or injury, there may be healthy portions of the anatomy and unhealthy (e.g., diseased or damaged) portions of the anatomy. The diseased or damaged portions of the anatomy are referred to as pathological anatomy, and the healthy portions of the anatomy are referred to as non-pathological anatomy.

[0034]

[0067] As used in this disclosure, pre-morbid characterization refers to characterizing a patient's anatomy as it existed before the patient suffered a disease or injury, and thus, even before the implantation of an implant. Because patients typically do not see a physician or surgeon until their anatomy is already diseased or injured, pre-morbid characterization of an anatomy is often unavailable. As used in this disclosure, diseased pre-implant characterization refers to characterizing a patient's anatomy as it existed before the patient was implanted with an implant but after suffering a disease or injury. As used in this disclosure, post-implant characterization refers to characterizing a patient's anatomy as it exists after the implantation of an implant.

[0035]

[0068]

[0013] Figure 1 is a block diagram illustrating an example computing device that may be used to implement the techniques of this disclosure. Figure 1 illustrates device 100, which is an example of a computing device configured to perform one or more example techniques described in this disclosure.

[0036]

[0069] Device 100 may include various types of computing devices, such as a server computer, a personal computer, a smartphone, a laptop computer, and other types of computing devices. Device 100 includes processing circuitry 102, memory 104, and a display 112. Display 112 is optional, such as in examples where device 100 is a server computer.

[0037]

[0070] Examples of processing circuitry 102 include one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. In general, processing circuitry 102 may be implemented as a fixed-function circuit, a programmable circuit, or a combination thereof. A fixed-function circuit refers to a circuit that provides a specific function and is preconfigured with respect to the operations it can perform. A programmable circuit refers to a circuit that can be programmed to perform various tasks and provide flexible functionality in the operations it can perform. For example, a programmable circuit may execute software or firmware that causes the programmable circuit to operate in a manner defined by the software or firmware instructions. A fixed-function circuit may execute software instructions (e.g., to receive parameters or output parameters), but the types of operations the fixed-function circuit performs are generally invariant. In some examples, one or more of the units may be different circuit blocks (fixed function or programmable), and in some examples, one or more units may be integrated circuits.

[0038]

[0071] The processing circuitry 102 may include an arithmetic logic unit (ALU), an elementary function unit (EFU), digital circuits, analog circuits, and / or a programmable core formed from programmable circuitry. In examples where the operations of the processing circuitry 102 are performed using software executed by programmable circuitry, the memory 104 may store object code for the software received and executed by the processing circuitry 102, or another memory within the processing circuitry 102 (not shown) may store such instructions. An example of software is software designed for surgical planning.

[0039]

[0072] The memory 104 may be formed by any of a variety of memory devices, such as dynamic random access memory (DRAM), including synchronous dynamic random access memory (SDRAM), magnetoresistive random access memory (MRAM), resistive random access memory (RRAM), or other types of memory devices. Examples of the display 112 include a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or another type of display device.

[0040]

[0073] Device 100 may include a communications interface 114 that enables device 100 to output data and instructions to and receive data and instructions from visualization device 118 via network 116. For example, after determining pre-morbid characteristics of an anatomical object, using techniques described in this disclosure, communications interface 114 may output the pre-morbid characteristic information to visualization device 118 via network 116. The surgeon may then view a graph of the anatomical object using visualization device 118. The viewed anatomical object may be either a pre-morbid anatomical object, a pre-implantation diseased anatomical object, or a post-implantation anatomical object. In some examples, visualization device 118 may present, for example, a pre-implantation diseased anatomical object or a post-implantation anatomical object overlaid on an image of an injured or diseased anatomical object. Overlaying images from different times may allow a physician or surgeon to more easily observe deterioration of the anatomical object.

[0041]

[0074] Communications interface 114 may be hardware circuitry that allows device 100 to communicate (e.g., wirelessly or using wires) with other computing systems and devices, such as visualization device 118. Network 116 may include various types of communications networks, including the Internet, one or more wide area networks, such as a local area network, etc. In some examples, network 116 may include wired and / or wireless communications links.

[0042]

[0075] The visualization device 118 may utilize various visualization techniques to display image content to the surgeon. The visualization device 118 may be a mixed reality (MR) visualization device, a virtual reality (VR) visualization device, a holographic projector, or other device for presenting extended reality (XR) visualization. In some examples, the visualization device 118 may be a Microsoft HOLOLENS® headset available from Microsoft Corporation of Redmond, Washington, USA, or a similar device, such as a similar MR visualization device that includes a waveguide. The HOLOLENS® device can be used to present 3D virtual objects through holographic lenses or waveguides, while allowing a user to view actual objects in a real-world scene, i.e., in a real-world environment, through the holographic lenses.

[0043]

[0076] The visualization device 118 can facilitate preoperative planning for joint repair and replacement using visualization tools available to utilize patient image data to generate three-dimensional models of bone contours. These tools enable surgeons to design and / or select surgical guides and implant components that perfectly match the patient's anatomy. These tools can improve surgical outcomes by customizing the surgical plan for each patient. One example of such a visualization tool for shoulder repair is the BLUEPRINT® system available from Wright Medical Technology, Inc. The BLUEPRINT® system provides surgeons with a two-dimensional plan view of the bone repair area as well as a three-dimensional virtual model of the repair area. Surgeons can use the BLUEPRINT® system to select, design, or modify appropriate implant components, determine how to best position and orient the implant components and how to prepare the bone surfaces to accept the components, and design, select, or modify surgical guide tools or instruments to perform the surgical plan. The information generated by the BLUEPRINT® system is compiled into a pre-operative surgical plan for the patient that is stored in a database in an appropriate location (e.g., on a server within a wide area network, local area network, or global network) that is accessible by the surgeon or other caregivers, including before and during the actual surgery.

[0044]

[0077] As shown, memory 104 stores data representing a shape model 106, data representing an anatomical scan 108, and an implant library 110. Although shape model 106, data representing an anatomical scan 108, and implant library 110 are shown as being stored locally on device 100 in the example of FIG. 1, in other examples, the contents of shape model 106, data representing an anatomical scan 108, and implant library 110 may be stored remotely and accessed, for example, via network 116.

[0045]

[0078] The anatomical structure scan 108 is an example of a computed tomography (CT) scan of a patient, represented, for example, by CT scan image data. The anatomical structure scan 108 may be sufficient to construct a three-dimensional (3D) representation of the patient's anatomical structures, such as the scapula and glenoid cavity, through either automated or manual segmentation of the CT image data to produce segmented anatomical objects. One exemplary implementation of automated segmentation is described in U.S. Patent No. 8,971,606. There may be various other ways of performing automated segmentation, and the technique is not limited to automated segmentation using the technique described in U.S. Patent No. 8,971,606. As an example, segmentation of the CT image data to produce segmented objects includes comparing voxel intensities in the image data to determine bony anatomical structures and estimated sizes of the bony anatomical structures to determine the segmented objects. Additionally, exemplary techniques may be implemented using non-automated segmentation techniques in which a medical professional evaluates CT image data to segment anatomical objects, or some combination of automation and user input to segment anatomical objects. U.S. Provisional Patent Application No. 62 / 826,119, filed March 29, 2019, and U.S. Provisional Patent Application No. 62 / 826,190, filed March 29, 2019, describe aspects of segmentation and are both incorporated by reference herein in their entireties.

[0046]

[0079] In one or more examples, the anatomy scan 108 may be a scan of an anatomy that includes an implant and is therefore pathological due to injury or disease. A patient may have an injured shoulder that requires corrective treatment, and for the treatment, or perhaps as part of a diagnosis, a surgeon may have requested the anatomy scan 108 to plan the surgery. A computing device (such as device 100 or some other device) may generate a segmentation of the patient anatomy so that the surgeon can see the anatomical objects, their size, shape, and interconnections with other anatomical structures of the patient anatomy requiring surgery.

[0047]

[0080] As described in more detail below, processing circuitry 102 may utilize image data from scan 108 to compare (e.g., size, shape, orientation, etc.) with a statistical shape model (SSM) as a way of determining characteristics of the patient's anatomy before the patient suffers injury or disease, or after suffering injury or disease but before an implant is placed. In other words, in the case of a revision, the patient may have an injured or diseased bone into which an implant is implanted, and then, in the case of a revision procedure, processing circuitry 102 determines characteristics of the injured or diseased bone at the time the implant is implanted. The characteristics of the injured or diseased bone before implantation may help the surgeon perform the revision procedure. The SSM may represent either a model of the pre-diseased anatomy or a model of the diseased anatomy before implantation. In some examples, processing circuitry 102 may compare 3D point data of non-pathological points of anatomical objects of the patient's anatomy in the image data from scan 108 with points in the SSM.

[0048]

[0081] For example, scan 108 provides the surgeon with a view of the current characteristics of the damaged or diseased anatomical structure, including the implant, if one has been placed. In order to reconstruct the anatomical structure (i.e., to present a pre-diseased or diseased pre-implanted state), the surgeon may find it useful to have a model that shows the characteristics of the anatomical structure before the injury or disease and / or before the implant. For example, the model may be a predictor of the patient's anatomy before the injury or disease and / or before the implantation of the implant. Just as the patient may not likely have seen the surgeon until after the injury occurred or the disease had progressed, a model of the patient's anatomy before the injury or disease (e.g., the pre-diseased or native anatomy) may not be available. Similarly, even if the patient did not see the surgeon before the implant was placed, a model of the patient's anatomy before the implantation may not be available because the procedure was performed long ago, because the procedure was performed by a different surgeon or healthcare service, or for any number of other reasons. By using SSM as a way of modeling the pre-diseased and / or diseased pre-implant anatomy, surgeons can determine characteristics of the patient anatomy that can no longer be determined by simply looking at an anatomy scan alone. As explained in more detail below, using SSM in connection with treatment modifications presents unique challenges, and this disclosure describes techniques that may improve the device's ability to provide image data to the surgeon and provide surgical guidance.

[0049]

[0082] The implant library 110 represents a database of known implants. For each known implant, the implant library 110 may store the manufacturer's designation for the implant, the model for the implant, the surgical technique needed to install or remove the implant, the countries in which the implant was publicly available and the date ranges in which the implant was publicly available in those countries, and such information for other known implants. For each known implant, the implant library 110 may also store information about the implant, such as the type of implant (e.g., anatomical, reverse, etc.), the fixation method for the implant (e.g., cement, screw, etc.), and the sizes available for the implant. For each implant, the implant library 110 may also store removal information, such as the designation of the tool needed to remove the implant and instructions for removing the implant. The implant library 110 may also store associated 2D and / or 3D images for each known implant.

[0050]

[0083] As described in more detail below, as part of planning an implant removal operation, processing circuitry 102 may determine which implants from implant library 110 correspond to the patient's implants by using known information about the patient's implants and image data of the patient's implants. Accordingly, processing circuitry 102 may output information, such as removal information, regarding implants being removed from the patient as part of the surgery. Based on the determined removal information, the surgeon may be able to better pre-operatively plan for implant removal by knowing, for example, what tools will be needed to remove a particular implant and how long the removal is expected to take in advance of the operation.

[0051]

[0084] When a revision case is encountered, processing circuit 102 may implement different processing techniques than for non-revision cases. Processing circuit 102 may determine, for example, if a particular case is a revision case based on user input from a member of the surgical team or other user, or based on information stored in a patient profile. However, in some instances, such as when a patient has experienced trauma and is non-responsive, the user of device 100 may not know whether the patient has already had an implant installed, in which case processing circuit 102 may determine that the case is a revision case based on anatomy scan 108.

[0052]

[0085] In revision cases where the patient has an existing implant placed in or proximal to the joint, processing circuit 102 of device 100 may be configured to obtain an anatomical scan 108 from memory 104. Processing circuit 102 may be configured to determine that the joint includes an implant based on the anatomical scan 108 or based on user input. Processing circuit 102 may, for example, present a query to a user of device 100 asking the user whether the patient's joint includes an implant, and the user of device 100 may be able to respond to the query with a yes, no, or don't know answer. In other examples, processing circuit 102 may be configured to determine that the joint includes an implant by extracting such information, for example, from medical records stored in memory 104 or from medical records stored remotely from device 100 but otherwise accessible to device 100 via a network connection.

[0053]

[0086] Alternatively or additionally, processing circuitry 102 may be configured to determine the presence of an implant based on the image data by performing image analysis of the image data for an anatomy scan 108. Image data, in this context, should be understood to include both a displayed image and raw image data associated with an anatomy scan 108. In this context, raw image data refers to image data used to determine a displayed image. The raw image data may have a different resolution, bit depth, or dynamic range than the displayed image and may additionally be in a different color space or color model. Generally, raw image data includes data captured as part of imaging, including data used by the display 112 or visualization device 118 to display an image, and in some instances, even data not directly used by the display 112 or visualization device 118 to display an image. For example, raw image data may include data that is not even traditionally considered to be image data in the context of the present disclosure. As an example, within CT image data, pixels are associated with relative radiodensity values ​​corresponding to average attenuation measured in Hounsfield units (HU) using the Hounsfield scale. These HU values ​​are examples of raw image data. A display device converts HU values ​​to grayscale for display. Unless otherwise stated, it should be assumed that techniques of this disclosure described as being performed on image data can be performed using either displayed images or raw images.

[0054]

[0087] Additionally, it should be understood that image data in this disclosure refers to both 2D image data and 3D image data. For simplicity, some techniques are described as being performed on pixels, which represent locations in a 2D model, but unless otherwise stated, it should be assumed that the techniques can also be performed on voxels, which represent locations in a 3D model. Similarly, unless otherwise stated, it should be assumed that techniques described as being performed on voxels may also be performed on pixels.

[0055]

[0088] The processing circuitry 102 may determine the presence of an implant using intensity thresholding on the raw image data of the anatomical scan 108. When imaged using a CT scan, different materials produce pixels of different intensities. When measured in HU, for example, water has a value of approximately 0, air has a value of approximately -1000, cortical bone ranges from 200 to 500, and metal implants typically have an intensity above 1000. In some examples, implants may be cemented in place using bone cement to secure the implant to the bone. Bone cement may also produce pixels of unique intensities. For example, in a CT scan, polymethyl methacrylate (PMMA) bone cement typically produces voxels with an expected intensity of 1423±70, while other types of cement may have different intensities. Pixels with similar intensities generally belong to the same object (e.g., soft tissue, bone, implant, cement).

[0056]

[0089] By determining pixel intensities, processing circuit 102 can categorize objects in the image as belonging to either bone, soft tissue, implant components, cement, or some other category. As described in more detail below, some pixels may be categorized as either noise or indeterminable. By performing pattern matching and shape analysis on the objects using techniques described in this disclosure, processing circuit 102 may further categorize the objects as belonging to, for example, a particular bone or a particular component of an implant. For example, for an object in an image corresponding to a bone, processing circuit 102 may analyze the shape of the object to determine whether the bone is a humerus, a glenoid, or some other bone. Similarly, for an object in an image corresponding to an implant component, processing circuit 102 may analyze the shape of the object to determine whether the implant component is a stem, a glenoid prosthesis, a humeral prosthesis, or some other component of an implant.

[0057]

[0090] Figure 2 shows a modification unit 200. The modification unit 200 of Figure 2 may be, for example, a component of the device 100 of Figure 1 included within the processing circuit 102. The modification unit 200 includes an implant analysis unit (IAU) 202, which includes an implant segmentation unit 204, an implant identification unit 206, and a feature extraction unit 208. The modification unit 200 also includes a bone pseudo-segmentation unit 210, a projection unit 212, a trained machine learning (TML) system 214, and a post-processing unit 216.

[0058]

[0091] The implant segmentation unit 204 receives the anatomical scans 108, which include CT scans of the joint with the implant. The CT scans may collectively form a 3D model of the joint with the implant. The implant segmentation unit 204 segments the 3D model to isolate the implant by identifying voxels of the 3D model having HU values ​​indicative of corresponding to the implant. Because HU values ​​for metal implants are typically above 1000 and HU values ​​for cortical bone, connective tissue, and other parts of the joint are substantially lower, the implant segmentation unit 204 may, for example, classify voxels with HU values ​​above 1000 or some other threshold level as corresponding to implant components and voxels with HU values ​​below 1000 as corresponding to non-implant components.

[0059]

[0092] The CT scan includes voxels corresponding to the implant. In some cases, the implant generates noise outside of where the implant is located and where it overlaps with the patient's anatomy in the CT scan. Noise in the CT scan generated by a metal implant tends to have an HU value that is substantially below the HU value of the voxels corresponding to the metal implant.

[0060] Because voxels corresponding to implants have high HU, processing circuitry 102 may be configured to accurately identify implants based on HU thresholding. However, because voxels resulting from noise generated by implants have significantly lower HU values ​​when such voxels from the noise are mixed with voxels of the patient anatomy, processing circuitry 102 may not easily classify the voxels as belonging to the patient anatomy or to noise generated in the image by implants. Therefore, noise typically does not complicate the process of segmenting a 3D model to isolate implants in the same way that noise complicates segmenting a 3D model to isolate bones. The isolated voxels corresponding to implants determined by implant segmentation unit 204 may be referred to as an implant mask. The implant mask represents a 3D model in which all voxels that do not correspond to implants are set to a common value that is different from the values ​​of those voxels corresponding to non-implants. Thus, the implant mask isolates implants from all other non-implants in the 3D model.

[0061]

[0093] Figure 3A shows an axial slice of a CT scan of a shoulder joint including a reverse implant. Thus, Figure 3A represents a 2D slice of a 3D model of the joint that may be included, for example, in anatomy scan 108. The axial scan in Figure 3A shows the humeral cap component (302A), the glenoid ball component (304A), and the screw (306A) that attaches the glenoid ball component to the scapula. Figure 3A also shows bones (e.g., 308A and elsewhere) and other aspects of the joint.

[0062]

[0094] FIG. 3B shows the implant of FIG. 3A after segmentation to isolate the implant. Accordingly, FIG. 3B shows an axial scan of the implant mask, showing the humeral cap component (302B), the glenoid ball component (304B), and the screw (306B) attaching the glenoid ball component 304B to the scapula. However, all other non-implant pixels in FIG. 3A are set to a constant value in FIG. 3B, in stark contrast to the implant components. FIG. 3B represents a 2D example of an implant mask generated by the implant segmentation unit 204.

[0063]

[0095] The implant identification unit 206 determines a type of implant based on the implant mask determined by the implant segmentation unit 204 and outputs the type of implant to a user. In this context, the type of implant refers to, for example, a hemispherical implant type, a reverse implant type, or an anatomical implant type. As will be described in more detail with reference to FIG. 4 , the implant identification unit 206 may determine the type of implant based on the number of unconnected objects in the implant mask determined by the implant segmentation unit 204.

[0064]

[0096] As will be described in more detail with reference to FIG. 4, the feature extraction unit 208 analyzes the segmented image of the implant, i.e., the implant mask, to determine features of the implant, such as the geometric center of the implant sphere, which may serve as an approximation for the center of the glenoid fossa.

[0065]

[0097] Figure 4 shows an IAU 402, which represents one example implementation of IAU 202 in Figure 2. IAU 402 includes an implant segmentation unit 404 connected to a thresholding unit 406, a glenoid marker detection unit 408, and a feature extraction unit 410. Implant segmentation unit 404 generally operates in the same manner as described with respect to implant segmentation unit 204 of Figure 2 to determine an implant mask.

[0066]

[0098] Based on the determined implant mask, the connected thresholding unit 406 determines the number of disconnected objects in the implant mask. The connected thresholding unit 406, for example, identifies groups of adjacent voxels having HU values ​​corresponding to implants. A group of adjacent voxels corresponding to implants that is completely surrounded by voxels corresponding to non-implants is referred to herein as an object. A first group of voxels corresponding to implants that is completely surrounded by voxels corresponding to non-implants and a second group of voxels corresponding to implants that is completely surrounded by voxels corresponding to non-implants constitute a first object and a second object, respectively. If none of the first group of voxels corresponding to implants are adjacent to any of the second group of voxels corresponding to implants, the first object and the second object are considered disconnected objects. That is, there is a patient anatomical structure between the first object and the second object, and therefore they are not connected. To avoid identifying extraneous voxels or groups of voxels as unconnected objects, the connected thresholding unit 406 may identify only those groups of voxels that have a certain shape or minimum size as objects.

[0067]

[0099] If the connected thresholding unit 406 determines that the implant mask contains three unconnected objects, the connected thresholding unit 406 determines that the type of the existing implant comprises an undefined type, meaning that the type is unknown. If the connected thresholding unit 406 determines that the implant mask contains two unconnected objects, the connected thresholding unit 406 determines that the type of the existing implant comprises a reverse-type implant. Because the glenoid implant components of an anatomical implant are typically not primarily metallic, the connected thresholding unit 406 may not detect the glenoid implant objects as being present in the implant mask. In contrast, in the case of a reverse-type implant, both the humeral implant component and the glenoid implant component are primarily metallic. FIG. 3B, for example, shows two unconnected objects, the first of which is a humeral cap component (302B) and the second of which is a glenoid ball component (304B) and a screw (306B). Although the glenoid ball component 304B and the screw 306B appear to be separate objects in the axial scan of FIG. 3B, in the actual 3D model the two components are a common object.

[0068]

[0100] Figure 5 shows an axial slice of a CT scan of a shoulder joint including an anatomical implant. Figure 5 shows a spherical humeral implant component 502 and shows a glenoid marker 504. The glenoid marker 504 is a small component within the larger glenoid implant component. Because the larger glenoid implant component is mostly non-metallic, the larger glenoid implant component is not easily identifiable within the CT scan.

[0069]

[0101] If the connected thresholding unit 406 detects an object, the glenoid marker detector 408 may search a second search location for the presence of a glenoid marker. The second search location may be, for example, a reduced region within the unsegmented 3D model identified based on proximity to the detected object. For example, the glenoid marker detector 408 may be configured to search only radially outward from the spherical surface of the humeral implant 502, which generally corresponds to the area to the left of the humeral implant 502 in FIG. 5 . The glenoid marker detector 408 may not need to search an area radially inward from the humeral implant component 502, which generally corresponds to the area to the right of the humeral implant 502 in FIG. 5 . In the case of a hemispherical implant, the patient also has a spherically shaped implant component on the humeral side; however, unlike an anatomical implant, a hemi-implant does not include a scapular implant component and instead uses the patient's natural bone to receive the humeral implant component.

[0070]

[0102] Although primarily non-metallic, the glenoid implant component includes a metallic marker, such as marker 504 in FIG. 5. The metallic marker is, for example, a strip of metal that may appear as a roughly straight line or thin rectangle on an axial scan of a CT image. The straight line is typically used by the surgeon to determine or monitor the orientation of the glenoid component and to determine if the orientation changes over time.

[0071]

[0103] If the glenoid marker detection unit 408 determines that the second search location corresponds to metal and therefore includes voxels indicative of a glenoid marker, the glenoid marker detector unit 408 may determine that the implant type for the implant is an anatomical implant type. If the glenoid marker detection unit 408 determines that the second search location corresponds to metal and therefore does not include voxels indicative of a glenoid marker, the glenoid marker detector unit 408 may determine that the implant type for the implant is a hemi-implant type. The absence of a glenoid marker means that the patient does not have a scapular implant component, which corresponds to a hemi-implant that does not require modification of the patient's scapula.

[0072]

[0104] Because glenoid markers tend to be relatively small and therefore contain less metal, they typically do not cause the same intensity or particularly large objects in a CT scan. Thus, when identifying glenoid markers, glenoid marker detection unit 408 may utilize intensity and size thresholds that differ from the intensity and size thresholds used by connected thresholding unit 406 when determining the number of disconnected objects.

[0073]

[0105] The feature extraction unit 410 may be configured to determine features of the implant based on the implant type and the implant mask. As described in more detail below, the feature extraction unit 410 may be configured to determine a patient coordinate system that can be used to determine a diseased pre-implantation approximation of the bone using the SSM.

[0074]

[0106] FIG. 6 is a diagram illustrating how feature extraction may be performed on a segmented image isolating implant components for a reverse-type implant. FIG. 6 shows an axial slice 600 of the segmented image. The axial slice 600 may be, for example, a slice of a 3D implant mask determined by the implant segmentation unit 204 or the implant segmentation unit 404. The axial slice 600 shows a glenoid sphere 602, a coordinate system center 604, and a vector 606. For a reverse-type implant, the feature extraction unit 410 may be configured to use pattern matching to locate a partially spherical object, i.e., a group of voxels in the implant mask, that corresponds to the glenoid sphere. The partially spherical object may have, for example, a spherical surface and a flat circular surface. The feature extraction unit 410 may identify a center location on the flat circular surface as the coordinate system center (604 in FIG. 6 ). The feature extraction unit 410 may identify a vector (606 in FIG. 6 ) that is orthogonal to the flat circular surface.

[0075]

[0107] 7A and 7B are additional diagrams illustrating how feature extraction may be performed on a segmented image isolating implant components for a reverse-style implant. FIG. 7A shows an axial slice 700 of a segmented image. The axial slice 700 may be, for example, a slice of a 3D implant mask determined by the implant segmentation unit 204 or the implant segmentation unit 404. The axial slice 700 shows a glenosphere 702 and a humeral stem 708. For a reverse-style implant, the feature extraction unit 410 may be configured to locate a group of voxels in the implant mask that corresponds to the humeral stem (708 in FIG. 7A ). The feature extraction unit 410 may identify the humeral stem by searching the implant mask for elongated objects using pattern matching. In this context, an elongated object generally refers to an object in which one dimension, such as length, is significantly greater than another dimension, such as width or depth. For elongated objects, the feature extraction unit 410 may be configured to determine a vector (710 in FIG. 7A) pointing from the distal end of the humeral stem 708 to the proximal end of the humeral stem 708.

[0076]

[0108] FIG. 7B shows another view of axial slice 700B. FIG. 7B shows coordinate system center 704, which may correspond to coordinate system center 604 described above, and vector 706, which may correspond to vector 606 described above. FIG. 7B also shows vector 710 determined from humeral stem 708 (e.g., vector 710 in FIG. 7B illustrates vector 710 in FIG. 7A extended through center 604 beyond the distal end of humeral stem 708). As can be seen in FIG. 7B, vectors 706 and 710 are not perfectly orthogonal. Therefore, feature extraction unit 410 may determine a plane defined by vectors 706 and 710 and rotate vector 710 on that plane so that it is orthogonal to vector 706. This rotated vector is shown in FIG. 7B as vector 712. Thus, in FIG. 7B, vector 706 and vector 712 are orthogonal. The feature extraction unit may additionally determine a third vector centered at coordinate system center 704 that is orthogonal to, or perpendicular to, the plane formed by vector 706 and rotated vector 712. This orthogonal vector is perpendicular to (e.g., exits or enters) the 2D image of Figure 7B and is shown in Figure 7B as vector 714.

[0077]

[0109] Vectors 706, 712, and 714 thus form a patient coordinate system centered at coordinate system center 704. Vector 706 approximates the medio-lateral axis for the glenoid fossa, and vector 712 approximates the infero-superior axis for the glenoid fossa. In anatomy, positions closer to the patient's head are commonly referred to as superior, and positions closer to the patient's feet are commonly referred to as inferior. The median plane refers to an imaginary plane running through the center of the patient's body, for example, through the nose and pelvis. Positions closer to the medial plane are commonly referred to as medial, and positions further from the median plane are commonly referred to as lateral.

[0078]

[0110] In the case of an anatomical implant, the feature extraction unit 410 determines a first vector based on the spherical implant component and a second vector based on the humeral stem component in the same manner as described above for a reverse-type implant. However, in some implementations, the first vector may alternatively be determined based on a glenoid marker. However, in the case of an anatomical implant, the spherical implant component is a humeral implant component instead of a glenoid implant component. Similar to a reverse-type implant, the feature extraction unit 410 may also rotate the second vector determined from the humeral stem to determine an orthogonal second vector and to determine a third vector that is orthogonal, i.e., perpendicular, to the plane formed by the first vector and the rotated second vector. However, unlike a reverse-type implant, in the case of an anatomical implant, the feature extraction unit 410 determines a coordinate system center based on the location of the glenoid marker in the scapular implant component. The first vector, the rotated second vector, and the third vector form a coordinate system centered on the coordinate system center.

[0079]

[0111] Returning to FIG. 2 , the bone pseudo-segmentation unit 210 performs pseudo-segmentation of the bones of the joint. The bone pseudo-segmentation unit 210 may, for example, identify voxels of the 3D model having HU values ​​indicative of bone. Because the bone pseudo-segmentation unit 210 is configured to classify as bone only those voxels with HU values ​​within a narrow range of HU values ​​associated with bone, the segmentation performed by the bone pseudo-segmentation unit 210 may be referred to as “pseudo” segmentation. The bone pseudo-segmentation unit 210 classifies voxels outside that narrow range as not bone, even if those voxels may actually correspond to bone. As previously discussed, the metal of the implant generates noise that may make the classification of some voxels uncertain based on the CT scan alone. Some voxels of the 3D model that actually correspond to bone may not have HU values ​​within the narrow range, for example, due to the presence of noise. For purposes of determining pseudo-segmented bone, the bone pseudo-segmentation unit 210 may be configured to classify such uncertain voxels as not bone and not attempt to further classify such voxels.

[0080]

[0112] 8 shows an example of a pseudo-segmented bone image 800 that may be generated by the bone pseudo-segmentation unit 210. In the example of FIG. 8, area 802 is an example area in image 800 that the bone pseudo-segmentation unit 210 determined to be bone based on the HU values ​​of the voxels for that portion of the 3D model. Areas 804A and 804B represent example areas in image 800 that the bone pseudo-segmentation unit 210 determined to be definitively not bone based on the HU values ​​of the voxels for that portion of the 3D model. Such areas may, for example, correspond to bone but may be obscured by noise or may correspond to non-bone, such as tissue or an implant.

[0081]

[0113] The projection unit 212 generates an estimate of the affected pre-implant anatomical object, such as the scapula, based on the shape model 106, the pseudo-segmented bone determined by the bone pseudo-segmentation unit 210, and the shape model of the implant features determined by the feature extraction unit 206. The particular shape model of the shape model 106 used by the projection unit 212 may be selected by a member of the surgical team, for example, based on a known classification of the joint before the implant is placed. The known classification may be, for example, the Walch classification of the glenoid, which is discussed in more detail below.

[0082]

[0114] As described in more detail below, the projection unit 212 may use the pseudo-segmented bone (e.g., image 800 of FIG. 8 ) as an initial shape estimate for the bone. The projection unit 212 may align the initial shape estimate to a shape model to form an initially aligned shape estimate. The projection unit 212 may align the initial shape estimate to the shape model using a determined patient coordinate system, for example, as discussed elsewhere in this disclosure. The patient coordinate system may be determined based on features determined by the feature extraction unit 208 or based on other techniques described below for determining a patient coordinate system.

[0083]

[0115] Figure 9A shows the initial alignment of an initial shape estimate 900 to a shape model 902. The initial alignment is determined based on a determined patient coordinate system. Figure 9B shows the alignment of the initial shape estimate 900 to the shape model 902 after rotation. Techniques for determining the initial alignment and for performing the rotation are described in more detail below.

[0084]

[0116] After alignment, the projection unit 212 may then deform the shape model based on the aligned shape estimate to generate a diseased pre-embedded approximation of the bone. The projection unit 212 may perform the deformation using, for example, iterative closest point (ICP) and / or elastic registration. A more detailed description of the functionality of the projection unit 212 is provided below. FIG. 10 shows a diseased pre-embedded approximation 1000, which represents an example output of the projection unit 212.

[0085]

[0117] The modification unit 200 may output the diseased pre-implanted approximation of the bone to a member of the surgical team via a display device, for example, for use in pre-operative planning or for other purposes. The modification unit 200 may also use the pre-implanted approximation of the bone to perform segmentation on a 3D model of the patient anatomy.

[0086]

[0118] The projection unit 212 may also output the affected pre-implant approximation of the bone to the TML system 214. Based on the implant mask from the implant segmentation unit 204, the affected pre-implant approximation from the projection unit 212, and the determined implant features (e.g., information indicative of a coordinate system) from the feature extraction unit 208, the TML system 214 segments the 3D model (e.g., the anatomy scan 106 to isolate specific portions of the patient anatomy, such as the scapula, clavicle, glenoid, humerus, etc.). The TML system 214 may be configured to perform segmentation based on machine learning, as described in more detail below. The TML system 214 generates a probability classification map, which may associate each voxel or group of voxels with a probability that it corresponds to a specific piece of the anatomy.

[0087]

[0119] As described above, the bone pseudo-segmentation unit 210 can segment implants from non-implants in the 3D model with relatively little ambiguity. Identifying the boundary between the scapula and adjacent bones, for example, requires more complex decision-making. Therefore, the TML system 214 may be configured to remove implants from the 3D model. In this context, removal may mean setting voxels corresponding to the implants to a constant, different color, such as black. The TML system 214 may then compare the affected pre-implant approximation with the remaining non-implant voxels and, based on this comparison, determine the probability that a particular voxel or group of voxels corresponds to the scapula, the humerus, etc.

[0088]

[0120] The post-processing unit 216 may, for example, process the segmented image and the probability classification map to determine and output a final segmented image. In one example, the final segmented image may include annotations indicating boundaries corresponding to the pseudo-segmentation determined by the pseudo-segmentation unit 210, boundaries corresponding to the affected pre-embedding approximation, or other such boundaries. In FIG. 11 , for example, boundaries 1102A and 1102B may be presented in a first style, such as a color or line type, to indicate that they correspond to the pseudo-segmented image, and boundaries 1104A and 1104B may be presented in a different style to indicate that they correspond to the affected pre-embedding approximation.

[0089]

[0121] The probability classification map represents a confidence level for each voxel or group of voxels. The probability map may be used by the post-processing unit 216 to generate probabilities for multiple segmentations. FIG. 12 shows an example of an output image with three proposed segmentations, each with an associated probability. A 90% probability may mean, for example, that the area within the line has been determined to have a 90% probability of belonging to the scapula. Lines that are farther apart have lower probabilities, meaning that the segmentations are more likely to include bones that do not belong to the scapula.

[0090]

[0122] The post-processing unit 216 may be configured to receive user input, which may be, for example, a selection of one of the proposed segmentations or a manual segmentation guided by a probability classification map.

[0091]

[0123] Figure 13 shows an exemplary training system for the TML system 214. The training system of Figure 13 includes an IAU 1302, which includes a bone pseudo-segmentation unit 1304, an implant segmentation unit 1306, and a feature extraction unit 1308. The training system of Figure 13 also includes a manual segmentation unit 1310 and an ML system 1312. The IAU 1302, bone pseudo-segmentation unit 1304, implant segmentation unit 1306, and feature extraction unit 1308 generally function in the same manner as the IAU 202, bone pseudo-segmentation unit 204, implant segmentation unit 206, and feature extraction unit 208 described above to generate implant masks and determine implant features.

[0092]

[0124] For a particular anatomical scan from the anatomical scans 108, the IAU 1302 generates an implant mask and a pseudo-segmentation and determines implant features. For the same particular anatomical scan, the manual segmentation unit 1310 produces one or more manual segmentations of the anatomical scan. The manual segmentation may be generated, for example, based on input from a surgeon or other expert. The manual segmentation may serve as ground truth for the ML system 1312. The ML system 1312 determines how to segment the anatomical scan based on the pseudo-segmented image, the implant mask, and the determined features in a manner that results in the same segmentation as the manual segmentation, e.g., the mean or mode of the manual segmentation. The ML system 1312 may use machine learning to learn these segmentations using a large number of training samples. The machine learning techniques may then be implemented by the TML system 214.

[0093]

[0125] Aspects of shape modeling will now be described in more detail. The processing circuitry 102 and projection unit 212 may be configured, for example, to determine a pre-diseased or diseased pre-implantation approximation of the patient anatomy. SSM is a compact tool that represents shape variation among a population of (database) samples. For example, a clinician or researcher may generate a database of image scans, such as CT image data, of different people that collectively represent a population that does not suffer from a damaged or diseased glenoid, humerus, or adjacent bone. Additionally or alternatively, a clinician or researcher may generate a database of image scans of different people that collectively represent a population that does suffer from a particular type of damaged or diseased glenoid, humerus, or adjacent bone.

[0094]

[0126] As an example, the morphological characteristics of the glenoid are traditionally classified using a system known as the Walch classification. The Walch classification of diseased glenoids is determined based on axial sections of two-dimensional CT scans, and the morphological characteristics of the glenoid are divided into three major groups with subtypes. Type A is a centered or symmetrical arthritis without posterior subluxation of the humeral head. Type A1 has minor central wear or erosion, and Type A2 has severe or extensive central wear or erosion. Type B is characterized by asymmetrical arthritis with posterior subluxation of the humeral head. Type B1 has no obvious glenoid erosion, with posterior joint space narrowing, subchondral sclerosis, and osteophytes. Type B2 has obvious or pronounced erosion of the posterior glenoid, creating a biconcave appearance of the glenoid. Type C indicates glenoid retroversion greater than 25° (occurring in dysplasia) regardless of glenoid erosion or the location of the humeral head relative to the glenoid fossa.

[0095]

[0127] A clinician or researcher may determine an average shape for the patient anatomy from the shapes in the database. The clinician or researcher may determine an average shape for the patient anatomy for people who do not suffer from damaged joints, as well as an average shape for each of various types of diseased joints. For example, the clinician or researcher may determine an average shape for a type A glenoid, a type A1 glenoid, a type B glenoid, etc. Various techniques below are described with respect to a shape model 106, which may represent either a pre-diseased model or a diseased model. As described in more detail below, the average shapes for various types of diseased joints may be used to determine diseased pre-implantation characterizations of anatomical objects for patients with implants.

[0096]

[0128] Returning to FIG. 1 , memory 104 stores information indicative of shape model 106. As an example, shape model 106 represents the average shape for anatomical objects (e.g., glenoid fossa, humeral head, etc.) of a patient's anatomy from shapes in a database. Other examples of shape model 106, such as the mode or median of shapes in a database, are also possible. A weighted average is another possible example of shape model 106. Shape model 106 may be a surface or volume of points (e.g., a graphical surface or volume of points defining a 3D model), and the information indicative of the average shape may be coordinate values ​​for the vertices of primitives that interconnect to form the surface. Techniques for generating SSMs such as shape model 106 and values ​​associated with generating shape model 106 may be found in various publications, such as from http: / / www.cmlab.csie.ntu.edu.tw / ~cyy / learning / papers / PCA_ASM.pdf, dated August 22, 2006. Other ways of generating the shape model 106 are possible.

[0097]

[0129] Using the shape model 106, the processing circuit 102 can represent anatomical structure shape variations by adding points or values ​​of the shape model 106 to a covariance matrix. For example, the SSM can be interpreted as a linear equation.

[0098]

number

[0099]

[0130] In the above equation, s′ is the shape model 106 (e.g., a point cloud of an average shape, as an example, where the point cloud defines the coordinates of points in the shape model 106, such as the vertices of the primitives that form the shape model 106). i is the eigenvalue, v i are the eigenvectors of the covariance matrix (also called the modes of variation). The covariance matrix describes the variance within a dataset. The element at position i,j is the covariance between the ith and jth elements of the dataset array.

[0100]

[0131] SSM represents constructing the covariance matrix of the database and then performing a "singular value decomposition" to extract a matrix of principal vectors (also called eigenvectors) and another diagonal matrix of positive values ​​(called eigenvalues). i ) is the new coordinate system basis of the database. i ) is the eigenvector (v i ) . Both the eigenvectors and eigenvalues ​​can reflect the amount of variation about the corresponding axis.

[0101]

[0132] This formula is used when the processing circuit 102 calculates the weights b i Just change s j For example, to generate a new shape model, the processing circuitry may i Determine the value of s i In the example above, λ i and v i and s' are all known based on the manner in which s' was generated (e.g., based on the manner in which the shape model 106 was generated). i By selecting different values ​​of , the processing circuit 102 can generate different instances of the shape model (e.g., different s that are different variations of the shape model 106). i ) can be determined.

[0102]

[0133] The shape model may represent how the anatomical object should appear to the patient. As will be described in more detail, processing circuitry 102 may compare points (e.g., in the 3D cloud of points) of the shape model of the anatomical object to anatomical points represented in the image data of scan 108 as anatomical points of the anatomical object that are not or minimally affected by injury or disease (e.g., non-pathological points). Based on this comparison, processing circuitry 102 may determine a pre-diseased or diseased pre-implantation characterization of the anatomical object.

[0103]

[0134] As an example, assume that a surgeon wants to determine the pre-diseased characteristics of the glenoid cavity for shoulder surgery, or the diseased pre-implantation characteristics of the glenoid cavity for revision surgery. There is a correlation between the shape of the glenoid cavity and the bony zones around it, such as the medical glenoid vault, acromion, and coracoid. The shape model 106 may be the average shape of the scapula, which includes the glenoid cavity. Assume that in the patient, the glenoid cavity is pathological (e.g., diseased or damaged).

[0104]

[0135] According to example techniques described in this disclosure, processing circuit 102 may determine an instance of “s” (e.g., a shape model of the scapula with a glenoid cavity) that best matches the non-pathological anatomical structure of the patient's anatomical object (e.g., a non-pathological portion of the scapula in scan 108). In the instance of “s,” the glenoid cavity that best matches the non-pathological anatomical structure (referred to as s* and represented by a point cloud similar to the shape model) may indicate a pre-diseased or diseased pre-implantation characterization of the pathological glenoid cavity.

[0105]

[0136] Processing circuitry 102 or projection unit 212 may determine an instance of s* (e.g., a shape model of the scapula with a glenoid cavity that best matches a non-pathological portion of the patient's scapula. The non-pathological portion may be a portion of the anatomical object with minimal to no impact from disease or injury, where the anatomical object and its surrounding anatomical structures are obtained from the segmentation performed by scan 108 to segment the anatomical object.

[0106]

[0137] In some instances, to determine the pre-diseased or diseased pre-implantation characterization of a particular anatomical object, anatomical structures beyond the anatomical object may be required to align the shape model. For example, to determine the pre-diseased or diseased pre-implantation characterization of a glenoid, the anatomical structure of the scapula may be required to align the shape model. In that case, the glenoid of the shape model represents the pre-diseased or diseased pre-implantation characterization of the patient's glenoid. Thus, the shape model may not be limited to modeling just the anatomical object of interest (e.g., the glenoid), but may include additional anatomical objects.

[0107]

[0138] The processing circuitry 102 or the projection unit 212 may perform the following exemplary operations: (1) initially align the segmented anatomical object from the image data of the scan 108 to the coordinate system of the shape model 106 to generate an initial aligned shape; (2) compensate for errors in the initial alignment by under- and over-segmentation, where the under- and over-segmentation are due to imperfections in generating the scan 108 to generate the aligned shape (also called an intermediate aligned shape); and (3) perform iterative operations including an iterative closest point (ICP) operation and elastic registration to determine an instance of s* (i.e., an instance of s′ that most closely matches the patient anatomical structure identified by the segmentation object). With regard to operation (1), this disclosure describes different techniques for determining the patient coordinate system depending on whether the patient has an anatomical implant or a reverse implant, or whether the patient has not yet had an implant installed or has had a hemi-implant installed. Exemplary techniques for implementing operations (1)-(3) are described in more detail below.

[0108]

[0139] There may be several technical issues with generating a pre-morbid or diseased pre-implant characterization of the patient anatomy. One issue may be that the image data needed from the scan 108 is unavailable or distorted due to under-segmentation or over-segmentation, creating challenges in aligning the segmented object to the shape model 106. Another issue may be that once alignment to the shape model 106 occurs, the registration of the shape model 106 to the segmented object may be poor, resulting in an insufficient pre-morbid or diseased pre-implant characterization.

[0109]

[0140] This disclosure describes exemplary techniques that enable alignment of a segmented object to a shape model 106 even in situations where there is under-segmentation or over-segmentation and where the patient already has an implant. This disclosure also describes exemplary techniques for registering a shape model 106 to a segmented object (e.g., in a multi-level iterative process using ICP and elastic registration). The exemplary techniques described in this disclosure may be used separately or together. For example, in one example, the processing circuit 102 may be configured to perform the alignment of a segmented object to a shape model 106 utilizing the exemplary techniques described in this disclosure, but to perform the registration of the shape model 106 to the segmented object using some other technique. In one example, the processing circuit 102 may perform the alignment of a segmented object to a shape model 106 utilizing some other technique, but to perform the registration of the shape model 106 to the segmented object using one or more exemplary techniques described in this disclosure. In some examples, processing circuitry 102 may perform the example alignment and registration techniques described in this disclosure.

[0110]

[0141] As described above, the processing circuit 102 may align the segmented object (e.g., as segmented using an exemplary technique such as voxel intensity comparison) to the shape model 106. In this disclosure, alignment refers to a change to the coordinate system of the segmented object such that the segmented object is in the same coordinate system as the shape model 106. As also described above, an example of a shape model 106 is a shape model of the anatomical structure of a scapula with a glenoid cavity, where the glenoid cavity is an injured or diseased anatomical object. For example, the shape model 106 is defined in its own coordinate system, which may be different from the patient coordinate system (e.g., the coordinate system that defines the point cloud of 3D points in the scan 108). The patient coordinate system is the coordinate system used to define points in the patient's anatomy in the scan 108, including non-pathological points and pathological points. Non-pathological points refer to points within the scan 108 for non-pathological anatomical structures, and pathological points refer to points within the scan 108 for pathological anatomical structures.

[0111]

[0142] There may be various ways to determine pathological and non-pathological points. As one example, a surgeon may examine image data of the scan 108 to identify pathological and non-pathological points. As another example, a surgeon may examine a graphical representation of a segmented object to identify pathological and non-pathological points. As another example, there may be an assumption that some anatomical points are rarely injured or diseased and are present in the image data of the scan 108. For example, the medical glenoid vault, acromion, and coracoid are a few examples. Additional examples include the trigonum scapulae and the subscapular angle. However, one or more of the trigonum scapulae and the subscapular angle may be absent or distorted in the image data of the scan 108 due to over-segmentation and under-segmentation.

[0112]

[0143] In one or more examples, processing circuit 102 may determine a patient coordinate system, which is a coordinate system recorded for the CT scan data of scan 108. Shape model 106 may be defined in its own coordinate system. Processing circuit 102 may determine the coordinate system of shape model 106 based on metadata stored with shape model 106, which metadata was generated as part of determining the average of the shapes in the database. Processing circuit 102 may determine a transformation matrix based on the patient coordinate system and the coordinate system of shape model 106. The transformation matrix is ​​how processing circuit 102 transforms a segmented object (e.g., a point in a 3D volume of points) into the coordinate system of shape model 106. For example, a point in a 3D volume of points of a segmented object may be defined using (x, y, z) coordinate values. The result of the transformation may be (x', y', z') coordinate values ​​that are aligned to the coordinate system of shape model 106. Processing circuit 102 calculates the transformation matrix through a closed form.

[0113]

[0144] There are multiple ways to determine the transformation matrix. As an example, the coordinate system may be c = (x, y, z, o), where x, y, and z are orthogonal basis 3D vectors and o is the origin (c is a 4x4 homogeneous matrix, with the last column being (0,0,0,1)). The new coordinate system into which the segmented object will be transformed is C = (X, Y, Z, O). In this example, the transformation matrix is ​​then T = c^-1 x C. Thus, processing circuitry 102 may determine the patient coordinate system.

[0114]

[0145] 14 through 20 describe exemplary manners in which processing circuitry 102 determines a patient coordinate system. As will be described in more detail, even after determining the patient coordinate system, further adjustments to shape model 106 may be required to properly align the segmented object (e.g., patient anatomy) to shape model 106. For example, due to over-segmentation or under-segmentation, missing or distorted image data cannot be used to determine the patient coordinate system. Thus, processing circuitry 102 utilizes image data that is available to perform an initial alignment, but may then perform further operations to fully align the segmented object to shape model 106.

[0115]

[0146] FIG. 14 is an illustration of a scapula with points of interest used to determine a patient coordinate system. For example, FIG. 14 illustrates a scapula 118 with a glenoid center 120, a scapular triangle 122, and a subscapular angle 124. Some techniques utilize the glenoid center 120, the scapular triangle 122, and the subscapular angle 124 to determine a patient coordinate system. For example, the glenoid center 120, the scapular triangle 122, and the subscapular angle 126 may be considered to form a triangle, and the processing circuit 102 may determine the center point of the triangle (e.g., in 3D) as the origin of the patient coordinate system.

[0116]

[0147] However, such techniques may not be available in cases of over-segmentation and under-segmentation. As an example, in under-segmentation, scan 108 may be cut inferiorly and / or superiorly and / or medially, and scapular triangle 122 and subscapular angle 124 may not be present in the segmented object extracted from the image data of scan 108. Therefore, processing circuitry 102 may not be able to utilize all three of glenoid center 120, scapular triangle 122, and subscapular angle 124 for purposes of determining the patient coordinate system.

[0117]

[0148] In one or more examples, if a particular landmark (e.g., glenoid center 120, scapular triangle 122, and subscapular angle 124) is not available as a segmented anatomical object in the image data of scan 108, processing circuit 102 may define a patient coordinate system based on the scapula normal (e.g., a vector perpendicular to scapula 118) and a transverse axis based on 3D information in the image data of scan 108. As described in more detail, processing circuit 102 may determine the scapula normal based on the normal of a plane that best fits scapular body 118. Processing circuit 102 may determine the transverse axis by fitting a line through the supraspinous fossa, the spinous column, and the cancellous region, which is between the scapular body. Using the cancellous region, processing circuit 102 may define a Cartesian (e.g., x, y, z) coordinate system (although other coordinate systems are possible). The origin of the coordinate system may be glenoid center 120, although other origins are also possible.

[0118]

[0149] In this way, processing circuitry 102 may determine a coordinate system for the patient based on anatomical objects present in the image data of scan 108, even when there is over-segmentation or under-segmentation. Based on the coordinate system, processing circuitry 102 may align the segmentation object to the coordinate system of shape model 106 (e.g., the initial SSM) and iteratively deform shape model 106 to the patient segmentation object (e.g., a deformed version of shape model 106) in order to register the shape model for purposes of determining pre-diseased or diseased pre-implant characterization.

[0119]

[0150] 15A and 15B are illustrations of planar cuts through the scapula to determine a patient coordinate system that may not rely on anatomical objects for which image data is unavailable due to over- or under-segmentation. For example, unlike the example of FIG. 14 , which relies on the glenoid center 120, scapular triangle 122, and subscapular angle 124, which may not be available due to under- or over-segmentation, the exemplary technique illustrated with reference to FIGS. 15A and 15B may not rely on anatomical objects that are unavailable due to under- or over-segmentation.

[0120]

[0151] As illustrated in Figure 15A, one of the scans 108 may be a scan of an axial cut through the scapula 118, showing the scapular portion 126. Figure 15B is a top view of the scapular portion 126 of Figure 15A through the scapula 118.

[0121]

[0152] 15B illustrates a plurality of dots that pass through scapular portion 126, representing intersecting planes through portion 126. In one or more examples, processing circuit 102 may determine the planes that intersect through scapular portion 126. The intersections of the planes through portion 126 are indicated by dots. For example, FIG. 15B illustrates line 128. Line 128 intersects with the majority of the dots within scapular portion 126.

[0122]

[0153] The dots in the scapular portion 126 may be the "skeleton" of the surrounding contour. Skeletons are dots through the contour that are each the same distance to their respective nearest boundary. While skeletons are described, as another example, in some instances the dots in the scapular portion 126 may be center points.

[0123]

[0154] Processing circuitry 102 may determine image data forming portion 126. Processing circuitry 102 may determine dots (e.g., skeletal dots or center dots, as two non-limiting examples) in portion 126, as shown. Processing circuitry 102 may determine a plane extending from line 128 up toward the glenoid cavity and from line 128 down toward the inferior angle of the scapula.

[0124]

[0155] For example, while FIG. 15A illustrates one exemplary axial cut, processing circuitry 102 may determine multiple axial cuts and determine points through the axial cuts, similar to FIG. 15B. The result may be a 2D point for each axial cut, and processing circuitry 102 may determine a line, similar to line 128, through each of the axial cuts. Processing circuitry 102 may determine a plane extending through the line for each axial cut through scapula 118. This plane passes through scapula 118 and is illustrated in FIG. 16.

[0125]

[0156] FIG. 16 is a conceptual diagram of a perspective view of a normal to a plane determined with the examples illustrated in FIGS. 15A and 15B through the scapula for determining a patient coordinate system. For example, FIG. 16 illustrates plane 130. Plane 130 intersects with line 128 illustrated in FIG. 15B. Plane 130 may be considered the plane that best fits scapular body 118. Processing circuitry 102 may determine a vector 132 that is perpendicular to plane 130. In one or more examples, vector 132 may form one axis (e.g., the x-axis) of the patient coordinate system. Techniques for determining other axes are described in more detail below.

[0126]

[0157] Figure 17 is a conceptual illustration of another perspective of the example of Figure 16, such as another perspective of the normal to the plane through the scapula for determining the patient's coordinate system. Figure 17 is similar to Figure 16, but from a front perspective rather than the side perspective of Figure 16. For example, Figure 17 illustrates the patient's scapula 118 along with a best fit plane 130 and normal vector 132.

[0127]

[0158] In this manner, processing circuitry 102 may determine a first axis of the patient coordinate system that is perpendicular to scapula 118. Processing circuitry 102 may determine a second axis of the patient coordinate system as an axis that intersects scapula 118.

[0128]

[0159] Figure 18 is a conceptual diagram illustrating a transverse axis through the scapula for determining a patient's coordinate system. For example, Figure 18 illustrates an example of determining an axis in addition to the one determined in Figures 4 and 5 for aligning the shape model 106 to the coordinate system of the image data of the scan 108.

[0129]

[0160] FIG. 18 illustrates a transverse axis 134. Processing circuitry 102 may determine transverse axis 134 as a line fit through the cancellous region between the supraspinous fossa, the spinal column, and the body of the scapula. An exemplary technique for determining transverse axis 134 is described with reference to FIGS. 19A, 19B, 20A-20C, and 21A and 21B, in which processing circuitry 102 may determine multiple sagittal cuts through the scapula based on image data from scan 108, as illustrated in FIGS. 19A and 19B. For example, FIGS. 19A and 19B are different perspective views of sagittal cuts through the scapula based on axis 133A. Processing circuitry 102 may utilize axis 133A based on an approximation of the cancellous region between the supraspinous fossa, the spinal column, and the body of the scapula, and may be pre-programmed with the estimated axis.

[0130]

[0161] 20A-20C illustrate one result of the sagittal cuts of FIGS. 19A and 19B. For example, as illustrated in FIG. 20A, the sagittal cut is in the shape of a "Y." Processing circuit 102 may determine skeleton lines passing through the Y-shape, as illustrated in FIG. 20B. For example, processing circuit 102 may determine dots that are equidistant to the nearest boundary passing through the Y-shape and interconnect the dots to form lines passing through the Y-shape, as shown in FIG. 20B. Also, other points, such as center points, may be used rather than skeletons. Processing circuit 102 may determine the intersection points of the lines passing through the skeleton, as illustrated by intersection point 135. For example, intersection point 135 may be common to all of the lines that together form the Y-shape.

[0131]

[0162] Processing circuitry 102 may repeat these operations for each of the Y-shapes in each of the sagittal cuts to determine a respective intersection point, such as intersection point 135. Processing circuitry 102 may determine a line that intersects each of the intersection points to determine initial horizontal axis 133B illustrated in Figures 21A and 21B.

[0132]

[0163] Processing circuitry 102 may determine a sagittal cut through the scapula using initial horizontal axis 133B. For example, FIGS. 21A and 21B are different perspective views of a sagittal cut through the scapula based on axis 133B. Processing circuitry 102 may repeat the operations described with reference to FIGS. 20A-20C to determine multiple intersection points for the sagittal cuts shown in FIGS. 21A and 21B. For example, based on the sagittal cuts using axis 133B, similar to FIGS. 20A-20C, each sagittal cut may form a Y-shape, similar to FIG. 20A. Processing circuitry 102 may determine a skeleton line passing through the Y-shape, similar to FIG. 20B, and determine the intersection points, similar to FIG. 20C. Processing circuitry 102 may repeat these operations for each of the Y-shapes in each of the sagittal cuts to determine the respective intersection points, similar to the description above with respect to FIGS. 20A-20C. Processing circuitry 102 may determine a line that intersects with each of the multiple intersection points to determine horizontal axis 134 illustrated in FIG.

[0133]

[0164] Figure 22 is a conceptual diagram illustrating the transverse axis of Figure 18 through the scapula and the normal of Figures 16 and 17 to a plane through the scapula to determine the patient's coordinate system. For example, Figure 22 illustrates the scapula 118 and the glenoid center 120. Figure 22 also illustrates a vector 132 perpendicular to the plane 130, illustrating the transverse axis 134.

[0134]

[0165] For example, as described above, an initial step for pre-diseased or diseased pre-implant characterization is to determine the coordinate axes along which the segmented object will be aligned to shape model 106. Using the exemplary techniques described above, processing circuit 102 may determine the x-axis (e.g., vector 132) and the z-axis (e.g., horizontal axis 134). Processing circuit 102 may further determine the y-axis using the following technique: Once processing circuit 102 has determined a coordinate system to represent and define the location of the anatomical object determined from the segmentation of the image data of scan 108, processing circuit 102 may be able to align the segmented object to the coordinate system of shape model 106.

[0135]

[0166] 23 is a conceptual diagram illustrating the movement (or extension) of the transverse axis through the scapula and the normal to the plane through the scapula relative to the glenoid center location to determine the patient coordinate system. For example, FIG. 8 illustrates a patient coordinate system centered at the glenoid center 120. As shown, vector 132 forms the x-axis of the patient coordinate system, as determined above, transverse axis 134 forms the z-axis of the patient coordinate system, and axis 136 forms the y-axis of the patient coordinate system.

[0136]

[0167] The processing circuit 102 may determine the glenoid center based on multiple 2D and 3D motions. For example, the processing circuit 102 may determine the center of gravity of the glenoid cavity and project the center of gravity onto the glenoid cavity surface to determine the glenoid center 120.

[0137]

[0168] Processing circuitry 102 may then determine y-axis 136 based on x-axis 132 and z-axis 134. For example, processing circuitry 102 may determine y' = z * x, where * is a vector product. y' is perpendicular to the plane defined by x-axis 132 and z-axis 134, but x-axis 132 and z-axis 134 are not necessarily perfectly orthogonal. Thus, processing circuitry 102 may replace z-axis 134 by Z = x * y' or x-axis 132 by X = y' * z to have an orthogonal system (x, y', Z) or (X, y', z). In some examples, processing circuitry 102 may utilize Z = x * y' because calculating x-axis 132 (e.g., scapula normal) is more robust than calculating z-axis 134 (e.g., horizontal axis).

[0138]

[0169] In this manner, processing circuitry 102 may determine a patient coordinate system. In some examples, such as those described above, processing circuitry 102 may determine the patient coordinate system without relying on particular landmarks, such as the scapular triangle 122 and the subscapular angle 124, which may not be present in the segmentation of anatomical objects from the image data of scan 108 due to over- or under-segmentation. In particular, for this exemplary technique, processing circuitry 102 may determine the x-, y-, and z-axes without relying on particular anatomical objects, but rather may determine the x-, y-, and z-axes based on the scapula 118 itself, which should be present in the image data of scan 108, but the scapular triangle 122 and the subscapular angle 124 may not be present in the image data of scan 108 due to under- or over-segmentation.

[0139]

[0170] The various techniques described above for determining a patient coordinate system may be used, for example, for patients who do not yet have an implant installed, or for patients who have a hemi-implant installed and therefore do not have a glenohumeral implant component. For patients with anatomical or reverse-type implants, the present disclosure proposes additional techniques for determining a patient coordinate system. These additional techniques were described above with respect to the feature extraction unit 410 and FIGS. 6, 7A, and 7B. These additional techniques may be used either separately from or in conjunction with the above-described techniques for determining a patient coordinate system.

[0140]

[0171] In some cases for patients with implants, most commonly anatomical or reverse implants, processing circuitry 102 may not be able to determine the x-, y-, and z-axes using the techniques described above with respect to the transverse axis and scapular plane. Specifically, the transverse axis and the plane through the scapula may not be easily detectable in scan 108 due to the presence of the implant. For patients with implants, processing circuitry 102 may determine the x-, y-, and z-axes based on features of the implant, as determined by feature extraction unit 208 or feature extraction unit 410 described above with reference to FIGS. 6, 7A, and 7B.

[0141]

[0172] After determining the patient coordinate system either using the transverse axis and scapular plane or using the implant features, the processing circuit 102 may determine a transformation matrix to align the patient coordinate system to the shape model 106. The alignment process is generally performed in the same manner regardless of the technique used to determine the patient coordinate system. As explained above, one exemplary way to determine the transformation matrix is ​​to use a coordinate system where c = (x, y, z, o), where x, y, and z are orthogonal basis 3D vectors and o is the origin (c is a 4x4 homogeneous matrix with the last column being (0,0,0,1)). The new coordinate system to which the segmented object will be transformed is C = (X,Y,Z,O). In this example, the transformation matrix is ​​then T = ĉ-1 × C.

[0142]

[0173] Processing circuit 102 may multiply coordinates defining the segmentation object (e.g., where coordinates are determined from axes using the techniques described above) based on image data from scan 108 with a transformation matrix to align the segmentation object to shape model 106 (e.g., SSM retrieved from memory 104). The result of this alignment may be an initial shape. For example, FIG. 24 illustrates an initial aligned shape 138 aligned with shape model 106. Initial aligned shape 138 is shown in FIG. 24 as being superimposed on top of shape model 106. Initial aligned shape 138 is generated based on image data from scan 108 in which some portions of anatomical structures may be missing (e.g., due to under-segmentation) or distorted (e.g., due to over-segmentation). For example, initial aligned shape 138 may be generally positioned to have the same orientation on the x-axis, y-axis, and z-axis as shape model 106.

[0143]

[0174] However, following the transformation, the initial aligned shape 138 may not be perfectly aligned with the shape model 106. In cases where the scapula is clipped too inferiorly and / or medially (which may not be known but is true in the initial shape 138), there may be calculation errors in determining the scapula body normal (e.g., vector 132) and the transverse axis 134. These calculation errors may result in misalignment between the shape model 106 and the patient anatomy (e.g., scapula 118), and thus the initial aligned shape 138 may not be as perfectly aligned with the shape model 106 as desired. For example, as can be seen in FIG. 24 , the initial aligned shape 138 is rotated along the z-axis (e.g., transverse axis) 134.

[0144]

[0175] In one or more examples, because it may be unclear whether there is misalignment from an over-cut scapula 118, processing circuitry 102 may modify parameters of initial shape 138 to generate an aligned shape. The aligned shape is substantially proximal (e.g., in terms of location, size, and orientation) to shape model 106. As an example, to modify the parameters, processing circuitry 102 may iteratively adjust coordinates of initial aligned shape 138 such that the initial aligned shape model is rotated along the z-axis (e.g., points of initial aligned shape 138 are rotated along the z-axis). During each adjustment, processing circuitry 102 may determine the distance between initial aligned shape 138 and shape model 106 (e.g., points in the initial aligned shape and points represented in shape model 106).

[0145]

[0176] For example, processing circuitry 102 may determine the distance between points on initial aligned shape 138 (e.g., points on the acromion and coracoid) and corresponding points on shape model 106 (e.g., the acromion and coracoid on shape model 106). Corresponding points refer to points in initial aligned shape 138 and points in shape model 106 that identify the same patient anatomical structure. Processing circuitry 102 may continue to rotate initial aligned shape 138 until the distance between initial aligned shape 138 and shape model 106 about z-axis 134 meets a threshold (e.g., less than a threshold including where the distance is minimized). Processing circuitry 102 may then rotate initial aligned shape 138 along y-axis 138 until the point difference between initial aligned shape 138 and corresponding points in shape model 106 about the y-axis meets a threshold (e.g., less than a threshold including where the distance is minimized). The processing circuit 102 may rotate the initial aligned shape model about the x-axis 132 until the point difference between the initial aligned shape 138 about the x-axis 132 and the corresponding point in the shape model 106 meets a threshold (e.g., is less than a threshold, including when the distance is minimized).

[0146]

[0177] As an example, processing circuit 102 may modify parameters of initial aligned shape 138 such that initial aligned shape 138 is rotated 5 degrees along an axis within a search range of [-45 degrees, 45 degrees] or [-27 degrees, 27 degrees] (e.g., a first instance of the initial aligned shape model). Processing circuit 102 may determine a distance between a point of the first instance of initial aligned shape 138 and a corresponding point of shape model 106. Assuming the distance is not minimum or is less than a threshold, processing circuit 102 may then modify parameters of initial aligned shape 138 such that initial aligned shape 138 is rotated 10 degrees along an axis (e.g., a second instance of initial aligned shape 138) and determine a distance (e.g., between a point of the second instance of initial aligned shape 138 and a corresponding point of shape model 106). Processing circuit 102 may repeat these operations for each of the axes for each instance of initial aligned shape 138. Processing circuit 102 may continue to repeat these operations until processing circuit 102 determines an instance of initial aligned shape 138 that resulted in a distance less than a threshold (e.g., a minimum distance). The instance of initial aligned shape 138 that resulted in a distance less than the threshold between various iterative instances (including an example of a minimum distance) is selected as aligned shape 140, illustrated in FIG.

[0147]

[0178] The result of these operations may be an aligned shape (e.g., a model rotated along the x-axis, y-axis, and z-axis). For example, FIG. 25 illustrates an aligned shape 140 aligned to the shape model 106. The aligned shape 140 is shown superimposed on the shape model 106 in FIG. 25. The aligned shape 140 (e.g., simply the aligned shape 140) provides a better alignment to the shape model 106 compared to the initial aligned shape 138. For example, as shown in FIG. 25, the aligned shape 140 is better aligned to the shape model 106 than the initial aligned shape 138 for the shape model 106 shown in FIG. 24.

[0148]

[0179] For example, the initial aligned shape 138 and the shape model 106 may be aligned globally. However, due to over- or under-segmentation of image data of the anatomical object in the scan 108, there may be some tilt or misorientation between the initial aligned shape 138 and the shape model 106. To address this, the processing circuit 102 may separately rotate the initial aligned shape 138 about each axis (x, y, z) until the distance between the initial aligned shape 138 and the shape model 106 meets (e.g., is minimized) a threshold. Thus, by iteratively rotating the initial aligned shape 138, the processing circuit 102 may generate an aligned shape 140 (also referred to as an intermediate aligned shape 140). The distance between points on the aligned shape 140 and corresponding points of the shape model 106 may be minimized so that the aligned shape 140 (e.g., the intermediate aligned shape 140) is substantially within the coordinate system of the shape model 106.

[0149]

[0180] In some examples, the shape model 106 is deformed to the shape of the aligned shape 140 to generate pre-diseased or diseased pre-implantation characteristics of the patient anatomy, so the aligned shape 140 may be referred to as an intermediate aligned shape. The processing circuitry 102 may utilize techniques described below to deform the shape model 106 to register the shape model 106 to the aligned shape 140. However, in some examples, the processing circuitry 102 may utilize some other technique to deform the shape model 106 to register it to the aligned shape 140. Also, in the case of the exemplary technique for registering the shape model 106 to the aligned shape 140, such techniques may be implemented when the aligned shape 140 is generated using techniques other than those described above.

[0150]

[0181] As described above, the processing circuit 102 may be configured to register the shape model 106 to the aligned shape 140. Registering the shape model 106 to the aligned shape 140 may refer to iteratively deforming the shape model 106 until a cost function value falls below (e.g., is minimized) a threshold. The registration algorithm is a global loop that includes two inner iterative loops called an iterative closest point (ICP) loop and an elastic registration (ER) loop. There may be other exemplary ways of implementing the registration algorithm, and the use of two loops within a global loop is just one exemplary way of implementing the registration algorithm. For example, in some examples, it may be possible to avoid the ICP loop and implement only the ER loop. In some examples, it may be possible to implement only the ICP loop and avoid the ER loop.

[0151]

[0182] In the ICP loop for the first iteration of the global loop, the initial inputs are the aligned shape 140 and the shape model 106. For the ICP loop in the first iteration of the global loop, processing circuitry 102 iteratively modifies the aligned shape 140 based on a comparison of the aligned shape 140 with the shape model 106 to generate a primary shape. For example, for each iteration through the ICP loop, processing circuitry 102 determines a cost value, and the modified aligned shape that results in a cost value less than (e.g., minimized) a threshold is the output of the ICP loop and is referred to as the primary shape. An example of an ICP algorithm is described in more detail below.

[0152]

[0183] The primary shape is an input to the ER loop for the first iteration of the global loop. Another input to the ER loop for the first iteration of the global loop is the shape model 106. For the ER loop in the first iteration of the global loop, the processing circuit 102 may deform the shape model 106 to generate multiple estimated shape models (e.g., one for each pass through the ER loop). For each iteration of the ER loop, the processing circuit 102 may determine a total cost value and determine which iteration of the ER loop utilized an estimated shape model with a total cost value that meets a threshold (e.g., less than a threshold, including when minimized). The result of the ER loop (e.g., an estimated shape model with a total cost value that meets a threshold) is the primary registered shape model. This completes one iteration of the global loop. For the first iteration of the global loop, the total cost value may be based on the distance and orientation between the estimated shape model and the primary shape, constraints on medialization of anatomical structures, and parameter weightings.

[0153]

[0184] For example, assume that S1 is a primary shape model that processing circuit 102 may determine using an ER loop in a first iteration of a global loop. For the ER loop, processing circuit 102 may generate S11, S12, S13, etc., where each one of S11, S12, S13, etc. is a transformed version of shape model 106, and each of S11, S12, S13, etc. is an example of an estimated shape model. For each one of S11, S12, S13, etc., processing circuit 102 may determine a total cost value (e.g., S11_totalcostvalue, S12_totalcostvalue, S13_totalcostvalue, etc.). Processing circuit 102 may determine which of the total cost values ​​is less than (e.g., minimized) a threshold value. The estimated shape model (e.g., one of S11, S12, S13, etc.) that has a total cost value below (e.g., minimized) the threshold is S1 (e.g., the first registered shape model), after which the first iteration of the global loop is completed.

[0154]

[0185] For the second iteration of the global loop, processing circuitry 102 performs the operations of an ICP loop. In the second iteration of the global loop, for the ICP loop, one input is the primary shape model generated from the ER loop, and the other input is the primary shape generated by the previous ICP loop. In the second iteration of the global loop, for the ICP loop, processing circuitry 102 iteratively modifies the primary shape based on a comparison of the primary shape with the primary shape model. For example, for each iteration through the ICP loop, processing circuitry 102 determines a cost value, and the modified primary shape that results in a cost value less than (e.g., minimized) a threshold is the output of the ICP loop and is referred to as the secondary shape.

[0155]

[0186] The secondary shape is the input to the ER loop for the second iteration of the global loop. Another input to the ER loop for the second iteration of the global loop is the shape model 106 and / or the primary shape model. If the shape model 106 is used, then iThe value of b can vary within [-b +b]. If a first-order shape model is used, b i The value of b may vary within [-b+b_prev b+b_prev], where b_prev is the value found in the previous iteration (eg, used to determine the primary shape model).

[0156]

[0187] For the ER loop in the second iteration of the global loop, the processing circuit 102 may deform the shape model 106 and / or the primary shape model to generate multiple estimated shape models (e.g., one for each iteration of the ER loop). For each iteration of the ER loop, the processing circuit 102 may determine a total cost value and determine which iteration of the ER loop utilized an estimated shape model with a total cost value that satisfies (e.g., is minimized) a threshold. The result of the ER loop in the second iteration of the global loop (e.g., an estimated shape model with a total cost value that satisfies a threshold) is a secondary shape model. This completes the second iteration of the global loop. The total cost value for the second iteration of the global loop may be based on the distance and orientation between the estimated shape model and the secondary shape, constraints regarding anatomical structure medialization, and parameter weighting.

[0157]

[0188] For example, assume that S2 is a second shape model that processing circuit 102 may determine using an ER loop in the second iteration of the global loop. For the ER loop, processing circuit 102 may generate S21, S22, S23, etc., where each one of S21, S22, S23, etc. is a transformed version of shape model 106 and / or the first shape model, and S21, S22, S23, etc. is each an example of an estimated shape model. For each one of S21, S22, S23, etc., processing circuit 102 may determine a total cost value (e.g., S21_totalcostvalue, S22_totalcostvalue, S23_totalcostvalue, etc.). Processing circuit 102 may determine which of the total cost values ​​is less than (e.g., minimized) a threshold value. The estimated shape model (e.g., one of S21, S22, S23, etc.) that has a total cost value below (e.g., minimized) the threshold is S2 (e.g., the second registered shape model), after which the second iteration of the global loop is completed.

[0158]

[0189] In the above example, processing circuit 102 generated S11, S12, S13, etc. (e.g., a first set of estimated shape models) for the first iteration of the global loop, and S21, S22, S23, etc. (e.g., a second set of estimated shape models) for the second iteration of the global loop. In some examples, the number of estimated shape models in the first and second sets of estimated shape models may be the same. In some examples, the number of estimated shape models in the first and second sets of estimated shape models may be different (e.g., 6 estimated shape models in the first set of estimated shape models and 14 estimated shape models in the second set of estimated shape models). One reason for having different numbers of estimated shape models in different iterations of the global loop is that by increasing the number of estimated shape models in subsequent iterations of the global loop, it may be possible to more accurately determine the global minimum compared to when the same or fewer number of estimated shape models are used. That is, processing circuitry 102 may gradually increase the number of unknowns (eg, modes and scale factors) used to generate the estimated shape model through multiple ER loops.

[0159]

[0190] This process continues iteratively until the total cost value falls below (e.g., is minimized) a threshold. The registered shape model (e.g., the Nth registered shape model) that provides a total cost value below (e.g., minimized) the threshold provides a pre-diseased or affected pre-implant characterization of the patient anatomy. Some additional processing may be required to convert the pre-diseased or affected pre-implant characterization back to the patient coordinate system.

[0160]

[0191] As described above, to determine pre-diseased or affected pre-implantation characterization, there is a global loop including an ICP loop and an ER loop. The following describes an example of an ICP loop. In the ICP loop, there is a source point cloud and a target point cloud. The target point cloud remains fixed, and the processing circuit 102 or the projection unit 212 modifies (e.g., transforms) the source point cloud so that the transformed point cloud matches the target point cloud. The difference between the transformed point cloud and the target point cloud indicates how well the transformed point cloud matches the source cloud. In one or more examples, the processing circuit 102 or the projection unit 212 continues to transform the source point cloud until the difference falls below a threshold. For example, the processing circuit 102 or the projection unit 212 continues to transform the source point cloud until the difference is minimized.

[0161]

[0192] In one or more examples, in the first iteration of the global loop, for an ICP loop, the target point cloud is the shape model 106 and the source point cloud is the aligned shape 140. The processing circuit 102 or projection unit 212 may determine the distance between points on the aligned shape 140 and corresponding points on the shape model 106. The points on the aligned shape 140 used may be points known to be non-pathological (e.g., the medial glenoid vault, acromion, and coracoid, as a few non-limiting examples). Other examples include various other points on the scapula present in the image data of the scan 108. The processing circuit 102 or projection unit 212 may find points relating to the same anatomical structures on the shape model 106.

[0162]

[0193] 26 is a conceptual diagram illustrating an example for determining a disparity value for the ICP algorithm. For example, processing circuitry 102 or projection unit 212 may determine a point (p) on the target point cloud (e.g., shape model 106). Processing circuitry 102 or projection unit 212 may then determine the closest point to point p on aligned shape 140, which in the example of FIG. 26 is point p pp The processing circuit 102 or the projection unit 212 is shown in FIG. ct A point p on the aligned shape 140, such as pp A set number of points (e.g., 10 points) that are close to may be determined.

[0163]

[0194] For point p on shape model 106 and each of these points (e.g., the closest and neighboring points on aligned shape 140), processing circuitry 102 or projection unit 212 may determine a normal vector. For example, vector n is the normal vector to point p, and vector n is the normal vector to point p. pp is the normal vector to the point p ct is the normal vector to the point. One way to determine the normal vector is based on a vector perpendicular to the tangent plane to the point. Another way to calculate the normal of a point is to calculate the normal of each triangle that shares the point and then impute an average normal. To overcome local noise, the normal is smoothed by calculating the average normal of the points within a certain neighborhood.

[0164]

[0195] The processing circuitry 102 or the projection unit 212 may determine the orientation and distance difference between a point p on the shape model 106 and a point on the aligned shape 140 based on the following equations:

[0165]

number

[0166]

[0196] In the above equation, ps is the source point (e.g., point p on the shape model 106), pt is the point on the aligned shape 140 (e.g., point p pp or p ct where "w" is the closest point and nearest point such as ns (e.g., ns shown in FIG. 26), nt is the normal vector of the point on the aligned shape 140 (e.g., npp and nj), and w is a pre-programmed weighting factor. Typically, w is equal to 1 to give equal weight between distance and orientation. In some examples, high / low curvature regions may require higher / lower values ​​of "w". The value of "w" may be defined empirically.

[0167]

[0197] The processing circuit 102 or the projection unit 212 may determine which of the difference values ​​resulted in the smallest difference value. For example, the first difference value may be p, ns, p pp , and n pp Based on this, the second difference value is p, ns, p ct , and n j In this example, the second difference value may be smaller than the first difference value. The processing circuit 102 or the projection unit 212 may project p on the aligned shape 140 as a point corresponding to point p on the shape model 106. ct may be determined.

[0168]

[0198] 26 illustrates one point p on shape model 106, there may be multiple (e.g., N) points on shape model 106, and processing circuit 102 or projection unit 212 may perform operations similar to those described above to identify N corresponding points on aligned shape 140 for each of the N points on shape model 106. Thus, there may be N difference values. Processing circuit 102 or projection unit 212 may sum the N difference values ​​and divide the result by N. If the resulting value is greater than a threshold (including, for example, examples where the resulting value is not minimized), processing circuit 102 or projection unit 212 may continue the ICP loop.

[0169]

[0199] With the N points on the shape model 106 and the N points on the aligned shape, the processing circuit 102 or the projection unit 212 may determine a rotation matrix R and a translation vector t. Based on the rotation matrix and the translation vector, the processing circuit 102 or the projection unit 212 may rotate and translate the points on the aligned shape 140 (e.g., −R multiplied by (points on the aligned shape) plus the translation vector). The result is a first intermediate primary shape. One exemplary manner of generating the rotation matrix R and the translation vector t is described in Berthold K. P. Horn (1987), “Closed-form solution of absolute orientation using unit quaternions,” https: / / pdfs.semanticscholar.org / 3120 / a0e44d325c477397afcf94ea7f285a29684a.pdf.

[0170]

[0200] This may terminate one instance of the ICP loop. The processing circuit 102 or the projection unit 212 may then repeat these operations, where the processing circuit 102 or the projection unit 212 uses the first intermediate primary shape and the shape model 106 instead of the aligned shape 140. The processing circuit 102 or the projection unit 212 may determine N difference values ​​for each of the N points on the shape model 106 and the N points on the intermediate primary shape. The processing circuit 102 or the projection unit 212 may sum the N difference values ​​and divide by N. If the resulting value is greater than (or not minimized by) a threshold, the processing circuit 102 or the projection unit 212 may determine a rotation matrix and a translation vector and determine a second intermediate primary shape. This may terminate a second iteration through the ICP loop.

[0171]

[0201] Processing circuit 102 or projection unit 212 may continue to repeat these operations until processing circuit 102 or projection unit 212 determines the value that results from the sum of N difference values ​​between N points on shape model 106 and N points on the Xth intermediate primary shape divided by N when the value meets a threshold (e.g., when the value is minimized). In this case, the ICP loop is terminated and processing circuit 102 or projection unit 212 determines that the Xth intermediate primary shape is the primary shape.

[0172]

[0202] The primary shape becomes an input to an elastic registration (ER) loop. Another input in the ER loop is the shape model 106. In the ER loop, the processing circuitry 102 or the projection unit 212 may determine multiple estimated shape models based on the shape model 106. For example, the processing circuitry 102 or the projection unit 212 may generate a new shape model (s i ) can be determined.

[0173]

number

[0174]

[0203] In the above equation, s′ is the shape model 106 (e.g., a point cloud of an average shape, as an example, where the point cloud defines the coordinates of points in the shape model 106, such as the vertices of the primitives that form the shape model 106). i is the eigenvalue, v i are the eigenvectors of the covariance matrix (also called the modes of variation). The covariance matrix describes the variance within a dataset. The element at position i,j is the covariance between the ith and jth elements of the dataset array.

[0175]

[0204] The processing circuitry 102 or the projection unit 212 i By selecting different values ​​of b, multiple estimated shape models can be determined. iis a scaling factor that scales an eigenvalue or an eigenvector. The eigenvalue (λ i ), and the eigenvector (v i ) are known from the generation of the shape model 106 (e.g., s’). For the sake of simplicity of the figure, the processing circuit 102 or the projection unit 212 may determine ten estimated shape models based on ten selected (e.g., randomly or pre-programmed) values of b i . There may be more or fewer estimated shape models than ten.

[0176]

[0205] In the ER loop, the processing circuit 102 or the projection unit 212 may perform the following operations on the estimated shape models. As part of the ER loop, the processing circuit 102 or the projection unit 212 may determine which of the estimated shape models yields a cost function value that is below a threshold (e.g., minimized). The cost function value may be based on three sub-cost function values, but more or fewer than three sub-cost function values are also possible. For example, assume that the first sub-cost function value is Cf1, the second sub-cost function value is Cf2, and the third sub-cost function value is Cf3. In some examples, the cost function value (Cf) is equal to Cf1 + Cf2 + Cf3. In some examples, weights may be applied such that Cf = w1*Cf1 + w2*Cf2 + w3*Cf3, where 0 < wi < 1 and 0 ≤ Cfi ≤ 1. The weights (wi) may be pre-programmed. To complete the ER loop, the processing circuit 102 or the projection unit 212 may determine which of the estimated shape models yields a Cf value that meets the threshold (e.g., less than the threshold including being minimized).

[0177]

[0206] The first sub-cost function value (Cf1) is based on the distance and orientation between the estimated shape model (e.g., generated based on different values for bi) and the primary shape generated by the ICP loop. For example, for each of the estimated shape models, the processing circuit 102 or the projection unit 212 may determine the Cf1 value. The formula for Cf1 may be the same as the formula used in the ICP loop.

[0178]

[0207] For example, Cf1=Σnorm(p s -p t ) 2 +w*norm(n s -n t ) 2 / N. However, in this case, p s and n s are for the points and vectors of those points on each of the estimated shape models, and p t and n t are for the points and vectors of those points on the primary shape. N refers to the number of points on the primary shape and on each of the estimated shape models.

[0179]

[0208] Processing circuit 102 or projection unit 212 may determine the value of Cf1 using the example techniques described above with respect to the ICP loop. For example, for a first estimated shape model, processing circuit 102 or projection unit 212 may determine a value of Cf1 (e.g., the first Cf1), and for a second estimated shape model, processing circuit 102 or projection unit 212 may determine a value of Cf1 (e.g., the second Cf1), and so on. In this case, processing circuit 102 or projection unit 212 may not translate or rotate any of the estimated shape models but may select one of the estimated shape models using the calculation of Cf1 for each of the estimated shape models.

[0180]

[0209] The value of Cf1 is one of the sub-cost function values ​​used to determine the cost function value of the ER loop. In some examples, it may be possible to determine an estimated shape model that minimizes the Cf1 value and terminate the ER loop. However, simply minimizing the difference (e.g., point distance) between the output of the ICP loop and the estimated shape generated from the shape model 106 may not be sufficient to determine a pre- or post-implantation characterization of the patient's anatomy. For example, there may be logical constraints regarding the location of the shape generated by the ICP loop (e.g., the primary shape after the initial termination of the ICP loop) relative to the patient's body, and minimizing the difference (e.g., distance) between the primary shape and the estimated shape may violate such logical constraints in some cases. For example, the sub-cost function value Cf1 may be non-convex in some cases and thus result in a false local minimum. With additional sub-cost function values, the processing circuit 102 or the projection unit 212 may correct for non-convexity of the total cost function value Cf.

[0181]

[0210] For example, when predicting pre-diseased or diseased pre-implantation characterization of a glenoid, the glenoid of the estimated shape model should not be more medial than the current patient glenoid. In this disclosure, medialized or medial means toward the center of the patient's body. When a patient suffers injury or disease in the glenoid, bone erosion may cause the pathological glenoid to be more medial than before the injury or disease (e.g., shifted closer to the center of the patient's body). Therefore, a glenoid in one instance of an estimated shape model that is more medial than the current glenoid is likely not an adequate estimation of the pre-diseased or diseased pre-implantation patient anatomy. Also, injury or disease may have caused the glenoid to be more medial; therefore, if one of the estimated shape models includes a glenoid that is more medial than the current location of the glenoid, the instance of the estimated shape model may not have the pre-diseased or diseased pre-implantation characteristics of the patient anatomy.

[0182]

[0211] That is, the first estimated shape model generated from shape model 106 is assumed to minimize the value of Cf1 based on the distance and orientation between the first estimated shape model and the primary shape generated by the ICP loop. In this example, if the glenoid of the first estimated shape model is more medial than the current location of the glenoid, the first estimated shape model may not be an appropriate or best estimate of the pre-diseased or diseased pre-implantation shape of the pathological anatomy.

[0183]

[0212] To ensure that medialized instances of the estimated shape model are not used to determine pre-diseased or diseased pre-implantation characterizations, processing circuit 102 may determine a value of Cf2. The value of Cf2 indicates whether the estimated shape model is more medial than the patient anatomy or not. That is, a second exemplary sub-cost function value is Cf2. The value of Cf2 is a measure of the constraint on medialization of the anatomy.

[0184]

[0213] To determine the value of Cf2, processing circuitry 102 determines a threshold point (p th ) can be determined. th represents the threshold of glenoid medialization that must be crossed by an instance of the intermediate shape model used to determine the premorbid characterization. th 1 illustrates an exemplary manner in which the image data from the scan 108 of the glenoid may be divided into four quadrants of interest (e.g., superior, posterior, inferior, and anterior).

[0185]

[0214] 27A and 27B are conceptual diagrams illustrating a portion of a glenoid for determining parameters of a cost function used to determine a pre-diseased or diseased pre-implant shape of a patient's anatomy. As will be explained in more detail, in conjunction with the illustrated portion of the glenoid in FIGS. 27A and 27B, processing circuitry 102 calculates the parameters of a cost function used to determine the pre-diseased or diseased pre-implant shape of a patient's anatomy. thProcessing circuitry 102 may also determine an internalization value for each of the estimated shape models (e.g., generated based on the eigenvalues ​​and eigenvectors described above from shape model 106 using the values ​​of b i ). th Based on the value of and the internalization value, processing circuit 102 may determine a sub-cost value for Cf2.

[0186]

[0215] FIG. 27A illustrates a transverse axis 134 (e.g., as described above) that divides the glenoid surface into an anterior side 142 and a posterior side 144. FIG. 27B illustrates a transverse axis 134 that divides the glenoid surface into a superior side 150 and an inferior side 152. For example, returning to FIG. 18 , processing circuitry 102 has determined transverse axis 134. Processing circuitry 102 may then determine anterior side 142, posterior side 144, superior side 150, and inferior side 152 based on an axial or sagittal cut through scapula 118. The result may be the example illustrated in FIGS. 27A and 27B.

[0187]

[0216] Processing circuitry 102 may project points on the glenoid surface for each of the segments (e.g., the anterior and posterior sides in FIG. 27A and the superior and inferior sides in FIG. 27B) onto the horizontal axis 134 and determine the center of mass of the projected points for each segment. The projection of a point onto a line is the closest point on that line. The line and vector defined by the point and its projection form a right angle. This can be calculated by choosing a random point on the line that forms the hypotenuse with the point to be projected. Using trigonometry, the projection is the hypotenuse length multiplied by the cosine of the angle defined by the hypotenuse and the line. The center of mass is the average point of the projected points on that line and may also be thought of as the center of gravity.

[0188]

[0217] For example, point 146 in FIG. 27A is an example of a center of mass of a projection point for anterior side 142. Point 148 in FIG. 27A is an example of a center of mass of a projection point for posterior side 144. Point 154 in FIG. 27B is an example of a center of mass of a projection point for superior side 150. Point 156 in FIG. 27B is an example of a center of mass of a projection point for inferior side 152.

[0189]

[0218] In one or more examples, processing circuit 102 may determine the outermost centroid point (e.g., the point farthest from the center of the patient's body) from points 146, 148, 154, and 156. Processing circuit 102 may compare the outermost centroid point with a threshold point p th The processing circuitry 102 may also set the outermost quadrant as Q th In some instances, the outermost centroid point does not necessarily lie in the outermost quadrant. For example, point 156 may be the outermost centroid point (e.g., the threshold point p th ) in this example, we assume that the outermost quadrant (Q th ) may be on the lower side 152 if the outermost centroid point was in the outermost quadrant. However, the outermost centroid point may not be in the outermost quadrant.

[0190]

[0219] The processing circuit 102 th Processing circuitry 102 may determine a quadrant within the instance of the estimated shape model that corresponds to the quadrant (e.g., the outermost quadrant). Processing circuitry 102 may project points of the quadrant of the instance of the estimated shape model onto horizontal axis 134. For example, for each of the estimated shape models, processing circuitry 102 may determine points such as points 146, 148, 154, and 156. However, due to different interiorizations of the estimated shape models (e.g., due to different values ​​of b i used to generate the estimated shape models), each projected point may be at a different location on horizontal axis 134.

[0191]

[0220] For example, processing circuitry 102 may determine an anchor point on horizontal axis 134. This anchor point is at the same location on horizontal axis 134 for each of the estimated shape models. Processing circuitry 102 may determine the distance of the projected point to the anchor point. In some examples, p th may be an anchor point.

[0192]

[0221] The processing circuit 102 calculates the projection point p th The processing circuitry 102 may determine the distance to dmed The average distance of the projected points may be determined as a value equal to:

[0193]

[0222] d med If the value of d is zero or positive, it means that the instance of the estimated shape model is more medial than the current patient anatomy (e.g., glenoid and scapula) and therefore is not a good predictor of the pre-diseased or diseased pre-implantation characteristics of the patient anatomy. med If the value of is negative, it means that the instance of the estimated shape model is not more medial than the current patient anatomy and therefore may be a possible predictor of the pre-diseased or diseased pre-implantation characteristics of the patient anatomy.

[0194]

[0223] In some examples, the processing circuit 102 med may set Cf2 equal to a value based on med (It is a function of d.) For example, med The function used to calculate Cf2 based on d greater than zero med increasing function for values ​​of d less than zero med It may be a decreasing function of the value of d. med If d is greater than or equal to 0, processing circuitry 102 med Set Cf2 equal to d medWhen w1 is less than zero, processing circuit 102 may set Cf2 equal to 0. In this manner, if an instance of the estimated shape model is more medial than the current patient anatomy, the value of Cf2 is positive and the overall cost value of the cost function is increased. Also, the cost function (Cf) is equal to Cf1 plus Cf2 plus Cf3 (and optionally updated using weights w1, w2, and w3), and when Cf2 is positive, the value of Cf is increased compared to when the value of Cf2 is zero. Because processing circuit 102 has determined a cost value that meets a threshold (e.g., minimizes the cost function), having a positive value of Cf2 causes processing circuit 102 to avoid using an instance of the estimated shape model that is more medial than the current patient anatomy to determine pre-diseased or diseased pre-implantation characteristics.

[0195]

[0224] In some examples, the cost function value (Cf) may be based only on Cf1 and Cf2. For example, for an ER loop, processing circuit 102 or projection unit 212 may determine which estimated shape model results in minimizing Cf. However, in such cases, it may be possible for the estimated shape model to be a model with high variance (e.g., having a lower probability of representing pre-disease or diseased pre-implantation characteristics in the general population). Thus, in some examples, for an ER loop, processing circuit 102 may determine a parameter weight value (e.g., Cf3) that weights more likely estimated shape models as having a higher probability of being a representation of pre-disease or diseased pre-implantation anatomy and less likely estimated shape models as having a lower probability of being a representation of pre-disease or diseased pre-implantation anatomy.

[0196]

[0225] For example, in some instances, processing circuitry 102 may also determine a sub-cost function value Cf3 that is used to penalize more complex solutions in increasing variance of data error. For example, processing circuitry 102 may determine a gradient smoothing term using an eigenvalue that penalizes participation modes of variation according to the following equation:

[0197]

number

[0198]

[0226] In the above equation, N is the number of modes used to instantiate the estimated shape model. Cf3 may not be necessary in all instances. For example, in the equation for Cf3, there are larger numerical values ​​to sum when an instance of the estimated shape model has many modes compared to when the instance of the estimated shape model has fewer modes. Thus, the summation result (e.g., the value of Cf3) is larger for those estimated shape models with more modes than for those estimated shape models with fewer modes. Thus, larger values ​​of Cf3 cause the cost function value Cf to be larger than smaller values ​​of Cf3. Because processing circuitry 102 may be minimizing the value of Cf, estimated shape models with larger values ​​of Cf3 are less likely to be determined as the pre-diseased or diseased pre-implantation shape of the patient anatomy compared to estimated shape models with smaller values ​​of Cf3.

[0199]

[0227] Generally, Cf3 consists of two terms: a sparse or hard term and a fine or smoothing term. The eigenvalues ​​are the positive values ​​on the diagonal of the decomposition matrix. Their order reflects the occurrence of the corresponding eigenvector in the database. The first eigenvalue represents the most "important" or frequent variation in the database, while the last eigenvalue represents the least frequent (usually representing noise). In the sparse term of Cf3, processing circuit 102 may remove those eigenvectors that contribute to the least significant K% of the database variance (i.e., bi = 0 if lambda(i) > q, where vi for i ≤ q represents (100 - K)% of the database variance). In the fine term of Cf3, processing circuit 102 may gradually penalize higher eigenvectors. As a result, the algorithm avoids complex or noisy solutions. Generally, the term "smoothing" is used to refer to the addition of an adjustment term to the optimization method.

[0200]

[0228] Processing circuit 102 may determine the estimated shape model for which w1*Cf1+w2*Cf2+w3*Cf3 is less than (e.g., minimized from) a threshold value. This estimated shape model is the primary shape model for the first iteration through the global loop.

[0201]

[0229] For example, for the first iteration through the global loop, for the ER loop, assume that processing circuit 102 determines the following estimated shape model based on the shape model 106 and different values ​​of b i : For example, the first estimated shape model may be s 11 , where s 11 is

[0202]

number

[0203] where s' is the shape model 106, λ i are the eigenvalues ​​used to determine the shape model 106, v i are the eigenvectors used to determine the shape model 106. In this example, b iis a first weighting parameter. The processing circuit 102 may determine a second estimated shape model (e.g., s12), where s12 is

[0204]

number

[0205] where b12 i is a second weighting parameter. In this manner, processing circuitry 102 may determine multiple estimated shape models (e.g., s11, s12, s13, etc.).

[0206]

[0230] For each estimated shape model, processing circuitry 102 may determine values ​​of Cf1, Cf2, and Cf3. Also, not all of Cf1, Cf2, and Cf3 may be required. For example, processing circuitry 102 may determine s11_Cf1 based on estimated shape model s11 and the first shape generated by the ICP loop. Processing circuitry 102 may determine s12_Cf1 based on estimated shape model s12 and the first shape generated by the ICP loop, and so on. Processing circuitry 102 may determine s11_Cf2, s12_Cf3, etc., as described above with reference to the exemplary techniques for determining Cf2. Similarly, processing circuitry 102 may determine s11_Cf3, s12_Cf3, etc., as described above with reference to the exemplary techniques for determining Cf3.

[0207]

[0231] Processing circuit 102 may determine s11_Cf as w1*s11_Cf1+w2*s11_Cf2+w3*s11_Cf3 and s12_Cf as w1*s12_Cf1+w2*s12_Cf2+w3*s13_Cf3, and so on. The weights applied to Cf1, Cf2, and Cf3 may be different for each of s11, s12, s13, etc., or the weights may be the same. Processing circuit 102 may determine which one of s11_Cf, s12_Cf, s13_Cf, etc. is smallest (or possibly smaller than a threshold). Assume s12_Cf is smallest. In this example, processing circuit 102 may determine estimated shape model s12 as a result of an ER loop, and the result of the ER loop for the first iteration of the global loop is a first-order shape model.

[0208]

[0232] This may terminate the first iteration of the global loop. For example, in the first iteration of the global loop, the ICP loop generated a primary shape, and the ER loop generated a primary shape model. Then, for the second iteration of the global loop, the input to the ICP loop is the primary shape model generated from the previous ER loop and the primary shape generated from the previous ICP loop. In the ICP loop for the second iteration of the global loop, processing circuit 102 generates a secondary shape based on the primary shape model and the primary shape. The secondary shape is input to the ER loop in the second iteration of the global loop. Also, in the ER loop, in the second iteration of the global loop, processing circuit 102 may determine an estimated shape model (e.g., s21, s22, s23, etc.) based on shape model 106 and / or based on the primary shape model (e.g., s12 in this example). The output of the ER loop may be the secondary shape model, which may terminate the second iteration of the global loop.

[0209]

[0233] This process repeats until processing circuit 102 determines the instance of the estimated shape model that minimizes the Cf value. For example, assume that after the first iteration of the global loop, processing circuit 102 outputs estimated shape model s12 having a cost value of s12_Cf. After the second iteration of the global loop, processing circuit 102 outputs estimated shape model s23 having a cost value of s23_Cf. After the third iteration of the global loop, processing circuit 102 outputs estimated shape model s36 having a cost value of s36_Cf. In this example, processing circuit 102 may determine which one of s12_Cf, s23_Cf, or s36_Cf has the smallest value and determine the estimated shape associated with the smallest Cf value as the patient's pre-diseased or affected pre-implant anatomical structure. For example, assume that s23_Cf was the smallest. In this example, processing circuitry 102 may determine that estimated shape model s23 represents the patient's pre-diseased or diseased pre-implant characteristics.

[0210]

[0234] In the above example, processing circuit 102 may loop through the global loop until the value of Cf is minimized. However, in some examples, processing circuit 102 may be configured to loop through the global loop for a set number of iterations. Processing circuit 102 may then determine which of the estimated shape models resulted in the smallest value of Cf. In some examples, processing circuit 102 may loop through the global loop until the value of Cf falls below a threshold.

[0211]

[0235] As described above, the processing circuit 102 may determine the minimum of the cost function value Cf. Minimizing the value of Cf or determining the minimum Cf value from multiple Cf values ​​are two exemplary ways of satisfying the cost function. There may be other ways to satisfy the cost function. As an example, satisfying the cost function may mean that the cost value of the cost function is less than a threshold. It may also be possible to reconfigure the cost function so that satisfying the cost function means maximizing the cost function. For example, one of the coefficients in the cost function may be the distance between a point in the estimated shape model and a corresponding point in the output from the ICP loop. In some examples, the processing circuit 102 may minimize the distance between the estimated shape model and the output from the ICP loop and may generate a number that is inversely correlated with the distance (e.g., the closer the distance, the more numbers the processing circuit 102 generates). In this example, to satisfy the cost function, the processing circuit 102 may maximize the cost value that is inversely correlated with the distance. Thus, although examples are described with respect to minimizing as part of the global loop and the ICP loop, in some examples, to satisfy (e.g., optimize) the cost function, processing circuit 102 may maximize the value of Cf. Such techniques are contemplated by this disclosure. For example, a cost value satisfying a threshold means determining that the cost value is less than the threshold if a cost value less than the threshold indicates a pre-diseased or diseased pre-implantation shape, or greater than the threshold if a cost value greater than the threshold indicates a pre-diseased or diseased pre-implantation shape.

[0212]

[0236] FIG. 28 is a flowchart illustrating an exemplary method of operation of a computing device in accordance with one or more exemplary techniques described in this disclosure. The technique of FIG. 28 is described with reference to device 100 of FIG. 1 but is not limited to any particular type of computing device. Computing device 100 acquires image data of a patient's joint including an implant, the image data comprising a 3D model of the joint with the implant (2802). Computing device 100 identifies a first object within the 3D model corresponding to a first component of the implant (2804). Computing device 100 determines a first vector based on the first object (2806). Computing device 100 identifies a second object within the 3D model corresponding to a second component of the implant (2808). Computing device 100 determines a second vector based on the second object (2810). Computing device 100 determines a third vector that is perpendicular to a plane defined by the first and second vectors (2812). The computing device 100 determines (2814) a patient coordinate system based on the first vector, the second vector, and the third vector. The computing device 100 identifies (2816) an anatomical object within the 3D model. The computing device 100 generates (2818) an initial aligned shape that initially aligns the anatomical object from the image data to the shape model based on the determined patient coordinate system. The computing device 100 generates (2820) information indicative of the diseased pre-implantation shape of the anatomical object based on the initial aligned shape.

[0213]

[0237] FIG. 29 is a flowchart illustrating an exemplary method of operation of a computing device in accordance with one or more exemplary techniques described in this disclosure. The technique of FIG. 29 is described with reference to device 100 of FIG. 1 but is not limited to any particular type of computing device. Computing device 100 determines a shape model for an affected anatomical object (2902). Computing device 100 acquires image data of a patient's joint including an implant, the image data including a 3D model of the joint with the implant (2904). Computing device 100 identifies voxels in the 3D model of the joint with the implant that correspond to bones (2906). Computing device 100 determines an initial shape estimate for the bone based on the voxels identified as corresponding to bones (2908). Computing device 100 aligns the initial shape estimate to the shape model to form an aligned initial shape estimate (2910). The computing device 100 deforms 2912 the shape model based on the aligned initial shape estimate to generate a pre-implantation affected approximation of the bone.

[0214]

[0238] 30 is a flowchart illustrating an exemplary method of operation of a computing device in accordance with one or more exemplary techniques described in this disclosure. The technique of FIG. 30 is described with reference to device 100 of FIG. 1, but is not limited to any particular type of computing device. Computing device 100 acquires image data of a patient's joint (3002). Computing device 100 determines that the joint includes an existing implant (3004). Computing device 100 segments the image data of the joint (3006). Computing device 100 generates a type identification for the existing implant (3008).

[0215]

[0239] 31 is a flowchart illustrating an example method of operation of a computing device in accordance with one or more example techniques described in this disclosure. The technique of FIG. 31 is described with respect to device 100 of FIG. 1, but is not limited to any particular type of computing device.

[0216]

[0240] Computing device 100 acquires image data of a patient's joint with an existing implant (3102). Computing device 100 may acquire the image data using, for example, any of the techniques described above for acquiring image data. Computing device 100 identifies an image of the existing implant within the image data of the patient's joint with an existing implant (3104). To identify the image of the existing implant, computing device 100 may, for example, segment the image data of the patient's joint with the existing implant to determine a segmented image of the existing implant.

[0217]

[0241] In some examples, the image data of the patient's joint can be a 3D model of the joint, the image of the existing implant can be a 3D model of the existing implant, and the image of the multiple implant products can be a 3D image. In some examples, the image data of the patient's joint can be a 2D image of the joint, the image of the existing implant can be a 2D image of the existing implant, and the image of the multiple implant products can be a 2D image. In some examples, the image data of the patient's joint can be a 3D model of the joint, the image of the existing implant can be a 3D model of the existing implant, and the image of the multiple implant products can be a 2D image. The image data of the patient's joint with existing implants can be, for example, an X-ray image.

[0218]

[0242] The computing device 100 accesses a database in a memory device that associates a plurality of implant products with images for the plurality of implant products, each implant product of the plurality of implant products being associated with at least one image (3106). The memory device may be either local or remote. In the database, each respective implant product may be associated with information such as an implant type for each respective implant product, a manufacturer for each respective implant product, a model type for each implant, a set of tools recommended or required to remove the implant, a set of surgical recommendations for implant removal, specifications such as dimensions of the implant product, or any other such information.

[0219]

[0243] The computing device 100 identifies at least one implant product that corresponds to the existing implant based on a comparison of the image of the existing implant to the images for the plurality of implant products (3108). The computing device 100 may determine a digitally reconstructed radiograph of the existing implant from the 3D model of the existing implant, and to identify the at least one implant product that corresponds to the existing implant, the computing device 100 may compare the digitally reconstructed radiograph of the existing implant to the images for the plurality of implant products.

[0220]

[0244] The computing device 100 may be configured to compare the shape of the existing implant in the image data with the shapes of the plurality of implant products to identify at least one implant product that corresponds to the existing implant based on a comparison of the image of the existing implant with the images of the plurality of implant products. Different implants typically have different shapes. For example, the stems of the different implant products may be symmetrical or asymmetrical, may include a collar or not, may have a straight or anatomical design, may have any one of a number of different cross-sectional shapes, may be flanged or rounded, or may have any number of other characteristic shape characteristics. The humeral platforms of the different implant products may have different neck shaft angles, thicknesses, polyethylene thicknesses, polyethylene angulations, or any number of other characteristic shape characteristics. The glenosphere components of the implant products may also have any number of characteristic shape characteristics, such as whether the glenosphere component is lateralized.

[0221]

[0245] Based on the identified at least one implant product, computing device 100 may be configured to output, for example via a display, a manufacturer's indicator for the at least one implant product corresponding to the existing implant or a product identification indicator for the at least one implant product corresponding to the existing implant. In some examples, computing device 100 may also output an indicator of a set of required or recommended tools to be used to remove the respective implant product for the at least one implant product corresponding to the existing implant and / or a set of instructions for removing the respective implant product for the at least one implant product corresponding to the existing implant.

[0222]

[0246] In some examples, the database may also store, for each respective implant product, information such as an indication of the country in which the respective implant product was installed or approved for installation, the date or date range in which the respective implant was approved for installation, the date or date range in which the respective implant was discontinued, the date or date range in which the respective implant was approved for installation in a particular country, the date or date range in which the respective implant was discontinued in a particular country, or other such information. Computing device 100 may receive user input that may also be used to identify at least one implant product corresponding to an existing implant. For example, if a patient is known to have had an implant installed in a certain country on a certain date, computing device 100 may be configured to eliminate as potential implant products any implant products that were not available in that certain country or until after that certain date.

[0223]

[0247] 32 is a flowchart illustrating an example method of operation of a computing device in accordance with one or more example techniques described in this disclosure. The technique of FIG. 32 is described with respect to device 100 of FIG. 1, but is not limited to any particular type of computing device.

[0224]

[0248] Computing device 100 acquires 3202 image data of a patient's joint with an existing implant. Computing device 100 may acquire the image data using, for example, any of the techniques described above for acquiring image data. Computing device 100 identifies 3204 portions of the image corresponding to cement that secures a portion of the existing implant to the bone. Computing device 100 may identify the portions of the image corresponding to the cement based on the segmentation techniques described above. In some cases, after performing the initial segmentation, computing device 100 may perform local processing, for example, around the stem and base plate of the implant, to more accurately determine the amount and shape or pattern of cement present. Because implant components generally have smooth and known shapes, computing device 100 may be able to perform a pixel-by-pixel analysis on portions of the implant component to determine the shape and pattern of cement that secures the implant component to the bone.

[0225]

[0249] Based on the portion of the image corresponding to the cement, the computing device 100 determines (3206) a cement strength value for the patient's joint with the existing implant. The cement strength value may, for example, be or correspond to an estimate determined from the image data of the cement's fixation strength. The cement strength may be determined based on one or more of, for example, the cement volume, the cement volume relative to the surface area of ​​the implant or bone, the cement thickness, the density of the cement region of interest (ROI), the surface seating of the implant relative to the cement, or other such factors. The computing device 100 may also determine the cement strength based on the cement pattern. As one example, even if the overall volume or average thickness of the cement is the same for two different implants, the cement strength may differ based on whether more cement is near the base or the end of the stem. As another example, even if the overall volume or average thickness of the cement is the same for two different implants, the cement strength may differ based on whether the cement edge is smooth or rough.

[0226]

[0250] The computing device 100 generates an output based on the determined cement strength (3208). The output may be, for example, one or more of an estimate of the difficulty associated with removing the existing implant or a surgical recommendation for removing the existing implant. The computing device 100 may additionally or alternatively output a surgical recommendation based on factors such as cement volume, cement volume relative to the surface area of ​​the implant or bone, cement thickness, cement ROI density, implant surface seating relative to the cement, cement pattern, or other such factors.

[0227]

[0251] The computing device 100 may access a database that stores, for a plurality of cement strengths, an associated measure of the difficulty required to remove an implant component held by a cement having that strength. Based on a regression analysis of the plurality of cement strengths, the computing device 100 may determine an estimate of the difficulty required to remove an existing implant based on the cement strength for a patient's joint with an existing implant. The measure of the difficulty required to remove the implant may be, for example, one or more of the number of blows required to remove the implant, the measured force required to remove the implant, surgeon feedback (e.g., a scale of 1 to 10), the duration of time required to perform the removal phase, or any such feedback. In this regard, the database may store information regarding or obtained from previously performed surgeries. The difficulty estimate may take virtually any form, including a numerical value (e.g., a scale of 1 to 10), a classification (e.g., easy, medium, difficult), an estimated amount of time required to perform the removal phase, or any other such type, as output to a user of the device 100.

[0228]

[0252] In some implementations, cement strength may be associated with a particular type of implant or a particular make and model of implant. Thus, once the implant type or implant make or model is determined, computing device 100 may determine an estimate of the difficulty required to remove the existing implant based on the existing implant type or model. For example, computing device 100 may determine the estimate of difficulty based on a regression analysis of multiple cement strengths for the determined implant type or model.

[0229]

[0253] While the techniques have been disclosed with respect to a limited number of examples, those skilled in the art, having the benefit of this disclosure, will appreciate numerous modifications and variations therefrom. For example, it is contemplated that any reasonable combination of the described examples may be implemented. It is intended that the appended claims cover all such modifications and variations as fall within the true spirit and scope of the present invention.

[0230]

[0254] It should be appreciated that, depending on the example, some acts or events of any of the techniques described herein may be performed in a different sequence, added, merged, or omitted altogether (e.g., not all described acts or events are necessarily required for the practice of a technique). Furthermore, in some examples, acts or events may be performed simultaneously rather than sequentially, e.g., through multithreaded processing, interrupt processing, or multiple processors.

[0231]

[0255] In one or more examples, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted to a computer-readable medium as one or more instructions or code and executed by a hardware-based processing unit. Computer-readable media may include computer-readable storage media, which correspond to tangible media such as data storage media, or communication media, including any medium that facilitates transfer of a computer program from one place to another, for example via a communication protocol. In this manner, computer-readable media may generally correspond to (1) non-transitory tangible computer-readable storage media or (2) a communication medium such as a signal or carrier wave. Data storage media may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, code, and / or data structures for implementation of the techniques described in this disclosure. A computer program product may include a computer-readable medium.

[0232]

[0256] By way of example, and not limitation, such computer-readable storage media may comprise RAM, ROM, EEPROM, CD-ROM, or other optical disk storage, magnetic disk storage, or other magnetic storage devices, flash memory, or any other medium usable to store desired program code in the form of instructions or data structures and accessible by a computer. Also, any connection is properly referred to as a computer-readable medium. For example, if instructions are transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included within the definition of medium. However, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transitory media, but instead cover non-transitory, tangible storage media. As used herein, disk and disc include compact discs (CDs), laser discs, optical discs, digital versatile discs (DVDs), floppy disks, and Blu-ray discs, where disks typically reproduce data magnetically and discs reproduce data optically using a laser. Combinations of the above should also be included within the scope of computer-readable media.

[0233]

[0257] The operations described in this disclosure may be performed by one or more processors, which may be implemented as fixed-function processing circuitry, programmable circuitry, or a combination thereof, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. A fixed-function circuit refers to a circuit that provides a specific function and is preconfigured with respect to the operations it can perform. A programmable circuit refers to a circuit that can be programmed to perform various tasks and provide flexible functionality in the operations it can perform. For example, a programmable circuit may execute instructions specified by software or firmware that cause the programmable circuit to operate in a manner defined by the software or firmware instructions. A fixed-function circuit may execute software instructions (e.g., to receive or output parameters), but the types of operations it performs are generally invariant. Thus, as used herein, the terms “processor” and “processing circuit” may refer to any of the foregoing structures or any other structure suitable for implementing the techniques described herein.

[0234]

[0258] Various examples have been described. These and other examples are within the scope of the following claims.

Claims

1. determining a shape model for the affected anatomical object; acquiring, by a computing system, image data of a patient's joint including an implant, wherein the image data comprises a 3D model of the joint with the implant; identifying voxels corresponding to bones in the 3D model of the joint with the implant; determining an initial shape estimate for the bone based on the voxels identified as corresponding to the bone; aligning the initial shape estimate to the shape model to form an aligned initial shape estimate; deforming the shape model based on the aligned initial shape estimate to generate a pre-implantation affected approximation of the bone; and A method for providing the above.

2. outputting the pre-implantation diseased approximation of the bone to a display device. The method of claim 1 further comprising:

3. identifying voxels in the 3D model of the joint with the implant that correspond to the implant; removing the voxels corresponding to the implant from the 3D model of the joint with the implant to generate a 3D model of the joint without the implant; The method of claim 1 or 2, further comprising:

4. outputting the 3D model of the joint without the implant to a display device. The method of claim 3 further comprising:

5. Segmenting the 3D model of the joint without the implant based on the pre-implantation diseased approximation of the bone. The method of claim 1 , further comprising:

6. 6. The method of claim 5, wherein segmenting the 3D model of the joint without the implant based on the pre-implantation diseased approximation of the bone comprises inputting the pre-implantation diseased approximation into a machine learning system.

7. The method of claim 5 , further comprising: outputting the segmented 3D model of the joint without the implant.

8. 8. The method of claim 7, wherein outputting the segmented 3D model of the joint without the implant comprises outputting a plurality of possible segmentations for the 3D model.

9. The method of claim 7 , further comprising: outputting confidence values ​​of the plurality of possible segmentations for the 3D model.

10. identifying voxels in the 3D model of the joint with the implant that correspond to non-implant features; 5. The method of claim 1, further comprising: removing the voxels corresponding to the non-implant features from the 3D model of the joint with the implant to generate a 3D model of the implant.

11. outputting the 3D model of the implant to a display device. The method of claim 10 further comprising:

12. identifying voxels in the 3D model of the joint with the implant that correspond to the implant to determine a 3D model of the implant; determining one or more features for the implant based on the 3D model of the implant; aligning the initial shape estimate to the shape model to form the aligned initial shape estimate based on the determined one or more features for the implant; The method of claim 1 further comprising:

13. The method of claim 12 , wherein aligning the initial shape estimate to the shape model further comprises: performing an initial alignment of the initial shape estimate to the shape model; and rotating the initial alignment relative to the shape model.

14. The method of claim 1 , wherein determining the shape model for the pre-implant diseased bone comprises receiving a classification of the pre-implant diseased bone from a user.

15. identifying the voxels corresponding to bone; classifying voxels in the 3D model as corresponding to one of bone, implant, soft tissue, or noise; The method of claim 1 , comprising:

16. 16. The method of claim 1, wherein deforming the shape model based on an initially aligned shape estimate comprises performing one or both of an iterative closest point registration or an elastic registration.

17. The method of claim 1 , wherein the image data comprises a computed tomography (CT) image.

18. acquiring, by a computing system, image data of a patient's joint including an implant, wherein the image data comprises a 3D model of the joint with the implant; identifying a first object within the 3D model, wherein the first object corresponds to a first component of the implant; determining a first vector based on the first object; identifying a second object within the 3D model, wherein the second object corresponds to a second component of the implant; determining a second vector based on the second object; determining a third vector that is perpendicular to a plane defined by the first vector and the second vector; determining a patient coordinate system based on the first vector, the second vector, and the third vector; identifying anatomical objects within the 3D model; generating an initial aligned shape that initially aligns the anatomical object from the image data to a shape model based on the determined patient coordinate system; generating information indicative of a diseased pre-implantation shape of the anatomical object based on the initial aligned shape; A method for providing the above.

19. iteratively adjusting coordinates of the initially aligned shape to rotate the initially aligned shape until a distance between the initially aligned shape and the shape model satisfies a threshold to generate an aligned shape. Furthermore, wherein generating information indicative of the diseased pre-implantation shape of the anatomical object comprises generating information indicative of the diseased pre-implantation shape based on the aligned shapes.

20. The method of claim 18.

20. acquiring, by a computing system, image data of a joint of a patient; determining, by the computing system, that the joint includes an existing implant; segmenting the image data of the joint; generating a type identification for the existing implant; A method for providing the above.

21. 21. The method of claim 20, wherein determining that the joint includes the implant comprises determining, by the computing system, that the joint includes the implant based on analysis of the image data.

22. 21. The method of claim 20, wherein determining that the joint includes the implant comprises determining, by the computing system, that the joint includes the implant based on user input.

23. 23. The method of any one of claims 20 to 22, wherein the identification of the type for the existing implant comprises one of a reverse implant, an anatomical implant, or a hemispherical implant.

24. the image data comprising a 3D model of the joint, wherein segmenting the image data comprises identifying voxels in the 3D model that correspond to the existing implant based on intensities of voxels in the 3D model, the method comprising: determining that the voxels that correspond to the existing implant form three unconnected objects; determining, in response to determining that the voxels corresponding to the existing implant form the three unconnected objects, that the type for the existing implant comprises an undefined type; 24. The method of any one of claims 20 to 23, further comprising:

25. generating an output indicating that the type for the existing implant comprises the undefined type.

25. The method of claim 24, further comprising:

26. The image data comprises a 3D model of the joint, and wherein segmenting the image data comprises identifying voxels in the 3D model that correspond to the existing implant based on intensities of voxels in the 3D model, the method comprising: determining that the voxels that correspond to the existing implant form a single connected object; responsive to determining that the voxels corresponding to the existing implant form the one connected object, identifying a second location of voxels based on the location of the voxels corresponding to the existing implant; searching for a glenoid marker at a second location of the voxel; determining, in response to not detecting the glenoid marker, that the type for the existing implant comprises a hemispherical implant type.

24. The method of any one of claims 20 to 23, further comprising:

27. generating an output indicating that the type for the existing implant comprises the hemispherical implant type.

27. The method of claim 26, further comprising:

28. The image data comprises a 3D model of the joint, and wherein segmenting the image data comprises identifying voxels in the 3D model that correspond to the existing implant based on intensities of voxels in the 3D model, the method comprising: determining that the voxels that correspond to the existing implant form a single connected object; responsive to determining that the voxels corresponding to the existing implant form the one connected object, identifying a second location of voxels based on the location of the voxels corresponding to the existing implant; searching for a glenoid marker at a second location of the voxel; determining, in response to detecting the glenoid marker, that the type for the existing implant comprises an anatomical implant type; 24. The method of any one of claims 20 to 23, further comprising:

29. generating an output indicating that the type for the existing implant comprises the anatomical implant type; 30. The method of claim 28, further comprising:

30. The image data comprises a 3D model of the joint, and wherein segmenting the image data comprises identifying voxels in the 3D model that correspond to the existing implant based on intensities of voxels in the 3D model, the method comprising: determining that the voxels that correspond to the existing implant form two unconnected objects; determining that the type for the existing implant comprises a reverse type in response to determining that the voxels corresponding to the existing implant form the two unconnected objects; 24. The method of any one of claims 20 to 23, further comprising:

31. generating an output indicating that the type for the existing implant comprises the reverse type.

31. The method of claim 30, further comprising:

32. acquiring, by a computing system, image data of a patient's joint having an existing implant; identifying, by the computing system, an image of the existing implant within the image data of the joint of the patient with the existing implant; accessing, in a memory device, a database associating a plurality of implant products with images relating to the plurality of implant products, wherein each implant product of the plurality of implant products is associated with at least one image; identifying at least one implant product corresponding to the existing implant based on a comparison of the image of the existing implant to the images of the plurality of implant products; A method for providing the above.

33. Each respective implant product is associated with a manufacturer for said respective implant product, and said method further comprises: outputting the manufacturer's indicia for the at least one implant product corresponding to the existing implant; 33. The method of claim 32, further comprising:

34. Each respective implant product is associated with a product identification for said respective implant product, said method comprising: outputting the product identification indicia for the at least one implant product corresponding to the existing implant; 34. The method of claim 32 or 33, further comprising:

35. Each respective implant product is associated with a set of tools used to remove said respective implant product, said method comprising: outputting an indication of the set of tools used to remove the respective implant product for the at least one implant product corresponding to the existing implant; 35. The method of any one of claims 32 to 34, further comprising:

36. Each respective implant product is associated with a set of instructions for removing said respective implant product, said method comprising: outputting the set of instructions for removing the respective implant product for the at least one implant product corresponding to the existing implant.

36. The method of any one of claims 32 to 35, further comprising:

37. Receiving user input Furthermore, wherein identifying the at least one implant product corresponding to the existing implant is further based on the user input.

37. The method of any one of claims 32 to 36.

38. The user input: an indication of the country in which the existing implant was installed; the date or date range when the existing implant was placed; the manufacturer of said existing implant; The type of existing implant 38. The method of claim 37, comprising one or more of:

39. determining a subset of implant products from the plurality of implant products based on the user input; Furthermore, wherein identifying the at least one implant product corresponding to the existing implant based on the comparison of the image of the existing implant to the images for the plurality of implant products comprises comparing the image data of the joint with images for only a subset of the implant products.

39. The method of claim 37 or 38.

40. 40. The method of any one of claims 32-39, wherein the image data of the joint of the patient comprises a 3D model of the joint, the images of the existing implants comprise 3D models of the existing implants, and the images of the plurality of implant products comprise 3D images.

41. 40. The method of any one of claims 32-39, wherein the image data of the joint of the patient comprises a 2D image of the joint, the image of the existing implant comprises a 2D image of the existing implant, and the images of the plurality of implant products comprise 2D images.

42. 40. The method of any one of claims 32-39, wherein the image data of the joint of the patient comprises a 3D model of the joint, the images of the existing implants comprise a 3D model of the existing implants, and the images of the plurality of implant products comprise 2D images.

43. determining a digitally reconstructed radiograph of the existing implant from the 3D model of the existing implant; Furthermore, wherein identifying the at least one implant product corresponding to the existing implant based on the comparison of the image of the existing implant to the images of the plurality of implant products comprises comparing the images of the plurality of implant products to the digitally reconstructed radiograph of the existing implant.

43. The method of claim 42.

44. 40. The method of any of claims 32 to 39, wherein the image data of the joint of the patient with the existing implant comprises an X-ray image.

45. 45. The method of any one of claims 32 to 44, wherein identifying the image of the existing implant comprises segmenting the image data of the joint of the patient with the existing implant to determine a segmented image of the existing implant.

46. 46. ​​The method of any one of claims 32 to 45, wherein identifying at least one implant product corresponding to the existing implant based on the comparison of the image of the existing implant to the images for the plurality of implant products comprises comparing a shape for the plurality of implant products with a shape of the existing implant in the image data.

47. acquiring, by a computing system, image data of a patient's joint having an existing implant; identifying, by the computing system, a portion of the image corresponding to cement securing a portion of the existing implant to bone; determining a cement strength of the joint of the patient with the existing implant based on the portion of the image corresponding to cement; generating an output based on the determined cement strength; and A method for providing the above.

48. 48. The method of claim 47, wherein the output comprises an estimate of the difficulty associated with removing the existing implant.

49. 49. The method of claim 47 or 48, wherein the output comprises a surgical recommendation to remove the existing implant.

50. storing a database of a plurality of cement strengths, each of which is associated with a measure of the difficulty required to remove the implant; determining an estimate of the difficulty required to remove the existing implant based on the cement strength for the joint of the patient with the existing implant based on the regression analysis of the plurality of cement strengths; 50. The method of any one of claims 47 to 49, further comprising:

51. 51. The method of claim 50, wherein the measure of difficulty required to remove the implant comprises a number of blows required to remove the implant.

52. 52. A computer-readable storage medium storing instructions that, when executed, cause one or more processors to perform the method of any one of claims 1 to 51.

53. Memory and Implemented in the circuit, determining a shape model for the affected anatomical object; acquiring image data of a patient's joint including an implant, wherein the image data comprises a 3D model of the joint with the implant; identifying voxels in the 3D model of the joint with the implant that correspond to bone; determining an initial shape estimate for the bone based on the voxels identified as corresponding to the bone; aligning the initial shape estimate to the shape model to form an aligned initial shape estimate; Deforming the shape model based on the aligned initial shape estimate to generate a pre-implantation affected approximation of the bone. one or more processors configured to 1. A device comprising:

54. the one or more processors: outputting the pre-implant diseased approximation of the bone to a display device.

54. The device of claim 53, further configured to:

55. the one or more processors: identifying voxels in the 3D model of the joint with the implant that correspond to the implant; 55. The device of claim 53 or 54, further configured to remove the voxels corresponding to the implant from the 3D model of the joint with the implant to generate a 3D model of the joint without the implant.

56. the one or more processors:

56. The device of claim 55, further configured to output the 3D model of the joint without the implant to a display device.

57. the one or more processors: Segmenting the 3D model of the joint without the implant based on the pre-implantation diseased approximation of the bone.

57. The device of any one of claims 53 to 56, further configured to:

58. 58. The device of claim 57, wherein the one or more processors are further configured to input the pre-implantation diseased approximation into a machine learning system to segment the 3D model of the joint without the implant based on the pre-implantation diseased approximation of the bone.

59. the one or more processors: Outputting the segmented 3D model of the joint without the implant 58. The device of claim 57, further configured to:

60. 60. The device of claim 59, wherein the one or more processors are further configured to output a plurality of possible segmentations for the 3D model to output the segmented 3D model of the joint without the implant.

61. the one or more processors: outputting confidence values ​​of the plurality of possible segmentations for the 3D model; 60. The device of claim 59, further configured to:

62. the one or more processors: identifying voxels in the 3D model of the joint with the implant that correspond to non-implant features; removing the voxels corresponding to the non-implant features from the 3D model of the joint with the implant to generate a 3D model of the implant.

57. The device of any one of claims 53 to 56, further configured to:

63. the one or more processors: Outputting the 3D model of the implant to a display device.

63. The device of claim 62, further configured to:

64. the one or more processors: identifying voxels in the 3D model of the joint with the implant that correspond to the implant to determine a 3D model of the implant; determining one or more features for the implant based on the 3D model of the implant; aligning the initial shape estimate to the shape model to form the aligned initial shape estimate based on the determined one or more features for the implant.

54. The device of claim 53, further configured to:

65. 65. The device of claim 64, wherein to align the initial shape estimate to the shape model, the one or more processors are further configured to perform an initial alignment of the initial shape estimate to the shape model and rotate the initial alignment relative to the shape model.

66. 54. The device of claim 53, wherein the one or more processors are further configured to receive a classification of the pre-implant diseased bone from a user to determine the shape model for the pre-implant diseased bone.

67. To identify the voxels corresponding to bone, the one or more processors classify voxels in the 3D model as corresponding to one of bone, implant, soft tissue, or noise.

54. The device of claim 53, further configured to:

68. 68. The device of any one of claims 53 to 67, wherein the one or more processors are further configured to perform one or both of iterative closest point registration or elastic registration to deform the shape model based on an initially aligned shape estimate.

69. 69. The device of any one of claims 53 to 68, wherein the image data comprises a computed tomography (CT) image.

70. Memory and Implemented in the circuit, acquiring image data of a patient's joint including an implant, wherein the image data comprises a 3D model of the joint with the implant; identifying a first object within the 3D model, wherein the first object corresponds to a first component of the implant; determining a first vector based on the first object; identifying a second object within the 3D model, wherein the second object corresponds to a second component of the implant; determining a second vector based on the second object; determining a third vector that is perpendicular to a plane defined by the first vector and the second vector; determining a patient coordinate system based on the first vector, the second vector, and the third vector; Identifying anatomical objects within the 3D model; generating an initial aligned shape that initially aligns the anatomical object from the image data to a shape model based on the determined patient coordinate system; generating information indicative of a diseased pre-implantation shape of the anatomical object based on the initial aligned shape; one or more processors configured to 1. A device comprising:

71. the one or more processors: Iteratively adjusting coordinates of the initially aligned shape to rotate the initially aligned shape until a distance between the initially aligned shape and the shape model satisfies a threshold to generate an aligned shape. further configured as follows: wherein, to generate the information indicative of the diseased pre-implantation shape of the anatomical object, the one or more processors are further configured to generate information indicative of the diseased pre-implantation shape based on the aligned shapes.

71. The device of claim 70.

72. Memory and Implemented in the circuit, acquiring image data of the patient's joint; determining that the joint contains an existing implant; segmenting the image data of the joint; Generate a type identification for the existing implant one or more processors configured to 1. A device comprising:

73. 73. The device of claim 72, wherein the one or more processors are further configured to determine that the joint includes the implant based on analysis of the image data.

74. 73. The device of claim 72, wherein the one or more processors are further configured to determine that the joint includes the implant based on user input to determine that the joint includes the implant.

75. 75. The device of any one of claims 72 to 74, wherein the identification of the type for the existing implant comprises one of a reverse implant, an anatomical implant, or a hemispherical implant.

76. The image data comprises a 3D model of the joint, and the one or more processors identify voxels in the 3D model that correspond to the existing implant based on intensities of voxels in the 3D model to segment the image data; determining that the voxels corresponding to the existing implants form three unconnected objects; and determining that the type for the existing implant comprises an undefined type in response to determining that the voxels corresponding to the existing implant form the three unconnected objects.

76. The device of any one of claims 72 to 75, further configured to:

77. the one or more processors: Generate an output indicating that the type for the existing implant comprises the undefined type.

77. The device of claim 76, further configured to:

78. The image data comprises a 3D model of the joint, wherein the one or more processors: identifying voxels in the 3D model that correspond to the existing implant based on the intensity of the voxels in the 3D model to segment the image data; determining that the voxels corresponding to the existing implants form a single connected object; In response to determining that the voxels corresponding to the existing implant form the one connected object, identifying a second location of voxels based on the location of the voxels corresponding to the existing implant; Searching for a glenoid marker at a second location of the voxel; In response to not detecting the glenoid marker, determining that the type for the existing implant comprises a hemispherical implant type.

76. The device of any one of claims 72 to 75, further configured to:

79. the one or more processors: Generate an output indicating that the type for the existing implant comprises the hemispherical implant type.

79. The device of claim 78, further configured to:

80. The image data comprises a 3D model of the joint, wherein the one or more processors: identifying voxels in the 3D model that correspond to the existing implant based on the intensity of the voxels in the 3D model to segment the image data; determining that the voxels corresponding to the existing implants form a single connected object; In response to determining that the voxels corresponding to the existing implant form the one connected object, identifying a second location of voxels based on the location of the voxels corresponding to the existing implant; Searching for a glenoid marker at a second location of the voxel; and determining that the type for the existing implant comprises an anatomical implant type in response to detecting the glenoid marker.

76. The device of any one of claims 72 to 75, further configured to:

81. the one or more processors: generating an output indicating that the type for the existing implant comprises the anatomical implant type; 81. The device of claim 80, further configured to:

82. The image data comprises a 3D model of the joint, wherein the one or more processors: identifying voxels in the 3D model that correspond to the existing implant based on the intensity of the voxels in the 3D model to segment the image data; determining that the voxels corresponding to the existing implants form two unconnected objects; determining that the type for the existing implant comprises a reverse type in response to determining that the voxels corresponding to the existing implant form the two unconnected objects.

76. The device of any one of claims 72 to 75, further configured to:

83. the one or more processors: generating an output indicating that the type for the existing implant comprises the reverse type; 83. The device of claim 82, further configured to:

84. Memory and Implemented in the circuit, acquiring image data of a patient's joint with an existing implant; identifying an image of the existing implant within the image data of the joint of the patient with the existing implant; accessing a database associating a plurality of implant products with images relating to the plurality of implant products, wherein each implant product of the plurality of implant products is associated with at least one image; identifying at least one implant product corresponding to the existing implant based on a comparison of the image of the existing implant to the images for the plurality of implant products; one or more processors configured to 1. A device comprising:

85. Each respective implant product is associated with a manufacturer for the respective implant product, and wherein the one or more processors: outputting the manufacturer's identification for the at least one implant product corresponding to the existing implant; 85. The device of claim 84, further configured to:

86. Each respective implant product is associated with a product identification for the respective implant product, wherein the one or more processors: outputting the product identification indicia for the at least one implant product corresponding to the existing implant; 86. The device of claim 84 or 85, further configured to:

87. each respective implant product is associated with a set of tools used to remove said respective implant product, wherein said one or more processors: and outputting an indication of the set of tools used to remove the respective implant product for the at least one implant product corresponding to the existing implant.

87. The device of any one of claims 84 to 86, further configured to:

88. Each respective implant product is associated with a set of instructions for removing said respective implant product, wherein said one or more processors: and outputting the set of instructions for removing the respective implant product for the at least one implant product corresponding to the existing implant.

88. The device of any one of claims 84 to 87, further configured to:

89. the one or more processors: Accepts user input, Identifying the at least one implant product that corresponds to the existing implant based on the user input.

89. The device of any one of claims 84 to 88, further configured to:

90. The user input: an indication of the country in which the existing implant was installed; the date or date range when the existing implant was placed; the manufacturer of said existing implant; The type of existing implant 90. The device of claim 89, comprising one or more of:

91. the one or more processors: determining a subset of implant products from the plurality of implant products based on the user input; comparing the image data of the joint with images relating to only a subset of the implant products to identify the at least one implant product corresponding to the existing implant based on the comparison of the images of the existing implant to the images of the plurality of implant products.

91. The device of claim 89 or 90, further configured to:

92. 92. The device of any one of claims 84-91, wherein the image data of the joint of the patient comprises a 3D model of the joint, the images of the existing implant comprise a 3D model of the existing implant, and the images of the plurality of implant products comprise 3D images.

93. 92. The device of any one of claims 84 to 91, wherein the image data of the joint of the patient comprises a 2D image of the joint, the image of the existing implant comprises a 2D image of the existing implant, and the image of the plurality of implant products comprises a 2D image.

94. 92. The device of any one of claims 84-91, wherein the image data of the joint of the patient comprises a 3D model of the joint, the images of the existing implant comprise a 3D model of the existing implant, and the images of the plurality of implant products comprise 2D images.

95. the one or more processors: determining a digitally reconstructed radiograph of the existing implant from the 3D model of the existing implant; comparing the images of the plurality of implant products with the digitally reconstructed radiograph of the existing implant to identify the at least one implant product that corresponds to the existing implant based on the comparison of the images of the existing implant to the images of the plurality of implant products.

95. The device of claim 94, further configured to:

96. 92. The device of any one of claims 84 to 91, wherein the image data of the joint of the patient with the existing implant comprises an X-ray image.

97. 97. The device of any one of claims 84 to 96, wherein the device segments the image data of the joint of the patient with the existing implant to determine a segmented image of the existing implant to identify the image of the existing implant.

98. 98. The device of any one of claims 84 to 97, wherein the device compares shapes for the plurality of implant products with shapes of the existing implants in the image data to identify at least one implant product that corresponds to the existing implant based on the comparison of the image of the existing implant to the images for the plurality of implant products.

99. Memory and Implemented in the circuit, acquiring image data of a patient's joint with an existing implant; identifying a portion of the image corresponding to cement securing a portion of the existing implant to bone; determining a cement strength of the joint of the patient with the existing implant based on the portion of the image corresponding to cement; generating an output based on the determined cement strength; one or more processors configured to 1. A device comprising:

100. 100. The device of claim 99, wherein the output comprises an estimate of the difficulty associated with removing the existing implant.

101. 101. The device of claim 99 or 100, wherein the output comprises a surgical recommendation to remove the existing implant.

102. the one or more processors: storing a database of a plurality of cement strengths, each of said cement strengths being associated with a measure of the difficulty required to remove the implant; determining an estimate of the difficulty required to remove the existing implant based on the cement strength for the joint of the patient with the existing implant based on the regression analysis of the plurality of cement strengths; 102. The device of any one of claims 99 to 101, further configured to:

103. 103. The device of claim 102, wherein the measure of difficulty required to remove the implant comprises a number of blows required to remove the implant.

Images