Soft tissue modeling and planning system for plastic surgery procedures

Abstract

A surgical planning system for use in a surgical procedure to repair an anatomical structure of interest, comprising: a pre-planning system that generates a virtual surgical plan; and a mixed reality system, comprising a visualization device that can be worn by a user to view the virtual surgical plan projected in a real environment. The virtual surgical plan includes a 3D virtual model of the anatomical structure of interest. When wearing the visualization device, the user can align the 3D virtual model with the real anatomical structure of interest, thereby achieving registration between the details of the virtual surgical plan and the real anatomical structure of interest. The registration enables the surgeon to implement the virtual surgical plan on the real anatomical structure of interest without the use of tracking markers.

Description

Background Art

[0001] Surgical joint repair procedures involve the repair and / or replacement of damaged or diseased joints. Many times, surgical joint repair procedures (such as arthroplasty, etc., as an example) involve replacing a damaged joint with a prosthesis implanted into the patient's bone. Properly selecting a prosthesis of the appropriate size and shape and properly positioning the prosthesis to ensure the best surgical outcome is challenging. To aid in positioning, surgical procedures often involve the use of surgical instruments to control the shaping of the surface of the damaged bone and control the cutting or drilling of the bone to receive the prosthesis.

[0002] Today, surgeons have access to visualization tools that use 3D modeling of bone shape to facilitate preoperative planning for joint repair and replacement. These tools can help surgeons design and / or select surgical guides and implants that closely match the patient's anatomy and can improve surgical outcomes by customizing the surgical plan for each patient. Summary of the invention

[0003] The present disclosure describes various systems, devices, and techniques for providing patient analysis, preoperative planning, and / or training and training for surgical joint repair procedures. For example, the system described herein can determine soft tissue (e.g., muscle, connective tissue, adipose tissue, etc.) dimensions and / or features from patient imaging data. The features of the soft tissue can include fat infiltration, atrophy rate, range of motion, or other similar features. The system can use these soft tissue dimensions and / or features to determine the range of motion values ​​of one or more joints of the patient. Based on these range of motion values, the system can suggest one or more types of surgical interventions that may be suitable for treating one or more health conditions of one or more joints. In one embodiment, the system can use soft tissue dimensions to determine whether anatomical shoulder replacement surgery or reverse shoulder replacement surgery is suitable for a specific patient.

[0004] Or, alternatively, the system described herein can also determine the bone density measurement of at least a portion of the humeral head of the patient based on the patient-specific image data of the patient.For example, the bone density measurement can be a single indication of the overall density of the humeral head or a part of the humeral head.As another embodiment, the bone density measurement can include the bone density values ​​of the various parts of the humeral head of the patient.In fact, the bone density measurement can not indicate the density of the bone, but can be a measurement representing bone density (for example, the standard deviation of the voxel intensity from the image data, the compressibility, etc.). The system can control the user interface to present a graphical representation of the bone density measurement and / or generate suggestions about the implant type of the humeral head based on the bone density measurement.For example, the bone density measurement indicating enough trabecular bone density in the humeral head can cause the system to suggest a stemless humeral implant opposite to a stemmed humeral implant.

[0005] In one embodiment, a system for modeling a soft tissue structure of a patient includes: a memory configured to store patient-specific image data of the patient; and a processing circuit configured to: receive the patient-specific image data; determine a patient-specific shape representing the patient's soft tissue structure based on an intensity of the patient-specific image data; and output the patient-specific shape.

[0006] In another embodiment, a method for modeling a soft tissue structure of a patient includes: storing patient-specific image data of the patient by a memory; receiving the patient-specific image data by a processing circuit; determining, by the processing circuit, a patient-specific shape representing the soft tissue structure of the patient based on the intensity of the patient-specific image data; and outputting the patient-specific shape by the processing circuit.

[0007] In another embodiment, a computer-readable storage medium includes instructions that, when executed by a processing circuit, cause the processing circuit to: store patient-specific image data of a patient in a memory; receive the patient-specific image data; determine a patient-specific shape representing a soft tissue structure of the patient based on an intensity of the patient-specific image data; and output the patient-specific shape.

[0008] In another embodiment, a system for modeling a soft tissue structure of a patient includes: a device for storing patient-specific image data of the patient; a device for receiving the patient-specific image data; a device for determining a patient-specific shape representing the patient's soft tissue structure based on the intensity of the patient-specific image data; and a device for outputting the patient-specific shape.

[0009] The details of various embodiments of the present disclosure are set forth in the accompanying drawings and the following description. Various features, objects, and advantages will become apparent from the specification, drawings, and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 is a block diagram of a plastic surgery system according to an embodiment of the present disclosure.

[0011] Figure 2 is a block diagram of an orthopedic surgery system including a mixed reality (MR) system according to an embodiment of the present disclosure.

[0012] Figure 3 is a flow chart illustrating example stages of a surgical life cycle.

[0013] Figure 4 is a flow chart showing the preoperative, intraoperative, and postoperative workflow in support of a plastic surgery procedure.

[0014] Figure 5A and Figure 5B is an illustration of exemplary muscles and bones associated with a patient's shoulder.

[0015] Figure 6 is a block diagram illustrating exemplary components of a system configured to determine soft tissue structure dimensions and / or characteristics and other information related to surgical intervention associated with a joint in accordance with an embodiment of the present disclosure.

[0016] Figure 7 is a diagram of exemplary insertion points of the muscles of the rotator cuff;

[0017] Fig. 8A is a conceptual diagram of exemplary patient-specific image data.

[0018] Figure 8B is based on Fig. 8A Conceptual illustration of a Hesse feature image generated from patient-specific image data.

[0019] Figure 8C is a conceptual diagram of an exemplary initial shape and an exemplary contour superimposed on patient-specific image data.

[0020] Fig. 9 is a conceptual diagram of an exemplary procedure for changing an initial shape toward a patient-specific shape representing a soft tissue structure of the patient.

[0021] Fig.10 is a conceptual illustration of an exemplary procedure for changing an intermediate shape toward a patient-specific shape representing the patient's soft tissue structure.

[0022] Fig.11 is a conceptual diagram representing exemplary patient-specific shapes of a patient's soft tissue structures as compared to actual contours in the patient-specific image data.

[0023] Fig.12is a conceptual diagram showing an exemplary initial shape and patient-specific shape of a scapular muscle superimposed on patient-specific image data.

[0024] Fig.13 is a conceptual diagram showing an exemplary initial shape and patient-specific shape of the supraspinatus muscle superimposed on patient-specific image data.

[0025] Fig.14 is a conceptual axial view showing an exemplary final patient-specific shape of the rotator cuff muscles superimposed on the patient-specific image data.

[0026] Fig.15 is a conceptual sagittal suture illustration showing an exemplary final patient-specific shape of the rotator cuff muscles superimposed on patient-specific image data.

[0027] Fig.16A is a post-concept three-dimensional view showing an exemplary final patient-specific shape of the rotator cuff muscles and bones from the patient-specific image data.

[0028] Fig. 16B is a conceptual front three-dimensional view showing an exemplary final patient-specific shape of the rotator cuff muscles and bones from patient-specific image data.

[0029] Fig.17 is a conceptual end-on three-dimensional view showing an exemplary final patient-specific shape of the rotator cuff muscles and bone from the patient-specific image data.

[0030] Fig.18A and Fig.18B is a conceptual illustration of exemplary patient-specific CT data in which an initial shape associated with a soft tissue structure is registered with a skeletal structure and modified into a patient-specific shape representing the soft tissue structure.

[0031] Fig.19 is a conceptual diagram of an exemplary final patient-specific shape that is masked and thresholded to determine soft tissue characteristics such as fat infiltration.

[0032] Fig. 20 Conceptual illustration of exemplary final patient-specific shapes and pre-morbid assessment of soft tissue structures.

[0033] Fig.21 and Fig. 22 is a conceptual diagram of an exemplary spring-modeled muscle that contributes to range of motion analysis of the shoulder joint.

[0034] Fig.23A is a flow chart illustrating an exemplary process for modeling soft tissue structures using patient-specific image data in accordance with the techniques of this disclosure.

[0035] Fig. 23B is a flow chart illustrating another exemplary process for modeling soft tissue structures using patient-specific image data in accordance with the techniques of this disclosure.

[0036] Fig.24 is a flow chart illustrating an exemplary process for modeling soft tissue structures using patient-specific image data in accordance with the techniques of this disclosure.

[0037] Fig.25 is a flow chart illustrating an exemplary process for determining fat infiltration values ​​of soft tissue structures of a patient in accordance with the presently disclosed techniques.

[0038] Fig.26 is a flow chart illustrating an exemplary process for determining atrophy rate of a patient's soft tissue structure in accordance with the presently disclosed techniques.

[0039] Fig. 27 is a flow chart illustrating an exemplary process for determining a type of shoulder treatment based on a determined soft tissue structure of a patient in accordance with the presently disclosed techniques.

[0040] Fig.28 is a flow chart illustrating an exemplary process for determining a type of shoulder treatment based on patient-specific image data in accordance with the techniques of this disclosure.

[0041] Fig.29 is a block diagram illustrating an exemplary computing system implementing a deep neural network (DNN) that may be used to determine one or more aspects of a patient's anatomy, diagnosis, and / or treatment recommendations in accordance with the techniques of this disclosure.

[0042] Fig.30 It shows that it can be Fig.29 An exemplary DNN implemented by an exemplary computing system in .

[0043] Fig.31 is a flow chart illustrating example operation of a computing system using a DNN to determine a recommended type of shoulder surgery for a patient in accordance with the techniques of this disclosure.

[0044] Fig.32 is an illustration of exemplary bones associated with a patient's shoulder.

[0045] Fig.33A , Fig.33B ,as well as Fig.33C is a conceptual illustration of an exemplary humerus prepared for a humeral implant.

[0046] Fig.34 is a conceptual diagram of an exemplary humeral implant.

[0047] Fig.35 is a conceptual diagram of an exemplary stemmed humeral implant.

[0048] Fig.36 is a conceptual illustration of an exemplary stemless humeral implant implanted on a humeral head.

[0049] Fig.37 is a conceptual diagram of an exemplary reverse humeral implant.

[0050] Fig.38 is a block diagram illustrating exemplary components of a system configured to determine bone density from patient-specific image data according to an embodiment of the present disclosure.

[0051] Fig.39A is a flow chart illustrating an exemplary process for determining a type of humeral implant based on bone density.

[0052] Fig.39B is a flow chart illustrating an exemplary process for applying a neural network to patient-specific image data to determine a stem size for a humeral implant.

[0053] Fig.39C is a flow chart illustrating an exemplary process for determining a recommendation for shoulder treatment based on soft tissue structure and bone density determined from patient-specific image data.

[0054] Fig.40 is a flow chart illustrating an exemplary process for displaying bone density information.

[0055] Fig.41 is a conceptual diagram of an exemplary user interface including a femoral head and a cutting plane.

[0056] Fig.42 is a conceptual diagram of an exemplary user interface including a representation of a femoral head and internal bone density.

[0057] Fig.43 is a conceptual diagram of an exemplary user interface including a representation of the humeral head and internal bone density associated with a type recommendation for a humeral implant. DETAILED DESCRIPTION

[0058] The present disclosure describes various systems, devices, and techniques for providing patient analysis, preoperative planning, and / or training and training for surgical joint repair procedures. Orthopedic surgery can involve implanting one or more prosthetic devices to repair or replace a patient's damaged or diseased joint. Virtual surgical planning tools use image data of the diseased or damaged joint to generate an accurate three-dimensional bone model that can be viewed and manipulated by a surgeon preoperatively. These tools can enhance surgical results by allowing a surgeon to simulate surgery, select or design an implant that more strictly matches the contour of the patient's actual bone, and select or design surgical instruments and guide tools that are particularly suitable for repairing a particular patient's bone.

[0059] These planning tools can be used to generate preoperative surgical plans, implants and surgical instruments selected or manufactured for individual patients. These systems can rely on the patient's skeletal model to determine the type of individual patient's program and / or specific implant. However, the soft tissue structure information (e.g., muscle and / or connective tissue) derived from the patient's imaging data is unavailable. In the case of this imaging data of the soft tissue of the patient, planning tools and clinicians can determine certain aspects of surgical operations or implants without the help of the patient's soft tissue how to affect the current joint and postoperative joint.

[0060] As described herein, the system can determine soft tissue (e.g., muscle, tendon, ligament, cartilage, and / or connective tissue) scale and other features from patient imaging data. Therefore, the system can be configured to use these soft tissue scales and / or other features derived from patient imaging data to select or suggest certain types of medical intervention, types of surgical treatment or even types, scales, and / or placement of one or more medical implants. In this way, the system can use the soft tissue information derived from patient imaging data to determine or help determine the surgical plan of a specific patient. For example, the system can select between anatomical shoulder replacement surgery or reverse shoulder replacement surgery, and then based on the soft tissue scales and other features derived from patient imaging data, the option is output to users such as surgeons by, for example, presentation on a display. These suggestions to shoulder replacement described herein can be applied to new replacement or first replacement of the shoulder, or in other embodiments, can be applied to revision surgery in which the patient has undergone shoulder replacement. Typically, shoulder surgery can be used to restore shoulder function and / or reduce patient's pain.

[0061] In one embodiment, the system can receive patient imaging data (e.g., including X-ray images, magnetic resonance imaging (MRI) images, or computed tomography (CT) of other imaging modalities) and construct a three-dimensional (3D) imaging data set. From the imaging data set, the system can identify the location of bones associated with the soft tissue structure of interest and the approximate location of the soft tissue structure itself. For example, if a patient may need shoulder replacement surgery, the system can identify the various parts of the scapula and humerus and the muscles of the rotator cuff. For each soft tissue structure (e.g., for each muscle of the rotator cuff), the system can determine a representation of the soft tissue structure from the imaging data. The system can place an initial shape within the estimated position of the soft tissue structure and then fit the initial shape to the imaging data to determine a representation of the actual soft tissue structure. The estimated position can be based on one or more markers or landmarks (e.g., muscle insertion points or muscle origins) on the associated bones or other skeletal structures or portions of skeletal structures. The initial shape can be a statistical mean shape (SMS) derived from multiple subjects or arbitrary geometric shapes.

[0062] For the initial shape, the system can use vectors orthogonal to the surface of the initial shape to identify the voxels outside or inside the initial shape and exceeding the intensity threshold representing the boundary of the soft tissue structure in the imaging data. In some embodiments, the boundary of the soft tissue structure can be estimated from the separation zone identified between adjacent soft tissue structures. For each vector, from the corresponding position on the initial shape, the system can move the surface of the initial shape toward the corresponding voxel of the identified voxel. This movement of the surface of the initial shape may occur several iterations until the initial shape is modified to be close to the contour of the identified voxel. In other embodiments, the system can use the correspondence from the initial shape to the associated bones and / or a minimization or maximization algorithm (e.g., a cost function) to fit and scale the initial shape to patient-specific image data. Then, the final modified shape can be used as a representation of soft tissue structures such as the muscles of the patient's rotator cuff.

[0063] The system can determine one or more features of one or more soft tissue structures from the determined representation. For example, the system can employ these features for surgical planning. In some embodiments, the system can calculate the volume of the soft tissue structure used in other determinations. Within the representation of the soft tissue structure, the system can determine the fat permeability value of the soft tissue structure based on the thresholded intensity of the voxels from the imaging data, that is, compare the voxel intensity and the threshold intensity value to characterize the voxel as a representation of fat tissue. For example. A voxel with an intensity value exceeding a threshold within the segmented representation of the soft tissue structure can be considered to be a representation of fat tissue rather than muscle tissue. A voxel that does not exceed the threshold intensity value or a group of two or more voxels can be considered to represent non-fat tissue. As used herein, the term "exceeds" can refer to a value greater than a threshold or less than a threshold.

[0064] The ratio of adipose tissue to total tissue (including adipose tissue and non-adipose tissue) within the representation can be determined as a fat permeability value. The system can also calculate the atrophy rate by dividing the volume of the SMS fitted to the patient's skeleton (e.g., the pre-disease tissue volume estimated for the patient) by the volume of the representation of the soft tissue structure. The system can then determine the spring constant (or some other representation of muscle function) of the soft tissue structure and other soft tissue structures associated with the joint to determine the range of motion of the joint. The system can then determine the type of intervention for the joint based on the range of motion and / or other characteristics discussed herein. For example, the system can determine whether an anatomical shoulder replacement or a reverse shoulder replacement is a more appropriate treatment for the patient.

[0065] In some embodiments, the system can determine the bone density characteristics of the humeral head of the humerus based on patient-specific image data (e.g., 2D or 3D image data). For example, the system can characterize the bone density value assigned to the voxel or voxel combination of the trabecular bone in at least a portion of the humeral head. In other embodiments, the system can determine the overall bone density measurement or score of the entire volume of the trabecular bone in at least a portion of the humeral head indicating the humeral head. The system can control the display device to display a user interface, and the user interface includes representations of bone density such as graphical indications of bone density. In some embodiments, the system can generate suggestions for the type of humeral implant (e.g., with or without a stem) based on the determined bone density. In some embodiments, the suggestion of the type of humeral implant can be based on the historical surgical data of the humeral implant, wherein the system associates the type of the humeral implant used for the patient with the bone density value identified in the patient-specific image data for the same patient.

[0066] As an embodiment herein, shoulder replacement surgery is described. However, other anatomical structures or structure groups of patients can be analyzed using systems, equipment, and technology described herein, to determine the treatment type of other joints (e.g., elbow, hip, knee, etc.) of the patient, or to select a certain type of implant for the patient's specific anatomical health condition. In addition, in other embodiments, other structures such as bones can be identified using technology described herein for determining soft tissue structure from patient imaging data.

[0067] In some embodiments, systems, devices, and methods may employ a mixed reality (MR) visualization system to assist in creating, implementing, validating, and / or modifying surgical plans prior to and during a surgical procedure, such as determining the associated processes for the type of treatment being provided to a patient (e.g., a joint replacement procedure such as a shoulder replacement). Because MR, or in some instances VR, may be used to interact with the surgical plan, the present disclosure may also refer to the surgical plan as a "virtual" surgical plan. According to the techniques of the present disclosure, a mixed reality visualization system and a visualization tool may be used together or a visualization tool may be used in addition to a mixed reality visualization system.

[0068] For example, by the system BLUEPRINT TM A surgical plan or suggestion generated by the system, available from Wright Medical, Inc., or another surgical planning platform may include information defining various features of a surgical procedure, such as suggested steps for surgical treatment (e.g., anatomical or reverse shoulder surgery) of a particular surgical procedure step to be performed on a patient by a surgeon according to a surgical plan (e.g., including bone or tissue preparation steps and / or steps for selecting, modifying, or replacing implant components). In various embodiments, the information may include the size, shape, angle, surface contour, and / or orientation of an implant component selected or modified by a surgeon, the size, shape, angle, surface contour, and / or orientation of bone or soft tissue defined by a surgeon in a bone or tissue preparation step, and / or the position, axis, plane, angle, and / or entry point of placement of an implant component defined by a surgeon relative to the patient's bone or other tissue. Information such as the size, shape, angle, surface contour, and / or orientation of an anatomical feature of a patient may be derived from imaging (e.g., x-ray, CT, MRI, or ultrasound or other images), direct observation, or other techniques.

[0069] Some visualization tools utilize patient image data to generate three-dimensional models of bone contours to facilitate preoperative planning for joint repair and replacement. These tools can allow surgeons to design and / or select surgical guides and implant components that closely match the patient's anatomy. These tools can improve surgical outcomes by customizing the surgical plan for each patient. An example of such a visualization tool for shoulder repair is the BLUEPRINT® tool identified above. TM SYSTEM. BLUEPRINT TM The system provides surgeons with a 2D plan view of the bone repair area and a 3D virtual model of the repair area. Surgeons can use BLUEPRINT TMsystems to select, design, or modify appropriate implant components, determine how best to position and orient the implant components and how to shape the surface of the bone to receive the components, and design, select, or modify surgical guide tools or instruments to accomplish the surgical plan. Powered by BLUEPRINT TM The information generated by the system can be compiled in the patient's preoperative surgical plan (including before and during the actual surgery) stored in an appropriate location in a database (e.g., on a server on a wide area network, local area network, or global network), where the surgeon or other care provider can access the appropriate location in the database.

[0070] Specific embodiments of the present disclosure are described with reference to the attached drawings, wherein like reference numerals represent like elements. However, it should be understood that the attached drawings only illustrate various implementations described herein and are not intended to limit the scope of the various techniques described herein. The drawings illustrate and describe various embodiments of the present disclosure.

[0071] In the following description, many details are set forth to provide an understanding of the present disclosure. However, it should be understood by those skilled in the art that one or more aspects of the present disclosure may be implemented without these details, and that various modifications or alterations of the described embodiments are possible.

[0072] Figure 1 is a block diagram of a plastic surgery system 100 according to an embodiment of the present disclosure. The plastic surgery system 100 includes a set of subsystems. Figure 1 In an embodiment of the present invention, the subsystems include a virtual planning system 102, a planning support system 104, a manufacturing and delivery system 106, an intraoperative guidance system 108, a medical education system 110, a monitoring system 112, a predictive analysis system 114, and a communication network 116. In other embodiments, the plastic surgery system 100 may include more, fewer, or different subsystems. For example, the plastic surgery system 100 may omit the medical education system 110, the monitoring system 112, the predictive analysis system 114, and / or other subsystems. In some embodiments, the plastic surgery system 100 can be used for surgical tracking, in which case the plastic surgery system 100 can be referred to as a surgical tracking system. In other cases, the plastic surgery system 100 can be generally referred to as a medical device system.

[0073] A user of the orthopedic surgery system 100 can use the virtual planning system 102 to plan an orthopedic surgery. As discussed herein, for example, the virtual planning system 102 and / or another surgical planning system can analyze patient imaging data (e.g., bone and / or soft tissue) and determine the proposed surgical treatment based on the bone and / or soft tissue features determined from the imaging data. A user of the orthopedic surgery system 100 can use the planning support system 104 to review the surgical plan generated using the orthopedic surgery system 100. The manufacturing and delivery system 106 can help manufacture and deliver items needed to perform orthopedic surgery. The intraoperative guidance system 108 provides guidance to help users of the orthopedic surgery system 100 to perform orthopedic surgery. The medical education system 110 can help train users, such as medical professionals, patients, and other types of individuals. The preoperative and intraoperative monitoring system 112 can help monitor patients before and after they undergo surgery. The predictive analysis system 114 can help medical professionals make various types of predictions. For example, the predictive analysis system 114 can apply artificial intelligence techniques to determine a classification (e.g., diagnosis) of the health condition of an orthopedic joint, determine which type of surgery to perform on a patient and / or which type of implant to use in a procedure, determine the types of items that may be needed during a surgical procedure, and the like.

[0074] The subsystems of the plastic surgery system 100 (e.g., the virtual planning system 102, the planning support system 104, the manufacturing and delivery system 106, the intraoperative guidance system 108, the medical education system 110, the preoperative and intraoperative monitoring system 112, and the predictive analysis system 114) may include various systems. The systems in the subsystems of the plastic surgery system 100 may include various types of computing systems, computing devices, including server computers, personal computers, tablet computers, smart phones, display devices, Internet of Things (IoT) devices, visualization devices (e.g., mixed reality (MR) visualization devices, virtual reality (VR) visualization devices, holographic projectors, or other devices for presenting extended reality (XR) visualization), surgical tools, etc. In some embodiments, the holographic projector can project holograms for overall viewing by multiple users or a single user without headphones, rather than only for viewing by users wearing headphones. For example, the virtual planning system 102 may include an MR visualization device and one or more server devices, the planning support system 104 may include one or more personal computers and one or more server devices, etc. A computing system is a collection of one or more computing devices and / or systems configured to operate as a system. In some embodiments, one or more devices may be shared between two or more subsystems of the orthopaedic surgical system 100. For example, in the previous embodiments, the virtual planning system 102 and the planning support system 104 may include the same server device.

[0075] Exemplary MR visualization devices include the Microsoft HOLOLENS 300, available from Microsoft Corporation of Redmond, Washington. TM The headset, which includes a see-through holographic lens, sometimes called a waveguide, allows the user to see real-world objects through the lens and see the projected 3D holographic objects at the same time. MICROSOFT HOLOLENS TM Headphones or similar waveguide-based visualization devices are embodiments of MR visualization devices that can be used according to some embodiments of the present disclosure. Some holographic lenses can present holographic objects with a certain degree of transparency through a see-through holographic lens so that the user sees real-world objects and virtual holographic objects. In some embodiments, some holographic lenses may sometimes completely prevent users from seeing real-world objects, but may allow users to see a complete virtual environment. The term "mixed reality" may also cover scenes in which one or more users are able to perceive one or more virtual objects generated by holographic projections. In other words, "mixed reality" may cover situations in which a holographic projector generates a hologram of an element that appears to exist in the user's actual physical environment. Although the MR visualization device is described as one embodiment herein, in other embodiments, display screens such as cathode ray tube (CRT) displays, liquid crystal displays (LCDs), and light emitting diode (LED) displays may be used to present any aspect of the information described herein.

[0076] exist Figure 1 In some embodiments, the devices included in the subsystems of the plastic surgery system 100 can communicate using a communication network 16. The communication network 16 can include various types of communication networks, including one or more wide area networks such as the Internet, local area networks, etc. In some embodiments, the communication network 116 can include wired and / or wireless communication links.

[0077] Various variations of the orthopaedic surgical system 100 are possible. The variations may include: Figure 1 The versions of the plastic surgery system 100 shown in FIG. 1 may include more or fewer subsystems. For example, Figure 2 1 is a block diagram of an orthopaedic surgery system 200 including one or more mixed reality (MR) systems according to an embodiment of the present disclosure. The orthopaedic surgery system 200 can be used to create, verify, update, modify, and / or implement a surgical plan. In some embodiments, such as by using a virtual surgical planning system (e.g., BLUEPRINT TMIn some embodiments, the plastic surgery system 200 can be used to create a surgical plan immediately before or during surgery as needed. In some embodiments, the plastic surgery system 200 can be used for surgical tracking, in which case the plastic surgery system 200 can be referred to as a surgical tracking system. In other cases, the plastic surgery system 200 can be generally referred to as a medical device system.

[0078] exist Figure 2 In an embodiment of the present invention, the orthopedic surgery system 200 includes a preoperative surgical planning system 202, a healthcare facility 204 (e.g., a surgical center or hospital), a storage system 206, and a network 208 that allows a user at the healthcare facility 204 to access stored patient information such as medical history, image data corresponding to a damaged joint or bone, and various parameters corresponding to a surgical plan that has been created preoperatively (as an example). The preoperative surgical planning system 202 can be used with Figure 1 and in some embodiments may generally correspond to the virtual planning system 102 in BLUEPRINT TM Virtual planning systems that are similar or identical to the system.

[0079] exist Figure 2In some embodiments of the present disclosure, the healthcare facility 204 includes a mixed reality (MR) system 212. In some embodiments of the present disclosure, the MR system 212 includes one or more processing devices (P) 210 to provide functions such as presenting visual information related to preoperative planning, intraoperative guidance, or even postoperative review and follow-up to the user. The processing device 210 may also be referred to as a processor. In addition, one or more users of the MR system 212 (e.g., a surgeon, a nurse, or other care provider) can use the processing device (P) 210 to generate a request for a specific surgical plan or other patient information, which is sent to the storage system 206 via the network 208. In response, the storage system 206 returns the requested patient information to the MR system 212. In some embodiments, the user can request and receive information using other processing devices, such as one or more processing devices that are part of the MR system 212 but not part of any visualization device, or one or more processing devices that are part of the visualization device (e.g., visualization device 213) of the MR system 212, or a combination of one or more processing devices that are part of the MR system 212 but not part of any visualization device and one or more processing devices that are part of the visualization device (e.g., visualization device 213), and the visualization device (e.g., visualization device 213) is part of the MR system 212. In other words, a user can request and receive information using other processing devices, such as one or more processing devices that are part of the MR system 212 but not part of any visualization device, or one or more processing devices that are part of the visualization device (e.g., visualization device 213). TM Exemplary MR visualization devices such as devices may include all components of the MR system 212 , or utilize one or more external processors and / or memories to perform some or all of the processing functions necessary for the passive visualization device 213 .

[0080] In some embodiments, multiple users can use the MR system 212 simultaneously. For example, the MR system 212 can be used in a spectator mode, in which multiple users each use their own visualization device so that the users can see the same information at the same time and from the same perspective. In some embodiments, the MR system 212 can be used in a mode in which multiple users each use their own visualization device so that the users can see the same information from different perspectives.

[0081] In some embodiments, the processing device 210 can provide a user interface for displaying data and receive input from a user at the healthcare facility 204. The processing device 210 can be configured to control the visualization device 213 to present the user interface. Further, the processing device 210 can be configured to control the visualization device 213 (e.g., one or more optical waveguides such as a holographic lens) to present virtual images, such as 3D virtual models, 2D images, surgical planning information, etc. The processing device 210 can include a variety of different processing or computing devices, such as servers, desktop computers, laptop computers, tablet computers, mobile phones, and other electronic computing devices, or processors within the device. In some embodiments, one or more processing devices 210 can be located away from the healthcare facility 204. In some embodiments, the processing device 210 is located within the visualization device 213. In some embodiments, at least one processing device 210 is located outside the visualization device 213. In some embodiments, one or more processing devices 210 are located within the visualization device 213 and one or more processing devices 210 are located outside the visualization device 213.

[0082] exist Figure 2 In some embodiments, the MR system 212 also includes one or more memories or storage devices (M) 215 for storing data and instructions of software that can be executed by the processing device 210. The instructions of the software can correspond to the functions of the MR system 212 described herein. In some embodiments, it is also possible to store software such as BLUEPRINT TM The processing device 210 may be configured to perform the functions of a virtual surgical planning application such as a visualization device 213 and the memory storage device (M) 215. For example, the memory or storage system 215 may be configured to store data corresponding to at least a portion of the virtual surgical plan. In some embodiments, the storage system 206 may be configured to store data corresponding to at least a portion of the virtual surgical plan. In some embodiments, the memory or storage device (M) 215 is located within the visualization device 213. In some embodiments, the memory or storage device (M) 215 is located external to the visualization device 213. In some embodiments, the memory or storage device (M) 215 includes a combination of one or more memories or storage devices within the visualization device 213 and one or more memories or storage devices located external to the visualization device.

[0083] The network 208 can be equivalent to the network 116. The network 208 can include one or more wide area networks, local area networks, and / or global networks (e.g., the Internet) that connect the preoperative surgical planning system 202 and the MR system 212 to the storage system 206. The storage system 206 can include one or more databases containing patient information, medical information, patient image data, and parameters that define the surgical plan. For example, in preparation for an orthopedic surgical procedure, medical images of the patient's diseased or damaged bones and / or soft tissues are usually generated preoperatively. The medical images can include images of the relevant bones and / or soft tissues taken along the sagittal and coronal planes of the patient's body. The medical images can include X-ray images, magnetic resonance imaging (MRI) images, computed tomography (CT) images, ultrasound images, and / or any other type of 2D or 3D images that provide information about the relevant surgical area. The storage system 206 can also include data identifying implant components selected for a particular patient (e.g., type, size, etc.), surgical guides selected for a particular patient, and details of the surgical procedure such as entry point, cutting plane, drilling axis, reaming depth, etc. As an embodiment, the storage system 206 can be a cloud-based storage system (as shown) or can be located at the healthcare facility 204 or at the location of the preoperative surgical planning system 202, or can be part of the MR system 212 or visualization device (VD) 213.

[0084] Before (e.g., preoperatively) or during (e.g., intraoperatively) a surgical procedure, a surgeon can use the MR system 212 to create, review, verify, update, modify, and / or implement a surgical plan. In some embodiments, the MR system 212 can also be used after (e.g., postoperatively) a surgical procedure to review the results of the surgical procedure, assess whether a revision is needed, or perform other postoperative tasks. In this way, the MR system 12 enables a user to see a real-world scene such as an anatomical object in addition to a virtual image (e.g., a virtual scapula or humerus image, a guide image, or other text or image) placed at the real-world scene. To this end, the MR system 212 may include a visualization device 213 that is worn by the surgeon and (as described in further detail below) is operable to display various types of information, including a 3D virtual image of the patient's diseased, damaged, or post-operative joint and details of the surgical plan, such as an image of the patient's bone and / or soft tissue derived from the patient's imaging data, a generated model of the bone or soft tissue, a 3D virtual image of the prosthetic implant components selected for the surgical plan, a 3D virtual image of entry points for locating the prosthetic components for proper orientation and positioning of the prosthetic components, surgical guides, and instruments and their placement on the damaged joint during the surgical procedure, a 3D virtual image of alignment axes and cutting planes for aligning cutting or reaming tools to shape bone planes or a 3D virtual image of a drilling tool that defines one or more holes in a bone surface, and any other information that the surgeon may use to implement the surgical plan, etc. The MR system 212 is capable of generating an image of this information that a user of the visualization device 213 is capable of perceiving before and / or during the surgical procedure.

[0085] In some embodiments, the MR system 212 includes multiple visualization devices (e.g., multiple instances of the visualization device 213) so that multiple users can see the same image and share the same 3D scene at the same time. In some of these embodiments, one visualization device can be designated as a master device and other visualization devices can be designated as observers or spectators. Users of the MR system 212 can re-designate any observer device as a master device at any time as needed.

[0086] Figure 3 is a flow chart showing exemplary stages of a surgical procedure lifecycle 300. Figure 3In an embodiment of the invention, the surgical life cycle 300 begins with a preoperative phase (302). During the preoperative phase, a surgical plan is developed. The preoperative phase is followed by a manufacturing and delivery phase (304). During the manufacturing and delivery phase, patient-specific items such as parts and equipment required to execute the surgical plan are manufactured and delivered to the surgical site. In some embodiments, patient-specific items do not need to be manufactured to execute the surgical plan. The intraoperative phase (306) follows the manufacturing and delivery phase. The surgical plan is executed during the intraoperative phase. In other words, one or more people perform surgery on the patient during the intraoperative phase. The intraoperative phase is followed by a postoperative phase (308). The postoperative phase includes activities that occur after the surgical plan is completed. For example, the patient may be monitored for complications during the postoperative phase.

[0087] As described in the present disclosure, the orthopaedic surgical system 100 may be used in one or more of a preoperative stage 302, a manufacturing and delivery stage 304, an intraoperative stage 306, and a postoperative stage 308. Figure 1 ). For example, the virtual planning system 102 and the planning support system 104 may be used in the preoperative stage 302. In some embodiments, the preoperative stage 302 may include a system that analyzes patient imaging data, models bone and / or soft tissue, and / or determines or recommends a type of surgical treatment based on the patient's health. The manufacturing and delivery system 106 may be used in the manufacturing and delivery stage 304. The intraoperative guidance system 108 may be used in the intraoperative stage 306. Figure 3 It can be used in multiple stages Figure 1 For example, the medical education system 110 may be used in one or more of the preoperative stage 302, the intraoperative stage 306, and the postoperative stage 308; the preoperative and postoperative monitoring system 112 may be used in the preoperative stage 302 and the postoperative stage 308. The predictive analysis system 114 may be used in the preoperative stage 302 and the postoperative stage 308. Figure 3 There may be various workflows during a surgical procedure. For example, Figure 3 Different workflows within a surgical procedure may be appropriate for different types of surgical procedures.

[0088] Figure 4 is a flow chart illustrating an exemplary preoperative, intraoperative, and postoperative workflow in support of a plastic surgery procedure. Figure 4In an embodiment of the present invention, the surgical procedure begins with a medical consultation (400). During the medical consultation (400), the medical professional assesses the patient's medical health. For example, the medical professional may question the patient regarding the patient's symptoms. During the medical consultation (400), the medical professional may also discuss various treatment options with the patient. For example, the medical professional may describe one or more different surgical procedures that address the patient's symptoms.

[0089] Further, Figure 4 Embodiments in the present invention include a case creation step (402). In other embodiments, the case creation step is performed before the medical consultation step. During the case creation step, a medical professional or other user creates an electronic case file for a patient. The electronic case file for a patient may include information related to the patient, such as data about the patient's symptoms, patient range of motion observations, data about the patient's surgical plan, medical images of the patient, notes about the patient, billing information about the patient, and the like.

[0090] Figure 4 The embodiment in includes a preoperative patient monitoring stage (404). During the preoperative patient monitoring stage, the patient's symptoms can be monitored. For example, the patient may suffer from pain associated with arthritis in the patient's shoulder. In this embodiment, the patient's symptoms may not yet rise to the level of requiring arthroplasty to replace the patient's shoulder. However, arthritis typically worsens over time. Accordingly, the patient's symptoms can be monitored to determine whether it is time to perform surgery on the patient's shoulder. Observations from the preoperative patient monitoring stage can be stored in the patient's electronic medical record file. In some embodiments, the predictive analysis system 114 can be used to predict when a patient may need surgery, predict delaying a treatment process or avoiding surgery, or make other predictions about the patient's health.

[0091] In addition, Figure 4 In an embodiment, a medical image acquisition step (406) is performed during the preoperative phase. During the image acquisition step, a medical image of the patient is generated. The medical image of a particular patient can be generated in a variety of ways. For example, the image can be generated using a computed tomography (CT) process, a magnetic resonance imaging (MRI) process, ultrasound processing, or other imaging process. The medical image generated during the image acquisition step includes an image of an anatomical structure of interest for the particular patient. For example, if the patient's symptoms involve the patient's shoulder, a medical image of the patient's shoulder can be generated. The medical image can be added to the patient's electronic medical record. The medical professional can use the medical image in one or more of the preoperative, intraoperative, and postoperative phases.

[0092] Further, in Figure 4In the embodiment of the present invention, an automatic processing step (408) can be performed. During the automatic processing step, the virtual planning system 102 ( Figure 1 ) can automatically develop a preliminary surgical plan for the patient. For example, the virtual planning system 102 can generate a model or representation of the patient's bones and / or soft tissues. Based on these representations, the virtual planning system 102 can determine bone and / or soft tissue characteristics such as soft tissue volume, fat infiltration of muscles, muscle atrophy rate, and range of motion of bones. The virtual planning system 102 can determine what type of treatment should be performed based on these characteristics (e.g., whether the shoulder replacement is an anatomical replacement or a reverse replacement). In some embodiments of the present disclosure, the virtual planning system 102 can use machine learning techniques to develop a preliminary surgical plan based on information in the patient's virtual case file.

[0093] Figure 4 Embodiments also include a manual correction step (410). During the manual correction step, one or more human users can review and correct the determinations made during the automatic processing step. In some embodiments of the present disclosure, one or more users can use a mixed reality or virtual reality visualization device during the manual correction step. In some embodiments, the changes made during the manual correction step can be used as training data to improve the machine learning techniques applied by the virtual planning system 102 during the automatic processing step.

[0094] The virtual planning step (412) may be performed at Figure 4 After the manual correction step in the virtual planning step, the medical professional can develop a surgical plan for the patient. In some embodiments of the present disclosure, one or more users can use a mixed reality or virtual display visualization device during the process of developing a surgical plan for the patient.

[0095] Further, in Figure 4 In some embodiments, intraoperative guidance (414) may be generated. Intraoperative guidance may include guiding the surgeon on how to perform the surgical plan. In some embodiments of the present disclosure, the virtual planning system 102 may generate at least a portion of the intraoperative guidance. In some embodiments, the surgeon or other users may contribute to the intraoperative guidance.

[0096] In addition, Figure 4 In the embodiment of the present invention, the step of selecting and manufacturing a surgical item (416) is performed. During the step of selecting and manufacturing a surgical item, the manufacturing and delivery system 106 ( Figure 1) can manufacture surgical items used in the surgical procedures described by the surgical plan. For example, surgical items may include surgical implants, surgical tools, and other items required to perform the surgical procedures described by the surgical plan.

[0097] exist Figure 4 In an embodiment of the present invention, the intraoperative system 108 ( Figure 1 ) to perform the surgical procedure (418). For example, the surgeon may perform the surgical procedure while wearing a head-mounted MR visualization device of the intraoperative system 108 that presents guidance information to the surgeon. The guidance information may help guide the surgeon through the surgical procedure, thereby providing guidance for each step in the surgical workflow, including the order of steps in each step of the surgical procedure workflow, details of each step, and tool or implant selection, implant placement and positioning, and providing bone surface preparation for each step in the surgical workflow.

[0098] After the surgical procedure is completed, postoperative patient monitoring (420) may be performed. During the postoperative patient monitoring step, the patient's recovery outcomes may be monitored. Recovery outcomes may include relief of symptoms, range of motion, complications, performance of implanted surgical items, etc. The preoperative and postoperative monitoring system 112 ( Figure 1 ) can aid in the postoperative patient monitoring step.

[0099] Figure 4 The medical consultation, case creation, preoperative patient monitoring, image acquisition, automatic processing, manual correction, and virtual planning steps in Figure 3 Part of the preoperative phase 302. Figure 4 The guided steps in the surgical procedure are Figure 3 Part of the intraoperative phase 306. Figure 4 The steps in postoperative patient monitoring are Figure 3 Part of the postoperative phase 308.

[0100] As described above, one or more subsystems of the orthopaedic surgical system 100 may include, for example, an MR system 212 ( Figure 2 ) or one or more mixed reality (MR) systems. Each MR system may include a visualization device. For example, in Figure 2In an embodiment of the present invention, the MR system 212 includes a visualization device 213. In some embodiments, in addition to including a visualization device, the MR system may also include external computing resources that support the operation of the visualization device. For example, the visualization device of the MR system may be communicatively connected to a computing device (e.g., a personal computer, a laptop, a tablet computer, a smartphone, etc.) that provides external computing resources. Alternatively, sufficient computing resources may be provided on or within the visualization device 213 to perform the necessary functions of the visualization device.

[0101] The virtual planning system 102 and / or other systems may analyze patient imaging data that is also used in planning surgical interventions, such as joint surgery. As discussed herein, as an example, shoulder replacement surgery is one type of surgical procedure that may be planned using the systems and techniques herein. Figure 5A and Figure 5B is an illustration of exemplary muscles and bones associated with a patient's shoulder.

[0102] like Figure 5A 500 includes a sternum 502, a shoulder 504, and ribs 506. Some bones associated with the structure and function of the shoulder 504 include the scapular process 510 and the scapula 512 (not shown in their entirety). Muscles associated with the shoulder 504 include the serratus anterior 508, the teres major, and the biceps 518. The subscapularis 514 is Figure 5A One of the rotator cuff muscles is shown in . Figure 5B In the embodiment of FIG. 5 , the other rotator cuff muscles, the supraspinatus 526 , the infraspinatus 530 , and the teres minor 532 are shown in a posterior view of the patient 500 . Figure 5B Also shown are bony features of the spine including the humeral head 520 and scapula 528. Other muscles associated with the shoulder 504 include the triceps 522 and deltoid 524.

[0103] When evaluating the shoulder 504 for treatment, such as what type of shoulder treatment or replacement might be appropriate, the system may make decisions such as Figure 5A and Figure 5B The patient-specific imaging data of the bones and soft tissues discussed in the virtual planning system 102 are analyzed. For example, the virtual planning system 102 can generate a representation of the soft tissue (e.g., muscle) from the patient imaging data and determine various characteristics of the soft tissue. These characteristics may include muscle volume associated with the muscle, fat infiltration (e.g., fat ratio), muscle atrophy rate, and range of motion of the joint.

[0104] From this information, virtual planning system 102 can determine the treatment type suggested, such as whether the patient benefits from anatomical shoulder replacement or reverse shoulder replacement etc.In anatomical shoulder replacement, utilize artificial humeral head (for example, partial sphere) to replace humeral head, and utilize the shoulder acetabular surface of the artificial curved surface that cooperates with artificial humeral head to replace scapula.In reverse shoulder replacement, artificial partial sphere is implanted into shoulder acetabular surface and implants the artificial curved surface (for example, cup body) that cooperates with sphere to replace bone head.Virtual planning system 102 can also suggest scale and / or the placement of implant based on patient imaging data and / or muscle feature.

[0105] In one embodiment, a system such as virtual planning system 102 can be configured to model the soft tissue structure of a patient. Virtual planning system 102 may include a memory and processing circuitry configured to store patient-specific image data of a patient. The processing circuitry may be configured to receive patient-specific image data (e.g., CT data), determine a patient-specific shape representing the patient's soft tissue structure based on the intensity of the patient-specific image data, and output the patient-specific shape. Thus, the patient-specific shape may be a model of the patient's actual soft tissue structure.

[0106] The virtual planning system 102 can generate a patient-specific shape of a soft tissue structure using various methods. For example, the processing circuit can be configured to receive an initial shape (e.g., a geometric shape or a statistical average shape based on a patient population) and determine a plurality of surface points on the initial shape. Then, the virtual planning system 102 can align the initial shape with the patient-specific image data (e.g., placing the initial shape into the patient-specific image data based on muscle insertion points on adjacent bones) and identify one or more contours in the patient-specific image data, which represent the boundaries of the patient's soft tissue structure. The one or more contours can be voxels or pixels within the patient-specific imaging data that have an intensity exceeding a threshold indicating the boundary of the soft tissue structure. In some embodiments, the contour can be determined by identifying a separation zone between adjacent soft tissue structures (e.g., using a Hesse feature image representing an intensity gradient within the patient-specific image data). In contrast to identifying structure boundaries based only on very similar intensities between muscles (e.g., and fat tissue), identifying a Hesse feature image of separation zones between adjacent structures can improve the accuracy between these structure boundaries. Then, the virtual planning system 102 iteratively moves a plurality of surface points toward the corresponding positions of the one or more contours to change the initial shape into a patient-specific shape representing the patient's soft tissue structure. In this way, each iterative movement causes the modified initial shape to become increasingly similar to the actual shape of the patient's soft tissue structure as indicated in the image data.

[0107] In some embodiments, the virtual planning system 102 can display the patient-specific shape that has been modeled using the imaging data. The virtual planning system 102 can also perform additional determinations as part of the surgical planning. For example, the virtual planning system 102 can use the patient-specific imaging data to determine a fat volume ratio of the patient-specific shape, determine a shrinkage rate of the patient-specific shape, determine a range of motion of the patient's humerus based on the fat volume ratio and shrinkage rate of the patient-specific shape of the patient's soft tissue structure, and then determine one of a plurality of types of shoulder treatment procedures for the patient based on the range of motion of the humerus.

[0108] The virtual planning system 102 can determine the range of motion of the humerus of the patient by determining the range of motion of the humerus of the patient based on the fat volume ratio and atrophy rate of one or more muscles of the patient's rotator cuff. Based on this information, the virtual planning system 102 can select the type of shoulder treatment from one of the anatomical shoulder replacement surgery or the reverse shoulder replacement surgery. In some embodiments, for the situation when the patient's bones and / or muscles cannot support the anatomical shoulder replacement, the virtual planning system 102 may suggest the reverse shoulder replacement surgery. In this way, the patient determined to have greater fat infiltration and greater atrophy rate may be more suitable for reverse shoulder replacement (for example, compared with one or more appropriate thresholds). In some embodiments, the planning system 102 can adopt a decision tree or a neural network and use fat infiltration values ​​and other parameters such as patient age, gender, activity, and / or other factors that may indicate that the patient is more suitable for reverse or anatomical shoulder replacement as input. In some embodiments, the fat infiltration value can be a type of quality metric for soft tissue structures such as muscles. In other embodiments, the quality of the muscle can be represented by other types of values ​​of the presence of fat that may or may not be incorporated into the muscle.

[0109] Figure 6 is a block diagram illustrating exemplary components of a system 540 configured to determine soft tissue structure dimensions and other information related to surgical intervention associated with a joint according to an embodiment of the present disclosure. The system 540 may be used with Figure 1 The virtual planning system 102 in and / or a system configured to perform the processes discussed herein are similar. Figure 6 In the embodiment of FIG. 5 , system 514 includes processing circuitry 542, power supply 546, display device 548, input device 550, output device 552, storage device 554, and communication device 544. Figure 6In some embodiments, the display device 548 can display an image to present a user interface such as an opaque or at least partially transparent screen to the user. The display device 548 can present visual information, and in some embodiments, present audio information or other information to the user. For example, the display device 548 may include one or more speakers, a tactile device, etc. In other embodiments, the output device 552 may include one or more speakers and / or a tactile device. The display device 548 may include an opaque screen (e.g., an LCD or LED display). Alternatively, the display device 548 may include an MR visualization device (e.g., including a see-through holographic lens) and a device that allows the user to see real-world objects in the real-world environment through the lens and also through, for example, a device such as Microsoft HOLOLENS TM The user interface can be used to present a virtual image of a surgical procedure plan for a particular patient.

[0110] In some embodiments, the user can interact with the system 540 and control the system 540 in various ways. For example, the input device 550 may include one or more microphones and associated speech recognition processing circuits or software, which can recognize the voice commands issued by the user, and responsively perform various arbitrary operations such as the selection, activation, or disabling of various functions associated with surgical planning, intraoperative guidance, etc. As another embodiment, the input device 550 may include one or more cameras or other optical sensors that detect and interpret gestures to perform the operations described above. As another embodiment, the input device 550 includes one or more devices that sense the gaze direction and perform various operations as described elsewhere in this disclosure. In some embodiments, the input device 550 can receive manual input from the user via a handheld controller that includes, for example, one or more buttons, keypads, keyboards, touch screens, joysticks, trackballs, and / or other manual input media, and perform the various operations described above in response to manual user input.

[0111] The communication device 544 may include one or more circuits or other components that facilitate data communication with other devices. For example, the communication device 544 may include one or more physical drives (e.g., DVD, Blu-ray, or Universal Serial Bus (USB) drives) that allow data to be transferred between the system 540 and the drive when physically connected to the system 540. In other embodiments, the communication device 544 may include. The communication device 544 may also support wired and / or wireless communication with another computing device and / or network.

[0112] The storage device 544 may include one or more memories and / or repositories that store data of corresponding types in common and / or independent devices. For example, the user interface module 556 may include instructions defining how the system 540 controls the display device 548 to present information to the user. The preoperative module 558 may include instructions for analysis of patient data such as imaging data and / or determination of treatment options based on the patient data. The intraoperative module 560 may include instructions defining how the system 540 operates when providing information such as details about a planned surgical procedure and / or feedback about a surgical procedure to a clinician for display.

[0113] The surface fitting module 562 may include instructions defining how the processing circuit 542 determines a representation of soft tissue (e.g., a patient-specific shape) from patient-specific imaging data. For example, the surface fitting module 564 may specify an initial shape, a number of iterations, and other details about adjusting the initial shape to a patient-specific shape based on the intensity of the patient imaging data. The image registration module 564 may include instructions defining how to align the initial shape or other anatomical structures with the patient image data. For example, before generating a patient-specific shape in the surface fitting process, the image registration module 564 may instruct the processing circuit 542 how to align the statistical mean shape (SMS) (e.g., an anatomical shape derived from a multi-person population) with the skeleton of the patient imaging data. The patient data 566 may include any type of patient data, such as patient imaging data (e.g., CT scans, X-ray scans, or MRI data), patient characteristics (e.g., age, height, weight), patient diagnosis, patient health, previous surgical operations or implants, or any other information related to the patient, etc.

[0114] As discussed above, the surgical life cycle 300 may include a preoperative phase 302 ( Figure 3). One or more users may use the orthopedic surgery system 100 in the preoperative stage 302. For example, the orthopedic surgery system 100 may include a virtual planning system 102 (which may be similar to the system 540) that facilitates one or more users to generate a virtual surgical plan customized for the anatomical structure of interest of a particular patient. As described herein, the virtual surgical plan may include a 3-dimensional virtual model corresponding to the anatomical structure of interest of the particular patient and a 3-dimensional model of one or more prosthetic components matched to the particular patient to repair the anatomical structure of interest or selected to repair the anatomical structure of interest. The virtual surgical plan may also include a 3-dimensional virtual model of guidance information to guide the surgeon when performing the surgical procedure, for example, when preparing a bone surface or tissue and placing implantable prosthetic hardware relative to the bone surface or tissue.

[0115] As discussed herein, the system 540 can be configured to model the soft tissue structure of the patient using the patient imaging data. For example, the system 540 can include a memory (e.g., a storage device 554) configured to store patient-specific image data (e.g., patient data 566) of the patient. The system 540 also includes a processing circuit 542 configured to receive the patient-specific image data and determine a patient-specific shape representing the soft tissue structure of the patient based on the intensity of the patient-specific image data. The processing circuit 542 can then output the patient-specific shape, such as for display or use in further analysis of the patient. For example, the processing circuit 542 can use the patient-specific shape or other features from the patient-specific image data to generate a surgical procedure recommendation described herein (e.g., which type of treatment should be performed on the patient).

[0116] The processing circuit 542 may determine the patient-specific shape using one or more processes. For example, the processing circuit 542 may receive an initial shape (e.g., a geometric shape or SMS), determine a plurality of surface points on the initial shape, and register the initial shape with the patient-specific image data. The processing circuit 542 may register the initial shape by determining one or more muscle insertion points and / or origins on a pre-segmented bone in the patient-specific image data or by otherwise identifying the approximate location of the soft tissue structure of interest. Then, the processing circuit 542 may identify one or more contours representing the boundaries of the patient's soft tissue structures (which may be based on separation zones between soft tissue structures) in the patient-specific image data and iteratively move a plurality of surface points toward the corresponding positions of the one or more contours to change the initial shape into a patient-specific shape representing the patient's soft tissue structures. In this way, when the boundaries of the initial shape are iteratively moved toward a closer fit to the contour, the processing circuit 542 may generate one or more intermediate shapes. The contour may represent a set of voxels that exceed a certain threshold or fall within a threshold range and indicate the boundaries of the soft tissue structure.

[0117] In some embodiments, the initial shape and the patient-specific shape are three-dimensional shapes. However, in other embodiments, the initial shape and / or the patient-specific shape can be defined as two-dimensional. In these embodiments, a collection of several two-dimensional shapes can be used to define the entire volume, or the three-dimensional shape. In one embodiment, the processing circuit 542 can iteratively move the surface points of the initial shape and the intermediate shape in the direction of each vector of three dimensions so that the processing circuit 542 processes the data in the three-dimensional space. In other embodiments, the processing circuit 542 can operate in two-dimensional slices to change the initial shape toward the contour in the patient-specific image data. Then, the processing circuit 542 can combine several two-dimensional slices to generate a full three-dimensional volume of the final patient-specific shape of the patient.

[0118] Soft tissue structures may include muscles, tendons, ligaments, or other connective tissues other than bones. Even though joint replacement therapy may typically involve the transformation of bones (e.g., replacing at least a portion of bones with artificial materials such as metals and / or polymers), however, the soft tissue state may inform what type of replacement may be suitable for a particular joint being replaced. In this way, system 540 can analyze the soft tissues of patients such as muscles around joints to obtain information that may affect the type of joint replacement. In the case of shoulder replacement, the soft tissue structures of interest of the joint may include rotator cuff muscles such as the subscapularis, supraspinatus, infraspinatus, and teres minor. Other muscles associated with the shoulder such as the teres major, deltoid, serratus anterior, triceps, and biceps may be analyzed for shoulder replacement therapy. For the purpose of surgical planning, system 540 may determine the individual features of each soft tissue structure for the purpose of determining what type of range of motion and / or stress the newly repaired joint can withstand.

[0119] In some embodiments, processing circuit 542 can determine the type of shoulder treatment for the patient based on various criteria, such as the range of motion of the humerus relative to the acetabular surface of the scapula or the rest of the scapula. The type of shoulder treatment can include anatomical shoulder replacement or reverse shoulder replacement, and processing circuit 542 can suggest what type of replacement is preferred for the patient based on soft tissue features. In addition, processing circuit 542 suggests other parameters for treatment, such as implant placement position, angle, orientation, type of implant, etc. For example, processing circuit 542 can determine the fat volume ratio (e.g., fat penetration value) of the patient's specific shape from patient-specific image data. Processing circuit 542 can also determine the atrophy rate of the patient's specific shape based on the assessed pre-disease state or previous health state of the soft tissue structure of interest. Then, processing circuit 542 can determine the range of motion of the humerus of the patient based on the fat volume ratio and atrophy rate of the patient's specific shape of the soft tissue structure of the patient. For example, the higher fat penetration and atrophy of the muscle may indicate that the range of motion of the joint is lower (or narrower). In some embodiments, the range of motion can be one or more specific angles of the corresponding movement of the joint or a measure or other composite value representing the overall range of motion of the joint. Processing circuit 542 can determine the range of motion of the joint based on several muscles. For example, processing circuit 542 can determine one or more range of motion values ​​(e.g., one or more individual angles or one or more composite values ​​representing the entire range of motion) of the humerus relative to the scapula based on the fat ratio and atrophy rate of several corresponding muscles of the rotator cuff and / or other muscles or connective tissues associated with the shoulder joint. In addition, the range of motion may be affected by bone-to-bone collisions or other mechanical impacts. From this information, processing circuit 542 can recommend the type of shoulder treatment for the patient during the preoperative planning phase.

[0120] Figures 7 to 18B Exemplary steps involved in modeling a patient's soft tissue structure from patient-specific image data are shown. The processing circuitry 542 of system 540 is described as an exemplary system for performing these processes, however, other devices, systems, or combinations thereof may perform similar determinations. Figure 7 is a diagram of exemplary insertion points for some of the muscles of the rotator cuff.

[0121] like Figure 7 In the embodiment shown in, processing circuit 542 can start soft tissue modeling by making initial shape and patient's associated bone registration.Shoulder socket 578 of scapula is shown together with humeral head 520 of humerus 570.Can determine patient's bone from automatic segmentation process, in this process, processing circuit 542 or other systems determine bone based on the intensity value of patient-specific image data.From these bones, processing circuit 542 can identify the insertion point or attachment point (or other origin of muscle) of one or more soft tissue structures of interest of shoulder joint.In other embodiments, can directly determine insertion point or other bone mark from patient-specific image data rather than bone segmentation.For example, insert supraspinatus 572 to indicate the position that supraspinatus is attached to humeral head 520, insert infraspinatus 574 to indicate the position that infraspinatus is attached to humeral head 520, and insert scapula 576 to indicate the position that scapula muscle is attached to humeral head 520.Processing circuit 542 can identify each insertion point in these insertion points based on comparison with anatomical diagram or other instructions based on general human anatomy. In some embodiments, the processing circuit 542 may determine additional insertion points on the scapula or other bones as additional points for aligning the initial shape with the patient's anatomy.

[0122] As discussed herein, the initial shape that is converted and fitted to the patient's image data can start as a geometric shape or a more specific SMS. The SMS can be selected for a general group or from a plurality of different SMS based on one or more demographic factors of the patient group (e.g., gender, age, race, etc.). In some embodiments, the SMS can be used because the SMS may more closely match the patient's muscles. Thus, the processing circuit 542 can reduce the number of iterations or calculations required to modify the initial shape and generate a patient-specific shape that fits the image data. In addition, the SMS can include a pre-identified position that matches an identified insertion point on the associated bone.

[0123] Fig. 8A is a conceptual diagram of exemplary patient-specific image data. Fig. 8A, a two-dimensional image data 600 showing the intensity of a CT, or x-ray image is shown. Higher intensities represent dense tissues such as bones in the form of scapula 602 that absorb x-ray energy. Typically, soft tissue structures such as muscles are more difficult to identify in CT data because tissues absorb x-ray energy at similar low levels. For example, the infraspinatus muscle can have a boundary defined in part by edge 601 (i.e., the edge of the scapula 602) and edge 603 (i.e., the separation zone between the outer edge of the infraspinatus muscle and the adjacent layer of fat tissue). Although edge 603 may be difficult to identify from the x-ray data, however, specific data processing techniques can help identify edge 603 or identify the separation zone from which edge 603 can be determined.

[0124] Figure 8B is based on Fig. 8A Conceptual diagram of a Hesse feature image 605 generated based on patient-specific image data in FIG. Figure 8B In the embodiment shown in FIG. 1 , the processing circuit 542 may be configured to receive a signal from a processor such as Fig. 8A A Hesse signature image 605 is determined from patient-specific CT data such as image data 600. The Hesse signature image 605 is shown as a two-dimensional image, however, it may be part of a three-dimensional data field. The Hesse signature image 605 indicates an area of ​​the patient-specific CT data, the area including a higher intensity gradient between two or more voxels in the patient-specific CT data. For example, the processing circuit 542 may calculate second-order derivatives between adjacent voxels, or groups of voxels, to determine the gradients between the voxels or groups of voxels, and then generate a Hesse signature image based on the second-order derivatives. Although the Hesse signature image is described as a three-dimensional image, the same technique may also be used to generate a two-dimensional image.

[0125] Because there are variations in intensity between the voxels of these structures, such as between a bone object and soft tissue, between two groups of bone objects that are close to each other, and between two groups of soft tissue that are close to each other, the Hesse feature image can show the separation between two anatomical objects. The processing circuit 542 can determine one or more contours, or at least a portion of a contour, based on the voxel-based separation information (for example, based on the Hesse feature image).

[0126] Although the interval between the contour indication anatomical objects, however, the contour may not provide the complete shape of the anatomical object.For example, due to lack of intensity gradient or noise between the imperfect imaging, voxel or voxel group, there may be holes, gaps or other errors causing the discontinuity of the contour representing the boundary of anatomical structure (for example, bone or soft tissue).As an embodiment, the contour may be incomplete and may not be the closed surface representing the anatomical object.Usually, the contour can provide the initial estimation of the size, shape and position of the anatomical object.However, as described in more detail, because the contour may be imperfect based on image information and 2D scanning, the contour may be the inaccurate indicator (for example, due to the projection in the hole or other missing parts and the contour) of the actual anatomical object.Therefore, the processing circuit 542 can use the initial shape with closed surface and the initial shape is modified to approach the contour defined at least by the Hesse feature image part.

[0127] The Hesse feature image 605 includes several lines of various intensities (e.g., white represents areas where the gradient between voxels is higher than dark or black areas). Whiter and wider lines represent areas of higher gradient. The clavicle 602 can be identified as the lines around the outer surface of the clavicle 602, i.e., representing larger gradients between voxels with higher intensity from the bone and those voxels with lower intensity from the soft tissue. Figure 8B As shown in , the infraspinatus muscle can have a boundary determined in part by contour 607A (i.e., the edge of the scapula 602) and contour 607B (i.e., the separation zone between the outer edge of the infraspinatus muscle and the adjacent fat tissue layer). Each of contours 607A and 607B can be identified as extending through the middle of the separation zone indicated by the line of the Hessian feature image 605. As such, contours 607A and 607B can indicate the correspondence between adjacent structures near the boundary of each structure.

[0128] Each of the contours 607A and 607B may be discontinuous, however, in other embodiments, these contours may be continuous separation zones. Thus, contours 607A and 607B may constitute at least a portion of a contour representing the boundary of the infraspinatus muscle. In some embodiments, other separation zones may be used to provide contours for other muscles. For example, the Hesse feature image 605 may represent skin boundaries such as contour 607C, which may be used to identify the boundaries of these muscles (which may typically be located adjacent to the skin with minimal fat tissue between the muscle and the skin).

[0129] Based on the Hesse feature image 605, the processing circuit 542 can identify one or more separation zones between the soft tissue structure and the adjacent soft tissue structure. In other words, the separation zone can indicate the intensity gradient between the two soft tissue structures in the patient-specific image data. Therefore, the processing circuit 542 can determine that at least a portion of the one or more contours passes through the one or more separation zones. The processing circuit 542 can determine that the contour passes through the middle of the separation zone or through the intensity-based weighted middle of the separation zone.

[0130] Figure 8C 604 and an exemplary segmentation contour superimposed on patient-specific image data. Figure 8C , a two-dimensional image data 600 is shown illustrating a process of modeling a soft tissue structure according to the exemplary process described herein. Although the soft tissue structure of the infraspinatus muscle is shown as an example, the same process can be performed on any soft tissue structure. Figure 8C Two-dimensional images are shown in FIG. 5 for illustrative purposes, however, processing circuitry 542 may perform these processes in three-dimensional space.

[0131] An initial shape, such as initial shape 604 shown as a dashed line, is selected for the soft tissue structure of interest (i.e., the infraspinatus muscle in this embodiment). The initial shape 604 is then registered to one or more associated bones including the scapula 602. For example, an insertion point identified on the scapula 602 can be used to match a corresponding attachment point identified from the initial shape 604. The initial shape 604 can be an SMS, i.e., an anatomical shape representing the soft tissue structure of multiple subjects different from the patient. Because the SMS is specific to the muscle of interest, the initial shape 604 may be similar to the patient's structure. However, as Figure 8C As shown in , initial shape 604 does not accurately reflect the boundary of scapula 602 or other intensities of image data 600. Therefore, processing circuitry 542 may deform or modify initial shape 604 to fit the image data of this particular patient.

[0132] The processing circuit 542 can move portions of the initial shape 604 toward the actual soft tissue structure of the patient represented by the segmentation contour 606 (e.g., similar to the synthesized patient-specific shape) through one or more iterations. As shown by the arrow groups 608A, 608B, and 608C (collectively, "arrow groups 608"), portions of the initial shape 604 are deformed toward other corresponding portions of the segmentation contour 606. Figure 8CEach arrow in the corresponding arrow group shown moves the corresponding surface point on the surface of the initial shape 604 toward the segmentation contour 606 by at least a portion of the distance during one iteration. In this way, after two or more iterations, the processing circuit 542 can deform the initial shape 604 to fit the segmentation contour 606. For example, a portion of the segmentation contour 606 is moved in the direction of the arrow group 608B to fit a portion of the scapula 602. The portion of the initial shape 604 that is also moved in the direction of the arrow group 608A and the arrow group 608C is now fit to the contour of the soft tissue structure indicated by the intensity of the image data 600. In some embodiments, this process can be referred to as closed surface fitting.

[0133] In some embodiments, the processing circuit 542 may receive a segmentation contour 606, i.e., an initial analysis of the patient-specific image data, to generate a representation of a soft tissue structure of interest such as the infraspinatus muscle. For example, the segmentation contour 606 may be a three-dimensional shape or a plurality of two-dimensional slices. However, the segmentation contour 606 may not be a complete model representing the soft tissue structure. For example, individual voxels in the image data may be inaccurate or missing, and the synthesized segmentation contour 606 may be incomplete. For example, the segmentation contour 606 may be at least partially determined by a contour identified from a separation zone according to a Hessian feature image. Therefore, deforming a fully enclosed SMS or geometric shape to fit the image data 600 may result in a complete model of the soft tissue structure.

[0134] In other embodiments, the processing circuit 542 may not receive any segmented contours 606 of the soft tissue structure. Instead, the processing circuit 542 may place the initial shape according to the insertion points on the associated bones within the patient-specific image data. The processing circuit 542 may then extend vectors from various locations on the surface of the initial shape toward corresponding voxels, or otherwise indicate the edges of the soft tissue structure or bones in the image data 600, which exceed a threshold intensity, differ from surrounding voxels by a predetermined value or a predetermined percentage. For example, these voxels exceeding the threshold intensity may together constitute one or more contours of the soft tissue structure represented by the patient-specific image data. Alternatively, as discussed herein, the contour representing the edge of the soft tissue structure may be determined based on a gradient between voxels in an image such as a Hessian feature. Therefore, after one or more iterations of deforming the initial shape 604 toward an identified voxel that exceeds a threshold or exceeds a relative change in voxel intensity, the initial shape 604 may then be converted into a patient-specific shape representing the patient's soft tissue structure. Thus, the patient-specific shape may be a model of the patient's structure, and various features such as volume, length, etc. may be calculated by the processing circuit 542. Figures 9 to 11 An exemplary process for determining a patient-specific shape from patient-specific image data by iteratively moving points on the surface of an initial shape is shown.

[0135] Fig. 9 is a conceptual diagram of an exemplary procedure for changing an initial shape 616 toward a patient-specific shape 614 representing the patient's soft tissue structure. Fig. 9 610. The embodiment in the figure shows a sagittal view 610 of the patient image data, the initial shape 616, and the scapula 612. The initial shape 616 can be placed and deformed into a soft tissue structure representing the subscapularis muscle, that is, it can be similar to the outline 614 represented by the dotted line. The processing circuit 542 aligns the initial shape 616 in the image data by aligning multiple positions on the initial shape 616 with corresponding insertion positions on one or more bones identified in the patient image data. For example, the initial shape 616 can be aligned with the insertion position on the scapula 612. This alignment may not require that any part of the initial shape 616 actually contacts the scapula 612, but the alignment uses the insertion position on the scapula 612 as a guide for aligning the initial shape 616 in the patient-specific image data. The initial shape 616 is shown as an SMS to approach the patient's structure. In other embodiments, geometric shapes such as a sphere, an ovoid, or other structures can be used as the initial shape 616.

[0136] Because the initial shape 616 needs to be deformed or modified, the processing circuit 542 can move portions of the initial shape 616 toward the voxels of the image data representing the surface of the soft tissue structure as needed. For example, the processing circuit 542 can select a plurality of surface points on the initial shape 616. Each of the vectors 618A and 618B (collectively referred to as "vectors 618") extends from a corresponding surface point on the initial shape 616 and in a direction orthogonal to the surface of the initial shape 616. The processing circuit 542 can extend the vectors 618 inside and outside the initial shape 616, which can enable the processing circuit 542 to identify the contours of the soft tissue structure located outside or inside the initial shape 616 according to the position of the initial shape 616 and the imaging data. For example, because a portion of the scapula 612 is located within the initial shape 616, the vector 618B points inward from the surface of the initial shape 616. In contrast, the vector 618A around the rest of the initial shape 616 ultimately points outward from the surface of the initial shape 616.

[0137] Thus, as indicated by the contour 614 and the surface of the scapula 612, the processing circuit 542 can use the vector 618 to identify one or more contours corresponding to the edge of the soft tissue structure. For example, the processing circuit 542 can extend the corresponding vector 618 from each surface point (e.g., based on the point of each vector 618) in at least one of an outward extension or an inward extension from the corresponding surface point of the plurality of surface points on the initial shape 616. Therefore, the processing circuit 542 can determine, for each vector of the surface point, a corresponding location in the patient-specific image data where the voxel intensity exceeds the threshold intensity value. These corresponding locations of at least one surface point of the plurality of surface points at least partially define one or more contours. In other words, for each vector in the vector 618, it can be determined that the voxel or pixel including the intensity value exceeding the threshold intensity value is part of the contour 614 or an adjacent bone of the soft tissue structure. In other embodiments, the processing circuit 542 can receive the contour 614 from a previous segmentation of the image data indicating the soft tissue structure of the patient, or can use the Hesse feature data to determine at least a portion of the contour 614 identified from the separation region in the Hesse feature image. However, the processing circuit 542 can still deform the initial shape 616 toward a known contour to generate a fully enclosed surface model of the soft tissue structure of interest. The fully enclosed surface model can also be pre-marked (e.g., the initial shape 616 can be pre-marked) as which parts should face the bone or which parts should be adjacent to another specific muscle. This marking can enable or initiate further segmentation of the soft tissue structure.

[0138] As discussed above, the position that each vector 618 is looking for in the patient-specific image can be based on the separation zone and the contour determined from the Hesse feature image. In other embodiments, the processing circuit 542 can determine the corresponding position in the patient-specific image data where the intensity of its voxel exceeds the threshold intensity value by determining the corresponding position in the patient-specific image data that is greater than or less than the predetermined intensity value. In other words, the intensity can exceed (e.g., become higher than) a high threshold intensity value indicating that the voxel represents a bone or exceed (e.g., become lower than) a low threshold intensity value indicating that the voxel represents a fluid, adipose tissue, or other tissue indicating the boundary of a soft tissue structure. For example, the threshold intensity value can represent the intensity of the bone. Because soft tissue (e.g., muscle) exists against the surface of the bone, once the vector reaches the position of the bone, the processing circuit 542 infers that the position is the boundary of the soft tissue structure. The threshold intensity value can also be set to a structure other than the muscle to identify the volume where the vector has left the soft tissue structure. In other embodiments, the threshold intensity value can be less than the expected intensity value of the muscle. For example, once the threshold intensity in the path of the vector decreases below a threshold intensity value, the processing circuit 542 may interpret the lower intensity value as fluid or other structure distinct from the soft tissue structure of interest. In some embodiments, the threshold intensity value may be a predetermined magnitude.

[0139] In other embodiments, the threshold intensity value can be a difference value calculated based on where the vector originated from and / or the previous voxel or pixel that most recently intersected the vector. In this way, the processing circuit 542 is able to identify relative changes in the intensity of the patient image data (e.g., voxel-to-voxel changes) or a filter mask that can indicate the boundary of a soft tissue structure or other structure associated with a portion of the contour 614 of the soft tissue structure of interest. For example, the processing circuit 542 can be configured to determine the corresponding location in the patient-specific image data that exceeds the threshold intensity value by determining the corresponding location in the patient-specific image data that is greater than a difference threshold, wherein the difference threshold is the difference between the intensity associated with the corresponding surface point and the intensity of the corresponding location in the patient-specific image data. In some embodiments, when analyzing image data for the boundary of the soft tissue structure of interest, the processing circuit 542 can employ one or more of these types of thresholds.

[0140] Typically, the processing circuit 542 can identify the corresponding position of the boundary of the soft tissue structure in the patient-specific image data for each vector. However, in some embodiments, the processing circuit 542 may not identify the voxel or pixel that exceeds the threshold intensity value. This problem may occur due to incomplete information in the patient-specific image data, data corruption, patient movement during data generation, or any other type of abnormality. The processing circuit 542 can use an algorithm to avoid this problem in the deformation process of the initial shape 616. For example, the processing circuit 542 can use the maximum distance in the image data, and identify the voxel or pixel that exceeds the threshold intensity value for this maximum distance. The maximum distance can be a predetermined distance or a scale distance from a point on the initial shape 616 where the vector begins. The maximum distance can be selected in the range from about 5 millimeters (mm) to about 50 mm. In one embodiment, the maximum distance can be set to about 20 mm.

[0141] If the processing circuit 542 does not identify a voxel that exceeds a threshold intensity value within the distance from the origin of the vector, the processing circuit 542 may remove the vector and the corresponding point on the surface of the initial shape 616 from the deformation process and deform only relying on other points and vectors. Alternatively, the processing circuit 542 may select a new surface point from the initial shape 616 and extend a new vector from the new surface point in an attempt to find a voxel that exceeds a threshold intensity value or to find a voxel corresponding to a contour or boundary of a soft tissue structure determined from the Hessian feature image. The new surface point on the surface of the initial shape 616 may be within or at a predetermined distance from the removed surface point, at a predetermined distance between the removed surface point and another surface point, or at some other location on the initial shape 616 selected by the processing circuit 542 to replace the removed surface point.

[0142] The processing circuit 542 may then use one or more iterations to deform the surface of the initial shape 616 so that the closed initial shape 616 approximates or fits the contour 614 and the portions of the scapula 612. During each iteration, the processing circuit 542 may move some or all of the surface points on the initial shape 616. For example, the processing circuit 542 may extend a corresponding vector 618 from a corresponding surface point of each of the plurality of surface points and the vector 618 is orthogonal to the surface of the initial shape 616 including the corresponding surface point. As discussed above, these vectors may point inward and / or outward from the surface of the initial shape 616. The processing circuit 542 may then determine a corresponding point in the patient-specific image data that exceeds a threshold intensity value for the corresponding vector 618 of each surface point. These points that exceed the threshold intensity value may form one or more contours similar to the contour 614 and / or the bone surface.

[0143] At each corresponding point of the contour 614, the processing circuit 542 can determine a plurality of potential positions that are within the envelope of the corresponding point and that exceed a threshold intensity value in the patient-specific image data. In this way, the plurality of potential positions of the single vector can at least partially define the surface of the contour 614. These potential positions within the envelope indicate potential directions in which the surface points on the initial shape 616 should move. The processing circuit 542 selects one of the potential positions to guide the movement of the corresponding surface point to resolve how the surface of the contour at the potential positions is oriented relative to the surface points on the initial shape 616. In other words, the processing circuit 542 can select such potential positions so that the difference between the orientation of the surface of the initial shape 616 and the orientation of the surface of the contour 614 at the potential positions is reduced.

[0144] For example, the processing circuit 542 may determine a corresponding normal vector orthogonal to the surface for each of the plurality of potential locations. Taking the vector 620 of the corresponding surface point on the surface of the initial shape 616 as an example, a number of normal vectors are generated from each potential location. An exemplary normal vector is a vector 622 from a potential location. Then, the processing circuit 542 may determine, for each of the corresponding normal vectors, an angle between the corresponding normal vector and the vector from the corresponding surface point. Using an embodiment of vector 624, the processing circuit 542 may determine an angle 622 between vector 624 and vector 620. This angle 622 may be referred to as a cosine angle. The processing circuit 542 may perform this calculation for each potential location corresponding to a corresponding vector 618 such as the exemplary vector 620.

[0145] The processing circuit 542 can then select, for each corresponding surface point on the initial shape 616, a potential position from the plurality of potential positions that has a minimum angle between a vector of the corresponding surface point (e.g., vector 620) and a corresponding normal vector (e.g., vector 624) of each of the plurality of potential positions. In other words, the processing circuit 542 can identify a position on the contour 614 that provides an appropriate movement of the surface point and a deformation of the surface point. The processing circuit 542 can then, for each corresponding surface point, cause the corresponding surface point to move at least partially toward the selected one of the potential positions. This movement of the corresponding surface point causes the initial shape 616 to be modified or deformed toward a patient-specific shape corresponding to the contour 614 and a portion of the scapula 612. The processing circuit 542 can repeat this process at each iteration until the initial shape 616 has been deformed to a voxel or pixel that is close to exceeding a threshold intensity value or corresponding to a previously segmented boundary of a soft tissue structure of interest.

[0146] The processing circuit 542 can cause the corresponding surface point to move at least half of the distance between the corresponding surface point and a selected potential position corresponding to the contour 614. However, in other situations or iterative processes, the distance moved can be changed. Because the deformation caused in a single step may not produce an accurate final patient-specific shape, the surface point may not be completely moved toward the potential position. In other words, the slight adjustment made to the selected position on the contour 614 based on the normal vector of each potential position in each iteration can provide a final shape that matches the contour 614 more strictly. In other words, each surface point on the initial shape 616 may not necessarily move completely in a linear direction during all iterative processes. This nonlinear movement combination of each surface point can allow the final patient-specific shape to match or fit more strictly to the contour 614 and (if applicable) adjacent bone surfaces.

[0147] like Fig.10 As shown in the sagittal view 640, Fig. 9 Initial shape 616 in FIG. 6 has been deformed into intermediate shape 632 during one iteration, and intermediate shape 632 is closer to contour 614 than initial shape 616. When processing circuit 542 determines that intermediate shape 632 is still not close to contour 614 or additional deformation is otherwise required, processing circuit 542 may perform one or more additional iterative deformations. For example, processing circuit 542 may determine vectors 634A and 634B (collectively, “vectors 634”) from corresponding surface points on intermediate shape 632. Fig. 9 Similar to the depicted vector 618 , vector 634B points inwardly toward the surface of the scapula 612 , and vector 634B points outwardly toward portions of the scapula 612 and the contour 614 .

[0148] Processing circuitry 542 then determines another set of potential locations for each vector 634 at the point at which the corresponding vector reaches a voxel or pixel that exceeds a threshold intensity value. From each potential location for each vector 634, processing circuitry 542 may select a potential location that has a normal vector with a minimum angle when compared to the vector for the surface point. For example, for an exemplary vector 636 for a surface point on intermediate shape 632, processing circuitry 542 may determine vector 637 as a vector from a potential location in contour 614 that is within the envelope of the point in contour 614 where vector 636 reaches contour 614. Processing circuitry 542 may select a potential location associated with vector 637 when angle 638 is the minimum angle between the vector for intermediate shape 632 and the normal vector for the potential location. In this way, processing circuitry 542 may select a location on contour 614 that is different from the point at which vector 634 reaches contour 614 for at least some of vectors 634. This process enables processing circuitry 542 to more closely approximate contour 614 by moving surface points on intermediate shape 632 in a direction different from the direction normal to the surface at the corresponding surface point.

[0149] Processing circuitry 542 may then cause the surface points of intermediate shape 632 to move at least partially toward the selected potential location on contour 614. For example, processing circuitry 542 may cause intermediate shape 632 to deform into Fig.11 The fully enclosed final patient-specific shape 642 is shown in FIG. Fig.11 As shown in , the patient-specific shape 642 can approximate the contour 614 and at least some surfaces of the scapula 612, against which the soft tissue structures of the patient are arranged. In some embodiments, the patient-specific shape 642 can be identical or similar to the contour 614. The contour 614 can represent an initial segmentation of soft tissue structures in the patient-specific image data and / or represent the surface of the super-threshold voxels identified from each vector. In other embodiments, the processing circuit 542 can deform the initial shape 616 more than twice before arriving at the final patient-specific shape 642. In some embodiments, the processing circuit 542 can perform a predetermined number of iterations to approximate the contours of the voxels or pixels that exceed the threshold intensity value. In other embodiments, the processing circuit 542 can continue to perform additional iterative deformations of the initial shape until a certain number, a certain percentage, or all of the surface points of the shape are within a predetermined distance of the voxels or pixels that exceed the threshold. In other words, the processing circuit 542 can continue to deform the initial shape and its intermediate shapes until the deformed shape is within a certain acceptable tolerance or error of the contour within the patient-specific image data.

[0150] In some embodiments, processing circuitry 542 may follow the same deformation instructions in each iteration. Alternatively, processing circuitry 542 may adjust one or more factors that determine how processing circuitry 542 moves surface points of a shape during iterations of the deformation process. For example, these factors may specify the number of surface points from an initial or intermediate shape, the number of potential positions identified within a contour, the size of an envelope of potential positions that can be selected for each vector of surface points, the distance each surface point can move in one iteration, the allowable deviation that each surface point can move relative to each other in one iteration, or other such factors. These types of factors can limit the extent to which processing circuitry 542 deforms an initial or intermediate shape in a single iteration.

[0151] For example, the processing circuit 542 may deform the initial shape in a more uniform manner and the intermediate shape in a less uniform manner to more closely approximate the actual dimensions of the soft tissue structure identified in the image data. In one embodiment, the processing circuit 542 may be configured to iteratively move the plurality of surface points toward corresponding potential positions of one or more contours (e.g., contour 614) by generating a second shape (e.g., intermediate shape 632) by moving each of the plurality of surface points from the initial shape 616 by a first corresponding distance in a first iteration, the first corresponding distance being within a first tolerance of the first modified distance. The first tolerance may be selected by the processing circuit 542, a user, or the first tolerance may be otherwise predetermined to maintain the smoothness of the second shape relative to the initial shape 616. In other words, the tolerance may be an allowed deviation from a value. The tolerance may be as low as zero so that all surface points must move the same distance. However, the tolerance may be larger to allow the processing circuit 542 to change the distance that each surface point can move in an iteration.

[0152] Then, the processing circuit 542 may move each of the plurality of surface points of the second shape by a second corresponding distance in a second iteration after the first iteration to generate a third shape (e.g., another intermediate shape or a final shape) from the second shape, the second corresponding distance being within a second tolerance of the second modified distance. In the second iteration, the second tolerance is greater than the first tolerance so that the change in the distance moved by each surface point can be greater than the change allowed in the previous iteration. In this way, a smaller tolerance can promote smoothness when the shape is deformed, while a larger tolerance can promote further accuracy in the degree of proximity of the next deformed shape to the soft tissue structure. Typically, the processing circuit 542 may increase the tolerance of the modified distance of the surface point between iterations. However, in some embodiments, the processing circuit 542 may switch between increasing or decreasing the tolerance, or the processing circuit 542 may maintain the same value of the tolerance for each iteration. In other words, the elasticity of the later iterations may increase so that each surface point can be moved more strictly to its corresponding point (e.g., toward the contour representing the boundary of the soft tissue structure). In addition, because the system becomes more confident in the correspondence with the contour 614, the search distance may increase for each iteration. For more confident correspondences, such as the correspondence of portions of contour 614 associated with bone structures, the system may use higher iterations that provide more elasticity or tolerance for movement of each surface point on initial shape 616. Conversely, portions of contour 614 associated with other soft tissues may require an initial iteration with less elasticity because the confidence in the correspondence with contour 614 is lower.

[0153] In some embodiments, this process of registration and modification may be similar to the B-spline algorithm. The internal parameters of the B-spline algorithm may include the spline order, the number of control points, and the number of modification iterations of the shape. The first two parameters of the spline order and the number of control points can control the "elasticity" of the algorithm. The higher the spline order and the number of control points, the more elastic behavior occurs at each surface point on the shape. In other words, as long as the confidence level of the correspondence with the contour remains high, later iterations lead to a more specific solution to modify the shape.

[0154] In one embodiment, for a first iteration, the algorithm can divide the space of the initial shape 616 into a plurality of surface points. The processing circuit 542 can determine a deformation field based on these surface points and the spline order using a spline relationship. For further iterations, the number of surface points can be doubled, tripled, etc., to enable a more specific deformation field that changes the initial shape 616. Each "iteration" can refer to an internal iteration of the B-spline algorithm. The processing circuit 542 can use an external iteration to perform registration of the initial shape or modified shape, which has "N times" of internal iterations. "N" may increase with external iterations to enable a more flexible output. An exemplary manner in which the processing circuit 542 performs B-spline registration is described in Lee et al. "Scattered Data Interpolation with Multilevel B-Splines" IEEE Transactions on Visualization and Computer Graphics, Vol. 3, No. 3, July-September, 1997. Additional exemplary ways in which b-spline registration is performed can be found at https: / / itk.org / Doxygen411 / html / classitk_1_1BSplineScatteredDataPointSetToImageFilter.html and http: / / www.insight-journal.org / browse / publication / 57.

[0155] In some embodiments, the processing circuit 542 can move the surface points of the initial shape or intermediate shape toward the contour by a larger distance or the entire distance based on the identified intensity value of the voxel or pixel at the position. For example, a voxel with a higher intensity can indicate the presence of a bone. Typically, a soft tissue structure can be arranged against a portion of a bone. Therefore, if the voxel is identified as a bone, the processing circuit 542 can directly move the corresponding surface points of the initial shape or intermediate shape to or adjacent to the identified bone structure. In other embodiments, when the bone is identified as a part of the contour, the processing circuit 542 can increase the tolerance of the modified distance so that the next iteration can be more accurately close to the contour of the bone. As discussed herein, in other embodiments, the contour 614 can be determined based on the Hessian feature image representing the separation zone between adjacent structures. In some embodiments, the processing circuit 542 can track the distribution (profile, curve) behavior of the Hessian feature image along the vector to determine the correspondence with the boundary of the soft tissue structure. The Hessian feature image can include a distribution similar to the rectangular function that provides voxels for the correspondence of the vector. For a bone structure, the processing circuit 542 may know the voxel of the bone surface so that the surface point is moved directly to the voxel.

[0156] Once the final patient-specific shape 642 is determined, the processing circuitry 542 may then output the patient-specific shape 642. In some embodiments, the processing circuitry 542 may control the patient-specific shape 642 displayed to the user. In other embodiments, the processing circuitry 542 may perform additional calculations on the patient-specific shape 642. For example, the processing circuitry 542 may determine the volume, linear dimensions, cross-sectional dimensions, or other characteristics of the patient-specific shape 642. As described herein, the processing circuitry 542 may use these features in other determinations.

[0157] Figure 12 to Figure 1 8 shows an exemplary modeling of the rotator cuff muscles based on the deformation process described herein. In some embodiments, a system such as system 540 can display similar images to a user via a user interface. Some illustrations are two-dimensional, while other illustrations are three-dimensional of the same modeled structure. Fig.12 is a conceptual illustration of an exemplary axial view 650 of patient image data including a scapula 652. An initial shape 654 represents the SMS of the scapular muscle, which has been deformed into a patient-specific shape 656 representing the patient's scapular muscle. Fig.13 is a conceptual illustration of an exemplary sagittal view 660 of patient image data including a scapula 662. An initial shape 664 represents the SMS of the supraspinatus muscle, which has been deformed into a patient-specific shape 666 representing the patient's supraspinatus muscle.

[0158] Fig.14 6 is a conceptual axial image 670 showing exemplary final patient-specific shapes 676, 678, and 680 of the three rotator cuff muscles superimposed on the patient-specific image data. Fig.15 As shown in FIG. 6 , the scapula 672 is shown relative to the humeral head 674. Specifically, the patient-specific shape 676 represents the subscapularis muscle, the patient-specific shape 678 represents the supraspinatus muscle, and the patient-specific shape 680 represents the infraspinatus muscle. Fig.15 As shown in FIG. 6 , a sagittal view 682 of the patient image data includes a scapula 673 relative to a patient-specific shape 676 (e.g., subscapularis), a patient-specific shape 678 (e.g., supraspinatus), and a patient-specific shape 680 (e.g., infraspinatus). It should be noted that additional rotator cuff muscles or other muscles associated with the joint of interest are not shown, but may be determined in other embodiments.

[0159] Fig.16A 690 is a conceptual three-dimensional view showing an example final patient-specific shape of the three rotator cuff muscles and bones from the patient-specific image data. Fig.16A, the subscapularis muscle is shown represented by a patient-specific shape 676 relative to the scapula 692 and the humeral head 694. View 690 also shows the supraspinatus muscle as a patient-specific shape 678 and the infraspinatus muscle as a patient-specific shape 680. Fig. 16B As shown in , the front view 702 is a three-dimensional representation of similar structures shown in the rear view 690, such as the subscapularis muscle as the patient-specific shape 676, the supraspinatus muscle as the patient-specific shape 678, and the infraspinatus muscle as the patient-specific shape 680. Fig.17 690, the end view 704 is a three-dimensional view of similar structures shown in the rear view 690. The end view 704 is a diagram of these shoulder structures in a plane aligned with the acetabular surface 706. Thus, the subscapularis is shown as a patient-specific shape 676, the supraspinatus is shown as a patient-specific shape 678 between the coracoid process 708 and the acromion 710, and the infraspinatus is shown as a patient-specific shape 680.

[0160] Fig.18A and Fig.18B is a conceptual diagram of exemplary patient-specific CT data in which an initial shape associated with a soft tissue structure is registered with a skeletal structure and modified to a patient-specific shape representing the soft tissue structure. For example, the processing circuit 542 may initially identify a skeletal structure within the CT data (e.g., one or more x-ray images), i.e., may be a basic landmark for registering the initial shape of the soft tissue structure and then scaling the initial shape to fit the initial shape to the CT data of the patient.

[0161] like Fig.18A As shown in the axial slice embodiment of FIG. 5 , the processing circuit 542 can determine a set of basic landmarks 713A, 713B, and 713C (collectively referred to as "landmarks 713") from the patient image based on the target muscle (e.g., soft tissue structure). For example, the subscapularis muscle is adjacent to the thorax, and therefore, the ribs may be segmented and used to locate the landmark (the ribs of landmark 713). The initial shape 715 can be a statistical mean shape (SMS) of the subscapularis muscle that is strictly aligned with the patient. The initial shape 715 can be a pathological shape, such as an SMS generated from other patients with similar health conditions as the patient. In other words, because the SMS is only aligned and scaled, the healthy SMS may not provide an appropriate model of the patient's soft tissue. However, in some embodiments, the SMS can be used based on the shape relative to the patient's soft tissue. FIG. 8A to FIG. 11 The registered and / or scaled SMS is further modified by the described closed surface modification.

[0162] Connections 714A, 714B, and 714C (collectively “connections 714 ”) indicate the correspondence between the marker 713 and the corresponding point on the SMS of the initial shape 715 . Fig.18A is an axial slice of the patient-specific CT data.

[0163] Fig.18B is a sagittal slice of the patient-specific CT data and shows a different view of the ribs and the subscapularis muscle. Single rib 716 is a landmark comprising several points that correspond to corresponding points on the initial shape 715, and connections 717 are lines indicating these correspondences. Fig.18A and Fig.18B is a two-dimensional representation of the registration process, and in some embodiments, a three-dimensional registration can be performed by the processing circuit 542. The process described with respect to the subscapularis muscle can also be performed for other muscles. For example, landmarks related to the supraspinatus muscle can be identified on the underside of the clavicle and the acromion, and landmarks related to the infraspinatus muscle can be identified based on the surrounding skin.

[0164] Typically, once a basic landmark (e.g., landmarks 713 and 716) is identified, its closest correspondence on the SMS is located. The processing circuit 542 can determine an intensity-based curve along a line connecting landmark "1" and its corresponding point on SMS 'c' (e.g., connecting 713 and 717). When a specific change in the curve is detected at a certain position 'v', the Euclidean distance (d) between 'v' and 'c' and the intensity-based value (i) of 'v' are stored. Therefore, the processing circuit 542 can use the following equation:

[0165] Cf=fun(d n ,i n ), n∈sign(1)

[0166] The intensity-based metric may be simply the intensity value of the CT image data or its gradient. For intensity, the particular change may be a step size from high (bone density) to low (soft tissue structure), while for gradient embodiments, there may be positive spikes along the curve that connects to indicate the boundaries of soft tissue structures. The processing circuit 542 may determine the registration of the initial shape 715 using a minimization algorithm such as a cost function. The minimization algorithm may refer to a general type of algorithm in which the processing circuit 542 deforms the initial shape 715 (e.g., SMS) by satisfying a threshold of the algorithm so that the deformed version of the SMS fits the bone-to-muscle scale of the soft tissue structure. For example, the cost function may be a combination of the Euclidean distance d=||vc|| and i; for example,

[0167] Cf=∑ n w1×d n +w2×i n , (2)

[0168] Among them, w1 and w2 are empirically determined weights.

[0169] The cost function can also have another term that is independent of the patient and is used to smooth the final estimate of the registration. This term is called the regularization term and can provide adaptive weights for the parameters of the initial shape 715 (based on their relevance and the noise in the SMS). This term (Cf2) can be added to the previous "difference" term (now called (Cf1)) and optimized to give the sum:

[0170] Cf=Cf1+Cf2 (3)

[0171] For comparability, Cf1 can be in the same scale as Cf2. Cf2 can have its value normalized (i.e., between 0 and 1). In order to scale Cf to the same interval, normalization is performed using Cf1 values ​​after strict registration (i.e., between SMS and patient-specific CT data).

[0172] Once the initial shape 715 is rigidly registered with the patient, it can be elastically deformed to match the patient's soft tissue structure using its finite parameter equation:

[0173]

[0174] where s' is the initial shape (e.g., SMS), λ i is an eigenvalue and v i are the eigenvectors (e.g., also called the variation patterns) of the corresponding covariance matrix. The covariance matrix represents the variation of a data set, such as the variation in patient-specific patient data. b i The value of determines the shape of s (eg, the final patient-specific shape of the soft tissue structure). i is the scaling factor of the original shape 715. The processing circuit 542 can use this process to find b i The value of s is such that a final shape (s) that estimates the patient-specific structure of the target muscle is provided. For example, the best fit is performed by the processing circuit 542 to minimize a cost function that defines the "difference" between s and the patient's muscle (e.g., m) in the patient-specific CT image. In other words, the optimization algorithm can minimize Cf=|sm| or Cf'=|sm| -1 maximize.

[0175] It may be important to define Cf or Cf' now because based on its convex shape, the optimizer will or will not fall into a global or local minimum or maximum. In order to obtain a good estimate at the end of the optimization process, the processing circuit 542 can determine Cf that reflects the "shape difference" between the modified final object (s) and the target (m). The variable for calculating this difference may be related to the estimated Euclidean distance of the final shape to a limited number of basic landmarks located on the patient.

[0176] The processing circuit 542 may use an optimization algorithm that minimizes the cost function by applying the SMS parameter equation in an iterative manner and changing the parameter values ​​based on the value of Cf after each iteration. The processing circuit 542 may stop the loop when Cf cannot be optimized any further (minimized or maximized), that is, an optimal value is reached, or when a maximum number of iterations is reached. Upon completion of the optimization algorithm, the processing circuit 542 may cause the modified initial shape 715 to ultimately become the final patient-specific shape of the soft tissue structure. A minimization or maximization algorithm may refer to a general type of algorithm in which the processing circuit 542 deforms the initial shape 715 (e.g., SMS) by satisfying a threshold of the algorithm so that the deformed version of the SMS fits the bone-to-muscle scale of the soft tissue structure. The threshold may indicate when the deformed version of the SMS is the best fit or reduces error with the soft tissue structure in the patient-specific image data.

[0177] Fig.19 is a conceptual diagram of an exemplary final patient-specific shape that is masked and thresholded to determine fat infiltration. Fat infiltration, or a value representing the fat ratio or fat volume within a muscle, can indicate changes in structure and / or function relative to an otherwise healthy muscle. Thus, the amount of fat within a muscle can indicate the health of that muscle. In turn, the health of the muscle can affect what type of joint treatment is appropriate for the patient.

[0178] As discussed herein, a system such as system 540 can determine a fat infiltration value of a modeled soft tissue structure from patient-specific image data within a representation of the soft tissue structure relative to a threshold intensity value based on thresholded voxels or groups of voxels. A ratio of fat tissue to total tissue within the representation can be determined as the fat infiltration value. System 540 can determine the fat infiltration value from pixels of multiple two-dimensional slices of the patient-specific image data or from voxels of a three-dimensional image data set. Here, a three-dimensional approach is described as an embodiment, but for purposes of illustration, a two-dimensional illustration is used. For example, for CT image data, the intensity threshold for fat can be approximately -29 Hounsfield units (HU) and the intensity threshold for muscle can be approximately 160 HU. In one embodiment, the fat infiltration (FI) value can be calculated as: FI = 100*(1-x / X), where x is the muscle volume within a mask and X is the total volume within the mask. In some embodiments, areas with fat or muscle less than a specified threshold (e.g., less than one cubic millimeter) can be considered noise and eliminated.

[0179] As Fig.19As shown in , patient image data 720 includes a patient-specific shape 722 generated for a muscle of interest using a mask applied to the voxels. First, the system can apply a mask to the patient-specific shape 722 to remove data outside the patient-specific shape 722. Next, the system can apply a threshold to the voxels under the mask. In some embodiments, a threshold can be applied to the average voxel intensity of groups of voxels or two or more voxels to avoid noise that may exist in a single voxel. In this way, the system can analyze the intensity and / or spatial characteristics of the patient-specific image data to determine the area of ​​fat tissue. The black area indicated by voxel 724 indicates muscle tissue above the threshold intensity indicating muscle tissue. In contrast, the lighter voxel 726 is below the threshold, lower intensity, and indicates fat tissue. In the absence of a mask shown, the intensity of the fat tissue voxel will be lower than the intensity of the voxel associated with the muscle. The system can then determine the fat volume of the soft tissue structure by adding the voxel 726 under the threshold. The system can then determine the fat permeability value based on the fat volume and the total volume of the patient-specific shape 722 of the soft tissue structure. For example, system 540 can make fat volume divided by total volume, to determine the fat ratio (for example, percentage) of soft tissue structure.Then, system 540 can output the fat volume ratio of soft tissue structure.In certain embodiments, when determining what type of joint replacement may be suitable for the patient, system 540 can use fat volume ratio as input.In certain embodiments, system 540 can use muscle quality indicators such as fat penetration and range of motion to determine the positioning and / or orientation of implant components such as humeral implant or acetabulum implant.For example, system 540 can suggest to move one or more implants laterally or centrally, to improve the range of motion and / or strength of the shoulder of this patient.Because the strength or flexibility of muscle may depend at least in part on the distance between acetabulum and humeral head, so change the position of humeral implant (or select humeral implant of different sizes) and make humerus move to be closer to or more away from acetabulum can make clinician improve the range of motion and / or strength of shoulder after implantation.In certain embodiments, bone transplantation can be used to add to humeral head or acetabulum, to realize the required lateralization or centralization (with or without humeral implant) of humerus.

[0180] Fig. 20 is a conceptual diagram of an exemplary final patient-specific shape 736 and pre-morbid prediction 734 of a soft tissue structure. Fig. 20 As shown in the embodiment of FIG. 54 , the sagittal view 730 includes a patient-specific shape 736 of the scapula 732 and the subscapularis muscle. The patient-specific shape 736 can be determined based on the closed surface fitting described above. However, it can provide useful information when selecting a type of joint treatment to understand how the muscle changes from a healthy state to a pre-disease state. In this way, the system 540 can determine the rate of atrophy of the muscle.

[0181] For example, the processing circuit 542 of the system 540 can be configured to determine the bone to muscle scale of the patient's soft tissue structure. The processing circuit 542 can determine the length, width, and / or volume of the scale of bones such as scapula 732 relative to muscles based on the patient's specific shape 736. The bone to muscle scale can identify the patient's specific anatomical structure size. The processing circuit 542 can obtain the statistical mean shape (SMS) of the soft tissue structure. The SMS can be a representation of a typical muscle based on the calculation of a large number of multiple healthy subjects. However, in some embodiments, the SMS of pathological structures can be used in some embodiments.

[0182] Next, the processing circuit 542 can deform the SMS using a minimization algorithm so that the SMS fits the bone-to-muscle scale of the soft tissue structure. For example, the processing circuit 542 can modify the SMS to fit the bone-to-muscle scale of the patient more strictly and thereby estimate the pre-disease state of the muscle. The generated pre-disease prediction 734 can be used as an estimate of the health state of the muscle. The processing circuit 542 can then determine the atrophy rate of the soft tissue structure (e.g., a muscle such as the supraspinatus muscle represented by the patient-specific shape 736) by dividing the deformed SMS volume by the soft tissue structure volume represented by the patient-specific shape 736. The result is the atrophy rate of the soft tissue structure. In some embodiments, in addition to calculations during preoperative planning, the processing circuit 542 can output the atrophy rate for display or further use.

[0183] Fig.21 and Fig. 22 is a conceptual diagram of an exemplary spring-modeled muscle that contributes to range of motion analysis of the shoulder joint. Fig.21 and Fig. 22 Preoperative planning for a reverse shoulder replacement is shown, however, in other embodiments, an anatomical shoulder replacement may be planned. Fig.21 As shown in, virtual diagram 740 comprises the posterior view of humeral head 744 and scapula 742 on the left side and the anterior view of humeral head 744 and scapula 742 on the right side.Humeral head has been equipped with spacer 748, and spacer 748 is configured to make the assembling shoulder acetabular sphere 750 cooperate with shoulder acetabular surface.Other parts may also be related to implants such as cup and plate of spacer 748.In anatomical shoulder replacement, these parts are inverted so that implant sphere is attached to humeral head 744 instead.Some exemplary parts can comprise the reverse shoulder acetabular component with wedge-shaped base plate, center screw, glass ball and symmetrical implant.In reverse shoulder replacement, the sessile retrograde (Ascend) FLEX reverse humeral component with standard insert and 1.5 offset disk can also be used.

[0184] For example, when the shoulder joint has deteriorated, the system 540 can perform a determination as to whether the patient should receive an anatomical or reverse shoulder replacement. Part of this determination can include consideration of dedicating one or more muscles to the movement of the joint. In one embodiment, the processing circuitry 542 of the system 540 can model the three rotator cuff muscles as springs with spring constants K1, K2, and K3. Fig.21 As shown in , the infraspinatus has been modeled as spring 746 and the subscapularis has been modeled as spring 747. In other embodiments, fewer or additional muscles may be modeled as part of the analysis. Fig. 22 As shown in FIG. 7 , virtual view 760 shows the subscapularis muscle modeled as spring 762. Processing circuit 542 can assign spring constants to each spring, such as assigning spring constants K1, K2, and K3 to springs 746, 747, and 762.

[0185] As described herein, the processing circuit 542 can determine each spring constant based on the calculated fat infiltration (e.g., fat volume ratio) and atrophy rate of the corresponding muscle that has been modeled. For example, the processing circuit 542 can determine the spring constant of each muscle using an equation such as K=f(R(FI), R(A)), where K is the spring constant, FI is the fat infiltration, and A is the atrophy of the corresponding muscle. In other embodiments, when determining the spring constant, additional factors such as the total volume, length, and / or cross-sectional thickness of the patient-specific shape of the muscle, the patient's age, the patient's gender, the patient's injury history, or any other type of factors that may affect the function of the muscle can be used. For example, the processing circuit 542 can determine the attachment points of each spring 746, 747, and 762 based on the insertion point of the muscle. In some embodiments, each muscle can be represented by two or more springs, such as different springs representing each attachment point from the muscle to the bone. In other embodiments, each muscle can be represented by a more complex model of muscle function other than the spring. In some embodiments, the processing circuit 542 can determine the load applied to the spring. The load may combine the weight of the bone structure with an external load (eg, lifting a standard object). Although springs may be used as some models, in other embodiments, a finite element model may be used.

[0186] Processing circuit 542 can determine the range of motion of the humerus of the patient by determining the fat volume ratio and atrophy rate of one or more corresponding muscles of the patient's rotator cuff. For example, processing circuit 542 can calculate each spring constant K1, K2, and K3, and then determine the range of motion of the humerus relative to the scapula. In some embodiments, processing circuit 542 can determine the range of motion in one or more planes or in three dimensions. The range of motion axis can be predetermined, and in some embodiments, when soft tissue is not a limiting factor along these axes, bone collision can be determined to establish the range of motion of some angles. Processing circuit 542 can perform this calculation for each possible type of treatment, such as anatomical replacement or reverse replacement. In other embodiments, processing circuit 542 can perform range of motion analysis on currently damaged joints and bones and use this calculation to identify which type of treatment is suitable for the patient. In any case, processing circuit 542 can determine which type of shoulder treatment such as anatomical shoulder replacement or reverse shoulder replacement should be selected for the patient and then output the selected shoulder treatment type for display. The processing circuitry 542 may present an analysis of each type of therapy to the user, such as presenting a numerical score or calculation for each type of therapy, etc. The user may then determine from this presented information for each type of therapy which type of therapy to provide to the patient.

[0187] Fig.23A is a flow chart illustrating an exemplary procedure for modeling soft tissue structures using patient-specific image data in accordance with the techniques of the present disclosure. The processing circuitry 542 of the system 540 is described as executing Fig.23A In the embodiment of the present invention, however, other devices or systems such as the virtual planning system 102 can perform one or more parts of the present technology. Further, some parts of the present technology can be performed by a combination of two or more devices and / or systems via a distributed system. Fig.23A The embodiments can be compared with those in the above referenced Figures 8 to Fig.11 The example diagram and discussion are similar.

[0188] like Fig.23A As shown in , the processing circuit 542 can obtain patient-specific image data (800) of a patient of interest. The patient-specific image data can be generated by one or more imaging modalities (e.g., x-ray, CT, MRI, etc.) and stored in a data storage device. Then, the processing circuit 542 obtains an initial shape of a soft tissue structure of interest (802). The initial shape can be a geometric shape or a statistical mean shape (SMS). The soft tissue structure can be muscle or other non-skeletal structure. However, in other embodiments, the processing described herein can be performed on bones. Fig.23AThe processing circuit 542 then registers the initial shape with the patient-specific image data (804). The registration may include registering the initial shape with bone insertion points and / or bones identified by the already segmented bones in the patient-specific image data. In other embodiments, a preliminary muscle segmentation has been performed on the soft tissue structures of interest in the patient-specific image data, and the processing circuit 542 may register the initial shape with the preliminary muscle segmentation.

[0189] The processing circuitry 542 then identifies one or more contours in the patient-specific image data that represent boundaries of the soft tissue structure (806). The one or more contours may be identified as voxels associated with the segmented bones and / or muscles in the patient-specific image data. In other embodiments, the processing circuitry 542 may determine each contour by normal vectors extending inwardly and / or outwardly from the surface of the initial shape from the initial shape. Voxels or pixels encountered by each vector and exceeding a threshold intensity value in the patient-specific image data may be identified as defining at least a portion of the contour.

[0190] The processing circuit 542 then moves the surface points on the surface of the initial shape toward corresponding points on one or more contours (808). The movement of these surface points causes the entire surface of the initial shape to deform. If the processing circuit 542 determines that the surface points need to be moved again to make the initial shape fit more closely to the one or more contours (the "yes" branch in block diagram 810), the processing circuit 542 moves the surface points of the deformed surface of the initial shape again (808). If the processing circuit 542 determines that the surface points do not need to be moved again and the deformed shape fits the one or more contours (the "no" branch in block diagram 810), the processing circuit 542 outputs the final deformed shape as a patient-specific shape representing the soft tissue structure of the patient (812). The patient-specific shape can be presented via a user interface and / or used for further analysis, such as part of a preoperative plan for treatment of the patient.

[0191] Fig. 23B is a flow chart illustrating another exemplary procedure for modeling soft tissue structures using patient-specific image data in accordance with the techniques of the present disclosure. The processing circuit 542 of the system 540 is described as executing Fig. 23B In the embodiment of the present invention, however, other devices or systems such as the virtual planning system 102 can perform one or more parts of the present technology. Further, a combination of two or more devices and / or systems can perform some parts of the present technology via a distributed system. Fig. 23B The embodiments in the above reference FIG. 8A to FIG. 11 In some respects, Fig. 23B The technology in Fig.23AThe technique is similar to .

[0192] like Fig. 23B As shown in , the processing circuit 542 can obtain patient-specific image data (820) of the patient of interest. The patient-specific image data can be generated by one or more imaging modalities (e.g., x-ray, CT, MRI, etc.) and stored in a data storage device. The processing circuit 542 then obtains an initial shape, which is an SMS of the soft tissue structure of interest (821). The processing circuit 542 then aligns the SMS with the insertion points of one or more corresponding bones of the patient-specific image data (822). In other embodiments, the alignment can include: aligning the SMS with a preliminary muscle segmentation (in addition to the bone insertion points) in the patient-specific image data.

[0193] Then, the processing circuit 542 selects a plurality of surface points around the surface of the SMS and determines a vector orthogonal to the surface of each surface point (823). The surface points may be uniformly distributed around the surface of the SMS at a predetermined density, a predetermined interval, and / or according to other selection factors. In some embodiments, the processing circuit 542 may guide these normal vectors outwardly and inwardly from at least some of the surface points. For each of these vectors, the processing circuit 542 determines a point in the patient-specific image data that exceeds a threshold intensity value and a potential position within the envelope of the point (824). The determined point may be a voxel, a pixel, or a point in space associated with the voxel or pixel. The determined points may correspond to the outer surface (e.g., one or more contours) of the identified soft tissue structure of interest within the patient-specific image data. Potential positions within the envelope are those positions that also exceed the threshold and are part of one or more contours. The envelope may be determined as a predetermined distance from the point identified by the vector, the number of potential positions adjacent to the point (e.g., the eight potential positions closest to the point), or other such criteria.

[0194] These potential positions are analyzed to identify changes in the profile that is fitted to the SMS. In other words, the potential positions are analyzed more accurately in the direction of the surface point in which the SMS is moved. Processing circuit 542 determines, for each vector, the potential position that has the minimum angle difference between its vector and the vector of the surface point (825). This angle can be referred to as the cosine between the two vectors. Fig. 9The angle 622 in is an embodiment of the angle between the vector 620 of the surface point and the vector 624 of the potential position on the contour. After selecting the potential position for each surface point on the SMS, the processing circuit 542 moves each surface point a certain distance toward the corresponding potential position of the contour (826). The distance moved can be a portion or percentage of the total distance to the potential position. In one embodiment, the distance is about half of the total distance to the potential position. In this way, each iteration brings the surface point closer to one or more contours, but the distance is getting smaller and smaller with each iteration. In other embodiments, the distance can be less than half of the total distance to the potential position or greater than half of the total distance to the potential position.

[0195] The movement of these surface points causes the entire surface of the SMS to deform. If the processing circuit 542 determines that the surface points need to be moved again to make the SMS fit more closely to one or more contours (the "yes" branch in block diagram 827), the processing circuit 542 updates the SMS shape change function (828) before re-determining the points and SMS surface points that exceed the threshold for each vector. The SMS change function can define how the processing circuit 542 deforms the SMS in this iteration. For example, the SMS change function can define how the deformation of the SMS balances the "smoothness" and "precision" of the next shape. For example, early deformations may be smoother or more uniform than later deformations, and later deformations can optimize the deformation accuracy of one or more contours in the patient-specific image data.

[0196] In one embodiment, the SMS change function can utilize a tolerance factor that defines the amount of movement that each surface point deviates from another surface point. For example, a zero tolerance can indicate that all surface points must move the same distance in this iteration. A larger tolerance can allow surface points to move different distances, which may result in a lower smoothness of the deformed SMS, but a higher accuracy toward the contour in the patient-specific image data. In some embodiments, the SMS change function can specify different tolerances for different threshold intensities. For example, if a point in the patient-specific image data exceeds a threshold indicating a bone, the SMS change function can specify a larger tolerance because it can be expected that muscles are arranged against the bone surface, so the processing circuit 542 can move the surface point closer to the bone surface. In some embodiments, the processing circuit 542 may not change the SMS shape change function between two iterations.

[0197] If the processing circuit 542 determines that the surface points do not need to be moved again and the deformed SMS shape fits one or more contours (the "No" branch in block 827), the processing circuit 542 outputs the final deformed shape as a patient-specific shape representing the patient's soft tissue structure (829). Fig.25 , Fig.26 ,as well as Fig. 27 As described in one or more embodiments of the present invention, a patient-specific shape can be presented via a user interface and / or used for further analysis, such as as part of a preoperative plan for treatment of the patient.

[0198] Fig.24 is a flow chart illustrating an exemplary procedure for modeling soft tissue structures using patient-specific image data in accordance with the techniques of the present disclosure. The processing circuitry 542 of the system 540 is described as executing Fig.24 In the embodiment of the present invention, however, other devices or systems such as the virtual planning system 102 can perform one or more parts of the present technology. Further, a combination of two or more devices and / or systems can perform some parts of the present technology via a distributed system. Fig.24 The embodiments can be compared with the above reference Fig.18A and Fig.18B The example diagram and discussion are similar.

[0199] like Fig.24 As shown in , the processing circuit 542 can obtain patient-specific image data (830) of the patient of interest. The patient-specific image data can be generated by one or more imaging modalities (e.g., x-ray, CT, MRI, etc.) and stored in a data storage device. Then, the processing circuit 542 obtains an initial shape of the soft tissue structure of interest (831). The initial shape can be a geometric shape or a statistical mean shape (SMS). For example, the SMS can be a pathological shape to capture a health condition similar to that of the patient. The soft tissue structure can be a muscle or other non-skeletal structure. Then, the processing circuit 542 aligns the initial shape with one or more locations associated with one or more bones of the patient-specific CT data (832). The alignment can include: aligning points on the initial shape with corresponding points or locations associated with adjacent bones.

[0200] The processing circuit 542 then determines the correspondence between each position of the bone and the corresponding point on the initial shape (833). The processing circuit 542 then determines the distance between each position and the corresponding point on the initial shape based on the intensity curve of the correspondence (834). For example, the intensity or gradient curve can be used to identify the boundaries of soft tissue structures in the patient-specific CT data. The processing circuit 542 can use a cost function to fit the initial shape to all available basic positions of the bone.

[0201] Processing circuitry 542 may then select a scaling factor that minimizes the difference between the initial shape and the variation from the patient-specific CT data (835). For example, processing circuitry 542 may analyze different scaling factors and use a cost function to obtain the best fit of the initial shape to the patient-specific CT data. Processing circuitry 542 may then output a final patient-specific shape representing the soft tissue structure of the patient. For example, processing circuitry 542 may perform this analysis on several muscles associated with the shoulder to be replaced.

[0202] Fig.25 is a flow chart illustrating an exemplary procedure for determining fat permeability values ​​for soft tissue structures of a patient in accordance with the techniques of the present disclosure. The processing circuit 542 of the system 540 is described as executing Fig.25 In the embodiment of the present invention, however, other devices or systems such as the virtual planning system 102 can perform one or more parts of the present technology. Further, a combination of two or more devices and / or systems can perform some parts of the present technology via a distributed system. Fig.25 The embodiments in the above reference Fig.19 The example diagram and discussion are similar. Fig.25 However, in other embodiments, a plurality of two-dimensional slice data may be analyzed in a similar manner.

[0203] like Fig.25As shown in , the processing circuit 542 can obtain or receive the final patient-specific shape of the patient's soft tissue structure (840). The processing circuit 542 then applies a mask (842) to the patient-specific shape. The mask can remove data outside the patient-specific shape. Next, the processing circuit 542 can apply a threshold to the voxel or volume under the mask (844). In some embodiments, the processing circuit 542 can apply a threshold to the average intensity of two or more voxels and / or voxel groups in a group to determine whether the voxel group should be identified as fat tissue. This grouping of voxels can reduce the impact of noise from whether the voxel is determined to be fat tissue. The processing circuit 542 then determines the fat volume of the soft tissue structure by adding voxels with intensity values ​​determined to be below the threshold (846). In other words, voxels less than the threshold are determined to be fat tissue and voxels greater than the threshold are determined to be muscle. The processing circuitry 542 then determines a fat infiltration value (848) based on the fat volume of the patient-specific shape of the soft tissue structure (i.e., the number of voxels having intensity values ​​less than the intensity threshold, representing the fat volume) and the total volume (i.e., the total represented by all voxels in the masked patient-specific shape, including the number of voxels having intensity values ​​less than the intensity threshold and the number of voxels having intensity values ​​greater than or equal to the intensity threshold). For example, the processing circuitry 542 can divide the fat volume by the total volume of the patient-specific shape to determine the fat ratio of the soft tissue structure. The processing circuitry 542 can calculate the total volume of the patient-specific shape or obtain the volume previously calculated. The processing circuitry 542 can then output the fat volume ratio of the soft tissue structure as the fat infiltration value (850). The fat volume ratio can be presented via a user interface and / or used for additional analysis.

[0204] Fig.25 is a flow chart illustrating an exemplary procedure for determining the rate of atrophy of a patient's soft tissue structure in accordance with the techniques of the present disclosure. The processing circuit 542 of the system 540 is described as executing Fig.26 In the embodiment of the present invention, however, other devices or systems such as the virtual planning system 102 can perform one or more parts of the present technology. Further, a combination of two or more devices and / or systems can perform some parts of the present technology via a distributed system. Fig.26 The embodiments in the above reference Fig. 20 The reference 3D dataset describes Fig.26 However, in other embodiments, a plurality of two-dimensional slice data may be analyzed in a similar manner.

[0205] like Fig.26As shown in the embodiment of the present invention, first, the processing circuit 542 determines the bone to muscle scale (860) of the patient's soft tissue structure. The processing circuit 542 can determine the length, width, and / or volume of the scale of bones such as scapula relative to muscles based on the patient's specific shape. The bone to muscle scale can identify the patient's specific anatomical structure size. Then, the processing circuit 542 obtains the statistical mean shape (SMS) (862) of the soft tissue structure. The SMS can be a representation of a typical muscle scale calculated based on a large number of multiple subjects. The SMS can be based on a healthy population. In some embodiments, the SMS can be based on the patient's race, gender, age, height, or other factors similar to the patient.

[0206] Next, the processing circuit 542 deforms the SMS using a minimization algorithm to fit the SMS to the bone-to-muscle scale of the soft tissue structure (864). For example, the processing circuit 542 may modify the SMS to fit the bone-to-muscle scale of the patient more closely and thereby assess the pre-morbid state of the muscle. Various different types of minimization algorithms may be used to fit the SMS to the patient's anatomical structure. The pre-morbid prediction of the muscle produced may be used as an assessment of the scale of the healthy state of the muscle. The processing circuit 542 then determines the atrophy rate of the soft tissue structure by dividing the volume of the deformed SMS by the volume of the soft tissue structure represented by the patient-specific shape (866). The result is the atrophy rate of the soft tissue structure, i.e., the ratio of the volume of healthy tissue to the volume of actual tissue. In other embodiments, the atrophy rate may be calculated based on other scale comparisons between the healthy state of the soft tissue structure and the current state. The processing circuit 542 then outputs the atrophy rate for display to the user or for further use (868), in addition to calculations during preoperative planning. For example, the processing circuit 542 may output the current tissue or patient-specific shape superimposed on a pre-morbid or healthy assessment of the same soft tissue structure. When performing preoperative planning for a joint of interest (e.g., the rotator cuff muscles of a shoulder joint of interest), the processing circuit 542 may perform this atrophy rate calculation for each muscle of interest.

[0207] Fig. 27 is a flow chart illustrating an exemplary procedure for determining a type of shoulder treatment based on a patient's determined soft tissue structure in accordance with the techniques of the present disclosure. The processing circuit 542 of the system 540 is described as executing Fig. 27 However, other devices or systems such as the virtual planning system 102 may perform one or more parts of the present technology. Further, a combination of two or more devices and / or systems may perform some parts of the present technology via a distributed system. Fig.26 However, in other embodiments, a plurality of two-dimensional slice data may be analyzed in a similar manner.

[0208] like Fig. 27 As shown in the embodiment of , the processing circuit 542 can model one or more rotator cuff muscles and / or other muscles associated with the shoulder joint into springs (870). Then, the processing circuit 542 obtains the fat infiltration value and atrophy rate of each soft tissue structure (872). Then, the processing circuit 542 determines the spring constant (874) for each muscle based on the fat infiltration value and atrophy rate. Then, the processing circuit 542 is able to determine the range of motion of the humerus in the shoulder joint based on the spring constant of one or more muscles (876). Then, the processing circuit 542 determines the type of shoulder treatment based on the determined range of motion (878). For example, the processing circuit 542 can choose between anatomical shoulder replacement and reverse shoulder replacement. In some embodiments, pathologies such as concentric osteoarthritis or massive tearing of the rotator cuff, muscle mass index, age, and adenoid deformity status may also affect whether anatomical or reverse shoulder replacement is suitable for the patient. In some embodiments, the processing circuit 542 can determine other aspects of the surface from this information, such as the size of the implant, the location of the implant, or other relevant information. In some embodiments, processing circuit 542 may use one or more decision trees or neural networks to determine the type of shoulder replacement. The suggested shoulder treatment may also be based on other patient information such as age, gender, activity, or any other aspect.

[0209] Processing circuitry 542 may control the user interface to display the suggested shoulder replacement. In response to the clinician selecting, accepting, or confirming the suggestion, processing circuitry 542 may initiate additional preoperative planning for the patient and the selected shoulder replacement type. For example, processing circuitry 542 or a user interface such as Figure 1 Other systems such as the virtual planning system 102 in can generate a surgical plan for the selected shoulder replacement. The processing circuit 542 can control the user interface to guide the clinician through additional customized steps for the patient, such as the required implant, cutting plane, anchor position, reaming axis, screw drilling and / or placement, size, implant placement, specific steps of the surgical procedure, and any other aspects of the selected surgical procedure. The clinician can also interact with the surgical plan by observing the system-generated visualization of the patient's procedure, anatomical structure, and / or implant. These types of processes can apply surgery to other joints such as the ankle, elbow, wrist, hip, knee, etc. In addition, Fig. 27Process in can be used to determine the position or rotation angle of one or more implants associated with shoulder treatment.For example, processing circuit 542 can determine whether to center (for example, make humeral head implant move closer to scapula or make shoulder mortise move closer to patient's midline) or lateralization (for example, make humeral head implant move further away from scapula or make shoulder mortise move closer to humerus), to improve range of motion based on spring constant.In certain embodiments, processing circuit 542 can suggest bone transplantation (removing or adding bone) to the size and / or position of humeral head and / or shoulder mortise, to realize the desired position of humerus relative to scapula.Processing circuit can use the stiffness of shoulder muscle (for example, spring constant when being used for muscle modeling) to determine the proper position of humeral head, to realize suitable range of motion and / or strength of shoulder.

[0210] As described herein, processing circuitry 542 and / or other systems can determine multiple measurements of morphological features of the patient using patient-specific imaging data. The measurements can include distance measurements, angle measurements, or other types of numerical representations of measured relationships of the patient's structures and / or between the patient's structures. For example, the measurements can include any combination of values ​​related to one or more of the following:

[0211] Acetabular version: The angular orientation of the axis of the acetabular articular surface relative to the transverse axis of the scapula.

[0212] Acetabular tilt: The superior / inferior spread of the acetabulum relative to the scapula.

[0213] Acetabular orientation / direction: The three-dimensional orientation of the acetabulum in three-dimensional space.

[0214] · Best-fit sphere radius of the acetabulum: The radius of the best-fit sphere for the patient's acetabulum. The best-fit sphere is a conceptual sphere whose size allows a sector of the sphere to sit as flat as possible with the patient's acetabulum.

[0215] · Root mean square error of the best fitting sphere of the shoulder socket: the mean square error of the difference between the patient's shoulder socket and the sector of the best fitting sphere.

[0216] Reverse shoulder angle: The inclination of the lower part of the shoulder socket.

[0217] Critical shoulder angle: The angle between the plane of the acetabulum and the line connecting the most inferior point of the acromion.

[0218] Acromiohumeral space: The space between the acromion and the top of the humerus.

[0219] Acetabular space: The space between the acetabulum and the humerus.

[0220] Humeral version: The angle between the humeral orientation and the lateral condylar axis.

[0221] Humeral neck axis angle: The angle between the normal vector of the anatomical humeral neck and the intramedullary axis.

[0222] Humeral head best-fit sphere radius and RMS error: The radius of the best-fit sphere for the patient's humeral head. The best-fit sphere is a conceptual sphere whose size allows a sector of the sphere to fit as closely as possible to the surface of the humeral head. The RMS error is the error between the best-fit sphere and the patient's actual humeral head.

[0223] Humeral subluxation: measurement of the subluxation of the humerus relative to the acetabulum.

[0224] Humeral orientation / direction: The orientation of the humeral head in three-dimensional space.

[0225] · Measurement of the epiphysis of the patient's humerus.

[0226] • Measurement of the metaphysis of the patient's humerus.

[0227] • Measurement of the diaphysis of the patient's humerus.

[0228] Posterior tilt of the skeleton.

[0229] Fig.28 is a flow chart illustrating an exemplary procedure for determining a type of shoulder treatment based on patient-specific image data in accordance with the techniques of the present disclosure. The processing circuitry 542 of the system 540 is described as executing Fig.28 However, other devices or systems such as the virtual planning system 102 may perform one or more parts of the present technology. Further, a combination of two or more devices and / or systems may perform some parts of the present technology via a distributed system. Fig.28 However, in other embodiments, a plurality of two-dimensional slice data may be analyzed in a similar manner.

[0230] like Fig.28 As shown in the embodiment of , the processing circuit 542 can receive patient-specific image data (e.g., CT image data) of the patient. The processing circuit 542 then determines one or more soft tissue features from the patient-specific image data of one or more soft tissue structures of the patient (892). Exemplary soft tissue features may include soft tissue shape and volume, fat infiltration value, atrophy rate, range of motion value, or any other such parameter. The processing circuit 542 then generates a recommendation for the type of shoulder surgery based on the determined one or more soft tissue features (894). For example, the processing circuit 542 can select between an anatomical total shoulder replacement or a reverse total shoulder replacement. The processing circuit 542 can then output the determined recommendation for the type of shoulder surgery for the patient for display in the user interface (896).

[0231] One or more of the determinations described herein and related to plastic surgery classification and surgical planning may employ artificial intelligence (AI) techniques such as neural networks. In one embodiment, processing circuitry 542 may employ various AI techniques to generate one or more characteristics of tissue, such as atrophy rate, fat infiltration, range of motion, recommendations for implant types (e.g., stem size for humeral implants), and / or recommendations for specific types of surgical treatment (e.g., anatomical or reverse shoulder replacement), etc. In some embodiments, during the preoperative phase 302 ( Figure 3 ) or other stages of the surgical life cycle. Deep neural networks (DNNs) are a type of artificial neural network (ANN) that have shown great promise as a classification tool. DNNs include an input layer, an output layer, and one or more hidden layers between the input layer and the output layer. DNNs may also include one or more other types of layers such as pooling layers.

[0232] Each layer may include a set of artificial neurons, i.e., often referred to simply as "neurons". Each neuron in the input layer receives an input value from an input vector. The output of the neurons in the input layer is provided as the input of the next layer in the DNN. Each neuron in the layer after the input layer may apply a propagation function to the output of one or more neurons in the previous layer to generate an input value to the neuron. The neuron may then apply an activation function to the input to calculate an activation value. The neuron may then apply an output function to the activation value to generate an output value of the neuron. The output vector of the DNN includes the output value of the output layer of the DNN.

[0233] There are several challenges associated with the application of DNNs to planning orthopedic surgery, specifically, with respect to shoulder pathology. For example, some challenges relate to how to construct and train a DNN so that the DNN is able to provide meaningful output with respect to shoulder pathology. In another embodiment of the challenges associated with the application of DNNs to planning orthopedic surgery, it is understood that patients and medical professionals are reluctant to trust decisions made by a computer, especially when it is unclear how the computer made those decisions. Thus there is the question of how to generate output in a manner that helps ensure that patients and medical professionals are willing to trust aspects of the output of the DNN.

[0234] The present disclosure describes techniques that can address these challenges and provide a DNN structure that provides meaningful outputs about shoulder pathology and / or suggested shoulder treatment based on one or more inputs. For example, an artificial neural network (ANN) has an input layer, an output layer, and one or more hidden layers between the input layer and the output layer. The input layer includes a plurality of input layer neurons. Each of the plurality of input layer neurons corresponds to a different input element of a plurality of input elements. The output layer includes a plurality of output layer neurons.

[0235] Each of the plurality of output layer neurons corresponds to a different output element in the plurality of output elements. Each of the plurality of output elements corresponds to a different classification in one or more shoulder pathology classification systems. In this embodiment, the computing system may generate a plurality of training data sets from previous shoulder surgery cases. Each respective training data set corresponds to a different training data patient in the plurality of training data patients and includes a respective training input vector and a respective target output vector.

[0236] For each corresponding training data set, the training input vector of the corresponding training data set includes the value of each element of the plurality of input elements. For each corresponding training data set, the target output vector in the corresponding training data set includes the value of each element of the plurality of output elements. In this embodiment, the computing system may train the neural network using the plurality of training data sets. In addition, in this embodiment, the computing system may obtain a current input vector corresponding to the current patient. The computing system may apply the DNN to the current input vector to generate a current output vector. The computing system may then determine a diagnosis of the shoulder health of the current patient based on the current output vector, which may also be referred to as a shoulder classification.

[0237] In this embodiment, by having different output elements of the plurality of output elements correspond to different categories in one or more shoulder pathology classification systems, the DNN is able to provide meaningful output information that can be used in the diagnosis of a patient's shoulder health condition, the determination of anatomical features, or treatment recommendations. For example, this can be more efficient in terms of computation and training time than a system in which different values ​​of neurons in the output layer correspond to different categories. Further, in some embodiments, the output values ​​of the neurons in the output layer indicate a measure of confidence that the patient's classified shoulder health condition belongs in a corresponding category of a shoulder pathology classification system. The confidence value can help the user consider the possibility that the patient may have a different category of shoulder health condition than that determined by the computer system using the DNN. Further, for the output of the same output layer neuron, it may be computationally efficient to express a confidence level and be used as a basis for determining a diagnosis (e.g., classification) of a patient's shoulder health condition, a specific feature of the tissue (e.g., fat infiltration value, atrophy value, range of motion value), or even a recommendation for a type of surgical procedure based on one or more such tissue features.

[0238] Fig.29 is a block diagram illustrating an exemplary computing system 902 implementing a DNN that may be used to determine one or more aspects of a patient's anatomy, diagnosis, and / or treatment recommendations in accordance with the techniques of the present disclosure. The computing system 902 may be part of the orthopedic surgery system 100 ( Figure 1). The computing system 902 can use the DNN to determine soft tissue features and / or recommendations for types of treatment, such as whether the patient would benefit from an anatomical or reverse total shoulder replacement. In some embodiments, the computing system 902 includes an XR visualization device (e.g., an MR visualization device or an XR visualization device), which includes one or more processors that perform operations of the computing system 902.

[0239] like Fig.29 In the embodiment shown in FIG. 1 , system 900 includes a computing system 902, a collection of one or more client devices (collectively referred to as “client devices 904”). In other embodiments, system 900 may include more, fewer, or different devices and systems. In some embodiments, computing system 902 and client devices 904 may communicate via one or more communication networks such as the Internet.

[0240] The computing system 902 may include one or more computing devices. The computing system 902 and the client device 904 may include various types of computing devices, such as server computers, personal computers, smart phones, laptops, and other types of computing devices. Fig.29 In the embodiment of the present invention, the computing system 902 includes a processing circuit 908, a data storage system 910, and a collection of one or more communication interfaces 912A to 912N (collectively referred to as "communication interfaces 912"). The data storage system 910 is configured to store data. The communication interface 912 enables the computing system 902 to communicate with other computing systems and devices such as client devices 912 (e.g., wirelessly or using wires). For ease of description, the present disclosure may describe actions performed by the processing circuit 906, the data storage system 910, and the communication interface 912 as performed by the computing system 902 as a whole. One or more subsystems ( Figure 1 ) may include a computing system 902 and a client device 904. For example, the virtual planning system 102 may include a computing system 902 and a client device 904.

[0241] The user may use the client device 904 to access information generated by the computing system 902. For example, the computing system 902 may generate a recommendation for the type of shoulder treatment for the current patient. The recommendation may be represented by a shoulder category among a plurality of shoulder categories in a shoulder treatment classification system. In this embodiment, the user may use the client device 904 to access information about intervention recommendations. Because the computing system 902 may be remote from the client device 904, the user of the client device 904 may consider the computing system 902 to be in a cloud-based computing system. In other embodiments, some or all of the functions of the computing system 902 may be performed by one or more client devices 904.

[0242] The computing system 902 may implement the DNN. The storage system 910 may include one or more computer-readable data storage media. The storage system 910 may store parameters of the DNN. For example, the storage system 910 may store weights of neurons of the DNN, bias values ​​of neurons of the DNN, and the like.

[0243] The computing system 902 can determine recommendations for treatment of the patient's shoulder health condition based on the output of the DNN. According to the technology of the present disclosure, the output elements of the DNN include output elements corresponding to different categories in one or more shoulder recommendation classification systems. The shoulder recommendation classification system may include different classification systems for each type of treatment for the patient, or different classifications for different pathological types that may lead to different recommendations. For example, different treatments may include anatomical shoulder replacement and reverse shoulder replacement. However, other treatments or surgeries may have corresponding classification systems. For example, the Walch classification system and the Favard classification system are two different primary glomerular osteoarthritis classification systems. The Warner classification system and the Goutallier classification system are two different rotator cuff classification systems. In some embodiments, the shoulder pathology classification system may include classifications of more general categories of shoulder pathology, such as one or more of the following: primary acetabulum humeral osteoarthritis (PGHOA), rotator cuff tear arthropathy (RCTA) instability, massive rotator cuff tear (MRCT), rheumatoid arthritis, post-traumatic arthritis, and osteoarthritis. These classification systems can be used to determine recommendations for treatment.

[0244] For example, the Walch classification system specifies five categories: 1A, 1B, 2A, 2B, and 3. As another embodiment, the Favard classification system specifies five categories: E0, E1, E2, E3, and E4. As yet another embodiment, the Warner classification system specifies four categories of rotator cuff atrophy: none, mild, moderate, and severe. As yet another embodiment, the Goutallier classification system specifies five categories: 0 (completely normal muscle), I (some fat streaks), II (muscle mass greater than fat infiltration), III (muscle mass equals fat infiltration), IV (fat infiltration greater than muscle). In other embodiments, the classification system may use fat infiltration values ​​or atrophy rates calculated from patient-specific image data.

[0245] In some embodiments, computing system 902 may determine a recommendation for treatment based on a diagnosis of the patient's shoulder health condition based on a comparison with the value of the output element generated by the DNN. For example, the value of the output element may correspond to a confidence value indicating a confidence level that the patient's shoulder health condition belongs to the category corresponding to the output layer neuron that generated the value. For example, the value of the output element may be a confidence value or computing system 902 may calculate a confidence value based on the value of the output element.

[0246] In some embodiments, the output function of the output layer neurons generates confidence values. Further, the computing system 902 can identify which confidence value is the highest. In this embodiment, the computing system 902 can determine that the shoulder pathology category corresponding to the highest confidence value is a diagnosis of the shoulder health condition of the current patient. In some embodiments, if no confidence value is above a threshold, the computing system 902 can generate an output indicating that the computing system 902 cannot make a conclusive diagnosis. Thus, the computing system 902 cannot determine a recommendation for treatment.

[0247] As mentioned above, in some embodiments, the output element of the DNN includes a confidence value. In one such embodiment, the confidence value function outputs the confidence value. The confidence value function may be an output function of an output layer neuron of the DNN. In this embodiment, all possible confidence values ​​output by the confidence value function are within a predetermined range. Further, in this embodiment, the computing system 902 may apply the DNN to the input vector to generate an output vector. As part of applying the DNN, the computing system 902 may calculate the output value of the corresponding output layer neuron for each corresponding output layer neuron in a plurality of output layer neurons.

[0248] The computing system 902 may then apply a confidence value function using the output value of the corresponding output layer neuron as an input to the confidence value function. The confidence value function outputs a confidence value of the corresponding output layer neuron. In this embodiment, for each corresponding output layer neuron in the plurality of output layer neurons, the output element corresponding to the corresponding output layer neuron specifies the confidence value of the corresponding output layer neuron. Further, for each corresponding output layer neuron in the plurality of output layer neurons, the confidence value of the corresponding output layer neuron is a measure of the confidence that the current patient's shoulder health condition belongs to a category corresponding to the output element (corresponding to the corresponding output layer neuron) in one or more shoulder pathology classification systems.

[0249] The computing system 902 may use various confidence value functions. For example, the computing system 902 may apply a hyperbolic tangent function, a sigmoid function, or other types of functions that output values ​​within a predetermined range. The hyperbolic tangent function (tanh) has the form γ(c)=tanh(c)=(e c –e -c) / (e c +e -c ). The hyperbolic tangent function takes a real-valued argument, such as the output value of an output layer neuron, and transforms it to the range (–1, 1). The sigmoid function has the form γ(c) = 1 / (1+e -c ). The sigmoid function takes a real-valued independent variable, such as the output value of an output layer neuron, and transforms it to the range (0,1).

[0250] The computing system 902 may train the DNN using a plurality of training data sets. Each respective training data set may correspond to a different training data patient from a plurality of previously diagnosed training data patients. For example, a first training data set may correspond to a first training data patient, a second training data set may correspond to a second training data patient, and so on. The training data sets may correspond to training data patients in the sense that the training data sets may include information about the patients. The training data patients may be real patients with diagnosed shoulder health conditions. In some embodiments, the training data patients may include simulated patients.

[0251] Each corresponding training data set may include a corresponding training input vector and a corresponding target output vector. For each corresponding training data set, the training input vector of the corresponding training data set includes the value of each of the multiple input elements. In other words, the training input vector may include the value of each input layer neuron of the DNN. For each corresponding training data set, the target output vector of the corresponding training data set may include the value of each of the multiple output elements. In other words, the target output vector may include the value of each output layer neuron of the DNN.

[0252] In some embodiments, the values ​​in the target output vector are based on confidence values. The confidence values, in turn, can be based on confidence levels expressed by one or more trained medical professionals, such as orthopedic surgeons. For example, the trained medical professionals can be given the information in the training input vectors of the training data set (or the information from which the training input vectors of the training data set are derived) and can be consulted to provide a confidence level that the training data patient has a shoulder health condition that belongs to each category in each shoulder pathology classification system.

[0253] For example, in an embodiment where the shoulder pathology classification system includes the Walch classification system, the medical professional may indicate that her confidence level that the training data patient's shoulder health condition belongs to category A1 is 0 (i.e., she does not believe at all that the training data patient's shoulder health condition belongs to category A1), indicate that her confidence level that the training data patient's shoulder health condition belongs to category A2 is 0, indicate that her confidence level that the training data patient's shoulder health condition belongs to category B1 is 0.75 (i.e., she is fairly confident that the training data patient's shoulder health condition belongs to category B1), indicate that her confidence level that the training data patient's shoulder health condition belongs to category B2 is 0.25 (i.e., she believes that there is a small chance that the training data patient's shoulder health condition belongs to category B2), and may indicate that her confidence level that the training data patient's shoulder health condition belongs to category C is 0. In some embodiments, the computing system 902 may apply the inverse of the confidence value function to the confidence values ​​provided by the medical professional to generate the values ​​included in the target output vector. In some embodiments, the confidence values ​​provided by the medical professional are the values ​​included in the target output vector.

[0254] Different medical professionals may have different confidence levels for the same training data patient having a shoulder health condition belonging to each category in each shoulder pathology classification system. Therefore, in some embodiments, the confidence value on which the value in the target output vector is based can be an average or otherwise determined from the confidence levels provided by multiple medical professionals. Similar confidence values ​​can be calculated for recommendations for treatment types based on identified pathologies, or features determined from the DNN.

[0255] In some such embodiments, the confidence levels of some medical professionals may be given greater weight in the weighted average of the confidence levels than the confidence levels of other medical professionals. For example, the confidence levels of outstanding plastic surgeons may be given greater weight than the confidence levels of other plastic surgeons. In another embodiment, the confidence levels of medical professionals or training data patients in a particular region or hospital may be given greater weight than those of medical professionals or training data patients in other regions or hospitals. Advantageously, this weighted average may allow the DNN to be adjusted according to various criteria and preferences.

[0256] For example, a medical professional may prefer to use a DNN that has been trained so that confidence levels are weighted in a specific manner. In some embodiments where the training data set includes a training data set based on the medical professional's own cases, a medical professional (e.g., a plastic surgeon) may prefer to use a DNN trained using such a training data set in which the medical professional's own cases are weighted more heavily or exclusively. In this way, the DNN can generate outputs that are customized for the medical professional's own practice style. Moreover, as mentioned above, medical professionals and patients can have different levels of confidence in the output of the computing system. Accordingly, in some embodiments, the computing system 902 can provide information indicating that the DNN is trained to simulate the decisions of the medical professional himself and / or a plastic surgeon with a particular confidence.

[0257] In some embodiments, different medical professionals' confidence levels for the same training data patient may be used when generating different training data sets. For example, a first training data set may be generated using a first medical professional's confidence level for a particular training data patient and a second training data set may be generated using a second medical professional's confidence level for the same training data patient.

[0258] Further, in some embodiments, computing system 902 can provide confidence values ​​for output to one or more users. For example, computing system 902 can provide confidence values ​​to client device 904 for display to one or more users. In this way, one or more users can better understand how computing system 902 can obtain a diagnosis and / or recommendation for treatment of the patient's shoulder.

[0259] In some embodiments, in order to expand the full domain of training data sets, the computing system 902 can automatically generate confidence values ​​from electronic medical records. For example, in one embodiment, the patient's electronic medical records may include such data, that is, the computing system 902 can form an input vector from the data, and may include data indicating the surgeon's diagnosis of the patient's shoulder health and the selected shoulder treatment. In this embodiment, the computing system 902 can infer a default confidence level from the diagnosis. The default confidence level can have various values ​​(e.g., 0.75, 0.8, etc.). While the default confidence level may not reflect the surgeon's actual confidence level, the input confidence level can help increase the number of available training data sets, which can improve the accuracy of the DNN.

[0260] In some embodiments, the training data set is weighted based on the health outcomes of the training data patients. For example, if the training data patients associated with the training data set have all positive health outcomes, then the training data set can be given a higher weight. However, if the associated training data patients have less positive health outcomes, then the training data set can be given a lower weight. During training, the computing system 902 can use a loss function that weights the training data set based on the weights given to the training data sets.

[0261] In some embodiments, as part of generating a training data set, the computing system 902 may select multiple training data sets from a database of training data sets based on one or more training data set selection criteria. In other words, if a training data set does not meet the training data set selection criteria, the computing system 902 may exclude a particular training data set from the training process of the DNN. Fig.29 In the embodiment of the present invention, the data storage system 910 stores a database 914 containing training data sets from previous shoulder surgery cases.

[0262] There may be a wide variety of training data set selection criteria. For example, in one embodiment, one or more training data set selection criteria may include which surgeon performed surgery on a plurality of training data patients. In some embodiments, one or more training data set selection criteria include the region in which the training data patients live. In some embodiments, one or more training data set selection criteria include a region associated with one or more surgeons (e.g., the region in which one or more surgeons practice, live, obtain licenses, train, etc.).

[0263] In some embodiments, one or more training data set selection criteria include postoperative health outcomes of patients in the training data. In this embodiment, the postoperative health outcomes of patients in the training data may include one or more of the following: postoperative range of motion, presence of postoperative infection, or postoperative pain. Thus, in this embodiment, the training data set based on which the DNN is trained may exclude training data sets in which adverse health outcomes occur.

[0264] Additional training data sets can be added to the database over time and the computing system 902 can use the additional training data sets to train the DNN. Thus, as more training data sets are added to the database, the DNN can continue to improve over time.

[0265] The computing system 902 may apply one of various techniques to train the DNN using a training data set. For example, the computing system 902 may use one of various standard back propagation algorithms known in the art. For example, as part of training the DNN, the computing system 902 may apply a cost function to determine a cost value based on the difference between an output vector generated by the DNN and a target output vector. The computing system 902 may then use the cost value in the back propagation algorithm to update the weights of the neurons in the DNN. In this way, the computing system 902 may train the DNN to determine various features of the patient's soft tissue (e.g., fat infiltration, atrophy rate, range of motion, etc.) based on inputs such as tissue volume, voxel grouping, and pre-disease shape volume. In some embodiments, the computing system 902 may train the DNN to determine recommendations for shoulder treatment using inputs from patient-specific image data and / or determined features of soft tissue such as these features that can be determined from different DNNs. In some embodiments, the computing system 902 can train the DNN to determine recommendations for shoulder treatment using a bone density metric that indicates or is related to the bone density of one or more bones (e.g., the humeral head) that receive an implant. For example, the computing system 902 can train the DNN using patient-specific imaging data (e.g., CT data) associated with the humeral head and the surgeon-selected type of humeral implant for each respective patient (e.g., a stem that may or may not include the length of the stem, or a stemless humeral implant). The output of this training is a recommended type of humeral implant based on patient-specific image data associated with the humeral head of a new patient. In this way, the type of shoulder treatment and / or implant can be determined based on the density of trabecular bone or other features related to density within the humerus.

[0266] Fig.30 It is shown that by having Fig.29 An exemplary DNN 930 implemented by the computing system 902 of the system in FIG. Fig.30 In the embodiment of the invention, the DNN 930 includes an input layer 932, an output layer 934, and one or more hidden layers 936 located between the input layer 932 and the output layer 934. Fig.30 In the embodiment of FIG. , neurons are represented as circles. Although in Fig.30 In the embodiment of FIG. 9 , each layer is shown as including six neurons, however, the layers in DNN 930 may include more or fewer neurons. Further, although in Fig.30 930 is shown as a fully connected network, however, DNN 930 may have a different architecture. For example, DNN 930 may not be a fully connected network, may have one or more convolutional layers, or may have other layers similar to those in FIG. Fig.30 Different architectures shown in .

[0267] In some implementations, DNN 930 can be or include one or more artificial neural networks (also referred to as neural networks for short). A neural network can include a set of connected nodes, which can also be referred to as neurons or perceptrons. A neural network can be organized into one or more layers. A neural network including multiple layers can be referred to as a "deep" network. A deep network can include an input layer, an output layer, and one or more hidden layers between the input layer and the output layer. The nodes of a neural network can be connected or incompletely connected.

[0268] DNN 930 can be or include one or more feed-forward neural networks. In a feed-forward neural network, the connections between nodes do not form loops. For example, each connection can connect a node from a previous layer to a node from a subsequent layer.

[0269] In some instances, DNN 930 can be or include one or more recursive neural networks. In some instances, at least some nodes in the recursive neural network can form loops. Recursive neural networks can be particularly useful for processing input data that is sequential in nature. Specifically, in some instances, the recursive neural network can transfer or store information from a previous portion of an input data sequence to a later portion of an input data sequence by using recursive or directed loop node connections.

[0270] In some embodiments, the sequential input data can include event sequence data (e.g., sensor data over time or images captured at different times). For example, the recurrent neural network can analyze sensor data over time to detect or predict swipe directions, to perform handwriting recognition, etc. The sequential input data can include words in a sentence (e.g., for natural language processing, language detection or processing, etc.), notes in a piece of music, sequential actions made by a user (e.g., to detect or predict sequential application usage), sequential target states, etc. Exemplary recurrent neural networks include long short-term (LSTM) recurrent neural networks, gated recurrent units, bidirectional recurrent neural networks, continuous-time recurrent neural networks, neural history compressors, echo state networks, Elman networks, Jordan networks, recurrent neural networks, Hopfield networks, fully recurrent networks, sequence-sequence configurations, etc.

[0271] In some implementations, DNN 930 can be or include one or more convolutional neural networks. In some instances, a convolutional neural network can include one or more convolutional layers that perform convolutions on input data using learned filters. Filters can also be referred to as kernels. Convolutional neural networks can be particularly useful for vision problems such as when the input data includes imagery such as still images or videos. However, convolutional neural networks can also be applied to natural language processing.

[0272] DNN 930 may be or include one or more other forms of artificial neural networks such as, for example, a deep Boltzmann machine, a deep belief network, a stacked autoencoder, etc. Any of the neural networks described herein can be combined (e.g., stacked) to form a more complex network.

[0273] exist Fig.30 In the embodiment of the present invention, the input vector 938 includes multiple input elements. Each input element can be a numerical value. The input layer 932 includes multiple input layer neurons. Each input layer neuron in the multiple input layer neurons included in the input layer 932 can correspond to different input elements in the multiple input elements. In other words, for each input element in the input vector 938, the input layer 932 can include different neurons.

[0274] Further, in Fig.30 In the embodiment of the present invention, the output vector 940 includes multiple output elements. Each output element can be a numerical value. The output layer 934 includes multiple output layer neurons. Each output layer neuron in the multiple output layer neurons corresponds to a different output element in the multiple output elements. In other words, each output layer neuron in the output layer 934 corresponds to a different output element in the output vector 940.

[0275] The input vector 938 may include a wide variety of information. For example, the input vector 938 may include morphological measurements of the patient. In some embodiments where the input vector 938 includes morphological measurements of the patient, the input vector 938 may determine the measurements based on medical images of the patient such as CT images, MRI images, or other types of images. For example, the computing system 902 may obtain a medical image of the current patient (e.g., patient-specific image data). For example, the computing system 902 may obtain a medical image from an imaging machine (e.g., a CT machine, an MRI machine, or other types of imaging machines), an electronic medical record of the patient, or another data source. As described herein, in this embodiment, the computing system 902 may segment the medical image to identify the internal structures of the current patient such as soft tissue and bone, and in some embodiments, generate patient-specific shapes of bones and / or soft tissues. Further, in this embodiment, the computing system 902 may determine multiple measurements based on the relative positions of the identified internal structures of the current patient. In this embodiment, multiple input elements may include input elements for each measurement in multiple measurements. Other inputs may include other determinations from patient-specific image data, such as fat permeability values, atrophy rates, and / or range of motion values ​​of joints, etc.

[0276] As mentioned elsewhere in this disclosure, computing system 902 may include one or more computing devices. Thus, various functions of computing system 902 may be performed by various combinations of computing devices in computing system 902. For example, in some embodiments, a first computing device in computing system 902 may segment an image, a second computing device in computing system 902 may train a DNN, a third computing device in computing system 902 may apply the DNN, etc. In other embodiments, a single computing device in computing system 902 may segment an image, train a DNN, and apply the DNN.

[0277] In some embodiments, input vector 938 may include information based on an assessment of the patient's rotator cuff (e.g., in combination with zero or more of the other exemplary types of input data described herein). For example, input vector 938 may include information about fat infiltration of the rotator cuff, atrophy of the rotator cuff, and / or other information about the patient's rotator cuff, either alone or in combination with the morphological inputs described above. In some embodiments, for example, fat infiltration measurements and atrophy measurements of soft tissue used as input to the neural network may be derived by any of the soft tissue modeling techniques described herein. In some embodiments, information about the patient's rotator cuff may be expressed in terms of categories in a shoulder pathology classification system such as the Warner classification system or the Goutallier classification system.

[0278] In some embodiments, input vector 938 may include (e.g., in combination with zero or more of the other exemplary types of input data described herein) range of motion information for the patient. As described elsewhere in this disclosure, in some embodiments, a motion tracking device may be used to generate the patient's range of motion information. As described herein, in other embodiments, one or more range of motion values ​​may be determined from an analysis of patient-specific image data.

[0279] Further, in some embodiments, the input vector 938 may include (e.g., in combination with zero or more other exemplary types of input data described herein) information specifying a category in one or more shoulder pathology classification systems. In this embodiment, the output vector may include output elements corresponding to categories in one or more different shoulder recommendations for shoulder treatment. For example, the input vector 938 may include information specifying a category in a rotator cuff muscle classification system and the output vector 940 may include output elements corresponding to the type of shoulder treatment recommended (e.g., anatomical or reverse shoulder replacement).

[0280] In some embodiments, input vector 938 may include information specifying a bone density score for the humerus and / or acetabulum (e.g., in conjunction with zero or more other exemplary types of input data described herein, including morphological input and / or rotator cuff input). Other information included in input vector 938 may include demographic information such as patient age, patient activity, patient gender, patient body mass index (BMI). In some embodiments, input vector 938 may include information about the onset rate of symptoms (e.g., gradually or suddenly). Multiple input elements in input vector 938 may also include patient goals for participating in activities such as specific exercise / sports types, range of motion, etc.

[0281] In certain embodiments, output vector can comprise the output element of multiple surgical operation types.The output element of every kind of surgical operation type can correspond with different types of shoulder surgeries.The shoulder surgery type of the exemplary type that can be presented as output can comprise the standard total shoulder arthroplasty of stemless, the standard total shoulder arthroplasty of stemless, the reverse shoulder arthroplasty of stemless, the reverse shoulder arthroplasty of stem, the standard total shoulder arthroplasty of shoulder mortise with enhancement, the reverse shoulder arthroplasty of shoulder mortise with enhancement and the plastic shoulder surgery of other types.With regard to after the surgical operation, the shoulder joint of the patient has standard anatomical configuration aspect, wherein, the scapula side of the shoulder joint has concave surface and the humerus side of the shoulder surgery has convex surface, and the shoulder surgery can be " standard ".On the other hand, " reverse " shoulder surgery produces opposite structure, wherein, convex surface is attached to scapula and concave surface is attached to humerus.

[0282] In addition, the computing system 902 can determine a recommended type of shoulder surgery for the patient based on the current output vector. For example, the computing system 902 can determine which output element in the output vector corresponds to the type of shoulder surgery with the largest confidence value.

[0283] Fig.31 is a flow chart illustrating exemplary operation of a computing system using a DNN to determine a recommended type of shoulder surgery for a patient in accordance with the techniques of the present disclosure. Fig.31 In an embodiment of the invention, the computing system 902 generates a plurality of training data sets (950). In this embodiment, the DNN has an input layer, an output layer, and one or more hidden layers between the input layer and the output layer. The input layer includes a plurality of input layer neurons. Each of the plurality of input layer neurons corresponds to a different input element of the plurality of input elements. The output layer includes a plurality of output layer neurons.

[0284] Each output layer neuron in the plurality of output layer neurons corresponds to a different output element in the plurality of output elements. The plurality of output elements include output elements of a plurality of surgical types. The output elements of each surgical type in the output elements of the plurality of surgical types correspond to different types of shoulder surgeries in the plurality of types of shoulder surgeries. Each corresponding training data set corresponds to a different training data patient in the plurality of training data patients and includes a corresponding training input vector and a corresponding target output vector. For each corresponding training data set, the training input vector in the corresponding training data set includes the value of each element in the plurality of input elements. For each corresponding training data set, the target output vector in the corresponding training data set includes the value of each element in the plurality of output elements.

[0285] Further, in Fig.31 In an embodiment of the present invention, the computing system 902 trains the DNN using a plurality of training data sets (952). In addition, the computing system 902 may obtain a current input vector corresponding to the current patient (954). The computing system 902 may apply the DNN to the current input vector to generate a current output vector (956). The computing system 902 may determine a recommended type of shoulder surgery for the current patient based on the current output vector (958). The computing system 13 / 208 may perform these activities according to embodiments provided elsewhere in this disclosure.

[0286] Fig.32 is an illustration of exemplary bones associated with a patient's shoulder 1000. Fig.32 As shown in the embodiment of , shoulder 1000 includes humerus 1004, scapula 1010, and clavicle 1016. The shaft 1006 of humerus 1004 is connected to humeral head 1008, and humeral head 1008 and acetabulum 1012 form glenohumeral joint 1002. The acromion 1014 of scapula 1010 is attached to clavicle 1016 at the acromion-clavicular joint.

[0287] The connective tissue between humeral head 1008, shoulder mortar 1012 and / or humeral head 1008 and shoulder mortar 1012 may degenerate over time due to wear and tear and / or disease.In some cases, the patient who suffers from the degeneration of glenohumeral joint 1002 may benefit because of shoulder replacement, and in shoulder replacement, at least humeral head 1008, shoulder mortar 1012 or a part of both are replaced by artificial implant.For example, humeral head 1008 may be cut, to expose the trabecular bone of lower density in humeral head 1008.Humerus implant can be inserted in trabecular bone, so that new humeral implant is fixed to the shaft 1006 of humerus 1004.

[0288] Fig.33A , Fig.33B ,as well as Fig.33Cis a conceptual diagram of an exemplary humeral head 1022 prepared for use in a humeral implant. Fig.33A As shown in , as part of a shoulder arthroplasty procedure, a clinician can perform the surgical step of resection of the humeral head 1022 of the humerus 1020 by visually assessing (e.g., "eyeballing") and marking the anatomical neck 1024 of the humeral head 1020. The anatomical neck 1024 can refer to a plane that bisects a portion of the humeral head 1022 to create or expose a surface to which a humeral implant can be attached to the humerus 1020. Then, as shown in FIG. Fig.33B 1024, the clinician can perform the resection of the humeral head 1022 by guiding a cutting tool 1026 (e.g., a blade of an oscillating saw) along the marked anatomical neck 1024 using the clinician's free hand without mechanical or visual guidance. After the resection of the humeral head 1022 is completed, the trabecular bone region 1028 is exposed along the plane corresponding to the anatomical neck 1024. Typically, a humeral implant can be inserted into a portion of the trabecular bone region 1028 and fixed in place. The density of the trabecular bone region 1028, or the density variation within the volume of the trabecular bone region 1028, can affect what type of humeral implant can be used for the humerus 1020. For example, a trabecular bone region 1028 of greater density can support a humeral implant with a humeral implant of a shorter "handle", compared to a trabecular bone with a lower density or softer trabecular bone that may require a longer handle on the humeral implant.

[0289] Fig.34 It is a conceptual diagram of an exemplary humeral implant intended for anatomical shoulder replacement procedures. In total shoulder arthroplasty surgery (for example, the type of shoulder replacement or shoulder treatment), the humeral implant mechanical fixation strength in the humeral shaft can be determined mainly by the fixation of the stem handle of the implant. When the handle of the humeral implant is introduced, the implantation of the handle can cause the shaft to suffer higher mechanical forces and other blunt impact forces during drilling, reaming, and pulling holes. These procedures may cause complications related to intraoperative fractures (humeral shafts), and postoperative loosening and often cause stress avoidance of revision surgery. Extraction of the humeral head implant for revision may also be difficult, especially when implanted with bone cement.

[0290] The humeral implant without stem or with a shorter stem can eliminate potential shaft fracture and allow preservation of natural bone material by avoiding implantation in the shaft of the humerus or the parts of the shaft, which is beneficial in the case of revision. The stemless design can also enable the surgeon to restore the mortar-humeral rotation center independently of the humeral shaft orientation and avoid the complications associated with stem implantation. The skeletal fixation of the humeral implant without stem can be realized mainly in the humeral head and the trabecular bone mesh therein.

[0291] Contraindications for the stemless humeral implant for conduit saving may include poor bone quality (osteopenia, osteoporosis), other metabolic bone diseases (cysts, tumors, etc.), or the presence of previous fractures that may affect bone support, ingrowth, and the integration of metal components. In this way, the density of the bone in the femoral head (density and / or position of different bone densities, etc.) may need to be enough to support a shorter diaphysis or stemless femoral implant type. The surgeon can use the "thumb test" as a main intraoperative assessment tool, and the surface (proximal humeral tissue disease) of the neck incision of the humeral head with the thumb can be determined for implantation. This substrate vitality may be related to or represented by the density of the trabecular bone in the humeral head. In other words, the bone density of the trabecular bone in the humeral head can be assessed using the surgeon's thumb. The bone (for example, low density) easily compressed with minimal force is considered as unacceptable stemless component implantation. The bone that is not easily compressed or provides higher elasticity (for example, higher density) can be considered as acceptable stemless type humeral implant.

[0292] In this way, the surgeon can make the selection of stemless or stemless (or even the length of stem) based on the result of " thumb test ".As described herein, the historical data relevant to the humeral head bone quality (such as density, compression rate, selection of stemless or stemless implant or other suitable properties of the humeral head of support implant, etc.) may be associated with the patient-specific imaging data of these patients.For example, this association can map the intensity deviation of the standard of the voxel in the humeral head of the CT data of intensity threshold, Hornsfield unit or with a specific patient to the type of humeral implant selected by the patient for these corresponding patients.Once complete association, the system can use this association to suggest the humeral implant of specific type based on the CT data mapped to each type of humeral implant.In this way, as described herein, the system can suggest the humeral implant of specific type (for example, stemless and / or the length of stem) based on the analysis (for example, the intensity magnitude and / or magnitude position of voxel or a group of voxels) of patient-specific image data.

[0293] like Fig.34 1040 is an embodiment of a "stemless" humeral implant having a sliding surface 1042 configured to contact a shoulder socket or shoulder socket implant. A fixing structure 1044 includes a projection configured to be embedded in the trabecular bone 1028 to fix the humeral implant 1040, but there is no stem extending down near or in the shaft of the humerus.

[0294] Humerus implant 1050 is the embodiment of "handled " humerus implant, but handle fixing structure 1054 comprises the short handle that is convenient for humerus implant 1050 to be anchored in the trabecular bone of humerus.Humerus implant 1050 comprises the sliding surface 1052 that is configured to contact shoulder mortar or shoulder mortar implant.Humerus implant 1060 is the embodiment of "handled " humerus implant, but handle fixing structure 1064 comprises the long handle that is convenient for humerus implant 1060 to be anchored in the trabecular bone of humerus.Humerus implant 1060 comprises the sliding surface 1062 that is configured to contact shoulder mortar or shoulder mortar implant.When trabecular bone is damaged so that the handle that fully anchors humerus implant needs to be provided from healthy bone, humerus implant 1050 and 1060 can be used. For example, trabecular bone may not provide sufficient bone density to anchor the humeral implant 1040 and therefore a "stemmed" humeral implant like humeral implant 1050 or 1060 may instead be suggested or selected for the patient.

[0295] For humeri that have the lowest density of trabecular bone or otherwise require greater stability, a long stem fixation structure 1064 may be used. Conversely, when the density of trabecular bone is high enough to adequately anchor the humeral implant with a stemless fixation structure 1044, a humeral implant 1040 may be used. The benefits of a stemless design similar to the fixation structure 1044 may be that less trabecular bone needs to be removed from the humerus, as well as faster healing and a lower risk of cortical bone damage during stem expansion and / or insertion into the humerus.

[0296] However, until excision and manual manipulation of the humeral head, the clinician can determine whether the trabecular bone in the humeral head can support the stemless design as the humeral implant 1040. As described herein, the system can use patient-specific image data (e.g., CT data) to determine the trabecular bone density of the humeral head to assist the shoulder replacement plan before surgery. In this way, exemplary technology provides various practical applications using computational analysis to improve the mode of determining bone density and reduce surgical time (e.g., determine what kind of humeral head type has been selected to use) and / or improve the selection accuracy of the humeral head implant type before surgery (e.g., improve preoperative planning) technology.

[0297] Fig.35 1070 is a conceptual diagram of an exemplary stemmed humeral implant 1080 implanted into a humerus 1072. Fig.35 As shown in the embodiment of FIG. 1 , a stemmed humeral implant 1080 includes a glide surface 1084 and a stem 1080. The stem 1080 has been inserted into the trabecular bone 1078 to anchor the humeral implant 1080 to the humeral head 1076. Although the stem 1080 may be Fig.34However, the stem 1080 can still be at least partially inserted into the humeral shaft 1074. The exemplary humeral implant 1080 can be similar to the Aequalis Ascend manufactured by Wright Medical Group NV of Memphis, Tennessee. TM Flex is similar.

[0298] Fig.36 1 is a conceptual diagram 1100 of an exemplary stemless humeral implant 1112 implanted in a humeral head 1110. Fig.36 As shown in the embodiment of FIG. 1 , the stemless humeral implant 1112 includes a sliding surface 1116 and a fixation structure 1114. The fixation structure 1114 has been stemlessly embedded in the trabecular bone of the humeral head 1110. The humeral implant 1112 can be used with Fig.34 The exemplary humeral implant 1112 may be similar to the Simpliciti manufactured by Wright Medical Group N.V. of Memphis, Tennessee. TM The gliding surface 1112 can be configured to contact the acetabular implant 1106 implanted into the acetabular surface of the scapula 1102. Because the humeral implant 1112 includes a spherical surface similar to the spherical surface of a healthy humeral head, the stemless humeral implant 1112 and the acetabular implant 1106 can be part of an anatomical shoulder replacement.

[0299] Fig.37 It is a conceptual diagram of an exemplary reverse humeral implant 1124. The reverse humeral implant 1124 can be constructed as a stem (similar to humeral implant 1050 or 1060) or a stemless design (similar to implant 1040) and is implanted in the humeral head 1122 of the humerus 1020. However, the reverse humeral implant 1124 has a concave shape and is intended to contact the spherical or convex shape of the corresponding shoulder acetabulum implant. The system can be based on the characteristics of soft tissue structure (for example, one or more muscles of the rotator cuff muscle or other shoulder muscles) and / or the bone density of the humeral head 1122. The reverse shoulder replacement including the reverse humeral implant 1124.

[0300] Fig.38 1 is a block diagram illustrating exemplary components of a system 1140 configured to determine estimated bone density from patient-specific image data according to an embodiment of the present disclosure. The system 1140 and components therein may be used in conjunction with Figure 6 The system 540 and components described in and / or Figure 1 Thus, system 540 or virtual planning system 102 can perform the functions attributed to system 1140 herein.

[0301] like Fig.38 As shown in the embodiment of FIG. 1 , the system 1140 may include a processing circuit 1142, a power supply 1146, a display device 1148, an input device 1150, an output device 1152, a storage device 1154, and a communication device 1144. The display device 1148 may display an image to present a user interface such as an opaque or at least partially transparent screen to a user. The display device 1148 may present visual information, and in some embodiments, audio information or other information presented to the user. For example, the display device 1148 may include one or more speakers, a tactile device, etc. In other embodiments, the output device 1152 may include one or more speakers and / or a tactile device. The display device 1148 may include an opaque screen (e.g., an LCD or LED display). Alternatively, the display device 1148 may include an MR visualization device, for example, including a see-through holographic lens, in combination with a projector, to allow a user to see real-world objects in a real-world environment through the lens and also through, for example, a device such as Microsoft HOLOLENS TM The user interface can present a virtual image of a virtual surgical plan for a specific patient, such as information related to bone density, etc.

[0302] The input device 1150 may include one or more microphones and associated speech recognition processing circuits or software, which can recognize voice commands issued by the user and responsively perform any of a variety of operations, such as selection, activation, or disabling of various functions associated with surgical planning, intraoperative guidance, etc. As another embodiment, the input device 1150 may include one or more cameras or other optical sensors that detect and interpret gestures to perform the operations described above. As yet another embodiment, the input device 1150 includes one or more devices that sense gaze direction and perform various operations as described elsewhere in this disclosure. In some embodiments, the input device 1150 may receive manual input from a user via a handheld controller that includes, for example, one or more buttons, keypads, keyboards, touch screens, joysticks, trackballs, and / or other manual input media and perform the various operations described above in response to manual user input.

[0303] The communication device 1144 may include one or more circuits or other components that facilitate data communication with other devices. For example, the communication device 1144 may include one or more physical drives (e.g., DVD, Blu-ray, or Universal Serial Bus (USB) drives) to allow data to be transferred between the system 1140 and the drives when physically connected to the system 1140. In other embodiments, the communication device 1144 may include. The communication device 1144 may also support wired and / or wireless communication with another computing device and / or network.

[0304] The storage device 1154 may include one or more memories and / or repositories that store data of corresponding types in common and / or separate devices. For example, the user interface module 1156 may include instructions that define how the system 1140 controls the display device 1148 to present information such as information related to the bone density of the humerus or associated with a suggestion for a surgical treatment to a user. The preoperative module 1158 may include instructions for analysis of patient data such as imaging data and / or determination of treatment options based on the patient data. The intraoperative module 1160 may include instructions that define how the system 1140 operates when providing information for display to a clinician such as details about a planned surgical procedure and / or feedback about a surgical procedure. The patient data 1166 may be a repository that stores patient-specific image data.

[0305] The bone density modeling module 1162 may include instructions defining how the processing circuit 1142 determines one or more bone density metrics of a portion of at least one or more bones such as the humeral head. For example, the bone density modeling module 1162 may determine the bone density metric based on the intensity of the voxels within the patient-specific patient data (e.g., CT image data). The processing circuit 1142 may execute the bone density modeling module 1162 to determine different bone density categories of a group of pixels or voxels according to a predetermined range (e.g., Hornsfield units) of the intensity of a single or a group of pixels or voxels. In some embodiments, the processing circuit 1142 may generate a bone density metric based on the standard deviation of the voxels within the patient-specific image data. The bone density metric may include different bone density values ​​across a two-dimensional or three-dimensional region of the humeral head. In some embodiments, the bone density metric may be a single value determined based on the average pixel or voxel intensity across the humeral head or in a certain region of the humeral head. In some embodiments, the bone density modeling module 1162 may include instructions for determining the type of humeral implant (e.g., stemmed or stemless) and / or the position at which the humeral implant can be implanted into the humeral head. In fact, the bone density metric may not indicate the density of the bone, but may be a metric representing the bone density. For example, the bone density metric may only indicate the type of implant (e.g., stemmed or stemless) corresponding to the patient-specific image data being analyzed. As another embodiment, the bone density metric may include the voxel intensity from the image data, the standard deviation of the voxel intensity from the image data, compressibility, index, or some other indication related to or representing density, without actually providing a density measurement of the bone.

[0306] Processing circuit 1142 can perform calibration module 1164, with patient-specific image data and selected implant type (for example, based on the selected implant type of thumb test information history in this surgical procedure) of other patients from historical surgery, bone density measurement is calibrated.Historically, clinicians can use their thumb to press the trabecular bone in the humeral head (exposed by cutting head) to determine the rigidity of trabecular bone and thus determine the density of trabecular bone.This thumb test can be performed to identify what type of handle (if any) is needed for humeral implant.Calibration module 1164 can use this thumb test data obtained from historical patients to make the known surgical decision of the humeral implant type made based on thumb test program be associated with the patient-specific image data of the corresponding same patient, to determine the bone density measurement of current patient.Like this, calibration module 1164 can be used to identify one or more scopes of bone density measurement corresponding to corresponding humeral implant type. For example, using the calibration module 1164, the processing circuit 1142 can determine whether the stemless humeral implant 1040 is used for bone density measurements within a first range, whether the short stem humeral head 1050 is used for bone density measurements within a second range, and whether the long stem humeral head 1060 is used for bone density measurements within a third range.

[0307] As discussed above, the surgical life cycle 300 may include a preoperative phase 302 ( Figure 3 ). One or more users may use the orthopedic surgery system 100 in the preoperative stage 302. For example, the orthopedic surgery system 100 may include a virtual planning system 102 (which may be similar to system 1140) to assist one or more users in generating a virtual surgical plan customized for an anatomical structure of interest for a particular patient. As described herein, the virtual surgical plan may include a three-dimensional virtual model corresponding to the anatomical structure of interest for the particular patient and a three-dimensional model of one or more prosthetic components (e.g., implants) matched to the particular patient to repair the anatomical structure of interest or selected to repair the anatomical structure of interest. The virtual surgical plan may also include a three-dimensional virtual model of guidance information to guide the surgeon when performing the surgical procedure, for example, when preparing a bone surface or tissue and placing implantable prosthetic hardware relative to the bone surface or tissue.

[0308] As discussed herein, processing circuit 1142 can be configured to determine a bone density measurement of at least a portion of a patient's humeral head based on the patient-specific image data of the patient. For example, the bone density measurement can be a single indication of the overall density of the humeral head or a portion of the humeral head. As another embodiment, the bone density measurement can include the bone density value of the corresponding portion of the patient's humeral head. The system can control the user interface via user interface module 1156 to present a graphical representation of the bone density measurement (which can directly or indirectly indicate bone density) and / or generate suggestions for the implant type of the humeral head based on the bone density measurement. For example, a bone density measurement indicating sufficient trabecular bone density in the humeral head can produce a system suggesting a stemless humeral implant opposite to a stemmed humeral implant.

[0309] In one embodiment, processing circuit 1142 can be configured to identify the humeral head in patient-specific image data by, for example, segmenting the bone or otherwise identifying a sign or shape indicating the humeral head. Then, processing circuit 1142 can determine a bone density metric representing the bone density of at least a portion of the humeral head based on the patient-specific image data. Based on the bone density metric, processing circuit 1142 can generate a suggestion for the patient's humeral implant type. For example, processing circuit 1142 can suggest a stemmed humeral implant (stem implant type) for indicating a bone density metric for lower density bones and processing circuit 1142 can suggest a stemless humeral implant (stemless implant type) for indicating a bone density metric for higher density bones. Then, processing circuit 1142 can output a suggestion for the patient's humeral implant type to be displayed via a user interface.

[0310] In certain embodiments, processing circuit 1142 can determine the stem length including stem humeral implant type. Processing circuit 1142 can determine that a longer stem is needed to be anchored to the less dense bone of humerus or determine that the position of less dense trabecular bone in humerus needs a longer stem. Can identify and stem length itself is presented to the user, or processing circuit 1142 can suggest a specific humeral implant that meets the suggested length range. In this way, processing circuit 1142 can suggest a specific implant or implant type selected between three or more different types of humeral implants based on the bone density metric determined from patient-specific image data.

[0311] In some embodiments, the bone density metric can represent the overall density score of the trabecular bone in at least a portion of the humeral head (e.g., based on the value, index, or category of the voxel or pixel value from the image data). For example, the processing circuit 1142 can determine the average or weighted average density of the region of the humeral head and assign a specific metric value to the region of the humeral head. In other embodiments, the bone density metric indicating the lowest density bone found in the region can be determined to establish the lower limit of the bone density in the region. On the contrary, the bone density metric can indicate the highest density in the region of the humeral head. The bone density metric can include a plurality of bone density values ​​of the corresponding parts in the humeral head. For example, the bone density metric can include a density value matrix in the region of the humeral head, that is, including a specific bone density value of a corresponding voxel or a group of voxels. In this way, the bone density metric can provide a higher resolution representation of the bone density in the humeral head. In any case, the bone density metric can indicate actual bone density value, image data intensity, and / or the implant type proposed.

[0312] The processing circuit 1142 can determine the bone density metric using different techniques. In some embodiments, the processing circuit 1142 can determine the bone density metric by identifying the intensity of the corresponding voxel in at least a portion of the humeral head based on the patient-specific image data, classifying the intensity of the corresponding voxel into one of two or more intensity levels, and determining the bone density metric based on at least one of the number of voxels classified in each level of the two or more intensity levels or the position of the humeral head of the voxel classified in each level of the two or more intensity levels. In this way, the processing circuit 1142 can be configured to classify the different intensities in the patient-specific image data into different intensity levels and / or the positions of these intensity levels to determine the bone density metric. For example, the position of the intensity level may be related to whether the trabecular bone is dense enough to support the stemless humeral implant. If the trabecular bone has a lower overall bone density, but the center of the humeral head is still located above the threshold density required to support the stemless humeral implant, the processing circuit 1142 can still determine that the bone density metric is sufficient to support the stemless humeral implant. In other embodiments, even with a certain relatively high bone density level, if a pocket of less dense trabecular bone is identified at the location where a stemless humeral implant is implanted, the processing circuit 1142 may determine that the bone density metric indicates a need for a stemmed humeral implant.

[0313] In some embodiments, processing circuit 1142 can determine the bone density measurement of the volume of trabecular bone in the whole humeral head. In other embodiments, processing circuit 1142 can determine the plane of the humeral incision made by the humeral head and represented in the humerus, to prepare the humerus for receiving the humeral implant. The humeral incision exposes the surface of the trabecular bone in which the humeral implant is implanted. Then, processing circuit 1142 determines the bone density measurement of at least a portion of the humeral head bisected by the plane. In some embodiments, processing circuit 1142 can determine the bone density measurement of the pixel or voxel corresponding to the plane (for example, exposed by the plane or bisected by the plane). In other embodiments, processing circuit 1142 can determine the bone density measurement of the volume of trabecular bone extending from the plane and toward the axis of the humerus. In some embodiments, the volume of the trabecular bone analyzed can extend upward to the cortical bone limiting the outer surface of the humerus.

[0314] In some embodiments, the bone density metric may be displayed via a user interface (such as using user interface module 1156, etc.). Processing circuit 1142 may output such a user interface for display by display device 1148 or a display device of another system, the user interface including a graphical representation of a bone density metric located on a representation of at least a portion of the patient's humeral head. The graphical representation of the bone density metric may include a two-dimensional or three-dimensional graphic, which may include one or more shapes or colors displayed above or in place of the trabecular bone of the humerus. In one embodiment, the bone density metric may include a heat map of multiple colors, wherein each color in the multiple colors represents a different range of bone density values. Thus, different colors may represent different bone density magnitudes indicating a spatial representation of bone density variations within the volume of trabecular bone. The graphical representation of the bone density metric may include a two-dimensional representation of bone density variations within the plane of the humeral head. In other embodiments, the graphical representation of the bone density metric may include a three-dimensional representation of bone density variations and within at least the trabecular bone of the humeral head. In some embodiments, display device 1148 may include a mixed reality display, and processing circuit 1142 may control the mixed reality display to present a user interface including a graphical representation of the bone density metric.

[0315] In some embodiments, bone density measurement can be associated with bone density data (e.g., image data or other data indicating the bone structure in the humeral head) from other historical patients and the type of humeral implant selected by the clinician for this specific bone density data. The patient-specific image data of each patient and the final type of the humeral implant selected by the surgeon for each corresponding patient can be used to generate bone density data for these historical patients (e.g., can be based on "thumb test", wherein the clinician uses his thumb to press the trabecular bone in the humeral head and classifies the trabecular bone as being sufficient or insufficient for stemless humeral implant). Processing circuit 1142 can utilize these implant types selected based on thumb test to classify bone density measurement as being suitable for or not suitable for stemless humeral implant of future patients. In this way, processing circuit 1142 can associate bone density measurement with the type of humeral implant selected by the surgeon in the surgical operation performed to other subjects before, wherein the thumb test data indicates the density range of trabecular bone manually determined in the corresponding humeral head of other subjects (or the compression rate representing bone density). Based on this correlation, processing circuit 1142 can determine the suggestion of the humeral implant type to the patient. In some embodiments, processing circuit 1142 can use one or more neural networks to associate the implant type previously selected with the corresponding patient-specific image data to determine the bone density measurement of each type of implant available to the patient in the future. For example, processing circuit 1142 can use bone density measurement, patient-specific image data, and selected humeral implant type (with stem, stemless, and / or stem length) as input to the neural network. The output of the neural network can be these bone density measurements corresponding to the humeral implant type.

[0316] In some embodiments, processing circuit 1142 can use soft tissue features and bone density measurement to generate the shoulder surgery suggestion to the patient.For example, processing circuit 1142 can determine one or more soft tissue features (for example, soft tissue volume, fat penetration ratio, atrophy rate, and / or range of motion value) and the bone density measurement associated with the humerus of the patient based on the imaging data specific to the patient.As described herein, processing circuit 1142 can generate the suggestion (for example, anatomical or reverse shoulder surgery) of the shoulder surgery type performed by the patient and generate the suggestion of the humerus implant type to the patient based on the bone density measurement associated with the humerus.Then, processing circuit 1142 can output the suggestion of the shoulder surgery type and the humerus implant type to the patient for display.In some embodiments, the user interface can include one or more soft tissue features and / or the bone density measurement associated with the humerus as a part of the mixed reality user interface.

[0317] Fig.39Ais a flow chart illustrating an exemplary procedure for determining the type of humeral implant based on bone density. The processing circuit 1142 in the system 1140 is described as executing Fig.39A However, other devices or systems, such as system 542 or virtual planning system 102, may perform one or more portions of the present technology. Further, a combination of two or more devices and / or systems may perform some portions of the present technology via a distributed system. Fig.39A However, in other embodiments, a plurality of two-dimensional slice data may be analyzed in a similar manner.

[0318] like Fig.39A As shown in the embodiments of the present invention, the processing circuit 1142 can obtain patient-specific image data such as three-dimensional CT image data (e.g., from a memory or other system) (1200). The processing circuit 1142 can then identify the humeral head in the patient-specific image data (1202). For example, the processing circuit 1142 can segment the bone to identify the humeral head or determine a landmark or shape indicative of the humeral head. Using the patient-specific image data of the humeral head, the processing circuit 1142 can determine a bone density metric of at least a portion of the humeral head based on the intensity of a voxel or a group of voxels in the patient-specific image data (1204). The bone density metric can be an overall metric indicating the overall density of trabecular bone within the humeral head, or the bone density metric can include a value representing the density of each voxel in a group of voxels within the region of the humeral head.

[0319] Then, the processing circuit 1142 can determine the suggestion (1206) for the humeral implant type based on the bone density measurement. For example, when the bone density measurement indicates or represents that the density of trabecular bone is high enough to support a stemless humeral implant, the processing circuit 1142 can determine that the suggestion is a stemless humeral implant. The suggestion can be based on a selection algorithm (e.g., one or more tables, equations, or machine learning algorithms such as neural networks) that may be developed by the processing circuit 1142 (based on historical data related to the humeral implant received by the patient before). For example, the historical data may include patient-specific image data (e.g., CT data) and the type of humeral implant (e.g., stemless or stemless) selected by the surgeon for the corresponding patient (e.g., via using a thumb test to determine the trabecular bone quality or density in the humeral head). In one embodiment, the table can map the volume element intensity or a group of volume element intensities to the suggestion of a stemless or stemless implant type. In another embodiment, the first table can map the volume element intensity to a density value, and the second table can map the density value to the suggestion of a stemless or stemless implant type. The system can use this mapping of image data to implant selection to inform a suggestion of an implant type for a new patient based on the image data of the patient. The processing circuit 1142 can then output a suggestion (1208) for a humeral implant type. The suggestion can be sent for use in another suggestion or displayed to the user.

[0320] Fig.39B is a flow chart illustrating an exemplary procedure for applying a neural network to patient-specific image data to determine the stem size of a humeral implant. The processing circuit 1142 in the system 1140 is described as executing Fig.39B However, other devices or systems, such as system 542 or virtual planning system 102, may perform one or more portions of the present technology. Further, a combination of two or more devices and / or systems may perform some portions of the present technology via a distributed system. Fig.39B However, in other embodiments, a plurality of two-dimensional slice data may be analyzed in a similar manner.

[0321] like Fig.39B As shown in the embodiment of FIG. 1 , processing circuit 1142 may obtain patient-specific image data such as three-dimensional CT image data (e.g., from a memory or other system) (1210). Processing circuit 1142 may then select one or more subsets of the patient-specific image data to be applied to the neural network (1212). For example, processing circuit 1142 may select a specific portion of the three-dimensional CT image data corresponding to one or more anatomical regions of the humerus. In some embodiments, the three-dimensional CT image data may indicate bone density. Processing circuit 1142 may then apply the neural network to the selected subset of the patient-specific image data (1214).

[0322] The neural network may include or be based on a convolutional neural network (CNN). As discussed above, a convolutional neural network can include one or more convolutional layers that perform convolutions on input data using learned filters. The filters can also be referred to as kernels. Convolutional neural networks can be particularly useful for vision problems such as when the input data includes still images, such as three-dimensional CT data or other imaging modalities such as MRI data.

[0323] Exemplary CNNs may include an ingestion network (e.g., ingestion V-net), a residual neural network (e.g., ResNet), or other types of networks including transfer learning techniques applied based on prior data related to previous humeral diagnosis and / or humeral implant data. In some embodiments, CNN may include N convolutional layers, followed by M fully connected layers. Each layer may include one or more filters, and the layer may be a stacked convolutional layer. In some embodiments, one or more filters may be specified for cortical bone, and one or more other filters may be specified for cancellous (e.g., trabecular) bone. The layers of CNN may be constructed to evaluate several bone regions from the metaphysis to the diaphysis of the humerus.

[0324] Processing circuit 1132 or another processor can train CNN to predict humeral stem size, which may include: training the model for tens, hundreds, or thousands of test images (e.g., CT scan images). The output may be clinical data collected during or after the surgical procedure, such as stem size and fill ratio (i.e., the ratio of the stem of the humeral catheter to the stem of the filled humeral implant), etc. The stem size may specify the length and / or cross-sectional width or area of ​​the humeral implant. A shorter stem size may be referred to as a stemless humeral implant, while a longer stem size may be referred to as a stemmed humeral implant. CNN may also include one or more hyperparameters. Several different types of hyperparameters may be used to reduce the average classification ratio. An exemplary hyperparameter may include a rectified linear unit (ReLU) for a nonlinear portion, rather than traditional slower solutions such as Tanh or Sigmond functions. The learning rate of a CNN may depend on one or more variables of an essentially adaptive gradient descent algorithm, such as Adagrad, Adadelta, RMSprop, or Adam, etc., in which the learning rate is adaptive based on the type data fed back to the CNN. The batch size of the data may depend on the learning rate selected for the CNN. The CNN may utilize a momentum value tested based on performance. In one embodiment, the momentum value may be selected from a range of 0.90 to 0.99. However, in other embodiments, other momentum values ​​may be selected. Due to the relatively small training data set, the CNN may include larger weight values. However, for other types of training data sets, these weights may be different.

[0325] CNN can output the handle size (1216) of the humeral implant according to the three-dimensional patient-specific image data applied by CNN. The handle size may include handle length and / or cross-sectional dimensions (e.g., diameter, circumference, area, or other such parameters). In this way, a determined handle size corresponding to the specific dimensions and / or bone density of the trabecular bone and / or cancellous bone of the humerus of the patient can be selected. Based on the handle size, the processing circuit 1132 can output a suggestion (1218) for the type of humeral implant for the patient. In this way, the processing circuit 1132 can determine whether a humeral implant with or without a handle is suitable for treating the patient. In some embodiments, CNN can be applied to a bone density metric determined from patient-specific imaging data. In other embodiments, CNN can be used instead of a bone density metric so that CNN can directly output the handle size of the humeral implant according to patient-specific imaging data. In some embodiments, the bone density modeling module 1162 may include CNN and relevant parameters.

[0326] As discussed above, the processing circuit 1132 can train a convolutional neural network and apply the convolutional neural network to patient-specific image data (e.g., 3D imaging data) to generate a stem size (obtained from the patient-specific image data) recommended for the patient's humeral implant. In one embodiment, the system may include a memory configured to store the patient's patient-specific image data and a processing circuit (e.g., processing circuit 1132) configured to apply a convolutional neural network to the patient-specific image data (or a subset thereof) and output a stem size for the patient's humeral implant based on the convolutional neural network applied to the patient-specific image data. The processing circuit may also be configured to output a suggestion for a humeral implant type including a stem size. The patient-specific image data may represent some or all of the bone density of one or more bones of a patient such as a humerus. In some embodiments, the processing circuit may also output a representation of a bone density metric representing the bone density of at least a portion of the humeral head, but when the CNN is applied to the patient-specific image data, the bone density metric may or may not be used to generate a stem size suggestion.

[0327] Fig.39C is a flow chart illustrating an exemplary approach for determining a recommendation for shoulder treatment based on soft tissue structure and bone density determined from patient-specific image data. The processing circuit 1142 in the system 1140 is described as executing Fig.39C However, other devices or systems, such as system 542 or virtual planning system 102, may perform one or more portions of the present technology. Further, a combination of two or more devices and / or systems may perform some portions of the present technology via a distributed system. Fig.39CHowever, in other embodiments, a plurality of two-dimensional slice data may be analyzed in a similar manner.

[0328] like Fig.39C As shown in the embodiment of FIG. 1 , the processing circuit 1142 can determine the characteristics of one or more soft tissue structures based on the patient-specific image data (1220). For example, these characteristics can include reference Fig.23A , Fig. 23B , Fig.24 , Fig.25 , Fig.26 ,as well as Fig. 27 The volume, fat penetration rate, atrophy rate, and / or range of motion of one or more soft tissue structures described herein. Fig.39A As described above, the processing circuit 1142 may also determine a bone density metric of at least a portion of the humeral head based on the intensity of the patient-specific image data.

[0329] Processing circuit 1142 can determine one or more suggestions (1224) for shoulder treatment based on soft tissue features and bone density measurement. For example, processing circuit 1142 can determine whether shoulder replacement is reverse or anatomical replacement based on one or more soft tissue features. In addition, processing circuit 1142 can determine whether the humeral implant type used in shoulder replacement is a stemless or stemmed humeral implant type. In some embodiments, processing circuit 1142 can determine the position of at least one of humeral implant or acetabular implant based on soft tissue features and / or bone density measurement. Then, processing circuit 1142 can output one or more suggestions (1226) determined for the treatment of the patient's shoulder. In this way, processing circuit 1142 can use any feature, measurement, or other information derived from patient-specific image data and other patient information to provide suggestions related to shoulder treatment.

[0330] Fig.40 is a flow chart showing an exemplary process for displaying bone density information. The processing circuit 1142 in the system 1140 is described as executing Fig.40 However, other devices or systems, such as system 542 or virtual planning system 102, may perform one or more portions of the present technology. Further, a combination of two or more devices and / or systems may perform some portions of the present technology via a distributed system. Fig.40 However, in other embodiments, a plurality of two-dimensional slice data may be analyzed in a similar manner.

[0331] like Fig.40 As shown in the embodiment of FIG. 1 , the processing circuit 1142 may be based on, for example, Fig.39AThe processing circuit 1142 may then determine a bone density metric of at least a portion of the humeral head based on the intensity of the patient-specific image data (1230). The processing circuit 1142 may then determine a graphical representation of the bone density metric (1232). These graphical representations may be compared to Fig.42 and Fig.43 The processing circuit 1142 may then control the user interface to present a graphical representation of the bone density metric over at least a portion of the humeral head (1234).

[0332] Fig.41 13 is a conceptual diagram of an exemplary user interface 1300 including a humerus 1332 and a cutting plane 1338. Fig.41 As shown in the embodiment of FIG. 1 , user interface 1300 includes a navigation bar 1301 and toolbars 1318 and 1320. Navigation bar 1301 may include selectable buttons that, when selected by a user, cause user interface 1300 to change to a different function or information view related to shoulder treatment, such as planning a shoulder replacement, etc.

[0333] Navigation bar 1301 can include a welcome button 1302 that brings the user to a welcome screen showing information relevant to the patient or possible actions relevant to the treatment type. Plan button 1304 can change the view of user interface 130 to a virtual plan of shoulder surgery, can include the representation of bone and / or soft tissue structure, such as view 1330 including humerus 1332 etc. Transplant button 1306 can display the view of potential bone or soft tissue transplantation relevant to surgery, and humerus cutting button 1308 can display the representation of humeral head 1332 cut to expose trabecular bone therein. Installation guide button 1310 can display possible or suggested humerus implant. Shoulder acetabulum reaming button 1314 can display the view of the exemplary reaming performed to shoulder acetabulum, and shoulder acetabulum implant button 1316 can display the exemplary possible or suggested shoulder acetabulum implant for patient implantation. Toolbar 1318 can include optional buttons, when selected, causing user interface 1300 to change the size of view, rotation or view 1330. Toolbar 1320 may include selectable buttons that, when selected, cause user interface 1300 to transition between anatomical planes of the anatomical structure shown in view 1330 , such as a perspective or side view of the anatomical structure.

[0334] View 1330 includes a perspective view of a humerus 1332 showing an axis 1334 and a humeral head 1336. A cutting plane 1338 is shown to indicate how the humeral head 1336 can be cut prior to implanting the humeral implant. Although user interface 1300 may initially display a suggested location for cutting plane 1338, user interface 1300 may enable the user to move cutting plane 1338 as needed during the planning process. Fig.42 and Fig.43 As shown in FIG. 13 , once the user is satisfied with the location of the cutting plane 1338, the user interface 1330 can then remove the top of the humeral head 1336 to expose a representation of the trabecular bone into which the humeral implant can be implanted.

[0335] Fig.42 13 is a conceptual diagram of an exemplary user interface 1300 including a representation of a humeral head 1342 and a bone density metric 1344. Fig.42 In the embodiment shown in FIG. 1 , the user interface 1300 may include a view 1340, which shows a Fig.41 The humeral head 1342 after the cutting plane 1338 removes the top of the humeral head. The humeral head 1342 is a representation of the patient's humerus and can be derived from patient-specific image data. The bone density metric 1344 can be a graphical representation of a bone density metric generated for the trabecular bone of the humerus 1332.

[0336] The bone density metric 1344 may include different colors for voxels representing intensities that fall within corresponding ranges 1346A and 1346B of intensity for each color. Thus, the bone density metric 1344 may include bone density values ​​for different groups of voxels of trabecular bone within the humeral head 1342. For example, range 1346A represents a value greater than 0.30 g / cm 3 The bone density of 1346B is 0.15 g / cm 3 With 0.30g / cm 3 The bone density key 1347 indicates different colors of bone density within the possible range determined from the patient-specific image data. The three ranges shown in the bone density key 1347 are merely examples, and in other embodiments, one or more different numbers of ranges with different lower and upper limits may be used.

[0337] In other embodiments, the view 1340 may present a bone density metric 1344, i.e., an image of a range of voxel intensities representing patient-specific image data or an intensity representation of a single or a group of voxels. As one embodiment, the bone density metric 1344 may include only the voxel intensities of the patient-specific image data corresponding to the same cutting plane 1338. In other words, the view 1340 may include a picture of CT data in a 2D plane corresponding to the cutting plane 1338 superimposed on the representation of the exposed humerus 1332. As another embodiment, the view 1340 may include a heat map with different colors and patterns, for example, corresponding to different ranges of Hounsfield units (e.g., CT data). Thus, although bone density metrics such as the bone density metric 1344 may be related to a representation of bone density, however, the actual bone density metric may not actually reflect a measurement of bone density in the region.

[0338] Fig.43 1 is a conceptual diagram of an exemplary user interface 1300 including a representation of a humeral head 1342 and a bone density metric 1352 associated with a type of humeral implant recommendation. Fig.43 As shown in the embodiment of FIG. 1 , the user interface 1300 may include a view 1350, wherein Fig.42 Similarly, it is shown along Fig.41 Humeral head 1342 after the cutting plane 1338 in FIG. 134 is a representation of the patient's humerus and may be derived from patient-specific image data. Bone density metric 1352 may be a graphical representation of a bone density metric generated for the trabecular bone of humerus 1332 .

[0339] The bone density metric 1352 indicates the type of humeral implant that is implanted in the trabecular bone based on the bone density determined for the humerus 1332. As such, the bone density metric 1352 includes the bone density determined from the patient-specific patient data as part of a category associated with the type of humeral implant supported by the density of the bone in the humerus 1332. The metric key 1354 indicates the color of the bone density metric 1352 that corresponds to the type of humeral implant. For example, a light color indicates that a stemless humeral implant can be implanted and a dark color indicates that a stemmed humeral implant can be implanted in the humerus 1332. Fig.43 1352, bone density measurement 1352 indicates that the density of trabecular bone is enough to support the implantation of stemless humeral implant. In certain embodiments, bone density measurement 1352 can be distinguished by different colors, patterns, shapes or other graphical representations of different types of humeral implants. In one embodiment, bone density measurement 1352 can even be a graphical representation of the type of humeral implant itself, such as the length of the handle of the humeral implant, or the image of the stemless type, etc.

[0340] The following examples are described herein.

[0341] Example 1: A system for modeling a soft tissue structure of a patient, the system comprising: a memory configured to store patient-specific image data of the patient; and a processing circuit configured to: receive the patient-specific image data; determine a patient-specific shape representing the patient's soft tissue structure based on the intensity of the patient-specific image data; and output the patient-specific shape.

[0342] Example 2: A system according to Example 1, wherein the processing circuit is configured to: receive an initial shape; determine multiple surface points on the initial shape; align the initial shape with patient-specific image data; identify one or more contours in the patient-specific image data, the one or more contours representing at least a partial boundary of the patient's soft tissue structure; and iteratively move the multiple surface points toward corresponding positions of the one or more contours to change the initial shape into a patient-specific shape representing the patient's soft tissue structure.

[0343] Example 3: A system according to Example 2, wherein the processing circuit is configured to identify one or more contours by: extending a vector from each surface point in at least one manner outward or inward from a corresponding surface point in a plurality of surface points; and determining, for each surface point, a corresponding position in the patient-specific image data that exceeds a threshold intensity value, wherein the corresponding position of at least one surface point in the plurality of surface points at least partially defines the one or more contours.

[0344] Example 4: A system according to any of Examples 2 and 3, wherein the processing circuit is configured to identify one or more contours by: determining a Hesse feature image from patient-specific image data, wherein the Hesse feature image represents a region of the patient-specific image data, the region including a higher intensity gradient between two or more elements; identifying one or more separation regions between a soft tissue structure and an adjacent soft tissue structure based on the Hesse feature image; and determining at least a portion of one or more contours as passing through the one or more separation regions.

[0345] Example 5: A system according to any of Examples 2 to 4, wherein the processing circuit is configured to determine corresponding locations in the patient-specific image data that exceed a threshold intensity value by determining corresponding locations in the patient-specific image data that are greater than a predetermined intensity value.

[0346] Example 6: A system according to Example 5, wherein a predetermined threshold intensity value represents a bone in the patient-specific image data, and wherein, for each corresponding position in the patient-specific image data that exceeds the predetermined threshold intensity value representing the bone, the processing circuit is configured to move the surface point to the corresponding position.

[0347] Example 7: A system according to any of Examples 2 to 6, wherein the processing circuit is configured to determine corresponding locations in the patient-specific image data that exceed a threshold intensity value by determining corresponding locations in the patient-specific image data that are less than a predetermined intensity value.

[0348] Example 8: A system according to any of Examples 2 to 7, wherein the processing circuit is configured to determine corresponding locations in the patient-specific image data that exceed a threshold intensity value by determining corresponding locations in the patient-specific image data that are greater than a difference threshold between the two, namely, the intensity associated with the corresponding surface point and the intensity at the corresponding location in the patient-specific image data.

[0349] Example 9: A system according to any one of Examples 2 to 8, wherein the processing circuit is configured to iteratively move multiple surface points toward corresponding positions of one or more contours by the following items, for each iteration of moving the multiple surface points: extending a vector starting from each of the multiple surface points and orthogonal to the surface including the corresponding surface point; determining a corresponding point in the patient-specific image data that exceeds a threshold intensity value for the vector of each surface point; determining a plurality of potential positions that are located within an envelope of the corresponding point and exceed the threshold intensity value in the patient-specific image data for each corresponding point, wherein the plurality of potential positions at least partially define the surface of the one or more contours; for each of the multiple potential positions, determining a corresponding normal vector that is orthogonal to the surface; for each of the corresponding normal vectors, determining an angle between the corresponding normal vector and the vector of the corresponding surface point; for each of the corresponding surface points, selecting one of the multiple potential positions that includes the minimum angle between the vector of the corresponding surface point and the corresponding normal vector of each of the multiple potential positions; and for each of the corresponding surface points, at least partially moving the corresponding surface point toward the selected one of the potential positions, wherein moving the corresponding surface point causes the initial shape to be modified toward the patient-specific shape.

[0350] Example 10: The system of Example 9, wherein the processing circuit is configured to move the corresponding surface point by at least half a distance between the corresponding surface point and the selected one of the potential locations.

[0351] Example 11: A system according to any of Examples 9 and 10, wherein the processing circuit is configured to iteratively move multiple surface points toward corresponding potential positions in one or more contours by: in a first iteration, moving each of the multiple surface points from an initial shape by a first corresponding distance to generate a second shape, the first corresponding distance being within a first tolerance of a first modified distance, the first tolerance being selected to maintain smoothness of the second shape; and in a second iteration after the first iteration, moving each of the multiple surface points by a second corresponding distance to generate a third shape from the second shape, the second corresponding distance being within a second tolerance of the second modified distance, wherein the second tolerance is greater than the first tolerance.

[0352] Example 12: A system according to any one of Examples 2 to 11, wherein the processing circuit is configured to identify one or more contours by: determining a Hesse feature image from patient-specific image data, wherein the Hesse feature image represents a region of the patient-specific image data, the region including a higher intensity gradient between two or more elements; identifying one or more separation regions between a soft tissue structure and an adjacent soft tissue structure based on the Hesse feature image; and determining at least a portion of one or more contours as passing through the one or more separation regions.

[0353] Example 13: A system according to any of Examples 2 to 12, wherein the processing circuit is configured to align the initial shape by aligning multiple locations on the initial shape with corresponding insertion locations on one or more bones identified in the patient-specific image data.

[0354] Example 14: A system according to any of Examples 2 to 13, wherein the initial shape and the patient-specific shape are three-dimensional shapes.

[0355] Example 15: A system according to any of Examples 1 to 14, wherein the initial shape includes a geometric shape.

[0356] Example 16: A system according to any of Examples 1 to 15, wherein the initial shape includes anatomical shapes representing soft tissue structures of multiple subjects different from the patient.

[0357] Example 17: The system of Example 16, wherein the anatomical shape comprises a statistically averaged shape generated from soft tissue structures imaged for a plurality of subjects.

[0358] Example 18: A system according to any of Examples 1 to 17, wherein the patient-specific image data includes computed tomography (CT) image data generated from the patient.

[0359] Example 19: A system according to any of Examples 1 to 18, wherein the soft tissue structure includes muscle.

[0360] Example 20: A system according to Example 19, wherein the muscle is associated with the patient's rotator cuff.

[0361] Example 21: A system according to any of Examples 1 to 20, wherein the patient-specific shape comprises a three-dimensional shape.

[0362] Example 22: A system according to any of Examples 1 to 21, wherein the processing circuit is configured to: determine a fat volume ratio of a patient-specific shape; determine an atrophy rate of a patient-specific shape; determine a range of motion of the patient's humerus based on the fat volume ratio and atrophy rate of the patient-specific shape of the patient's soft tissue structure; and determine a type of shoulder treatment for the patient based on the range of motion of the humerus.

[0363] Example 23: The system of Example 22, wherein the processing circuit is configured to determine the range of motion of the humerus of the patient by determining the range of motion of the humerus of the patient based on the fat volume ratio and atrophy rate of each muscle of the patient's rotator cuff.

[0364] Example 24: A system according to any of Examples 22 and 23, wherein the type of shoulder treatment is selected from one of an anatomical shoulder replacement procedure or a reverse shoulder replacement procedure.

[0365] Example 25: A system according to any one of Examples 1 to 24, wherein the processing circuit is configured to: apply a mask to a patient-specific shape; apply a threshold to the voxels under the mask; determine the fat volume based on the voxels under the threshold; determine the fat permeability value based on the fat volume and the volume of the patient-specific shape of the soft tissue structure; and output the fat permeability value of the soft tissue structure.

[0366] Example 26: A system according to any one of Examples 1 to 25, wherein the processing circuit is configured to: determine the bone-to-muscle dimensions of a patient's soft tissue structure; obtain a statistical mean shape (SMS) of the soft tissue structure; deform the SMS by satisfying a threshold of an algorithm so that the deformed version of the SMS fits the bone-to-muscle dimensions of the soft tissue structure; determine the atrophy rate of the soft tissue structure by dividing the SMS volume by the soft tissue structure volume; and output the atrophy rate of the soft tissue structure.

[0367] Example 27: A method for modeling a patient's soft tissue structure, the method comprising: storing patient-specific image data of the patient by a memory; receiving the patient-specific image data by a processing circuit; determining, by the processing circuit, a patient-specific shape representing the patient's soft tissue structure based on the intensity of the patient-specific image data; and outputting the patient-specific shape by the processing circuit.

[0368] Example 28: The method according to Example 27 further includes: receiving an initial shape; determining multiple surface points on the initial shape; aligning the initial shape with patient-specific image data; identifying one or more contours in the patient-specific image data, the one or more contours representing the boundaries of the patient's soft tissue structure; and iteratively moving multiple surface points toward corresponding positions of the one or more contours to change the initial shape into a patient-specific shape representing the patient's soft tissue structure.

[0369] Example 29: A method according to Example 28, wherein one or more contours are identified by: extending a vector from each surface point in at least one manner outward or inward from a corresponding surface point in a plurality of surface points; and determining a corresponding position in the patient-specific image data that exceeds a threshold intensity value for the vector of each surface point, wherein the corresponding position of at least one surface point in the plurality of surface points at least partially defines the one or more contours.

[0370] Example 30: According to the method of any one of Examples 28 and 29, identifying one or more contours includes: determining a Hesse feature image from the patient-specific image data, wherein the Hesse feature image represents a region of the patient-specific image data, the region including a higher intensity gradient between two or more voxels;

[0371] One or more separation regions between the soft tissue structure and an adjacent soft tissue structure are identified based on the Hessian feature image; and at least a portion of the one or more contours is determined to pass through the one or more separation regions.

[0372] Example 31: A method according to any one of Examples 28 to 30, wherein determining corresponding positions in the patient-specific image data that exceed a threshold intensity value includes: determining corresponding positions in the patient-specific image data that are greater than a predetermined intensity value.

[0373] Example 32: A method according to Example 32, wherein a predetermined threshold intensity value represents bone in patient-specific image data, and wherein the method further comprises: for each corresponding position in the patient-specific image data that exceeds the predetermined threshold intensity value representing bone, moving the surface point to the corresponding position.

[0374] Example 33: A method according to any one of Examples 28 to 32, wherein determining corresponding positions in the patient-specific image data that exceed a threshold intensity value includes: determining corresponding positions in the patient-specific image data that are less than a predetermined intensity value.

[0375] Example 34: A method according to any one of Examples 28 to 33, wherein determining a corresponding position in the patient-specific image data that exceeds a threshold intensity value includes: determining a corresponding position in the patient-specific image data that is greater than a difference threshold between the two, wherein the two include an intensity associated with the corresponding surface point and the intensity of the corresponding position in the patient-specific image data.

[0376] Example 35: A method according to any one of Examples 28 to 34, wherein iteratively moving multiple surface points toward corresponding positions of one or more contours includes, for each iteration of moving the multiple surface points: extending from each of the multiple surface points a vector starting from the corresponding surface point and orthogonal to the surface including the corresponding surface point; determining, for each surface point's vector, a corresponding point in the patient-specific image data that exceeds a threshold intensity value; determining, for each corresponding point, a plurality of potential positions that are within an envelope of the corresponding point and exceed the threshold intensity value in the patient-specific image data, wherein the plurality of potential positions at least partially define the surface of the one or more contours; determining, for each of the multiple potential positions, a corresponding normal vector orthogonal to the surface; determining, for each of the corresponding normal vectors, an angle between the corresponding normal vector and the vector of the corresponding surface point; selecting, for each corresponding surface point, one of the multiple potential positions that includes the minimum angle between the vector of the corresponding surface point and the corresponding normal vector of each of the multiple potential positions; and, for each corresponding surface point, at least partially moving the corresponding surface point toward the selected one potential position, wherein moving the corresponding surface point causes the initial shape to be modified toward a patient-specific shape.

[0377] Example 36: The method according to Example 35 further comprises: moving the corresponding surface point by at least half of the distance between the corresponding surface point and the selected potential position.

[0378] Example 37: A method according to any one of Examples 35 and 36, wherein iteratively moving multiple surface points toward corresponding potential positions of one or more contours includes: in a first iteration, moving each of the multiple surface points from an initial shape by a first corresponding distance to generate a second shape, the first corresponding distance being within a first tolerance of a first modified distance, the first tolerance being selected to maintain smoothness of the second shape; and in a second iteration after the first iteration, moving each of the multiple surface points by a second corresponding distance to generate a third shape from the second shape, the second corresponding distance being within a second tolerance of the second modified distance, wherein the second tolerance is greater than the first tolerance.

[0379] Example 38: A method according to any one of Examples 28 to 37, wherein the processing circuit is configured to identify one or more contours by: determining a Hesse feature image from patient-specific image data, wherein the Hesse feature image represents a region of the patient-specific image data, the region including a higher intensity gradient between two or more elements; identifying one or more separation regions between a soft tissue structure and an adjacent soft tissue structure based on the Hesse feature image; and determining at least a portion of the one or more contours as passing through the one or more separation regions.

[0380] Example 39: A method according to any one of Examples 28 to 38, wherein aligning the initial shape includes aligning multiple locations on the initial shape with corresponding insertion locations on one or more bones identified in the patient-specific image data.

[0381] Example 40: A method according to any one of Examples 38 to 30, wherein the initial shape and the patient-specific shape are three-dimensional shapes.

[0382] Example 41: A method according to any one of Examples 27 to 40, wherein the initial shape includes a geometric shape.

[0383] Example 42: A method according to any one of Examples 27 to 41, wherein the initial shape includes anatomical shapes representing soft tissue structures of multiple subjects different from the patient.

[0384] Example 43: The method of Example 42, wherein the anatomical shape comprises a statistically averaged shape generated from soft tissue structures imaged for a plurality of subjects.

[0385] Example 44: A method according to any one of Examples 27 to 43, wherein the patient-specific image data includes computed tomography (CT) image data generated from the patient.

[0386] Example 45: A method according to any one of Examples 27 to 44, wherein the soft tissue structure includes muscle.

[0387] Example 46: A method according to Example 45, wherein the muscle is associated with the patient's rotator cuff.

[0388] Example 47: A method according to any one of Examples 27 to 46, wherein the patient-specific shape comprises a three-dimensional shape.

[0389] Example 48: The method of any one of Examples 27 to 47, further comprising: determining a patient-specific shape fat volume ratio; determining a patient-specific shape atrophy rate; determining a range of motion of the patient's humerus based on the patient-specific shape fat volume ratio and atrophy rate of the patient's soft tissue structure; and determining a type of shoulder treatment for the patient based on the range of motion of the humerus.

[0390] Example 49: The method according to Example 48, wherein determining the range of motion of the humerus includes: determining the range of motion of the patient's humerus based on the fat volume ratio and atrophy rate of each muscle of the patient's rotator cuff.

[0391] Example 50: A method according to any one of Examples 48 and 49, wherein the type of shoulder treatment is selected from one of anatomical shoulder replacement surgery or reverse shoulder replacement surgery.

[0392] Example 51: The method according to any one of Examples 27 to 50 further includes: applying a mask to a patient-specific shape; applying a threshold to the voxels under the mask; determining the fat volume based on the voxels under the threshold; determining the fat permeability value based on the fat volume and the volume of the patient-specific shape of the soft tissue structure; and outputting the fat volume ratio of the soft tissue structure.

[0393] Example 52: The method of any one of Examples 27 to 51, further comprising: determining the bone-to-muscle scale of a patient's soft tissue structure; obtaining a statistical mean shape (SMS) of the soft tissue structure; deforming the SMS by satisfying a threshold of an algorithm so that the deformed version of the SMS fits the bone-to-muscle scale of the soft tissue structure; determining atrophy rate of the soft tissue structure by dividing the SMS volume by the soft tissue structure volume; and outputting the atrophy rate of the soft tissue structure.

[0394] Example 53: A computer-readable storage medium comprising instructions that, when executed by a processing circuit, cause the processing circuit to: store patient-specific image data of a patient in a memory; receive the patient-specific image data; determine a patient-specific shape representing a soft tissue structure of the patient based on the intensity of the patient-specific image data; and output the patient-specific shape.

[0395] Example 54: A system for modeling a soft tissue structure of a patient, the system comprising: a device for storing patient-specific image data of the patient; a device for receiving the patient-specific image data; a device for determining a patient-specific shape representing the patient's soft tissue structure based on the intensity of the patient-specific image data; and a device for outputting the patient-specific shape.

[0396] Example 101: A system for modeling soft tissue structures of a patient, the system comprising: a memory configured to store patient-specific computed tomography (CT) data of the patient; and processing circuitry configured to: receive the patient-specific CT data; identify one or more locations within the patient-specific CT data associated with one or more bone structures; align an initial shape with the one or more locations; modify the initial shape into a patient-specific shape representing the soft tissue structure of the patient; and output the patient-specific shape.

[0397] Example 102: A system according to Example 101, wherein the one or more locations associated with the one or more bone structures include one or more insertion locations of the one or more bone structures identified in the patient-specific CT data.

[0398] Example 103: A system according to Example 102, wherein the processing circuit is configured to: identify one or more contours in patient-specific CT data, wherein the one or more contours represent at least a partial boundary of the patient's soft tissue structure; and modify the initial shape into a patient-specific shape representing the patient's soft tissue structure based on the one or more contours.

[0399] Example 104: A system according to Example 103, wherein the processing circuit is configured to: determine multiple surface points on an initial shape; and modify the initial shape by iteratively moving the multiple surface points toward corresponding positions of one or more contours to change the initial shape into a patient-specific shape representing the patient's soft tissue structure.

[0400] Example 105: A system according to any of Examples 103 and 104, wherein the processing circuit is configured to modify the initial shape by: extending a vector from each surface point in at least one manner outward or inward from a corresponding surface point in a plurality of surface points; and determining, for each surface point, a corresponding position in the patient-specific CT data that exceeds a threshold intensity value, wherein the corresponding position for at least one surface point in the plurality of surface points at least partially defines one or more contours.

[0401] Example 106: A system according to any of Examples 103 to 105, wherein the processing circuit is configured to identify one or more contours by: determining a Hesse feature image from patient-specific CT data, wherein the Hesse feature image represents a region of the patient-specific CT data, the region comprising a higher intensity gradient between two or more elements; identifying one or more separation zones between a soft tissue structure and an adjacent soft tissue structure based on the Hesse feature image; and determining at least a portion of the one or more contours as passing through the one or more separation zones.

[0402] Example 107: A system according to any one of Examples 101 to 106, wherein the processing circuit is configured to align an initial shape with one or more locations by: determining a correspondence between each of the one or more locations and a corresponding point on the initial shape; determining an intensity profile along each correspondence in the patient-specific CT data; determining, for each of the one or more locations and based on the intensity profile of the corresponding correspondence, a distance between the location and a corresponding point on the initial shape; and orienting the initial shape in the patient-specific CT data based on the corresponding distances between the one or more locations and points on the initial shape.

[0403] Example 108: A system according to Example 107, wherein the processing circuit is configured to modify the initial shape into a patient-specific shape by scaling the initial shape to minimize a difference between the initial shape and a variation of the patient-specific CT data representing the soft tissue structure.

[0404] Example 109: The system of Example 108, wherein the processing circuit is configured to determine the patient-specific shape based on the following parameter equation:

[0405]

[0406] where s' is the initial shape of the soft tissue structure of multiple subjects different from the patient, λ i is an eigenvalue and v i are the eigenvectors of the covariance matrix representing the variation of the patient-specific CT data, and b i is the scaling factor that modifies the initial shape.

[0407] Example 110: A system according to any of Examples 101 to 109, wherein the initial shape includes anatomical shapes representing soft tissue structures of multiple subjects different from the patient.

[0408] Example 111: A system according to Example 110, wherein the anatomical shape comprises a statistically averaged shape generated from soft tissue structures imaged for multiple subjects.

[0409] Example 112: A system according to any of Examples 101 to 111, wherein the soft tissue structure includes muscle.

[0410] Example 113: A system according to Example 112, wherein the muscle is associated with the patient's rotator cuff.

[0411] Example 114: A system according to any of Examples 101 to 113, wherein the patient-specific shape comprises a three-dimensional shape.

[0412] Example 115: A system according to any of Examples 101 to 114, wherein the processing circuit is configured to: determine a fat volume ratio of a patient-specific shape; determine an atrophy rate of a patient-specific shape; determine a range of motion of the patient's humerus based on the fat volume ratio and atrophy rate of the patient-specific shape of the patient's soft tissue structure; and determine a type of shoulder treatment for the patient based on the range of motion of the humerus.

[0413] Example 116: A system according to Example 115, wherein the type of shoulder treatment is selected from one of an anatomical shoulder replacement surgery or a reverse shoulder replacement surgery.

[0414] Example 117: A method for modeling a patient's soft tissue structure, the method comprising: storing patient-specific computed tomography (CT) data of the patient in a memory; receiving the patient-specific CT data by a processing circuit; identifying one or more locations associated with one or more bone structures within the patient-specific CT data by the processing circuit; aligning an initial shape with the one or more locations by the processing circuit; modifying the initial shape into a patient-specific shape representing the patient's soft tissue structure by the processing circuit; and outputting the patient-specific shape by the processing circuit.

[0415] Example 118: A method according to Example 117, wherein the one or more locations associated with one or more bone structures include one or more insertion locations of the one or more bone structures identified in the patient-specific CT data.

[0416] Example 119: The method according to Example 118 further includes: identifying one or more contours in patient-specific CT data, wherein the one or more contours represent at least a partial boundary of the patient's soft tissue structure; and modifying the initial shape into a patient-specific shape representing the patient's soft tissue structure based on the one or more contours.

[0417] Example 120: The method according to Example 119 further includes: determining multiple surface points on an initial shape; and modifying the initial shape by iteratively moving the multiple surface points toward corresponding positions of one or more contours to change the initial shape into a patient-specific shape representing the patient's soft tissue structure.

[0418] Example 121: According to the method described in any of Examples 129 and 120, modifying the initial shape includes: extending a vector from each surface point in at least one manner outward or inward from the corresponding surface point in a plurality of surface points; and determining a corresponding position in the patient-specific CT data that exceeds a threshold intensity value for the vector of each surface point, wherein the corresponding position of at least one surface point in the plurality of surface points at least partially defines one or more contours.

[0419] Example 122: A method according to any one of Examples 119 to 121, wherein identifying one or more contours includes: determining a Hesse feature image from patient-specific CT data, wherein the Hesse feature image represents a region of the patient-specific CT data, the region comprising a higher intensity gradient between two or more elements; identifying one or more separation zones between a soft tissue structure and an adjacent soft tissue structure based on the Hesse feature image; and determining at least a portion of the one or more contours as passing through the one or more separation zones.

[0420] Example 123: A method according to any one of Examples 117 to 122, wherein aligning the initial shape with one or more positions includes: determining a correspondence between each of the one or more positions and a corresponding point on the initial shape; determining an intensity curve along each correspondence in the patient-specific CT data; determining, for each of the one or more positions and based on the intensity curve of the corresponding correspondence, a distance between the position and a corresponding point on the initial shape; and orienting the initial shape in the patient-specific CT data according to the corresponding distances between the one or more positions and the points on the initial shape.

[0421] Example 124: The method of Example 123, wherein the initial shape is modified to a patient-specific shape by scaling the i...

Claims

1. A system for modeling a soft tissue structure of a patient, the system comprising: a memory configured to store patient-specific computed tomography (CT) image data generated from the patient; and The processing circuit is configured to: receiving the patient-specific CT image data from the memory; determining a patient-specific shape representing the soft tissue structure of the patient based on the intensity of the patient-specific CT image data, wherein, in order to determine the patient-specific shape, the processing circuit is configured to: receiving an initial shape; determining a plurality of surface points on the initial shape; registering the initial shape with the patient-specific CT image data; identifying one or more contours in the patient-specific CT image data, the one or more contours representing at least a portion of a boundary of the soft tissue structure of the patient, wherein the one or more contours are identified by: determining a Hesse feature image from the patient-specific CT image data, wherein the Hesse feature image represents a region of the patient-specific CT image data that includes a higher intensity gradient between two or more voxels; identifying one or more separation regions between the soft tissue structure and adjacent soft tissue structures based on the Hessian feature image; and determining at least a portion of the one or more contours as passing through the one or more separation regions, iteratively moving the plurality of surface points toward corresponding positions of the one or more contours to change the initial shape into the patient-specific shape representing the soft tissue structure of the patient; and The patient-specific shape representing the soft tissue structure of the patient is output.

2. The system according to claim 1, wherein: The processing circuit is configured to identify, for each respective surface point in the plurality of surface points, the one or more contours by: extending a vector from the corresponding surface point at least one of outwardly or inwardly; and For the vector extending from the corresponding surface point, a corresponding position in the patient-specific CT image data exceeding a threshold intensity value is determined, wherein the corresponding position determined for the vector extending from at least one surface point of the plurality of surface points at least partially defines the one or more contours.

3. The system according to claim 2, wherein: The processing circuit is configured to determine the corresponding locations in the patient-specific CT image data that exceed the threshold intensity value by determining the corresponding locations in the patient-specific CT image data that are greater than a predetermined intensity value.

4. The system according to claim 3, wherein: The predetermined intensity value represents a bone in the patient-specific CT image data, and wherein the processing circuit is configured to move the corresponding surface point to the corresponding position.

5. The system according to claim 2, wherein: The processing circuit is configured to determine the corresponding locations in the patient-specific CT image data that exceed the threshold intensity value by determining the corresponding locations in the patient-specific CT image data that are less than a predetermined intensity value.

6. The system according to claim 2, wherein: The processing circuit is configured to determine the corresponding position in the patient-specific CT image data that exceeds the threshold intensity value by determining the corresponding position in the patient-specific CT image data to be greater than a difference threshold, wherein the difference threshold is the difference between the intensity associated with the corresponding surface point and the intensity of the corresponding position in the patient-specific CT image data.

7. The system according to claim 1, wherein: The processing circuit is configured to iteratively move the plurality of surface points toward respective positions of the one or more contours by, for each iteration of moving the plurality of surface points, for each respective surface point in the plurality of surface points: extending a vector from the respective surface point and orthogonally to a surface including the respective surface point; for the vector extending from the corresponding surface point, determining a corresponding point in the patient-specific CT image data that exceeds a threshold intensity value; determining, for the corresponding point, a plurality of potential locations in the patient-specific CT image data that are within an envelope of the corresponding point and that exceed the threshold intensity value, wherein the plurality of potential locations at least partially define a surface of the one or more contours; For each respective potential location in the plurality of potential locations: determining, for each potential position, a corresponding normal vector orthogonal to the surface; and determining an angle between the respective normal vector and the vector originating from the respective surface point; selecting one of the plurality of potential locations that includes a minimum angle between the vector extending from the corresponding surface point and the normal vector determined for the plurality of potential locations; and The respective surface point is at least partially moved towards the selected one of the potential positions, wherein moving the respective surface point causes the initial shape to be modified towards the patient-specific shape.

8. The system according to claim 7, wherein: The processing circuit is configured to move the corresponding surface point by at least half a distance between the corresponding surface point and a selected one of the potential locations.

9. The system according to any one of claims 7 and 8, wherein: The processing circuit is configured to iteratively move the plurality of surface points toward corresponding potential locations in the one or more contours by: in a first iteration, moving each of the plurality of surface points from the initial shape by a first corresponding distance to generate a second shape, the first corresponding distance being within a first tolerance of a first modified distance, the first tolerance being selected to maintain smoothness of the second shape; and In a second iteration following the first iteration, each of the plurality of surface points is moved a second corresponding distance to generate a third shape from the second shape, the second corresponding distance being within a second tolerance of a second modified distance, wherein the second tolerance is greater than the first tolerance.

10. The system according to claim 1, wherein: The processing circuit is configured to register the initial shape by aligning a plurality of locations on the initial shape with corresponding insertion locations on one or more bones identified in the patient-specific CT image data.

11. The system according to claim 1, wherein: The initial shape and the patient-specific shape are three-dimensional shapes.

12. The system of claim 1, wherein: The initial shape comprises a geometric shape.

13. The system of claim 1, wherein: The initial shape includes anatomical shapes representing the soft tissue structure of a plurality of subjects different from the patient.

14. The system according to claim 13, wherein: The anatomical shape comprises a statistically averaged shape generated from the soft tissue structures imaged for the plurality of subjects.

15. The system of claim 1, wherein: The soft tissue structures include muscles.

16. The system of claim 15, wherein: The muscle is associated with the patient's rotator cuff.

17. The system of claim 1, wherein: The patient-specific shape comprises a three-dimensional shape.

18. The system of claim 1, wherein: The processing circuit is configured to: determining a fat volume ratio of the patient-specific shape; determining a shrinkage rate of the patient-specific shape; determining a range of motion of the patient's humerus based on the fat volume ratio and the atrophy rate of the patient-specific shape of the soft tissue structure of the patient; and A type of shoulder treatment for the patient is determined based on the range of motion of the humerus.

19. The system of claim 18, wherein: The processing circuit is configured to determine the range of motion of the humerus of the patient by determining the range of motion of the humerus of the patient based on a fat volume ratio and an atrophy rate of each muscle of the rotator cuff of the patient.

20. The system according to any one of claims 18 and 19, wherein: The type of shoulder treatment is selected from one of an anatomical shoulder replacement surgery or a reverse shoulder replacement surgery.

21. The system of claim 1, wherein: The processing circuit is configured to: applying a mask to the patient-specific shape; applying a threshold to the voxels under the mask; determining a fat volume based on the voxel below the threshold; determining a fat infiltration value based on the fat volume and the volume of the patient-specific shape of the soft tissue structure; and The fat permeability value of the soft tissue structure is output.

22. The system of claim 1, wherein: The processing circuit is configured to: determining the bone-to-muscle dimensions of the soft tissue structure of the patient; obtaining a statistically average shape of the soft tissue structure; deforming the statistical mean shape in a manner that satisfies a threshold of an algorithm so that the deformed version of the statistical mean shape fits the bone-to-muscle scale of the soft tissue structure; determining a shrinkage rate of the soft tissue structure by dividing the statistical mean shape volume by the volume of the soft tissue structure; and The atrophy rate of the soft tissue structure is output.

23. A method for modeling a soft tissue structure of a patient, the method comprising: storing, by a memory, patient-specific CT image data generated from the patient; receiving, by processing circuitry, the patient-specific CT image data from the memory; Determining, by the processing circuit, a patient-specific shape representing the soft tissue structure of the patient based on the intensity of the patient-specific CT image data, wherein determining the patient-specific shape comprises: receiving an initial shape; determining a plurality of surface points on the initial shape; registering the initial shape with the patient-specific CT image data; identifying one or more contours in the patient-specific CT image data, the one or more contours representing boundaries of the soft tissue structures of the patient, wherein identifying the one or more contours comprises: determining a Hesse feature image from the patient-specific CT image data, wherein the Hesse feature image represents a region of the patient-specific CT image data including a higher intensity gradient between two or more voxels; identifying one or more separation regions between the soft tissue structure and adjacent soft tissue structures based on the Hessian feature image; and determining at least a portion of the one or more contours as passing through the one or more separation regions; and iteratively moving the plurality of surface points toward corresponding positions of the one or more contours to change the initial shape into the patient-specific shape representing the soft tissue structure of the patient; and The patient-specific shape representing the soft tissue structure of the patient is output by the processing circuit.

24. The method according to claim 23, wherein: For each respective surface point in the plurality of surface points, the one or more contours are identified by: extending a vector from the corresponding surface point at least one of outwardly and inwardly; and For the vector extending from the corresponding surface point, a corresponding position in the patient-specific CT image data exceeding a threshold intensity value is determined, wherein the corresponding position determined for the vector extending from at least one surface point of the plurality of surface points at least partially defines the one or more contours.

25. The method according to claim 24, wherein: Determining the corresponding position in the patient-specific CT image data that exceeds the threshold intensity value includes determining the corresponding position in the patient-specific CT image data that is greater than a predetermined intensity value.

26. The method according to claim 25, wherein: The predetermined intensity value represents a bone in the patient-specific CT image data, and wherein the method further comprises: moving the corresponding surface point to the corresponding position.

27. The method according to claim 24, wherein: Determining the corresponding positions in the patient-specific CT image data that exceed the threshold intensity value includes determining the corresponding positions in the patient-specific CT image data that are less than a predetermined intensity value.

28. The method of claim 24, wherein: Determining the corresponding position in the patient-specific CT image data that exceeds the threshold intensity value includes: determining the corresponding position in the patient-specific CT image data that is greater than a difference threshold between the two, wherein the two include the intensity associated with the corresponding surface point and the intensity of the corresponding position in the patient-specific CT image data.

29. The method of claim 23, wherein: Iteratively moving the plurality of surface points toward respective positions of the one or more contours by, for each iteration of moving the plurality of surface points, for each respective surface point in the plurality of surface points: extending a vector from the respective surface point and orthogonally to a surface including the respective surface point; determining, for the vector extending from the surface point, a corresponding point in the patient-specific CT image data that exceeds a threshold intensity value; determining, for the corresponding point, a plurality of potential locations in the patient-specific CT image data that are within an envelope of the corresponding point and that exceed the threshold intensity value, wherein the plurality of potential locations at least partially define a surface of the one or more contours; For each potential location in the plurality of potential locations: For each potential position, determining a corresponding normal vector orthogonal to the surface; determining an angle between the respective normal vector and the vector extending from the respective surface point; selecting one of the plurality of potential locations that includes a minimum angle between the vector extending from the corresponding surface point and the normal vector determined for the plurality of potential locations; and The respective surface point is at least partially moved towards the selected one of the potential positions, wherein moving the respective surface point causes the initial shape to be modified towards the patient-specific shape.

30. The method of claim 29, further comprising: The corresponding surface point is moved by at least half the distance between the corresponding surface point and the selected one of the potential positions.

31. The method according to any one of claims 29 and 30, wherein: Iteratively moving the plurality of surface points toward corresponding potential positions of the one or more contours comprises: in a first iteration, moving each of the plurality of surface points from the initial shape by a first corresponding distance to generate a second shape, the first corresponding distance being within a first tolerance of a first modified distance, the first tolerance being selected to maintain smoothness of the second shape; and In a second iteration following the first iteration, each of the plurality of surface points is moved a second corresponding distance to generate a third shape from the second shape, the second corresponding distance being within a second tolerance of a second modified distance, wherein the second tolerance is greater than the first tolerance.

32. The method of claim 23, wherein: Registering the initial shape includes aligning a plurality of locations on the initial shape with corresponding insertion locations on one or more bones identified in the patient-specific CT image data.

33. The method of claim 23, wherein: The initial shape and the patient-specific shape are three-dimensional shapes.

34. The method of claim 23, wherein: The initial shape comprises a geometric shape.

35. The method of claim 23, wherein: The initial shape includes anatomical shapes representing the soft tissue structure of a plurality of subjects different from the patient.

36. The method of claim 35, wherein: The anatomical shape comprises a statistically averaged shape generated from the soft tissue structures imaged for the plurality of subjects.

37. The method of claim 23, wherein: The soft tissue structures include muscles.

38. The method of claim 37, wherein: The muscle is associated with the patient's rotator cuff.

39. The method of claim 23, wherein: The patient-specific shape comprises a three-dimensional shape.

40. The method of claim 23, further comprising: determining a fat volume ratio of the patient-specific shape; determining a shrinkage rate of the patient-specific shape; determining a range of motion of the patient's humerus based on the fat volume ratio and the atrophy rate of the patient-specific shape of the soft tissue structure of the patient; and A type of shoulder treatment for the patient is determined based on the range of motion of the humerus.

41. The method of claim 40, wherein: Determining the range of motion of the humerus includes determining the range of motion of the humerus of the patient based on a fat volume ratio and an atrophy rate of each muscle of the rotator cuff of the patient.

42. The method according to any one of claims 40 and 41, wherein: The type of shoulder treatment is selected from one of an anatomical shoulder replacement surgery or a reverse shoulder replacement surgery.

43. The method of claim 23, further comprising: applying a mask to the patient-specific shape; applying a threshold to the voxels under the mask; determining a fat volume based on the voxel below the threshold; determining a fat infiltration value based on the fat volume and the volume of the patient-specific shape of the soft tissue structure; and The fat volume ratio of the soft tissue structure is output.

44. The method of claim 23, further comprising: determining the bone-to-muscle dimensions of the soft tissue structure of the patient; obtaining a statistically average shape of the soft tissue structure; deforming the statistical mean shape in a manner that satisfies a threshold of an algorithm so that the deformed version of the statistical mean shape fits the bone-to-muscle scale of the soft tissue structure; determining a shrinkage rate of the soft tissue structure by dividing the statistical mean shape volume by the volume of the soft tissue structure; and The atrophy rate of the soft tissue structure is output.

Images