Surgical planning system with automated defect quantification

Abstract

To provide improved surgery planning systems, including multiple surgery planning systems with automated defect quantification and population-based decision making support capabilities.SOLUTION: Certain aspects provide multiple techniques for creating multiple medical treatment plans, comprising the steps of: acquiring medical image data associated with an anatomy of a patient; creating a three-dimensional anatomy model based on the medical image data; fitting a statistical shape model to the three-dimensional anatomy model; determining one or more quantitative measurement results based on the fitted statistical shape model; and classifying a defect associated with the anatomy of the patient based on the one or more quantitative measurement results.SELECTED DRAWING: Figure 8

Description

Technical Field

[0001] (Cross - reference to related applications) This application claims the benefit of and priority to U.S. Provisional Patent Application No. 62 / 845,676, filed May 9, 2019, the entire content of which is incorporated herein by reference.

[0002] Multiple aspects of the present disclosure relate to a plurality of surgery planning systems, including a plurality of surgery planning systems having automated defect quantification and population - based decision - making support capabilities.

Background Art

[0003] Conventional surgical planning tools handle preoperative planning procedures. They address conventional problems associated with a particular surgery, such as the size and design of various components including, for example, surgical instruments and implants, and the position and orientation of implants and fixation devices. They typically acquire the medical images of a patient as input, thereby enabling a user (a medical professional or a non - medical professional such as a technician or an engineer) to make decisions based only on the information available in those images.

Summary of the Invention

[0004] Certain embodiments provide a method for creating a plurality of medical treatment plans, including: obtaining medical image data related to a patient's anatomy; creating a three-dimensional anatomical structure model based on the medical image data; fitting a statistical shape model to the three-dimensional anatomical structure model; determining one or more quantitative measurement results based on the fitted statistical shape model; and classifying a defect related to the patient's anatomy based on the one or more quantitative measurement results.

[0005] Further embodiments provide a method for determining treatment for an anatomical defect, the method including: obtaining medical image data related to a patient's anatomy; creating a three-dimensional anatomical structure model based on the medical image data; fitting a statistical shape model to the three-dimensional anatomical structure model; identifying a defect based on the three-dimensional anatomical structure model and the statistical shape model; determining a default treatment based on the identified defect; receiving patient population data related to a plurality of other patients having the identified defect, the patient population data including a plurality of patient population data subsets related to various treatments of the identified defect; generating a visualization including a representation of each patient population data subset based on at least one patient characteristic and a representation of the patient based on the at least one patient characteristic; and selecting a final treatment for the patient.

[0006] Other embodiments include a plurality of processing systems configured to perform the above-described plurality of methods and the plurality of methods described herein; a non-transitory computer-readable medium including a plurality of instructions that, when executed by one or more processors of the processing system, cause the processing system to perform the above-described plurality of methods and the plurality of methods described herein; a computer program product embodied on a computer-readable storage medium including code for performing the above-described plurality of methods and the plurality of methods further described herein; and a processing system including means for performing the above-described plurality of methods and the plurality of methods further described herein.

[0007] The following description and the related drawings detail specific exemplary features of one or more embodiments.

Brief Description of the Drawings

[0008] The accompanying drawings illustrate certain aspects of one or more embodiments and, therefore, should not be considered as limiting the scope of the disclosure.

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21A

Figure 21B

Figure 21C

Figure 21D

Figure 22

Figure 23

Figure 24

Figure 25

Figure 26

Figure 27

Figure 28

Figure 29

Figure 30

[0009] For ease of understanding, the same reference numbers are used whenever possible to refer to the same elements common to multiple drawings. It is intended that multiple elements and multiple features of one embodiment can be advantageously incorporated into other multiple embodiments without further explanation.

Mode for Carrying Out the Invention

[0010] Multiple aspects of the present disclosure provide multiple devices, multiple methods, multiple processing systems, and multiple computer-readable media for multiple surgical planning systems, including multiple surgical planning systems having automated defect quantification and population-based decision-making support capabilities.

[0011] The multiple surgical planning systems described herein solve multiple problems associated with multiple conventional surgical planning tools.

[0012] For example, conventional multiple planning tools do not provide information about healthy anatomy and thus do not enable a user to appropriately evaluate the magnitude and location of an injury. In contrast, the multiple surgical planning tools described herein provide an automated defect classification system that describes characteristics of not only damaged anatomy but also healthy anatomy. Thus, the multiple surgical planning systems described herein overcome, inter alia, the problem of designing preoperative plans based only on damaged anatomy such as bone and cartilage. In this regard, the multiple surgical planning systems described herein provide a better, more detailed, and automated visual representation of the damaged bone anatomy based on defect classification.

[0013] As another example, while providing planning assistance for a particular surgery, conventional planning tools provide little assistance in selecting from among such particular surgeries. The multiple surgical planning systems described herein have different starting points and also enable a user to make more important, high-level surgical decisions. Thus, the multiple surgical planning systems described herein are more transparent to users such as surgeons. Specifically, the multiple surgical planning systems described herein provide statistical data that enables a surgeon to evaluate where a patient is located within a patient population, such that the surgeon can make an informed decision during preoperative planning. The transparency of the system enables the user to trace back any decision by providing the user with a complete patient profile. The surgical planning system is also aimed at reducing the number of manual interactions required to create a preoperative surgical plan.

[0014] The systems and methods disclosed in the present invention consist of a plurality of interconnected (related to each other) parts.

[0015] [Defect Quantification and Classification] Multiple embodiments of a defect quantification system can implement multiple methods for calculating the characteristics of defects or deformities in a patient's body, such as bones, organs, musculoskeletal regions, or any other anatomical part, starting from medical images. In some embodiments, the multiple defect quantification systems and methods described herein can be subsystems, modules, or other essential parts of a surgical planning system.

[0016] For example, the shape and size of a bone defect hold useful information for surgeons, implant or surgical tool manufacturers, implant positioning software providers, educational institutions, and, if necessary, the patient. Many classification systems have been used to describe the shape and size of bone defects. For example, the Paprosky classification system for the hip, the Dorr, Insall, and Rand classification systems for the knee, the Wallace, Walsch, and Antuna classification systems for the shoulder, and so on.

[0017] Conventional methods use multiple qualitative measurements based on standard radiography or two-dimensional (2D) computed tomography (CT) scans. They rely on the user to visually identify anatomical landmarks and infer where the defect begins and what the normal anatomical structure, i.e., what it would look like if it were a healthy anatomical structure. For example, in the case of bone or cartilage defects such as erosion of the glenoid, acetabulum, tibial plateau, vertebra, craniomaxillofacial region, or any other bone anatomical structure or cartilage surface, existing techniques would rely on the user to observe anatomical landmarks or abnormal bone geometry to assess which part of the anatomical structure has been eroded. However, without a shape of an undamaged anatomical structure as a reference, this generally cannot be more than a mere assessment. Similarly, in the evaluation of soft tissue or organs such as the heart, lungs, kidneys, and brain, under or overdeveloped parts, lobes, regions, chambers, vessels can be identified through visual evaluation or rules of thumb. However, without a shape of a normal or healthy anatomical structure as a reference, a truly meaningful quantification of such under or overdevelopment is impossible.

[0018] In addition, conventional methods use qualitative measurement results based on 2D images. These measurements are not accurate because some information is lost when converting a 3D object to its 2D representation. That is, the actual anatomical structure of a patient exists in 3D, but the images used to plan surgery are captured in 2D. These 2D techniques are unreliable as a result of their qualitative nature and also due to variations in imaging protocols and circumstances. For example, the scale of an object in a 2D X-ray depends on the distance between the radiation source and the acquisition plane, and the distance between the subject and the acquisition plane. Similarly, parallax effects also depend on those distances and whether the radiation source is stationary or moving. Furthermore, the orientation of the patient with respect to the radiation source and the acquisition plane affects the projection of the anatomical structure.

[0019] In a plurality of systems described herein, a defect or deformation is measured from a medical image of a patient using a model of a healthy body part as a reference (or as a template). The size of the defect can be calculated by a plurality of methods by measuring the distance between the points or surfaces of the actual damaged or deformed anatomical structure of the patient and the topological counterparts of such points or surfaces on the reference model. The distance can be measured, for example, by projecting a plurality of rays from a virtual model of the healthy anatomical structure and calculating the distance along those rays from the healthy body part to the damaged body part. A virtual model of the patient's anatomical structure can be obtained by segmenting a medical image of the actual patient's anatomical structure. A virtual model of the corresponding healthy anatomical structure is obtained by different methods, as described below. To enable the user to perform a visual assessment of the damage or deformation, 2D or 3D virtual models of the damaged or deformed body part and the healthy body part can be overlaid and presented to the user. One or both of these models can be shown semi-transparent.

[0020] Reference models of normal or healthy anatomical structures can arise from various sources. For example, a mirror image of a healthy contralateral anatomical part can be used. For this purpose, medical images of the contralateral anatomical part can be segmented, and the resulting virtual model can be mirrored.

[0021] In some embodiments, the various methods disclosed herein can perform quantitative measurements and predict the nature of defects or deformities by reconstructing healthy body parts using a 3D statistical shape model (SSM). Statistical shape modeling can be used to predict the original or healthy anatomical shape without the need for actual healthy bone (images). In such embodiments, a virtual model of a healthy anatomical structure can be obtained by fitting an SSM of the healthy anatomical structure to a portion of the patient's anatomical structure (medical image or virtual model).

[0022] In general, an SSM is a mathematical model that represents the average shape and shape variation within a population. Each shape generated by the SSM can be represented by a plurality of shape coefficients, referred to as SSM parameters.

[0023] In some examples, the method is performed, for example, in a 3D virtual model, a 3D biomechanical model (musculoskeletal model), an SSM, and / or an SSM instance, such that there is no approximation or conversion of measurement results between the 2D representation and the 3D world.

[0024] As an example, a fully automated defect classification system can be used to describe glenoid bone loss without the need for a healthy contralateral reference scapula, using three-dimensional measurement results based on a scapula and / or humerus model. In a plurality of other embodiments, the automated defect classification system can likewise be used to measure defects or deformities in other body parts such as the heart, knee, hip, spine, foot, lung, other joints, etc.

[0025] One example of a method can include: (1) acquiring (a plurality of) medical images of a patient having glenoid bone defects or arthroplasty; (2) segmenting the scapula to obtain a virtual three-dimensional surface model, for example, using Mimics by MATERIALISE (registered trademark); and (3) as shown in FIG. 1, fitting a statistical shape model (SSM) of a plurality of healthy scapulae to the healthy surface area of the patient's scapula.

[0026] Continuing with this example, the SSM should describe the shape of healthy scapulae within the population to which the patient belongs. By fitting the SSM to the healthy part of the patient's anatomy, the unhealthy surface of the scapula (e.g., the glenoid in this example) will be reconstructed. The shape correlation embedded within the SSM will create a reconstructed glenoid that statistically has the highest probability of resembling what the original, healthy or native shape of the currently unhealthy area would have looked like.

[0027] [Example of SSM Fitted to the Healthy Area of Bone] FIG. 1 shows an example of an SSM 102 fitted to reconstruct the original glenoid surface 106 with respect to the healthy area (e.g., 104) of the scapula.

[0028] To fit an SSM to partial data of a healthy anatomical structure or the like, various techniques such as posterior shape modelling can be used, and as a result, missing data (e.g., bone lost due to bone erosion) can be predicted. However, such techniques require an a priori (prior) identification of healthy regions and damaged or deformed regions. This process is known to exhibit high inter-user variability and high intra-user variability. Therefore, it is beneficial to automate this process.

[0029] [Division of SSM into multiple regions to improve fitting error] In one embodiment of the automated method, the SSM is subdivided into a plurality of phase regions (topological regions) such as regions 202 to 212 in the example of FIG. 2. For each of these multiple regions, it is tested whether including that region within the region used to fit the SSM results in a reduction or an increase in the fitting error. If including a region results in an unacceptable fitting error or an increase in the fitting error, that region is assumed to be damaged or deformed and excluded from the fitting. The SSM is then fitted to a subset of the remaining regions, resulting in an SSM instance. This represents what the patient's anatomical structure would have looked like in a healthy or non-deformed state.

[0030] In the example of FIG. 2, the surface of the SSM representing the scapula is divided into six regions: the base region 202, the acromion region 204, the coracoid region 206, the neck region 208, the acromion tip region 210, and the glenoid region 212. This is just one example, and other subdivisions are possible, such as subdivision into different regions, or more or fewer regions.

[0031] Thus, one example method may proceed as follows. First, based only on the points within the base region 202, the SSM shape is fitted to the target shape. After the convergence of the shape coefficients, the fit error is calculated as the root mean square error (RMSE) between the points on the SSM shape used for the fit and the identified corresponding points on the target shape.

[0032] If the fit error remains below a selected threshold, a second fit is performed using the points within the acromion region 204. If the fit error exceeds the threshold, the acromion region 204 of the target shape is considered unhealthy and the acromion region 204 is excluded from the subset of the phase regions. Subsequently, the same selection procedure is repeated for the points within the glenoid region 206, the acromion tip 210, and the neck region 208 in this example. In some cases, the glenoid fossa region 212 may be expected to be eroded and as a result may not be used for the fit.

[0033] For example, if both the acromion region 204 and the glenoid region 206 are excluded from the fit based on a fit error exceeding the threshold, in some cases, no further tests are performed on the acromion tip region 210 and the neck region 208.

[0034] In various embodiments, various fit error thresholds may be used. For example, a study of sensitivity has shown that a fit error threshold of 1.7 mm produces good results. Other values of the fit error threshold may also be selected. By way of example, fit error thresholds such as 0.5 mm, 0.6 mm, 0.7 mm, 0.8 mm, 0.9 mm, 1.0 mm, 1.1 mm, 1.2 mm, 1.3 mm, 1.4 mm, 1.5 mm, 1.6 mm, 1.8 mm, 1.9 mm, 2.0 mm, 2.5 mm, 3.0 mm, etc. may be selected.

[0035] Similar approaches can be applied to other anatomical structures. Thus, in generalizing the process, an anatomical structure can be subdivided into a plurality of topological regions (e.g., 202-212 in FIG. 2). A first region (e.g., the base region 202 in FIG. 2) can be selected to initiate a subset of the topological regions, and in some cases, this can be a region away from the defect or deformation. Next, the first region can be fitted, and a fitting error can be calculated and compared to a threshold such as the above-described threshold. Subsequently, additional topological regions can be added to the subset, and the subset can be fitted to the target model.

[0036] After each topological region is added to the subset, the fitting error can be recalculated, and depending on whether the calculated fitting error exceeds a set threshold such as the above-described threshold, additional topological regions can be removed from the subset or kept within the subset. To speed up the process, a plurality of topological regions that are not directly connected to the base region can be ignored if one or more intervening regions are classified as damaged or deformed. To further speed up the process and also to improve the results, topological regions known to be damaged or deformed can also be ignored.

[0037] By analyzing a bone defect (e.g., a glenoid bone defect) by comparing its shape to the predicted original shape (e.g., of an undamaged glenoid bone), quantitative measurement results such as, for example, in the case of the glenoid bone, glenoid vault loss, glenoid vault loss percentage, glenoid erosion area, glenoid erosion area percentage, maximum erosion depth, and others can be obtained.

[0038] [Distance measurement techniques for comparing a plurality of anatomical structure shapes] In one embodiment, to compare a plurality of anatomical structure shapes (e.g., between a predicted shape and an actual shape), distances between a plurality of topologically equivalent points on a plurality of models of each shape can be measured. For example, to list a few options, the distance between the closest points, or the distance between points along a ray emitted from one model to the other model, can be measured.

[0039] For example, for a substantially spherical or hemispherical anatomical part, such as the acetabulum 302 of FIG. 3, the ray 304 can be emitted concentrically outward from the center 306 of the sphere, as shown in the example of FIG. 3.

[0040] As another example, for a substantially flat or planar anatomical part, the ray 404 can be emitted parallel and perpendicular to the best-fitting plane 402, as shown in FIG. 4.

[0041] As yet another example, for an elongated anatomical part, the ray can be emitted outward and perpendicular to the central axis of the anatomical part. For other anatomical parts, the ray can be emitted perpendicular to the surface of the SSM instance. Note that these are just a few options, and it is possible to have multiple other ray-casting strategies or combinations of strategies.

[0042] Accordingly, the plurality of methods described herein can automatically calculate SSM-based metrics such as, for example: glenoid vault loss (total volume of the glenoid vault lost due to bone erosion), glenoid vault loss percentage (percentage of the volume of the glenoid vault lost due to bone erosion), local vault loss percentage (vault loss in the superior, inferior, anterior, and posterior regions), erosion area (surface area of the glenoid cavity affected by bone erosion), maximum erosion depth (maximum distance measured between the actual anatomical structure surface and a healthy reference model), erosion area percentage (percentage of the surface area of the glenoid cavity affected by bone erosion), subluxation distance, etc. Although the glenoid is used as an example herein, it should be noted that similar metrics can be calculated for other anatomical parts such as other bones, joints, and others. Based on this calculation, the plurality of systems described herein can automatically classify the defects.

[0043] [Examples of measurements of metrics related to bone loss] When using the glenoid bone as an example, the glenoid roof loss percentage measurement indicates how much glenoid roof volume has been eroded and represents the severity of the glenoid bone defect. The superior, anterior, inferior, and posterior roof loss percentages indicate how much of the roof has been eroded in each anatomical region or quadrant of the glenoid and provide a better understanding of the shape of the defect. The maximum erosion depth describes the amount of bone erosion at the deepest point of erosion. This measurement can assist the surgeon in determining whether to use a bone graft or ream during surgery. The erosion area percentage indicates how much of the original glenoid surface is no longer intact and provides an indication of the amount of possible implant-bone support. Finally, the subluxation distance and region describe the amount and direction of humeral subluxation, which provides a better understanding of the cause of the glenoid bone defect.

[0044] Figure 5 shows an example for measuring metrics related to bone loss in the glenoid bone.

[0045] To measure these metrics, a ray-casting algorithm (such as the one described above) can be used. For example, first, a plane (e.g., 506) is fitted to pass through the glenoid surface of the SSM to be fitted, and parallel rays are projected in the opposite direction of the surface normal from the glenoid points of the SSM shape to be fitted. The distance at which ray i intersects the SSM shape to be fitted is referred to as the vault depth (d i vault )(e.g., 502) and has a maximum value d max (e.g., 504) selected as such.

[0046] Then, the amount of bone erosion is evaluated by emitting rays (e.g., 502 and 506) parallel to the normal of the glenoid plane from the glenoid point of the conforming SSM shape towards the bone defect. The measured distance at which the ray intersects the bone defect is defined as the erosion depth (d i ero )(e.g., 506), and is limited up to d max (e.g., 504). If the erosion depth is infinite, there is no bone at that location. Next, the loss depth (d i loss ) is defined as the depth of the lost lid. The loss depth is similar to the erosion depth, except that it cannot exceed the lid depth.

[0047] Thus, in one example, for each ray i: d i vault >d max then: d i vault =d max , d i ero >d max and d i ero ≠infinite then: d i ero =d max , d i ero ≦d i vault then: d i loss =d i ero , d i ero >d i vault then: d i loss =d i vault .

[0048] Based on the depth measurement results, nine parameters can be calculated to describe the glenoid bone defect.

[0049] For example, the vault volume is calculated as the sum of all vault depths multiplied by the size (A i ) of the corresponding surface element. Similarly, the vault loss volume is calculated as the sum of the loss depths multiplied by the corresponding surface area. And the vault loss percentage is calculated as the percentage of the vault loss volume to the vault volume.

[0050] For the superior (sup), anterior (ant), inferior (inf), and posterior (post) vault loss percentages, the glenoid fossa center point is used to divide the glenoid fossa surface into four quadrants. The vault loss percentage in these regions is equal to the local vault loss volume divided by the local vault volume.

[0051] Next, in one example, the maximum erosion depth is calculated as the 95th percentile value of all erosion depth values. The erosion area is calculated as the area A i of all surface elements where the erosion depth is greater than one-third of the maximum erosion depth. To obtain the erosion area percentage, in one example, the erosion area is divided by the total area of the glenoid fossa. After projecting the center point of the humeral head onto the glenoid fossa plane, the subluxation distance is calculated as the in-plane distance from the center point of the humeral head to the center point of the glenoid fossa. The subluxation region is defined as the region (sup, ant, inf, post) where the center point of the humeral head is projected onto the glenoid fossa.

[0052] Thus, in one example: Vault volume = Σ i (d i vault ·A i)、 Cover loss volume = Σ i (d i loss ·A i )、 Cover loss percentage = (Cover loss volume) / (Cover volume), For all i within the region, local cover volume = Σ i (d i vault ·A i )、 For all i within the region, local cover loss volume = Σ i (d i loss ·A i )、 Local cover loss percentage = (Local cover loss volume) / (Local cover volume), Maximum erosion depth = p95(d i ero )、 d i ero > 1 / 3 maximum erosion depth for all i, erosion area = Σ i A i 、 Erosion area percentage = (Erosion area) / (Σ i A i ).

[0053] In some examples, multiple classification systems such as the Antuna classification in the frontal view (as described above) and the Wallace classification in the axial view can be combined. This advantageously provides the user (e.g., a surgeon) with a three - dimensional classification of defects as compared to conventional two - dimensional classifications.

[0054] Note that for other anatomical parts such as other joints, other bones, organs (heart, lungs, kidneys, brain and others), similar quantifications can be performed and damage, deformities or diseases can be evaluated. Based on this quantification, similar classification systems can be defined. The system and method use an appropriate and / or known classification system or a combination thereof based on the body part requiring treatment.

[0055] [Pre - operative surgical planning tool] For example, existing pre-operative planning tools such as SurgiCase Knee Planner by MATERIALISE (registered trademark) provide the possibility of generating a pre-operative surgical plan for a specific type of surgery (generally related to implants of a specific type, brand or product line). Pre-operative planning generally starts after the following major surgical decisions have been made by the surgeon: type of surgical treatment, type of implant and surgical instruments to be used, standard implant versus patient-matched, etc.

[0056] Furthermore, these decisions are based on medical images taken from the patient. For orthopedic treatments, for example, these medical images can depict the anatomical structure of the damaged bone / cartilage. Existing planners generate an initial or default plan based on the damaged anatomical structure (e.g., bone and / or cartilage), which is then reviewed by the surgeon. During the review, the surgeon may propose specific changes such as the position or size of the implant. The changes are then incorporated by the planner and a new pre-operative plan is generated for use during the actual surgical procedure.

[0057] Unfortunately, since existing planners only acquire medical images as input, the pre-operative plan only takes into account information visible in those medical images. The pre-operative plan does not address all the complexities associated with the surgery that the surgeon encounters in the operating room, or any aspect that cannot be easily derived from the medical images. This could affect the surgical outcome, the risk of intra-operative or post-operative complications, or patient satisfaction.

[0058] A surgical planning system can use integrated prediction techniques based on one or more known preoperative plan sets to utilize more than just the patient-specific medical images. For example, such a planning system can supply historical data from preoperative plans, data collected during surgery, and data collected after surgery. Further, the planning system can select multiple preoperative plans into a preoperative plan set and then apply prediction techniques such as techniques based on machine learning, deep learning, neural networks, or other artificial intelligence (AI) to create an integrated preoperative plan and can propose changes to the user. However, this method of preoperative plan generation is generally not transparent to the surgeon. That is, the surgeon does not know how or why the planner incorporated the proposed changes, which patient-specific characteristics led to the proposed changes, how sensitive the system is to those characteristics, or the impact of those changes on the patient. Thus, although the system itself can be self-learning, it does not enable the surgeon to make an informed decision.

[0059] The multiple systems disclosed herein overcome the drawbacks of existing surgical planning tools by providing more information to the surgeon and by functioning as a guide to the surgeon. As a guide, multiple embodiments of the multiple systems described herein provide timely proposals, advice, and warnings, along with detailed information justifying such proposals, advice, and warnings, enabling the surgeon to make an information-based decision. The control of the system is under the surgeon's control so that the surgeon can consciously make any decision, making it a transparent and user-friendly system. The multiple systems disclosed herein advantageously reduce the time spent in the operating room and the changes the surgeon has to deal with in the operating room, increase the probability of a favorable surgical outcome, and as a result, overall reduce the number of revision surgeries the patient may need.

[0060] The multiple systems described in this specification may use multiple feedback loops to provide information to a surgeon, such as through proposals, warnings, advice, and / or default preoperative plans. This involves establishing one or more interconnected databases.

[0061] [Surgical planning workflow] (For example, as performed by the multiple surgical planning systems described in this specification) A surgical planning method may include: (1) loading a plurality of medical images; (2) processing the plurality of medical images to identify, for example, a plurality of anatomical landmarks and / or create a virtual 3D model of one or more anatomical structures; (3) automatically creating a default surgical plan that is generally based on multiple geometric calculations based on the identified plurality of landmarks and typically includes selection of one or more implants, multiple implant sizes, multiple positions and orientations for all implants, corresponding multiple resection or multiple reaming procedures, etc.; (4) enabling a clinician to modify the default plan to obtain an approved preoperative plan; and (5) creating a preoperative plan available for intraoperative implementation. In some embodiments, the preoperative plan may be used, for example, in a navigation system, in a robotics system, for designing a patient-specific guide, in an augmented reality and / or virtual reality system, and for other purposes.

[0062] For example, FIG. 6 shows a workflow of a conventional surgical planning method and tool, including steps 602 to 618.

[0063] A database or other data store may be used to store (memorize) the approved plan, along with the relevant patient data such as medical images and landmark information and any virtual 3D models in the database. In addition, the multiple systems described in this specification add one or more feedback loops to the workflow shown in FIG. 6.

[0064] For example, as shown in FIG. 7, the first feedback loop 702 can mine information from a plurality of approved pre-operative surgical plans for use before or during the planning process and store it in the database 708. The second feedback loop 704 can collect information during the operation, store the data in the database 708, and mine the information for use before or during the planning process. The third feedback loop 706 can collect information after the operation, store the data in the database 708, and mine the information for use before or during the planning process.

[0065] Further improvements to the data flows shown in FIGS. 6 and 7 are shown in FIG. 8. The available historical data within the database is used to perform a historical-data analysis 802. This either associates patient characteristics with the planning decision-making or associates one or more planning parameters with the surgical outcome. Further, the results of this historical data analysis can be presented to a user (e.g., a surgeon) such that the planning parameters or the position of the patients within the population are shown, respectively, (e.g., at 804) together with the distribution of the surgical outcomes or the planning decision-making over the population. Advantageously, presenting this information does not force the user to blindly choose whether to accept or reject the proposed plan changes. Rather, it shows the user which planning decision-making options or parameter values are appropriate and to what extent they are more appropriate than other options or values, along with the possible postoperative scenarios.

[0066] For example, when considering options A, B, and C, instead of simply suggesting "take option A", the system can show how the patient population is distributed across options A, B, and C, and where that patient lies within the population. From the presentation of the results of the historical data analysis, the user can see not only whether the patient lies exactly on option A, or rather on the boundary between option A and option B, but also whether that boundary is clearly defined or rather a broad one with a smooth transition.

[0067] [Surgical planning data and databases] The multiple systems described herein may utilize one or more databases. The one or more databases are connected to various parts of the surgical planning system via one or more feedback loops. For example, data can be collected and stored in the database at one or more stages of the workflow, as described above with respect to FIG. 7.

[0068] In some implementations, the data collected can be (logically or physically) split into subsets such as patient data, preoperative data including a collection of pre-existing plans (i.e., preoperative plans already used for future preoperative plan optimization), retrospective data, intraoperative data, and postoperative data. The links between data that are in different subsets but relate to an individual patient are maintained; in other words, the database keeps track of which patient data, preoperative plans, intraoperative data, and postoperative data belong to the same patient. As described above, the data can be stored within a single database or within various databases.

[0069] Which of these types of data are stored in the (multiple) databases depends on which feedback loops are implemented within the system. A subset of the patient data is always stored. However, a basic system may implement, for example, a feedback loop for only the approved preoperative plan. Other systems may also implement feedback loops for intraoperative data and / or postoperative data. Other combinations are possible. One or more feedback loops may be called at regular intervals. In some embodiments, in the case of a replacement surgery, all feedback loops may be called to obtain the entire patient profiled from the previous surgery.

[0070] In some embodiments, the multiple systems described herein may operate locally or "on-premises". In this case, the (multiple) databases may include only data regarding one or more local users, such as a surgeon, physician, or clinician, or their team. In other embodiments, the system may be a network-based system, such as a web-based system or a cloud-based system, in which case the (multiple) databases may include data regarding a larger user base.

[0071] Patient data can be stored in the form of one or more of medical images, age, gender, weight, height, ethnicity, lifestyle, activity level, personal information such as medical history, and any data collected during pre-operative examinations (tests) such as complaints, pain scores, gait measurements, range of motion measurements, degenerative or congenital defects, sports or age-related injuries, genetic information, dental casts, and others. In some embodiments, the patient data can be anonymized to protect patient privacy or to comply with various patient privacy regulations such as the Health Insurance Portability and Accountability Act (HIPAA) or the General Data Protection Regulations (GDPR).

[0072] Pre-operative data can be stored, for example, in the form of a pre-operative treatment plan (e.g., 614 in FIGS. 6-8). The pre-operative treatment plan can alternatively be referred to as a pre-op plan or a pre-op surgical plan. The pre-operative data can capture some or all of the medical decision-making related to the treatment of the patient's medical condition, such as the type of treatment (both invasive and non-invasive); the type, brand, product line, size, implantation location and orientation of the planned implant (if any); the delivery system and approach for any implant; the design of any patient-specific instruments (if any); the details of any reaming process; the type or design of any defect filling components such as autografts, allografts, porous structures, and other modalities.

[0073] Intra-operative data (e.g., 710 in FIGS. 7-8) can be stored in any form of data captured during the operation, such as measurement results, the positions of anatomical landmarks identified during the operation, observation results, or the occurrence of intra-operative complications. Intra-operative data can relate to information that cannot be easily derived from medical images or pre-operative examinations (tests), such as information regarding soft tissue, muscle, muscle attachment points, muscle ruptures, tendons, ligaments, ligament tension, etc. Intra-operative data can also include any changes made during the operation to the pre-operative plan. Intra-operative data can also include synthetic data. In one example, the synthetic data can be data that cannot be quantified, such as ligament force in the case of the knee joint, but can be recorded because it affects the surgical outcome. This can be stored in the form of a biomechanical model.

[0074] Post-operative data (e.g., 618 in FIGS. 6-8) can be stored in any form of data captured after the operation, such as the occurrence of any complications, any data captured during post-operative examinations (tests), pain scores, patient satisfaction, function scores, revisions, post-surgery imaging, recovery time, rehabilitation time, rehabilitation methods of treatment, details and observation results of physiotherapists; (if any) range of motion measurements, etc.

[0075] Data can be entered into the system either manually or automatically through a surgical planning system, through any device used during the operation such as a navigation system, a robotics system or an augmented reality (AR) or virtual reality (VR) system, through an electronic access device, through a wearable device, or through a sensor embedded in an implant or a chip embedded in the patient. Note that these are just a few examples.

[0076] The multiple surgical planning system examples described in this specification may implement the automated defect quantification system described above. Based on the classification and description of the defect, the surgical planner provides additional valuable information to the surgeon to assist in the planning and implementation of the surgery.

[0077] [Acquisition of Patient Data] Patient data can be loaded from a file, storage medium or database, or manually entered into the system. If the patient has previously undergone surgery, the patient's past file can be retrieved from the database. Otherwise, a new case file or record is generated.

[0078] For many applications, medical images will be a valuable part of the patient data.

[0079] Data Processing: Patient data can be processed. For example, medical images can be converted into one or more virtual 3D models of anatomical structures such as bone anatomical structures, cartilage, organs, organ walls, blood pool volumes, and others. Anatomical landmarks can be determined or indicated in the medical image or virtual 3D model. This can be done manually or automatically, for example, by a feature-recognition technique. Further information such as bone density information, bone loss, impingement of bone-to-bone contact, spread / range of the defect on the surrounding anatomical structure, characteristics of adjacent soft tissues such as muscles, ligaments, cartilage, tendons, meniscus, and thickness of the soft tissue can be obtained from the medical image. In addition, a biomechanical model can be generated to validate the musculoskeletal data such as the anatomical structure of the bone in addition to the further simulated soft tissue data.

[0080] In some embodiments, the defect or deformation is quantified and / or classified as described above.

[0081] [Creation of Default Treatment Plan] In some embodiments, the plurality of surgical planning systems described herein can relate to a particular surgery and / or to implants of a particular type, brand, or product line. Additionally, unlike conventional systems, the plurality of systems described herein can assist in more significant, higher-level treatment decisions, such as the type of treatment, including invasive treatment, non-invasive treatment, or treatment involving a referral. Since many pathologies can be treated by various types of implants, such as off-the-shelf, customized, or custom implants, or combinations thereof, further treatment decisions can include the type of implant. For example, in the case of joints: cartilage repair, resurfacing, or replacement; partial or total (e.g., unicondylar / total distal femur implant, unicompartmental / total proximal tibia implant); fixation strategy (cemented / non-cemented, stemmed / stemless, press-fit, screw); functional strategy (e.g., posterior-stabilized / cruciate-retaining femoral implant, anatomical / reversed shoulder implant); acceptable range of motion; and others, can be considered. In the case of cardiac applications: valve repair, stapling, replacement, ring annuloplasty, type of stent, and other aspects can be considered.For cranio-maxillofacial applications: orthognathic, reconstructive, trauma, TMJ, alveolar type of surgical procedures, treatment of the maxilla or mandible or both, orbital floor, or part of the cranium, and other related aspects may be considered.

[0082] For pulmonary applications: endoluminal stents and extraluminal stents, valve types and other aspects may be considered.

[0083] For types of instruments or guidance: conventional instruments (devices), patient-specific guides, navigation systems, AR systems, robotics systems and others may be considered.

[0084] To assist in these decisions, the surgeon may be presented with information or models obtained from medical images as described above, and / or additional relevant information for a more detailed understanding of the defect, such as the results of quantification and classification of the defect or deformation. For example, the surgeon may be presented with the results of quantification and classification, and / or a visual representation of the defect or deformation, by overlaying a virtual 3D model of the actual patient's anatomical structure with a model representing a healthy anatomical structure, such as from the fitting of an SSM to a part of the patient's anatomical structure. One or more models may be presented in a translucent manner as described above. Also, biomechanical model simulations may be presented together with the virtual 3D SSM model.

[0085] As further support for these decision - making processes, the system may perform one or more population analyses based on historical data collected in a database through one or more feedback loops. Such analyses may associate one or more patient characteristics with one or more of the plurality of treatment decisions. Thus, the system may utilize (1) the selection of a population, (2) the selection of the treatment decision to be supported, and (3) the selection of one or more patient characteristics that characterize the members of the population and the patient to be treated. These selections may be left to the user, for example, via dropdown boxes or checkboxes within a user interface. Alternatively, the system may present the user with one or more pre - programmed combinations of selections, for example, in a wizard - like process. Correlation analysis may reveal which patient characteristics may be related to which treatment decisions. Alternatively, the system may first track the user's actions and then present the most common combination by default. For example, an AI - based system may learn about frequently selected decision influencers and display them to the surgeon at appropriate times during future preoperative planning. Alternatively, an AI - based system may learn the correlation between certain characteristics, particularly "optimal characteristics", and their influence on treatment decisions and use them to optimize and provide treatment options based on the "optimal" characteristics or based on the surgeon's selection of "optimal characteristics".

[0086] Regarding the selection of a population, the historical data analysis may be based on all records in the database or a subset of the records. For example, the population may be limited to fully - complete records, i.e., records that contain the appropriate data required for the analysis. The population may also be limited to patients having one or more characteristics in common with the patient to be treated, such as gender, age, ethnicity, and others. The population may also be limited to patients treated in the same country, in the same hospital, or by the same clinician, physician, surgeon, school, or others, etc.

[0087] Historical data analysis can reveal how a selected population is distributed across various options for a selected treatment decision. Members of the population are characterized by the selected patient characteristic(s). The patient to be treated can be located through the specific patient characteristic(s) of that patient within the analyzed population. As a result, it can be revealed which decision-making option seems to be most appropriate for this specific patient according to the historical data in the database. Alternatively, in this case, the system can show a comparative analysis if it is based on the (multiple) "best" characteristics selected by the system and is different from the (multiple) patient characteristics that were selected, thereby potentially enabling the user to re-evaluate their decision.

[0088] In some embodiments, historical data analysis can associate one or more treatment decisions with intraoperative or postoperative events, observations, or expected occurrences of results. Historical data analysis can, for example, reveal how much the probability or risk of a certain event, observation, or result occurring increases or decreases along with certain preoperative planning parameters.

[0089] For example, historical data analysis can associate a selected size of a heart valve with the risk of leakage, or a selected amount of lateralization of a shoulder implant with the risk of an acromion fracture.

[0090] In certain embodiments, the historical data analysis may utilize retrospective data including data obtained from high-level surgeons or key opinion leaders (KOLs), and provide it to novice or low-level surgeons to guide their decisions such as on bone defect data, mimic treatment options, or provide their used or preferred treatment plans to low-level surgeons. In some embodiments, the retrospective data may include information provided and used by a school (e.g., surgeons using the same plan or treatment options or other aspects).

[0091] This type of analysis can be made more accurate or appropriate for the patient to be treated, for example, by limiting the population to patients showing similarity to the patient to be treated with respect to one or more patient characteristics. This type of historical data analysis may require (1) selection of zero or more patient characteristics for limiting the population, (2) selection of one or more types of events, observations or outcomes, and (3) selection of one or more treatment decisions. As before, these selections can be entrusted to the user, for example, through dropdown boxes or checkboxes within the user interface. Alternatively, the system can present the user with one or more pre-programmed combinations of selections, for example, in a wizard-style process. Correlation analyses can reveal which events, observations or outcomes may be related to which treatment decisions. Alternatively, the system can first track the user's actions and then present the most common combination by default. For example, an AI-based system can learn about frequently selected decision influencers and display them to the surgeon at appropriate times during future preoperative planning.

[0092] Regarding the selection criteria for the population, the population should preferably be limited to a plurality of members showing similarity to the patient to be treated. This similarity can relate to one or more patient characteristics.

[0093] For example, in the case of heart-valve leakage, those patient characteristics can be a set of measurements that describe the shape of the anatomical structure surrounding the valve, such as the minimum and maximum diameters of the annulus.

[0094] As another example, in the case of an acromion fracture, the patient characteristics can include information regarding bone density obtained from a CT scan or results from the defect quantification and classification described above.

[0095] For these analyses that are known or suspected to depend on the shape of the patient's anatomical structure, the patient characteristics can include the parameters of the SSM or a subset of the parameters that are adapted to a part of the patient's anatomical structure. These parameters or such a subset form an n-dimensional vector that describes the patient's shape within an n-dimensional space that encompasses all possible shape variations. Thus, the population for historical data analysis can be limited to all members for which the corresponding n-dimensional vector is within a predefined distance from the patient to be treated.

[0096] The results of the historical data analysis can be presented in various ways. Some examples of this are described below. Multiple example embodiments of population analysis are also described below.

[0097] As an alternative to historical data analysis, the system can search for the member within the selected population that most closely matches the patient characteristics of the patient to be treated and display the selected decision-making options for that member.

[0098] Once a high-level treatment decision has been made, either using the decision support process as described above or without using the decision support process as described above, the system described herein may create a default preoperative plan for the patient being treated. This plan typically relies on one or more algorithms or heuristics that calculate treatment parameters, such as implant position and orientation, based on patient data and processed patient data.

[0099] For example, SurgiCase Knee Planner uses geometric algorithms based on anatomical landmarks identified on virtual 3D models of the patient's femur and tibia to calculate a local anatomical coordinate system, as well as the default size, position, and orientation of femoral and tibial implants relative to the patient's anatomical structure. For certain input parameters of such algorithms, population-wide values may be used. Alternatively, the values may be selected manually or automatically based on assistance from the decision support process as described above.

[0100] For example, in the case of total knee arthroplasty, a default value for varus correction for 3° of varus may be used for all patients, or historical data analysis may suggest a certain value for varus correction, or the value for varus correction for the member of the population that most closely matches may be used. Thus, the decision support process of the present invention can be used for both high-level treatment decisions and low-level treatment-specific decisions.

[0101] Historical data collected through one or more feedback loops can also be used to improve an automatically created default plan or to create a new default plan. For example, AI-based techniques such as machine learning, deep learning, neural networks, and the like can be used to incorporate changes that are often or consistently made during the planning process or during treatment into the default plan. In addition, information about intraoperative or postoperative complications can be used to include certain changes and ignore other changes.

[0102] [Changes to the treatment plan] Once the default plan is created, it is presented to the user for further fine-tuning. The user may be presented with the possibility of changing one or more treatment plan parameters. For example, the user may have the possibility of changing the size of the implant, the position of the implant, or the orientation of the implant.

[0103] During the planning process, the system can assist the user's decision-making through the decision-making support process described above.

[0104] The result of the planning process is an approved preoperative plan, that is, the treatment plan that the clinician has decided to implement.

[0105] In some embodiments, the system includes a feedback loop that stores all approved pre-operative plans in a database. The information collected in this way can be used as historical data provided to the decision-making support process. For example, performing a population analysis on a user's approved pre-operative plan can inform the user whether any changes or parameter values are within the scope of the user's past practice or experience. In contrast, performing a population analysis on all users' approved pre-operative plans can enable a user to learn from the accumulated experience of a much larger group of people, or to compare the user's individual practice to the average practice of all users. Other options are also possible, such as limiting the historical data to the approved pre-operative plans of all users in the same hospital, or of all users in the same country.

[0106] [Treatment of patients according to the treatment plan] Once an approved pre-operative plan has been created, the clinician can proceed with its implementation, i.e., the treatment of the patient. In some - mainly non-invasive - treatments, the pre-approved treatment plan can take the form of a prescription, such as for medication or exercise. In other - mainly invasive - treatments, the pre-approved treatment plan can take the form of a data file that can be used within a surgical guidance system. For example, the plan can be used to design and manufacture patient-specific instruments that assist the surgeon in achieving the planned surgical outcome during the operation. Alternatively, the plan can be loaded into a surgical navigation system or an AR system to display guidance information to the surgeon during the operation. Alternatively, the plan can be loaded into a robotics system to automatically or semi-automatically perform part of the operation.

[0107] The system may include a feedback loop that stores intraoperative data in a database. This may include any of the aforementioned intraoperative data. The data may be automatically collected by sensors in the operating room, by special surgical instruments, or by a surgical guidance system such as a navigation system, an AR system, or a robotics system, or may be manually entered through an electronic access device.

[0108] For example, the system may prompt the surgeon to store any intraoperative changes or complexities encountered during the surgery. This information may relate to implants, the surrounding patient anatomy, the actual implants and surgical instruments used, comprehensive data that is important but not measurable, etc. The system may also act as a notebook for the surgeon to record any relevant information regarding the patient's anatomy that may be useful at a later stage. This data is stored in the database for two purposes: (1) to complete the patient case file and (2) to optimize future preoperative planning.

[0109] The information thus collected can be used as historical data provided to the above-described decision-making support process. For example, capturing intraoperative measurement results and observations makes it possible to present statistical information to the user in an earlier step than approving the preoperative plan, regarding patient characteristics that cannot be derived from available medical images or can only be measured in invasive ways such as ligament tension, the occurrence of infection or damage to soft tissue. As another example, capturing information regarding intraoperative complications makes it possible to present statistical information to the user in an earlier step than approving the preoperative plan, regarding the likelihood of such complications. Finally, capturing any changes added to the surgical plan or any deviation from the approved preoperative plan makes it possible to replace or expand the decision-making support process described under "Planning Process" from presenting information regarding the choices made during the planning process to presenting information regarding the choices made during the surgery.

[0110] [Collection of Post-treatment Data] After treatment, more information such as postoperative medical images, virtual 3D models based on such images, postoperative measurement results, functional measurement results, pain scores, function scores, function scores, patient satisfaction information, information on postoperative complications, activity data, information on re-replacement surgery, etc. can be collected and captured through the feedback loop. The data can be automatically collected, for example, by a plurality of sensors or a plurality of wearable devices embedded in one or more implants, or can be manually entered into an electronic access device.

[0111] The information collected in this way can be used as historical data provided to the above-described decision-making support process. For example, it enables presenting statistical information to the user in a step prior to approving the preoperative plan regarding the actual surgical results, potential complication risks, implant lifespan, or patient satisfaction.

[0112] [Exclusion of Ineffective Treatment Plans] A special form of the intraoperative or postoperative feedback loop collects intraoperative and postoperative information regarding complications and uses it to classify, tag, or flag less effective preoperative plans based on, for example, how much the implementation of the surgery deviates from the preoperative plan (which can be user-defined), the severity of the complications, or the lifespan of the implant. This feedback loop enables further optimization by excluding the least effective treatment plans from the training data of AI-based techniques that generate such default plans for automatically created default plans. This feedback loop also enables improving the decision-making support system by excluding the least effective treatment plans from the data used in historical data analysis.

[0113] A very basic form of the exclusion feedback loop enables the user to manually flag preoperative or treatment plans that should not be included in any training data or historical data analysis.

[0114] [Presentation of the Results of Historical Data Analysis] Information generated as part of the decision-making support process can be presented to the user in any practical way. For example, when assisting in a decision-making regarding a limited number of discrete options or discrete parameter values, such as a choice from multiple treatment options or available implant sizes, a distribution graph or histogram can be shown for each of these options, along with one patient characteristic as the independent variable. The value of the patient characteristic for the specific patient being treated can be displayed on the graph by a mark on the independent axis, such that the user can be shown which decision-making option seems most appropriate for that patient based on the historical data.

[0115] For example, FIG. 9 shows an example of the results of historical data analysis represented in the form of distribution plots 902 to 906. The position of the patient being treated within the patient population is indicated by the vertical line 908. From this, the user can deduce that treatment B seems to be the most appropriate.

[0116] This represents a significant improvement over conventional systems that simply present the user with proposals for discrete treatment options or discrete parameter values. For example, in FIG. 9, the results of the historical data analysis are presented to the user, preferably in an intuitive manner. Specifically, in FIG. 9, not only is the proposal "Treatment B" obtained. The user also learns where the patient is located within the patient population and whether there are abrupt or smooth transitions between the various options. For example, the user can derive from the graph that Treatment B seems to be the most appropriate, but also that Treatment A could be a promising competitor and that Treatment C is not. If the surgeon has other medical or non-medical reasons to prefer Treatment A over Treatment B, such as treatment cost or their own lack of experience with Treatment B, the system of the present invention will not only simply propose Treatment B, but will also teach the user that Treatment A is a viable option and then provide the user with all relevant information about Treatment A.

[0117] Alternative representations are possible. For example, the data of the above graph can also be presented as a surface chart or a bar graph. Alternatively, it can be shown as a gradient (e.g., color gradient) plot. In that case, each of the decision-making options is represented by a specific color, pattern or intensity (e.g., grayscale), and the distribution of the population on the decision-making options is represented by mixing proportional amounts of the respective colors, patterns or intensities.

[0118] For example, FIG. 10 shows an example of the results of the historical data analysis presented in the form of a color plot. The position of the patient to be treated within the patient population is indicated by white dots. From this, the user can derive that Treatment B seems to be the most appropriate.

[0119] From the plots in FIG. 10, the user can derive information similar to the above-described distribution graphs. Specifically, the user can determine the positions of patients within the population, how the population is distributed over various treatment options or parameter values, and, by looking at the color gradient, whether there are smooth or abrupt transitions between those multiple options and between those multiple values. Deriving numerical values from a color plot can be more difficult. However, a color plot can be more intuitive to interpret.

[0120] In other embodiments, the analysis associating discrete options with two patient characteristics can be presented by other visual means, such as a 3D bar graph or other 2D plots (e.g., using color, pattern, intensity, or other visual references).

[0121] As another example, the results of a historical data analysis to assist in the selection of a continuous value parameter (e.g., varus correction for a knee implant, lateral movement of a shoulder implant, implantation depth of a heart valve, or patient satisfaction score, etc.) can be presented by a line graph.

[0122] For example, FIG. 11 shows an example of the results of a historical data analysis represented in the form of a line plot 1002. The position of the patient to be treated within the patient population is indicated by the vertical line 1004. From this, the user can derive that a value between 0.1 and 0.2 for parameter A seems to be the most appropriate.

[0123] An example such as FIG. 11 represents an improvement over a conventional system that simply presents a user with a proposal for a continuous parameter value. For example, based on FIG. 11, the user does not merely obtain the proposal "0.15". Rather, the user also learns where in the patient population the patient is located and whether, within the patient's overall position, the parameter is highly variable with respect to patient characteristics. For example, the user can derive from the graph that the value of parameter A of 0.15 seems to be the most appropriate, but also that there is little variation in the value of parameter A among patients similar to the patient being treated. To provide even more information, the line graph can show confidence intervals, for example, by vertical bars (so-called "whiskers") or by a shaded area around the value curve.

[0124] As another example, the results of a historical data analysis that links the probabilities or risks of intraoperative or postoperative events, observations or outcomes to treatment decisions or parameters can be presented in a graph, surface chart or bar graph, or in a color plot or pattern plot (optionally together with confidence intervals). Similar to the display of the position of patients within a population in the above example, the current selection regarding a decision-making option or parameter value can be displayed. In some embodiments, the graph, chart or color plot can be presented together with a depiction of the patient's anatomical structure, and / or any device, instrument or implant that forms part of the planned treatment, such as a 2D or 3D image, line drawing, medical image or virtual model. The graph, chart, color plot and depiction can all be interactive, such that a change made in one is automatically reflected in the others.

[0125] For example, FIG. 12 shows an example of the result of historical data analysis presented in a bar graph format. Here, the risks of two complications are related to the selected device size. The currently selected device size is indicated by circle 1202, but other means, such as the opacity, saturation, color, or pattern of the bars in the chart, etc., are also possible. From this, the user can deduce that device sizes 3 and 4 seem to be the most appropriate.

[0126] FIG. 13 shows yet another example of the result of historical data analysis presented in a color plot format. Here, the risks of two complications are related to the selected parameter value. The currently selected parameter value is indicated by circle 1302. From this, the user can deduce that the selected value is within the safe zone.

[0127] The various methods of displaying decision-making support data within the multiple illustrations described herein represent a significant improvement over conventional systems that merely present proposals to the user regarding decision-making options or parameter values. For example, from the representations shown in FIGS. 12 and 13, the user not only obtains the proposal "device size 3" or the proposal "parameter X = x". Rather, the user also becomes aware of what implications deviate from the proposal, how great the odds of complication results or risks are, how steeply the odds or risks increase or decrease when changing a decision-making option or parameter value, and thus how much leeway the user has in changing the decision-making option or parameter value. For example, the user can deduce from the plot that the current value for parameter X is within the safe zone, but furthermore, that it may be safe to increase that value slightly, but it seems undesirable to decrease it. One or more such historical data analysis representations can be presented to the user in any given instance.

[0128] [Application Example: Shoulder Treatment Decision Support] For shoulder-related complaints, there are various treatments depending on the condition (pathology). For example, shoulder arthritis can be treated by rest, medication, corticosteroid injection, arthroscopic debridement, hemiarthroplasty, resection arthroplasty, total (anatomical) shoulder replacement (TSA), reverse shoulder replacement (RSA), and others. Depending on the condition (pathology) or the complexity of the treatment, some physicians may choose to consult colleagues or other hospitals about the patient, or follow the treatment options of one of the known peers.

[0129] The multiple systems and methods of the present invention can assist physicians when determining treatment based on patient characteristics and historical data.

[0130] For example, based on medical images of the patient's bone and / or cartilage anatomical structure, such as CT or MRI images, a virtual 3D model of the patient's shoulder anatomical structure can be created. The defect can be quantified by the methods described above. The results of the quantification can be presented to the user, for example, by a depiction as shown in FIG. 14.

[0131] In particular, FIG. 14 shows an example of the representation of the results of glenoid defect quantification. On the left side, there is a virtual 3D model 1402 of the anatomical structure of the patient's scapula bone, with the glenoid in the center. On the right side, the anatomical structure 1404 is shown overlaid (superimposed) on the SSM 1406 representing the healthy anatomical structure adapted to the part of the patient's scapula. Various results of defect quantification are shown. The erosion depth (the distance from the actual bone surface to the position where the surface would have been in a healthy state, here represented by the surface of the SSM instance) is shown in the form of a gradient plot 1408.

[0132] In the example of FIG. 14, the erosion depth is calculated perpendicular to the best-fit plane passing through the surface of the glenoid fossa of the SSM instance. Other measurement directions are possible, such as locally perpendicular to the surface of the glenoid fossa of the SSM instance.

[0133] Additional metrics are calculated and shown, such as the glenoid loss percentage (percentage of the volume of the glenoid lid lost due to bone erosion), the erosion area percentage (percentage of the surface area of the glenoid fossa affected by bone erosion), and the maximum erosion depth. In this example, the glenoid fossa is subdivided into four quadrants, and quantitative metrics such as the anterior, posterior, superior, or inferior glenoid loss percentage are shown within each quadrant. Additionally, the subluxation distance is calculated. For this purpose, the center of rotation of the humeral head is calculated by best-fitting a sphere to the articular surface of the humeral head; the center point of this sphere is projected vertically onto the best-fit plane passing through the surface of the glenoid fossa of the SSM instance; the distance between this projection point and the geometric center of the glenoid fossa is measured and displayed. Also, the subluxation region, i.e., the quadrant onto which the center of rotation of the humeral head is projected, is shown.

[0134] FIG. 14 demonstrates a significant improvement over conventional systems in that the user now has reproducible and objective information for evaluating the extent and location of bone defects from this information and from its depiction. This information is important for determining the most appropriate treatment.

[0135] As described above, the system can further assist in decision-making by presenting statistical information based on historical data. For example, a system that includes a feedback loop for an approved pre-operative plan can perform an analysis to associate any of the above metrics with a treatment selected in previous cases. The results of this historical data analysis can be presented to the user in any of the above ways (formats). For example, the results can be presented in a chart such as that in FIG. 15, for example.

[0136] Specifically, FIG. 15 shows an example of the representation of the results of the historical data analysis. FIG. 15 shows what percentage of patients have been treated in various ways, divided by the percentage of lid loss. The patients to be treated are indicated by the vertical line 1502.

[0137] All records in the database can be used as a basis for historical data analysis. Alternatively, the population selected as a basis for historical data analysis can be limited in a number of ways. For example, if the population is limited to only those cases that have been treated by the user, the user will gain insight into how the patient to be treated relates to their past experience. Including more or all user cases will provide insight into the practice of a larger surgeon community, such as all surgeons in a particular hospital, country or the world.

[0138] The population can also be limited to patients that show a certain similarity to the patient to be treated. Such similarity can be based on one or more patient characteristics such as gender, age, ethnicity, activity level and the like.

[0139] Consultation with colleagues or other hospitals can be one of several options. Based on information stored in the database, the system may have the function of proposing clinicians who are open to the consultation (clinicians who accept the consultation). Based on the historical data in the database, the system may propose a more experienced clinician regarding similar patients, i.e., patients showing similar pathologies and / or other patient characteristics, or may propose following the treatment plan of the surgeon being consulted.

[0140] Analysis of historical data based on approved preoperative plans, collected and stored through a feedback loop, has been described. However, similar and potentially more appropriate analyses can be performed on intraoperative or postoperative data collected through other feedback loops. Such data may represent the actual treatment performed rather than the treatment the surgeon intended to provide.

[0141] [Application Example: Shoulder Surgery Implant Type Decision Support] Similar to the previous example, the multiple systems described herein may provide support for the decision of which type of implant to use in shoulder arthroplasty. This may include, for example, off-the-shelf implants versus custom implants.

[0142] For example, the system may provide decision-making support in the form of historical data analysis that associates the selection from standard or off-the-shelf implants and custom implants with the quantification of bone defects described above. The results of the analysis may be presented to the user in the form of graphs, charts, color plots or pattern plots, or in the form of those like other examples described herein.

[0143] For example, FIG. 16 shows an example of the representation of the results of a historical data analysis associating the selection from a standard implant and a custom implant with the percentage of glenoid loss of a patient's glenoid fossa. The patient to be treated is indicated by the vertical line 1602.

[0144] FIG. 17 shows another example of the representation of the results of a historical data analysis associating the selection from a standard implant and a custom implant with the percentage of glenoid loss of a patient's glenoid fossa. The patient to be treated is indicated by the circle 1702.

[0145] As another example, the system may be provided with a library of implants, and the analysis may associate the selection of an implant with one or more defect characteristics calculated from the defect quantification.

[0146] [Application Example: Shoulder Surgery, RSA Lateral Movement] In reverse shoulder arthroplasty, lateral movement of the center of rotation is often employed as a method to improve the torque generated by the rotator cuff and increase internal and external rotation. However, excessive lateral movement can cause excessive muscle lengthening and may also cause acromion fractures due to increased loading (load). Insufficient lateral movement can cause joint instability due to decreased muscle loading.

[0147] Therefore, the multiple systems described herein may provide decision-making support through simulation of muscle elongation due to lateral movement.

[0148] For example, the system may provide a 2D or 3D depiction of the patient's anatomical structure and the implant. This depiction may include the anatomical structure of the scapula or humerus bone, the implant, and a virtual model of one or more shoulder muscles. The shoulder muscles may be shown in their actual shape or rather schematically, for example, by lines, curves, polylines (polygons) or cylindrical shapes. The depiction may simulate how the trajectory of the muscles changes with the lateral movement of the implant and may be displayed as a biomechanical model. As a reference, the depiction may display, in an overlay, the trajectory of the muscles and the bone model in the original (i.e., either preoperative or healthy) state. The preoperative state may be derived from medical images. The healthy state may be approximated by fitting an SSM representing the anatomical structure of a healthy shoulder to a portion of the patient's anatomical structure.

[0149] The system may be interactive. For example, as shown in FIG. 18, the system may enable the user to manually shift the center of rotation from a first position 1802 to a second position 1804 by operating the model of the implant, for example, by clicking and dragging an input device such as a computer mouse. Alternatively, the system may provide a user interface control, such as a button or a slider, for adjusting the lateral movement. The depiction is automatically updated to reflect the adjustments made. For example, the relative positions of the scapula, humerus, and implant components, as well as the trajectories of the corresponding muscles, are updated.

[0150] The system may display numerical values such as percentages, such as the quantification of the extension amount of individual muscles, or a decrease in the thickness of lines, curves, polylines (polygons) or cylindrical shapes, or an average value for some or all of the muscles. These values may be overlaid on the depiction of the anatomical structure or may be listed elsewhere within the user interface.

[0151] The multiple systems according to the present invention can provide additional decision-making support through the analysis of historical data of multiple past cases. As described above, data for such analysis can be collected through one or more feedback loops in the form of an approved preoperative plan or in the form of an actually implemented surgical plan collected during or after the surgery. The population can be based on all available records or can be restricted in various ways as described above. In a plurality of preferred embodiments, the population is limited to patients who exhibit a certain similarity to the patient to be treated in one or more patient characteristics. For example, bone density can be derived from a CT scan and can play an important role in assessing the risk of acromion fractures. Alternatively or additionally, shape characteristics such as the thickness of the acromion can play an important role. Those shape characteristics can be quantified by a plurality of certain measurement results or by a plurality of parameter values of the SSM adapted to the anatomical structure of the patient as described above. The results of the analysis can be presented in the form of a graph, chart or color plot showing how frequently those amounts of lateral movement have been previously planned or implemented for various amounts of lateral movement. The current lateral movement can be shown on the graph, chart or color plot by markers such as a line, dot, diamond or the like.

[0152] Alternatively, the analysis can investigate how frequently the amount of muscle extension has been previously planned or implemented. This could be the amount of muscle extension of an individual muscle and the average or weighted average of the selected shoulder muscles or all shoulder muscles.

[0153] Finally, in embodiments where the system collects and stores information regarding intraoperative or postoperative complications, the analysis can further include the risk of such complications such as acromion fractures or instability. And the user can see from the graph, chart or color plot whether the selected lateral movement is not only within the range of common practice but also within the range of the safe zone.

[0154] In addition to or as an alternative to the interactive features described above, graphs, charts or color plots may be interactive. For example, a user may select the amount of lateral movement by clicking on a graph, chart or color plot or by sliding a marker representing the current amount of lateral movement. Any depiction of the anatomical structure and the planned implant(s) may be automatically updated to reflect changes in the lateral movement.

[0155] The systems and methods described herein may be operative and executable, for example, by a computing device such as a desktop computer, a portable computer, a portable electronic device, a tablet computer, a smartphone and other computerized devices. In some implementations, the methods described herein may be executed by a native software application, while in other implementations, the methods described herein may be executed in server-client implementations. For example, in some implementations, software configured to execute the methods described herein may be hosted by a remote server or a cloud-based system. In some cases, various aspects of the systems and methods described herein may be distributed across various computing devices.

[0156] Furthermore, the systems and methods described herein may be operative and executable, for example, by a medical professional such as a surgeon, a physician or a nurse, or by a non-medical professional such as a clinical technician, a design engineer, an implant manufacturer (e.g., to give an implant manufacturer an overview of what type of implant a particular surgeon handles and to generate a plot depicting it), a residency student, or a patient (e.g., a person rehearsing a surgery prior to an actual surgery).

[0157] [Application Example: CMF Treatment Decision Support] The defect quantification system described in this specification can further detect one or more defects in the craniomaxillofacial (CMF) region and further classify it for use in trauma; orbital reconstruction; distraction osteogenesis; temporomandibular joint; cranial vault reconstruction; craniosynostosis, etc. congenital craniofacial deformities; alveolar surgery; or any other aesthetic (cosmetic) surgery or reconstructive surgery, etc. that includes one or more of a plurality of parts of the craniomaxillofacial region.

[0158] As an example of an embodiment, the defect quantification system described in this specification can use patient data (e.g., image data) and one or more feedback loops to detect the type of defect to be quantified and classify defects such as orthognathic defects.

[0159] In an example of the method, one or more medical images or scans (generally, image data) of the anatomical structure of a patient in need of correction can be obtained. For example, the image data can relate to the patient's jaw deformity. In this example, the image data can include, for example, image data of one or more of the patient's mandible, maxilla, or chin. As described above, the anatomical structures within the image data can be segmented (e.g., between the mandible, maxilla, and / or chin) to obtain a virtual three-dimensional surface model. And a statistical shape model of a healthy anatomical structure (e.g., a healthy jaw) can be fitted to the three-dimensional surface model to identify the healthy and damaged parts of the patient's anatomical structure (e.g., the damaged part of the patient's jaw).

[0160] FIG. 19 shows an example of the representation of defect quantification using patient image data and SSM. In this example, the image data includes a three-dimensional model of the mandibular anatomical structure 1902 of the patient's bone overlaid on the SSM 1904 of the original healthy mandible.

[0161] In this example, it is clear that this patient only requires treatment of the mandible and not the maxilla.

[0162] The manner of comparing the patient's actual anatomical structure (e.g., by a three-dimensional model created from medical image data) with a healthy anatomical structure model (e.g., an SSM model) enables the surgeon to visualize possible surgical approaches. For example, in this case, while providing a healthy anatomical structure as a reference, the surgeon can manipulate the positioning of the mandible. In this example, the defect shown in FIG. 19 and the proposed surgical treatment (e.g., mandibular reconstruction) can be identified.

[0163] During the planning stage, the system guides the surgeon by showing, as shown in FIG. 20, the part 2002 of the anatomical structure that can be resected. In particular, the system shows clear resection margins and can warn the surgeon if, based on the quantified defect, the surgeon decides to resect more bone than necessary for the resection or less bone than necessary for the resection.

[0164] Furthermore, as described above, the three-dimensional patient anatomical structure model may be accompanied by historical data related to the patient and may propose a patient-specific implant for the planned treatment. For example, the treatment plan may include the use of a graft bone, and based on the patient's medical history, the patient's left fibula may be selected as the graft. The system can further indicate healthy parts on the fibula and show the postoperative results. However, these are just a few examples.

[0165] In another example, the proposed treatment plan may include treatment of additional CMF regions, including maxillary, mandibular, and genioplasty.

[0166] FIG. 21A shows an example where the defect is classified as LeFort I. This is a type of skull fracture that involves the maxilla and the surrounding structures in either the horizontal, pyramidal, or transverse direction. For such a classification, the treatment plan may include bilateral sagittal split osteotomy (BSSO) and genioplasty osteotomy. As described above, the model of the patient's anatomical structure is divided into various regions 2102 - 2108, which can be used for defect quantification and considered during preoperative planning.

[0167] FIG. 21B shows multiple aspects of the treatment of the defect quantified in FIG. 21A. In particular, FIG. 21B shows the recommended distance of maxillary movement to treat the defect. In some cases, the recommended distance may be based on historical data, and the system may further show the range 2110A and range 2110B (e.g., in mm) of possible maxillary movement.

[0168] FIG. 21C shows an example of proximal overlap and resection margin 2112. In particular, FIG. 21C identifies that at 2114, restoration of the anatomical structure of the bone is required.

[0169] FIG. 21D shows another example of the proposed treatment of the defect. In this example, the system displays a warning 2116 that the gap needs to be filled.

[0170] In some embodiments, based on the quantified defects and the initial treatment plan, the system may further propose an appropriate type and size of implant for connecting various bone portions, such as to fill the identified gap in FIG. 21D. For example, the system may propose the use of a guide for the placement of a mandibular implant. The system may further enable the surgeon to visualize various implant options before making a selection and updating the treatment plan accordingly.

[0171] [Application Example: Decision Support for Orthognathic Surgery] Another application example of the plurality of surgical planning systems described herein is decision support for genioplasty. In one example, preoperative planning tools (e.g., Proplan CMF by MATERIALISE (registered trademark), and others) may be used to generate a preoperative surgical plan for a particular cranio-maxillofacial surgery. And the image data from the preoperative planning tool may be used by a defect quantification system as described herein.

[0172] In one example, the defect can be classified as a jaw deformity that requires a genioplasty to treat the defect. In this example, the defect quantification system can quantify the defect based on various existing osteotomy classifications familiar to surgeons, such as, for example, Limberg's oblique subcondylar osteotomy, Moose's procedures for mandibular reduction, Caldwell and Letterman's vertical ramus osteotomy, Trauner and Obwesefer's sagittal split osteotomy (SSO), bilateral sagittal split osteotomy (BSSO), Winstanley's intraoral vertical ramus osteotomy (IVRO), and others. The defect quantification system can further enable the user to visualize various fractures of the skull, such as LeFort I, LeFort II, modified LeFort I, and others, when the defect is in the maxilla.

[0173] Alternatively or additionally, the defect quantification system can classify the defect based on the type of incision or surgery. For example, based on the defect and the generated SSM model, the user can select to perform bimaxillary (maxilla + mandible), multi-segment maxilla, maxilla only, mandible only, or symphysis formation.

[0174] Once the defect is quantified, a default treatment plan can be created as described above. In some cases, three-dimensional cephalometry data (measuring deviation from a reference), asymmetry assessment, records of past surgeries, and other types of patient data stored with the patient profile can be considered.

[0175] In one example, where the defect is within the mandible, a bilateral sagittal split osteotomy (BSSO) may be proposed by the surgical planning system. In some cases, this treatment may be performed without any treatment of the upper (maxilla) jaw. The surgical planning system may enable a user (e.g., a surgeon or other healthcare provider) to visualize a surgical approach to the mandible along with appropriate modifications to the maxilla, allowing the user to determine the best approach.

[0176] In some embodiments, the system may further assist the surgeon in selecting the appropriate type of BSSO, such as Dalpont, Obwegeser, short ramus osteotomy, inverted L, and vertical ramus, etc. Depending on the defect and the type of osteotomy, the system may provide warnings such as proximity or damage to surrounding nerves and propose an appropriate osteotomy. The surgical planning system may further warn the user when the amount of bone resected in the planned treatment is too much or too little. The surgical planning system may further assist the user with appropriate resection margins (prompt) and issue a warning when the margin is exceeded compared to the historical data of the selected patient population (e.g., the population in which the patient for whom the surgery is planned is a part).

[0177] Based on the type of osteotomy, the surgical planning system may further assist the user in determining a suitable fixation method, such as a patient-specific (patient-specific) method or a standard method, etc., and the area where the fixation method is to be placed. Some of the options available to the user may include the selection of one or more plates, the type of plate (patient-specific plate or standard plate), the use of guides and / or the use of lag screws, etc.

[0178] In some embodiments, when the treatment plan includes treatment of the maxilla, the user may select from two or more plates based on the patient history, and may compare the number and type of plates selected for similar patients using historical data analysis and / or patient population plotting.

[0179] In some embodiments, when the treatment plan includes treatment of the mandible, the surgical planning system may enable the user to visualize the positioning and orientation of plates or lag screws, upper or lower fixation areas, etc. The surgical planning system may further enable selection of plate thickness and width, fixation materials (e.g., CPTi, TAIV, bioresorbable) based on the amount of bone available, the number and position of fixation screws on each side of the osteotomy, use of guides in combination with patient-specific or standard plates, and other options. All of the foregoing selections and configurations may be part of the treatment plan generated by the surgical planning system.

[0180] In one exemplary embodiment, the defect quantification system may classify the patient as having a class 2 narrow maxilla defect that requires treatment of mandibular advancement and maxillary impaction. The default treatment plan may include treatment of the maxilla, such as BSSO for the mandible and multi-segment Lefort I osteotomy. The default treatment plan may further recommend using a patient-specific plate for the maxilla and three lag screws on each side for the mandible. The user of the surgical planning system (e.g., the surgeon) may approve the default treatment plan or explore modifications to the plan through the surgical planning system's ability to visualize the treatment plan.

[0181] Next, the user can approve the treatment plan and use it during the surgery (e.g., in the operating room). In the operating room, changes or deviations from the treatment plan, such as the time required to perform the surgical steps, anastomosis, ischemic time for graft removal, verification of necessary surgical instruments before the start of the surgery, blood loss, timed checks on diseased tissue to determine the exact resection margin, and others, can be input into the surgical planning system.

[0182] After the surgery is completed, the patient's profile can be updated, and certain data regarding the treatment can be generated not only for historical data analysis as described above but also for future preoperative surgical planning. In particular, other postoperative data such as infection rate, stability and recurrence rate, pain score, hospital discharge and related notes, mouth openings scans and records, recurrence and recurrence rate for tumor cases, flap survival rate for reconstructive surgery, functional outcomes, and aesthetic outcomes can also be included in the patient profile.

[0183] [Application Example: Decision Support for Reconstructive Surgery] Another application example of the multiple surgical planning systems described herein is decision support for reconstructive surgery. In one example, a preoperative planning tool (e.g., Proplan CMF by MATERIALISE (registered trademark), etc.) can be used to generate a preoperative surgical plan for a specific craniofacial surgery. Then, the image data from the preoperative planning tool can be used by a defect quantification system such as those described herein.

[0184] In one example, the defect can be classified as a deformation that extends to the mandible or the midface that requires reconstructive surgery. Based on patient profile data, such as the patient history and patient image data, a three-dimensional SSM model of the patient can be generated. Then, the image data (indicating the defect) and the SSM can be compared to generate a defect classification. Based on the defect classification, a default treatment plan can be generated by a surgical planning system such as those described herein.

[0185] In the case of cancer patients, the defect quantification system can quantify the defect based on the type of cancer and / or lesion (benign or malignant), the area of the lesion to be resected (excised) and treated during surgery, the number of surgeries required, and other factors. Any other patient information, such as other treatments such as chemotherapy, radiation therapy, is also included within the patient profile.

[0186] In the case of corrective surgery, the patient history can be taken into account during treatment planning. For example, based on patient image data, a user (e.g., a surgeon) can evaluate the asymmetry and its deviation from the normal original anatomical structure. Using a three-dimensional model based on the patient data, the defect is simulated compared to the healthy anatomical structure.

[0187] In the case of trauma, the visualization function of the surgical planning system can be used together with patient population and history data analysis to efficiently create an appropriate treatment plan. In some embodiments, the surgical planning system can recommend a default plan based on the characteristics identified in trauma patients.

[0188] Furthermore, the historical data analysis performed by the surgical planning system may enable the user to compare the success rates of different surgical approaches for specific indications, such as vascularized graft versus bone non-vascularised graft, autologous versus bone substitute, and others.

[0189] In some embodiments, the system may also store the relevant information required to match a donor to a recipient, and the surgical planning system may further provide information about other users (e.g., other surgeons) or other facilities (e.g., other hospitals) that come into contact with potential donors. When tissue is removed (harvested), the surgical planning system may display information regarding the morbidity of the donor site, e.g., in the case of a harvested bone graft. In the case of trauma involving larger bone defects, the surgical planning system may prompt the user to use larger and stronger plates, and in some cases even patient-specific plates. However, these are just some examples, and others are possible.

[0190] [Application Example: Heart Treatment] Another application example of the multiple surgical planning systems described herein is heart treatment. In one example, preoperative planning tools (e.g., MIMICS and MIMICS Enlight by MATERIALISE®) may be used to generate preoperative surgical plans for structural heart and other vascular interventions. And the image data from the preoperative planning tool may be used by a defect quantification system such as those described herein.

[0191] For example, patient data including images, scans, and patient history, etc. are stored (memorized) by a surgical planning system. As described above, the image data can be converted into a three-dimensional model of the patient's anatomical structure. And the SSM model can be used by a defect quantification system to classify heart defects based on congenital or acquired diseases. In some examples, the defect can be classified into septal defect, valvular heart disease such as aorta or mitral valve, vascular obstruction, fistula, and other conditions. Each category can be further divided into a plurality of classes based on severity. Once the defect is quantified, a default treatment plan such as those described above can be generated.

[0192] In one example, a patient can be identified as having a defect in the aortic valve that requires a transcatheter aortic valve replacement (TAVR) procedure. Multiple factors, such as aortic valve morphology, evaluation of the aortic root, evaluation of the annulus (size and height), LVOT calcification, height of the sinotubular junction, evaluation of the coronary ostium (height), evaluation of the sinus of vulsava (diameter and height), evaluation of the risk of coronary artery occlusion, prediction of the optimal fluoroscopic projection angles for device deployment, evaluation of the transfemoral access route for the TAVR device, evaluation of alternative routes if the transfemoral route is not feasible, evaluation of the feasibility of carotid protection devices, and others, can be determined from the SSM model and the three-dimensional anatomical structure model as part of a defect classification system, which helps the user (e.g., a surgeon) generate a treatment plan. These factors can affect the determination of the treatment plan, such as catheter planning, device selection, access planning if the conventional transfemoral route is not accessible, incision size, device type, and others.

[0193] In one example, a patient may be identified as having a defect in the mitral valve that requires a transcatheter mitral valve replacement (TMVR) procedure. Evaluation of the landing zone, including evaluation of the size of the mitral annulus (diameter, height, APML, leaflet), calcification, risk of left ventricular outflow tract (LVOT) obstruction, risk of interaction with other intercardiac devices (newly or recently implanted, or to be implanted), distance from such devices, determination of the optimal trans-septal puncture position or transapical route, evaluation of the optimal fluoroscopy angle, height of the papillary muscle, volume and size of the left ventricle, evaluation of the delivery device and route, angulation of the mitral valve, access position, extension of the trans-septal crossing (e.g., fossa ovalis), and others, can be determined from the SSM model and three-dimensional anatomical structure model as part of the defect classification system, which helps the surgeon generate a treatment plan. These factors can affect, for example, the type and size of the surgical device, incision size, and entry points. For example, a user (e.g., a surgeon) can determine the entry point such that the apex / apical puncture is perpendicular to the mitral annulus with respect to device placement. In addition to historical data, using a defect quantification system, the surgeon may be able to predict the outcome of a neoLVOT procedure by using one or more visualization methods to place the patient within the selected patient population.

[0194] In one example, a patient can be identified as having a defect in the left atrial appendage (LAA) that requires closure of the LAA. Multiple factors, such as the evaluation of the landing zone for device placement, determination of the optimal transseptal puncture position, determination and evaluation of the optimal fluoroscopic projection angle for device delivery, selection and planning of the delivery device, selection of the catheter, and angulation to the LAA, can be determined from the SSM model and the three-dimensional anatomical structure model as part of the defect classification system, which helps the surgeon generate a treatment plan. Based on the diameter, height, depth, and shape of the LAA, an appropriate device and its size can be selected for the treatment plan.

[0195] Using historical data and patient populations, the surgical planning system can prompt the user, in some cases, regarding the type and size of the device, catheter selection, and delivery route. Based on the severity of the disease, patient age, associated health risks, and device availability and feasibility, the surgical planning system can prompt the user regarding alternative treatments. For example, open heart surgery may be considered last. Based on the historical data, the system can also store relevant information regarding catheter delivery and the route used, such as the catheter deformation percentage, and can warn the user to consider it if a more appropriate catheter is available.

[0196] Other structural heart interventions, such as paravalvular leak, atrial septal defect (ASD), patent foramen ovale (PFO), etc., can also be planned using the surgical planning system described herein.

[0197] Furthermore, the best viewing angle for fluoroscopy, or intraoperative measurements such as C-arm angles for accurately positioning the patient during surgery, can also be proposed, and appropriate warnings can be provided via one or more navigation systems during the planned treatment (e.g., surgery) and during preoperative planning.

[0198] For example, based on the image data of the patient's anatomical structure (e.g., CT image or MRI image), a virtual three-dimensional model of the patient's heart anatomical structure can be created. The defect can be quantified by the method described above.

[0199] In the example shown in FIG. 22, the patient is identified as having a defect in the mitral valve.

[0200] Structural heart interventions such as TMVR involve the placement of a mitral valve device 2202, as shown in FIG. 22. Based on a three-dimensional model of the patient's heart, such as that shown in FIG. 22, the user (e.g., surgeon) can determine the size, type, position, and location of the implant. Patient metrics such as angulation, available cross-sectional area corresponding to the flow path, and others can be considered. Furthermore, using one or more visualization tools of the surgical planning system, the risk of leakage can be determined while considering the type of implant. Additionally, the delivery method and access point can also affect the choice of implant.

[0201] In another example, the current delivery route for delivering an implant may need to be determined for a patient who requires an LAA procedure. In such cases, the selection of a catheter based on the patient's anatomical structure, along with its entry point and delivery trajectory, needs to be planned to ensure safe delivery of the implant to the patient during surgery.

[0202] Figure 23 depicts a target trajectory 2302 for implant delivery. The user of the surgical planning system can experiment with various catheters prior to creating the final treatment plan. Further, if the delivery path selected for the patient would lead to additional complications, the surgical planning system can warn the user to reconsider the delivery path.

[0203] [Application Example: Knee Joint Treatment Decision Support] Another application example of the plurality of surgical planning systems described herein is for joint deficiencies (e.g., ankle, hip joint) such as knee joint treatment decision support. In one example, a preoperative planning tool (e.g., SurgiCase Knee Planner by MATERIALISE®) can be used to generate a preoperative surgical plan for arthroplasty of joints such as the knee joint. Patient data including medical image data (e.g., MRI and CT scans), patient history, preoperative PROM score, new surgery or replacement information, patella height, axis, deformation type, and others can be utilized by the surgical planning system to generate a treatment plan.

[0204] For example, a defect quantification system can be used to classify the severity of the defect as requiring total knee arthroplasty or partial knee arthroplasty. As described above, the defect quantification system can assist in quantifying the defect by comparing a three-dimensional model of the patient's anatomical structure with the SSM model. Information regarding the varus / valgus angle, cartilage wear, and other soft tissue data, along with information such as the type (standard or patient-specific) and size of the implant, can be presented to the user so that the preoperative plan can be determined, either before or during the planning.

[0205] In some cases, the user may compare the generated default preoperative plan with the selected patient population as described above and may use historical data analysis. In particular, the surgical planning system may present to the user information about why a certain type of implant was proposed, the position and location of the implant, and the varus / valgus angles considered, and the system may enable the user to visualize how changes in the characteristics of the implant affect the patient's expected postoperative outcome.

[0206] For example, if the patient is young and active, the surgical planning system may retrieve data on treatment options for young patients and propose to the user to consider partial knee arthroplasty (PKA) instead of total knee arthroplasty (TKA). The surgical planning system may further propose to the user to use a guide with a patient-specific implant while showing the optimal treatment option with minimal cartilage wear and tear.

[0207] The surgical planning system may also enable the user to view the treatment plan on a biomechanical model that includes information on bone and cartilage in addition to soft tissue data such as ligaments and muscle attachments. Furthermore, the surgical planning system may also be configured to simulate the biomechanical model through rotation and translation and present data such as ligament elongation and knee joint loading so that minimal damage occurs to the soft tissues around the knee joint as a result of the treatment.

[0208] In some cases, the biomechanical model may be stored using one of the multiple feedback loops described above. The biomechanical model may be used as a reference during surgery (in real time) (in addition to the navigation system) to prompt and guide the user (e.g., the surgeon) with warnings if the actual treatment deviates from the treatment plan or if other complications arise.

[0209] In some embodiments, intraoperative measurement results, such as deviations from preoperative plans and soft tissue information, may be stored to complete the patient profile and create future preoperative plans and historical treatment data (treatment data history).

[0210] In some embodiments, intraoperative measurement results, including deviations from the plan, may be recorded by a surgical planning system. For example: the need for cementation (tibia / femur), patella, approach, alignment technique, femoral rotation, femoral valgus, patella release, medial and lateral release, the level of balance satisfaction achieved after surgery (e.g., not happy, happy, very happy), blood loss, surgery time, range of motion when closed, use of a robotic system or other navigation system, bone quality, diagnosis, PCL cut and size, limb alignment (varus / neutral / valgus), joint space opening before cut (medial / lateral), joint space opening after implant placement (medial / lateral), laxity score (e.g., high / good / low), flexion contracture, ligament release, patella resurfacing, use of tibial and / or femoral guides and guide conformity, tibial slope, proximal tibial cut, tibial implant, confirmation of whether the planned implant was used or other sizes and types of implants were used, tibial thickness and insert type, distal tibial cut, posterior femoral cut, AP-shift femur, anterior femoral cut, femoral implant rotation, ROM: maximum flexion, flexion balance, extension balance, and others.

[0211] Furthermore, postoperative data such as PROM scores currently used by surgeons, such as KSS, KOOS, OKS, EQ5D, FJS, etc., and other inputs provided by patients or their caregivers during follow-up can also be recorded by the surgical planning system.

[0212] In one example, medical images of the patient's bone and / or cartilage anatomical structure, such as CT images or MRI images, can be used to generate a three-dimensional model 2402 of the patient's knee joint. The defect can be quantified by the method described above.

[0213] For example, FIG. 24 shows a representation of cartilage thickness on the anatomical structure of the bones of the knee joint (tibia and femur). A certain specified area (e.g., 2404) is considered to be healthy, such as where a sufficient amount of cartilage is found. Other areas (e.g., 2406) indicate defects, such as areas of weaker cartilage. This information can be used by the user (e.g., the surgeon) when determining which treatment option to select for the treatment plan.

[0214] For example, based on the image in FIG. 24, the user may decide to treat the patient with unicompartmental knee arthroplasty rather than total knee arthroplasty so that the cartilage found within the healthy area can be preserved. Based on this decision, the surgical planning system can propose an implant, size, brand, and type for this patient from various implants.

[0215] Furthermore, the surgical planning system can be configured to enable the user to visualize the type and size of the implant for cartilage wear before making a final decision on the treatment plan. In some embodiments, the user can further use historical data and patient population analysis to compare implant types, as described above.

[0216] Furthermore, as shown in FIG. 25, the surgical planning system can also be configured to display the varus / valgus angle 2502 used for limb alignment.

[0217] Similarly, the surgical planning system can be configured to display other patient metrics, such as tibial slope, implant position (position) and location (location), resection values, and the like, via the three-dimensional model.

[0218] Once an implant, such as the implant shown in FIG. 26 for the patient's tibia, is selected for the patient's anatomical structure, the user (e.g., the surgeon) can further refine the position of the implant within the three-dimensional model. For example, if the implant is protruding (as shown in 2602), the surgical planning system can issue a warning to the user and suggest that the user re-evaluate the position of the implant. In some cases, if an appropriate position cannot be established, the surgical planning system can suggest a different implant.

[0219] [Multiple Method Examples] FIG. 27 shows an example method 2700 for classifying defects using a statistical shape model.

[0220] Method 2700 begins with step 2702 of acquiring medical image data related to the patient's anatomical structure.

[0221] Next, method 2700 proceeds to step 2704 of creating a three-dimensional anatomical structure model based on the medical image data.

[0222] Next, method 2700 proceeds to step 2706 of fitting a statistical shape model to the three-dimensional anatomical structure model.

[0223] Next, method 2700 proceeds to step 2708 of determining one or more quantitative measurement results based on the fitted statistical shape model.

[0224] Next, method 2700 proceeds to step 2710 of classifying the defect related to the anatomical structure of the patient based on the one or more quantitative measurement results.

[0225] In some embodiments of method 2700, the step of fitting the statistical shape model to the three-dimensional anatomical structure model further includes a step of subdividing the statistical shape model into a plurality of topological regions; and a step of determining a subset of the topological regions for use in fitting the statistical shape model to the three-dimensional anatomical structure model from the plurality of topological regions.

[0226] In some embodiments of method 2700, the step of determining the subset of the topological regions for use in fitting the statistical shape model to the three-dimensional anatomical structure model from the plurality of topological regions further includes a step of excluding each of the topological regions of the plurality of topological regions if the fitting error exceeds a threshold when each topological region is included in the subset of the topological regions.

[0227] In some embodiments of method 2700, the step of determining the subset of the topological regions for use in fitting the statistical shape model to the three-dimensional anatomical structure model from the plurality of topological regions further includes a step of selecting a first topological region from the plurality of topological regions; a step of fitting the statistical shape model to the three-dimensional anatomical structure model based only on the first topological region; and a step of calculating a first fitting error based on the first fitting of the statistical shape model based on the first topological region.

[0228] In some embodiments of method 2700, the first fitting error is calculated as the root mean square error (RMSE) between a plurality of points on the statistical shape model and a corresponding plurality of points on the three-dimensional anatomical structure model.

[0229] In some embodiments of method 2700, the step of determining the subset of the phase regions for use in fitting the statistical shape model to the three-dimensional anatomical structure model from the plurality of phase regions further includes: determining that the first fitting error is below a threshold; selecting a second phase region from the plurality of phase regions; fitting the statistical shape model to the three-dimensional anatomical structure model based on the second phase region; and calculating a second fitting error based on the second fitting of the statistical shape model based on the second phase region.

[0230] In some embodiments of method 2700, the step of determining the subset of the phase regions for use in fitting the statistical shape model to the three-dimensional anatomical structure model from the plurality of phase regions further includes: determining that the first fitting error exceeds a threshold; and excluding a second phase region of the plurality of phase regions from the subset of the phase regions based on the first fitting error exceeding the threshold.

[0231] In some embodiments, method 2700 further includes the step of excluding a third phase region of the plurality of phase regions from the subset of the phase regions based on the exclusion of the second phase region.

[0232] In some embodiments of method 2700, the threshold is about 1.7 mm. In some embodiments of method 2700, the threshold is in the range of 0.5 mm to 3 mm.

[0233] In some embodiments of method 2700, the step of determining the subset of the phase regions for use in fitting the statistical shape model to the three-dimensional anatomical structure model from the plurality of phase regions further includes excluding a phase region of the plurality of phase regions known to have damage or deformation from the subset of the phase regions.

[0234] In some embodiments of method 2700, the step of classifying the defect based on the one or more quantitative measurement results further includes combining two or more classification systems, each of which is based on a different perspective of the anatomical structure of the patient, to generate a three-dimensional classification.

[0235] In some embodiments, method 2700 further includes creating a default treatment plan based on the classified defect related to the anatomical structure of the patient.

[0236] In some embodiments, method 2700 further includes obtaining patient data related to a plurality of patients having the classified defect; selecting a population of patient data based on characteristics related to the patient; and displaying a treatment option analysis that compares a plurality of treatment options based on the population of patient data.

[0237] In some embodiments, method 2700 further includes indicating a patient reference on the treatment option analysis based on the characteristics related to the patient.

[0238] In some embodiments, method 2700 further includes changing the default treatment plan based on the treatment option analysis.

[0239] In some embodiments of method 2700, the plurality of treatment options relate to the treatment of a shoulder defect.

[0240] In some embodiments of method 2700, the plurality of treatment options relate to the treatment of a joint defect.

[0241] In some embodiments of method 2700, the plurality of treatment options relate to the treatment of an affected part of the anatomical structure.

[0242] In some embodiments of method 2700, the plurality of treatment options relate to treatment of a defective portion of the anatomical structure.

[0243] FIG. 28 shows an example method 2800 for determining treatment of an anatomical defect.

[0244] Method 2800 begins with step 2802 of obtaining medical image data related to a patient's anatomical structure.

[0245] Next, method 2800 proceeds to step 2804 of creating a three-dimensional anatomical structure model based on the medical image data.

[0246] Next, method 2800 proceeds to step 2806 of fitting a statistical shape model to the three-dimensional anatomical structure model.

[0247] Next, method 2800 proceeds to step 2808 of identifying the defect based on the three-dimensional anatomical structure model and the statistical shape model.

[0248] Next, method 2800 proceeds to step 2810 of determining a default treatment based on the identified defect.

[0249] Next, method 2800 proceeds to step 2812 of receiving patient population data related to a plurality of other patients having the identified defect, the patient population data including a plurality of patient population data subsets related to various treatments of the identified defect.

[0250] Next, method 2800 proceeds to step 2814 of generating a visual image including a representation of each patient population data subset based on at least one patient characteristic and a representation of the patient based on the at least one patient characteristic.

[0251] Next, method 2800 proceeds to step 2816 of selecting a final treatment for the patient.

[0252] In some embodiments of method 2800, the final treatment includes the modified default treatment.

[0253] In some embodiments of method 2800, the final treatment includes the default treatment.

[0254] In some embodiments, method 2800 further includes generating a new patient population data entry (input) based on the patient and treatment outcomes associated with the selected treatment.

[0255] In some embodiments of method 2800, the step of fitting the statistical shape model to the three-dimensional anatomical structure model further includes subdividing the statistical shape model into a plurality of topological regions; and determining a subset of the topological regions for use in fitting the statistical shape model to the three-dimensional anatomical structure model from the plurality of topological regions.

[0256] In some embodiments of method 2800, the step of determining the subset of the topological regions for use in fitting the statistical shape model to the three-dimensional anatomical structure model from the plurality of topological regions further includes excluding each of the topological regions of the plurality of topological regions if a fitting error exceeds a threshold when each topological region is included within the subset of the topological regions.

[0257] In some embodiments of method 2800, the step of determining the subset of the topological regions for use in fitting the statistical shape model to the three-dimensional anatomical structure model from the plurality of topological regions further includes selecting a first topological region from the plurality of topological regions; fitting the statistical shape model to the three-dimensional anatomical structure model based only on the first topological region; and calculating a first fitting error based on the first fitting of the statistical shape model based on the first topological region.

[0258] In some embodiments of method 2800, the first fitting error is calculated as the root mean square error (RMSE) between a plurality of points on the statistical shape model and a corresponding plurality of points on the three-dimensional anatomical structure model.

[0259] In some embodiments of method 2800, the step of determining the subset of the phase regions for use in fitting the statistical shape model to the three-dimensional anatomical structure model from the plurality of phase regions further includes determining that the first fitting error is below a threshold; selecting a second phase region from the plurality of phase regions; fitting the statistical shape model to the three-dimensional anatomical structure model based on the second phase region; and calculating a second fitting error based on the second fitting of the statistical shape model based on the second phase region.

[0260] In some embodiments of method 2800, the step of determining the subset of the phase regions for use in fitting the statistical shape model to the three-dimensional anatomical structure model from the plurality of phase regions further includes determining that the first fitting error exceeds a threshold; and excluding a second phase region of the plurality of phase regions from the subset of the phase regions based on the first fitting error exceeding the threshold.

[0261] In some embodiments, method 2800 further includes excluding a third phase region of the plurality of phase regions from the subset of the phase regions based on excluding the second phase region.

[0262] In some embodiments of method 2800, the threshold is about 1.7 mm.

[0263] In some embodiments of method 2800, the threshold is in the range of 0.5 mm to 3 mm.

[0264] In some embodiments of method 2800, the step of determining a subset of the phase regions for use in fitting the statistical shape model to the three-dimensional anatomical structure model from the plurality of phase regions further includes excluding from the subset of the phase regions those phase regions of the plurality of phase regions known to have damage or deformation.

[0265] In some embodiments of method 2800, the final treatment relates to the treatment of a shoulder defect.

[0266] In some embodiments of method 2800, the final treatment relates to the treatment of a joint defect.

[0267] In some embodiments of method 2800, the final treatment relates to the treatment of an affected part of the anatomical structure.

[0268] In some embodiments of method 2800, the final treatment relates to the treatment of a defective part of the anatomical structure.

[0269] FIG. 29 shows an example of a method for determining the treatment of an anatomical defect.

[0270] Method 2900 begins with step 2902 of acquiring medical image data related to a patient's anatomical structure.

[0271] Next, method 2900 proceeds to step 2904 of creating a three-dimensional anatomical structure model based on the medical image data.

[0272] Next, method 2900 proceeds to step 2906 of fitting a statistical shape model to the three-dimensional anatomical structure model.

[0273] Next, method 2900 proceeds to step 2908 of identifying a defect based on the three-dimensional anatomical structure model and the statistical shape model.

[0274] Next, method 2900 proceeds to step 2910 of receiving a default treatment plan using an analysis of historical data including a plurality of previously used pre-operative treatment plans for the identified defect.

[0275] Next, method 2900 optionally proceeds to step 2912 of generating a visual image including a representation of the treatment plan based on at least one patient characteristic and a representation of the patient based on the at least one patient characteristic.

[0276] Next, method 2900 proceeds to step 2914 of approving the final treatment for the patient.

[0277] In some embodiments of method 2900, the final treatment includes a modified default treatment.

[0278] In some embodiments of method 2900, the final treatment includes the default treatment.

[0279] In some embodiments, method 2900 further includes a step of generating a new patient population data entry based on the patient and treatment outcomes associated with the selected treatment.

[0280] In some embodiments of method 2900, the step of fitting the statistical shape model to the three-dimensional anatomical structure model further includes a step of subdividing the statistical shape model into a plurality of topological regions; and a step of determining a subset of the topological regions for use in fitting the statistical shape model to the three-dimensional anatomical structure model from the plurality of topological regions.

[0281] In some embodiments of method 2900, the step of determining the subset of the topological regions for use in fitting the statistical shape model to the three-dimensional anatomical structure model from the plurality of topological regions further includes a step of excluding each of the topological regions of the plurality of topological regions if a fitting error exceeds a threshold when each topological region is included within the subset of the topological regions.

[0282] In some embodiments of method 2900, the step of determining the subset of the phase regions for use in fitting the statistical shape model to the three-dimensional anatomical structure model from the plurality of phase regions further includes: selecting a first phase region from the plurality of phase regions; fitting the statistical shape model to the three-dimensional anatomical structure model based only on the first phase region; and calculating a first fitting error based on the first fitting of the statistical shape model based on the first phase region.

[0283] In some embodiments of method 2900, the first fitting error is calculated as the root mean square error (RMSE) between a plurality of points on the statistical shape model and a corresponding plurality of points on the three-dimensional anatomical structure model.

[0284] In some embodiments of method 2900, the step of determining the subset of the phase regions for use in fitting the statistical shape model to the three-dimensional anatomical structure model from the plurality of phase regions further includes: determining that the first fitting error is below a threshold; selecting a second phase region from the plurality of phase regions; fitting the statistical shape model to the three-dimensional anatomical structure model based on the second phase region; and calculating a second fitting error based on the second fitting of the statistical shape model based on the second phase region.

[0285] In some embodiments of method 2900, the step of determining the subset of the phase regions for use in fitting the statistical shape model to the three-dimensional anatomical structure model from the plurality of phase regions further includes: determining that the first fitting error exceeds a threshold; and excluding a second phase region of the plurality of phase regions from the subset of the phase regions based on the first fitting error exceeding the threshold.

[0286] In some embodiments, method 2900 further includes excluding a third phase region from the plurality of phase regions based on excluding the second phase region from the subset of phase regions.

[0287] In some embodiments of method 2900, the threshold is about 1.7 mm.

[0288] In some embodiments of method 2900, the threshold is in the range of 0.5 mm to 3 mm.

[0289] In some embodiments of method 2900, the step of determining the subset of phase regions for use in fitting the statistical shape model to the three-dimensional anatomical structure model from the plurality of phase regions further includes excluding from the subset of phase regions a phase region of the plurality of phase regions known to have damage or deformation.

[0290] In some embodiments of method 2900, the final treatment relates to the treatment of a shoulder defect.

[0291] In some embodiments of method 2900, the final treatment relates to the treatment of a joint defect.

[0292] In some embodiments of method 2900, the final treatment relates to the treatment of an affected part of the anatomical structure.

[0293] In some embodiments of method 2900, the final treatment relates to the treatment of a defective part of the anatomical structure.

[0294] [Example of a processing system] FIG. 30 shows an exemplary processing system 3000 configured to perform a method for detecting and removing personally identifiable information.

[0295] The processing system 3000 includes a CPU 3002 connected to a data bus 3008. The CPU 3002 processes computer-executable instructions stored (stored) in, for example, a memory 3010 or a storage 3030, and is configured to cause the processing system 3000 to execute a method such as the methods described herein with respect to FIGS. 27-29. The CPU 3002 is included as representative of a single CPU, multiple CPUs, a single CPU having multiple processing cores, and other forms of processing architectures capable of executing computer-executable instructions.

[0296] The processing system 3000 further includes an input / output device and an interface 3004 that enables the processing system 3000 to interface with input / output devices such as, for example, a keyboard, a display, a mouse device, a pen input, a touch-sensitive input device, a camera, a microphone, a medical imaging device (equipment), and other devices that enable interaction with the processing system 3000. Note that although not shown with an independent external input / output device, the processing system 3000 can be connected to an external input / output device (e.g., an external display device) through physical and wireless connections.

[0297] The processing system 3000 further includes a network interface 3006 that provides the processing system 3000 with access to external computing devices, for example, via a network 3009.

[0298] The processing system 3000 further includes a memory 3010. In this example, the memory 3010 includes various components configured to execute the multiple functions described herein. In this embodiment, the memory 3010 includes an imaging component 3012, a modeling component 3014, a conformity component 3016, a quantification component 3018, a classification component 3020, a decision-making component 3022, a selection component 3024, a display 3026, and an identification component 3028. These various components may include, for example, computer-executable instructions configured to execute the various functions described herein.

[0299] For simplicity, in FIG. 30, it is shown as a single memory 3010, but the various aspects stored within the memory 3010 may be stored within various physical memories, however, note that all are accessible to the CPU 3002 via a plurality of internal data connections such as a bus 3012. For example, some components of the memory 3010 may reside locally on the processing system 3000, while other components may be executed on a remote processing system in other embodiments, or within a cloud-based processing system. This is just an example.

[0300] The processing system 3000 further includes a storage 3030. In this example, the storage 3030 includes patient data 3032, medical image data 3034, patient population data 3036, treatment data 3038, surgical device data 3040, default plan data 3042, preoperative plan data 3044, intraoperative plan data 3046, postoperative plan data 3048, history data and a plurality of plots 3050, as well as SSM model data 3052. Although not shown in FIG. 30, a plurality of other aspects may be included within the storage 3030.

[0301] Similar to the case of memory 3010, a single storage 3030 is shown in FIG. 30 for simplicity. However, the various aspects stored within storage 3030 can be stored in various physical storages. Nevertheless, all are accessible to CPU 3002 via multiple internal data connections such as bus 3008, or via external connections such as network interface 3006.

[0302] [Additional Considerations] The above description has been provided to enable any person skilled in the art to make and use various embodiments described herein. The multiple examples discussed herein do not limit the scope, applicability, or multiple embodiments recited in the claims. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other multiple embodiments. For example, changes can be made to the functions and arrangements of the multiple elements discussed without departing from the scope of the present disclosure. Various examples may appropriately omit, substitute, or add various procedures or components. For example, the multiple methods described can be executed in an order different from that described, and various steps can be added, omitted, or combined. Also, the multiple features described for some examples can be combined in other some examples. For example, an apparatus can be implemented using any number of aspects described herein, or a method can be implemented using any number of aspects described herein. In addition, the scope of the present disclosure is intended to cover apparatuses or methods implemented using other structures, functionalities, or structures and functionalities in addition to, or other than, the various aspects related to the present disclosure described herein. It should be understood that any aspect related to the present disclosure disclosed herein can be embodied (implemented) by one or more elements according to the claims.

[0303] As used herein, the term "exemplary" means "serving as an example, instance, or illustration." Any aspect described herein as "exemplary" should not necessarily be construed as preferred or advantageous over other aspects.

[0304] As used herein, the phrase that refers to "at least one of" a list of items refers to any combination of those items, including a single member (element). For example, "at least one of: a, b, or c" is intended to cover a, b, c, a-b, a-c, b-c, and a-b-c, as well as any combination with multiple (duplicate) of the same element (e.g., a-a, a-a-a, a-a-b, a-a-c, a-b-b, a-c-c, b-b, b-b-b, b-b-c, c-c, and c-c-c, or any other arrangement of a, b, and c).

[0305] As used herein, the term "determining" encompasses a wide variety of acts. For example, "determining" can include calculating, computing, processing, deriving, investigating, searching (e.g., searching a table, database, or other data structure), ascertaining, etc. Also, "determining" can include receiving (e.g., receiving information), accessing (e.g., accessing data in a memory), etc. Also, "determining" can include solving, selecting, choosing, establishing, etc.

[0306] The multiple methods disclosed in this specification include one or more steps or acts for achieving the multiple methods. The steps and / or acts of the methods can be interchanged with each other without departing from the scope of the claims. In other words, unless a specific order of steps or acts is specified, the order and / or use of specific steps and / or acts can be modified (changed) without departing from the scope of the claims. Further, the various operations of the multiple methods described above can be performed by any suitable means capable of performing the corresponding functions. The means can include various hardware and / or software components and / or modules, including but not limited to circuits, application specific integrated circuits (ASICs), or processors. Generally, if there are operations shown in the figures, those operations can have corresponding equivalent means-plus-function components with similar numbering.

[0307] Various illustrative logical blocks, modules, and circuits described in connection with the present disclosure may be implemented or executed by a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device (PLD), discrete gates or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general purpose processor may be a microprocessor, but in the alternative, the processor may be any commercially available processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.

[0308] The processing system can be implemented by a bus architecture. The bus can include any number of interconnecting buses and bridges depending on the specific application of the processing system and the overall design constraints. The bus can be coupled to various circuits including, among other things, a processor, a machine-readable medium, and input / output devices. A user interface (e.g., keypad, display, mouse, joystick, etc.) can also be connected to the bus. The bus can also be coupled to various other circuits such as a timing source, peripherals, voltage regulators, power management circuits, and other circuit elements well known in the art and thus will not be described further. The processor can be implemented by one or more general-purpose processors and / or dedicated processors. For example, it includes a microprocessor, a microcontroller, a DSP processor, and other circuits capable of executing software. Those skilled in the art will recognize how best to implement the functionality described for the processing system depending on the particular application and the overall design constraints imposed on the system as a whole.

[0309] When implemented in software, the functions can be stored or transmitted as one or more instructions or codes on a computer-readable medium. Software should be interpreted broadly to mean instructions, data, or any combination thereof, whether referred to as software, firmware, middleware, microcode, hardware description language, or otherwise. A computer-readable medium includes both a computer storage medium and a communication medium such as any medium that facilitates transfer of a computer program from one place to another. A processor can perform general processing and bus management, including execution of software modules stored (stored) on a computer-readable storage medium. A computer-readable storage medium can be coupled to the processor so that the processor can read information from, and write information to, the storage medium. Alternatively, the storage medium can be integral to the processor. For example, a computer-readable medium can include a transmission line, a carrier wave modulated by data, and / or a computer-readable storage medium having instructions stored separately on a computer-readable storage medium from a wireless node, all of which can be accessed by the processor through a bus interface. Alternatively or additionally, a computer-readable medium, or any part thereof, can be integrated into the processor such that the case can have a cache and / or a general-purpose register file. Examples of machine-readable storage media can include, for example, RAM (Random Access Memory), flash memory, ROM (Read Only Memory), PROM (Programmable Read-Only Memory), EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), registers, magnetic disks, optical disks, hard drives, or any other suitable storage medium, or any combination thereof. A machine-readable medium can be embodied within a computer program product.

[0310] A software module can include a single instruction or many instructions and can be distributed among multiple and diverse code segments across various programs and over multiple storage media. A computer-readable medium can include a plurality of software modules. A software module includes instructions that, when executed by an apparatus such as a processor, cause a processing system to perform various functions. A software module can include a sending module and a receiving module. Each software module can exist within a single storage device or can be distributed across multiple storage devices. For example, a software module can be loaded from a hard drive to RAM when a trigger event occurs. During execution of a software module, the processor can load a portion of the instructions into cache memory to improve access speed. And, for execution by the processor, one or more cache lines can be loaded into the general-purpose register file. When referring to the functionality of a software module, it will be understood that such functionality is implemented by the processor when executing instructions from that software module.

[0311] The following claims are not intended to be limited to the multiple embodiments shown in this specification, but should be consistent with the full scope that conforms to the language of the claims. In the claims, reference to an element in the singular is not meant to mean "only one" unless explicitly stated to mean "only one", but rather means "one or more". Unless otherwise specified, the term "some" refers to one or more. No claim element should be construed under the provisions of 35 U.S.C. § 112(f) unless the element is explicitly recited using the phrase "means for" or, in the case of a method claim, the phrase "step for". All structural and functional equivalents to the elements described throughout this disclosure, whether known to those skilled in the art or later becoming known to those skilled in the art, are hereby expressly incorporated by reference and are intended to be encompassed within the claims. Further, nothing disclosed in this specification is intended to be dedicated to the public regardless of whether such disclosure is explicitly recited within the scope of the claims.

Claims

1. Processing a plurality of medical images of a patient's anatomical structure to generate processed data, Creating a default surgical plan based on the processed data, Receiving an input for modifying the default surgical plan to obtain an approved preoperative plan for the surgery of the patient's anatomical structure, and further, Obtaining historical surgical data including one or more of preoperative data, intraoperative data, or postoperative data related to the same anatomical type as the patient's anatomical structure, Obtaining a selection of a population of patients having a condition similar to the patient's anatomical structure, a selection of treatment decisions related to the default surgical plan, and a selection of one or more patient characteristics, Determining a portion of the historical surgical data related to the population, Performing an analysis associating the one or more patient characteristics with the treatment decision based on the portion of the historical surgical data including patient data associating one or more values regarding the one or more patient characteristics with the treatment decision for each of the one or more patients, Outputting the result of the analysis, A computer program configured to obtain an input regarding the treatment decision.

2. The computer program according to claim 1, wherein the preoperative data includes one or more preoperative surgical plans.

3. The computer program according to claim 1, wherein the intraoperative data includes the occurrence of intraoperative complications.

4. The computer program according to claim 1, wherein the postoperative data includes the occurrence of postoperative complications.

5. The computer program according to claim 1, wherein the processed data includes one or more of one or more virtual 3D models of anatomical parts of the patient's anatomical structure or data identifying one or more anatomical landmarks of the anatomical structure.

6. The computer program according to claim 1, wherein processing the plurality of medical images includes performing one or more of quantifying or classifying anatomical defects of the anatomical structure.

7. Performing one or more of quantifying or classifying the anatomical defects includes Creating a three-dimensional anatomical model of the anatomical structure based on the plurality of medical images, Fitting a statistical shape model to a portion of the three-dimensional anatomical model, Determining one or more quantitative measurements based on the adapted statistical shape model, and The computer program according to claim 6, comprising classifying the anatomical defect based on the one or more quantitative measurements.

8. The computer program according to claim 7, further configured to output one or more of the one or more quantitative measurements, a display of the classification of the anatomical defect, or a visual representation of an overlay of the three-dimensional anatomical model and the adapted statistical shape model.

9. The anatomical structure of the patient is the patient's shoulder, The anatomical defect is a defect of the glenoid fossa of the patient's shoulder, The one or more patient characteristics include quantification of the defect of the glenoid fossa, and The treatment decision is a choice of treatment method for an unspecified complaint related to the shoulder or a choice between a standard implant and a custom implant, the computer program according to claim 6.

10. Obtaining the selection of the population, the selection of the treatment decision, and the selection of the one or more patient characteristics includes obtaining the selection from a user, the computer program according to claim 1.

11. Obtaining the selection of the population, the selection of the treatment decision, and the selection of the one or more patient characteristics includes presenting a user with one or more pre-programmed combinations of selections, the computer program according to claim 1.

12. The selection criteria for selecting the population of patients are related to parameters of a statistical shape model that fits a part of the anatomical structure, the computer program according to claim 1.

13. The computer program according to claim 1, further configured to select the population by excluding treatment plans that are ineffective based on one or more of the intraoperative data or the postoperative data.

14. Outputting the result of the analysis includes outputting the histogram, distribution graph, surface chart, bar graph, or gradient plot that associates the distribution of treatment decision options across the population with the characteristics and positions of the one or more patients on the histogram, distribution graph, surface chart, bar graph, or gradient plot, the computer program according to claim 1.

15. The computer program according to claim 1, wherein the treatment decision relates to any one of shoulder treatment, shoulder surgery, craniofacial treatment, jaw orthopedic surgery, reconstructive surgery, heart treatment, or knee treatment.

16. The computer program according to claim 1, further configured to enable the approved preoperative plan to be executable in the surgery.

Images