Preoperative planning and intraoperative guidance for orthopedic surgery in cases of bone fragmentation.

Abstract

To provide a device, a system, and a method for a preoperative plan and an intraoperative guidance for orthopedic surgery of a fractured joint suffering bone fragmentation.SOLUTION: A surgery system can be configured to acquire image data of a joint including at least a portion of humerus, segment the image data to determine a shape of a diaphysis of the humerus, determine a shape of the humerus before it becomes the estimated pathosis on the basis of the determined shape of the diaphysis, identify one or more bone chips in the image data on the basis of the estimated shape of the humerus, and generate an output on the basis of the bone chips identified in the image data.SELECTED DRAWING: Figure 35

Description

[Technical Field]

[0001]

[0001] This application claims the interests of U.S. Provisional Patent Application No. 62 / 940,810, filed on 26 November 2019, and U.S. Provisional Patent Application No. 63 / 049,497, filed on 8 August 2020, the entire contents of both applications are incorporated herein by reference. [Background technology]

[0002]

[0002] Surgical joint repair procedures include the repair and / or replacement of damaged or diseased joints. One example of a surgical joint repair procedure, such as arthroplasty, involves replacing a damaged joint with a prosthesis that is implanted in the patient's bone. Appropriate selection or design of a properly sized and molded prosthesis and proper positioning of the prosthesis are important to ensure optimal surgical outcomes. The surgeon can analyze the damaged bone to assist in the surgical steps for prosthesis selection, design and / or positioning, and for preparing the bone or tissue to accommodate or interact with the prosthesis. [Overview of the project]

[0003]

[0003] With respect to fractured joints, fracture patterns can have a wide range of variations and can be difficult to classify and analyze. This disclosure describes devices, systems, and methods for preoperative planning and intraoperative guidance for orthopedic surgery of fractured joints suffering from bone fragmentation.

[0004]

[0004] In one example, the method includes: acquiring image data of a joint including at least a portion of the humerus; segmenting the image data to determine the shape of the diaphysis of the humerus; determining the shape of the humerus before it is estimated to be in a pathological state based on the determined shape of the diaphysis; identifying one or more bone fragments in the image data based on the estimated shape of the humerus; and generating an output based on the identified bone fragments in the image data.

[0005]

[0005] In another example, the system includes a memory for storing image data of a joint including at least a portion of the humerus, and one or more processors configured to acquire image data of the joint including at least a portion of the humerus, segment the image data to determine the shape of the shaft of the humerus, determine the shape of the humerus before it is estimated to be in a pathological state based on the determined shape of the shaft, identify one or more bone fragments in the image data based on the estimated shape of the humerus, and generate an output based on the identified bone fragments in the image data.

[0006]

[0006] In another example, a computer-readable storage medium stores instructions that, when executed, cause one or more processors of a surgical system to acquire image data of a joint including at least a portion of the humerus, segment the image data to determine the shape of the shaft of the humerus, determine the shape of the humerus before it is estimated to be in a pathological state based on the determined shape of the shaft, identify one or more bone fragments in the image data based on the estimated shape of the humerus, and generate an output based on the identified bone fragments in the image data.

[0007]

[0007] In another example, the surgical system includes means for acquiring image data of a joint including at least a portion of the humerus; means for segmenting the image data to determine the shape of the diaphysis of the humerus; means for determining the shape of the humerus before it is estimated to be in a pathological state based on the determined shape of the diaphysis; means for identifying one or more bone fragments in the image data based on the estimated shape of the humerus; and means for generating an output based on the identified bone fragments in the image data.

[0008]

[0008] In another example, the method includes: acquiring image data of a joint including at least a portion of the humerus; segmenting the image data to identify portions of the image data corresponding to cortical bone; generating a three-dimensional (3D) model based on the portions of the image data corresponding to cortical bone, the 3D model including one or more 3D meshes corresponding to the surface of the portions of the image data corresponding to cortical bone, identifying portions of 3D meshes corresponding to the diaphysis in one or more 3D meshes; determining the shape of the humerus before it is estimated to be in a pathological state based on the shape of the portions of 3D meshes corresponding to the diaphysis; and generating an output based on the shape of the humerus before it is estimated to be in a pathological state.

[0009]

[0009] In another example, the medical system includes a memory configured to store image data of a joint including at least a portion of the humerus, and a processing path, the processing circuit which acquires image data of the joint including at least a portion of the humerus, segments the image data to identify portions of the image data corresponding to cortical bone, generates a three-dimensional (3D) model based on the portions of the image data corresponding to cortical bone, the 3D model includes one or more 3D meshes corresponding to the surfaces of the portions of the image data corresponding to cortical bone, identifies portions of the 3D meshes corresponding to the diaphysis in one or more 3D meshes, determines the shape of the humerus before it is estimated to be in a pathological state based on the shape of the portions of the 3D meshes corresponding to the diaphysis, and generates an output based on the shape of the humerus before it is estimated to be in a pathological state.

[0010] According to another example, a computer-readable storage medium includes obtaining image data of a joint including at least a portion of the humerus, segmenting the image data to identify a portion of the image data corresponding to cortical bone, generating a three-dimensional (3D) model based on the portion of the image data corresponding to cortical bone, the 3D model including one or more 3D meshes corresponding to the surface of the portion of the image data corresponding to cortical bone, identifying a portion of the 3D mesh corresponding to the diaphysis in the one or more 3D meshes, determining a shape of the humerus before a presumed pathological state based on the shape of the portion of the 3D mesh corresponding to the diaphysis, and generating an output based on the shape of the humerus before the presumed pathological state.

[0011] According to another example, a medical system includes means for obtaining image data of a joint including at least a portion of the humerus, means for segmenting the image data to identify a portion of the image data corresponding to cortical bone, means for generating a three-dimensional (3D) model based on the portion of the image data corresponding to cortical bone, the 3D model including one or more 3D meshes corresponding to the surface of the portion of the image data corresponding to cortical bone, means for identifying a portion of the 3D mesh corresponding to the diaphysis in the one or more 3D meshes, means for determining a shape of the humerus before a presumed pathological state based on the shape of the portion of the 3D mesh corresponding to the diaphysis, and means for generating an output based on the shape of the humerus before the presumed pathological state.

[0012] Details of various examples of the present disclosure are set forth in the accompanying drawings and the following description. Various features, objects, and advantages will be apparent from the description, drawings, and claims.

Brief Description of the Drawings

[0013] [Figure 1] FIG. 1 is a block diagram illustrating an exemplary computing device that may be used to implement the techniques of the present disclosure. [Figure 2]

[0014] Figure 2 shows an exemplary workflow according to the technology of the present disclosure. [Figure 3A]

[0015] Figure 3A shows the Hessian eigenvector decomposition for a tubular structure. [Figure 3B] Figure 3B shows the Hessian eigenvector decomposition for a blob structure. [Figure 3C] Figure 3C shows the Hessian eigenvector decomposition for a sheet-like structure. [Figure 4A]

[0016] Figure 4A shows examples of different positions of a displaced fractured humeral head. [Figure 4B] Figure 4B shows examples of different positions of a displaced fractured humeral head. [Figure 4C] Figure 4C shows examples of different positions of a displaced fractured humeral head. [Figure 4D] Figure 4D shows examples of different positions of a displaced fractured humeral head. [Figure 5A]

[0017] Figure 5A shows an example of automatic detection of a fractured humeral head and related segmented zones for the example of Figure 4A. [Figure 5B] Figure 5B shows an example of automatic detection of a fractured humeral head and related segmented zones for the example of Figure 4B. [Figure 5C] Figure 5C shows an example of automatic detection of a fractured humeral head and related segmented zones for the example of Figure 4C. [Figure 5D] Figure 5D shows an example of automatic detection of a fractured humeral head and related segmented zones for the example of Figure 4D. [Figure 6A]

[0018] Figure 6A shows an example of a fractured fragment region of an identified object. [Figure 6B] Figure 6B shows an example of a damaged fragment region of an identified object. [Figure 6C] Figure 6C shows an example of a damaged fragment region of an identified object. [Figure 6D] Figure 6D shows an example of a damaged fragment region of an identified object. [Figure 6E] Figure 6E shows an example of a damaged fragment region of an identified object. [Figure 6F] Figure 6F shows an example of a damaged fragment region of an identified target. [Figure 7A]

[0019] Figure 7A shows an example of a detected humeral head. [Figure 7B] Figure 7B shows an example of a detected humeral head. [Figure 7C] Figure 7C shows an example of a detected humeral head. [Figure 8A]

[0020] Figure 8A shows an example of a removed humeral head point. [Figure 8B] Figure 8B shows an example of a removed humeral head point. [Figure 8C] Figure 8C shows an example of a removed humeral head point. [Figure 9A]

[0021] Figure 9A shows an example for calculating the start limit and end limit. [Figure 9B] Figure 9B shows an example for calculating the start limit and end limit. [Figure 9C] Figure 9C shows an example for calculating the start limit and end limit. [Figure 10A]

[0022] Figure 10A shows an example of calculating the diaphysis point. [Figure 10B] Figure 10B shows an example of diaphysis point calculation. [Figure 10C] Figure 10C shows an example of diaphysis point calculation. [Figure 10D] Figure 10D shows an example of diaphysis point calculation. [Figure 11]

[0023] Figure 11 illustrates the detection of the innermost and outermost elbow points. [Figure 12A]

[0024] Figure 12A shows the elbow marker. [Figure 12B] Figure 12B shows the elbow marker. [Figure 13A]

[0025] Figure 13A illustrates the pulley and the detection of the small head marker. [Figure 13B] Figure 13B illustrates the pulley and the detection of the small head marker. [Figure 13C] Figure 13C illustrates the pulley and small head marker detection. [Figure 14A]

[0026] Figure 14A shows an example of detecting the biceps groove. [Figure 14B] Figure 14B shows an example of detecting the biceps brachii groove. [Figure 15A]

[0027] Figure 15A illustrates an example of humeral marker detection. [Figure 15B] Figure 15B illustrates an example of humeral marker detection. [Figure 15C] Figure 15C illustrates an example of humeral marker detection. [Figure 15D] Figure 15D illustrates an example of humeral marker detection. [Figure 16A]

[0028] Figure 16A shows an example of the upper humeral zone for establishing correspondence. [Figure 16B] Figure 16B shows an example of the upper humeral zone for establishing correspondence. [Figure 17A]

[0029] Figure 17A shows examples of markers for rigid body alignment and non-rigid body alignment. [Figure 17B] Figure 17B shows examples of markers for rigid body alignment and non-rigid body alignment. [Figure 17C] Figure 17C shows examples of markers for rigid body alignment and non-rigid body alignment. [Figure 17D] Figure 17D shows examples of markers for rigid body alignment and non-rigid body alignment. [Figure 18A]

[0030] Figure 18A shows different variations in the average shape of the humerus. [Figure 18B] Figure 18B shows different variations in the average shape of the humerus. [Figure 18C] Figure 18C shows different variations in the average shape of the humerus. [Figure 18D] Figure 18D shows different variations in the average shape of the humerus. [Figure 18E]Figure 18E shows the different variations in the average shape of the humerus. [Figure 18F] Figure 18F shows different variations in the average shape of the humerus. [Figure 18G] Figure 18G shows different variations in the average shape of the humerus. [Figure 18H] Figure 18H shows different variations in the average shape of the humerus. [Figure 19A]

[0031] Figure 19A shows the effect of humeral length on variations in posterior tilt and inclination. [Figure 19B] Figure 19B shows the effect of humeral length on variations in posterior tilt and inclination. [Figure 20A]

[0032] Figure 20A illustrates different processes that lead to the fusion of fractured fragments. [Figure 20B] Figure 20B illustrates the different processes that lead to the fusion of fractured fragments. [Figure 20C] Figure 20C illustrates different processes that lead to the fusion of fractured fragments. [Figure 21A]

[0033] Figure 21A illustrates a step segmentation algorithm applied to the proximal portion of a fractured humerus. [Figure 21B] Figure 21B illustrates a step segmentation algorithm applied to the proximal portion of a fractured humerus. [Figure 21C] Figure 21C illustrates a step segmentation algorithm applied to the proximal portion of a fractured humerus. [Figure 21D] Figure 21D illustrates a step segmentation algorithm applied to the proximal portion of a fractured humerus. [Figure 21E] Figure 21E illustrates a step segmentation algorithm applied to the proximal portion of a fractured humerus. [Figure 21F] Figure 21F illustrates a step segmentation algorithm applied to the proximal portion of a fractured humerus. [Figure 22A]

[0034] Figure 22A shows a 3D image of a proximal humerus fracture with the apex along the fracture line. [Figure 22B] Figure 22B shows a 3D image of a proximal humerus fracture with the apex along the fracture line. [Figure 23A]

[0035] Figure 23A shows the 2D contour for top-level vertex detection performed using the discrete curve expansion (DCE) algorithm. [Figure 23B] Figure 23B shows the 2D contour for top-level vertex detection performed using the discrete curve expansion (DCE) algorithm. [Figure 24A]

[0036] Figure 24A shows one type of contour topology. [Figure 24B] Figure 24B shows one type of contour topology. [Figure 25A]

[0037] Figure 25A shows the steps associated with the segmentation algorithm for step segmentation. [Figure 25B] Figure 25B shows the steps associated with the segmentation algorithm for step segmentation. [Figure 26A]

[0038] Figure 26A shows the process for detecting and characterizing the diaphysis. [Figure 26B] Figure 26B shows the process for detecting and characterizing the diaphysis. [Figure 26C] Figure 26C shows the process for detecting and characterizing the diaphysis. [Figure 26D] Figure 26D shows the process for detecting and characterizing the diaphysis. [Figure 27A]

[0039] Figure 27A shows an example of the initial alignment of the fractured diaphysis with the statistical shape model (SSM) of the humerus. [Figure 27B] Figure 27B shows an example of the initial alignment of the fractured diaphysis with the statistical shape model (SSM) of the humerus. [Figure 27C] Figure 27C shows an example of the initial alignment of the fractured diaphysis with the statistical shape model (SSM) of the humerus. [Figure 27D]Figure 27D shows an example of the initial alignment of the fractured diaphysis with the statistical shape model (SSM) of the humerus. [Figure 27E] Figure 27E shows an example of the initial alignment of the fractured diaphysis with the statistical shape model (SSM) of the humerus. [Figure 27F] Figure 27F shows an example of the initial alignment of the fractured diaphysis with the statistical shape model (SSM) of the humerus. [Figure 27G] Figure 27G shows an example of the initial alignment of the fractured diaphysis with the statistical shape model (SSM) of the humerus. [Figure 27H] Figure 27H shows an example of the initial alignment of the fractured diaphysis with the statistical shape model (SSM) of the humerus. [Figure 28A]

[0040] Figure 28A shows an example of estimating the shape of a fractured humerus based on the shape of the diaphysis. [Figure 28B] Figure 28B shows an example of estimating the shape of a fractured humerus based on the shape of the diaphysis. [Figure 28C] Figure 28C shows an example of estimating the shape of a fractured humerus based on its diaphysis. [Figure 29]

[0041] This flowchart illustrates an exemplary way in which a computing device operates using one or more exemplary technologies described in this disclosure. [Figure 30A]

[0042] Figure 30A shows an example of a 3D mesh corresponding to a portion of the diaphysis. [Figure 30B] Figure 30B shows an example of a 3D mesh corresponding to a portion of the diaphysis. [Figure 30C] Figure 30C shows an example of a 3D mesh corresponding to a portion of the diaphysis. [Figure 30D] Figure 30D shows an example of a 3D mesh corresponding to a portion of the diaphysis. [Figure 31A]

[0043] Figure 31A shows an example of a predicted 3D mesh of the humerus with a sphere that identifies the region of the humeral head. [Figure 31B] Figure 31B shows an example of a predicted 3D mesh of the humerus with a sphere that identifies the region of the humeral head. [Figure 31C]Figure 31C shows an example of a predicted 3D mesh of the humerus with a sphere that identifies the region of the humeral head. [Figure 32A]

[0044] Figure 32A shows the allowable area of ​​the humeral head for positioning the humeral head fragment. [Figure 32B] Figure 32B shows the allowable area of ​​the humeral head for positioning the humeral head fragment. [Figure 32C] Figure 32C shows the allowable area of ​​the humeral head for positioning the humeral head fragment. [Figure 33A]

[0045] Figure 33A shows an example of fragment translation. [Figure 33B] Figure 33B shows an example of a segment rotation. [Figure 34A]

[0046] Figure 34A is a flowchart illustrating an exemplary way in which a computing device operates using one or more exemplary techniques described in this disclosure. [Figure 34B] Figure 34B is a flowchart illustrating an exemplary way in which a computing device operates using one or more exemplary techniques described in this disclosure. [Figure 35]

[0047] Figure 35 is a flowchart illustrating an exemplary way in which a computing device operates using one or more exemplary techniques described in this disclosure. [Modes for carrying out the invention]

[0014]

[0048] Patients may be suffering from diseases (e.g., illnesses) or injuries that cause damage to their anatomical structures. Regarding the shoulder, examples of diseases or injuries the patient may suffer from, in some cases, primary glenohumeral osteoarthritis (PGHOA), rotator cuff tear arthritis (RCTA), instability, extensive rotator cuff tear (MRCT), rheumatoid arthritis (RA), post-traumatic arthritis (PTA), osteoarthritis (OA), or acute fracture. According to some estimates, humeral fractures are the fourth most common type of fracture worldwide occurring in osteoporotic bones (after hip fractures, vertebral fractures, and radial fractures), accounting for approximately 8% of all fractures worldwide.

[0015]

[0049] Proximal humerus fracture patterns exhibit a wide range of variations and can be difficult to classify. Several classification systems have been proposed to help surgeons choose treatment for specific patients. In 1970, Charles Neer described the four-segment classification system, recognizing that existing systems did not distinguish between injuries of varying severity and did not group them as fractures. The classification systems at the time were based on the mechanism of injury or the level of the fracture line, but did not consider many surgically significant aspects or pathological features of the injury, such as tubercle displacement. Neer's classification system was based on an observation made much earlier by Codman that all proximal humerus fractures consist of four main segments: the lesser tubercle, the greater tubercle, the articular surface, and the humeral shaft. The four-segment classification system defines proximal humerus fractures by the number of displaced segments or parts, with further categories for articular fractures and dislocations. The potentially associated segments are the greater tubercle, lesser tubercle, articular surface, and humeral shaft. A segment is defined as displaced if there is a separation of more than 1 cm or a 45-degree angle formation. Dislocations represent more severe injuries, are more likely to involve ischemic necrosis, and progress to heterotopic ossification; therefore, a separate category has been added for dislocations.

[0016]

[0050] The AO classification, along with the Neer classification, is a frequently used technique for classifying fractures. The AO classification divides proximal humeral fractures into three groups, A, B, and C, each with subgroups, emphasizing the blood supply to the articular surface. If either the lesser or greater tubercle remains attached to the articular segment, the blood supply is presumably adequate to avoid ischemic necrosis. The risk of ischemic necrosis increases from type A (very low) to type C (high risk), thus determining the treatment.

[0017]

[0051] Due to the combination of complex anatomical structures and intricate classification systems, both described systems suffer from poor inter- and intra-observer reproducibility. Despite this known drawback, both systems are currently used for planning proximal humerus fracture surgeries. Some of the observed variability may be attributable to the difficulty in interpreting complex three-dimensional (3-D) fracture patterns on two-dimensional plain radiographs.

[0018]

[0052] To address illness or injury, surgeons may perform surgical procedures such as, in some cases, reverse arthroplasty (RA), extended reverse arthroplasty (RA), standard total shoulder arthroplasty (TA), extended total shoulder arthroplasty (TA), or hemisphere-shoulder arthroplasty. It may be beneficial for surgeons to determine the characteristics of the patient's anatomical structure (e.g., size, shape, and / or location) before surgery. For example, determining the characteristics of the patient's anatomical structure can assist in the selection, design, and / or positioning of prostheses, as well as the planning of surgical steps to prepare the damaged bone surface to receive or interact with the prosthesis. Pre-planning allows surgeons to determine, rather than during, the steps to prepare the bone or tissue, the tools required, the size and shape of the tools, the size and shape or other characteristics of one or more prostheses to be implanted, and similar items.

[0019]

[0053] Pre-pathological characterization refers to characterizing a patient's anatomical structures as they existed before the patient suffered from the illness or injury. However, pre-pathological characterization of anatomical structures is not generally available because patients may not consult a doctor or surgeon until after they have become ill or injured.

[0020]

[0054] Anatomical structures prior to the onset of a pathological condition, also known as innate anatomical structures, refer to anatomical structures before the onset of disease or injury. Even after disease or injury, there may be parts of anatomical structures that are healthy and parts that are not healthy (e.g., diseased or damaged). Diseased or damaged parts of anatomical structures are called pathological anatomical structures, and healthy parts of anatomical structures are called non-pathological anatomical structures.

[0021]

[0055] As will be described in more detail below, according to the technology of this disclosure, a computing device can be configured to receive image data (e.g., CT scans) of a patient's injured joint. The computing device may also receive other information, such as the patient's sex or age. Based on this data, the computing device may be configured to detect and classify the fracture pattern of the injured joint. Based on this classification, the computing device may be configured to recommend a treatment method (e.g., no surgery, minimally invasive surgery (e.g., nails), or implant placement). If implant placement is selected by the surgeon, the computing device may also be configured to suggest the implant type, size, orientation, position, etc. The computing device may also be configured to provide the surgeon with guidance, such as visual guidance, regarding how to move the fractured fragments to achieve the desired reduction.

[0022]

[0056] This disclosure also describes techniques for configuring a computing device to align a fracture with image data. Specifically, the computing device may be configured to perform a fully or partially automated process of manipulating fragments to reconstruct the proximal humeral shape, and thus provide guidance to the surgeon on how to manipulate the fragments during surgery. As will be described in more detail below, according to the techniques of this disclosure, the computing device may be configured to position the diaphysis of a fractured humerus within image data and, based on the image of the diaphysis, determine the estimated shape of the humerus before the fracture. In a fracture event, all fragments of the humerus, including the diaphysis, may be displaced relative to where those fragments are located in an unfractured humerus. Therefore, since the diaphysis may be displaced as a result of the fracture, it may not be usable as a reference point for determining the positions of other fragments of the humerus. According to the techniques of this disclosure, a part of the scapula, such as the glenoid cavity, can be used as a reference for determining the position of the reassembled humerus after the fracture. In some implementations, the computing device may first determine the position of the diaphysis based on the position of the glenoid cavity. Next, the shaft can be used as a reference point to move the other fragments of the fractured humerus. Typically, the glenoid cavity does not move in the fracture event, making it available as a reference point for positioning the humerus.

[0023]

[0057] Image data, in this context, includes both display images and raw image data related to anatomical structure scans. In this context, raw image data refers to the image data used to determine the display image. Raw image data may have different resolutions, bit depths, or dynamic ranges than the display image, and may also be in different color spaces or color models. Generally, raw image data includes data used by a display or visualization device to display an image, and, in some embodiments, even data not directly used by a display or visualization device to display an image, which is captured as part of imaging. For example, raw image data may include data that would not conventionally be considered image data in the context of this disclosure. As an example, in CT image data, pixels are associated with relative radiation concentration values ​​corresponding to mean attenuation measured in Hounsfield units (HU) using a Hounsfield scale. These HU values ​​are an example of raw image data. The display device converts the HU values ​​to grayscale for display. Unless otherwise stated, the techniques of this disclosure described as being performed on image data should be assumed to be able to be performed using either display images or raw images.

[0024]

[0058] Furthermore, it should be understood that image data in this disclosure refers to both 2D and 3D image data. For simplicity, certain techniques are described as being performed on pixels representing locations within a 2D model, but unless otherwise specified, it should be assumed that these techniques can also be performed on voxels representing locations within a 3D model. Similarly, unless otherwise stated, techniques described as being performed on voxels should also be assumed to be performable on pixels.

[0025]

[0059] Figure 1 is a block diagram illustrating an exemplary computing device that may be used to implement the technologies of this disclosure. Figure 1 illustrates Device 100, an example of a computing device configured to perform one or more exemplary technologies described in this disclosure.

[0026]

[0060] Device 100 may include various types of computing devices, such as server computers, personal computers, smartphones, laptop computers, and other types of computing devices. Device 100 includes processing circuits 102, memory 104, and a display 110. The display 110 is optional, as in the example where device 100 is a server computer.

[0027]

[0061] Examples of the processing circuit 102 include one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. Generally, the processing circuit 102 may be implemented as a fixed-function circuit, a programmable circuit, or a combination thereof. A fixed-function circuit refers to a circuit that provides a specific functionality and is pre-configured to perform certain operations. A programmable circuit refers to a circuit that can be programmed to perform a variety of tasks and provides flexible functionality in the operations it can perform. For example, a programmable circuit may execute software or firmware that operates the programmable circuit in a manner defined by software or firmware instructions. A fixed-function circuit may execute software instructions (e.g., to receive or output parameters), but the type of operation performed by a fixed-function circuit is generally immutable. In some examples, one or more units may be separate circuit blocks (fixed-function or programmable), and in some examples, one or more units may be integrated circuits.

[0028]

[0062] The processing circuit 102 may include an arithmetic logic unit (ALU), an elementary function unit (EFU), digital circuits, analog circuits, and / or a programmable core, formed from programmable circuits. In an example where the operation of the processing circuit 102 is performed using software executed by the programmable circuits, memory 104 may store object code of the software that the processing circuit 102 receives and executes, or another memory (not shown) within the processing circuit 102 may store such instructions. An example of software includes software designed for surgical planning.

[0029]

[0063] Memory 104 may be formed by any of various memory devices, such as DRAM, which includes synchronous dynamic random access memory (SDRAM), magnetoresistive RAM (MRAM), resistive RAM (RRAM®), or other types of memory devices. Examples of display 110 include liquid crystal displays (LCDs), plasma displays, organic light-emitting diode (OLED) displays, or other types of display devices.

[0030]

[0064] Device 100 may include a communication interface 112 that enables device 100 to output data and commands to a visualization device 116 via a network 114 and to receive data and commands from the visualization device 116. For example, after determining the pre-pathological features of an anatomical object using the techniques described herein, the communication interface 112 may output information about the pre-pathological features to the visualization device 116 via the network 114. The surgeon may then use the visualization device 116 to view a graphic representation of the pre-pathological anatomical object (for example, in some cases, with the pre-pathological anatomical object overlaid on an image of the damaged or pathological anatomical object).

[0031]

[0065] The communication interface 112 may be hardware circuitry that enables device 100 to communicate (for example, wirelessly or using wires) with other computing systems and devices such as the visualization device 116. The network 114 may include various types of communication networks, including one or more wide area networks such as the Internet, a local area network, etc. In some examples, the network 114 may include wired and / or wireless communication links.

[0032]

[0066] The visualization device 116 may display image content to the surgeon using various visualization technologies. The visualization device 116 may be a mixed reality (MR) visualization device, a virtual reality (VR) visualization device, a holographic projector, or other device for presenting extended reality (XR) visualization. In some examples, the visualization device 116 may be a Microsoft HOLOLENS® headset, available from Microsoft Corporation in Redmond, Washington, USA, or a similar device such as a similar MR visualization device including a waveguide. Using a HOLOLENS® device, 3D virtual objects can be presented via a holographic lens or waveguide while enabling the user to see a real-world scene, i.e., actual objects in a real-world environment, through the holographic lens.

[0033]

[0067] The visualization device 1116 may utilize visualization tools available to facilitate preoperative planning for joint repair and replacement by generating a three-dimensional model of the bone outline using patient image data. These tools enable surgeons to design and / or select surgical guides and implant components that closely match the patient's anatomical structure. These tools can improve surgical outcomes by customizing the surgical plan for each patient. An example of such visualization tools for shoulder repair is the BLUEPRINT® system, available from Wright Medical Technology, Inc. The BLUEPRINT® system provides surgeons with a two-dimensional plan view of the bone repair area and a three-dimensional virtual model of the repair area. Using the BLUEPRINT® system, surgeons can select, design, or modify appropriate implant components, determine how to best position and orient the implant components, and how to shape the bone surface to receive the components, and design, select, or modify surgical guide tools or instruments to execute the surgical plan. Information generated by the BLUEPRINT® system may be edited during the preoperative surgical planning for the patient, stored in a database at an appropriate location (e.g., on a server in a wide area network, local area network, or global network) that can be accessed by the surgeon or other healthcare provider, including before and during the actual surgery.

[0034]

[0068] As shown in the figure, memory 104 stores data representing the shape model 106 and data representing the anatomical structure scan 108. The anatomical structure scan 108 is an example of a computed tomography (CT) scan of a patient, such as represented by CT scan image data.

[0035]

[0069] A CT scan combines a series of X-ray images taken from different angles and uses computer processing to create cross-sectional images or slices of bones, blood vessels, and soft tissues inside the body. While CT scans have many applications, they are particularly well-suited for quickly examining people who may have internal injuries, such as fractures from car accidents or other types of trauma. CT scans can be used to visualize almost any part of the body and are used to diagnose illnesses or injuries, as well as to plan medical, surgical, or radiation therapies.

[0036]

[0070] The output of a CT scan is a volumetric image composed of basic cubes (i.e., voxels), each voxel associated with a numerical value representing the X-ray attenuation measured at its corresponding 3D location. This value is typically converted to a Haunsfeld unit (HU) scale. The HU scale is a linear transformation of the original linear attenuation coefficient measurements to one where the radiation density of distilled water at standard pressure and temperature (STP) is defined as 0 HU, while the radiation density of air at STP is defined as -1000 HU. High HU values ​​above 400 correspond to high-density materials such as bone, while low HU values ​​correspond to low-density materials such as fat (e.g., -100 to -50 HU), muscle (+10 to +40 HU), and blood (+30 to +45 HU). For computer visualization purposes, HU values ​​are mapped to grayscale intensities.

[0037]

[0071] An anatomical structure scan 108 may be sufficient to reconstruct a three-dimensional (3D) representation of a patient's anatomical structure, such as the scapula and glenoid fossa, as an example of the patient's anatomical structure. One exemplary method of automated segmentation is described in U.S. Patent No. 8,971,606. Various other methods of performing automated segmentation may exist, and this technique is not limited to automated segmentation using the technique described in U.S. Patent No. 8,971,606. As an example, segmentation of CT image data to generate segmented objects includes comparing voxel intensity in the image data to determine the anatomical structure of bone and comparing it with the estimated size of the anatomical structure of bone to determine the segmented object. Furthermore, exemplary techniques may be performed using non-automated segmentation techniques in which a medical professional evaluates the CT image data to segment the anatomical objects, or using some combination of automation and user input for segmenting the anatomical objects.

[0038]

[0072] In one or more examples, the anatomical structure scan 108 may be a scan of an anatomical structure that is pathological due to injury or disease. A patient may have an injured shoulder requiring surgery, and the surgeon may request the anatomical structure scan 108 for the surgery, or possibly as part of the diagnosis, to plan the surgery. A computing device (such as device 100 or some other device) can generate a segmentation of the patient's anatomical structure, thereby allowing the surgeon to see the size, shape, and interconnections of anatomical objects, as well as the object's size, shape, and interconnections with other anatomical structures of the patient's anatomical structure requiring surgery. In cases of trauma, not all fracture types necessarily require surgery. For example, one-part or two-part fractures usually do not require surgery and are typically sufficient with shoulder immobilization. In the case of two-part or three-part fractures, complete surgery or a minimally invasive intervention may be sufficient for treatment. As will be described in more detail elsewhere in this disclosure, device 100 may be configured to perform fracture classification during the diagnostic phase. If the surgeon decides to perform surgery, device 100 may also be configured to plan the surgical intervention.

[0039]

[0073] In one or more examples, the processing circuit 102 can use the image data from scan 108 to compare (e.g., size, shape, orientation, etc.) with a statistical shape model (SSM) as a way to determine the characteristics of the patient's anatomical structures before the patient became injured or ill. In some examples, the processing circuit 102 can compare 3D point data of undamaged points of anatomical objects of the patient's anatomical structures in the image data from scan 108 with points in the SSM.

[0040]

[0074] For example, scan 108 provides the surgeon with a view of the current features of the injured or diseased anatomical structure. To reconstruct the anatomical structure (i.e., to represent the state before the pathological condition), the surgeon may find it beneficial to have a model that features the anatomical structure before the injury or disease. For example, the model may be a predictor of the patient's anatomical structure before the injury or disease. However, the patient may not have consulted a surgeon until the injury occurred or the disease progressed, and therefore a model of the patient's anatomical structure before the injury or disease (e.g., pre-pathological or congenital anatomical structure) may not be available. By using SSM as a way to model the anatomical structure before the injury, the surgeon can determine the features of the patient's anatomical structure before the pathological condition.

[0041]

[0075] SSM is a compact tool for representing shape variations between sample populations (databases). For example, a clinician or researcher may generate a database of image scans, such as CT image data, representing an entire population unaffected by injury or disease to the glenoid fossa, humerus, or adjacent bones. The clinician or researcher can then determine the average shape of a patient's anatomical structure from the shapes in the database. In Figure 1, memory 104 stores information representing the shape model 106. As an example, the shape model 106 represents the average shape of anatomical objects (e.g., glenoid fossa, humeral head, etc.) of a patient's anatomical structure from the shapes in the database. Other examples of the shape model 106 could be modes or medians of shapes in the database, etc. A weighted average is another possible example of the shape model 106. The shape model 106 may also be a volume of surfaces or points (e.g., a graphical surface or volume of points defining a 3D model), and the information representing the average shape may be the coordinate values ​​of the vertices of primitives such as triangles that interconnect to form a surface. While the various techniques described herein are explained in relation to shape modes using triangles, it should be understood that shape models may also use primitives other than triangles. Furthermore, it should be understood that other types of modeling may also be used, such as basis spline (BSpline) based models, including non-uniform rational basis spline (NURBS) models.

[0042]

[0076] Using the shape model 106, the processing circuit 102 may represent anatomical structural shape changes by adding points or values ​​of the shape model 106 to the covariance matrix. For example, SSM can be interpreted as a linear equation.

number

[0043]

[0077] In the above equation, s' is the shape model 106 (for example, a point cloud of the average shape, which defines the coordinates of points within the shape model 106, such as the vertices of primitives that make up the shape model 106). In this equation, λ iis an eigenvalue of the covariance matrix, and v i is an eigenvector (also called a variation mode). The covariance matrix represents the variance in a dataset. The element at the i,j position is the covariance between the i-th and j-th elements of the dataset array.

[0044]

[0078] SSM represents constructing the covariance matrix of a database and then performing "singular value decomposition" to extract a matrix of principal vectors (also called eigenvectors) and another diagonal matrix of positive values (called eigenvalues). The eigenvector (v in the formula i ) is the basis of the new coordinate system of the database. The eigenvalue (λ in the formula i ) represents the variance around the eigenvector (v i ). Both the eigenvector and the eigenvalue may reflect the amount of variation around the corresponding axis.

[0045]

[0079] This formula enables the processing circuit 102 to create an infinite number of instances of s i by changing the weight b of the covariance matrix. For example, to generate a new shape model, the processing circuit may determine the value of b j and determine a new value of s i and determine a new value of s i . In the above example, λ i and ν i and s' are all known based on the manner in which s' was generated (e.g., based on the manner in which the shape model 106 was generated). By selecting different values of b i , the processing circuit 102 may determine different instances of the shape model (e.g., different s i which are different variations of the shape model 106). ) may be determined.

[0046]

[0080] The shape model may represent how the anatomical object should appear to the patient. As will be described in more detail, the processing circuit 102 can match points of the shape model of the anatomical object (e.g., in a 3D point cloud) to anatomical points represented in the image data of scan 108 as anatomical points of the anatomical object that are not affected or are minimally affected by injury or disease (e.g., non-pathological points). Based on the match, the processing circuit 102 may determine the pre-pathological characterization of the anatomical object.

[0047]

[0081] As an example, suppose a surgeon wants to determine the characteristics of the glenoid fossa before it becomes pathological for shoulder joint surgery. There is a correlation between the shape of the glenoid fossa and the surrounding bony zones such as the medical articular fornix, acromion, and coracoid. Shape model 106 may be the average shape of the scapula including the glenoid fossa. Assume that in the patient, the glenoid fossa is pathological (e.g., diseased or damaged).

[0048]

[0082] According to the exemplary techniques described herein, the processing circuit 102 can determine the instance of "s" (e.g., a shape model of the scapula with a glenoid fossa) that best matches a non-pathological anatomical structure of a patient's anatomical object (e.g., a non-pathological portion of the scapula in scan 108). The glenoid fossa that best matches a non-pathological anatomical structure (referred to as s* and represented by a point cloud similar to the shape model) can, in the instance of "s", indicate the pre-pathological characterization of the pathological glenoid fossa.

[0049]

[0083] The processing circuit 102 can determine an instance of s* (for example, a scapular shape model having a glenoid fossa that best matches the non-pathological portion of the patient's scapula). The non-pathological portion may be a part of an anatomical object that is minimally or completely unaffected by disease or injury, and the anatomical object and its surrounding anatomical structures are obtained from segmentation performed by scan 108 to segment the anatomical object.

[0050]

[0084] In some cases, anatomical structures beyond the anatomical object itself may be required to align the shape model in order to determine the pre-pathological characterization of a particular anatomical object. For example, the anatomical structure of the scapula may be required to align the shape model in order to determine the pre-pathological characterization of the glenoid fossa. The glenoid fossa of the shape model then represents the pre-pathological characterization of the patient's glenoid fossa. Thus, the shape model does not have to be limited to modeling only the anatomical object of interest (e.g., the glenoid fossa), but may include additional anatomical objects.

[0051]

[0085] The processing circuit 102 may perform the following exemplary operations: (1) first align the segmented anatomical objects from the image data of scan 108 into the coordinate system of shape model 106 to generate an initial aligned shape; (2) compensate for errors in the initial alignment due to under-segmentation and over-segmentation resulting from imperfections in generating scan 108 to generate an aligned shape (also called an intermediate aligned shape); and (3) perform iterative operations, including iterative closest point (ICP) calculations and elastic alignment, to determine an instance of s* (i.e., an instance of s' that most closely matches the anatomical structure of the patient identified by the segmented objects). Exemplary techniques for performing these operations are described in more detail below.

[0052]

[0086] Therefore, there may be several technical challenges in generating pre-pathological characterizations of the patient's anatomical structures. One problem is that, due to insufficient or excessive segmentation, the required image data from scan 108 may be unavailable or distorted, creating challenges in aligning the segmented objects to the shape model 106. Another problem is that, if alignment to the shape model 106 exists, the alignment of the shape model 106 to the segmented objects may be insufficient, resulting in inadequate pre-pathological characterizations.

[0053]

[0087] When encountering a fracture case, the processing circuit 102 may implement different processing techniques than those used for non-fracture cases. Figure 2 shows an exemplary workflow of the processing circuit 102 for a fracture case. In the example in Figure 2, as will be described in more detail below, the processing circuit 102 performs CT scan segmentation (202) on the anatomical structure scan 108, fusion fragment separation (204), original anatomical structure prediction (206), and fractured fragment reduction (208). The processing circuit 102 may also perform predicted shape adjustment based on the reduced fragment, if necessary.

[0054]

[0088] The human body has two types of bone tissue: cortical bone and reticular bone (also known as cancellous bone). Cortical bone typically has a robust, calcified matrix with very little space, forming a protective shell around the cancellous bone tissue. Cortical bone provides rigidity to the bone. Cortical bone may also be called compact bone because the cortical bones appear to be "compressed" together. This compactness makes cortical bone appear very bright on CT images. Higher cortical thickness is found on the diaphysis, where the cortical bone surrounds the medullary cavity, which consists of bone marrow. Towards the joints, the thickness of the cortical shell decreases, and the medullary cavity is replaced by reticular bone. Overall, cancellous bone makes up about 20% of the skeleton of a typical adult. Reticular bone is a highly vascular and porous structure that has the primary function of absorbing, dampening, and transferring the load acting on the joints. Furthermore, reticular bone contains the majority of the body's red bone marrow, which produces blood cells.

[0055]

[0089] The interior of the humeral head is formed by reticular bone surrounded by a thin layer of skin, with the cortical shell barely visible in some areas. As a result, the contrast with surrounding tissues is much lower compared to the diaphysis zone. This is especially true for elderly individuals or patients with osteoporosis, where bone mineral density is reduced.

[0056]

[0090] The processing circuit 102 may be configured to rely on a 3D model of the cortical shell so that each voxel belonging to the cortical bone can be identified in the CT scan. This procedure is called segmentation and can be divided into two parts: diaphysis segmentation and fracture head segmentation. Thresholding may be used for diaphysis segmentation, using high contrast between the diaphysis and soft tissue.

[0057]

[0091] In contrast, segmentation of the humeral head is an extremely difficult task due to the following problems: Firstly, the humeral head generates a weak bone boundary. As mentioned earlier, the cortical shell is very thin at the level of the humeral head, resulting in similar strength to the surrounding tissue on CT images. Secondly, the humeral head generates varying density. The density of long bones changes substantially along the longitudinal axis. As a result, bone and tissue strength histograms frequently overlap. Therefore, segmentation that relies solely on an overall estimate of strength is inaccurate. Thirdly, fusion between fractured fragments and the scapula makes segmentation difficult. Fractures near the joint can include multiple thin-walled bone fragments that are adjacent or in direct contact. Segmenting such data can be extremely difficult, especially when the fractured fragments are fused with the scapula.

[0058]

[0092] Numerous techniques currently exist for segmenting 3D medical images. These techniques can generally be divided into two groups: (1) low-level algorithms and (2) high-level algorithms. The processing circuit 102 can be configured to execute either a low-level or high-level segmentation algorithm. Low-level algorithms operate directly on voxels of a given intensity image. Such algorithms are usually not flexible enough to solve all the problems encountered during bone segmentation. High-level algorithms include additional knowledge, such as shape or appearance, in a mathematical model for determining segmentation.

[0059]

[0093] Thresholding is a fast, low-level segmentation technique widely used for bone segmentation. Thresholding alone may not be sufficient when the intensity of the object and background does not change substantially. Therefore, when segmenting bone segments near joints that have similar intensity to soft tissue (e.g., the proximal humerus), thresholding alone may be insufficient.

[0060]

[0094] Another low-level segmentation technique is region expansion. One common region-based segmentation algorithm is seed region growth, where, starting from a initially defined seed, adjacent voxels are incrementally added to the region based on a user-defined similarity measure, e.g., the strength between adjacent voxels. The main drawback of region-based algorithms for bone segmentation is the tendency to spill over into tissue due to weak boundaries and the connections between bones that are close together.

[0061]

[0095] Another low-level segmentation technique is edge detection based on the first or second derivative of the image intensity function. Such techniques yield good results for datasets with good contrast between diaphysis and soft tissue, but performance may be worse at joint boundaries that introduce discontinuities (e.g., the humeral head).

[0062]

[0096] Another low-level segmentation technique is called the watershed algorithm. The basic idea is to interpret image values ​​as height maps "overflowed" by water from different sources, which may be local minima or user-defined markers in the image. For each source, a catchment area (i.e., a labeled region) is constructed that grows during the continuous overflow. Watersheds, representing the boundaries of the segmented regions, are constructed for adjacent regions to avoid joining. While watershed transformations are effective for separating regions, achieving bone segmentation requires combination with other methods such as thresholding. Thus, watershed transformations share the same problems as thresholding.

[0063]

[0097] Various high-level segmentation techniques utilize statistical models of shape and appearance. For segmentation of anatomical structures, statistical shape models may be fitted to the provided data by deforming the mean shape with respect to trained modes of variation. Thus, the deformation is constrained in a statistically valid direction, resulting in robust segmentation. While statistical models have proven to be a powerful tool for segmentation of healthy bone, these methods are not particularly suitable for traumatic cases due to the large pathological variability caused by the replacement of fractured fragments.

[0064]

[0098] Various high-level segmentation techniques utilize energy minimization along with deformable models. A 3D parametric deformable model can be represented by a parameterized surface of the segmented object. The model is deformed according to external and internal forces, yielding corresponding terms in the overall energy function. Internal energy enhances the surface smoothness while the surface is pulled toward the segmentation target by the external image forces. These models have several limitations. The performance of the algorithms is insufficient for elongated structures with low strength, as well as for highly textured skeletal parts. In addition, the quality of the parametric model initialization is important for the segmentation results, and insufficient initialization can lead to slow convergence or induce the model to have inaccurate features.

[0065]

[0099] Various high-level segmentation techniques utilize energy minimization along with deformable graph cuts. One such technique provides interactive segmentation of n-dimensional images by energy minimization via graph cuts. An interactive approach based on image intensity has been proposed for bone segmentation, where the algorithm generates intensity distributions of objects and backgrounds from user-defined strokes. Such techniques are very time-consuming and may require precise placement of strokes in critical areas where the cortical shell is thin and inconspicuous. Another important issue is optimal parameter selection. Typically, each image has a different set of optimal parameters, and significant user interaction may be required if the parameter settings are far from optimal.

[0066]

[0100] The processing circuit 102 may be configured to implement one or more of the segmentation techniques described above. However, given the shortcomings of these techniques, the processing circuit 102 may also be configured to perform image enhancement using local structure analysis. For example, the processing circuit 102 may be configured to use first and second intensity derivatives to detect local structures in the image. To handle segmentation of cortical bone layers with varying thicknesses, the processing circuit 102 may be configured to use enhancement filters, such as vascular enhancement filters.

[0067]

[0101] The enhancement filter may allow the processing circuit 102 to segment images that have low contrast between objects and background regions. Furthermore, the enhancement filter is also efficient for noisy images. When applying the enhancement filter, the processing circuit 102 performs a local structure analysis around each voxel in the dataset and calculates a measure that can distinguish cortical voxels from other voxels (air, cancellous bone, noise, etc.). The processing circuit 102 can be configured to search for geometric tubular structures. Blood vessels appear at different sizes and can be measured on a measurement scale "s" that varies within a specific range. By analyzing the second derivative information (Hesse) of the Gaussian kernel at scale "s", the processing circuit 102 can measure the contrast between regions inside and outside the range (-s,s) in the direction of the derivative (gradient).

[0068]

[0102] As part of eigenvalue decomposition, the processing circuit 102 may extract three orthonormal directions (eigenvectors) and magnitudes (eigenvalues) that remain invariant up to the scaling factor when mapped by the Hessian matrix. The eigenvalues ​​can be sorted in descending order. A voxel is classified as a tubular structure if |λ1| < |λ2| < |λ3|. λ1 is small (ideally 0), and λ2 and λ3 have high magnitudes and the same sign (the sign is an indicator of brightness / darkness). Therefore, an ideal tubular voxel structure is characterized by the following:

number

number

number

[0069]

[0103] The processing circuit 102 can calculate a dissimilarity measure based on two common ratios. The first ratio explains deviations from blob-like structures but cannot distinguish between tubular and plate-like patterns.

number

[0070]

[0104] The second ratio explains the aspect ratio of the two largest second derivatives. Since only the latter case is zero, this ratio is essential for distinguishing between plate-like and tubular structures.

number

[0071]

[0105] To address the noise appearing in the acquired image, the processing circuit 102 can use Frobenius matrix normalization as follows:

number

[0072]

[0106] This measure may be expected to be low in backgrounds where no structure is present, and the eigenvalues ​​may be expected to be small for the lack of contrast. In regions with high contrast compared to the background, the norm may be large due to at least one of the eigenvalues ​​being large. The final combination of vascular function components is defined as follows:

number

[0073]

[0107] Here, α, β, and c are thresholds that control the sensitivity of the filter to the scales Ra, Rb, and S. The criteria are combined using their product to ensure that the filter response is maximized only when all three criteria are met. The processing circuit 102 analyzes the vascular scale of equation (2.7) at different scales s. The filter response may be maximized at the scale that approximately matches the size of the vessels to be detected.

number

[0074]

[0108] Figures 3A to 3C show the Hessian eigenvector decomposition for tubular, blob, and sheet-like structures. Figure 3A shows the eigenvectors of the central voxel of the tubular structure. Figure 3B shows the eigenvectors of the central voxel of the blob structure. Figure 3C shows the eigenvectors of the central voxel of the sheet structure.

[0075]

[0109] The processing circuit 102 can implement a bone enhancement filter, particularly for scenarios involving the segmentation of bone structures where cortical bone appears as a sheet-like structure. The processing circuit 102 may be configured to use the common ratios Rb, Ra, and S described above, or it may use the slightly modified Rb ratio shown below.

number

[0076]

[0110] This scale is equal to 2 for sheet-like structures, equal to 1 for tubular structures, and 0 for blob structures.

[0077]

[0111] To speed up processing, the processing circuit 102 may define zones to which Hessian segmentation is applied. Specifically, the processing circuit 102 may perform the following four steps, which are described in more detail below: (1) automatic classification of the humerus and scapula (no Hessian calculation at this stage), (2) automatic identification of fracture head zones, (3) automatic identification of other fracture fragment zones, and (4) estimation of optimal Hessian parameters.

[0078]

[0112] For the automatic classification of the humerus and scapula, the processing circuit 102 may be configured to perform image downsampling via an automated algorithm that optimizes image dimensions to reduce processing time. The processing circuit 102 may also be configured to perform coarse segmentation by applying thresholding to the downsampled dataset, where all voxels with HUS greater than 200 are set to 1 and the remaining voxels are set to 0. The processing circuit 102 may be configured to perform initial classification, starting with classification based on binary morphometric features detected after thresholding. Each morphology is then labeled as scapula, humerus, or other. The processing circuit 102 may be configured to perform diaphysis axis calculation using the centroid of the diaphysis morphology on each slice to best fit the axes. The processing circuit 102 may be configured to perform accurate scapula detection. Having accurate scapula segmentation may be important for further virtual surgical planning. During this stage, specific techniques are used to enhance scapula segmentation and classification.

[0079]

[0113] For automatic identification of fractured head zones, the processing circuit 102 may be configured to implement an automatic proximal head segmentation algorithm based on a bone enhancement filter. Since the humeral head is not attached to any muscles, displacement of the humeral head after fracture is not necessarily limited to a specific position or direction. Depending on the type of accident, the force applied, and the immobilization before hospitalization, the humeral head may be found in different positions, some close to its original position, and others far from it.

[0080]

[0114] To define the humeral head segmentation zone, the processing circuit 102 can be configured to run a fully automated algorithm that can identify the center of the fractured head and its radius on a CT scan image. To do this, the processing circuit 102 calculates a gradient vector for all voxels with a HU value greater than 0. The processing circuit 102 then performs non-maximal suppression, setting voxels with a larger neighborhood along the gradient vector to 0. For each non-0 voxel, the processing circuit 102 then constructs a gradient line centered on the voxel center, extending 35 mm in two directions: along the gradient vector and in the opposite direction. The value 35 mm represents the statistically largest humeral head radius, but other values ​​may be used. The processing circuit 102 then calculates the intersection between each gradient line and an image voxel, and if the line intersects a voxel, the counter is incremented by 1. The voxel with the maximum number of intersections can be used to identify the center of the humeral head. The processing circuit 102 may also determine gradient lines that intersect the center of the humeral head and their origins (voxel centers) in order to calculate the humeral head radius. The algorithm may be invariant with respect to translation and rotation, and therefore may be able to detect a damaged humeral head at any location on the CT image.

[0081]

[0115] Figures 4A to 4D show examples of different locations of displaced and fractured humeral heads. Figure 4A shows an example of a humeral head close to its original position. Figure 4B shows an example of a humeral head far from its original position. Figure 4C shows an exemplary axial view of the lesser tubercle with a compressed humeral head. Figure 4D shows an exemplary 3D view of the lesser tubercle with a compressed humeral head. Figures 5A to 5D show the automatic detection of the fractured humeral head and related segmentation zones for the examples in Figures 4A to 4D.

[0082]

[0116] The processing circuit 102 can be configured to perform automatic identification of other damaged fragment zones. The processing circuit 102 may also be configured to perform detection and segmentation of the lesser and greater tubercles. Restoring the anatomical position of the greater tubercle directly affects the surgical outcome. Detection of the greater and lesser tubercles is important because they contain the attachment points of the shoulder muscles. The greater tubercle attaches to the supraspinatus, infraspinatus, and teres minor muscles, while the lesser tubercle attaches to the subscapularis and teres major muscles. Due to this fact, damaged tubercles are often close to their original positions.

[0083]

[0117] To determine the volume of interest, the processing circuit 102 may be configured to use the central axis of the diaphysis calculated as described above. A cylinder is created around this axis and used as the region of interest (ROI) of the fractured nodule. Figures 6A to 6F show examples of identified fractured fragment ROIs.

[0084]

[0118] The processing circuit 102 may be configured to optimize the Hessian parameters described above. As mentioned above, two cost functions (vascular functions) are proposed for analyzing local image structures. One is developed for vascular detection, and the other for bone enhancement. Both of these functions include several parameters that control the sensitivity of the filter. In some examples, the processing circuit 102 may set α and β to 0.5 and c to half the value of the maximum Hessian norm. In other examples, the processing circuit 102 may set c to a range of 60 to 150 to adjust noise suppression.

[0085]

[0119] Another important parameter of the local structure analysis filter is the scale, which represents the size of the anatomical structure to be detected (cortical thickness for bone analysis, and vascular diameter for vascular segmentation). In some cases, a scale ranging from the smallest possible bone structure to the thickest bone structure (0.5 ≤ s ≤ 3.0) can be used. In other cases, such as segmentation of proximal humerus fractures, the sheet scale can be assessed on two scales, 0.75 and 1.0, representing the mean cortical thickness in the proximal portion of the humerus.

[0086]

[0120] The thresholding level applied to the output image after Hessian processing is another parameter for addressing image noise. Another parameter used during the segmentation process is σ of the Gaussian smoothing filter. The processing circuit 102 may smooth the input image using σ before applying the local structure analysis filter. Higher values ​​of σ can significantly blur bone edges and introduce significant errors, while lower values ​​may not be sufficient for significant horizontal noise.

[0087]

[0121] Therefore, there are six different parameters (Gaussian smoothing σ, α, β, c, Hessian scale s, threshold) that may be adjusted to obtain the most likely segmentation of a proximal humerus fracture. Exemplary determinations of these six different parameters are described in more detail elsewhere in this disclosure.

[0088]

[0122] The processing circuit 102 can be configured to determine the predicted humeral shape based on a humeral statistical shape model. SSM has become a widely used tool and is extensively studied for the statistical analysis of 3D shapes, including in medical applications. Such models can derive meaningful statistical conclusions about classes of shapes and can therefore be used as prior information for object segmentation, analysis, and processing tasks. A typical application is model-based segmentation, which is usually used when there is a lack of information, and thus the model provides speculative information about the missing parts. Using SSM, the processing circuit 102 may be configured to separate shape variations into groups that form a basis in shape space.

[0089]

[0123] In the field of orthopedics, SSM is frequently applied to several bones of the human body other than the humerus. According to the techniques of this disclosure, as part of determining the predicted humeral shape based on a humeral statistical shape model, the processing circuit 102 may be configured to perform anatomical landmark detection, including humeral head detection. The processing circuit 102 may be configured to detect the humeral head using the same techniques described above with respect to automatic identification of fracture head zones. For example, the processing circuit 102 may construct an octree from an input mesh, define a line with a vertex and normal for each vertex on the mesh, and increment an internal counter of a voxel by 1 when any line intersects with a structural element (e.g., a voxel) of the octree. Using these techniques, the processing circuit 102 can detect the center of the humeral head as an octree structural element with the maximum number of intersections. The processing circuit 102 may also record vertex IDs belonging to lines intersecting this structural element, corresponding to the humeral head vertices.

[0090]

[0124] Figures 7A to 7C show examples of detected humeral heads. The processing circuit 102 detects the center of the humeral head. However, some outliers may be detected during humeral head apex detection. Figure 7A shows an example of humeral head center detection. Figure 7B shows an example of humeral head apex detection. Figure 7C shows an example of humeral head apex detection.

[0091]

[0125] To remove these outliers, the processing circuit 102 may be configured to remove small, truncated detection regions (e.g., those with a size of fewer than 10 vertices) and to detect the direction from the center of the humeral head (Figure 8C, point 802) to the centroid of the detected vertices (Figure 8C, point 804). The processing circuit 102 may also remove any vertices whose direction is opposite to the humeral head direction. The processing circuit may also apply a morphological closure operation. Figures 8A to 8C show examples of removed humeral head points.

[0092]

[0126] The processing circuit 102 can be configured to detect the humeral shaft axis by first calculating a directed bounding box (OBB). The maximum vector of this OBB may be a first approximation of the shaft axis. The processing circuit 102 may then cut the humerus in a plane perpendicular to the maximum OBB axis.

[0093]

[0127] The processing circuit 102 can use planes 902 and 904 to calculate the start limit point and end limit point using the following equations, as shown in Figures 9A and 9B. startPoint=OBBCorner+0.2×OBBMaxV ecto (3.1) endPoint=OBBCorner+0.8×OBBMaxV ector (3.2) Here, 0.2 and 0.8 represent 20 and 80 percent of the humerus length, respectively. These values ​​were chosen based on statistically average shape, such that the plane passing through these levels never intersects the humeral head and elbow region. Other values ​​may also be used.

[0094]

[0128] Figures 9A to 9C show examples of diaphysis point calculations. Figure 9A shows an example of the humerus OBB and amputation limit for diaphysis axis calculation. Figure 9B shows an example of points for axis calculation and diaphysis axis. Figure 9C shows an example of diaphysis banding points.

[0095]

[0129] For each cut, the processing circuit 102 detects the intersection points between the 3D mesh and the cutting plane and calculates the centroid of these points. The diaphysis axis is calculated based on a line that fits the centroid, as shown by line 906 in Figure 9B.

[0096]

[0130] The processing circuit 102 may also be configured to calculate the diaphysis banding point as the centroid of the intersection between the plane perpendicular to the newly calculated diaphysis axis and the 3D humerus mesh. For this process, the processing circuit 102 expands the start and end limits as follows: startPoint=OBBCorner+0.1×OBBMaxVector (3.3) endPoint=OBBCorner+0.9×OBBMaxV ector (3.4) Here, 0.1 and 0.9 represent 10 and 90 percent of the humeral length, respectively. These values ​​were determined based on statistical analysis of the boundary positions that divide the humerus into three parts: the humeral head, the shaft, and the elbow. Other values ​​may also be used. Figure 9C shows an example of determined banding points that may be used during elastic alignment to match the humeral shaft.

[0097]

[0131] The processing circuit 102 may also be configured to determine the elbow axis and orientation. The processing circuit 102 detects the elbow plane passing through the medial and lateral ribs of the elbow. To detect points located on these ribs, the processing circuit 102 performs the following steps: (1) At the elbow level, the humerus 3D mesh is cut with a plane perpendicular to the diaphysis axis (black dots on Figures 10A and 10B), for each cut, the intersections between the mesh and the intersecting plane are detected, the detected points are ordered to create a closed contour, and for each closed contour, a discrete curve unfolding algorithm is applied to determine the two most important points of the contour (dots on Figures 10A and 10B).

[0098]

[0132] Figures 10A to 10D show examples of diaphysis point calculation. Figure 10A shows an example of cubital column rib detection. Figure 10B shows an example of cubital column rib detection. Figure 10C shows an example of the cubital plane. Figure 10D shows an example of the cubital axis (line 1002).

[0099]

[0133] The processing circuit 102 may be configured to calculate the elbow plane as the best-fitting plane for the detected rib point, as shown in Figure 10C. The elbow axis is calculated as the vector product of the diaphysis axis and the elbow plane normal. An example is shown in Figure 10D, where the elbow plane normal is indicated by line 1004 and the elbow axis is indicated by line 1002. The elbow axis points in the same direction as the humeral head. To ensure the correct elbow axis direction, the processing circuit 102 may be configured to determine the scalar product between the elbow axis and the humeral head direction, as described above with respect to humeral head detection, and to reverse the elbow axis if the product is negative.

[0100]

[0134] The processing circuit 102 may be configured to detect elbow corresponding points by determining the epicondyle axis based on the outermost and innermost points of the elbow. To identify these points, the processing circuit 102 constructs two planes whose normals are equal to the elbow axis direction. The processing circuit 102 obtains the origin by translating the center of the humeral head by, for example, 10 cm along the elbow axis direction (inner plane origin) and the opposite direction (outer origin), as shown by planes 1102 and 1104 in Figure 11. 10 cm is a much larger distance than the elbow size based on statistical analysis, ensuring that the constructed planes do not intersect the elbow.

[0101]

[0135] Figure 11 illustrates the detection of the innermost and outermost elbow points. To find the innermost elbow point, the processing circuit 102 finds the point closest to the inner surface 1104. Similarly, using the outer plane 1102, the processing circuit 102 determines the outermost point. The elbow is divided into two parts (inner and outer) based on the inner point of the epicondyle axis, as shown in Figure 12A.

[0102]

[0136] Figures 12A and 12B show elbow markers. Figure 12A shows a side view, and Figure 12B shows an internal view.

[0103]

[0137] In the inner portion, the processing circuit 102 determines the elbow point furthest from the elbow axis corresponding to the lowest pulley point (point 1202 in Figure 12B). Using this point, the processing circuit 102 determines the pulley zone by defining two planes. The first plane is defined by a normal direction equal to the elbow axis and a pulley point shifted inward, for example, 3 mm along the elbow axis. The value of 3 mm is selected based on the anatomical structure of the elbow, because the inner pulley resembles a plane, and as a result, a small shift includes the pulley portion. The second plane is defined by the same normal and a pulley point shifted laterally, for example, 10 mm along the elbow axis. In this case, the distance is chosen experimentally with the aim of obtaining at least half of the pulley volume. An example of a pulley zone is shown in Figure 13B.

[0104]

[0138] The processing circuit 102 may also be configured to detect the pulley center by applying the above-described pulley head detection algorithm to the detection of the upper arm head. This algorithm is a common method that can detect spherical or circular structures on a 3D image. The pulley center detected using this technique generally corresponds to an intuitive center that a human would choose. The processing circuit 102 may also be configured to use a similar approach for detecting the subcapital marker. The results obtained are shown in Figure 13C.

[0105]

[0139] Figures 13A to 13C illustrate pulley and head marker detection. Figure 13A shows examples of the medial and lateral portions of the elbow. Figure 13B shows an example of pulley detection. Figure 13C shows an example of pulley and head marker internal marker.

[0106]

[0140] The processing circuit 102 may also be configured to detect the cubital rib point by applying the discrete curve expansion algorithm as described above, in which instance the cutting of the humerus is initiated at the elbow axis level. Examples of the obtained elbow marker points are shown in Figures 12A and 12B.

[0107]

[0141] Figures 14A and 14B show examples of biceps groove detection. Figure 14A shows the minimum curvature of the biceps groove, and Figure 14B shows the biceps groove point. The processing circuit 102 may be configured to perform biceps groove detection by performing localization and detection that continues until it reaches the upper end of the humeral head. Firstly, the processing circuit 102 calculates the diaphysis start point (Pstart) as the humeral head center point shifted by two head radii along the diaphysis axis toward the elbow. At this level, the biceps groove shape is concave toward the diaphysis center and can be characterized by a low curvature value (e.g., Figure 14A) and a close distance to the diaphysis center. To localize the biceps groove point, the processing circuit 102 calculates the intersection of the diaphysis and the plane defined by the diaphysis axis and the Pstart point. For each intersection, the cost function is calculated as follows: Kgroovei = Di + 2 × C (3.5)

[0108]

[0142] Here, D is the Euclidean distance from the point to the diaphysis axis, and C is the minimum curvature value at the analyzed point. The processing circuit 102 determines the bicephalic groove point as the point with the minimum cost function. The processing circuit then continues bicephalic groove point detection by shifting the intersecting plane upward along the diaphysis axis. For each intersection, the processing circuit 102 calculates the cost function as shown in equation 3.5 above and identifies the point with the minimum cost function value. At the humeral head level, this algorithm may fail due to several zones with low curvature. To address this problem, the processing circuit 102 may also be configured to use tracking techniques to validate the selected points. An example of detected bicephalic groove points is shown in Figure 14B.

[0109]

[0143] The processing circuit 102 can use these points as references for detecting upper diaphysis markers. Starting from the center level of the humeral head, the processing circuit 102 may calculate the intersection between the humeral mesh and a plane perpendicular to the diaphysis axis. For each intersection, the processing circuit 102 can apply a discrete curve unfolding algorithm to detect the highest point. The processing circuit 102 can select the two points closest to the previously calculated bipectin groove points as upper diaphysis markers. As the intersection plane is moved downward along the diaphysis axis, the processing circuit 102 repeats the previous steps to calculate the upper diaphysis markers. The algorithm may stop after reaching a distance equal to, for example, two humeral head diameters (D). Examples of detected markers are shown in Figures 15A to 15D for four different humeruses.

[0110]

[0144] If the intersecting plane is at a distance D, the processing circuit 102 can implement a difference algorithm. The concave ridge descending from the lesser tubercle disappears at approximately this level (two humeral head diameters from the center of the humeral head). To detect points on the ridge descending from the greater tubercle, the processing circuit 102 applies a discrete curve expansion algorithm and then uses a tracking algorithm that can track points on the greater tubercle ridge based on previously detected markers.

[0111]

[0145] Posterior tilt of the humerus varies considerably across populations. To establish the correct correspondence between the humeral heads of two humers, the processing circuit 102 may be configured to divide the humerus into several specific zones to establish inter-zone correspondence. The processing circuit 102 divides the proximal portion of the humerus into three zones: the humeral head, the greater tubercle, and the lesser tubercle. To do so, the processing circuit defines two planes: (1) the humeral head plane - the best-fitting plane (in the least squares sense) to the previously detected humeral head point, passing through the articular center; and (2) the tubercle plane - the plane defined by the line (in the least squares sense) fitted to the previously detected proximal biceps groove point, and the articular center. The processing circuit 102 uses these planes to cut out the humeral head and separate the two tubercles. Examples of the obtained results are shown in Figures 16A and 16B, which illustrate examples of the upper humeral zone for correspondence establishment.

[0112]

[0146] To find a correspondence between two surfaces, the processing circuit 102 may be configured to minimize the distance between them. In this case, the correspondence is based on the minimum matching distance. To perform distance minimization between two meshes, the processing circuit 102 may first perform uniform scaling, which is a method of making meshes the same size by equalizing the longest vector of the directed bounding box. Secondly, the processing circuit 102 may perform rigid body alignment, which is a method of minimizing the distance between two surfaces overall by translation and rotation. More specifically, three marker points are automatically detected on each humerus. The two surfaces are aligned in a least-squares sense by calculating the best-fit mapping between the two sets of markers. Thirdly, the processing circuit may perform non-rigid body alignment, which is a technique for local distance minimization by surface deformation. For example, the processing circuit 102 may implement a hierarchical B-spline alignment technique that locally approaches the two surfaces using surface marker points. Based on the techniques described above for anatomical landmark detection, up to 85 landmarks can be automatically detected on the surface of the humerus (see, for example, Figures 17B and 17C) and within the surface (see, for example, Figure 17D). In addition to these points, the processing circuit 102 can establish inter-zone correspondences between previously defined proximal humeral zones (humeral head, greater tubercle, and lesser tubercle) that can be used to detect landmark points on these zones. This number of landmark points may be used for sufficient and reliable surface deformation, but other numbers of landmarks may also be used.

[0113]

[0147] Figures 17A to 17D show examples of markers for rigid body alignment and non-rigid body alignment. Figure 17A shows an example of a marker for rigid body alignment. Figure 17B shows an example of a surface marker for non-rigid body alignment. Figure 17C shows an example of a surface marker for non-rigid body alignment. Figure 17D shows an example of a non-surface marker for non-rigid body alignment.

[0114]

[0148] Once B-spline alignment is applied and the two meshes are as close as possible, the processing circuit 102 can find point-to-point correspondences using a minimum distance criterion. At this stage, the processing circuit 102 can calculate the average shape and apply a principal component analysis (PCA) algorithm to determine the major variation modes.

[0115]

[0149] Figure 18A shows the first major mode of variation as a scale factor for average shape. Humeral length increases when the mode decreases, and conversely, humeral length decreases when the mode increases.

[0116]

[0150] Figure 18B shows the effect of the second major mode on the average shape. This mode represents a variation consisting of uniform expansion when the mode value decreases and uniform contraction when the value increases.

[0117]

[0151] Figure 18C shows the effect of the third mode on the average shape. This mode can be observed to have a high influence on diaphysis curvature and humeral varus / valgus in the frontal plane. This third mode also generates corrected positions of the olecapitellum and medial epicondyle.

[0118]

[0152] Figure 18D shows the variation of the fourth mode with respect to the average shape. It can be seen that correcting the mode value changes the diaphysis curvature in the sagittal plane. The fourth mode also has a slight effect on the position of the humeral head and elbow.

[0119]

[0153] Figure 18E shows the effect of the fifth mode on the average shape. It can be observed that the mode controls the posterior tilt of the humeral head by rotating the humeral head in one direction and the epicondylar axis in the other. The fifth mode rotates the positions of the lesser and greater tubercles in the same direction as the humeral head.

[0120]

[0154] Figure 18F shows the variation of the sixth mode relative to the average shape. Decreasing the mode value only causes contraction of the humeral shaft, and consequently, increasing the mode value is observed to expand the shaft. This mode is somewhat similar to the second mode, the difference being the intensity of its effect on different bone zones. The current mode has a high effect on the shaft and a very light effect on the head and elbow, while the second mode has a uniform effect on the entire humerus.

[0121]

[0155] Figure 18G shows the effects of the seventh mode, with the main variations being humeroclavicular / eversion changes and tubercle and microcephalic displacements.

[0122]

[0156] Figure 18H shows the variation in the eighth mode. This mode significantly alters the position and shape of the capitellum, as well as the position, shape, and orientation of the trochlea of ​​the humerus. Furthermore, this variation in the eighth mode affects the positions of the lateral and medial epicondyles. The eighth mode also strongly affects the shape of the proximal humerus, more specifically, both tubercles. When the shape parameters decrease, the distance between the tubercles increases, and vice versa. Changes in the position of the tubercles also affect the shape of the biceps groove.

[0123]

[0157] The above example illustrates the influence of the first eight major variation modes on the average shape. By minimizing the squared distance between correspondences, the processing circuit 102 can minimize the compactness of the SSM. However, even with the compactness value minimized, the eighth mode still has a significant impact on the average shape. Therefore, to accurately represent all humeral variations, the processing circuit 102 may be configured to use more than eight modes.

[0124]

[0158] To determine the proximal humeral point / zone that correlates with posterior humeral tilt, the processing circuit 102 may be configured as follows: (1) Create N humeral instances Sn based on the humeral SSM, and (2) For each Sn, set the central axis of the diaphysis (A diaphysis ) and epicondyle axis (A transepicondylar) and retroversion angle (theta n ) calculate (3) the Sn point pi on the 3D surface of each humerus A diaphysis Project onto the axis, (4) for each projection point, vector V i =p i -p projected Determine I, (5) each vector V i Regarding the angle αi = ∠(Vi,A transepicondylar Calculate ).

[0125]

[0159] After processing all N generated humeruses, the processing circuit 102 calculates the angle value α for each humerus. i The angle α was determined, and then linear regression analysis was applied to calculate the humeral posterior tilt angle and the angle α for all humeral vertices. i Any potential correlation between them can be determined.

[0126]

[0160] The results reveal several zones on the proximal humerus that strongly correlate with posterior tilt of the humerus. These zones, which can be used as reliable landmarks, include the biceps groove, the lesser tubercle, the edge of the intertubercular groove, and the anterior surface of the proximal diaphysis of the humerus.

[0127]

[0161] The processing circuit 102 may also be configured to determine the correlation of articular surface diameters. Based on the regression analysis, the processing circuit 102 can determine the equation of the regression line as follows: y = 0.825 × x - 0.37 (3.6) Here, x is the articular surface radius and y is the articular surface thickness.

[0128]

[0162] It has been determined that humerus length correlates with posterior tilt / slope variation. Figures 19A and 19B show the effect of humerus length on posterior tilt and slope variation, where smaller humerus lengths correspond to higher posterior tilt and slope variation, and longer humerus length values ​​correspond to decreased posterior tilt and slope variation. Therefore, the processing circuit 102 can be configured to take object scale into account using SSM.

[0129]

[0163] Using the tools described above, the processing circuit 102 may be configured to perform fracture fragment processing. With respect to proximal humerus fractures, CT scan images often show several of the fractured fragments fused together. Generally there are three types of fragment fusion: (1) Class 1 fusion - bone that is not fractured but is just bent (e.g., Figure 20A); Class 2 fusion - physical displacement of fractured fragments hitting another fragment (e.g., Figure 20B); Class 3 fusion - error during auto-segmentation caused by narrow interosseous spacing (e.g., Figure 20C).

[0130]

[0164] Figures 20A to 20C illustrate different processes that cause the fusion of fractured fragments. Figure 20A shows a fragment where the fragment is bent relative to the diaphysis. Figure 20B shows a fragment that has moved until two fractured parts collide. Figure 20C shows a fragment where fusion has occurred between fractured fragments caused by segmentation errors.

[0131]

[0165] To manipulate the damaged fragments and restore the original patient's anatomical structure, the processing circuit 102 may be configured to separate all fused fragments. It has been found that any fracture line contains regions with high absolute minimum curvature values. The minimum curvature changes significantly along the fracture line and often has regions with very low absolute curvature values. Furthermore, it has been observed that the proximal humerus has many intact bone regions with moderate absolute minimum curvature values. These conditions make it difficult to separate fused fragments using thresholding alone. To solve the separation problem, the processing circuit 102 may be configured to perform step segmentation.

[0132]

[0166] The idea behind the step segmentation algorithm is to propagate the segmentation process gradually. Between each step, the segmentation process starts in a specific region of interest that guides the overall segmentation. Figures 21A to 21F illustrate the principle of the algorithm and show three steps of the proposed idea applied to the proximal humerus fracture. Figure 21A shows the minimum curvature. Figure 21B shows region initialization by curvature thresholding. Figure 21C shows the next step propagation point. Figure 21D shows the next propagation point, which is recalculated after the first propagation step. Figure 21E shows the result after the second propagation step, and Figure 21F shows the skeletal region obtained after four iterations.

[0133]

[0167] The minimum surface curvature is selected as the fracture line descriptor (Figure 21A). It is possible to find a region with a "high" absolute minimum curvature value along any fracture line. This value is problem-specific and must be chosen experimentally. A value of "0.3" can be considered a "high" absolute minimum curvature value. Therefore, to initialize the region for the step segmentation algorithm, the processing circuit 102 may be configured to select all vertices with a minimum curvature value less than -0.3 (Figure 21B).

[0134]

[0168] The next step is propagation source point detection. The processing circuit 102 is configured to detect the boundary contour of a previously defined region and apply a discrete curve expansion algorithm (Figure 21C). The processing circuit 102 then uses each source point as the center of wave propagation based on a fast marching method. The processing circuit 102 uses the absolute minimum curvature value as the velocity function of wave propagation, which means faster propagation in regions with high curvature and, consequently, slower propagation in regions with low curvature values. The propagation distance is limited to a specific value that allows for propagation in the wrong direction caused by image noise. The results obtained after the first propagation step are shown in Figure 21D.

[0135]

[0169] When the propagation process is stopped, the processing circuit 102 combines the acquired points with the initial region to create an input for the next iteration. Thus, for the new region, the processing circuit 102 recalculates the boundary contour and source points (some, but not all, are labeled as points 2110A to 2110D in Figure 21D) and reapplies the fast marching method for further propagation. This is an iterative algorithm where the number of iterations is the problem-defined value.

[0136]

[0170] An example of the propagation region after two iterations is shown in Figure 21E. For the separation of the proximal humerus fracture fragment, the processing circuit 102 may be configured to use, for example, four iterations. Figure 21F shows the resulting dashed line, which is the skeleton of the final region.

[0137]

[0171] The processing circuit 102 may be configured to perform step propagation marker detection. One "key to success" of the step segmentation algorithm is accurate and robust step propagation marker detection. To accurately detect these points, the processing circuit 102 may be configured to define which points are important to the propagation process. Figures 22A and 22B show 3D images of a proximal humerus fracture in which a surgeon has manually selected all vertices belonging to the fracture line. It can be observed that the selected vertices form an elongated line which may form a closed cycle.

[0138]

[0172] As described above, the minimum surface curvature is selected as the fracture line descriptor (see, for example, Figure 21A). It is possible to find a region along any fracture line that has a “high” absolute minimum curvature value. The term “high” may be a problem-specific value that can be defined experimentally. Such a region follows the fracture line and therefore takes an elongated shape. To detect the remainder of the fracture line, the processing circuit 102 may be configured to propagate waves from the protruding (end) points of the defined region. Figure 22A shows an example of curvature along the humeral head, where the top of the scale represents minimum curvature and the bottom represents higher curvature. Point 2202 in Figure 22B corresponds to an example of a potentially optimal point for the propagation process to detect the fracture line.

[0139]

[0173] The processing circuit 102 may also execute a discrete curve expansion (DCE) algorithm to compute the aforementioned points. The DCE algorithm serves to eliminate object contour distortion while preserving a sufficient level of perceptual appearance for object recognition. Essentially, this technique eliminates less important contour vertices (often vertices affected by noise) while retaining vertices important for contour description. This technique is widely used for shape similarity scales, skeletal pruning, and protrusion detection of binary objects on a typical 2D grid.

[0140]

[0174] The application of the DCE algorithm to a simple 2D fish shape is shown in Figures 23A and 23B, where the shape contour is disturbed with noise. Figure 23B shows the obtained results, with the most important points of the contour indicated in blue. After observing these results, it can be concluded that this algorithm can be used to detect step propagation points.

[0141]

[0175] Next, we introduce the basic concept of DCE technology. For each evolutionary step, the contour point with the smallest relevance scale is removed, and two adjacent points are connected to form a contour line segment. The relevance scale K (Equation 4.1) can be determined based on the Euclidean distance and angle as follows.

number

number

[0142]

[0176] Figures 24A and 24B show two types of contour topologies. Figure 24A shows a spike, and Figure 24B shows a cavity. The processing circuit 102 may also be configured to perform step propagation marker detection. The processing circuit 102 can be configured to implement a new cost function that can be easily computed on a triangulated surface and yields similar results on a normal 2D grid compared to conventional DCE. To apply the DCE algorithm to contours formed by vertices on a triangulated surface, the processing circuit may be configured to implement the same techniques as for contours on a normal 2D grid. The difference lies in a new relevance measure that can be computed on a 3D surface. The vertex relevance measure is directly proportional to the sum of the segment lengths connected to the analyzed vertices and inversely proportional to the distances between unconnected vertices of these segments.

number

[0143]

[0177] To find the distance c, the processing circuit 102 may be configured to search for the shortest path between contour vertices only within the interior of the object defining the contour under consideration (e.g., Figure 24A). When vertices are found within a concave region, the distance c is equal to the sum of the segments forming the cavity (e.g., Figure 24B). c = a + b (4.3)

[0144]

[0178] The final condition is that we do not distinguish between "concave" vertices. For example, consider the following two different concave regions. 1. a=10 cm, b=20 cm, c=30 cm 2.a=1cm, b=0.5cm, c=1.5cm

[0145]

[0179] The relevance scale (Equation 4.2) may be, for example, 1 for both cases, but in the first case the segment is larger and therefore more important. To distinguish between various concave segments, the processing circuit 102 can be configured to determine a size factor.

number

[0146]

[0180] In the above example, the size factor is equal to 4.48 in the first case and 1.22 in the second case. Therefore, the processing circuit 102 can distinguish such concave regions. This relevance measure can be applied to any triangulated mesh and is equal to the cost function of equation 4.2 multiplied by the size factor.

number

[0147]

[0181] As mentioned above, fracture lines can be characterized by minimum curvature. To separate fused bone fragments, the processing circuit 102 can be configured to use minimum curvature as a surface descriptor. The minimum curvature value varies significantly along the fracture line, and this variation depends on the angle between fused fragments, DICOM image quality, and subsequent segmentation. For almost all fracture lines, one or more zones with low minimum curvature values ​​are identifiable. The term "low minimum curvature" is application-dependent and can be defined for each specific problem. These "low curvature" zones form elongated shapes and follow the fracture line. Other humeral zones where the minimum curvature value varies between "low" and 0 may belong to the fracture line, belong to the biceps groove, or are simply parts of an intact bone surface.

[0148]

[0182] The processing circuit 102 may be configured to implement the following segmentation algorithm for step segmentation. This algorithm is an iterative process in which a wave propagates in a specific direction, enabling the segmentation of zones with very weak intensity during each iteration (step). The main steps of this algorithm are as follows: 1. Initialize the area: · Each vertex V on mesh M i Regarding the minimum curvature value

number

number

number

number

number

[0149]

[0183] The segmentation process progresses by repeating steps 2 through 7. The number of iterations is an important parameter and may depend on the application. Increasing the number of iterations typically increases the number of regions detected, but these regions do not necessarily belong to fracture lines. A large number of iterations may detect many noisy regions that are undesirable for the segmentation results.

[0150]

[0184] The number of iterations may be a value specific to the problem. For the fragment separation problem, the processing circuit 102 can be configured to perform three iterations to achieve the desired result, but other numbers of iterations can also be used.

[0151]

[0185] The processing circuit 102 may also be configured to perform fracture line detection. The processing circuit 102 may calculate the fracture line of the fractured proximal humerus by applying skeletal extraction to the detected region using step segmentation as described above. The problem of skeletal extraction for a set of vertices on an arbitrary triangulated 3D mesh is not a trivial task. One solution is a method in which a sequence of mathematical morphological operations is applied to a set of vertices on a triangulated surface. The technique described is efficient and easy to implement, but has several drawbacks that can lead to truncated skeletal lines. To overcome this problem, the processing circuit 102 may be configured to include the definition of additional vertex categories to improve the thinning process by integrating priorities between vertex classes. This method has been tested for meshes (homogeneous and heterogeneous) and vertex set configurations of different categories. The results obtained illustrate the high accuracy and efficiency of the approach.

[0152]

[0186] As described above, with respect to proximal humerus fractures, CT scan images often show some of the fractured fragments fused together. To address this challenge, the processing circuit 102 may be configured to characterize the fracture line based on minimum curvature. However, minimum curvature varies significantly along the fracture line, and it is often observed that regions with low absolute curvature values ​​are present along the fracture line. Furthermore, the proximal humerus has many intact bone regions with moderate absolute minimum curvature values. Therefore, the processing circuit 102 may be configured to implement the step segmentation algorithm described above. The principle of the step segmentation algorithm is a progressive propagation process. During each iteration, the segmentation process begins at a region of interest that guides the overall segmentation / propagation. Such a region of interest is called a “source point” and is computed using a discrete curve expansion algorithm.

[0153]

[0187] Using the tools introduced above, the processing circuit 102 may be configured to determine a procedure for fracture reduction. Fracture reduction is a surgical procedure for repairing a fracture or dislocation into correct alignment. In this context, the term "reduction" does not necessarily mean any kind of removal or quantitative decrease, but rather means restoration, or "returning to normal". In a fracture, the fragments lose their alignment with respect to displacement and / or angulation. In order for the fractured bone to heal without deformation, the bone fragments must be realigned to their normal anatomical position. Orthopedic surgery attempts to restore the normal anatomical structure or function of the fractured bone by reducing the displacement.

[0154]

[0188] There are several techniques for proximal humerus fracture management, including non-operative treatment, closed reduction, open reduction and internal fixation, hemiarthroplasty, and reverse total shoulder arthroplasty. For all of these techniques except non-operative treatment, the original anatomical structure (prior to fracture) is important information that the surgeon should have in order to obtain satisfactory results.

[0155]

[0189] Techniques for reducing proximal humerus fractures and finding the original anatomical structure include manual fragment reduction, a tactile-based approach for proximal fracture reduction, fragment reduction based on the intact contralateral proximal humerus shape, and fragment reduction based on a statistical anatomical atlas model.

[0156]

[0190] The processing circuit 102 may be configured to perform humerus SSM fitting onto the damaged bone fragments. The purpose of the coarse alignment is to align the available damaged fragments to a level where their boundary similarities can be captured, thereby enabling fine alignment based on the ICP algorithm. The processing circuit 102 may be configured to perform coarse humerus proximal fragment alignment by using the humerus SSM, and the SSM acts as a template for the damaged fragment placement. However, as described above, the humerus shape has significant variations across the population, and thus, a simple average shape cannot be used as a reliable template. The processing circuit 102 may be configured to implement a step algorithm, and for each step, the SSM incorporates additional information about the fracture. This improves the approximation of the SSM shape to the patient's native humerus proximal shape.

[0157]

[0191] The processing circuit 102 may be configured to perform diaphysis detection and removal. FIGS. 26A to 26D show the process for diaphysis detection and characterization. FIG. 26A shows an example of a diaphysis oriented bounding box (OBB). FIG. 26B shows an example of diaphysis damaged part removal. FIG. 26C shows an example of a diaphysis central axis point. FIG. 26D shows an example of a diaphysis without damaged part central axis calculation.

[0158]

[0192] The processing circuit 102 starts the reduction process by determining the diaphysis fragment from the damaged fragments. To do so, the processing circuit 102 calculates an oriented bounding box (OBB) for the entire damaged humerus and determines the corner closest to and the corner farthest from the humerus rotation center based on the Euclidean distance (points 2601 and 2602 in FIG. 26A). By iterating over the detected fragments, the processing circuit 102 determines the diaphysis as the fragment having the vertex closest to the OBB farthest point.

[0159]

[0193] The processing circuit 102 is also configured to perform diaphysis characterization. The processing circuit 102 characterizes the diaphysis using a central axis that always points toward the humeral head. As will be described in more detail below, the accurate calculation of this vector requires a diaphysis portion without fracture contours. Therefore, the procedure for robust fracture diaphysis central axis calculation consists of two steps: (1) fracture contour removal and (2) diaphysis central axis calculation.

[0160]

[0194] The processing circuit 102 may be configured to perform fracture contour removal, during which the fractured diaphysis is excised. Very often, complex proximal humerus fractures occur at the metaphysis level. To detect the metaphysis level, a method of detecting the difference in diaphysis radius may be used. To do so, the processing circuit 102 calculates the OBB around the fractured diaphysis fragment and cuts this diaphysis mesh by a plane perpendicular to the longest OBB vector. The processing circuit 102 starts from the furthest OBB point and moves toward the nearest point (see, for example, Figure 26B). For each cut, the processing circuit 102 determines the radius of the fitted circle and compares that radius to the previous cut. If the radius value increases by 10% or more, the metaphysis level has been achieved, and the processing circuit 102 stops the cutting process.

[0161]

[0195] The processing circuit 102 performs the diaphysis central axis calculation in two steps. First, the processing circuit 102 estimates the initial central axis by cutting the diaphysis without broken edges with a plane perpendicular to the longest OBB vector axis. For each cut, the processing circuit 102 calculates the centroid of the intersection point between the cutting plane and the diaphysis mesh. The processing circuit uses the obtained centroid to fit the line, which is the first diaphysis central axis estimate, in the least mean squares sense (Figure 26D).

[0162]

[0196] The processing circuit 102 calculates the final central axis by reapplying the previous procedure, but this time, the processing circuit 102 cuts the diaphysis with a plane perpendicular to the previously estimated central axis.

[0163]

[0197] In summary, the fractured diaphysis is cut by a plane perpendicular to the longest OBB axis, and for each cut, the centroid is calculated. To determine the central axis, the processing circuit 102 fits a line to the detected centroid in a least-squares sense. As can be seen in Figure 26C, the center point at the fracture contour level deviates from the central axis, which is why the processing circuit 102 may be configured to remove the portion of the diaphysis surface higher than the metaphysis level.

[0164]

[0198] The processing circuit 102 may be configured to perform internal surface removal. For accurate fracture fragment reconstruction, during CT scan segmentation, the processing circuit 102 may be configured to detect both the external and internal surfaces of the cortical bone. For fracture fragment reduction and prediction of the original humeral shape, the processing circuit 102 may require only the external cortical shape. To detect the internal cortical bone, each diaphysis triangle is tested for its normal orientation. This test is performed in three steps: (1) projecting the centroid of the triangle onto the diaphysis axis, (2) determining the direction to the diaphysis axis as a vector from the centroid of the triangle to its projection, and (3) marking the triangle as part of the internal cortical surface if the dot product of the triangle's normal and the direction to the axis is positive. To extract only the external cortical shape, all triangles marked as internal surfaces may be removed.

[0165]

[0199] The processing circuit 102 may be configured to perform a rough fit. As described above, humeral length and diaphysis diameter have significant variations across populations, but can still be modeled using a humeral SSM. Each humeral variation (length, posterior tilt, inclination, etc.) can be controlled by a specific dominant mode of variation. Thus, the processing circuit 102 can initialize the shape of the SSM by fitting the proximal diaphysis radius to the radius of the fractured diaphysis fragment.

[0166]

[0200] The processing circuit 102 can perform the following: (1) determine the extraction of the proximal diaphysis (without the metaphysis) as described in the previous section; (2) determine the extraction surface section using a plane perpendicular to the central axis of the diaphysis; (3) fit a circle to the plane-surface intersection for each section; and (4) determine the proximal diaphysis radius as the average value of the fitted radii.

[0167]

[0201] The first and second variation modes have a significant effect on the proximal diaphysis radius. The processing circuit 102 may be configured to apply a nonlinear simplex optimization that minimizes the difference between the humeral SSM diaphysis radius and the fractured diaphysis radius by varying the first two major variation modes. Such an optimization estimates the original shape diaphysis radius, which is important for further posture improvement. This is the first basic approximation of the shape of the original bone. Since the humeral SSM diaphysis radius is approximately equal to that of the fractured one, the processing circuit 102 can approximately detect the metaphyseal zone at the same level for both diaphysis. The processing circuit 102 calculates a rigid body transformation between the fractured diaphysis and the humeral SSM using a point on the diaphysis-central axis at the metaphyseal level and the axis itself. The resulting transformation allows for the initialization of the posture of the fractured diaphysis relative to the humeral SSM.

[0168]

[0202] The processing circuit 102 can be configured to translate and rotate the fractured diaphysis along and around the central axis. Based on the initialization described above, the processing circuit 102 obtains the position of the fractured diaphysis on the humerus SSM. This position is a first approximation of the actual position and therefore needs to be improved. There are two parameters that can be adjusted to determine the precise position of the fractured diaphysis: vertical translation along the central axis and rotation around the central axis.

[0169]

[0203] The aforementioned initialization is an accurate technique for determining the height of a fractured diaphysis. However, this initialization can introduce errors, and therefore, existing techniques can benefit from an improved determination of the height of a fractured diaphysis. This disclosure introduces an algorithm that potentially improves the determination of the height of a fractured diaphysis. The algorithm involves translating a fractured diaphysis fragment vertically along its central axis. For each translation, the processing circuit 102 calculates the RMS distance between the diaphysis fragment and the humerus SSM. The processing circuit 102 searches for the translation that minimizes the RMS distance.

[0170]

[0204] After initial alignment and vertical translation, the processing circuit 102 obtains a very accurate vertical positioning of the fractured diaphysis, but knows nothing about its orientation around its central axis. Often, the diaphysis is modeled as a cylinder, making it impossible to find the correct orientation. Fortunately, the proximal diaphysis includes the biceps groove and two ridges descending from the greater and lesser tubercles. Such shape deformation allows the processing circuit 102 to determine the correct orientation of the fractured diaphysis relative to the humeral SSM. The algorithm is based on the rotation of the fractured diaphysis around its central axis. For each turn, the RMS distance between the fractured diaphysis and the SSM is calculated. The best orientation corresponds to the minimum RMS distance.

[0171]

[0205] Figures 27A to 27D show the first example of initial alignment of the fractured diaphysis with the humeral SSM, and Figures 27E to 27H show the second example of initial alignment of the fractured diaphysis with the humeral SSM. Figures 27A to 27D show a patient with a proximal diaphysis radius smaller than the diaphysis radius of the SSM. Figure 27D shows the simplex optimization output, where the SSM size is reduced. The initial translation and orientation were calculated very accurately. Therefore, translation and rotation optimization adjusted the initial position by only 2 mm and 3 degrees, respectively.

[0172]

[0206] Figures 27E to 27H show a patient with a diaphysis radius similar to that of a humerus SSM. Simplex optimization hardly alters the size of the humerus. The initial vertical alignment was accurately estimated, and the vertical adjustment procedure did not change the height of the fractured diaphysis. However, Figure 27G shows an incorrect orientation of the fractured diaphysis. The optimization process rotates the fractured diaphysis by 68 degrees, and Figure 27H shows the accurately oriented fractured diaphysis.

[0173]

[0207] The processing circuit 102 is configured to fit the humeral SSM as well as possible onto the available portion of the fractured diaphysis. The processing circuit 102 is configured to perform shape optimization. More specifically, the processing circuit 102 applies a simplex minimization algorithm that attempts to find the combination of major variation modes that minimizes the RMS distance between the fractured diaphysis and the SSM.

[0174]

[0208] Figures 28A to 28C show examples of fractured humerus shape estimation based on diaphysis. Figures 28A to 28C show the application of the aforementioned algorithm to three different patients. Visual observation of the results indicates that the estimated model shape is very close to the available diaphysis. The results also show that the algorithm can handle diaphysis of different lengths with the same accuracy. Examples of humerus shape estimation with the long portion of the available diaphysis are shown in Figures 28A and 28B. Figure 28C shows an example of predicted humerus shape based on the short portion of the diaphysis.

[0175]

[0209] The humeral head has one anatomical and one geometrical property that enables and / or requires the creation of a specific algorithm for its reduction. Firstly, the humeral head is the only fragment that is not attached to any muscle. This anatomical property allows for displacement of the humeral head over any direction and over long distances in cases of proximal humeral fractures. Secondly, the surface of the humeral head can be accurately approximated as a spherical shape. This geometrical property allows the processing circuit 102 to calculate the humeral head center and align that humeral head center with the humeral SSM head center.

[0176]

[0210] Using the geometric properties of the spherical humeral head, the processing circuit 102 may implement the following algorithm to align the damaged humeral head with the humerus SSM atlas. · Damaged humeral head detection: - Determine the center C of the damaged humeral head broken as the voxel intersected by the maximum number of surface normals as described above; - Determine the humeral head from the damaged fragments Fj, j ∈ M, where M is the set of fragments · Number of detected damaged fragments; · For each fragment Fj, fit a sphere in the least squares sense and store the sphere center Cj; For each Cj, calculate the Euclidean distance Dj = dist(Cj, Cbroken); - The minimum distance Dmin = minj∈M{Dj} corresponds to the damaged humeral head fragment FBrokenHead ← Fj, where j is the index corresponding to the distance Dmin. CBrokenHead corresponds to the detected center of the damaged head; · Humerus SSM head center detection: - Manually selected humeral head vertex IDs on the average shape enable the locations of those on the humerus SSM for any dominant mode configuration to be known; - Fitting a sphere to these points enables the humerus SSM head center Chssm to be calculated. · Alignment of the damaged humeral head center: - Define the transformation T brokenHead as the translation vector V tr = C hssm - C BrokenHead ; - Apply the transformation T BrokenHead to the damaged head fragment: F BrokenHeadAtHSSM ← Tr(F BrokenHead , T BrokenHead ); · Restoration of humeral head fracture: -After the previous cephalic alignment, the orientation of the fractured humeral head may be inaccurate. To find the correct orientation, the processing circuit 102 applies a simplex optimization algorithm. By averaging the rotations of the fractured humeral head around its center of rotation, the optimization process minimizes the RMS distance between the fractured humeral head and the humeral SSM surface.

[0177]

[0211] Examples of humeral head reduction are shown in Figures 31A-31C and 32A-32C. The fractured humeral head fragment perfectly conforms to the predicted humeral shape. The obtained visual results demonstrate the correct behavior of the fractured head reduction algorithm described.

[0178]

[0212] Proximal humerus fractures are ranked as the fourth most common fracture worldwide in osteoporosis (after hip, spine, and radius fractures). They account for approximately 8% of all fractures worldwide and are associated with significant costs. To treat a fractured shoulder, surgeons must restore its shape and / or function. Such operations are highly complex and success cannot be guaranteed. Readmissions are common after proximal humerus fractures, and the costs of managing these readmissions are typically higher than those of the initial hospitalization. The main reason for unsatisfactory outcomes is the unknown inherent humeral shape caused by the displaced fragment.

[0179]

[0213] An overview of the technology described above is provided here. Processing circuit 102 performs CT scan segmentation. The core of the proposed segmentation is a multiscale Hessé-based bone enhancement filter. This filter is the most suitable solution in the case of proximal humerus fractures, but it has several drawbacks, including speed, excessive memory usage, and application-specific parameter dependence.

[0180]

[0214] To overcome the two initial problems, this disclosure describes a technique for automatically determining the location of the fractured humeral head, diaphysis, and remaining fragments. A region of interest (ROI) is created around the located fragment, and the segmentation process is performed only within the defined ROI.

[0181]

[0215] Numerous possible sets may be tested to determine the application-specific parameter set for the filter. For each test, the results obtained are compared to ground truth data. Then, binary classification is used to determine the best parameter set. The procedure described is applied to seven patients. The best parameter set is the one that simultaneously maximizes the segmentation results for all patients.

[0182]

[0216] To validate the proposed segmentation algorithm, a set of 20 CT scans with proximal humerus fractures was used. The RMS distance between the automatically segmented fracture plane and the ground truth plane was selected to quantify the segmentation error. The mean RMS error obtained for all 20 patients was less than 1 mm. These results demonstrate the high accuracy of the proposed segmentation algorithm.

[0183]

[0217] Processing circuit 102 performs fracture fragment processing. In cases of proximal humerus fractures, CT scan images often show some of the fractured fragments fused together. All fused fragments must be separated in order to manipulate the fractured fragments and restore the original patient's anatomical structure. Few attempts have been made in the literature to solve the above problem, and all described methods are either manual or semi-automatic. Separating fused fragments is no trivial task. To be able to handle all possible fusion types, this disclosure describes a novel segmentation algorithm called step segmentation. The idea behind this algorithm is the gradual propagation of the segmentation process. Between each step, a specific region of interest is used as the starting point for wave propagation. Such “local attack” propagation allows the overall segmentation process to be guided toward the desired region.

[0184]

[0218] The fragment separation process acts on the fractured bone surface. To identify the “region of interest” for guided propagation, the processing circuit 102 executes a “discrete curve unfolding on an arbitrary triangulated 3D mesh” algorithm, as described in “Discrete Curve Unfolding on an Arbitrary Triangulated 3D Mesh” by Sergii Poltaretskyi, Jean Chaoui, and Chafiaa Hamitouche-Djabou, in Lecture Notes 8668, pp. 410–421, of “Discrete Geometry for Computer Imagery,” Computer Science, edited by Elena Barcucci, Andrea Frosini, and Simone Rinaldi, Springer International Publishing, September 2014. These methods enable the identification of protruding vertices for any contour defined on the 3D mesh. The techniques described above work on any triangulated mesh. However, the same idea can be applied to 2D segmentation problems.

[0185]

[0219] The processing circuit 102 performs statistical shape modeling of the humerus. Statistical shape models can be considered as inferential information about the modeled shape. These models are widely used in medical applications for purposes such as segmentation, shape prediction, and morphometric analysis. While SSM has been frequently applied to several human bones in the field of orthopedics, there is no comprehensive research on humeral SSM and its applications.

[0186]

[0220] This disclosure describes an automated algorithm for detecting correspondences between two humeruses. These correspondences allow the processing circuit 102 to calculate the SSM by principal component analysis. The automated approach plays a crucial role in improving accuracy by eliminating errors introduced by human experts during manual correspondence establishment.

[0187]

[0221] Using SSM, the processing circuit 102 performs virtual fragment reduction. The aforementioned technique serves as a tool for achieving estimation / prediction of the shape of the proximal humerus fracture.

[0188]

[0222] To successfully reconstruct the fracture shape, two main problems must be solved. The problems identified are low bone density caused by osteoporosis and the comminution effect that often occurs during proximal humerus fractures. The first problem is that not all bone surfaces may be detected, which negatively affects segmentation quality. The second problem makes it impossible to use the fracture contour in the reduction procedure.

[0189]

[0223] In light of these two issues, this disclosure describes a technique for predicting fracture shape based on the humeral SSM. The diaphysis surface can be used as reliable information for posterior tilt, inclination, and humeral HH estimation. To estimate these parameters, processing circuit 102 performs a fit-based procedure. This procedure fits the humeral SSM model to the available diaphysis fragment. The resulting shape is used as an innate humerus prediction.

[0190]

[0224] An exemplary application for fracture reduction using the techniques described above is described here. Device 100 acquires image data of the joint including at least a portion of the humerus and a portion of the scapula. The image data may be 3D image data containing multiple voxels. Device 100 segments the image data to identify portions of the image data corresponding to cortical bone. Device 100 can use, for example, one of the various segmentation techniques described above to identify voxels corresponding to cortical bone in the 3D image data. Segmentation may result in the identification of multiple fragments of the humerus in the image data, as well as the identification of cortical bone corresponding to the scapula. Each fragment in the image data generally corresponds to a group of mostly continuous cortical bone voxels that are partially or completely separated from other groups of mostly continuous cortical bone voxels.

[0191]

[0225] Depending on the type and severity of the fracture, segmented image data may include several fragments, such as the diaphysis, humeral head, lesser tubercle, and / or greater tubercle. Some of these parts of the humerus may still remain together in the 3D image data, i.e., they may not be fractured. For example, in many fractures, the humeral head remains together with the lesser tubercle. Some parts of the humerus may be fractured into multiple fragments. For example, in some fractures, the greater tubercle may be fractured into two, three, four, or sometimes even more fragments.

[0192]

[0226] Device 100 can create a 3D surface model including a scapula and a 3D surface mesh for each segment identified within the segmented image data. Device 100 can, for example, run a decimation algorithm on the segmented image data to determine a 3D mesh for each identified segment. Each 3D surface mesh may be represented by multiple triangles, each having three vertices, where vertices are positions in 3D model space. Device 100 can generate a 3D surface model at the same scale as the 3D image data, such that voxels are mapped to vertices and vice versa. Unlike voxels, vertices may not have any kind of related color information, such as luminance or chrominance values, RGB values, monochrome values, or similar.

[0193]

[0227] Device 100 can be configured to identify a portion of a 3D mesh corresponding to the humeral head in a 3D surface model. In a fractured humerus, the humeral head can move relatively large distances in several different directions compared to the movement of smaller and larger tubercles, which are typically limited by, for example, the portions of the humerus attached to muscles. To identify the humeral head, device 100 may identify inward normal vectors on the surface of the 3D mesh, for example, using the sphere detection algorithm described above. Device 100 may determine the normal vector for each vertex as the average of the normal vectors for all triangles sharing that vertex. Device 100 can then determine the point that intersects most frequently with the normal vectors. Since the humeral head is nearly spherical, the point of most intersections represents the approximate center of the nearly spherical shape corresponding to the humeral head.

[0194]

[0228] Figure 29 shows an example of a 2D slice of a 3D mesh for a fractured humerus. As can be seen in Figure 29, vectors 2902A, 2902B, 2902C, and other unlabeled vectors in Figure 29 are all normal vectors from the surface of fragment 2904. Based on vectors 2902A through 2902C, which all intersect at or near point 2906, device 100 can determine that 3D mesh 2904 is approximately spherical and therefore corresponds to the humeral head. In contrast, vectors 2908A, 2908B, and other vectors (unlabeled in Figure 29) are all normal vectors from the surface of 3D mesh 2910. Since vectors 2908A, 2908B, and other vectors do not have a common intersection point, device 100 can determine that 3D mesh 2910 does not correspond to the humeral head. Although the example in Figure 29 is shown in 2D for illustrative purposes, the device 100 can be configured to determine a normal vector and identify point 2906 in 3D space.

[0195]

[0229] Device 100 can also detect a 3D mesh corresponding to the diaphysis of the humerus. Device 100 may, for example, identify the vertex in the 3D model furthest from the humeral head. This vertex typically corresponds to the distal end of the diaphysis, i.e., toward the elbow. In shoulder trauma, since the fragment does not move to this part of the patient's arm due to the presence of muscles, this furthest vertex may be assumed to correspond to the diaphysis. In this context, device 100 may be configured to determine the vertex furthest from the humeral head using the approximate sphere center identified above as a reference point. That is, device 100 can identify the vertex furthest from the sphere center (e.g., point 2906 in Figure 29) as belonging to the 3D mesh corresponding to the patient's diaphysis.

[0196]

[0230] After identifying the 3D mesh corresponding to the diaphysis, device 100 can be configured to remove the inner surface of the diaphysis from the 3D mesh corresponding to the diaphysis. That is, with respect to the 3D mesh determined to correspond to the diaphysis, device 100 may be configured to identify and remove the vertices corresponding to the inner surface of the diaphysis, and the triangles associated with those vertices.

[0197]

[0231] Figure 30A shows a 2D slice of an exemplary 3D mesh 3000A corresponding to a portion of the diaphysis. The 3D mesh 3000A has a left surface 3002 and a right surface 3012. The left surface 3002 corresponds to the left cortical bone of the 2D slice of the diaphysis of the 3D mesh 3000A and has an outer surface 3004 and an inner surface 3006. The 3D mesh 3000A also has a right surface 3012. The right surface 3012 corresponds to the right cortical bone of the 2D slice of the diaphysis of the 3D mesh 3000A and has an outer surface 3014 and an inner surface 3016. Device 100 can determine the central axis 3022 by oriented a boundary box around the identified diaphysis fragment, and the long axis of the boundary box corresponds to the diaphysis axis, i.e., the central axis 3022. Device 100 can orient the bounding box, for example, by moving the bounding box around the diaphysis fragment to determine the orientation in which the diaphysis fragment is completely inside the box.

[0198]

[0232] Device 100 can also be configured to calculate normal vectors for the vertices of the 3D mesh 3000A, including all vertices of the outer surfaces 3004 and 3014 and the inner surfaces 3006 and 3016. Vectors 3020A to 3020D are examples of such normal vectors. Other examples of normal vectors are also shown in Figure 30A, but are unlabeled. Device 100 can then classify vertices with normal vectors pointing toward the central axis 3022 as corresponding to the inner surfaces of the bone walls of the diaphysis, and vertices with normal vectors pointing away from the central axis 3022 as corresponding to the outer surfaces of the bone walls of the diaphysis. Thus, device 100 can identify and remove triangles corresponding to the inner surfaces 3006 and 3016 by identifying vertices with normal vectors pointing toward the central axis 3022 and removing all triangles connected to or associated with these vertices.

[0199]

[0233] Figure 30B shows a 2D slice of an example of 3D mesh 3000B corresponding to the same diaphysis as 3D mesh 3000A, but with the inference surfaces 3002 and 3016 removed in 3D mesh 3000B. Similarly, Figure 30C shows an exemplary 3D mesh 3030 corresponding to the humeral wall with all vertices corresponding to both the medial and lateral walls, and Figure 30D shows an example of 3D mesh 3032 corresponding to the outer surface of the humeral wall without vertices corresponding to the outer surface of the humerus.

[0200]

[0234] Device 100 can similarly remove interiors from the 3D mesh of other fragments. With respect to the humeral head fragment, for example, device 100 may identify interiors by determining vertices that have normal vectors pointing toward the center point of the humeral head, such as point 2906 in Figure 29, and remove them from the 3D mesh triangles containing those vertices.

[0201]

[0235] Based on the shape of the outer surface of the 3D mesh corresponding to the diaphysis wall, device 100 can determine the patient's predicted humerus based on statistical shape modeling using the technique described above. That is, device 100 generates another 3D mesh corresponding to the predicted humerus.

[0202]

[0236] In some implementations, device 100 may also be configured to remove 3D meshes corresponding to smaller fragments from the 3D model. That is, device 100 removes 200 mm from the 3D model. 2 Any 3D mesh determined to have a smaller surface area, or smaller than several other such thresholds, may be removed. Surgeons generally remove small fragments rather than reattach them. Therefore, for the purposes of preoperative planning and intraoperative guidance, surgeons may prefer not to see small fragments.

[0203]

[0237] Since the diaphysis is frequently displaced during proximal humeral fractures, surgeons are likely to need to move the diaphysis to repair the fracture. Therefore, the position of the diaphysis in the obtained image data is unlikely to indicate where the diaphysis was before the fracture. Device 100 may be configured to align the predicted humerus and diaphysis to an estimated pre-fracture position using known criteria such as the glenoid fossa. That is, device 100 can determine the position of the 3D mesh of the predicted humerus based on the position of the 3D mesh of the glenoid fossa. For example, device 100 may be configured to identify the scapula in a 3D model based on factors such as position, size, and shape, and to identify the glenoid fossa in the 3D mesh of the scapula. Device 100 can then align the predicted humerus to the glenoid fossa. As an example, device 100 can align the predicted humerus such that the distance between the surface of the humeral head and the surface of the glenoid fossa is 3 mm, and the angle between the diaphysis axis and the vertical axis of the scapula is 30 degrees. Other distances and angles may be used. The device 100 may, as an addition or alternative, be configured to align the predicted humerus with the scapula based on input from a user such as a surgeon.

[0204]

[0238] Device 100 may be configured to mark all 3D meshes of fragments that do not correspond to the humeral head or shaft as corresponding to unknown fragments. Device 100 may then perform an initial projection and alignment of the 3D mesh for the unknown fragments onto the humeral head portion of the 3D mesh for the predicted humerus. To project and align the fragments, Device 100 may determine a sphere in 3D model space centered on the center point of the humeral head of the predicted humerus. Device 100 may determine the sphere to have a radius slightly larger than the radius of the humeral head of the predicted humerus, for example, 10% larger.

[0205]

[0239] Figure 31A shows an example of a 3D mesh of a predicted humerus 3102 with a humeral head 3104. Figure 31A also shows a 3D mesh 3106 corresponding to the humeral head fragment. Figure 31A also shows several unknown fragments. Sphere 3108 is a sphere with a radius slightly larger than the radius of the humeral head 3104. The center of sphere 3108 corresponds approximately to the center of the humeral head 3104. Figures 31B and 31C show the same fragments and spheres as in Figure 31A, but from different viewpoints. By aligning the fragments with a sphere having a radius slightly larger than the predicted humeral head, device 100 may separate the fragments, for example, to reduce contact or overlap.

[0206]

[0240] To project the 3D mesh of a fragment onto a sphere (e.g., sphere 3108), device 100 can, for example, calculate the centroid of the 3D mesh of the fragment. The centroid may be, for example, the average vertex of all the vertices of the fragment. Thus, device 100 can calculate the average vertex by dividing the sum of all vertex coordinates by the number of vertices. Then, device 100 may project the centroid of the 3D mesh of the fragment onto the sphere. To project the centroid of the 3D mesh onto the sphere, device 100 can determine a line that passes through the centroid of the 3D mesh of the fragment and through the center of the sphere. Device 100 can determine the intersection point where this line intersects the sphere. Then, device 100 can determine a translation vector from the centroid of the fragment to the intersection point and apply that translation vector to all the vertices of the 3D mesh of the fragment. Thus, the translation vector moves the fragment in 3D model space without changing the size or shape of the fragment.

[0207]

[0241] Next, device 100 can align the 3D mesh of the fragment by determining an orientation that minimizes the distance between the vertices of the fragment's 3D mesh and the vertices of the sphere's surface. Device 100 can determine the estimated alignment by rotating the 3D mesh of the fragment around the centroid of the fragment corresponding to the intersection with the sphere after device 100 has applied translation vectors to the 3D mesh of the fragment.

[0208]

[0242] After device 100 projects an unknown fragment and aligns it with the sphere, device 100 can move the 3D mesh of the unknown fragment until it no longer intersects with the 3D meshes of other fragments. To do this, device 100 can calculate the global centroid, for example, the average vertex of all vertices for all unknown fragments. Device 100 may then move the fragments away from the global centroid in an iterative manner, such as 1 mm per iteration, until there are no more intersections. Device 100 may, for example, determine a vector extending from the global centroid through the centroid of each individual fragment for each fragment, and then move the individual fragments along that vector. In some examples, device 100 may move only the fragment furthest from the global centroid in each iteration, and then calculate a new global centroid in each iteration.

[0209]

[0243] Next, device 100 can perform approximate alignment of the 3D mesh of the humeral head fragment with respect to the predicted humeral head of the humerus. Device 100 may translate the 3D mesh of the damaged humeral head by, for example, calculating a translation vector, which is a vector from the determined center point of the detected damaged humeral head 3D mesh to the predicted center of the humeral head. That is, device 100 may be configured to calculate a vector from the center of the 3D mesh of the humeral head fragment to the predicted center of the humeral head, and then use the calculated vector to correct all the vertices of the 3D mesh of the humeral head fragment in order to move the entire 3D mesh of the humeral head fragment.

[0210]

[0244] Next, device 100 may use nonlinear minimization to find the initial orientation of the 3D mesh of the humeral head fragment relative to the predicted humeral head. To do so, device 100 may construct a set of three angles (1,0,0;0,1,0;0,0,1) with related axes. Rotations around these axes represent any possible rotations in 3D space. Device 100 can then rotate the 3D mesh of the humeral head fragment in the defined space to determine the minimum distance between the vertices of the 3D mesh of the humeral head fragment and the vertices of the humeral head portion of the predicted humerus 3D mesh. Device 100 may, for example, determine the sum of the distances between the vertices on the 3D mesh of the damaged humeral head fragment and the vertices on the 3D mesh of the predicted humerus. For each vertex on the 3D mesh of the humeral head fragment, device 100 may determine the distance to the nearest vertex on the 3D mesh of the predicted humerus. Device 100 can rotate the 3D mesh of the upper arm head fragment around a center to determine the orientation that minimizes the sum of these distances. This minimization can serve as an initial estimate and does not need to be perfect.

[0211]

[0245] Device 100 can be configured to return the 3D mesh of the undefined fragment toward a new center of gravity in order to reduce a fracture. After the above-described movement and orientation, Device 100 may calculate the new global center of gravity relative to the 3D mesh of the unknown fragment and iteratively move the 3D mesh of the unknown fragment toward the new global center of gravity until the 3D mesh of the unknown fragment begins to contact or intersect with it. Device 100 can, for example, determine a vector from the new global center of gravity through the center of gravity of the individual 3D mesh of the unidentified fragment for each 3D mesh of the unidentified fragment and move the 3D mesh of the individual fragment along that vector. For each iteration, Device 100 can move the 3D mesh of the unidentified fragment by a small amount, such as 1 mm or some other such distance. Similar to the orientation of the 3D mesh for the humeral head fragment relative to the predicted humeral head of the humerus, this minimization can serve as a sufficient initial estimate of the location of the 3D mesh of the unknown fragment and does not need to be perfect.

[0212]

[0246] Next, device 100 may calculate an allowable region for the 3D mesh of the unknown fragment. The allowable region represents the possible locations of the 3D mesh of the unknown fragment and can correspond to areas on the predicted 3D mesh of the humerus that do not overlap with the 3D meshes of the humeral head fragment and the diaphysis fragment. Device 100 can determine the allowable region by subtracting the 3D meshes of the humeral head fragment and the diaphysis fragment from the predicted 3D mesh of the humerus, with the remaining area being the allowable region.

[0213]

[0247] Figures 32A to 32C show the allowable region 3202 from three different viewpoints. The allowable region 3202 corresponds to a portion of the humerus that does not overlap with the diaphysis fragment 3204 or the humeral head fragment 3206.

[0214]

[0248] Next, device 100 can project the 3D mesh of the undefined fragment onto the allowable region. Device 100 can implement, for example, the ICP algorithm or other types of mathematical optimization algorithms to perform the projection. Device 100 may perform nonlinear position optimization on the undefined fragment, for example, to adjust the position of the undefined fragment with respect to both other fragments and the allowable region. To do so, for each undefined fragment, device 100 may be configured to define a set of six parameters (x,y,z translation and z,y,z rotation) and three axes (x(1,0,0), y(0,1,0), z(0,0,1)). The translation and rotation may be fixed, meaning that applying translation and rotation to the 3D mesh may change the orientation or location of the 3D mesh, but does not affect the shape or size of the 3D mesh.

[0215]

[0249] With each iteration of the minimization algorithm, device 100 can align the contours of the 3D mesh by applying transformations from a defined set of transformations to the 3D mesh of each fragment. The defined set of transformations may include translations and rotations that transform the fragments. Figure 33A shows an example of translation, moving fragment 3302 relative to fragment 3304. In the embodiment of Figure 33A, device 100 applies a translation vector (represented by arrow 3306) to fragment 3302, and after translation, moves fragment 3302 relative to fragment 3304. Figure 33B shows an example of rotation, moving fragment 3312 relative to fragment 3314. In the embodiment of Figure 33B, device 100 applies a rotation vector (represented by arrow 3316) to fragment 3312, and after rotation, moves fragment 3312 relative to fragment 3314.

[0216]

[0250] Device 100 may then perform translation on the aligned fragments to move them toward the overall centroid of all unknown fragments. Device 100 may be configured to calculate a minimization function as the product of two terms: (1) the distance between fragments and (2) the distance from the allowable region boundary and the fragment boundary.

[0217]

[0251] Next, device 100 may refine the orientation of the 3D mesh for the humeral head fragment. Once the approximate pre-fracture position of the unknown fragment is determined for the 3D mesh, device 100 can refine the orientation of the 3D mesh for the humeral head fragment by identifying the humeral head contour and fitting a plane to this contour. Device 100 may identify the humeral head contour by, for example, identifying boundary vertices, e.g., vertices of edges in the mesh for the humeral head fragment that are connected to only one triangle. Device 100 may rotate the humeral head around a normal defined by a plane. The plane may, for example, define the normal for the humeral head orientation rotation. Device 100 can rotate the humeral head, for example, from -180 degrees to 180 degrees in steps of 4 degrees, or in any other such steps. Device 100 can calculate the distance from the humeral head contour to the undefined fragment and diaphysis at each iteration and then select an orientation with the minimum distance.

[0218]

[0252] Device 100 may then perform a global fragment adjustment to adjust the positions of all fragments relative to each other. To do so, for each fragment (excluding the diaphysis), device 100 can define a set of six parameters (x, y, z translation and x, y, z rotation) and three axes (x(1,0,0), y(0,1,0), z(0,0,1)). With each iteration of the minimization algorithm, device 100 can apply a transformation from the defined set to each fragment. With each iteration, device 100 can calculate distance values ​​between all fragments. These distance values ​​may be, for example, the sum of the shortest distances between the vertices of the fragment contours. Using minimization, device 100 can find the globally optimal position for all fragments. When performing these calculations, device 100 can use the diaphysis to calculate the fragment distances without transforming the diaphysis, because the diaphysis is the reference point for all fragments based on the positioning described above.

[0219]

[0253] Figures 34A and 34B show examples of minimization algorithms for performing the overall fragment adjustment described above. After performing the initial fragment positioning described above, device 100 can execute the minimization algorithms in Figures 34A and 34B to determine the final position of the unknown fragment in the 3D mesh.

[0220]

[0254] Device 100 applies an initial rotation and / or translation to each 3D mesh of the unknown fragment in order to adjust the position of the unknown fragment's 3D mesh (3400). Device 100 may, for example, select rotations and translations from a defined set of available rotations and translations. The defined set can limit the range of possible available rotations and translations in order to limit the rotations and / or distances that the 3D mesh can move. Device 100 then calculates the overall centroid of the 3D mesh of the unknown fragment (3410).

[0221]

[0255] Device 100 calculates the distance for each 3D mesh of the unknown fragment (3420). The distance to a fragment can be represented by Fragment_d_gc_i, where "i" represents the fragment ID. The distance may be, for example, the distance between the centroid of the fragment and the overall centroid, or some other such distance. Device 100 determines the distance of the furthest 3D mesh to the unknown fragment (3430). That is, Device 100 determines the fragment with the maximum distance. This maximum distance can be called Max_d.

[0222]

[0256] Next, device 100 transforms the unknown fragments based on the distance to the furthest fragment. Figure 34B shows an exemplary process for transforming fragments based on the value of Max_d. For each 3D mesh of the unknown fragments, device 100 calculates a displacement vector (3442). The displacement vector, referred herein to as Displacement_i for fragment i, can have a set magnitude, such as 1 mm. Device 100 may determine Displacement_i as, for example, a vector from the centroid of fragment i to the overall centroid for all unknown fragments. Next, device 100 determines the translation for each 3D mesh of the unknown fragments (3444). Device 100 may determine the translation as, for example, (Fragment_d_gc_i / Max_d)*Displacement_i. Thus, for the furthest fragment, Fragment_d_gc_i is equal to Max_d, and therefore the furthest fragment is moved by the full increment of the magnitude of Displacement_i, while the other unknown fragments are moved by a portion of the increment. This ratio corresponds to (Fragment_d_gc_i / Max_d). Device 100 then uses the translation determined for a particular fragment to transform each 3D mesh of the unknown fragment (3446).

[0223]

[0257] After transforming the unknown fragments, device 100 determines whether any of the 3D meshes of the unknown fragments intersect (3448). If the 3D meshes of the unknown fragments do not intersect (3448, NO), device 100 repeats steps 3442, 3444, and 3446 until the 3D meshes of the unknown fragments intersect.

[0224]

[0258] When unknown fragments intersect (3448, YES), device 100 calculates a minimized value (Figures 34A, 3450). Device 100 may determine the minimized value based, for example, on the total distance between the fragments and the total distance between the unknown fragment and the outer contour of the allowable region, or both. To calculate the total distance between fragments, device 100 may, for example, identify the nearest vertex on the boundary edge of another fragment for each vertex on the boundary contour of each fragment and calculate the distance between these two vertices. Device 100 can determine the total distance between fragments as the sum of the distances between each boundary contour vertex of a fragment and the nearest boundary contour vertex of a different fragment.

[0225]

[0259] Device 100 may also calculate a between the unknown fragment and the outer contour of the allowed region. To do this, device 100 may identify the nearest vertex on any unknown fragment boundary contour for each vertex on the boundary contour of the allowed region and determine the distance between the allowed region vertex and the nearest unknown fragment vertex. Device 100 may determine the total distance between the unknown fragment and the outer contour of the allowed region as the sum of the distances between the allowed region vertices and the nearest unknown fragment vertices.

[0226]

[0260] Device 100 may determine the minimum value as, for example, the sum or product of the total distance between the fragments and the total distance between the unknown fragment and the outer contour of the allowable region. In some examples, device 100 may weight these two factors so that one has a greater impact on the minimum value than the other. In some examples, instead of the total distance between the unknown fragment and the outer contour of the allowable region, for example, the percentage of the allowable region covered by the unknown fragment can be used to determine the minimum value. After calculating the minimum value, device 100 determines whether the stopping condition has been met (3460). If the stopping condition has not been met (3460, NO), device 100 repeats steps 3400, 3410, 3420, 3430, 3440, and 3450 until the stopping condition is met. For each iteration of steps 3400, 3410, 3420, 3430, 3440, and 3450, device 100 applies different initial rotations and / or translations to each of the unknown fragments. If the stopping condition is met (3460, YES), device 100 determines a reduction with the smallest minimization value (3470) because this reduction can be considered a good reconstruction of the unknown fragment.

[0227]

[0261] The termination conditions described above may be, for example, the number of iterations or the amount of time. In some implementations, the termination conditions may be based on achieving a minimum value below a threshold, either alternatively or additionally.

[0228]

[0262] Figure 35 is a flowchart illustrating an exemplary way in which a computing device operates using one or more exemplary techniques described herein. The techniques of Figure 35 are described in reference to device 100 in Figure 1, but are not limited to any particular type of computing device.

[0229]

[0263] Device 100 acquires image data of the joint (3502). The image data may be, for example, a CT scan, a 3D model developed from a CT scan, or any other type of image data as described herein. Device 100 segments the image data to determine the shape of the diaphysis of the humerus (3504). Device 100 may perform segmentation to determine the shape of the diaphysis in response to determining, for example, that the joint contains a fracture. Device 100 may determine that the joint contains a fracture based on an analysis of the image data or based on user input from a user of Device 100, such as a surgeon.

[0230]

[0264] Based on the determined shape of the shaft, device 100 determines the shape of the humerus before it reaches the estimated pathological state (3506). Device 100 may determine the shape of the humerus before it reaches the estimated pathological state, for example, using the SSM technique described above. In most fracture cases, image data of the patient's undamaged humerus is not available. Therefore, device 100 may be configured to determine the shape of the humerus before it reaches the estimated pathological state, as described above. However, in instances where image data of the patient's undamaged humerus is available, device 100 may use image data of the patient's undamaged humerus instead of the shape of the humerus before it reaches the estimated pathological state.

[0231]

[0265] Based on the estimated shape of the humerus, device 100 identifies one or more bone fragments in the image data (3508). Device 100 can, for example, perform various techniques described above to identify one or more bone fragments. For example, device 100 may be configured to identify a region of interest based on the estimated pre-pathological shape of the humerus, and to perform segmentation within the region of interest to identify fragments within the region of interest. In some examples, device 100 may also refine the estimated shape of the humerus after identifying one or more bone fragments. As part of identifying one or more bone fragments, device 100 may also be configured to compare joint image data with images of the estimated pre-pathological shape of the humerus and, based on the comparison, identify fracture locations in the joint image data. Device 100 may be configured to determine the shape of one or more bone fragments in the image data and, based on the shape of one or more bone fragments and the identified fracture locations, determine predicted locations corresponding to one or more bone fragments.

[0232]

[0266] Based on the identified bone fragments in the image data, device 100 generates an output (3510). The output may be, for example, a classification of the fracture determined for the joint (for example, determined for the humerus), and / or an identification of a surgical procedure for fixing the joint. The classification may be, for example, the AO classification, the Neer classification, or any other type of classification. Surgical recommendations may include, for example, an identification of a procedure for fixing the joint. The procedure may be, for example, an indication that surgery is not recommended, that minimally invasive surgery such as the placement of nails or plates is recommended, or that the placement of implants is recommended.

[0233]

[0267] In some examples, device 100 may be configured to determine the number of fragments present in the image data and output a surgical recommendation based on the determined number of fragments. For example, in response to identifying four or more fragments, device 100 may be configured to generate a surgical recommendation indicating that a surgical intervention, such as joint replacement surgery, is necessary for the patient. In another example, in response to detecting only one or two fragments, device 100 may be configured to generate a surgical recommendation indicating that a non-surgical procedure, such as a nail or plate, is recommended for the patient.

[0234]

[0268] In some examples, device 100 may be configured to determine the shape of one or more bone fragments in the image data and, based on the shape of one or more bone fragments, identify the orthographic position of the humerus corresponding to one or more bone fragments before the estimated pathological condition. By detecting the shape of the fragments, device 100 can determine, for example, which part of the patient's anatomical structure the fragment corresponds to. For example, if the shape of the fragment is spherical, the fragment may correspond to the humeral head. After all the fragments have been segmented, device 100 can determine which part of the patient's anatomical structure the fragments correspond to, for example, the greater and lesser tubercles, the humeral head, the biceps groove, or other parts of the patient's anatomical structure.

[0235]

[0269] In some examples, device 100 may be configured to determine the displacement and / or angle of displacement of one or more bone fragments in the image data. Based on the displacement and / or angle of displacement of one or more bone fragments present in the image data, device 100 may be configured to output a surgical recommendation or fracture classification determined for the joint. Based on the displacement and / or angle of displacement of fragments present in the image data, device 100 may be configured to classify fragments. For example, if a fragment has a rotation greater than 45 degrees and / or a displacement of 10 mm or more, the processing circuit 102 may classify the fragment as displaced. Based on the number of displaced fragments and other criteria, device 100 may be configured to output a surgical recommendation or fracture classification determined for the joint.

[0236]

[0270] The output generated by device 100 may be an output image, a still image, a series of images, a video, an animation, or any other type of graphical output. Device 100 may generate an output image for display on display 110, a visualization device 116, or any other such display.

[0237]

[0271] The output image may be, for example, an output image of the patient's joint. The output image may include a visual representation of the patient's injured humerus, at least a portion of the shape of the humerus before it was presumed to be pathological, and at least one of one or more bone fragments. Device 100 may, for example, overlay an image of the shape of the humerus before it was presumed to be pathological with an image of the patient's injured humerus to identify the portion of the patient's humerus damaged by the fracture. For example, for the undamaged portion of the patient's humerus, the image of the shape of the humerus before it was presumed to be pathological and the image of the patient's actual humerus should closely match in shape and other features. In the damaged portion of the patient's humerus, the image of the shape of the humerus before it was presumed to be pathological may deviate from the image of the patient's actual humerus.

[0238]

[0272] Device 100 may be configured to present, for example, regions of the humerus's pre-pathological shape that do not match the patient's actual humerus image, using one or more distinguishing features. Distinguishing features may include contours, shading, coloring, and highlighting. Similarly, Device 100 may be configured to present, for example, regions of the humerus's pre-pathological shape that match the patient's actual humerus image, and to present portions of the patient's actual humerus image that do not match the pre-pathological shape in a contrasting manner. Thus, Device 100 may be configured to generate output images in which the undamaged portion of the patient's humerus, the damaged portion of the patient's humerus, and the predicted portion of the patient's humerus are uniquely identified. In this context, the predicted portion of the patient's humerus generally indicates what those portions of the patient's humerus would look like if they were undamaged.

[0239]

[0273] Device 100 may also be configured to produce an output image that includes a visual representation of at least one of one or more bone fragments. Device 100 may also add annotations to the output image that identify a portion of the visual representation of the pre-pregnancy shape of the humerus corresponding to one of the one or more bone fragments. Device 100 may use contouring, shading, coloring, highlighting, or any other technique to identify where the fragment will be moved, for example, as part of fixing the injured humerus. In addition, or alternatively, Device 100 may also add other annotations, such as arrows or text directions, that identify the technique for moving the fragment as part of fixing the injured humerus.

[0240]

[0274] Device 100 may be configured to generate output images for display via display 110, for example, as part of the surgical planning stage. Output images generated by device 100 as part of the surgical planning stage may enable the surgeon to better estimate the extent and duration of the surgery, determine the tools required to perform the surgery, determine the techniques to be performed during the surgery, and make other such surgical decisions. Device 100 may be configured to output to the surgeon, during the surgical planning stage prior to the actual surgery, an output image containing some or all of the pre-morbid shape of the humerus and / or image data corresponding to the patient's actual injured humerus. The output image may show one or more fragments from the image data of the patient's injured humerus that can be identified using the techniques described above. The output image may also show the pre-morbid shape of the humerus or a portion of the patient's uninjured humerus. The pre-morbid shape of the humerus or the patient's uninjured humerus may be annotated to indicate the location corresponding to the fragment. In other words, the surgeon can be shown the location where the fragments should be moved during surgery by annotating the shape of the humerus before the presumed pathological condition occurred, or by marking the undamaged humerus of the patient.

[0241]

[0275] When a patient is undergoing joint replacement surgery, during the surgical planning phase, the processing circuit 102 may be configured to output an output image containing a visual representation of the humeral head implant corresponding to the humeral head implant to be placed for the patient. The humeral head implant may be shown, for example, as being placed in the diaphysis corresponding to the patient's actual, injured humerus. The output image may also show one or more fragments from image data of the patient's injured humerus that can be identified using the techniques described above. The output image, showing the humeral head implant and the pre-pregnancy shape of the humerus before the estimated pathological condition, or the patient's uninjured humerus, may be annotated to indicate the location corresponding to the fragment. The annotations may be used to advise the surgeon on where to place the fragment during implant surgery.

[0242]

[0276] Device 100 may also be configured to generate output images for display via visualization device 116, for example, during surgery. In some instances, the surgical planning stage described above may occur several days or weeks before the actual surgery is performed. In other instances, such as in cases of major trauma, the surgical planning stage may occur immediately before or concurrently with the actual surgical procedure. Device 100 may also be configured to generate the same output images described above for the surgical planning stage during the actual surgery.

[0243]

[0277] Device 100 may generate a series of images to guide the surgeon on how to place an implant and how to move the fragments to repair the damaged humerus. For example, based on the predicted humerus, device 100 may determine the humeral height, humeral head size, and pulp canal size. Using this information, device 100 may be able to suggest the most appropriate implant size to the surgeon. Furthermore, device 100 may also be able to identify one or more feature points on the fractured anaphysis (e.g., points that the surgeon can easily find during surgical intervention). From these points, device 100 can calculate the height to which the implant should be positioned in order to respect the shape before the predicted pathological condition occurred. During surgery, the surgeon can locate these points, measure the height, and accurately place the implant. In some examples, device 100 may also be configured to display information that may guide the surgeon in placing implant components, such as the humeral head, via a visualization device 116, for example. For example, device 100 may generate markings or annotations that show the surgeon how to position the implant at the correct height, orientation, etc.

[0244]

[0278] After the placement of the humeral head, device 100 may output an image showing the humeral head implant. Device 100 may also annotate the image to guide the surgeon regarding where various fragments should be placed relative to the humeral head implant.

[0245]

[0279] As part of generating the output image described above, device 100 may be configured to align joint image data with an image of the humerus's pre-estimated pathological shape, thereby generating a composite image showing a portion of the joint image data and a portion of the image of the humerus's pre-estimated pathological shape. The composite image may also show one or more bone fragments, a visual representation of the humeral head implant, and / or annotations identifying the position where one or more bone fragments are moved.

[0246]

[0280] The following examples illustrate the devices and technologies described above.

[0247]

[0281] Example 1: The method includes acquiring image data of a joint including at least a portion of the humerus, segmenting the image data to determine the shape of the diaphysis of the humerus, determining the shape of the humerus before it is estimated to be in a pathological state based on the determined shape of the diaphysis, identifying one or more bone fragments in the image data based on the estimated shape of the humerus, and generating an output based on the identified bone fragments in the image data.

[0248]

[0282] Example 2: The method according to Example 1, further comprising determining the number of fragments present in the image data based on the identification of one or more bone fragments.

[0249]

[0283] Example 3: The method according to Example 2, wherein the output includes a surgical recommendation determined based on the determined number of fragments.

[0250]

[0284] Example 4: Surgical recommendations are as described in Example 3, including identification of procedures for joint immobilization.

[0251]

[0285] Example 5: The output is the method of any one of Examples 1 to 4, including the classification of fractures determined for each joint.

[0252]

[0286] Example 6: Identifying one or more bone fragments in image data is the method of any one of Examples 1 to 5, comprising identifying a region of interest based on the presumed pre-pathological shape of the humerus and performing segmentation within the region of interest to identify fragments within the region of interest.

[0253]

[0287] Example 7: The method of any one of Examples 1 to 6, further comprising determining the shape of one or more bone fragments in the image data.

[0254]

[0288] Example 8: The method of Example 7, further comprising determining the presumed pre-pathological position of the humerus corresponding to one or more bone fragments based on the shape of one or more bone fragments.

[0255]

[0289] Example 9: The method according to any one of Examples 1 to 8, further comprising determining the displacement angle of one or more bone fragments in the image data.

[0256]

[0290] Example 10: The method according to any one of Examples 1 to 9, further comprising determining the displacement of one or more bone fragments in the image data.

[0257]

[0291] Example 11: The method according to Example 9 or 10, further comprising determining a surgical recommendation based on the displacement and / or displacement angle of one or more bone fragments present in the image data, wherein the output includes a surgical recommendation.

[0258]

[0292] Example 12: Surgical recommendations are as described in Example 11, including the classification of fractures determined for a joint.

[0259]

[0293] Example 13: The output is an output image, as described in any of Examples 1 through 12.

[0260]

[0294] Example 14: The method according to Example 13, wherein the output image includes a visual representation of at least a portion of the shape of the humerus before the presumed pathological condition.

[0261]

[0295] Example 15: The method according to Example 14, wherein the output image also includes a visual representation of at least one of the one or more bone fragments.

[0262]

[0296] Example 16: The output image is the method described in Example 14 or 15, and also includes a visual representation of the humeral head implant.

[0263]

[0297] Example 17: The method according to any one of Examples 13 to 16, wherein the output image includes a visual representation of at least one of one or more bone fragments.

[0264]

[0298] Example 18: The output image is obtained using any of the methods in Examples 13 to 17, including a visual representation of the humeral head implant.

[0265]

[0299] Example 19: Further comprising generating annotations on the output image, the annotations identifying a portion of the visual representation of the presumed pathological shape of the humerus corresponding to one of one or more bone fragments, in any of the methods in Examples 13 to 18.

[0266]

[0300] Example 20: The method according to any one of Examples 1 to 19, wherein generating an output includes aligning joint image data with an image of the humerus's pre-estimated pathological shape and generating a composite image showing a portion of the joint image data and a portion of the image of the humerus's pre-estimated pathological shape.

[0267]

[0301] Example 21: The method according to Example 20, wherein the composite image data further shows one or more bone fragments.

[0268]

[0302] Example 22: The composite image data further illustrates the visual representation of the humeral head implant according to the method in Example 20 or 21.

[0269]

[0303] Example 23: The composite image further shows annotations identifying the positions where one or more bone fragments are moved, as described in any of Examples 20 to 22.

[0270]

[0304] Example 24: The method according to any one of Examples 1 to 23, further comprising comparing joint image data with images of the humerus's shape before the estimated pathological condition; identifying the fracture site in the joint image data based on the comparison; determining the shape of one or more bone fragments in the image data; and determining the predicted location corresponding to one or more bone fragments based on the shape of one or more bone fragments and the identified fracture site.

[0271]

[0305] Example 25: Determining the shape of the humerus before it is estimated to be pathological is the method of any one of Examples 1 to 24, which includes determining the shape of the humerus before it is estimated to be pathological based on a statistical shape model.

[0272]

[0306] Example 26: The method according to any one of Examples 1 to 24, further comprising determining one or more of the following based on the estimated shape of the humerus: estimated humeral height, compressed and estimated humeral head size, or estimated canal size.

[0273]

[0307] Example 27: The method of any one of Examples 1 to 26, further comprising determining a recommended implant size for a patient based on one or more of the estimated humeral height, estimated humeral head size, or estimated pulposus size.

[0274]

[0308] Example 28: The method of any one of Examples 1 to 27, further comprising determining a recommended implant size for the patient based on the estimated shape of the humerus.

[0275]

[0309] Example 29: The method of any one of Examples 1 to 28, further comprising determining one or more feature points on the fractured shaft of the humerus.

[0276]

[0310] Example 30: The methods of Examples 26 and 29, further comprising determining one or more feature points on the fractured humeral shaft based on one or more of the estimated humeral height, estimated humeral head size, or estimated canal size.

[0277]

[0311] Example 31: The method of Example 29 or 30, further comprising determining the implant height of the humeral head component based on one or more feature points on the fractured diaphysis.

[0278]

[0312] Example 32: The method of Example 31, further comprising outputting a visual representation of the determined implant height.

[0279]

[0313] Example 33: The system includes a memory for storing image data of a joint including at least a portion of the humerus, and one or more processors, the one or more processors being configured to acquire image data of the joint including at least a portion of the humerus, segment the image data to determine the shape of the diaphysis of the humerus, determine the shape of the humerus before it is estimated to be in a pathological state based on the determined shape of the diaphysis, identify one or more bone fragments in the image data based on the estimated shape of the humerus, and generate an output based on the identified bone fragments in the image data.

[0280]

[0314] Example 34: The system according to Example 33, wherein one or more processors are further configured to determine the number of fragments present in the image data based on the identification of one or more bone fragments.

[0281]

[0315] Example 35: The system described in Example 34, wherein the output includes a surgical recommendation determined based on the determined number of fragments.

[0282]

[0316] Example 36: The system described in Example 35, wherein surgical recommendations include identification of procedures for joint fixation.

[0283]

[0317] Example 37: The output is a system described in any of Examples 33 to 36, including the classification of fractures determined for each joint.

[0284]

[0318] Example 38: A system according to any one of Examples 33 to 37, wherein, in order to identify one or more bone fragments in image data, one or more processors are configured to identify a region of interest based on the presumed pre-pathological shape of the humerus and to perform segmentation within the region of interest to identify fragments within the region of interest.

[0285]

[0319] Example 39: The system according to any one of Examples 33 to 38, wherein one or more processors are further configured to determine the shape of one or more bone fragments in the image data.

[0286]

[0320] Example 40: The system according to Example 39, wherein one or more processors are further configured to identify the presumed pre-pathological position of the humerus corresponding to one or more bone fragments, based on the shape of one or more bone fragments.

[0287]

[0321] Example 41: The system according to any one of Examples 33 to 40, wherein one or more processors are further configured to determine the displacement angles of one or more bone fragments in the image data.

[0288]

[0322] Example 42: The system according to any one of Examples 33 to 41, wherein one or more processors are further configured to determine the displacement of one or more bone fragments in the image data.

[0289]

[0323] Example 43: The system according to Example 41 or 42, wherein one or more processors are further configured to determine a surgical recommendation based on the displacement and / or angle of displacement of one or more bone fragments present in the image data, and the output includes a surgical recommendation.

[0290]

[0324] Example 44: Surgical recommendations include the classification of fractures determined for each joint, as described in Example 43.

[0291]

[0325] Example 45: The output includes the output image, as described in any of Examples 33 through 44.

[0292]

[0326] Example 46: The system according to Example 45, wherein the output image includes a visual representation of at least a portion of the shape of the humerus before the estimated pathological condition.

[0293]

[0327] Example 47: The system described in Example 46, wherein the output image also includes a visual representation of at least one of the one or more bone fragments.

[0294]

[0328] Example 48: The system described in Example 46 or 47, where the output image also includes a visual representation of the humeral head implant.

[0295]

[0329] Example 49: The system according to any one of Examples 45 to 48, wherein the output image includes a visual representation of at least one of one or more bone fragments.

[0296]

[0330] Example 50: The output image is a system described in any of Examples 45 to 49, including a visual representation of the humeral head implant.

[0297]

[0331] Example 51: A system according to any one of Examples 45 to 50, wherein one or more processors are further configured to generate annotations on an output image, the annotations identifying a portion of the visual representation of the shape of the humerus before it became presumed pathological, corresponding to one of the one or more bone fragments.

[0298]

[0332] Example 52: The system according to any one of Examples 43 to 51, wherein one or more processors are configured to align joint image data with an image of the humerus's pre-estimated pathological shape in order to generate an output, and to generate a composite image showing a portion of the joint image data and a portion of the image of the humerus's pre-estimated pathological shape.

[0299]

[0333] Example 53: The system described in Example 52, wherein the composite image data further shows one or more bone fragments.

[0300]

[0334] Example 54: The composite image data further illustrates the visual representation of the humeral head implant, as described in Example 52 or 53.

[0301]

[0335] Example 55: The composite image further includes annotations indicating the position for moving one or more of the bone fragments, as described in any of Examples 52 to 54.

[0302]

[0336] Example 56: The system according to any one of Examples 33 to 45, further configured to compare joint image data with images of the humerus's shape before it was estimated to be in a pathological state, to identify fracture locations in the joint image data based on the comparison, to determine the shape of one or more bone fragments in the image data, and to determine predicted locations corresponding to one or more bone fragments based on the shape of one or more bone fragments and the identified fracture locations.

[0303]

[0337] Example 57: A system according to any one of Examples 33 to 46, wherein one or more processors are configured to determine the shape of the humerus before it is estimated to be in a pathological state, based on a statistical shape model.

[0304]

[0338] Example 58: The system described in any of Examples 33 to 57, wherein the system includes a display device configured to display an output.

[0305]

[0339] Example 59: The system described in Example 58, wherein the display comprises a visualization device.

[0306]

[0340] Example 60: A system according to any one of Examples 35 to 59, further comprising determining an estimated humeral height and one or more of the estimated humeral head size or estimated canal size, based on the estimated shape of the humerus.

[0307]

[0341] Example 61: A system according to any of Examples 35 to 60, further comprising determining a recommended implant size for a patient based on one or more of the following: estimated humeral height, estimated humeral head size, or estimated pulposus size.

[0308]

[0342] Example 62: A system according to any of Examples 35 to 61, further comprising determining a recommended implant size for the patient based on the estimated shape of the humerus.

[0309]

[0343] Example 63: A system according to any one of Examples 35 to 62, further comprising determining one or more feature points on the fractured shaft line of the humerus.

[0310]

[0344] Example 64: The system according to Examples 61 and 63, further comprising determining one or more feature points on the fractured humeral shaft based on one or more of the estimated humeral height, estimated humeral head size, or estimated canal size.

[0311]

[0345] Example 65: The system according to Example 63 or 64, further comprising determining the implant height of the humeral head component based on one or more feature points on the fractured diaphysis.

[0312]

[0346] Example 66: The system described in Example 65, further comprising outputting a visual representation of the determined implant height.

[0313]

[0347] Example 67: A computer-readable storage medium that stores instructions that, when executed, cause one or more processors of a surgical system to perform any combination of the methods of Examples 1 to 32.

[0314]

[0348] Example 68: A virtual surgical system including means for performing any combination of methods from Examples 1 to 32.

[0315]

[0349] Example 69: A method comprising: acquiring image data of a joint including at least a portion of the humerus; segmenting the image data to identify portions of the image data corresponding to cortical bone; generating a three-dimensional (3D) model based on the portions of the image data corresponding to cortical bone; the 3D model comprising one or more 3D meshes corresponding to the surfaces of the portions of the image data corresponding to cortical bone; identifying portions of 3D meshes corresponding to the diaphysis in one or more 3D meshes; determining the shape of the humerus before the estimated pathological condition based on the shape of the portions of 3D meshes corresponding to the diaphysis; and generating an output based on the shape of the humerus before the estimated pathological condition.

[0316]

[0350] Example 70: The method according to Example 69, further comprising identifying a portion of the 3D mesh corresponding to the humeral head in one or more 3D meshes.

[0317]

[0351] Example 71: The method according to Example 70, wherein identifying a portion of a 3D mesh corresponding to the head of the arm comprises determining the normal vectors of the vertices of one or more 3D meshes, determining the most common intersection of the normal vectors of the vertices of one or more 3D meshes, and identifying a vertex having a normal vector that intersects the most common intersection as a vertex belonging to the portion of the 3D mesh corresponding to the head of the arm.

[0318]

[0352] Example 72: The method according to Example 70 or 71, wherein identifying a portion of the 3D mesh corresponding to the diaphysis includes determining the vertex furthest from the portion of the 3D mesh corresponding to the humeral head in one or more 3D meshes.

[0319]

[0353] Example 73: The method of any one of Examples 69 to 72, further comprising aligning the humerus to its shape prior to the presumed pathological condition.

[0320]

[0354] Example 74: The method of Example 74, further comprising identifying a 3D mesh corresponding to a scapula in a 3D model, and identifying a reference point based on the 3D mesh corresponding to the scapula.

[0321]

[0355] Example 75: The method according to any one of Examples 69 to 74, further comprising identifying a 3D mesh in a 3D model that corresponds to an unknown fragment, wherein the 3D mesh corresponding to the unknown fragment includes a 3D mesh that is not the 3D mesh corresponding to the diaphysis or the 3D mesh corresponding to the humeral head.

[0322]

[0356] Example 76: The method according to Example 76, further comprising determining an acceptable area of ​​a 3D mesh corresponding to an unknown fragment based on the presumed pre-pathological shape of the humerus, a 3D mesh corresponding to the shaft, and a 3D mesh corresponding to the humeral head.

[0323]

[0357] Example 77: The method of Example 76, further comprising determining the location of an unknown fragment within an acceptable region.

[0324]

[0358] Example 78: The method of Example 76, which determines the minimum value based on the distance between the boundary of the tolerance region and the 3D mesh corresponding to the unknown fragment.

[0325]

[0359] Example 79: The method of Example 76, wherein the minimum value is determined based on the percentage of the allowable area covered by the 3D mesh corresponding to the unknown fragment.

[0326]

[0360] Example 80: The method according to Example 78 or 79, further determining the minimum value based on the distance between each 3D mesh corresponding to the unknown fragment.

[0327]

[0361] Example 81: The method of Example 76, further comprising performing multiple transformations on a 3D mesh corresponding to an unknown fragment, determining a minimum value for each of the multiple transformations based on one or more of the following: the percentage of the tolerance region covered by the 3D mesh corresponding to the unknown fragment, the distance between the boundary of the tolerance region and the 3D mesh corresponding to the unknown fragment, or the distance between each of the 3D meshes corresponding to the unknown fragment, and selecting fragment reduction based on the minimum values ​​for the multiple transformations.

[0328]

[0362] Example 82: The method of any one of Examples 69 to 81, further comprising determining the number of fragments present in the image data.

[0329]

[0363] Example 83: The method according to Example 82, wherein the output includes a surgical recommendation determined based on the determined number of fragments.

[0330]

[0364] Example 84: Surgical recommendations are as described in Example 83, including identification of procedures for joint fixation.

[0331]

[0365] Example 85: The output is the method of any of Examples 69 to 84, including the classification of fractures determined for each joint.

[0332]

[0366] Example 86: The output is an output image, as described in any of Examples 69 to 85.

[0333]

[0367] Example 87: The method according to Example 86, wherein the output image includes a visual representation of at least a portion of the shape of the humerus before the presumed pathological condition.

[0334]

[0368] Example 88: The method according to Example 87, wherein the output image also includes a visual representation of at least one of the unknown fragments.

[0335]

[0369] Example 89: The output image is the method described in Example 87 or 88, and also includes a visual representation of the humeral head implant.

[0336]

[0370] Example 90: The method of any one of Examples 86 to 89, further comprising generating annotations on the output image, wherein the annotations identify a portion of the visual representation of the presumed pre-pathological shape of the humerus, corresponding to an unknown fragment of the unknown fragment.

[0337]

[0371] Example 91: The method according to any one of Examples 69 to 90, wherein generating an output includes aligning joint image data with an image of the humerus's pre-estimated pathological shape and generating a composite image showing a portion of the joint image data and a portion of the image of the humerus's pre-estimated pathological shape.

[0338]

[0372] Example 92: The method according to Example 91, wherein the composite image data further shows one or more bone fragments.

[0339]

[0373] Example 93: The method according to Example 91 or 92, wherein the composite image data further shows one or more visual representations of unknown fragments.

[0340]

[0374] Example 94: The composite image further indicates annotations that identify the positions to move one or more of the unknown fragments, as described in any of Examples 91 to 93.

[0341]

[0375] Example 95: The method according to any of Examples 69 to 94, further comprising determining the angle of displacement of one or more bone fragments from a 3D model.

[0342]

[0376] Example 96: The method according to any of Examples 69 to 95, further comprising determining the displacement of one or more bone fragments from a 3D model.

[0343]

[0377] Example 97: The method of Example 95 or 96, further comprising determining a surgical recommendation based on the displacement and / or angle of displacement of one or more bone fragments, wherein the output includes a surgical recommendation.

[0344]

[0378] Example 98: Surgical recommendations, including the classification of fractures determined for a joint, as described in Example 97.

[0345]

[0379] Example 99: The method according to any one of Examples 95 to 98, further comprising determining the displacement angle of one or more bone fragments from a 3D model by determining the amount of rotation required for fragment reduction.

[0346]

[0380] Example 100: The method according to any one of Examples 95 to 99, further comprising determining the displacement of one or more bone fragments from a 3D model by determining the amount of translation required for fragment reduction.

[0347]

[0381] Example 101: A computer-readable storage medium for storing instructions, wherein, when executed, the instructions cause one or more processors of a surgical system to perform any combination of methods of Examples 69 to 100.

[0348]

[0382] Example 102: A virtual surgical system comprising means for performing any combination of methods from Examples 69 to 101.

[0349]

[0383] While the technology has been disclosed with respect to a limited number of examples, those skilled in the art who are interested in this disclosure will understand that there are numerous modifications and variations therefrom. For example, it is intended that any reasonable combination of the examples described may be carried out. The appended claims are intended to cover modifications and variations that fall within the true spirit and scope of the invention.

[0350]

[0384] It should be recognized that, depending on the example, certain operations or events of any of the techniques described herein may be performed in different sequences, and may be added, combined, or omitted entirely (for example, not all described operations or events are necessary for the implementation of those techniques). Furthermore, in certain examples, operations or events may be performed not sequentially, but concurrently, for example, through multithreading, interrupt handling, or multiple processors.

[0351]

[0385] In one or more examples, the functions described may be implemented in hardware, software, firmware, or any combination thereof. Where implemented in software, these functions may be stored as one or more instructions or codes on a computer-readable medium, or transmitted through a computer-readable medium, or executed by a hardware-based processing unit. The computer-readable medium may include, for example, a communication medium, which includes any medium that facilitates the transfer of a computer program from one location to another according to a communication protocol, or a computer-readable storage medium corresponding to a tangible medium such as a data storage medium. Thus, the computer-readable medium may generally correspond to (1) a tangible computer-readable storage medium that is not transient, or (2) a communication medium such as a signal or carrier wave. The data storage medium may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, codes, and / or data structures for the implementation of the technology described herein. A computer program product may include a computer-readable medium.

[0352]

[0386] Such computer-readable storage media may include, but are not limited to, computer-readable storage media, RAM, ROM, EEPROM®, CD-ROM or other optical disk storage devices, magnetic disk storage devices or other magnetic storage devices, flash memory, or any other media that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. Also, any connection is strictly referred to as computer-readable media. For example, if instructions are transmitted from a website, from a server, or from another remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of media. However, it should be understood that computer-readable storage media and data storage media do not include connections, carriers, signals, or other temporary media, but instead focus on non-temporary tangible storage media. The terms "disk" and "disc" used herein include compact discs (CDs), laserdiscs (registered trademark), optical discs, digital general-purpose discs (DVDs), floppy disks (registered trademark), and Blu-ray discs (registered trademark). Generally, a "disk" reproduces data magnetically, while a "disc" reproduces data optically using a laser. Combinations of these should also be included within the scope of computer-readable media.

[0353]

[0387] The operations described herein may be performed by one or more processors, which may be implemented as fixed-function processing circuits, programmable circuits, or combinations thereof, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other equivalent integrated circuits or discrete logic circuits. Fixed-function circuits refer to circuits that provide a specific functionality and are pre-configured to perform operations. Programmable circuits refer to circuits that can be programmed to perform a variety of tasks and provide flexible functionality in the operations they can perform. For example, a programmable circuit may execute instructions specified by software or firmware that operate the programmable circuit in a manner defined by software or firmware instructions. Fixed-function circuits may execute software instructions (e.g., to receive or output parameters), but the type of operation performed by a fixed-function circuit is generally immutable. Therefore, the terms “processor” and “processing circuit,” as used herein, may refer to any of the aforementioned structures or any other structure suitable for the implementation of the technologies described herein.

[0354]

[0388] We have described various examples. These and other examples are within the scope of the following claims. The original claims of the patent application are listed here. [1] A method, To obtain image data of the joint including at least a portion of the humerus, Segmenting the image data in order to identify the portion of the image data corresponding to the cortical bone, Based on the portion of the image data corresponding to the cortical bone, a three-dimensional (3D) model is generated, and the 3D model includes one or more 3D meshes corresponding to the surface of the portion of the image data corresponding to the cortical bone. In the aforementioned one or more 3D meshes, a portion of the 3D mesh corresponding to the bone shaft is identified, Based on the shape of the 3D mesh portion corresponding to the aforementioned diaphysis, the shape of the humerus before the estimated pathological condition is determined, A method comprising generating an output based on the presumed pre-pathological shape of the humerus. [2] The method according to [1], further comprising aligning the shape of the humerus prior to the estimated pathological condition with a reference point. [3] In the 3D model, the 3D mesh corresponding to the scapula is identified, The method according to [2], further comprising identifying the reference point based on a 3D mesh corresponding to the scapula. [4] The method according to [1], further comprising identifying a portion of the 3D mesh corresponding to the humeral head in one or more of the 3D meshes. [5] Identifying the portion of the 3D mesh corresponding to the upper arm head is: Determining the normal vectors of the vertices of one or more 3D meshes, The most common intersection point of the normal vectors of the vertices of the one or more 3D meshes, The method according to [4], which includes identifying a vertex having a normal vector that intersects the most common intersection point as a vertex belonging to the portion of the 3D mesh corresponding to the upper arm head. [6] The method according to [4], wherein identifying the portion of the 3D mesh corresponding to the shaft of the bone includes determining the vertex in one or more 3D meshes that is furthest from the portion of the 3D mesh corresponding to the humeral head. [7] The method according to [4], further comprising identifying a 3D mesh in the 3D model that corresponds to an unknown fragment, wherein the 3D mesh corresponding to the unknown fragment includes a 3D mesh that is not the 3D mesh corresponding to the shaft or the 3D mesh corresponding to the humeral head. [8] The method according to [7], further comprising determining an acceptable area for a 3D mesh corresponding to the unknown fragment based on the shape of the humerus prior to the presumed pathological condition, a 3D mesh corresponding to the shaft, and a 3D mesh corresponding to the humeral head. [9] The method according to [8], further comprising determining the location of a 3D mesh corresponding to the unknown fragment in the allowable region.

[10] The method according to [8], further comprising determining a minimum value based on the distance between the boundary of the allowable region and the 3D mesh corresponding to the unknown fragment.

[11] The method according to [8], further comprising determining a minimum value based on the percentage of the allowable area covered by the 3D mesh corresponding to the unknown fragment.

[12] The method according to

[11] further comprises determining the minimized value based on the distance between each 3D mesh corresponding to the unknown fragment.

[13] Performing multiple transformations on the 3D mesh corresponding to the unknown fragment, For each of the aforementioned transformations, a minimum value is determined based on one or more of the following: the percentage of the allowable region covered by the 3D mesh corresponding to the unknown fragment, the distance between the boundary of the allowable region and the 3D mesh corresponding to the unknown fragment, or the distance between each of the 3D meshes corresponding to the unknown fragment. The method according to [8] further comprises selecting fragment rearrangement based on the minimum value for the plurality of transformations.

[14] The method according to [1], further comprising determining the number of unknown fragments present in the image data, wherein the unknown fragments are represented by a 3D mesh that is not the 3D mesh corresponding to the diaphysis or the 3D mesh corresponding to the humeral head.

[15] The method according to

[14] , wherein the output includes a surgical recommendation determined based on the number of unknown fragments determined.

[16] The surgical recommendation is the method of

[15] , which includes identifying a procedure for immobilizing the joint.

[17] The method according to [1], wherein the output includes a classification of fractures determined for the joint.

[18] The method according to [1], wherein the output includes an output image.

[19] The method according to

[18] , wherein the output image includes a visual representation of at least a portion of the shape of the humerus prior to the presumed pathological condition.

[20] The method according to

[18] , wherein the output image includes a visual representation of at least one unknown fragment, the unknown fragment including a 3D mesh that is not the 3D mesh corresponding to the shaft or the 3D mesh corresponding to the humeral head.

[21] The method according to

[18] , which includes a visual representation of the humeral head implant in the output image.

[22] The method according to

[18] , further comprising generating annotations on the output image, the annotations identifying a portion of a visual representation of the shape of the humerus before the estimated pathological condition corresponding to an unknown fragment, the unknown fragment comprising a 3D mesh that is not the diaphysis or the humeral head.

[23] The output described above is generated by Aligning the image data of the joint with an image of the humerus before it entered the estimated pathological state, The method according to [1], comprising generating a composite image showing a portion of the image data of the joint and a portion of the image showing the shape of the humerus before it entered the estimated pathological state.

[24] The composite image further shows a visual representation of one or more unknown fragments, the one or more unknown fragments corresponding to a 3D mesh that is not the 3D mesh corresponding to the shaft or the 3D mesh corresponding to the humeral head, the method according to

[23] .

[25] The composite image further provides annotations identifying positions for moving one or more unknown fragments, the one or more unknown fragments corresponding to a 3D mesh that is not the 3D mesh corresponding to the shaft or the 3D mesh corresponding to the humeral head, the method according to

[23] .

[26] Determining the displacement angle of one or more unknown fragments from the 3D model, wherein the one or more unknown fragments are represented by a 3D mesh that is not the 3D mesh corresponding to the shaft or the 3D mesh corresponding to the humeral head, The method according to [1], further comprising determining a surgical recommendation based on the displacement angles of one or more unknown fragments, wherein the output includes the surgical recommendation.

[27] The method according to

[26] , further comprising determining the displacement angle of one or more unknown fragments from the 3D model by determining the amount of rotation required for fragment reduction.

[28] Determining the displacement of one or more unknown fragments from the 3D model, wherein the one or more unknown fragments are represented by a 3D mesh that is not the 3D mesh corresponding to the shaft or the 3D mesh corresponding to the humeral head, The method according to [1], further comprising determining a surgical recommendation based on the displacement of one or more unknown fragments, wherein the output includes the surgical recommendation.

[29] The method according to

[28] , further comprising determining the displacement of one or more unknown fragments from the 3D model by determining the amount of translation required for fragment reduction.

[30] The surgical recommendation is the method described in

[28] , which includes the classification of fractures determined for the joint.

[31] A healthcare system, A memory configured to store image data of a joint including at least a portion of the humerus, A processing circuit is provided, The aforementioned processing circuit is Image data of the joint including at least a portion of the humerus is obtained. The image data is segmented in order to identify the portion of the image data corresponding to the cortical bone. A three-dimensional (3D) model is generated based on the portion of the image data corresponding to the cortical bone, and the 3D model includes one or more 3D meshes corresponding to the surface of the portion of the image data corresponding to the cortical bone. In the aforementioned one or more 3D meshes, a portion of the 3D mesh corresponding to the bone shaft is identified, Based on the shape of the 3D mesh portion corresponding to the aforementioned diaphysis, the shape of the humerus before the estimated pathological condition is determined. A medical system configured to generate an output based on the shape of the humerus prior to the estimated pathological condition.

[32] The processing circuit is The medical system according to

[31] , further configured to align the shape of the humerus before the estimated pathological condition occurred with a reference point.

[33] The processing circuit is In the aforementioned 3D model, the 3D mesh corresponding to the scapula is identified, The medical system according to

[32] , further configured to identify the reference point based on a 3D mesh corresponding to the scapula.

[34] The processing circuit is The medical system according to

[31] , wherein one or more 3D meshes are further configured to identify a portion of the 3D mesh corresponding to the humeral head.

[35] In order to identify the portion of the 3D mesh corresponding to the upper arm head, the processing circuit: Determine the normal vectors of the vertices of the one or more 3D meshes mentioned above. Determine the most common intersection point of the normal vectors of the vertices of the one or more 3D meshes. The medical system according to

[34] , further configured to identify a vertex having a normal vector intersecting the most common intersection as a vertex belonging to the portion of the 3D mesh corresponding to the upper arm head.

[36] In order to identify the portion of the 3D mesh corresponding to the shaft of the bone, the processing circuit is further configured to determine the vertex in one or more of the 3D mesh that is furthest from the portion of the 3D mesh corresponding to the humeral head, as in the medical system of

[34] .

[37] The processing circuit is The medical system according to

[34] , wherein the 3D model is further configured to identify a 3D mesh corresponding to an unknown fragment, the 3D mesh corresponding to the unknown fragment includes a 3D mesh that is not the 3D mesh corresponding to the diaphysis or the 3D mesh corresponding to the humeral head.

[38] The processing circuit is The medical system according to

[37] , further configured to determine an acceptable area for a 3D mesh corresponding to the unknown fragment, based on the shape of the humerus prior to the presumed pathological condition, a 3D mesh corresponding to the shaft, and a 3D mesh corresponding to the humeral head.

[39] The processing circuit is The medical system according to

[38] , further configured to determine the location of a 3D mesh corresponding to the unknown fragment within the allowable region.

[40] The processing circuit is The medical system according to

[38] , further configured to determine a minimum value based on the distance between the boundary of the allowable region and the 3D mesh corresponding to the unknown fragment.

[41] The processing circuit is The medical system according to

[38] , further configured to determine a minimum value based on the percentage of the allowable area covered by the 3D mesh corresponding to the unknown fragment.

[42] The processing circuit is The medical system according to

[41] , further configured to determine the minimum value based on the distance between each 3D mesh corresponding to the unknown fragment.

[43] The processing circuit is Perform multiple transformations on the 3D mesh corresponding to the aforementioned unknown fragment. For each of the above-mentioned transformations, a minimum value is determined based on one or more of the following: the percentage of the allowable region covered by the 3D mesh corresponding to the unknown fragment, the distance between the boundary of the allowable region and the 3D mesh corresponding to the unknown fragment, or the distance between each of the 3D meshes corresponding to the unknown fragment. The medical system according to

[38] , further configured to select fragment reduction based on the minimum value for the plurality of transformations.

[44] The processing circuit is The medical system according to

[31] , further configured to determine the number of unknown fragments in the image data, wherein the unknown fragments are represented by a 3D mesh that is not the 3D mesh corresponding to the diaphysis or the 3D mesh corresponding to the humeral head.

[45] The medical system according to

[44] , wherein the output includes a surgical recommendation determined based on the number of unknown fragments determined.

[46] The medical system described in

[45] , wherein the surgical recommendation includes the identification of a procedure for immobilizing the joint.

[47] The medical system according to

[31] , wherein the output includes a classification of fractures determined for the joint.

[48] ​​The medical system according to

[31] , wherein the output includes an output image.

[49] The medical system according to

[48] , wherein the output image includes a visual representation of at least a portion of the shape of the humerus prior to the presumed pathological condition.

[50] The medical system according to

[48] , wherein the output image includes a visual representation of at least one unknown fragment, the unknown fragment including a 3D mesh that is not the 3D mesh corresponding to the shaft or the 3D mesh corresponding to the humeral head.

[51] The medical system described in

[48] , which includes a visual representation of the humeral head implant in the output image.

[52] The processing circuit is A medical system according to

[48] , further configured to generate annotations on the output image, wherein the annotations identify a portion of a visual representation of the shape of the humerus before it was presumed to be in a pathological state, the unknown fragment comprising a 3D mesh that is not the diaphysis or the humeral head.

[53] The processing circuit is The image data of the joint is aligned with an image of the humerus before it was estimated to be in a pathological state. The medical system according to

[31] , further configured to generate a composite image showing a portion of the joint image data and a portion of the image of the humerus before it was estimated to be in a pathological state.

[54] The medical system according to

[53] , wherein the composite image further shows a visual representation of one or more unknown fragments, the one or more unknown fragments corresponding to a 3D mesh that is not the 3D mesh corresponding to the shaft or the 3D mesh corresponding to the humeral head.

[55] The medical system according to

[53] , wherein the composite image further indicates annotations that identify the position of moving one or more unknown fragments, the one or more unknown fragments corresponding to a 3D mesh that is not the 3D mesh corresponding to the shaft or the 3D mesh corresponding to the humeral head.

[56] The processing circuit is The displacement angle of one or more unknown fragments is determined from the 3D model, and the one or more unknown fragments are represented by a 3D mesh that is not the 3D mesh corresponding to the shaft of the bone or the 3D mesh corresponding to the humeral head. The medical system according to

[31] , further configured to determine a surgical recommendation based on the displacement angle of one or more unknown fragments, wherein the output includes the surgical recommendation.

[57] The processing circuit is The medical system according to

[56] , further configured to determine the displacement angle of one or more unknown fragments from the 3D model by determining the amount of rotation required for fragment reduction.

[58] The processing circuit is The displacement of one or more unknown fragments is determined from the 3D model, and the one or more unknown fragments are represented by a 3D mesh that is not the 3D mesh corresponding to the shaft of the bone or the 3D mesh corresponding to the humeral head. A medical system according to

[31] , further configured to determine a surgical recommendation based on the displacement of one or more unknown fragments, wherein the output includes the surgical recommendation.

[59] The processing circuit is The medical system according to

[58] , further configured to determine the displacement of one or more unknown fragments from the 3D model by determining the amount of translation required for fragment reduction.

[60] The medical system described in

[58] , which includes the classification of fractures determined for the joint, as described in the surgical recommendations.

[61] The medical system according to

[31] , further comprising a display device configured to display the output.

[62] The medical system according to

[61] , wherein the display device comprises a mixed reality visualization device.

[63] A computer-readable storage medium, To obtain image data of the joint including at least a portion of the humerus, Segmenting the image data in order to identify the portion of the image data corresponding to the cortical bone, Based on the portion of the image data corresponding to the cortical bone, a three-dimensional (3D) model is generated, and the 3D model includes one or more 3D meshes corresponding to the surface of the portion of the image data corresponding to the cortical bone. In the aforementioned one or more 3D meshes, a portion of the 3D mesh corresponding to the bone shaft is identified, Based on the shape of the 3D mesh portion corresponding to the aforementioned diaphysis, the shape of the humerus before the estimated pathological condition is determined, A computer-readable storage medium, which includes generating an output based on the shape of the humerus prior to the estimated pathological condition.

[64] A healthcare system, Means for acquiring image data of a joint including at least a portion of the humerus, Means for segmenting the image data in order to identify the portion of the image data corresponding to the cortical bone, A means for generating a three-dimensional (3D) model based on a portion of the image data corresponding to the cortical bone, wherein the 3D model includes one or more 3D meshes corresponding to the surface of the portion of the image data corresponding to the cortical bone. In the aforementioned one or more 3D meshes, means for identifying a portion of the 3D mesh corresponding to the bone shaft, A means for determining the shape of the humerus before it is estimated to be in a pathological state, based on the shape of a portion of the 3D mesh corresponding to the diaphysis, A medical system comprising means for generating an output based on the presumed shape of the humerus before it entered a pathological state.

Claims

1. A method of operating a medical system, The computing device acquires image data of the joint, including a portion of the diaphysis of the humerus in a pathological condition. The computing device segments the image data in order to determine a three-dimensional (3D) model of the shaft of the humerus, The computing device determines a 3D model of the estimated normal shape of the humerus using statistical shape modeling techniques based on the 3D model of the shaft of the humerus, The computing device identifies a region of interest based on the 3D model of the estimated normal shape of the humerus, performs segmentation in the region of interest to identify one or more bone fragments in the region of interest, and identifies one or more bone fragments in the image data. An operating method comprising: a computing device generating an output based on identified bone fragments in the image data.

2. The method of operation according to claim 1, further comprising a computing device determining the number of one or more bone fragments present in the image data based on the identification of one or more bone fragments, wherein the output includes a surgical recommendation determined based on the determined number of one or more bone fragments, wherein the surgical recommendation includes the identification of a procedure for fixing the joint.

3. The computing device determines the shape of one or more bone fragments in the image data, The operating method according to claim 1, further comprising a computing device determining the location on the 3D model of the estimated normal shape of the humerus corresponding to the one or more bone fragments, based on the shape of the one or more bone fragments.

4. The computing device determines the displacement angle of one or more bone fragments in the image data, The method of operation according to claim 1, further comprising a computing device determining a surgical recommendation based on the displacement angles of one or more bone fragments in the image data, wherein the output includes the surgical recommendation.

5. The computing device determines the displacement of one or more bone fragments in the image data, The operating method according to claim 1, further comprising a computing device determining a surgical recommendation based on the displacement of one or more bone fragments present in the image data, wherein the output comprises the surgical recommendation.

6. The output includes an output image which includes a visual representation of at least a portion of the 3D model of the estimated normal shape of the humerus, a visual representation of at least one of the one or more bone fragments, and a visual representation of the humeral head implant. The operating method according to claim 1, further comprising generating annotations on the output image, wherein the annotations identify a portion of the visual representation of the 3D model of the estimated normal shape of the humerus corresponding to one or more bone fragments.

7. The computing device compares the image data of the joint with an image of the 3D model of the estimated normal shape of the humerus, The computing device identifies the fracture location in the joint image data based on the comparison, The computing device determines the shape of one or more bone fragments in the image data, The method of operation according to claim 1, further comprising a computing device determining a predicted location corresponding to one or more bone fragments based on the shape of one or more bone fragments and the identified fracture locations.

8. The method of operation according to claim 1, further comprising a computing device determining a recommended implant size based on one or more of the estimated humeral height, estimated humeral head size, or estimated ductal pulp size.

9. The computing device determines one or more feature points on the fractured metaphysis of the humerus based on one or more of the estimated humeral height, estimated humeral head size, or estimated canal size. The method of operation according to claim 1, further comprising a computing device determining the implant height of the humeral head component based on one or more feature points on the fractured metaphysis.

10. It is a medical system, A memory configured to store image data of a joint including a portion of the diaphysis of the humerus in a pathological state, It includes a processing circuit, and the processing circuit is In order to determine a three-dimensional (3D) model of the shaft of the humerus, the image data is segmented, Based on the 3D model of the shaft of the humerus, a 3D model of the estimated normal shape of the humerus is determined using statistical shape modeling techniques. Based on the 3D model of the estimated normal shape of the humerus, a region of interest is identified, and segmentation is performed in the region of interest to identify one or more bone fragments in the region of interest, and one or more bone fragments are identified in the image data. A medical system configured to generate an output based on identified bone fragments in the aforementioned image data.

11. The processing circuit is further configured to determine the number of bone fragments present in the image data based on the identification of one or more bone fragments. Here, the output includes a surgical recommendation determined based on the determined number of the one or more bone fragments. The medical system according to claim 10, wherein the surgical recommendation comprises identification of a procedure for immobilizing the joint.

12. The aforementioned processing circuit is Determine the shape of one or more bone fragments in the aforementioned image data, The medical system according to claim 10, further configured to identify the position on the 3D model of the estimated normal shape of the humerus corresponding to the one or more bone fragments, based on the shape of the one or more bone fragments.

13. The aforementioned processing circuit is Determine the displacement angle of one or more bone fragments in the aforementioned image data. The medical system according to claim 10, further configured to determine a surgical recommendation based on the displacement angles of one or more bone fragments present in the image data, wherein the output includes a surgical recommendation.

14. The aforementioned processing circuit is Determine the displacement of one or more bone fragments in the aforementioned image data. The medical system according to claim 10, further configured to determine a surgical recommendation based on the displacement of one or more bone fragments present in the image data, wherein the output includes a surgical recommendation.

15. The output includes the output image. The output image includes a visual representation of at least a portion of the 3D model of the estimated normal shape of the humerus, a visual representation of at least one of the one or more bone fragments, and a visual representation of the humeral head implant. The medical system according to claim 10, wherein the processing circuit is further configured to generate annotations on the output image, the annotations identify a portion of a visual representation of the estimated normal shape of the humerus corresponding to one of one or more bone fragments.

16. The processing circuit compares the image data of the joint with an image of the 3D model of the estimated normal shape of the humerus. Based on the comparison, the fracture location in the joint image data is identified. Determine the shape of one or more bone fragments in the aforementioned image data, The medical system according to claim 10, further configured to determine a predicted location corresponding to one or more bone fragments based on the shape of one or more bone fragments and the identified fracture location.

17. The medical system according to claim 10, wherein the processing circuit is further configured to determine a recommended implant size based on one or more of the estimated humeral height, estimated humeral head size, or estimated ductal medullary size.

18. The processing circuit determines one or more feature points on the fractured humeral shaft based on one or more of the estimated humeral height, estimated humeral head size, or estimated canal size. The medical system according to claim 10, further configured to determine the implant height of the humeral head component based on one or more feature points on the fractured diaphysis.

Images