Automatic placement and estimation of a reference grid for an anatomical coordinate system

Abstract

A system and method for a computerized framework are disclosed that provide a novel mechanism for determining the automatic placement of a bone reference grid and an anatomical reference frame (ARF). The disclosed framework is operable for activation of a computerized mechanism that can determine, provide, and / or display the anatomically correct positions of femoral tunnels and / or other forms of surgical landmarks that a surgeon relies on for anterior cruciate ligament (ACL) procedures, based on a three-dimensional (3D) model of the distal femur. The disclosed framework is also operable for activation of a computerized mechanism that can determine, provide, and / or display the anatomically correct positions of tibial tunnels and / or other forms of surgical landmarks that a surgeon relies on for ACL procedures, based on a three-dimensional (3D) model of the proximal tibia.

Description

Technical Field

[0001] (Cross - Reference to Related Applications) This application claims the benefit of U.S. Provisional Patent Application No. 63 / 354,953, filed on June 23, 2022, entitled "Automatic Placement of Bernard - Hertel’s grid and Estimation of Anatomical Coordinate System in a 3D Model of a Distal Femur." This provisional application is hereby incorporated by reference herein as if fully reproduced below.

[0002] This disclosure relates to preoperative and intraoperative surgical analysis and treatment, and more specifically, to methodologies for determining an anatomical reference frame (ARF) of bone, and to the automatic placement of reference grids such as the Bernard - Hertel (BH) grid and / or the tibial grid.

Background Art

[0003] The anterior cruciate ligament (ACL) is one of the major ligaments that provides stability to the knee joint. Playing sports involving sudden stops or changes in direction is one of the main causes of ACL injury, and one example is a complete rupture. For this reason, ACL rupture is a common medical condition with over 200,000 cases per year in the United States alone. The standard method of treatment is arthroscopic reconstruction, in which the ruptured ligament is replaced by a tissue graft that is pulled into the knee joint through tunnels drilled in both the femur and the tibia. By opening these tunnels in anatomically correct positions, knee stability and patient satisfaction are ensured, but the current failure rate in primary ACL reconstruction ranges from 10% to 15%.

[0004] The position and orientation of the femoral tunnel significantly affect the success of surgery, promoting the need for preoperative planning to properly define the optimal femoral tunnel. Compared to the femoral side, significantly less attention has been directed to the surgical techniques for accurate tibial tunnel formation. However, recent studies have emphasized the importance of the tibial tunnel position, and it has been shown that the tibial tunnel position affects the outcome of an anatomical ACL reconstruction. A tibial tunnel placed extremely anteriorly may result in increased graft inclination and subsequent impingement, while a graft placed extremely posteriorly may result in increased anterior translational laxity.

SUMMARY OF THE INVENTION

[0005] Referring first to the femur, to determine the anatomically correct position of the femoral tunnel in the related art, some surgeons rely on specific anatomical landmarks. However, studies have suggested that these landmarks are unreliable and may not even exist for some patients. To obtain a more accurate femoral tunnel position, the Bernard Helfer (BH) grid can be utilized. The BH grid proposes an ACL reconstruction technique and can be used to evaluate the tunnel placement after ACL reconstruction.

[0006] As background, the BH grid involves a quadrant method for determining the femoral insertion site. Using a radiographic lateral image, the Blumensaat line can be identified, and two other lines perpendicular to that line can be drawn passing through the shallow and deep edges of the lateral femoral condyle. The fourth line drawn is parallel to the Blumensaat line and touches the inferior edge of the condyle. The resulting BH grid consists of a normalized reference frame independent of the size, shape, and distance of the knee from which the X-ray was obtained. The coordinates on this reference frame are given as percentages along the Blumensaat line and in the vertical direction.

[0007] Some current research has suggested that, in addition to tunnel position, the orientation of the femoral tunnel can also play an important role in the success of ACL reconstruction surgery. Tunnel orientation is defined as the direction with respect to three anatomical directions (i.e., the sagittal, axial, and coronal directions) that define an Anatomical Reference Frame (ARF) and / or Anatomical Coordinate System (ACS) of the bone.

[0008] Accordingly, the disclosed systems and methods provide a novel framework that automatically places the BH grid and determines the ARF of the bone, considering a three-dimensional (3D) model of the distal femur. The disclosed method is applicable to different femur shapes and sizes and is reliable enough to replace the manual process with negligible errors.

[0009] In fact, the BH grid system was originally applied to the lateral radiographs of the knee, and this approach is still being used in recent research. However, there are studies showing that the image quality and orientation of the X-ray tube with respect to the patient can affect the accuracy of tunnel position measurement. It has also been observed that computed tomography (CT) imaging may be more reliable for performing this task. A common aspect among all the related art methodologies is that, regardless of whether radiographic or CT imaging is used, the placement of the BH grid is a manual process plagued by observer variability. For example, such a process is unreliable because the variability in measuring the femoral tunnel position remains non-negligible, despite the clearly improved inter-observer agreement (when compared to radiographs) obtained from CT scans.

[0010] Currently, there are several related art methods for automatically estimating the ARF in an automated fashion from a 3D model of the femur that are hypothesized to be more reliable than manual approaches, particularly due to eliminating variability within and between observers. These automated related art methods rely on a template model or atlas, use the entire femur including the femoral head, and / or perform a cylindrical / spherical or elliptical fitting to the posterior condyles. Despite being fully automated and presenting good reliability, by relying on prior knowledge (e.g., pre - constructed models of the femur and / or the entire template model / atlas), the related art approaches require more information than the disclosed method, are not general, and tend to suffer from local minimum problems. Further, by obtaining the sagittal direction through the fitting of a cylinder / sphere / ellipse to the condyles, it is not guaranteed that the direction will result in a lateral radiographic view where both condyles are perfectly overlapping, which is used for applying the BH grid system.

[0011] For this reason, among others, the related art automated ARF estimation methods are not suitable for replacing the sagittal direction estimation step in an automated BH grid placement algorithm. For example, the related art approach has attempted to provide an automated estimation of the ARF without requiring knowledge of the entire femur model. However, this approach is not only not suitable for generating the sagittal direction, but also does not guarantee condyle overlap. Further, this approach does not have the ability to reliably determine the axial direction.

[0012] The present disclosure at least partially corrects these drawbacks by providing a computerized methodology that automatically provides the placement of the BH grid and determines the ARF given a 3D model of the distal femur. According to some embodiments, as discussed in more detail below, the disclosed method operates by determining the sagittal direction through the alignment of the medial and lateral condyles and then generating a radiograph of the distal femur for detecting the inter - condylar contour. The BH grid is then obtained as a rectangle that contacts this contour and encloses the radiograph of one or both of the condylar regions.

[0013] Accordingly, an improved mechanism is disclosed that provides a new way to improve the placement and reliability efficiency and accuracy of the BH grid. For example, for 21 different femur models, the tunnel input points obtained via the disclosed automated processing method (e.g., algorithm) differ from the manual process by an average of only 0.28 mm ± 0.16 mm. This error is approximately seven times smaller than the smallest error reported for existing automated and manual methods.

[0014] According to some embodiments, the disclosed method continues by automatically determining the axial direction using the sagittal plane estimated to search for the sagittal view of the diaphysis, from which circles can be extracted. By joining the centers of these circles, the axial direction is obtained. The coronal plane is obtained from the cross product between the sagittal direction and the axial direction, providing a complete ARF.

[0015] Unlike the methods of related art for the automatic estimation of the BH grid, the disclosed and developed method does not require alignment with a template model, thereby demonstrating an operation that works with less information and assumptions, and thereby being able to find the sagittal direction in a more efficient and accurate manner without requiring initialization. Indeed, as discussed herein, the disclosed method does not tend to suffer from the problem of local minima. Also, the disclosed detection of the Blumensaat line is applicable to different types of morphologies because it does not depend on the curvature pattern of the intercondylar contour. Similarly, compared to existing automatic approaches for ARF estimation, the disclosed executable method is applicable to a wider range of input models because it does not require prior information or the entire femur model.

[0016] Referring now to the tibia, the measurement of the position of the tibial tunnel has recently been achieved using a 3D model of the proximal tibia obtained from a CT scan or a three-dimensional volume rendering image in which a rectangular grid is arranged to surround the tibial plateau, considering the axial view of the bone. This approach is called the quadrant method. The first step of the algorithm is to define the axial direction that is considered the normal to the tibial plateau. Then, from the axial view of the tibia, the approach is to find a line that is tangent to the posterior contour of the plateau and obtain the remaining part of the grid such that the sides of the rectangle are tangent to the anterior, medial, and lateral contours of the plateau. The resulting grid consists of a normalized reference frame that is independent of the size and shape of the knee. The coordinates of this reference frame are given as percentages along the anterior-posterior (AP) and medial-lateral (ML) directions.

[0017] The purpose of the tibial grid is to find the internal point of the ACL tibial tunnel. To fully define the tunnel, the external points are also determined. Recent studies have proposed that, considering the location of the tibial rough surface for the task of finding the external point, the external point should be within 1 cm to 2 cm medial to the tibial tubercle.

[0018] Related art approaches for defining and measuring the position of the tibial tunnel also rely on knee radiographs. However, recent studies have also shown that, due to the inadequate position of the knee relative to the x-ray tube, the use of radiographic images may not be reliable, and CT scans and three-dimensional volume rendering images provide more reliable results. The quadrant method with a 3D model of the proximal tibia is the most widely used related art technique for measuring the position of the internal point of the tibial tunnel. An aspect common to many related arts is that the placement of the tibial grid is a manual process, which is time-consuming and necessarily plagued by observer variability.

[0019] The position of the tibial rough surface may be used to determine around the external opening of the tibial tunnel and is often determined by visual observation of the patient.

[0020] This document discloses a method that automatically provides the placement of a tibial grid, an anatomical reference frame of the tibia, and the position of the tibial tuberosity when a 3D model of the proximal tibia is provided. The method is robust against different tibial shapes and sizes and has sufficient reliability with negligible errors to replace a manual process.

[0021] The systems and methods of the present disclosure provide a computerized framework that addresses current drawbacks in existing technologies, particularly by providing novel mechanisms for BH grid, automatic placement of the tibial grid, and automatic determination of ARF for the femur and tibia.

[0022] According to one or more embodiments, the present disclosure provides a non-transitory computer-readable storage medium for performing the above-described technical processes. The non-transitory computer-readable storage medium tangibly stores thereon or is tangibly encoded thereon computer-readable instructions that, when executed by a device, cause at least one processor to implement a method for providing novel mechanisms for automatic placement of the BH grid and automatic determination of ARF.

[0023] According to one or more embodiments, there is provided one or more computing devices and / or a system comprising the devices configured to provide functionality in accordance with such embodiments. According to one or more embodiments, the functionality is embodied in steps of a method implemented by one or more computing devices and / or the devices. According to one or more embodiments, program code (or program logic) executed by a processor of a computing device to implement the functionality in accordance with one or more such embodiments is embodied in, thereby, and / or on a non-transitory computer-readable medium.

[0024] The features and advantages of the present disclosure will become apparent from the following description of the embodiments illustrated in the accompanying drawings, and the reference characters refer to the same parts throughout the various drawings. The drawings are not necessarily to scale, and instead, emphasis is placed on illustrating the principles of the present disclosure.

Brief Description of the Drawings

[0025]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10A

Figure 10B

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

DETAILED DESCRIPTION OF THE INVENTION

[0026] Here, the present disclosure will be described more fully hereinafter with reference to the accompanying drawings, which form a part of this specification and show, by way of non-limiting illustration, specific exemplary embodiments. However, the subject matter may be embodied in a variety of different forms, and thus, it is intended that the subject matter covered or claimed is not limited to any of the exemplary embodiments described herein, and the exemplary embodiments are provided merely by way of illustration. Similarly, a reasonably broad scope of the claims or the subject matter is intended. In particular, for example, the subject matter may be embodied as a method, a device, a component, or a system. Accordingly, the embodiments may take the form of, for example, hardware, software, firmware, or any combination thereof (other than software itself). Thus, the following detailed description is not intended to be taken in a limiting sense.

[0027] Throughout this specification and the claims, terms may have nuanced meanings that are suggested or implied in the context beyond their explicitly stated meanings. Similarly, as used herein, the phrase "in one embodiment" does not necessarily refer to the same embodiment, and as used herein, the phrase "in another embodiment" does not necessarily refer to a different embodiment. For example, the claimed subject matter is intended to include combinations of all or some of the exemplary embodiments.

[0028] Generally, terms can be understood at least in part from their use in context. For example, as used herein, terms such as "and," "or," or "and / or" can include a variety of meanings that may depend at least in part on the context in which such terms are used. Typically, when used to associate a list such as A, B, or C, this is intended to mean A, B, and C used in an inclusive sense, as well as A, B, or C used in an exclusive sense. Further, as used herein, the term "one or more" may be used to describe any feature, structure, or property in a single sense, or in a plural sense to describe a combination of features, structures, or properties, depending at least in part on the context. Similarly, terms such as "a," "an," or "the" can be understood to convey a single use or a plurality of uses, depending at least in part on the context. Additionally, the term "based on" may not be intended to necessarily convey an exclusive set of factors, but rather may allow, depending at least in part on the context, for the presence of additional factors that are not necessarily explicitly described.

[0029] The present disclosure will be described below with reference to block diagrams and operational diagrams of methods and devices. It is understood that each block of the block diagram or operational diagram, and combinations of blocks of the block diagram or operational diagram, can be implemented by analog or digital hardware and computer program instructions. These computer program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, an ASIC, or other programmable data processing device to change the functions detailed herein, such that the instructions executed via the processor of the computer or other programmable data processing device implement the functions / operations specified in the block diagram or operational block or block. In some alternative implementations, the functions / operations described in the block may occur out of the order described in the operational diagram. For example, two blocks shown consecutively may actually be executed substantially simultaneously, or the blocks may be executed in the reverse order, depending on the functions / acts involved, in some cases.

[0030] Unless otherwise limited, the terms "connected", "coupled", and "attached" in this specification, and their variations, are widely used and include direct and indirect connections, couplings, and attachments. Further, the terms "connected" and "coupled", and their variations, are not limited to physical or mechanical connections or couplings. Further, terms such as "up", "down", "top", "front", "rear", "upper", "lower", "upwardly", "downwardly", and other directional descriptors are intended to facilitate the description of exemplary embodiments of the present disclosure and are not intended to limit the structure of the exemplary embodiments of the present disclosure to a particular position or direction. Terms of degree such as "substantially" or "approximately" are understood by those skilled in the art to refer to a reasonable range around and including a given value, and a range outside the given value, such as, for example, general tolerances associated with the manufacture, assembly, and use of embodiments. When referring to a structure or characteristic, the term "substantially" includes characteristics that are almost or completely present in the characteristic or structure.

[0031] For the purposes of the present disclosure, a non-transitory computer-readable medium (or computer-readable storage medium / media) stores computer data, which can include computer program code (or computer-executable instructions) executable by a computer in machine-readable form. By way of example and not limitation, the computer-readable medium may comprise a computer-readable storage medium for tangible or fixed storage of data, or a communication medium for transient interpretation of code-containing signals. A computer-readable storage medium, as used herein, refers to physical or tangible storage (as contrasted with signals) and includes any method or technology implemented in volatile and non-volatile, removable and non-removable media for the tangible storage of information such as computer-readable instructions, data structures, program modules, or other data, but is not limited thereto. Computer-readable storage media include RAM, ROM, EPROM, EEPROM, flash memory or other solid state memory technologies, optical storage devices, cloud storage devices, magnetic storage devices, or any other physical or material medium that can be used to tangibly store the desired information or data or instructions and that can be accessed by a computer or processor, but is not limited thereto.

[0032] For the purposes of the present disclosure, the term “server” should be understood to refer to a service point that provides processing, database, and communication facilities. By way of example and not limitation, the term “server” can refer to a single physical processor having associated communication and data storage and database functions, or to a networked or clustered complex of processors and associated networks and storage devices, as well as the operating software and one or more database systems and application software that support the services provided by the server. A cloud server is an example thereof.

[0033] For the purposes of the present disclosure, a network is to be understood to refer to a network that can couple devices such that communications can be exchanged, for example, between a server and a client device, or between other types of devices including between wireless devices coupled via a wireless network. The network may also include mass storage such as, for example, network attached storage (NAS), storage area network (SAN), content delivery network (CDN), or other forms of computer or machine-readable media. The network can include the Internet, one or more local area networks (LANs), one or more wide area networks (WANs), wired type connections, wireless type connections, cellular, or any combination thereof. Similarly, sub-networks that may employ different architectures, or conform to different protocols, or are interoperable can operate with each other within a larger network.

[0034] For the purposes of the present disclosure, a wireless network is to be understood to couple a client device to a network. The wireless network may employ a stand-alone ad hoc network, a mesh network, a wireless local area network (WLAN) network, a cellular network, etc. The wireless network may further employ multiple network access technologies including Wi-Fi, Long-Term Evolution (LTE), WLAN, wireless router (WR) mesh, or second, third, fourth, or fifth generation (2G, 3G, 4G or 5G) cellular technologies, mobile edge computing (MEC), Bluetooth, 802.11b / g / n, etc. The network access technologies can enable wide area coverage for devices such as, for example, client devices having varying degrees of mobility. Briefly stated, a wireless network can include substantially any type of wireless communication mechanism by which signals can be communicated between devices such as client devices or computing devices, between networks, or within a network.

[0035] A computing device may be able to transmit and receive signals, such as via a wired or wireless network, or may be able to process or store signals, such as in memory as a physical memory state, and thus may operate as a server. Thus, examples of devices that can operate as a server include dedicated rack-mounted servers, desktop computers, laptop computers, set-top boxes, and integrated devices that combine various features such as two or more features of the aforementioned devices.

[0036] For the purposes of the present disclosure, a client (or consumer or user) device, referred to as a user equipment (UE), may include a computing device capable of transmitting and receiving signals, such as via a wired or wireless network. Client devices can include, for example, desktop computers, or portable devices such as mobile phones, smartphones, display pagers, radio frequency (RF) devices, infrared (IR) devices, near field communication (NFC) devices, personal digital assistants (PDAs), handheld computers, tablet computers, phablets, laptop computers, set-top boxes, wearable computers, smartwatches, and integrated or distributed devices that combine various functions such as the functions of the aforementioned devices.

[0037] In some embodiments, as described below, the client device may also be communicatively coupled to any type of known or future-known medical device (e.g., for example, MRI devices, CT scanners, electrocardiogram (ECG or EKG) devices, photoplethysmographs (PPG), Doppler and transit time flow meters, laser Doppler, endoscopic devices, neuromodulation devices, nerve stimulation devices, etc., or combinations thereof, but not limited to these), any type of Class I, II, or III medical device.

[0038] System or framework Referring to FIG. 1, a system (or framework) 100 is illustrated that includes a UE 106 (e.g., a client device), a network 102, a cloud system 104, and a surgical engine 200. The UE 106 can be any type of device, such as a mobile phone, tablet, laptop, personal computer, sensor, Internet of Things (IoT) device, autonomous machine, and any other device equipped with a cellular, wireless, or wired transceiver, but is not limited thereto. In some embodiments, as described above, the UE 106 can also be a medical device or another device communicatively coupled to a medical device that enables receipt of readings from sensors of the medical device. For example, in some embodiments, the UE 106 can be a user's smartphone (or an office / hospital device) connected to, for example, a peripheral nerve modulation device via WiFi, Bluetooth Low Energy (BLE), or NFC. Thus, in some embodiments, the UE 106 can be configured to receive data from sensors associated with a medical device, as discussed in more detail below. For further consideration of the UE 106, see at least FIGS. 10A and 10B.

[0039] The network 102 can be any type of network, such as a wireless network, cellular network, Internet, local area network, or wide area network, but is not limited thereto. As discussed herein, the network 102 facilitates the connection of the components of the system 100, as shown in FIG. 1.

[0040] The cloud system 104 can be any type of cloud operating platform and / or network-based system where applications, operations, and / or other forms of network resources can be located. For example, the system 104 can correspond to a service provider, network provider, and / or medical provider, from which services and / or applications can be accessed, sourced, or executed. In some embodiments, the cloud system 104 can include servers and / or databases of information that are accessible via the network 102. In some embodiments, a database (not shown) of the system 104 can store a dataset of data and metadata associated with local and / or network information related to users, patients of the UE 106, and the UE 106, as well as services and applications provided by the cloud system 104 and / or the surgical engine 200.

[0041] The surgical engine 200 includes components for determining the automatic placement of a reference grid of bones, such as a BH grid and an ARF-related tunnel arrangement through the femur, or a reference grid and an ARF-related tunnel arrangement through the tibia, as discussed in more detail below. Embodiments of how the engine 200 operates and functions, as well as embodiments of the capabilities that the engine 200 includes and executes, are discussed in more detail below in particular in relation to FIGS. 3-8.

[0042] According to some embodiments, the surgical engine 200 may be a special-purpose machine or processor and may be hosted by devices on the network 102, within the cloud system 104, and / or on the UE 106. In some embodiments, the engine 200 may be hosted by a peripheral device (e.g., a medical device as described above) connected to the UE 106.

[0043] According to some embodiments, the surgical engine 200 may function as an application provided by the cloud system 104. In some embodiments, the engine 200 may function as an application installed on the UE 106. In some embodiments, such an application may be a web-based application accessed by the UE 106 via the network 102 from the cloud system 104 (e.g., as shown by the connection between the network 102 and the engine 200 and / or the dashed line between the UE 106 and the engine 200 in FIG. 1). In some embodiments, the engine 200 may be configured and / or installed as an extension script, program or application (e.g., a plugin or extension) to another application or program provided by the cloud system 104 and / or executed on the UE 106.

[0044] As shown in FIG. 2, according to some embodiments, the surgical engine 200 includes a model module 202, an estimation module 204, a placement module 206, and a display module 208. Of course, the engines and modules discussed herein are non-exhaustive as additional or fewer engines and / or modules (or sub-modules) may be applicable to embodiments of the systems and methods discussed. The operations, configurations, and functions of each of the engine 200 and its modules, and their roles within the embodiments of the present disclosure, will be described in more detail below. This specification first focuses on the processes related to the femur.

[0045] Reference Grid and Anatomical References for Femur Referring to FIG. 3, a process 300 is illustrated that details a non-limiting and exemplary embodiment of the computerized operations of the disclosed framework for determining the ARF of a bone according to the automatic placement of the BH grid.

[0046] According to some embodiments, as discussed herein, a framework (e.g., engine 200) executes a methodology (provided at least via process 300) that operates by determining a sagittal direction through alignment of the medial and lateral condyles, and may then generate a radiograph of the distal femur for detecting the intercondylar contour. The BH grid then contacts this contour and is obtained as a rectangle surrounding the radiograph of one or both condyles. The disclosed method continues by automatically determining the axial direction by obtaining a sagittal view of the diaphysis using the estimated sagittal plane, from which a circle can be extracted. By firmly joining the centers of these circles, the axial direction is obtained. A coronal plane is obtained from the cross product between the sagittal direction and the axial direction, providing a complete ARF.

[0047] The disclosed framework, realized through execution of the engine 200 for the operations detailed as part of process 300, is applicable to different femur shapes and sizes, and its improved reliability, accuracy, and ease of implementation can replace the conventional processing medicine that experts currently rely on.

[0048] According to some embodiments, step 302 of process 300 can be performed by the model module 202 of the surgical engine 200, steps 304 - 306 and 310 - 312 can be performed by the estimation module 204, steps 308 and 314 can be performed by the placement module 206, and step 316 can be performed by the display module 208.

[0049] Process 300 begins at step 302, where engine 200 receives an input that identifies a 3D model of the distal femur. According to some embodiments, the identification of the 3D model may be based on, but is not limited to, a request to generate a 3D model, a search for and acquisition of a 3D model, and / or an upload and / or download of a 3D model. In some embodiments, the input may be in the form of an image, a message, a multimedia item, and / or any other type of format known or to be known in the future for engine 200 to receive and display digital content corresponding to a model of the patient's bone, particularly the distal femur (e.g., a 3D model).

[0050] The discussion herein focuses on 3D models of the distal femur, but it should be understood that it is not to be construed as limiting, as models of other types, styles, and forms of other bones (e.g., the proximal tibia discussed below) may be utilized without departing from the scope of the present disclosure.

[0051] At step 304, engine 200 performs an estimation of the sagittal direction based on the input received from step 302. According to some embodiments, step 304 involves an engine that estimates the sagittal direction of the bone based on pairs of points where the normal vectors are orthogonal to the vector joining them (e.g., without registration or initialization, as in conventional methodologies). According to some embodiments, this is illustrated in example 400 of FIG. 4, where exemplary distal femur 402 is identified on pairs of points P1 and P2, and normal vectors N1 and N2, respectively.

[0052] According to some embodiments, step 304 may involve calculating a normal for each point (or at least one set of points on the bone) based on the input of the 3D model. According to some embodiments, step 304 can be restricted within the search domain when searching for points by finding a region of interest (ROI) that includes the surface of the condyle, where only the points on that ROI (instead of the complete 3D model) are considered. In some embodiments, the ROI can be found by registering the 3D model to a template model, by utilizing a statistical shape model (SSM), through 3D curvature analysis, by using a deep learning framework, or by some combination of them.

[0053] The engine 200 can then analyze the calculated normals and determine pairs of points (e.g., P1 and P2) where the corresponding normals (N1 and N2) are parallel and the vector "v" connecting P1 and P2 is orthogonal to N1 and N2, as shown in FIG. 4.

[0054] Next, the engine 200 can determine the sagittal direction based on the following hypothesis: v = P2 - P1. According to some embodiments, the vector v connecting each selected pair of points consists of a hypothesis for the sagittal direction. Based on such sagittal direction hypotheses, an estimation of the sagittal direction can be determined. In some embodiments, the hypotheses are represented as 3D points and can then be clustered, whereby the median of the cluster can be calculated. In some other embodiments, random sample consensus (RANSAC), or other robust estimation models (e.g., Hough transform), can be applied to the set of hypotheses to estimate the sagittal direction.

[0055] According to some embodiments, step 304 may further involve determining the medial-to-lateral orientation with respect to the sagittal direction by identifying the outer and inner condyles (e.g., P1 and P2 respectively).

[0056] In some embodiments, the estimation of the sagittal direction may further include refining the sagittal direction by generating a simulated radiographic view of the femur or a two-dimensional (2D) cross map (as discussed below, at least in relation to step 306 and FIG. 5), and adjusting the outer boundary of the condyle so that they overlap. Such generated views / maps can be generated by considering orthographic or perspective projections.

[0057] Process 300 proceeds from step 304 to step 306, where, after determining the sagittal direction estimate (step 304), engine 200 performs an estimation of the Blumensaat line. According to some embodiments, engine 200 accesses a 3D model of the femur and constructs a 2D projection of the number of intersections between the projection rays and the 3D model (referred to as a "cross map"). A non-limiting example of such mapping is provided in FIG. 5, and cross map 500 includes regions 502, 504, and 506, as well as curve 508. As shown in FIG. 5, region 502 corresponds to zero (0) intersections, region 504 corresponds to a region having two (2) intersections, and region 506 corresponds to a region having four (4) intersections. According to some embodiments, cross map 500 enables the identification of curve 508 (e.g., curve "C"). Curve 508 corresponds to the contour of the intercondylar region, which can be determined by performing edge detection analysis (or calculation) and searching for a curve between regions having intersection points 2 and 4 (e.g., between regions 504 and 506 in FIG. 5, respectively).

[0058] According to some embodiments, the determination of the Blumensaat line in step 306 can include finding a straight line that touches curve 508 at the maximum number of points that do not intersect curve 508. In other words, regardless of the shape, the Blumensaat line touches curve 508 but does not intersect curve 508 despite being in the tangent direction. This is a novel executable step compared to different types of existing Blumensaat line forms in that it does not depend on a specific curvature pattern, unlike previous methods.

[0059] Referring to FIG. 6, it is illustrated in a 2D projection 600 of the distal femur from a side view, and the Blumensaat line 602 is shown relative to the curve 508. Thus, as shown in FIG. 6 and described herein in connection with step 306, the Blumensaat line touches the curve 508 at the maximum number of points and does not intersect the curve 508 elsewhere. According to some embodiments, the Blumensaat line can be determined based on the use of a template model / SSM, 2D curvature analysis, deep learning schemes, voting schemes, clustering, and / or any other type of known or future-known heuristics.

[0060] In some embodiments, in situations where the curve 408 is mound-shaped, the Blumensaat line can intersect some regions of the intercondylar contour and touch it only at a designated location (e.g., near the intercondylar notch). In some embodiments, the Blumensaat line may be further based on the back-projection of the 2D intersection map 500 onto points on the 3D model. In some embodiments, the back-projection can be based on a cross-sectional plane defined by the sagittal direction (from step 304). Thus, according to some embodiments, 3D points within the 3D model can be obtained / determined by back-projecting the intercondylar contour / Blumensaat line onto the 3D model. Having such points, for example, an appropriate cross-sectional plane of the model can be obtained based on a plane having a sagittal direction that includes such points. In some embodiments, such cut planes can be used when determining the axial direction through a circular fitting within the shaft, as discussed below.

[0061] Next, process 300 proceeds from step 306 to step 308, and engine 200 determines the placement of the BH grid. Engine 200 determines the placement (and other characteristics such as, for example, size, ratio, and dimensions) of the BH grid on the estimated values of the sagittal direction (from step 304) and the Blumensaat line (from step 306).

[0062] According to some embodiments, step 308 includes an engine 200 that places a BH grid so as to surround the condyle when shown in a lateral view of the distal femur. According to some embodiments, edge detection is applied to an intersection map (e.g., map 500 of FIG. 5), and an edge corresponding to a curve surrounding the region of zero crossings is obtained (corresponding to the sagittal contour of the condyle). In other words, when considering the orthographic projection, the engine 200 utilizes a 2D projection of the number of intersections of the bone model with projection rays along the sagittal direction. Next, the Blumensaat line (line 602 in FIG. 6) intersects the obtained contour, and the long side of the BH grid is obtained. Finally, a line parallel to the Blumensaat line tangent to the contour is obtained, and the distance between the two lines is the length of the short side (width) of the BH grid. An example of this is illustrated in FIG. 7, in which a 2D projection 700 including the Blumensaat line 602 and the BH grid 702 is shown. According to some embodiments, the projection 700 can be an intersection map obtained from any of a radiographic view, an orthographic projection, or a perspective projection. In some embodiments, if the position of the lateral condyle is known, the edge detection of step 306 can be achieved by first slicing the model sagittally and considering only the lateral condyle for constructing the intersection map.

[0063] In some embodiments, step 308 may further include back-projecting the BH grid from the 2D model to the 3D model. In such embodiments implemented at step 310, the engine 200 can perform the back-projection such that the BH grid is displayed as part of the 3D model or as an overlay. In some embodiments, the back-projection can be based on a cross-sectional plane defined by the sagittal direction and the Blumensaat line.

[0064] Next, process 300 proceeds from step 310 to step 312, where the engine 200 determines the estimation of the axial and coronal directions (e.g., the remaining anatomical directions). As discussed herein, the axial and coronal directions are utilized to determine the ARF.

[0065] According to some embodiments, step 312 can include a set of sub-steps. The first sub-step involves the engine 200 obtaining a sagittal view of the bone whose shaft profile is to be obtained. In some embodiments, the sagittal view may be an orthographic projection of the entire femur model or a cross-section, or may be obtained from the intersection of the cross-sectional plane and the model. In some embodiments, the sagittal view may be an intersection map (as described above with respect to FIG. 5), and the shaft profile can be obtained based on the transition between 0 and 2 intersection regions (e.g., regions 502 and 504 respectively).

[0066] In the next sub-step, the engine 200 searches for a circle that is in the tangential direction (e.g., simultaneously in the tangential direction) with respect to the shaft profile, and the line joining the centers of the obtained circles provides an estimate of the axial direction. In some embodiments, if the anterior and posterior cortices of the femur are known, the search can be restricted by considering only pairs that contain one point from each cortex. According to some embodiments, the axial direction can alternatively be determined based on a determined relationship between a fixed angle (by a predetermined value) with respect to the Blumensaat line. In some embodiments, the axial direction can alternatively be determined based on a cylinder fitting methodology that utilizes the dimensions and values of the shaft region.

[0067] Next, the engine 200 can determine the coronal direction via the cross product between the sagittal direction and the axial direction.

[0068] FIG. 8 shows an example of a femur model 800 in which circles 802 and 804 are illustrated. Circles 802 and 804 are tangent to the shaft profile as described above and can be obtained as follows. First, the normal at each point of the shaft profile is calculated. Next, all pairs of contour points are generated, and the lines passing through the contour points parallel to their respective normal vectors intersect. According to FIG. 8, the line in the direction n A containing the point P A is on the center of the circle 802 belonging to the axial direction and contains the point P P in the direction n PIt intersects the line of . The axial direction is obtained by joining all the intersection points that are equidistant from the considered point, corresponding to the center of the circle tangent to the shaft contour.

[0069] According to some embodiments, the step of joining the points can be performed using any known or future-known technique, algorithm, or mechanism such as standard or robust line fitting, clustering schemes, Hough transform, and / or any other known or future-known technique for estimating and determining lines (and their distances / lengths) from a set of points, but is not limited thereto.

[0070] Returning to process 300, process 300 proceeds from step 312 to step 314, and engine 200 generates an ARF for the distal femur. As shown in FIG. 9, an example of the generated ARF 900 is illustrated, including a sagittal direction 906, an axial direction 904, and a coronal direction 902.

[0071] In step 316, the generated ARF can be displayed as an overlay or part of a 3D model that can be used for an ACL procedure, as discussed above. In some embodiments, information related to the ARF, direction, BH grid, and Blumensaat line can be saved and utilized for subsequent ARF projections.

[0072] Thus, at least based on the above considerations, the disclosed methodology, unlike the prior art, functions without the need for alignment with a template model or initialization of the sagittal direction. In fact, the disclosed method can be implemented without the entire femur model and does not depend on the curvature pattern of the intercondylar contour. This, among other advantages, demonstrates a system that makes the disclosed methodology applicable to a more diverse range of input models and different types of morphologies, is not prone to local minimum problems, and functions in a more computationally efficient and accurate manner.

[0073] Reference Grid and Anatomical References for the Tibia Referring to FIG. 10 (shown in FIGS. 10A and 10B), process 1000 details a non-limiting example of the computerized operation of the disclosed framework for the placement of the tibial grid and the determination of the ARF.

[0074] According to some embodiments, as discussed herein, a framework (e.g., engine 200) can execute a methodology (provided at least via process 1000) that operates by determining the locations of the tibial plateau and the tibial tuberosity. In particular, the methodology takes as input a 3D model of the proximal tibia and outputs a rectangle surrounding the tibial plateau (e.g., the tibial grid) that directly provides estimates in the 3D axial, sagittal, and coronal directions, as well as the location of the tibial tuberosity, all of which are calculated in an automatic and unsupervised manner. The tibial grid can be used by a surgeon to determine the footprint of the ACL using any of a number of studies that provide this location in normalized coordinates, and the location and axis of the tuberosity are useful in determining the tunnel outside orifice.

[0075] Considering that the quadrant method is applied to the tibial plateau, in various examples, the first step consists of a coarse segmentation of the tibial plateau points, followed by a fine segmentation. Then, to determine the axis direction, robust plane fitting is applied. From the axial view of the proximal tibia, a line that is a double tangent to the posterior contour of the plateau is estimated, and then parallel and perpendicular lines are determined with respect to the posterior, medial, and lateral plateau contours, providing the grid placement. The last step of the automatic algorithm consists of tibial tuberosity localization.

[0076] The tunnel entry points obtained using the method with 13 different femur models differ from the manual process by an average of 0.55 mm ± 0.35 mm. The error is approximately four times smaller than the minimum error reported in the literature for the procedures of the related art. Regarding the semi - automatic grid placement algorithm of the related art, the disclosed approach also presents very favorably compared, with average errors for the anterior - posterior distance and the medial - lateral distance between the automatic and manual placements of 0.63 mm instead of 2.4 mm and 0.59 mm instead of 1.6 mm, respectively. The method disclosed herein does not depend on user intervention, on large reconstructions of the tibial shaft, on specific patient positioning during image acquisition, or on the availability of 3D models of other knee components such as the patella. For these reasons, the disclosed methodology is applicable to a more diverse set of input models.

[0077] The disclosed framework, realized through the execution of the engine 200 of the operations detailed as part of process 1000, is applicable to different tibial shapes and sizes, and its improved reliability, accuracy, and ease of implementation can replace the processing methods of the related art that experts currently rely on.

[0078] According to some embodiments, step 1002 of process 1000 can be performed by the model module 202 of the surgical engine 200, steps 1004 - 1010 and 1018 can be performed by the estimation module 204, steps 1010 - 1014 can be performed by the placement module 206, and part of step 1016 and step 1018 can be performed by the display module 208.

[0079] Process 1000 begins with step 302 where engine 200 receives a 3D model of the proximal tibia. According to some embodiments, the reception of the 3D model may be based on, but not limited to, a request to generate a 3D model, a search for and acquisition of a 3D model, and / or an upload and / or download of a 3D model. In some embodiments, the input may be in the form of an image, a message, a multimedia item, and / or any other type of form, known or later to be known, of engine 200 for receiving and displaying digital content corresponding to a model of a patient's bone, particularly the proximal tibia (e.g., a 3D model). As previously described, although the discussion herein focuses on the 3D model of the proximal tibia, it should not be construed as limiting as other types of bone model types, forms, and morphologies can be utilized without departing from the scope of the present disclosure.

[0080] In step 1004, the engine 200 identifies the first approximate tibial axis and the deepest point (hereinafter, the lowest point) of the tibial plateau. In some embodiments, identifying the first approximate tibial axis and the lowest point of the tibial plateau may involve deforming a statistical shape model (SSM) corresponding to the 3D model to create a deformed SSM, and identifying the first approximate tibial axis and the lowest point of the tibial plateau from the deformed SSM. The SSM in this context is itself a three-dimensional model created based on a plurality of models of the proximal tibia, and the models are statistically combined to reach an SSM representing the standard shape and size of the proximal tibia. The SSM has its tibial axis and the previously identified lowest point. In this case, "morphing" may refer to adjusting or modifying the shape and / or scale of the SSM in a three-dimensional coordinate system so as to correspond to a 3D model with a certain degree of certainty. Thus, in some embodiments, the morphing can be repeated in an iterative process, morphing, confirmation of correspondence, and until the correspondence meets or exceeds a predetermined correspondence. Since the deformed SSM is only used to find the first approximate tibial axis and the lowest point of the tibial plateau, the predetermined degree of correspondence may be relatively low (e.g., 0.6 - 0.8). By morphing the SSM to be similar to the 3D model, the approximate axial direction of the 3D model, as well as the apex of the tibial plateau, is determined, and the lowest point is located on the inner plateau of the overall tibial plateau.

[0081] In yet other embodiments, identifying the first approximate tibial axis and the lowest point of the tibial plateau may involve the use of a template model, an atlas, a deep learning framework, or the like. In yet other embodiments, identifying the first approximate tibial axis may involve performing principal component analysis (PCA) on the 3D model to determine the three most dominant, mutually orthogonal directions. Thus, the three most dominant, mutually orthogonal directions correspond to the three anatomical planes. Identifying the first approximate tibial axis and the lowest point of the tibial plateau will be discussed in more detail below with visual references.

[0082] In step 1006, the engine 200 segments the 3D model using the first approximate tibial axis and the lowest point of the tibial plateau to create segmented data. In one embodiment, segmenting the 3D model includes selecting data points from the 3D model that are within and above the segmentation plane, which is perpendicular to the first approximate tibial axis and is a predetermined distance below or distal to the lowest point of the tibial plateau. Otherwise, all points that are above or proximal to a specified distance along the first approximate tibial axis (e.g., 5 mm below or distal to the lowest point) from the first approximate tibial axis and the lowest point are selected as the segmented data. In this way, the segmented data is likely to include all the data points of the tibial plateau, but additional data points exist as long as the segmentation is based on the first approximate tibial axis.

[0083] Note that segmentation can take many suitable forms. In some embodiments, data that is within and / or above the segmentation plane may be extracted and placed within a new data file or at a memory location different from the unsegmented data. However, in other cases, data that is within and / or above the segmentation plane may be identified in some suitable way within the 3D model, but the 3D model may retain all the original data. Thus, segmentation may alternatively refer to identifying data that is within and / or above the segmentation plane and should not be read to mean separation of the data. In other words, the segmented data need not be separate data. The use of the first approximate tibial axis and the lowest point of the tibial plateau to segment the 3D model to create segmented data will be described in more detail below with visual reference.

[0084] Coarse segmentation that yields segmented data may include points belonging to the diaphysis of the tibia. In some cases, the segmented data may also include points belonging to the fibula and may not be something to be considered when placing the tibial grid. For this reason, the exemplary methodology further refines the determination of the tibial plateau as discussed in exemplary steps 1008 and 1010.

[0085] Accordingly, in step 1008, the engine 200 determines or outlines the outer perimeter of the tibial plateau from within the segmented data. In one embodiment, outlining the outer perimeter of the tibial plateau includes identifying a contour region within the segmented data and assigning an outer perimeter based on the contour region. More specifically, in one embodiment, the refinement functions by determining the tibial plateau and the 3D contour within the tibial plateau. To determine the 3D contour in this example, a filter having a high response to regions of large 3D curvature is applied to the segmented data, and points having a curvature above a predetermined value are specifically selected. In one embodiment, a set of control points is selected using the deformed SSM, and the control points are a set of high-curvature points of the deformed SSM. Next, the entire contour or outer perimeter is obtained by selecting the maximum curvature points within the segmented data that do not exceed a predetermined distance (e.g., 2 mm) from the control points. Otherwise, the outer perimeter of the tibial plateau can be considered to be a line passing through a plurality of discrete data points that lie on the selected points.

[0086] Thereafter, in one embodiment, points inside the outer perimeter are selected, for example, by performing an orthographic projection along the first approximate tibial axis. That is, data points from the segmented data that are present inside the outer perimeter are identified, for example, by orthographic projection, resulting in a set of data points that includes the data points of the outer perimeter and at least a portion of the segmented data. Determining or depicting the outer perimeter of the tibial plateau from within the segmented data will be considered in more detail below with visual reference.

[0087] In step 1010, the engine 200 conforms to the tibial plane based on at least a portion of the outer perimeter and segmented data. A reference perpendicular to the tibial plane defines the final tibial axis. In one embodiment, considering the outer perimeter of the tibial plateau and a portion of the segmented data present within the outer perimeter, a robust plane fitting is performed to determine a plane that includes at least some of the data points of the outer perimeter and at least some of the segmented data. In some cases, the 3D model of the tibial plateau may include the cervical spine spine, and thus, instead of considering all the data points that fit the plane, a robust scheme may perform the selection of the inner layer as part of the plane fitting process. Thus, the tibial plane so determined mathematically defines the "final" position of the tibial plateau, and the normal to the plane defines the final tibial axis. In one embodiment, since the motion information is the direction rather than the specific position of the tibial axis, the "final tibial axis" is simply any line perpendicular to the tibial plateau.

[0088] In another embodiment, the tibial plane can be determined directly from the 3D model. That is, an exemplary robust plane fitting can be performed across the entire 3D model as received in step 1002. Any suitable plane fitting algorithm such as a voting scheme, Hough transform, clustering, hypothesis testing algorithm, etc. may be used. Thus, an exemplary direct approach may omit the use of the SSM to determine the first approximate tibial axis. In some cases, the PCA analysis discussed above can be used to limit or restrict the search and plane fitting with respect to the entire 3D model. When determining the tibial plane based on a robust plane fitting performed on the 3D model by PCA analysis, the tibial plane so determined directly provides the axial direction via the normal to the tibial plane, and thus this alternative eliminates the need to use the SSM to determine the first approximate tibial axis.

[0089] In yet other embodiments, the tibial plane, regardless of how it is identified, can be determined by planar fitting only to the data defining the outer perimeter outlined at step 1008. The outer perimeter data may be used as an exclusive data set provided to a planar fitting algorithm from which the tibial plane is determined. Any suitable planar fitting algorithm such as a voting scheme, Hough transform, clustering, hypothesis testing algorithm, etc. may be used.

[0090] Regardless of the exact methodology used to provide the tibial plane and the final tibial axis, the next step in the exemplary methodology is the placement of the tibial grid. Exemplary steps 1012 and 1014 implement the placement. At step 1012, the engine 200 finds a double tangent at a rear location based on the outer perimeter and a portion of the segmented data. In one embodiment, finding the double tangent may include orthogonally projecting the outer perimeter and the portion of the segmented data within the outer perimeter onto the tibial plane to create projected data, and finding a double tangent on the rear side of the projected data.

[0091] That is, to determine a line that doubly contacts the rear side or the rear contour of the tibial plateau, the method projects onto an axial plane via an orthographic projection. Next, the data points on the rear side of the outer perimeter are selected, in one example, with the aid of an approximate anterior-posterior direction obtained using the SSM. Next, the rear contour may be divided into inner and outer sides, and all lines are generated that include one point from each side having equal tangents. The line that is the double tangent is selected as the line on which all points either exist on the line or on the same side of the line and whose direction is equal to the direction of the tangent.

[0092] In step 1014, the engine 200 forms a tibial grid including a rectangle having a first long side that coincides with or is coaxial with the double tangent, and includes a second long side that is parallel to the first long side and identifies the outermost edge on the front side of the outer periphery, and a first short side that is perpendicular to the first long side and identifies the innermost part of the outer periphery, and a second short side that is perpendicular to the first long side and identifies the outermost part of the outer periphery. From the correct arrangement of the tibial grid, the sagittal direction and the coronal direction can be obtained as the directions of the long side and the short side of the tibial grid, respectively. The formation of the tibial grid will be discussed in more detail below with visual reference.

[0093] In the above example regarding step 1012, the SSM is used to help determine the anterior-posterior direction. In an example where the SSM is not used, the anterior-posterior direction can be estimated based on the cervical spine. In particular, the cervical spine is the highest curvature point in the segmented data located near the centroid of the tibial plane, and the cervical spine belongs to the posterior side. In the segmented data or the 3D model, the anterior-posterior direction can be estimated by finding the cervical spine with an algorithm. In still other embodiments, determining the medial-lateral direction for the arrangement of the double tangent can be determined algorithmically by finding the side of the tibial plateau that contains more points with higher curvature (e.g., based on the tibial plane). The side of the tibial plateau with more data points and higher curvature corresponds to the medial side.

[0094] In a further embodiment, the double tangent can be determined without explicit identification of the medial and lateral directions. In particular, in a further embodiment, the tangents of all posterior points are determined. Using the tangents, all possible pairs of posterior points and a part or all of the corresponding tangents are established. Then, the embodiment selects two pairs in which the two tangents and the line junctions have the same direction. The embodiment selects the pair that results in a line that does not cut or excise the contour. Thus, the double tangent is the line corresponding to the pair with the maximum distance between points.

[0095] In step 1016, the engine 200 overlays and displays a tibial grid on the 3D model. That is, the surgeon may be provided with a view of the tibia showing the tibial grid (e.g., along the final tibial axis). The surgeon may select the tunnel opening position on the tibial plateau based on the tibial grid.

[0096] In a recently published paper, the location of the tibial tuberosity is considered to determine the external point of the ACL tibial tunnel. Due to the importance of this landmark, this method may further include the automatic detection of the tibial tuberosity or the tibial tuberosity, which may be useful for the surgeon in determining the position of the tibial tunnel.

[0097] Accordingly, the exemplary process 1000 may also include step 1018. In step 1018, the engine 200 identifies the tibial tuberosity in the 3D model and displays a display of the location of the tibial tuberosity in the 3D model. In one embodiment, identifying the tibial tuberosity may include projecting the 3D model onto the sagittal plane, segmenting the 3D model along a line perpendicular to the second long side to result in tuberosity segmentation, and identifying the foremost part of the tuberosity segmentation as the tibial tuberosity. By back-projecting onto the 3D model, the center of the tuberosity is reconstructed. The elevation of the tuberosity can be obtained as the intersection of the tuberosity region with a plane passing through the foremost point (center of the tuberosity), where the vertical is in the sagittal direction.

[0098] In other embodiments, the orthographic projection and segmentation with respect to the line perpendicular to the second long side may be omitted, and instead, a curved cue and / or sphere may be fitted to the data from the 3D model to estimate the position of the tibial tuberosity. In still other cases, the position of the tibial tuberosity may be estimated using a template model, an atlas, a deep learning framework, etc. Here, the present specification turns to a visual explanation of the exemplary methodology.

[0099] FIG. 11 shows a rendering of an exemplary proximal tibia 3D model. In particular, visible in FIG. 11 are a tibia 1100, a tibial plateau 1102, a cervical spine 1104, a tibial tuberosity 1106, and a portion of the tibial axis 1108. On the inner side of the tibial plateau 1102 there is a lowest point 1110, which in this case is also the lowest or most distal point of the tibial plateau (more specifically, the medial plateau). The coordinate system of FIG. 11 shows an inner axis and an outer axis, a proximal axis and a distal axis, and an anterior axis and a posterior axis.

[0100] FIG. 12 shows a rendering of a 3D model showing a first approximate tibial axis 1200 and a lowest point 1110. In an exemplary methodology (e.g., step 1004), the first approximate tibial axis 1200 and the lowest point 1110 are determined, for example, by morphing an SSM. However, any suitable technique can be used to find the first approximate tibial axis 1200 and the lowest point 1110. In an exemplary methodology, data points are segmented by identifying all data points of the 3D model that exist above a segmentation plane. The segmentation plane exists at a predetermined distance (e.g., 5 mm) below the lowest point 1110, and the segmentation plane is perpendicular to the first approximate tibial axis 1200. In FIG. 12, the segmented data 1202 is highlighted with a stick ring.

[0101] Figure 13 shows a rendering of a 3D model including an exemplary outer perimeter 1300. In an exemplary methodology (e.g., step 1008), the segmented data is refined to better depict the outer perimeter of the tibial plateau. In one embodiment, the deformed SSM defines a control set or an initial set of peripheral data points of the tibial plateau, and the initial set of peripheral data points 1302 is shown as a square, with only a portion thereof being associated with reference numbers. Further, the segmented data is provided to an algorithm that identifies high-profile regions. High-profile regions, such as region 1304, are connected to the initial set of peripheral data points 1302 to define the outer perimeter 1300 (a combination of a square and a circle). Thus, the data points of the outer perimeter 1300 and the data points existing within the outer perimeter can be considered as detailed or fine segmentation.

[0102] Figure 14 shows a rendering of a 3D model, highlighting the segmented data, the outer perimeter, and a portion of the segmented data existing within the outer perimeter. In particular, in Figure 14, the segmented data 1202, the first approximate tibial axis 1200, and the lowest point 1110 are visible. Within the segmented data 1202, the data points define a data point of the segmented data existing within the outer perimeter and the outer perimeter, region 1400.

[0103] Figure 15 shows a perspective view of a rendering of a 3D model including the tibial plane. In an exemplary methodology (e.g., step 1010), the data points defining the outer perimeter and the data points of the segmented data existing within the outer perimeter are provided to a plane fitting algorithm to find a well-fitting mathematical plane, such as the tibial plane 1500. Note that not all data points of the outer perimeter and / or data points of the segmented data existing within the outer perimeter are present within the tibial plane 1500. Once the tibial plane 1500 is determined, a line perpendicular to the tibial plane 1500 defines the final tibial axis 1502. Note the cervical spine 1104 that rises above the tibial plane 1500 and is defined and present within the posterior portion of the tibial plane 1500 while existing within the outer perimeter (not specifically depicted in Figure 15).

[0104] FIG. 16 shows a side elevation view of a rendering of a 3D model including the tibial plane. FIG. 16 better shows the normal or perpendicular relationship between the final tibial axis 1502 and the tibial plane 1500. Further, the figure of FIG. 16 also shows whether the portion of the data existing within the outer perimeter rises above (e.g., the cervical spine 1104) or falls below (e.g., the lateral condyle 1504) the tibial plane 1500.

[0105] FIG. 17 shows the outer perimeter data points and the data points existing within the outer perimeter, projected onto the tibial plane, together with the double tangent. In an exemplary methodology (e.g., step 1012), the data points of the outer perimeter 1300 and the data (points 1700) existing within the outer perimeter can be orthogonally projected onto a plane perpendicular to the final tibial axis 1502, such as the tibial plane 1500 itself (e.g., the plane of the page). The front portion 1702 and the rear portion 1704 of the projection can be determined in any suitable manner, such as by using the SSM or by the cervical spine. With the rear portion 1704 determined, the double tangent 1706 may be arranged as discussed in detail above. The double tangent 1706 defines a line existing within the coronal plane. A line perpendicular to the final tibial axis 1502 and perpendicular to the double tangent 1706 can define the coronal direction.

[0106] Figure 18 shows the projection of FIG. 17 with the tibial grid disposed thereon. In an exemplary methodology (e.g., step 1014), placing the tibial grid may involve fitting or discovering a minimum rectangle around the projected data points, the rectangle having a first long side 1800 coaxial with the double tangent 1706, a second long side 1802 parallel to the double tangent 1706, and two short edge portions 1804 and 1806 that are parallel to each other and perpendicular to the long sides 1800 and 1802. Thus, the short sides 1804 and / or 1806 define the coronal direction. The rectangle formed by the edges 1800, 1802, 1804, and 1806 can be considered to define a tibial grid 1820 having a base point or position (0%, 0%) at the lower left of FIG. 18 and (100%, 100%) at the upper right. The surgeon can then use the tibial grid 1820 to determine the ACL footprint using one of a number of studies that provide positions in normalized coordinates.

[0107] Figure 19 shows an axial view of a rendering of a 3D model including the tibial grid. In an exemplary methodology (e.g., step 1016), the tibial grid 1820 determined as discussed above can be overlaid and displayed on a computer screen within the surgical setting, for example, so that the surgeon can select the opening position of the tibial tunnel on the tibial plateau. Note the position of the tibial tuberosity 1106.

[0108] In still further embodiments, the sagittal direction can be estimated as the direction of a line passing through the outermost and innermost points of the back portion of the axial orthographic projection. For example, from the perspective of FIG. 19, the sagittal direction can be considered as a line 1900 that intersects the most posterior outermost point 1902 and the rearmost innermost point 1904. Note that the sagittal direction determined according to the outermost and innermost points of the back portion may be slightly different from the sagittal direction determined to coincide with the double tangent.

[0109] FIG. 20 shows a sagittal or side view of a rendering of a 3D model including the tibial plane. In an exemplary methodology (e.g., step 1018), the tibial tuberosity 1106 may be programmatically determined by projecting the 3D model in the sagittal direction and narrowing or segmenting the data points that are anterior to the segmentation line 2000, where the segmentation line 2000 is parallel to the final tibial axis (not specifically shown), and the segmentation line 2000 intersects the anterior long side 1802 of the tibial grid 1808. Thus, the tibial tuberosity 1106 may be programmatically determined as the maximum or apex (e.g., the most anterior point 2002) of the segmented data points. With the tibial grid 1820 programmatically positioned and the tibial tuberosity 1106 programmatically determined, a clinician may plan the tibial tunnel position and transmit the information to intraoperative equipment.

[0110] Referring further to FIG. 20. In a further embodiment, additional scale invariant information may be generated to further assist a surgeon in the placement of the tibial tunnel. In particular, with the 3D model projected in the sagittal direction as shown in FIG. 20, a further embodiment finds the final point 2030. The final point 2030, and a line 2032 parallel to the tibial grid 1820 together define an anterior-posterior distance (AP in the figure, hereinafter referred to as the AP distance). The AP distance may be marked as shown in FIG. 20, such as 25% of the AP distance, 43% of the AP distance, and 62% of the AP distance. These depictions may be useful in determining the placement of the tibial tunnel, or at least the location of the opening of the tibial tunnel on the tibial plateau.

[0111] In the embodiments discussed above, the segmentation plane was selected as a plane perpendicular to the first approximate tibial axis at a predetermined distance below or distal to the lowest point 1110 (FIG. 11). However, in other embodiments, the segmentation plane may be based on the last point 2030 of the tibia. In particular, in these embodiments, the sagittal direction can be estimated in any suitable manner, such as using the sagittal direction determined from the analysis of the 3D model of the femur from the same patient (imaging taken with the patient's leg extended). In the estimated sagittal direction, the 3D model can be projected as shown in FIG. 20, and the last point 2030 is determined. Using the last point 2030, the segmentation plane may again be a plane perpendicular to the first approximate tibial axis and including the last point 2030. In other cases, the segmentation plane may be at a predetermined distance below or distal to the last point 2030 (e.g., 3 mm), and again may be a segmentation plane perpendicular to the first approximate tibial axis. When the segmented data is determined based on an alternative segmentation plane, the tibial grid may be placed using the various exemplary steps discussed above.

[0112] FIG. 21 shows a rendering of 3D curvature values projected onto an axial plane. As described above, in alternative embodiments, the tibial plateau can be determined without using prior information such as SSM. In particular, in alternative embodiments, the 3D model, or at least the segmented data of the 3D model, can be evaluated to determine 3D curvature values in regions such as 2100, 2102, and the position of the cervical spine spine in region 2104. The 3D curvature values can be used to generate a two-dimensional image by projecting the 3D curvature values onto an axial plane (the projection being of the kind shown in FIG. 21). To find a closed contour, any suitable morphological operation may be applied to the 2D projection, and the closed contour thus defines the outer perimeter. Optionally, the data defining the outer perimeter may then be projected onto the segmented data to identify a portion of the segmented data that lies within the outer perimeter.

[0113] FIG. 22 is a block diagram showing a computing device 2200 (e.g., UE 106 as discussed above) that illustrates an example of a client device or a server device used in various embodiments of the present disclosure.

[0114] The computing device 2200 may include more or fewer components than those shown in FIG. 22, depending on the deployment or use of the device 2200. For example, a server computing device such as a server mounted in a rack may not include an audio interface 2252, a display 2254, a keypad 2256, an illumination device 2258, a tactile interface 2262, a GPS receiver 2264, or a camera / sensor 2266. Some devices may include additional components not shown, such as a GPU device, an encryption coprocessor, an AI accelerator, or other peripheral devices.

[0115] As shown in FIG. 22, the device 2200 includes a central processing unit (CPU) 2222 that communicates with a mass memory 2230 via a bus 2224. The computing device 2200 also includes one or more network interfaces 2250, an audio interface 2252, a display 2254, a keypad 2256, an illumination device 2258, an input / output interface 2260, a tactile interface 2262, an optional GPS receiver 2264 (and / or a replaceable or additional GNSS receiver), and a camera or other optical sensor, a thermal sensor, or an electromagnetic sensor 2266. The device 2200 may include one camera / sensor 2266 or multiple camera / sensors 2266. The positioning of the camera / sensor 2266 on the device 2200 may vary by device 2200 model, by device 2200 capabilities, and the like, or combinations thereof.

[0116] In some embodiments, the CPU 2222 may include a general-purpose CPU. The CPU 2222 may include a single-core or multi-core CPU. The CPU 2222 may be provided with a (system-on-a-chip) SoC or a similar embedded system. In some embodiments, the GPU may be used instead of the CPU 2222 or in combination with the CPU 2222. The large-capacity memory 2230 may include a dynamic random access memory (DRAM) device, a static random access memory device (SRAM), or a flash (e.g., NAND flash) memory device. In some embodiments, the large-capacity memory 2230 may include a combination of such memory types. In one embodiment, the bus 2224 may include a peripheral component interconnect express (PCIe) bus. In some embodiments, the bus 2224 may include a plurality of buses instead of a single bus.

[0117] The large-capacity memory 2230 represents another example of a computer storage medium for storing information such as computer-readable instructions, data structures, program modules, or other data. The large-capacity memory 2230 stores a basic input / output system ("BIOS") 2240 for controlling the low-level operations of the computing device 2200. The large-capacity memory also stores an operating system 2241 for controlling the operation of the computing device 2200.

[0118] The application 2242 may include computer-executable instructions that, when executed by the computing device 2200, perform any of the methods (or portions of the methods) described above in the description of the foregoing figures. In some embodiments, the software or program implementing the method embodiments may be read from a hard disk drive (not shown) and temporarily stored in the RAM 2232 by the CPU 2222. Then, the CPU 2222 may read software or data from the RAM 2232, process them, and save them back to the RAM 2232.

[0119] The computing device 2200 may optionally communicate directly with a base station (not shown) or with another computing device. The network interface 2250 may be known as a transceiver, a receiving device, or a network interface card (NIC).

[0120] The audio interface 2252 generates and receives audio signals such as the sound of a human voice. For example, the audio interface 2252 may be coupled to speakers and a microphone (not shown) to enable communication with others or to generate an audio confirmation for some action. The display 2254 may be a liquid crystal display (LCD), gas plasma, light emitting diode (LED), or any other type of display used in a computing device. The display 2254 may also include a touch-sensitive screen arranged to receive input from an object such as a stylus or a finger from a human hand.

[0121] The keypad 2256 may comprise any input device arranged to receive input from a user. The illuminator 2258 may provide a status indication or may provide light.

[0122] The computing device 2200 also comprises an input / output interface 2260 for communicating with external devices using communication technologies such as USB, infrared, Bluetooth™. The tactile interface 2262 provides tactile feedback to a user of the client device.

[0123] The GPS transceiver 2264 can typically determine the physical coordinates of a computing device 2200 on the surface of the earth that outputs its position as latitude and longitude values. The GPS transceiver 2264 can also employ other geolocation mechanisms, including but not limited to triangulation, assisted GPS (AGPS), E-OTD, CI, SAI, ETA, BSS, etc., to further determine the physical location of the computing device 2200 on the surface of the earth. However, in one embodiment, the computing device 2200 can communicate via other components and provide other information that can be used to determine the physical location of the device, including, for example, MAC addresses, IP addresses, etc.

[0124] For the purposes of the present disclosure, a module is a software, hardware, or firmware (or combination thereof) system, process, or function, or components thereof, that performs or facilitates the processes, features, and / or functions (with or without human interaction or augmentation) described herein. A module can include sub-modules. The software components of a module may be stored on a computer-readable medium for execution by a processor. A module may be integrated into one or more servers or may be loaded and executed by one or more servers. One or more modules may be grouped and made into an engine or an application.

[0125] Those skilled in the art will recognize that the methods and systems of the present disclosure may be implemented in many ways and are therefore not limited by the foregoing exemplary embodiments and examples. In other words, various combinations of hardware and software or firmware, and functional elements executed by single or multiple components in individual functions, may be distributed among software applications at the client level or server level, or both. In this regard, any number of the features of the different embodiments described herein may be combined into a single or multiple embodiments, and alternative embodiments are possible that have fewer or more than all of the features described herein.

[0126] Functionality may also be distributed, in whole or in part, among multiple components in manners that are currently known or will become known. Accordingly, an infinite number of software / hardware / firmware combinations are possible in achieving the functions, features, interfaces, and preferences described herein. Further, the scope of the present disclosure includes conventional methods for performing the described features and functions and interfaces, and those variations and modifications that can be made to the hardware or software or firmware components described herein, as will be understood by those skilled in the art now and in the future.

[0127] Furthermore, embodiments of the methods presented and described as flowcharts in the present disclosure are provided by way of example to provide a more complete understanding of the technology. The methods of the present disclosure are not limited to the operations and logical flows presented herein. Alternative embodiments are contemplated in which the order of various operations is changed and sub-operations described as part of larger operations are performed independently.

[0128] Although various embodiments have been described for purposes of this disclosure, such embodiments should not be considered as limiting the teachings of this disclosure. Various changes and modifications can be made to the above elements and operations to obtain results that remain within the scope of the systems and processes described in this disclosure.

[0129] The following sections define various non-limiting examples.

[0130] Item 1: A computer-implemented method comprising: identifying a three-dimensional (3D) model of a femur by a device; determining, by the device, a pair of points on the femur having a normal vector orthogonal to a vector joining the pair of points by analyzing the 3D model; determining, by the device, a sagittal direction by analyzing the vector; further analyzing, by the device, the 3D model based on the determined sagittal direction to generate an intersection map of the 3D model, wherein the intersection map includes information related to the curvature value of the intercondylar region; determining, by the device, a Blumensaat line based on the intersection map; determining, by the device, a Bernnard-Helter (BH) grid by analyzing the intersection map according to the Blumensaat line, wherein the BH grid includes a region surrounding the condyle when the femur is presented in a lateral view; and arranging, by the device, a digital representation of the BH grid as an overlay of the 3D model.

[0131] Item 2: The computer-implemented method according to Item 1, further comprising: determining values for the pair of points and each normal vector; clustering the determined values; and determining a median value based on the clustering, wherein the determined sagittal direction corresponds to the median value.

[0132] Item 3: The computer-implemented method according to any of the preceding items, further comprising: executing, by the device, an estimation model; and determining, by the device, a sagittal direction based on the output of the estimation model.

[0133] Item 4: Further comprising identifying an outer condyle and an inner condyle, and determining a sagittal outer-inner orientation based on the identified outer condyle and inner condyle, the computer-implemented method according to any one of the preceding items.

[0134] Item 5: The computer-implemented method according to item 4, further comprising refining the sagittal direction based on adjustment of the outer boundaries of the outer condyle and the inner condyle.

[0135] Item 6: Further comprising back-projecting a BH grid onto a 3D model, the BH grid being a two-dimensional (2D) model, and the back-projection being based on a cross-sectional plane defined by the sagittal direction and the Blumenzahl line, the computer-implemented method according to any one of the preceding items.

[0136] Item 7: Back-projecting a point on an intersection map onto a point on a 3D model, the point on the intersection map being two-dimensional (2D), further comprising obtaining a cross-sectional plane of the 3D model based on the sagittal direction, and determining an axial direction via circle fitting, the computer-implemented method according to any one of the preceding items.

[0137] Item 8: Determining a contour of the femur based on a sagittal view of the femur, the sagittal view being obtained through a 2D projection along the sagittal direction, further comprising searching for a circle tangent to the contour at two points, and estimating a line connecting the respective centers of the circles, the estimated line providing information related to the axial direction, the computer-implemented method according to any one of the preceding items.

[0138] Item 9: Further comprising analyzing the femur based on the sagittal direction, determining a sagittal view of the femur, determining an axial direction based on the sagittal view, and determining a coronal plane based on the cross product of the sagittal plane and the axial direction, the computer-implemented method according to any one of the preceding items.

[0139] Item 10: The computer-implemented method according to item 9, further comprising determining an anatomical reference frame (ARF) of the femur based on the sagittal, axial, and coronal directions.

[0140] Item 11: The computer-implemented method according to any of the preceding items, further comprising identifying a frontal projection of the femur, identifying a set of projection rays within a 3D model associated with the femur based on the frontal projection, and determining a set of intersections corresponding to the set of projection rays, wherein the intersection map includes information related to the set of intersections.

[0141] Item 12: The computer-implemented method according to item 12, further comprising analyzing a set of intersections via edge detection analysis and determining a curve value of the intercondylar region based on the edge detection analysis, wherein the Blumensaat line is in the tangent direction with respect to a curve associated with the curve value.

[0142] Item 13: A non-transitory computer-readable storage medium tangibly encoded with computer-executable instructions that, when executed by one or more devices, cause the one or more devices to identify a three-dimensional (3D) model of the femur, analyze the 3D model, determine a pair of points on the femur having a normal vector orthogonal to a vector joining the pair of points, analyze the vector to determine the sagittal direction, further analyze the 3D model based on the sagittal direction to generate an intersection map of the 3D model, wherein the intersection map includes information related to a curve value of the intercondylar region, determine the Blumensaat line based on the intersection map, analyze the intersection map according to the Blumensaat line to determine the Bernard-Helter (BH) grid, wherein the BH grid includes a region surrounding the condyles when the femur is presented in a lateral view, and arrange a digital representation of the BH grid as an overlay of the 3D model.

[0143] Item 14: The non - transitory computer - readable storage medium according to item 13, wherein the command causes one or more devices to determine a pair of points and values of each normal vector for one or more devices, cluster the values, determine a median value based on the clustering, and the sagittal direction corresponds to the median value.

[0144] Item 15: The non - transitory computer - readable storage medium according to any one of items 13 to 14, wherein the command causes one or more devices to identify the lateral condyle and the medial condyle, determine the lateral - medial orientation of the sagittal direction based on the identified lateral condyle and medial condyle, and refine the sagittal direction based on the adjustment of the outer boundaries of the lateral condyle and the medial condyle.

[0145] Item 16: The non - transitory computer - readable storage medium according to any one of items 13 to 15, wherein the command further causes one or more devices to back - project points on an intersection map to points on a 3D model, the points on the intersection map being two - dimensional (2D), obtain a cross - sectional plane of the 3D model based on the sagittal direction, and determine the axial direction through circle fitting.

[0146] Item 17: The non - transitory computer - readable storage medium according to any one of items 13 to 16, wherein the command causes one or more devices to determine the contour of the femur based on a sagittal view of the femur, the sagittal view being obtained through a 2D projection along the sagittal direction, search for a circle that touches the contour at two points, estimate a line connecting the centers of each circle, and the estimated line provides information related to the axial direction.

[0147] Item 18: The non - transitory computer - readable storage medium according to any one of items 13 to 17, wherein the command further causes one or more devices to analyze the femur based on the sagittal direction, determine a sagittal view of the femur, determine the axial direction based on the sagittal view, determine the coronal direction based on the product of the sagittal direction and the axial direction, and determine an anatomical reference frame (ARF) of the femur based on the sagittal direction, the axial direction, and the coronal direction.

[0148] Item 19: A device including one or more processors configured to perform the methods of Items 1 to 12 and / or to execute the instructions of Items 13 to 18. Item 20: Intentionally skipped.

[0149] Item 21: A computer-implemented method comprising receiving, by a device, a three-dimensional model (3D model) of the proximal tibia; identifying, by the device, a first approximate tibial axis; segmenting, by the device, the 3D model using the first approximate tibial axis to create segmented data; defining, by the device, the outer perimeter of the tibial plateau from the segmented data; fitting, by the device, a tibial plane based on the outer perimeter and at least a portion of the segmented data; forming, by the device, a tibial grid consisting of a rectangle having a first long side coaxial with a double tangent, a second long side parallel to the first long side and identifying the outermost edge of the anterior portion of the outer perimeter, a first short side perpendicular to the first long side and identifying the innermost portion of the outer perimeter, and a second short side perpendicular to the first long side and identifying the outermost portion of the outer perimeter; and overlaying and displaying, by the device, the tibial grid on the 3D model.

[0150] Item 22: The computer-implemented method according to Item 21, further comprising morphing a statistical shape model (SSM) corresponding to the 3D model to create a morphed SSM, and identifying the first approximate tibial axis from the morphed SSM.

[0151] Item 23: The computer-implemented method according to any one of Items 21 to 22, further comprising identifying the lowest point of the tibial plateau from the SSM, and segmenting comprising further segmenting the 3D model using the first approximate tibial axis and the lowest point to create segmented data.

[0152] Item 24: The computer-implemented method according to item 23, wherein segmenting the 3D model further includes selecting data points from the 3D model that exists within and above the segmentation plane, the segmentation plane is perpendicular to the first approximate tibial axis, and the segmentation plane is at a predetermined distance distal to the lowest point of the tibial plateau.

[0153] Item 25: The computer-implemented method according to item 24, wherein the lowest point of the tibial plateau is the most distal point of the medial plateau.

[0154] Item 26: The computer-implemented method according to any one of items 21-25, wherein segmenting the 3D model further includes estimating the sagittal direction of the 3D model, projecting the 3D model in the sagittal direction to obtain a sagittal projection, finding the last point of the tibial plateau from the sagittal projection, and segmenting the 3D model using the last point and the first approximate tibial axis.

[0155] Item 27: The computer-implemented method according to item 26, wherein segmenting the 3D model further includes selecting data points from the 3D model that exists within and above the segmentation plane, the segmentation plane is perpendicular to the first approximate tibial axis, and the segmentation plane is at a predetermined distance distal to the last point of the tibial plateau.

[0156] Item 28: The computer-implemented method according to any one of items 21-27, wherein drawing the outer perimeter of the tibial plateau further includes identifying a contour region within the segmented data and assigning the outer perimeter based on the contour region.

[0157] Item 29: The computer-implemented method according to any one of items 21-28, wherein finding the double tangent further includes projecting the data of the outer perimeter and the segmented data within the outer perimeter onto the tibial plane to create projected data, and finding the double tangent on the posterior side of the projected data.

[0158] Item 30: The computer-implemented method according to any one of Items 21 to 29, further comprising identifying the tibial tuberosity in the 3D model by a device and displaying the position of the tibial tuberosity in the 3D model.

[0159] Item 31: The computer-implemented method according to Item 30, wherein identifying the tibial tuberosity further comprises projecting the 3D model onto a sagittal plane, segmenting the 3D model along a 3D line perpendicular to the second long side to result in tuberosity segmentation, and identifying the foremost part of the tuberosity segmentation as the tibial tuberosity.

[0160] Item 32: The computer-implemented method according to any one of Items 21 to 31, wherein segmenting to create segmented data further comprises selecting data proximal to a predetermined point distal to the lowest point.

[0161] Item 33: The computer-implemented method according to any one of Items 21 to 32, wherein fitting the tibial plane further comprises fitting based on the segmented data present within and on the outer periphery.

[0162] Item 34: A non-transitory computer-readable storage medium tangibly encoded with computer-executable instructions that, when executed by one or more devices, cause the one or more devices to receive a three-dimensional model (3D model) of the proximal tibia, identify a first approximate tibial axis, use the first approximate tibial axis to segment the 3D model to create segmented data, trace the outer perimeter of the tibial plateau from the segmented data, fit to a tibial plane based on the outer perimeter and at least a portion of the segmented data, where a normal to the tibial plane defines a final tibial axis, find a double tangent on the posterior based on the outer perimeter and portions of the segmented data, form a tibial grid including a rectangle having a first long side coaxial with the double tangent, a second long side parallel to the first long side and identifying the outermost edge of the anterior portion of the outer perimeter, a first short side perpendicular to the first long side and identifying the innermost portion of the outer perimeter, and a second short side perpendicular to the first long side and identifying the outermost portion of the outer perimeter, and overlay and display the tibial grid on the 3D model.

[0163] Item 35: The non-transitory computer-readable storage medium of item 34, wherein when one or more devices identify a first approximate tibial axis and the lowest point of the tibial plateau, the instructions further cause the one or more devices to associate a statistical shape model (SSM) with the 3D model to create a deformed SSM and identify the first approximate tibial axis from the deformed SSM.

[0164] Item 36: The non-transitory computer-readable storage medium according to any one of items 34-35, wherein the instructions further cause the one or more devices to identify the lowest point of the tibial plateau from the SSM, and when the one or more devices segment the 3D model, the instructions cause the one or more devices to segment the 3D model using the first approximate tibial axis and the lowest point of the tibial plateau.

[0165] Item 37: When the one or more devices segment the 3D model, the instructions cause the one or more devices to select data points from the 3D model that are within the segmentation plane and above the segmentation plane, the segmentation plane being perpendicular to the first approximate tibial axis and the segmentation plane being distally located a predetermined distance from the lowest point of the tibial plateau, the non-transitory computer-readable medium of item 36.

[0166] Item 38: The instructions cause one or more devices to estimate the sagittal direction of a 3D model, project the 3D model in the sagittal direction, resulting in a sagittal projection, find the last point of the tibial plateau from the sagittal projection, and when the one or more devices segment the 3D model, the instructions cause the one or more devices to segment the 3D model using the last point and the first approximate tibial axis, the non-transitory computer-readable storage medium of any of items 34 - 37.

[0167] Item 39: When the one or more devices segment the 3D model, the instructions further cause the one or more devices to select data points from the 3D model that are within and above the segmentation plane, the segmentation plane being perpendicular to the first approximate tibial axis and the segmentation plane being distally located a predetermined distance from the last point of the tibial plateau, the computer-implemented method of any of items 34 - 38.

[0168] Item 40: When the one or more devices depict the outer perimeter of the tibial plateau, the instructions further cause the one or more devices to identify a contour region within the segmented data and assign the outer perimeter based on the contour region, the non-transitory computer-readable storage medium of any of items 34 - 39.

[0169] Item 41: When one or more devices find a double tangent, the instructions cause the one or more devices to project data points of the outer perimeter and the segmented data within the outer perimeter onto the tibial plane to create projected data, and further cause the one or more devices to find a double tangent on the posterior side of the projected data, the non-transitory computer-readable storage medium according to any one of Items 34 to 40.

[0170] Item 42: The instructions further cause the one or more devices to identify the tibial tuberosity in the 3D model and display the position of the tibial tuberosity grid in the 3D model, the non-transitory computer-readable storage medium according to any one of Items 34 to 41.

[0171] Item 43: When one or more devices identify the tibial tuberosity, the instructions cause the one or more devices to project the 3D model onto the sagittal plane, segment the 3D model along a line perpendicular to the second long side to result in tuberosity segmentation, and identify the foremost part of the tuberosity segmentation as the tibial tuberosity, the non-transitory computer-readable storage medium according to Item 42.

[0172] Item 44: When one or more device segments create segmented data, the instructions further cause the one or more devices to select data proximal to a predetermined point distal to the lowest point, the non-transitory computer-readable storage medium according to any one of Items 34 to 43.

[0173] Item 45: When one or more devices fit to the tibial plane, the instructions cause the one or more devices to fit based on the outer perimeter and the segmented data present within the outer perimeter, the non-transitory computer-readable storage medium according to any one of Items 34 to 44.

[0174] Item 46: A device including one or more processors configured to implement the methods according to Items 21 to 33 and / or the instructions according to Items 34 to 45.

Claims

1. A computer-implemented method, comprising: identifying, by a device, a three-dimensional (3D) model of a femur; determining, by the device, a pair of points on the femur having a normal vector orthogonal to a vector joining the pair of points by analyzing the 3D model; analyzing, by the device, the vector to determine a sagittal direction; further analyzing, by the device, the 3D model based on the determined sagittal direction to generate an intersection map of the 3D model, the intersection map including information related to a curve value of an intercondylar region; determining, by the device, a Blumensaat line based on the intersection map; analyzing, by the device, the intersection map according to the Blumensaat line to determine a Bernard-Helter (BH) grid, the BH grid including a region surrounding a condyle when the femur is presented in a lateral view; placing, by the device, a digital representation of the BH grid as an overlay on the 3D model.

2. determining values of the pair of points and each normal vector; clustering the determined values; determining a median value based on the clustering, the determined sagittal direction corresponding to the median value, the computer-implemented method according to claim 1.

3. executing, by a device, an estimation model; determining the sagittal direction based on an output of the estimation model, the computer-implemented method according to claim 1.

4. identifying a lateral condyle and a medial condyle; determining a lateral-medial orientation of the sagittal direction based on the identified lateral condyle and medial condyle, the computer-implemented method according to claim 1.

5. further comprising refining the sagittal direction based on an adjustment of an outer boundary of the lateral condyle and the medial condyle, the computer-implemented method according to claim 4.

6. further comprising back-projecting the BH grid onto the 3D model, the BH grid being a two-dimensional (2D) model, the back-projection being based on a cutting plane defined by the sagittal direction and the Blumensaat line, the computer-implemented method according to claim 1.

7. Back-projecting a point on the intersection map to a point on the 3D model, wherein the point on the intersection map is two-dimensional (2D), the back-projecting; Obtaining a cross-sectional plane of the 3D model based on the sagittal direction; Determining an axial direction via circle fitting, the computer-implemented method according to claim 1.

8. Determining a contour of the femur based on a sagittal view of the femur, wherein the sagittal view is obtained through a 2D projection along the sagittal direction, the determining; Searching for a circle that touches the contour at two points; Estimating a line connecting the centers of each of the circles, wherein the estimated line provides information related to the axial direction, the estimating, the computer-implemented method according to claim 1.

9. Analyzing the femur based on the sagittal direction and determining a sagittal view of the femur; Determining an axial direction based on the sagittal view; Determining a coronal direction based on a cross product of the sagittal direction and the axial direction, the computer-implemented method according to claim 1.

10. Further comprising determining an anatomical reference frame (ARF) of the femur based on the sagittal direction, the axial direction, and the coronal direction, the computer-implemented method according to claim 9.

11. Identifying a front projection of the femur; Identifying a set of projection rays within the 3D model related to the femur based on the front projection; Determining a set of intersections corresponding to the set of projection rays, wherein the intersection map includes information related to the set of intersections, the determining, the computer-implemented method according to claim 1.

12. Analyzing the set of intersections via edge detection analysis; Determining a curve value of the intercondylar region based on the edge detection analysis, wherein the Blumensaat line is a tangent direction to a curve associated with the curve value, the determining, the computer-implemented method according to claim 11.

13. When executed by one or more devices, the one or more devices, Identifying a three-dimensional (3D) model of the femur; Analyzing the 3D model and determining a pair of points on the femur having a normal vector orthogonal to a vector connecting the pair of points; Analyze the vector and determine the sagittal direction, Further analyze the 3D model based on the sagittal direction and generate an intersection map of the 3D model, wherein the intersection map includes information related to the curve value of the intercondylar region, Determine the Blumensaat line based on the intersection map, Analyze the intersection map according to the Blumensaat line and determine the Bernard-Helfet (BH) grid, wherein the BH grid includes the region surrounding the condyle when the femur is provided in a lateral view, A non-transitory computer-readable storage medium tangibly encoded with computer-executable instructions that cause one or more devices to perform placing a digital representation of the BH grid as an overlay on the 3D model.

14. The instructions cause the one or more devices to Determine the values of the pair of points and each normal vector, Cluster the values, Determine a median value based on the clustering, wherein the sagittal direction corresponds to the median value, and further perform the determination,

15. The instructions cause the one or more devices to Identify the lateral condyle and the medial condyle, Determine the lateral-medial orientation of the sagittal direction based on the identified lateral condyle and medial condyle, Refine the sagittal direction based on the adjustment of the outer edges of the lateral condyle and the medial condyle, and further perform the refinement,

16. The instructions cause the one or more devices to Back-project a point on the intersection map to a point on the 3D model, wherein the point on the intersection map is two-dimensional (2D), Obtain a cross-sectional plane of the 3D model based on the sagittal direction, Determine an axial direction through circle fitting, and further perform the determination,

17. The instructions cause the one or more devices to Determine the contour of the femur based on the sagittal view of the femur, wherein the sagittal view is obtained through a 2D projection along the sagittal direction, Search for a circle tangent to the contour at two points. Estimate a line connecting the centers of each of the circles, and further cause the estimated line to provide information related to the axial direction, the non-transitory computer-readable medium according to claim 13.

18. The instructions cause the one or more devices to Analyze the femur based on the sagittal direction and determine a sagittal view of the femur; Determine an axial direction based on the sagittal view; Determine a coronal direction based on the cross product of the sagittal direction and the axial direction; Determine an anatomical reference frame (ARF) of the femur based on the sagittal direction, the axial direction, and the coronal direction, the non-transitory computer-readable medium according to claim 13.

19. A device, One or more processors, Identify a three-dimensional (3D) model of a femur, Analyze the 3D model and determine a pair of points on the femur having a normal vector orthogonal to a vector connecting the pair of points; Analyze the vector and determine a sagittal direction; Further analyze the 3D model based on the determined sagittal direction and generate an intersection map of the 3D model including information related to a curve value of an intercondylar region; Determine a Blumensaat line based on the intersection map; Analyze the intersection map according to the Blumensaat line and determine a Bernard-Helter (BH) grid, wherein the BH grid includes a region surrounding a condyle when the femur is presented in a lateral view; A device comprising one or more processors configured to place a digital representation of the BH grid as an overlay on the 3D model.

20. The one or more processors Analyze the femur based on the sagittal direction and determine a sagittal view of the femur; Determine an axial direction based on the sagittal view; Determine a coronal direction based on the cross product of the sagittal direction and the axial direction; Determine an anatomical reference frame (ARF) of the femur based on the sagittal direction, the axial direction, and the coronal direction, the device according to claim 19, further configured as such.

21. A computer-implemented method, Receiving, by a device, a three-dimensional model (3D model) of a proximal tibia; Identifying, by the device, a first approximate tibial axis; The device segments the 3D model using the first approximate tibial axis to create segmented data; The device draws a perimeter of the tibial plateau from the segmented data; The device fits a tibial plane based on the perimeter and at least a part of the segmented data, wherein a normal to the tibial plane defines a final tibial axis; The device finds a double tangent on a posterior part based on the perimeter and a part of the segmented data; The device forms a tibial grid consisting of a rectangle having a first long side coaxial with the double tangent, a second long side parallel to the first long side and specifying an outermost edge of a front part of the perimeter, a first short side perpendicular to the first long side and specifying an innermost part of the perimeter, and a second short side perpendicular to the first long side and specifying an outermost part of the perimeter; The device overlays and displays the tibial grid on the 3D model, a computer-implemented method comprising. Identifying a first approximate tibial axis further comprises morphing a statistical shape model (SSM) corresponding to the 3D model to create a morphed SSM, and identifying the first approximate tibial axis from the morphed SSM, the computer-implemented method according to item 21. Identifying a lowest point of the tibial plateau from the SSM; The segmenting further comprises segmenting the 3D model using the first approximate tibial axis and the lowest point to create segmented data, the computer-implemented method according to claim 21. The segmenting the 3D model further comprises selecting data points from the 3D model existing within and above a segmentation plane, the segmentation plane being perpendicular to the first approximate tibial axis, the segmentation plane being distally at a predetermined distance from the lowest point of the tibial plateau, the computer-implemented method according to claim 23. The lowest point of the tibial plateau is the most distal point of the medial plateau, the computer-implemented method according to claim 24. ​ ​ ​ ​ ​ estimating the sagittal direction of the 3D model; projecting the 3D model in the sagittal direction, resulting in a sagittal projection; finding the last point of the tibial plateau from the sagittal projection, and the segmenting further includes segmenting the 3D model using the last point and the first approximate tibial axis, the computer-implemented method according to claim 21. **Claim 27** the segmenting the 3D model further includes selecting data points from the 3D model that exists within and above the segmentation plane, the segmentation plane being perpendicular to the first approximate tibial axis and the segmentation plane being at a predetermined distance distal to the last point of the tibial plateau, the computer-implemented method according to claim 26. **Claim 28** the outlining the outer perimeter of the tibial plateau further includes identifying a contour region within the segmented data and assigning the outer perimeter based on the contour region, the computer-implemented method according to claim 21. **Claim 29** finding the double tangent includes projecting the data of the outer perimeter and the segmented data within the outer perimeter onto the tibial plane to create projected data, and finding the double tangent on the posterior side of the projected data, the computer-implemented method according to claim 21. **Claim 30** further including identifying the tibial tuberosity within the 3D model by a device and displaying a display of the position of the tibial tuberosity within the 3D model, the computer-implemented method according to claim 21 of the 3D. **Claim 31** identifying the tibial tuberosity includes projecting the 3D model onto a sagittal plane, segmenting the 3D model along a line perpendicular to the second long side, resulting in a tuberosity segmentation, and identifying the foremost part of the tuberosity segmentation as the tibial tuberosity, the method according to claim 30. **Claim 32** the segmenting for creating the segmented data further includes selecting data proximal to a predetermined point distal to the lowest point, the computer-implemented method according to claim 21. **Claim 33** The computer-implemented method of claim 21, wherein fitting the tibial plane further comprises fitting based on the segmented data present within the outer periphery and the outer periphery.

34. When executed by one or more devices, cause the one or more devices to receive a three-dimensional model (3D model) of the proximal tibia; identify a first approximate tibial axis; segment the 3D model using the first approximate tibial axis to create segmented data; define an outer periphery of the tibial plateau from the segmented data; fit to a tibial plane based on the outer periphery and at least a portion of the segmented data, wherein a reference perpendicular to the tibial plane defines a final tibial axis; find a double tangent on the posterior based on the outer periphery and portions of the segmented data; form a tibial grid having a rectangle with a first long side coaxial with the double tangent, a second long side parallel to the first long side identifying an outermost edge of the anterior portion of the outer periphery, a first short side perpendicular to the first long side identifying an innermost portion of the outer periphery, and a second short side perpendicular to the first long side identifying an outermost portion of the outer periphery; overlay and display the tibial grid on the 3D model.

35. When the one or more devices identify the first approximate tibial axis and the lowest point of the tibial plateau, the instructions further cause the one or more devices to morph a statistical shape model (SSM) corresponding to the 3D model to create a morphed SSM and identify the first approximate tibial axis from the morphed SSM.

36. The instructions further cause the one or more devices to identify the lowest point of the tibial plateau from the SSM, and when the one or more devices segment the 3D model, the instructions cause the one or more devices to segment the 3D model using the first approximate tibial axis and the lowest point of the tibial plateau.

37. When the one or more devices segment the 3D model, the instructions cause the one or more devices to select data points from the 3D model that are in and above the segmentation plane, the segmentation plane being perpendicular to the first approximate tibial axis and the segmentation plane being distally located a predetermined distance from the lowest point of the tibial plateau, the non - transitory computer - readable storage medium of claim 36.

38. The instructions cause the one or more devices to estimate the sagittal direction of the 3D model, project the 3D model in the sagittal direction to obtain a sagittal projection as a result, find the last point of the tibial plateau from the sagittal projection, When the one or more devices segment the 3D model, the instructions further cause the one or more devices to segment the 3D model using the last point and the first approximate tibial axis, the non - transitory computer - readable medium of claim 36.

39. When the one or more devices segment the 3D model, the instructions further cause the one or more devices to select data points from the 3D model that are in and above the segmentation plane, the segmentation plane being perpendicular to the first approximate tibial axis and the segmentation plane being distally located a predetermined distance from the last point of the tibial plateau, the computer - implemented method of claim 34.

40. When the one or more devices define the outer perimeter of the tibial plateau, the instructions cause the one or more devices to identify a contour region within the segmented data and assign the outer perimeter based on the contour region, the non - transitory computer - readable storage medium of claim 34.

41. When the one or more devices find the double - tangent, the instructions cause the one or more devices to project the data points of the outer perimeter and the segmented data within the outer perimeter onto the tibial plane to create projected data, find the double - tangent on the posterior side of the projected data, the non - transitory computer - readable medium of claim 34.

42. The non - transitory computer - readable storage medium according to claim 34, wherein the command further causes the one or more devices to identify a tibial tuberosity in the 3D model and to display a display of the position of the tibial tuberosity grid in the 3D model.

43. When the one or more devices identify the tibial tuberosity, the command causes the one or more devices to project the 3D model onto a sagittal plane, segment the 3D model along a line perpendicular to the second long side to result in a tuberosity segmentation, identify the foremost part of the tuberosity segmentation as the tibial tuberosity. The non - transitory computer - readable medium according to claim 42.

44. When the one or more devices are segmented to create segmented data, the command causes the one or more devices to further select data proximal to a predetermined point distal to the lowest point. The non - transitory computer - readable storage medium according to claim 34.

45. When the one or more devices fit to the tibial plane, the command causes the one or more devices to fit based on the outer perimeter and the segmented data existing within the outer perimeter. The non - transitory computer - readable storage medium according to claim 34.

46. A device, comprising one or more processors, receiving a three - dimensional model (3D model) of the proximal tibia, identifying a first approximate tibial axis, segmenting the 3D model using the first approximate tibial axis to create segmented data, drawing a perimeter of the tibial plateau from the segmented data, fitting to a tibial plane based on the outer perimeter and at least a portion of the segmented data, wherein a reference perpendicular to the tibial plane defines a final tibial axis, finding a double - tangent on the posterior based on the outer perimeter and the portion of the segmented data, forming a tibial grid comprising a rectangle having a first long side coaxial with the double - tangent, a second long side parallel to the first long side and identifying the outermost edge of the anterior portion of the outer perimeter, a first short side perpendicular to the first long side and identifying the innermost portion of the outer perimeter, and a second short side perpendicular to the first long side and identifying the outermost portion of the outer perimeter. One or more processors configured to overlay and display the tibia grid on the 3D model, a device comprising.

Images