Automatic placement of reference grid and estimation of anatomical coordinate system

Abstract

Systems and methods are disclosed for a computerized framework that provides novel mechanisms for determining automatic placement of a reference grid and an anatomical reference frame (ARF) of a bone. The disclosed framework operates to implement computerized mechanisms that may determine, provide, and / or display an anatomically correct position of a femoral tunnel and / or other forms of surgical markers that are surgeon dependent for an anterior cruciate ligament (ACL) procedure based on a three-dimensional (3D) model of the distal femur. The disclosed framework also operates to implement computerized mechanisms that may determine, provide, and / or display an anatomically correct position of the tibial tunnel and / or other forms of surgical markers dependent on an ACL procedure surgeon based on a three-dimensional (3D) model of the proximal tibia.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of U.S. Provisional Application No. 63 / 354,953, filed on June 23, 2022, entitled “Automatic Placement of Bernard-Hertel's Grid and Estimation of Anatomical Coordinate System in a 3D Model of a Distal Femur”. This provisional application is incorporated herein by reference as if fully reproduced below. Technical Field

[0003] The present disclosure relates to pre-operative and intra-operative surgical analysis and processing, and more particularly to methods for bone anatomical reference frame (ARF) determination and automated placement of reference grids, such as the Bernard-Hertel (BH) grid and / or the tibial grid. Background Art

[0004] The anterior cruciate ligament (ACL) is one of the key ligaments that provides stability to the knee joint. Participating in sports that involve sudden stops or changes in direction is one of the leading causes of ACL injuries, an example of which is a complete tear of the ACL. As a result, ACL tears are a common medical condition, with more than 200,000 cases each year in the United States alone. The standard treatment is arthroscopic reconstruction, in which the torn ligament is replaced with a tissue graft that is pulled into the knee joint through tunnels opened with a drill in both the femur and tibia. Opening these tunnels in anatomically correct locations ensures knee stability and patient satisfaction, although current failure rates for primary ACL reconstructions range from 10-15%.

[0005] The position and orientation of the femoral tunnel significantly influence the success of the procedure, motivating the need for preoperative planning to properly define the optimal femoral tunnel. Significantly less attention has been given to the surgical technique for accurate tibial tunnel creation compared to the femoral side. However, recent studies have emphasized the importance of tibial tunnel position and revealed that tibial tunnel position influences the outcome of anatomic ACL reconstruction. Placing the tibial tunnel too far anteriorly may result in increased graft tilt and subsequent impingement, whereas placing the graft too far posteriorly may result in increased anterior translation laxity. Summary of the invention

[0006] First, referring to the femur, in order to determine the anatomically correct location of the femoral tunnel in the related art, some surgeons rely on specific anatomical landmarks. However, studies have shown that these landmarks are unreliable and may not even exist in some patients. In order to obtain a more accurate femoral tunnel location, the Bernard-Hertel (BH) grid can be utilized. The BH grid can be used to propose ACL reconstruction techniques and to evaluate tunnel placement after ACL reconstruction.

[0007] As background, the BH grid relates to a quadrant approach for determining the positioning of the femoral insertion. Using lateral radiography, the Blumensaat line can be identified and two additional lines can be drawn perpendicular to this line such that the lines pass through the superficial and deep borders of the lateral femoral condyle. The fourth line to be drawn is parallel to the Blumensaat line and tangent to the inferior border of the condyle. The resulting BH grid consists of a normalized reference system that is independent of the size, shape, and distance of the knee at which the x-ray was acquired. Coordinates on this reference system are given in percentages along the Blumensaat line and perpendicularly.

[0008] Several studies have recently shown that in addition to tunnel location, the orientation of the femoral tunnel can also play an important role in the success of ACL reconstruction surgery. Tunnel orientation is defined as the direction with respect to three anatomical directions (i.e., sagittal, axial, and coronal) that define the anatomical reference frame (ARF) and / or the anatomical coordinate system (ACS).

[0009] Thus, the disclosed system and method provides a novel framework that automatically places a BH mesh and determines the ARF of the bone given a three-dimensional (3D) model of the distal femur. The disclosed method is applicable to different femoral shapes and sizes and reliably replaces the manual process with negligible error.

[0010] In practice, the BH grid system was originally applied to lateral radiography of the knee, and more recent work still utilizes this approach. However, there are studies that have shown that image quality and the orientation of the X-ray tube relative to the patient can affect the accuracy of tunnel positioning measurements. Additionally, it has been observed that computerized tomography (CT) imaging may be more reliable for performing this task. A common aspect between all related technical methods is that, whether using radiography or CT imaging, the placement of the BH grid is a manual process that suffers from observer variability. For example, despite significantly improved inter-observer agreement obtained using CT scanning (compared to radiography), the variability in measuring femoral tunnel positioning is still not negligible, making this process unreliable.

[0011] There are currently several related art methods for estimating the ARF from a 3D model of the femur in an automatic manner, which are considered to be more reliable than manual methods, particularly due to the elimination of intra- and inter-observer variability. These automatic related art methods rely on a template model or atlas, utilize the entire femur (including the femoral head), and / or perform a cylindrical / spherical or elliptical fit to the posterior condyle. Although fully automatic and exhibiting good reliability, the related art methods require more information than the disclosed methods due to their reliance on prior knowledge (e.g., a pre-built model of the entire femur and / or a template model / atlas), and are less versatile and more susceptible to local minima problems. In addition, the sagittal direction is retrieved by the cylindrical / spherical / elliptical fit to the condyle, and there is no guarantee that this direction produces a lateral radiographic view in which the two condyles completely overlap, which is used to apply the BH grid system.

[0012] For this reason, among other things, the automatic ARF estimation method of the related art is not suitable for replacing the sagittal direction estimation step in the automatic BH grid placement algorithm. For example, the related art method attempts to provide an automatic estimation of the ARF without the need to understand the entire femoral model. However, this method is not only not suitable for generating the sagittal direction, but also does not guarantee the overlap of the condyles. In addition, the method cannot reliably determine the axial direction.

[0013] The present disclosure remedies these shortcomings at least in part by providing a computerized method that automatically provides placement of a BH grid and determines the ARF given a 3D model of the distal femur. According to some embodiments, as discussed in more detail below, the disclosed method operates by determining the sagittal direction through alignment of the medial and lateral condyles, and then generates a radiographic view of the distal femur to detect the intercondylar contour. The BH grid is then obtained as a rectangle that is tangent to the contour and encloses the radiographic view of one or both condyles.

[0014] Thus, an improved mechanism is disclosed that provides a novel way to improve the efficiency and accuracy of BH mesh placement and the reliability of BH meshes. For example, the tunnel entry points obtained by the disclosed automated processing method (e.g., algorithm) on 21 different femoral models differed from the manual process by an average of 0.28 mm +- 0.16 mm. This error is approximately seven (7) times smaller than the minimum error reported by existing automated and manual methods.

[0015] According to some embodiments, the disclosed method continues by automatically determining the axial direction using the estimated sagittal plane to retrieve a sagittal view of the bone axis, from which circles can be extracted. By connecting the centers of these circles, the axial direction is obtained. The cross product between the sagittal direction and the axial direction produces the coronal direction, thereby providing a complete ARF.

[0016] Unlike related art methods for automatic estimation of the BH grid, the disclosed and deployed method does not require alignment with a template model, thereby proving that the operation uses less information and assumptions, which enables finding the sagittal direction in a more efficient and accurate manner without the need for initialization. In fact, as discussed herein, the disclosed method is not susceptible to local minima problems. In addition, the disclosed detection of the Blumensaat line does not depend on the curvature pattern of the intercondylar contour, thereby enabling it to be applicable to different types of morphologies. Similarly, compared to existing automatic methods for ARF estimation, the disclosed executable method does not require prior information and does not require an entire femoral model, which enables it to be applicable to a wider variety of input models.

[0017] Turning now to the tibia, the measurement of the position of the tibial tunnel has recently been completed using a 3D model of the proximal tibia obtained from a CT scan or a three-dimensional volume rendering image, in which a rectangular grid is placed so that when the axial view of the bone is considered, it surrounds the tibial plateau. This method is called the quadrant method. The first step of the algorithm is to define the axial direction, which is considered to be the normal of the tibial plateau. Then, from the axial view of the tibia, the method looks for a line that is double-cut with the posterior contour of the platform, and obtains the remainder of the grid so that the sides of the rectangle are tangent to the anterior contour, the medial contour, and the lateral contour of the platform. The grid obtained consists of a normalized reference system that is independent of the size and shape of the knee. The coordinates in this reference system are given in percentages along the anterior-posterior (AP) and medial-lateral (ML) directions.

[0018] The purpose of the tibial mesh is to locate the internal point of the ACL tibial tunnel. In order to fully define the tunnel, an external point is also determined. Recent work has considered the localization of the tibial tuberosity to find the external point, claiming that the external point should be 1 cm to 2 cm medial to the tibial tuberosity.

[0019] Related art methods for defining and measuring the positioning of the tibial tunnel also rely on radiographs of the knee. However, recent studies have also shown that the use of radiographs can be unreliable due to inadequate positioning of the knee relative to the X-ray tube, and that CT scans and three-dimensional volume-rendered images provide more reliable results. With a 3D model of the proximal tibia, the quadrant method has become the most widely used related art for measuring the location of the internal points of the tibial tunnel. A common aspect between many of the related art techniques is that the placement of the tibial mesh is a manual process, time consuming and inevitably suffers from observer variability.

[0020] The location of the tibial tuberosity can be used to determine the external orifice of the tibial tunnel and, in many cases, this location is determined by visual observation of the patient.

[0021] This document discloses a method that, given a 3D model of the proximal tibia, automatically provides placement of the tibial mesh, the anatomical reference frame of the tibia, and the positioning of the tibial tuberosity. The method is robust to different tibia shapes and sizes and is sufficiently reliable to replace the manual process with negligible error.

[0022] The disclosed system and method provide a computerized framework that addresses current shortcomings in the prior art by, among other things, providing a novel mechanism for automatically placing a BH mesh, a tibial mesh, and automatically determining the ARF of the femur and tibia.

[0023] According to one or more embodiments, the present disclosure provides a non-transitory computer-readable storage medium for executing the above technical steps. The non-transitory computer-readable storage medium has computer-readable instructions tangibly stored or tangibly encoded thereon, and when the computer-readable instructions are executed by a device, at least one processor executes a method for providing a novel mechanism for automatically placing a BH grid and automatically determining an ARF.

[0024] According to one or more embodiments, a system is provided, comprising one or more computing devices and / or apparatuses configured to provide functionality according to such embodiments. According to one or more embodiments, the functionality is embodied in steps of a method performed by at least one computing device and / or apparatus. According to one or more embodiments, program code (or program logic) executed by (one or more) processors of a computing device to implement functionality according to one or more such embodiments is contained in, contained by, and / or contained on a non-transitory computer-readable medium. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Features and advantages of the present disclosure will be apparent from the following description of the embodiments shown in the accompanying drawings, in which reference numerals refer to the same parts in various views. The accompanying drawings are not necessarily drawn to scale, but emphasis is placed on illustrating the principles of the present disclosure:

[0026] Figure 1 is a block diagram of an exemplary configuration in which the systems and methods disclosed herein may be implemented according to some embodiments of the present disclosure;

[0027] Figure 2 is a block diagram illustrating components of an exemplary system according to some embodiments of the present disclosure;

[0028] Figure 3 shows an exemplary data flow according to some embodiments of the present disclosure;

[0029] Figure 4 depicts non-limiting exemplary embodiments of the disclosed technology in accordance with some embodiments of the present disclosure;

[0030] Figure 5 depicts non-limiting exemplary embodiments of the disclosed technology in accordance with some embodiments of the present disclosure;

[0031] Figure 6 depicts non-limiting exemplary embodiments of the disclosed technology in accordance with some embodiments of the present disclosure;

[0032] Figure 7 depicts non-limiting exemplary embodiments of the disclosed technology in accordance with some embodiments of the present disclosure;

[0033] Figure 8 depicts non-limiting exemplary embodiments of the disclosed technology in accordance with some embodiments of the present disclosure;

[0034] Fig. 9 depicts non-limiting exemplary embodiments of the disclosed technology in accordance with some embodiments of the present disclosure; and

[0035] Figure 10 (including Fig. 10A and 10B ) shows an exemplary data flow according to some embodiments of the present disclosure;

[0036] Fig.11 A rendering of a 3D model of an exemplary proximal tibia is shown.

[0037] Fig.12 A rendering of the 3D model is shown showing the first approximate tibial axis and the lowest point.

[0038] Fig.13 A rendering of a 3D model including an exemplary outer perimeter is shown.

[0039] Fig.14 A rendering of the 3D model is shown, highlighting the outer perimeter as well as portions of the segmented data.

[0040] Fig.15 A perspective view of a rendering of the 3D model including the tibial plane is shown.

[0041] Fig.16 A rendering of the 3D model including the tibial plane is shown in a side elevation view.

[0042] Fig.17 The projection of the data points of the outer perimeter onto the final tibial plane and the projection of the data points within the outer perimeter as well as the bitangent lines are shown.

[0043] Fig.18 A tibial mesh is shown. Fig.17 Projection.

[0044] Fig.19 A rendered axial view of the 3D model including the tibial mesh is shown.

[0045] Fig. 20 A rendering of the 3D model including the tibial plane is shown in a side elevation view.

[0046] Fig.21 A rendering of the 3D curvature values ​​projected onto the axial plane is shown.

[0047] Fig. 22 is a block diagram illustrating a computing device, which shows an example of the device used in various embodiments of the present disclosure. DETAILED DESCRIPTION

[0048] The present disclosure will now be described more fully below with reference to the accompanying drawings, which form a part of the present disclosure and illustrate certain exemplary embodiments by non-limiting illustrations. However, the subject matter may be embodied in various different forms, and thus the subject matter covered or claimed is intended to be interpreted as not being limited to any exemplary embodiment set forth herein; the provision of exemplary embodiments is merely illustrative. Likewise, it is intended to provide a reasonably broad scope for the subject matter claimed or covered. For example, among other things, the subject matter may be embodied as a method, device, component, or system. Thus, an embodiment may, for example, take the form of hardware, software, firmware, or any combination thereof (except the software itself). Therefore, the following detailed description is not intended to be understood in a limiting sense.

[0049] Throughout the specification and claims, terms may have nuanced meanings that are suggested or implied by the context beyond those explicitly stated. Likewise, the phrase "in one embodiment" as used herein does not necessarily refer to the same embodiment, and the phrase "in another embodiment" as used herein does not necessarily refer to a different embodiment. For example, it is intended that the claimed subject matter includes a combination of all or part of the exemplary embodiments.

[0050] In general, terms can be understood at least in part from the use in the context. For example, as used herein, terms such as "and", "or" or "and / or" can include various meanings that can depend at least in part on the context in which such terms are used. Generally, if used in an association list, such as A, B or C, "or" is intended to represent A, B and C used in an inclusive sense here, and A, B or C used in an exclusive sense here. In addition, as used herein, the term "one or more", which depends at least in part on the context, can be used to describe any feature, structure or characteristic in a singular sense, or can be used to describe a combination of features, structures or characteristics in a plural sense. Similarly, terms such as "a", "an" or "the" can again be understood to express a single use or to express multiple uses, depending at least in part on the context. In addition, the term "based on" can be understood to not necessarily be intended to express a set of exclusive factors, but can allow the presence of additional factors that are not necessarily explicitly described again, which depends at least in part on the context.

[0051] The present disclosure is described below with reference to the block diagram and operation diagram of method and device.It should be understood that each frame of block diagram or operation diagram and the combination of frames in block diagram or operation diagram can be implemented by means of analog or digital hardware and computer program instructions.These computer program instructions can be provided to the processor of general-purpose computer to change its function as described in detail herein, provided to special-purpose computer, ASIC or other programmable data processing equipment, so that the instruction implementation block diagram or the function / action specified in one or more operation frames via the processor of computer or other programmable data processing equipment is performed.In some alternative embodiments, the function / action pointed out in the frame can not be carried out in the order pointed out in the operation diagram.For example, two frames shown in succession can actually be performed substantially simultaneously, or these frames can sometimes be performed in reverse order, depending on the function / action involved.

[0052] Unless otherwise limited, the terms "connect", "couple" and "install" and their variations are used extensively herein and cover direct and indirect connections, couplings and installations. In addition, the terms "connect" and "couple" and their variations are not limited to physical or mechanical connections or couplings. In addition, terms such as "upper", "lower", "bottom", "top", "front", "rear", "upper", "lower", "upward", "downward" and other orientation descriptors are intended to facilitate the description of the exemplary embodiments of the present disclosure, and are not intended to limit the structure of the exemplary embodiments of the present disclosure to any particular position or orientation. Those skilled in the art understand that degree terms such as "substantially" or "approximately" refer to a reasonable range around a given value, and include a given value and a range outside a given value, such as general tolerances associated with the manufacture, assembly and use of an embodiment. When referring to a structure or characteristic, the term "substantially" includes characteristics that are primarily or completely present in the characteristic or structure.

[0053] For the purposes of this disclosure, a non-transitory computer-readable medium (or computer-readable storage medium) stores computer data, which may include computer program code (or computer-executable instructions) that can be executed by a computer in a machine-readable form. As an example and not limitation, a computer-readable medium may include a computer-readable storage medium for tangible or fixed data storage, or a communication medium for transient interpretation of code-containing signals. As used herein, a computer-readable storage medium refers to physical or tangible storage (as opposed to a signal), and includes but is not limited to volatile and non-volatile, removable and non-removable media implemented in any method or technology for tangibly storing information such as computer-readable instructions, data structures, program modules, or other data. Computer-readable storage media include but are not limited to RAM, ROM, EPROM, EEPROM, flash memory or other solid-state memory technology, optical storage, cloud storage, magnetic storage devices, or any other physical or material medium that can be used to tangibly store the desired information or data or instructions and can be accessed by a computer or processor.

[0054] For purposes of this disclosure, the term "server" should be understood to refer to a service point that provides processing, database, and communication facilities. By way of example and not limitation, the term "server" may refer to a single physical processor with associated communications and data storage and database facilities, or it may refer to a networked or clustered complex of processors and associated network and storage devices, along with operating software and one or more database systems and application software supporting the services provided by the server. A cloud server is an example.

[0055] For purposes of this disclosure, a "network" should be understood to refer to a network that can couple devices so that communications can be exchanged, for example, between a server and a client device or other type of device, including between wireless devices coupled via, for example, a wireless network. For example, a network may also include a mass storage device, such as a network attached storage device (NAS), a storage area network (SAN), a content distribution network (CDN), or other form of computer or machine readable media. The network may include the Internet, one or more local area networks (LANs), one or more wide area networks (WANs), a wired type connection, a wireless type connection, cellular, or any combination thereof. Likewise, subnetworks that may employ different architectures or may be compatible with or with different protocols may interoperate within a larger network.

[0056] For the purposes of this disclosure, a "wireless network" should be understood as connecting a client device to a network. A wireless network may employ an independent ad hoc network, a mesh network, a wireless LAN (WLAN) network, a cellular network, and the like. A wireless network may also employ a variety of network access technologies, including Wi-Fi, Long Term Evolution (LTE), WLAN, a wireless router (WR) mesh, or 2nd, 3rd, 4th, or 5th generation (2G, 3G, 4G, or 5G) cellular technology, mobile edge computing (MEC), Bluetooth, 802.11b / g / n, and the like. Network access technologies may enable, for example, wide area coverage of devices with client devices of varying degrees of mobility. In short, a wireless network may include almost any type of wireless communication mechanism by which signals may be transmitted between devices such as client devices or computing devices, between networks, or within a network.

[0057] The computing device may be capable of sending or receiving signals, for example, via a wired or wireless network, or may be capable of processing or storing signals, for example, in a memory as a physical memory state, and thus may operate as a server. Thus, a device capable of operating as a server may include, for example, a dedicated rack-mounted server, a desktop computer, a laptop computer, a set-top box, an integrated device combining various features, such as two or more features of the aforementioned devices, etc.

[0058] For purposes of this disclosure, a client (or consumer or user) device, referred to as a user equipment (UE), may include a computing device capable of sending or receiving signals, for example, via a wired or wireless network. A client device may include, for example, a desktop computer or a portable device, such as a cellular phone, a smart phone, a display pager, a radio frequency (RF) device, an infrared (IR) device, a near field communication (NFC) device, a personal digital assistant (PDA), a handheld computer, a tablet computer, a phablet, a laptop computer, a set-top box, a wearable computer, a smart watch, an integrated or distributed device combining various features (such as the features of the aforementioned devices), and the like.

[0059] In some embodiments, as described below, the client device may also be or may be communicatively coupled to any type of known or to-be-known medical device (e.g., any type of Class I, Class II, or Class III medical device), such as, but not limited to, an MRI machine, a CT scanner, an electrocardiogram (ECG or EKG) device, a photoplethysmograph (PPG), a Doppler and emission time flow meter, a laser Doppler, an endoscopic device, a neuromodulation device, a neurostimulation device, the like, or some combination thereof.

[0060] System or framework

[0061] refer to Figure 1 , depicts a system (or framework) 100 that includes a UE 106 (e.g., a client device), a network 102, a cloud system 104, and a surgical engine 200. The UE 106 can be any type of device, such as, but not limited to, a mobile phone, a tablet, a laptop, a personal computer, a sensor, an Internet of Things (IoT) device, an autonomous machine, and any other device equipped with a cellular, wireless, or wired transceiver. In some embodiments, as discussed above, the UE 106 can also be a medical device, or another device that is communicatively coupled to the medical device so as to receive readings from the sensors of the medical device. For example, in some embodiments, the UE 106 can be a user's smartphone (or, for example, an office / hospital device) that is connected to a peripheral nerve modulation device via, for example, WiFi, Bluetooth Low Energy (BLE), or NFC. Thus, in some embodiments, the UE 106 can be configured to receive data from sensors associated with the medical device, as discussed in more detail below. The following at least references Fig. 10A and 10B Further discussion of UE 106 is provided.

[0062] Network 102 may be any type of network, such as, but not limited to, a wireless network, a cellular network, the Internet, a local area network, or a wide area network. As discussed herein, network 102 may facilitate Figure 1 Connections of the components of system 100 are shown.

[0063] The cloud system 104 can be any type of cloud operating platform and / or network-based system on which applications, operations, and / or other forms of network resources can be located. For example, the system 104 can correspond to a service provider, a network provider, and / or a medical provider from which services and / or applications can be accessed, obtained, or executed. In some embodiments, the cloud system 104 may include (one or more) servers and / or information databases accessible via the network 102. In some embodiments, a database (not shown) of the system 104 can store data and metadata associated with local and / or network information related to (one or more) users of the UE 106, patients, and the UE 106, as well as services and applications provided by the cloud system 104 and / or the surgical engine 200.

[0064] As discussed in more detail below, the surgical engine 200 includes components for determining automatic placement of a reference grid of bones, such as a BH grid and associated tunnel placement through the ARF of the femur, or a reference grid and associated tunnel placement through the ARF of the tibia. Figure 3-8 Embodiments of how the engine 200 operates and functions, as well as the capabilities and other functions it includes and performs, are discussed in greater detail.

[0065] According to some embodiments, the surgical engine 200 may be a dedicated machine or processor and may be hosted by a device on the network 102, within the cloud system 104, and / or on the UE 106. In some embodiments, the engine 200 may be hosted by a peripheral device (e.g., a medical device as discussed above) connected to the UE 106.

[0066] According to some embodiments, the surgical engine 200 may function as an application provided by the cloud system 104. In some embodiments, the engine 200 may function as an application installed on the UE 106. In some embodiments, such an application may be a web-based application (e.g., such as provided by the UE 106) accessed from the cloud system 104 via the network 102. Figure 1 In some embodiments, the engine 200 may be configured and / or installed as an enhancement script, program, or application (e.g., a plug-in or extension) of another application or program provided by the cloud system 104 and / or executed on the UE 106.

[0067] like Figure 2As shown, according to some embodiments, the surgical engine 200 includes a model module 202, an estimation module 204, a placement module 206, and a display module 208. It should be understood that the (one or more) engines and modules discussed herein are non-exhaustive, as additional or fewer engines and / or modules (or sub-modules) may be applicable to embodiments of the systems and methods discussed. Further details of the operation, configuration, and functionality of the engine 200 and each of its modules, as well as its role in embodiments of the present disclosure, will be discussed below. This specification first turns to processes associated with the femur.

[0068] Reference mesh and anatomical reference for the femur

[0069] Steering Figure 3 , depicts a process 300 detailing a non-limiting exemplary embodiment of computerized operation of the disclosed framework for determining the ARF of a bone based on automated placement of a BH grid.

[0070] According to some embodiments, as discussed herein, a framework (e.g., engine 200) may perform a method (such as provided at least via process 300) of determining a sagittal direction by alignment of the medial and lateral condyles, and then generating a radiographic view of the distal femur to detect an intercondylar contour. A BH grid is then obtained as a rectangle that is tangent to the contour and encloses a radiographic view of one or both condyles. The disclosed method continues by automatically determining an axial direction by retrieving a sagittal view of the bone axis using an estimated sagittal plane, from which a circle may be extracted. An axial direction is obtained by robustly connecting the centers of these circles. The cross product between the sagittal and axial directions produces a coronal direction, thereby providing a complete ARF.

[0071] The disclosed framework implemented by engine 200 performing the operations detailed as part of process 300 is adaptable to different femoral shapes and sizes, and its improved reliability, accuracy, and ease of implementation can replace conventional processing medical procedures currently relied upon by professionals.

[0072] According to some embodiments, step 302 of process 300 may be performed by model module 202 of surgical engine 200 ; steps 304 - 306 and 310 - 312 may be performed by estimation module 204 ; steps 308 and 314 may be performed by placement module 206 ; and step 316 may be performed by display module 208 .

[0073] Process 300 begins at step 302, where engine 200 receives input identifying a 3D model of the distal femur. According to some embodiments, the identification of the 3D model may be based on, but not limited to, a request to generate a 3D model, a search and retrieval of a 3D model, and / or the uploading and / or downloading of a 3D model. In some embodiments, the input may be in the form of an image, a message, a multimedia item, and / or any other type of known or yet to be known format for engine 200 to receive and process to display digital content corresponding to a patient's bone, particularly a model (e.g., a 3D model) of the distal femur.

[0074] It should be understood that while the discussion herein will focus on a 3D model of the distal femur, it should be construed as limiting, as other types, formats, and forms of models of other types of bones (e.g., the proximal tibia, discussed below) may be utilized without departing from the scope of the present disclosure.

[0075] In step 304, engine 200 performs an estimation of the sagittal orientation based on the input received from step 302. According to some embodiments, step 304 involves engine 200 estimating (e.g., not registering or initializing, as in conventional methods) the sagittal orientation of the bone based on each pair of points whose normal vectors are orthogonal to the vector connecting them. According to some embodiments, this Figure 4 This is shown in example 400 in which an exemplary distal femur 402 has pairs of points P1 and P2 and normal vectors N1 and N2 identified thereon, respectively.

[0076] According to some embodiments, step 304 may involve calculating the normal of each point (or at least a set of points on the bone) based on the input of the 3D model. According to some embodiments, when searching for points by finding a region of interest (ROI) that contains the condylar surface, step 304 may be limited in the search domain, where only points on the ROI are considered (rather than the complete 3D model). In some embodiments, the ROI can be found by registering the 3D model with a template model using a statistical shape model (SSM), by 3D curvature analysis, using a deep learning framework, or some combination thereof.

[0077] The engine 200 can then analyze the calculated normals and determine a pair of points (e.g., P1 and P2) for which the corresponding normals (N1 and N2) are parallel, and the vector "v" connecting P1 and P2 is orthogonal to N1 and N2, such as Figure 4 As shown in the figure.

[0078] The engine 200 can then determine the sagittal direction based on the assumption v=P2-P1. According to some embodiments, the vector v connecting each selected pair of points is composed of an assumption of the sagittal direction. Based on this sagittal direction assumption, an estimate of the sagittal direction can be determined. In some embodiments, the assumption can be represented as a 3D point and then clustered, whereby the median of the cluster can be calculated. In some other embodiments, a random sampling consensus (RANSAC) or other robust estimation model (e.g., Hough transform) can be applied to a set of assumptions for estimating the sagittal direction.

[0079] According to some embodiments, step 304 may also involve determining a lateral-medial orientation in the sagittal direction by identifying a lateral condyle and a medial condyle (eg, P1 and P2 , respectively).

[0080] In some embodiments, the estimation of the sagittal direction of step 304 may also involve generating a simulated radiographic view or a two-dimensional (2D) intersection map of the femur (as described below with respect to at least steps 306 and Figure 5 The sagittal direction is refined by adjusting the outer boundaries of the condyles so that they overlap. Such generated views / maps can be generated by considering orthogonal or perspective projections.

[0081] Process 300 proceeds from step 304 to step 306, where after having determined an estimate of the sagittal direction (step 304), engine 200 performs an estimate of the Blumensaat line. According to some embodiments, engine 200 accesses a 3D model of the femur and constructs a 2D projection of the number of intersections of the projection rays with the 3D model (referred to as an "intersection map"). Figure 5 A non-limiting example of such a mapping is provided in , where intersection graph 500 includes regions 502, 504, and 506 and curve 508. Figure 5 , region 502 corresponds to zero (0) intersections, region 504 corresponds to a region having two (2) intersections, and region 506 corresponds to a region having four (4) intersections. According to some embodiments, intersection graph 500 enables identification of curve 508 (e.g., curve "C"). Curve 508 corresponds to the contour of the intercondylar region, which can be determined by performing edge detection analysis (or calculation) and retrieving the region between regions having 2 intersections and 4 intersections (e.g., between Figure 5 The curve is determined by comparing the curves in FIG. 5 (between regions 504 and 506, respectively).

[0082] According to some embodiments, determining the Blumensaat line at step 306 may involve finding a line that is tangent to, but does not intersect, curve 508 at the maximum number of points. In other words, regardless of shape, the Blumensaat line is tangent to curve 508, but, while tangential, does not intersect curve 508. This is a novel executable step compared to different types of existing Blumensaat line morphologies because, unlike previous methods, it does not rely on a specific curvature pattern.

[0083] Go to Figure 6 , a 2D projection 600 of the distal femur is shown from a side view, wherein the Blumensaat line 602 is shown relative to the curve 508. Thus, as Figure 6 , and described herein with respect to step 306, the Blumensaat line is tangent to curve 508 at a maximum number of points and does not intersect curve 508 elsewhere. According to some embodiments, the Blumensaat line may be determined based on the use of a template model / SSM, 2D curvature analysis, a deep learning scheme, a voting scheme, clustering, and / or any other type of known or yet to be known heuristics.

[0084] In some embodiments, when curve 408 is a mountain shape, the Blumensaat line can intersect with some areas of the intercondylar profile, and only tangent to it in a specific location (for example, near the intercondylar notch). In some embodiments, the Blumensaat line can be further based on the reverse projection of the 2D points of the intersection diagram 500 to the points on the 3D model. In some embodiments, the reverse projection can be based on the interception plane (from step 304) limited by the sagittal direction. Therefore, according to some embodiments, the 3D points in the 3D model can be retrieved / determined by reversely projecting the intercondylar profile / Blumensaat line onto the 3D model. With these points, the appropriate interception plane of the model can be obtained based on, for example, a plane with a sagittal direction that includes these points. In some embodiments, when the axial direction is determined by performing a circle fit in the axis, such interception plane can be used, as discussed below.

[0085] Process 300 then proceeds from step 306 to step 308, where engine 200 determines the placement of the BH grid. Engine 200 determines the placement (and other characteristics, such as size, proportions, and dimensions) of the BH grid in the sagittal direction (from step 304) and the estimate of the Blumensaat line (from step 306).

[0086] According to some embodiments, step 308 involves the engine 200 placing the BH mesh so that it surrounds the condyle when depicted in a side view of the distal femur. According to some embodiments, edge detection is applied to the intersection graph (e.g., Figure 5500), where the edge corresponding to the curve enclosing the area with zero intersections (which corresponds to the sagittal profile of the condyle) is retrieved. In other words, when considering the orthogonal projection, the engine 200 uses the 2D projection of the number of intersections of the projection ray with the bone model along the sagittal direction. Then, the Blumensaat line ( Figure 6 The line 602) intersects the obtained contour, thereby generating the long edge of the BH grid. Finally, a line parallel to the Blumensaat line tangent to the contour is obtained, and the distance between the two lines is the length of the short edge (width) of the BH grid. Figure 7 An example of this is depicted in FIG, which depicts a 2D projection 700 including Blumensaat lines 602 and a BH grid 702. According to some embodiments, the projection 700 may be a radiographic view or an intersection map obtained from an orthogonal or perspective projection. In some embodiments, if the location of the lateral condyle is known, the edge detection of step 306 may be accomplished by first intercepting the model in the sagittal plane and considering only the lateral condyle to construct the intersection map.

[0087] In some embodiments, step 308 may further involve back-projecting the BH mesh from the 2D model to the 3D model. In these embodiments (which are implemented in step 310), the engine 200 may perform the back-projection so that the BH mesh is displayed as part of or overlaying the 3D model. In some embodiments, the back-projection may be based on the sagittal direction and the interception plane defined by the Blumensaat line.

[0088] Process 300 then proceeds from step 310 to step 312, where engine 200 determines estimates of the axial direction and the coronal direction (eg, the remaining anatomical direction). As discussed herein, the axial direction and the coronal direction are used to determine the ARF.

[0089] According to some embodiments, step 312 may include a set of sub-steps. The first sub-step involves the engine 200 obtaining a sagittal view of the bone from which the contour of the axis is retrieved. In some embodiments, the sagittal view may be an orthogonal projection of the entire or a sectioned femur model, or obtained from the intersection of a sectioning plane and the model. In some embodiments, the sagittal view may be an intersection view (as described above with respect to Figure 5 ), where the contour of the shaft can be obtained based on the transition between regions of 0 intersections and 2 intersections (e.g., regions 502 and 504, respectively).

[0090] In the next sub-step, the engine 200 performs a search for a circle that is tangent to the shaft contour at two points (e.g., simultaneously tangent), wherein a line connecting the centers of the obtained circles provides an estimate of the axial direction. In some embodiments, where the anterior and posterior cortices of the femur are known, the search can be limited by considering only pairs containing one point from each cortex. According to some embodiments, the axial direction may alternatively be determined based on a determined relationship between a fixed angle (at a predetermined value) relative to the Blumensaat line. In some embodiments, the axial direction may alternatively be determined based on a cylindrical fitting method utilizing the size and value of the shaft region.

[0091] The engine 200 may then determine the coronal direction via the cross product between the sagittal direction and the axial direction.

[0092] Figure 8 An example of a femur model 800 is depicted in which circles 802 and 804 are depicted. As described above, circles 802 and 804 are tangent to the shaft contour and can be obtained as follows. First, the normal at each point in the shaft contour is calculated. Then, all pairs of contour points are generated and lines passing through them parallel to the corresponding normal vectors are intersected. According to Figure 8 , with direction n A The point P A The line with direction n P The point P P The lines of φ intersect at the center of the circle 802, which belongs to the axial direction. The axial direction is obtained by connecting all the intersection points equidistant from the point under consideration, which corresponds to the center of the circle tangent to the shaft profile.

[0093] According to some embodiments, the step of connecting these points can be performed using any known or yet to be known technique, algorithm or mechanism, such as but not limited to standard or robust line fitting, clustering schemes, Hough transforms and / or any other known or yet to be known technique for estimating and determining lines (and their distances / lengths) from multiple groups of points.

[0094] Returning to process 300, process 300 proceeds from step 312 to step 314, where the engine 200 generates an ARF of the distal femur. Fig. 9 As shown in , an example of a generated ARF 900 is depicted, which includes a sagittal direction 906, an axial direction 904, and a coronal direction 902.

[0095] In step 316, the generated ARF may be displayed as an overlay or portion of the 3D model, which may be used in an ACL procedure, as discussed above. In some embodiments, information regarding the ARF, orientation, BH grid, and Blumensaat lines may be stored and used for subsequent ARF projections.

[0096] Thus, at least based on the above discussion, unlike conventional techniques, the disclosed method works without the need to align with a template model or initialize the sagittal orientation. In fact, the disclosed method can be performed without an entire femoral model and does not depend on the curvature pattern of the intercondylar contour. Among other benefits, this enables the disclosed method to be applicable to a wider variety of input models and different types of morphologies, and demonstrates a system that works in a more computationally efficient and accurate manner without being prone to local minima problems.

[0097] Reference mesh and anatomical reference for the tibia

[0098] Turning to Figure 10 (including Fig. 10A and 10B ), depicting process 1000 detailing a non-limiting example of computerized operation of the disclosed framework for placing a tibial mesh and determining an ARF.

[0099] According to some embodiments, as discussed herein, a framework (e.g., engine 200) can perform a method (as provided via at least process 1000) that operates by determining the location of a tibial plateau and a tibial tuberosity. In particular, the method takes as input a 3D model of the proximal tibia and outputs a 3D axial orientation, a rectangle (e.g., a tibial mesh) surrounding the tibial plateau that directly provides estimates of the sagittal and coronal orientations, and the location of the tibial tuberosity, all of which are calculated in an automatic, unsupervised manner. The surgeon can then use the tibial mesh to determine the ACL footprint by using one of the many studies that provide this location in normalized coordinates, and the location of the tuberosity, together with the axial orientation, can help determine the tunnel external orifice.

[0100] Given the quadrant approach applied to the tibial plateau, in various examples, the first step includes a rough and then fine segmentation of the tibial plateau points. Robust plane fitting is then applied to determine the axial orientation. From an axial view of the proximal tibia, a line bi-tangent to the posterior contour of the plateau is estimated, and then parallel and perpendicular lines tangent to the posterior, medial, and lateral plateau contours are determined, providing placement of the grid. The final step of the automated algorithm includes tibial tuberosity localization.

[0101] The tunnel entry points obtained using this method on 13 different femoral models differed from the manual process by an average of 0.55mm±0.35mm. This error is about four times smaller than the minimum error reported in the literature for related art procedures. With respect to the semi-automatic grid placement algorithms of the related art, the disclosed method also compares very favorably, with the average errors of the anterior-posterior distance between automatic placement and manual placement being 0.63mm and 2.4mm, respectively, and the average errors of the medial-lateral distance being 0.59mm and 1.6mm, respectively. The method disclosed herein depends neither on user intervention, nor on large reconstructions of the tibial axis, nor on specific patient positioning during image acquisition, nor on the availability of 3D models of other knee joint components such as the patella. For these reasons, the disclosed method is applicable to a wider variety of input models.

[0102] The disclosed framework, implemented by engine 200 performing the operations detailed as part of process 1000, is adaptable to different tibia shapes and sizes, and its improved reliability, accuracy, and ease of implementation may replace related art processing methods currently relied upon by practitioners.

[0103] According to some embodiments, step 1002 of process 1000 may be performed by the model module 202 of the surgical engine 200; steps 1004-1010 and 1018 may be performed by the estimation module 204; steps 1010-1014 may be performed by the placement module 206; and step 1016 and a portion of step 1018 may be performed by the display module 208.

[0104] Process 1000 starts from step 302, wherein engine 200 receives the 3D model of the proximal end of tibia. According to some embodiments, the reception of the 3D model can be based on but not limited to the request for generating the 3D model, the search and retrieval of the 3D model and / or the uploading and / or downloading of the 3D model. In some embodiments, the input can be in the form of an image, a message, a multimedia project and / or any other type of known or to-be-known format for the engine 200 to receive and process, to display the digital content corresponding to the patient's bone, particularly the model (e.g., 3D model) of the proximal end of tibia. As previously mentioned, although the discussion herein focuses on the 3D model of the proximal end of tibia, it should be understood to be restrictive, because without departing from the scope of the present disclosure, models of other types, formats and forms of other types of bones can be utilized.

[0105] In step 1004, the engine 200 identifies the deepest point (hereinafter referred to as the lowest point) of the first approximate tibial axis and the tibial plateau. In some examples, identifying the lowest point of the first approximate tibial axis and the tibial plateau can involve deforming a statistical shape model (SSM) to correspond to the 3D model to generate a deformed SSM, and identifying the lowest point of the first approximate tibial axis and the tibial plateau from the deformed SSM. In this context, the SSM itself is a three-dimensional model created based on multiple models of the proximal tibia, and the statistically combined model implements an SSM of a standard shape and size representing the proximal tibia. The tibial axis and the lowest point of the SSM have been previously identified. In this case, "deformation" can refer to adjusting or modifying the shape and / or scale of the SSM in the three-dimensional coordinate system to correspond to the 3D model with a predetermined degree of certainty. Therefore, in some examples, deformation can be an iterative process, deforming, checking correspondence and repeating until the correspondence reaches or exceeds a predetermined correspondence. The predetermined correspondence can be relatively low (e.g., 0.6 to 0.8) because the deformed SSM is only used to find the lowest point of the first approximate tibial axis and the tibial plateau. By deforming the SSM to resemble the 3D model, the approximate axial direction of the 3D model and the lowest point of the tibial plateau are determined, which is set in the medial plateau of the overall tibial plateau.

[0106] In yet other examples, identifying a first approximate tibial axis and the lowest point of the tibial plateau may involve the use of a template model, an atlas, a deep learning framework, etc. In yet other examples, identifying a first approximate tibial axis may involve performing a principal component analysis (PCA) on the 3D model to determine the three most dominant mutually orthogonal directions. Thus, the three most dominant mutually orthogonal directions will correspond to three anatomical planes. Identifying a first approximate tibial axis and the lowest point of the tibial plateau is discussed in more detail below with visual reference.

[0107] In step 1006, the engine 200 uses the first approximate tibial axis and the lowest point of the tibial plateau to segment the 3D model to create segmented data. In one example, segmenting the 3D model includes selecting data points from the 3D model that are located in and above the segmentation plane, the segmentation plane being perpendicular to the first approximate tibial axis, and the segmentation plane being a predetermined distance below or distal to the lowest point of the tibial plateau. In other words, all points above or near a specific distance (e.g., 5 mm below or distal to the lowest point) along the first approximate tibial axis are selected from the first approximate tibial axis and the lowest point as segmented data. In this way, the segmented data is very likely to contain all data points of the tibial plateau, but since the segmentation is based on the first approximate tibial axis, there will be additional data points.

[0108] It should be noted that segmentation can take many suitable forms. In some examples, data located in and / or above the segmentation plane can be extracted and placed in a new data file or a memory location different from non-segmented data. However, in other cases, data located in and / or above the segmentation plane can be identified in the 3D model in a certain appropriate manner, but the 3D model can retain all original data. Therefore, segmentation can alternatively refer to identifying data located in and / or above the segmentation plane, and should not be interpreted as the isolation of implied data. In other words, segmented data does not need to be isolated data. The following discusses in more detail the use of the first approximate tibial axis and the lowest point of the tibial plateau to segment the 3D model to produce segmented data with visual reference.

[0109] The rough segmentation that produces the segmented data may include points belonging to the axis of the tibia. In some cases, the segmented data may even contain points belonging to the fibula and should not be considered when placing the tibia mesh. For this reason, the exemplary method further refines the tibial plateau determination, as discussed in exemplary steps 1008 and 1010.

[0110] Therefore, in step 1008, engine 200 determines or describes the outer periphery of the tibial plateau from the segmented data. In one example, describing the outer periphery of the tibial plateau includes identifying the contour area in the segmented data, and allocating the outer periphery based on the contour area. More specifically, in one example, refinement works by determining the 3D contour of the tibial plateau and the 3D contour in the tibial plateau. In order to determine the 3D contour in this example, a filter with a high response to the region of large 3D curvature is applied to the segmented data, and specifically selects points with a curvature equal to or higher than a predetermined value. In one example, a group of control points are selected using a deformed SSM, and the control points are a group of high curvature points of the deformed SSM. The entire contour or outer periphery is then obtained by selecting the maximum curvature point within the segmented data that is not far from the control point by a predetermined distance (e.g., 2mm). In other words, the outer periphery of the tibial plateau can be considered as a line passing through a plurality of discrete data points located at a selected point.

[0111] Thereafter, in one example, points within the outer perimeter are selected, for example, by performing an orthogonal projection along the first approximate tibial axis. That is, data points from the segmented data that are within the outer perimeter are identified, for example, by an orthogonal projection, thereby generating a set of data points that include data points for the outer perimeter and at least a portion of the segmented data. Determining or delineating the outer perimeter of the tibial plateau from within the segmented data is discussed in more detail below with visual reference.

[0112] In step 1010, engine 200 fits the tibial plane based on at least a portion of the outer periphery and the segmented data. The normal of the tibial plane defines the final tibial axis. In one example, given the outer periphery of the tibial plateau and the portion of the segmented data located within the outer periphery, robust plane fitting is performed to determine the plane containing at least some of the data points in the data points of the outer periphery and at least some of the segmented data in the segmented data. In some cases, the 3D model of the tibial plateau can include tibial spines, and therefore robust schemes can be performed to select the inner periphery as a part of the plane fitting process, rather than considering all data points to fit the plane. The tibial plane so determined therefore mathematically defines the "final" location of the tibial plateau, and the normal of the plane defines the final tibial axis. In one example, the "final tibial axis" is simply any line perpendicular to the tibial plateau, because the operational information is the direction, rather than the specific location of the tibial axis.

[0113] In another example, the tibial plane can be directly determined according to the 3D model. That is, an exemplary robust plane fitting can be performed to the entire 3D model as received in step 1002. Any suitable plane fitting algorithm can be used, such as voting scheme, Hough transform, clustering, hypothesis and test algorithm, etc. Therefore, an exemplary direct method can avoid using SSM to determine the first approximate tibial axis. In some cases, the PCA analysis discussed above can be used to limit or restrict search and plane fitting relative to the entire 3D model. When the tibial plane is determined based on the robust plane fitting of all or subject to the PAC analysis limit performed on the 3D model, the tibial plane so determined directly provides the axial direction via the normal of the tibial plane, so this alternative avoids the need to use SSM to determine the first approximate tibial axis again.

[0114] In yet other examples, the tibial plane can be determined by plane fitting only the data defining the outer perimeter depicted in step 1008, regardless of how it is identified. The data for the outer perimeter can be used as a dedicated data set provided to a plane fitting algorithm, and the tibial plane is determined based on the dedicated data set. Any suitable plane fitting algorithm can be used, such as voting schemes, Hough transforms, clustering, hypothesis and test algorithms, etc.

[0115] Regardless of the precise method used to provide the tibial plane and the final tibial axis, the next step in the exemplary method is to place the tibial mesh. Exemplary steps 1012 and 1014 perform the placement. In step 1012, the engine 200 searches for a bitangent at a posterior position based on the outer perimeter and a portion of the segmented data. In one example, searching for the bitangent may include orthogonally projecting the outer perimeter and portions of the segmented data within the outer perimeter onto the tibial plane to generate projection data, and searching for the bitangent on the posterior portion of the projection data.

[0116] That is, to determine a line that is bitangent to the posterior or posterior contour of the tibial plateau, the method projects onto the axial plane by orthogonal projection. Then, in one example, data points on the posterior side of the lateral perimeter are selected with the aid of the approximate anterior-posterior direction retrieved using SSM. The posterior contour can then be divided into medial and lateral sides, and all lines with equal tangents containing one point from each side are generated. The bitangent line is selected as a line that has all points on the line or on the same side of the line and whose direction is equal to the direction of the tangent line.

[0117] In step 1014, the engine 200 forms a tibial mesh comprising a rectangle having: a first long edge, the first long edge being coincident with or coaxial with a bitangent; a second long edge, the second long edge being parallel to the first long edge and identifying an outermost edge of an anterior portion of an outer perimeter; a first short edge, the first short edge being perpendicular to the first long edge and identifying an innermost portion of an outer perimeter; and a second short edge, the second short edge being perpendicular to the first long edge and identifying an outermost portion of an outer perimeter. From the correct placement of the tibial mesh, the sagittal direction and the coronal direction can be obtained as the directions of the long edge and the short edge of the tibial mesh, respectively. Forming the tibial mesh is discussed in more detail below with visual reference.

[0118] In the example about step 1012 above, SSM is used to help determine the front-posterior direction.In the example not using SSM, the front-posterior direction can be estimated based on the tibial spine.Especially, the tibial spine is the highest curvature point in the segmented data, is arranged near the center of mass of the tibial plane, and the tibial spine belongs to the rear side.By finding the tibial spine with an algorithm in the segmented data or directly in the 3D model, the front-posterior direction can be estimated.In some other examples again, the medial-lateral direction for placing a bitangent can be determined with an algorithm by (for example, based on the tibial plane) finding the side of the tibial plateau that contains more points with higher curvature.The side with more data points and higher curvature of the tibial plateau corresponds to the medial side.

[0119] In another example, the bitangent can be determined without explicitly identifying the medial and lateral directions. In particular, in another example, the tangents of all posterior points are determined. Using the tangents, some or all possible tuples of posterior points and corresponding tangents are established. Then, the example selects two tuples for which the two tangents and the connecting line have the same direction. The example selects tuples derived from lines that do not cut through or excise the contour. Therefore, the bitangent is the line corresponding to the tuple with the maximum distance between points.

[0120] In step 1016, the engine 200 overlays and displays the tibial mesh relative to the 3D model. That is, a view of the tibia (e.g., along the final tibial axis) showing the tibial mesh can be provided to the surgeon. The surgeon can select the tunnel aperture location on the tibial plateau based on the tibial mesh.

[0121] A recent publication considered the localization of the tibial tuberosity for determining the external point of the ACL tibial tunnel. Due to the importance of this landmark, the method may also include automatic detection of the tibial tuberosity or tibial tuberosity, which may be helpful to the surgeon when deciding on the localization of the tibial tunnel.

[0122] Thus, exemplary process 1000 may also include step 1018. In step 1018, engine 200 identifies the tibial tuberosity within the 3D model and displays an indication of the location of the tibial tuberosity within the 3D model. In one example, identifying the tibial tuberosity may include projecting the 3D model onto a sagittal plane, segmenting the 3D model along a line perpendicular to the second long edge, thereby producing a tuberosity segmentation, and identifying the most anterior portion of the tuberosity segmentation as the tibial tuberosity. By back-projecting onto the 3D model, the center of the tuberosity is reconstructed. The ridge of the tuberosity can be obtained as the intersection of the tuberosity region with a plane whose normal is in the sagittal direction and passes through the most anterior point (the center of the tuberosity).

[0123] In other examples, the orthogonal projection and segmentation relative to the line perpendicular to the second long edge can be omitted, and instead a curvature hint and / or a sphere can be fitted to the data from the 3D model to estimate the location of the tibial tuberosity. In still other cases, the location of the tibial tuberosity can be estimated using a template model, an atlas, a deep learning framework, etc. The specification now turns to a visual explanation of an exemplary method.

[0124] Fig.11 A rendering of a 3D model of an exemplary proximal tibia is shown. In particular, Fig.11 100, tibial plateau 1102, tibial spine 1104, tibial tuberosity 1106, and a portion of tibial axis 1108. On the medial side of tibial plateau 1102 is nadir 1110, again the lowest or most distal point of tibial plateau 1102 (and more specifically, the medial plateau). Fig.11 The coordinate system of shows the medial and lateral axes, the proximal and distal axes, and the anterior and posterior axes.

[0125] Fig.12A rendering of a 3D model is shown showing a first approximate tibial axis 1200 and a nadir 1110. In an exemplary method (e.g., step 1004), the first approximate tibial axis 1200 and the nadir 1110 are determined, for example, by deforming the SSM. However, any suitable technique may be used to find the first approximate tibial axis 1200 and the nadir 1110. In the exemplary method, the data points are segmented by identifying all data points of the 3D model that are above a segmentation plane. The segmentation plane is located a predetermined distance (e.g., 5 mm) below the nadir 1110, and the segmentation plane is perpendicular to the first approximate tibial axis 1200. Fig.12 , segmented data 1202 is highlighted with stippling.

[0126] Fig.13 A rendering of a 3D model including an exemplary outer perimeter 1300 is shown. In an exemplary method (e.g., step 1008), segmented data is refined to better depict the outer perimeter of the tibial plateau. In one example, the deformed SSM defines a control set or initial set of peripheral data points of the tibial plateau, and the initial set 1302 of peripheral data points is shown as a square, only some of which are associated with reference numerals. In addition, the segmented data is subjected to an algorithm for identifying high-profile areas. High-profile areas such as area 1304 are connected to the initial set 1302 of peripheral data points to define an outer perimeter 1300 (a combination of squares and circles). Therefore, the data points of the outer perimeter 1300 and the data points located within the outer perimeter can be considered as detailed or refined segmentation.

[0127] Fig.14 A rendering of the 3D model is shown, which highlights the segmented data, the outer perimeter, and the portion of the segmented data that is within the outer perimeter. Fig.14 14. Visible in the segmented data 1202, the first approximate tibial axis 1200, and the lowest point 1110. Within the segmented data 1202, the data points of the outer perimeter and the data points of the segmented data that are within the outer perimeter (ie, region 1400) are defined.

[0128] Fig.15 A perspective view of the rendering of the 3D model including the tibial plane is shown. In the exemplary method (e.g., step 1010), the data points defining the outer periphery and the data points of the segmented data located within the outer periphery are subjected to a plane fitting algorithm to find a well-fitting mathematical plane, such as the tibial plane 1500. It should be noted that all data points of the outer periphery and / or the data points of the segmented data located within the outer periphery are not all located within the tibial plane 1500. Once the tibial plane 1500 is determined, the line perpendicular to the tibial plane 1500 defines the final tibial axis 1502. Note that the tibial spine 1104 is located within the outer periphery ( Fig.15The tibial plane 1500 rises above the tibial plane 1500 when not specifically depicted in the figure, and defines a posterior portion of the tibial plane 1500 and is located within the posterior portion.

[0129] Fig.16 A rendering of the 3D model including the tibial plane is shown in a side elevation view. Fig.16 The normal or perpendicular relationship of the final tibial axis 1502 to the tibial plane 1500 is better shown. Fig.16 The views show how portions of the data that were originally located within the outer perimeter may rise above the tibial plane 1500 (e.g., the tibial spine 1104) or fall below the tibial plane (e.g., the lateral condyle 1504).

[0130] Fig.17 Shown are the data points of the outer periphery projected onto the tibial plane and the data points and the bitangents in the outer periphery. In an exemplary method (e.g., step 1012), the data points of the outer periphery 1300 and the data (point 1700) in the outer periphery can be orthogonally projected onto a plane perpendicular to the final tibial axis 1502, such as the tibial plane 1500 itself (e.g., the plane of the page). The front portion 1702 and the rear portion 1704 of the projection can be determined in any suitable form, such as by using SSM or by the tibial spine. In the case of determining the rear portion 1704, the bitangent 1706 can be placed, as discussed in detail above. The bitangent 1706 defines the line in the coronal plane. The line perpendicular to the final tibial axis 1502 and perpendicular to the bitangent 1706 can define the coronal direction.

[0131] Fig.18 A tibial mesh is shown. Fig.17 In an exemplary method (e.g., step 1014), setting the tibial mesh can involve fitting or finding a minimum rectangle around the projected data points, the rectangle having a first long edge 1800 coaxial with the bitangent 1706, a second long edge 1802 parallel to the bitangent 1706, and two short edges 1804 and 1806 parallel to each other and perpendicular to the long edges 1800 and 1802. The short edges 1804 and / or 1806 thus define a coronal direction. The rectangle formed by the edges 1800, 1802, 1804, and 1806 can be considered to define a tibial mesh 1820, whose origin or location (0%, 0%) is at Fig.18 , and (100%, 100%) is in the upper right corner. The surgeon can then use the tibial mesh 1820 to determine the ACL coverage using one of many studies that provide locations in normalized coordinates.

[0132] Fig.19An axial view of a rendering of a 3D model including a tibial mesh is shown. In an exemplary method (e.g., step 1016), the tibial mesh 1820 determined as described above can be overlaid and displayed, for example, on a computer screen within a surgical scene, so that the surgeon can select the location of the orifice of the tibial tunnel on the tibial plateau. Note the location of the tibial tuberosity 1106.

[0133] In yet other examples, the sagittal direction may be estimated as the direction of a line passing through the most posterior lateral point and the most posterior medial point of orthogonal projections in the axial direction. Fig.19 In the view of FIG. 1 , the sagittal direction can be viewed as a line 1900 that intersects the most posterior lateral point 1902 and the most posterior medial point 1904. It should be noted that the sagittal direction determined from the most posterior lateral point and the most posterior medial point can be slightly different from the sagittal direction determined to coincide with the bitangent line.

[0134] Fig. 20 A rendered sagittal or lateral elevation view of a 3D model including a tibial plane is shown. In an exemplary method (e.g., step 1018), the tibial tuberosity 1106 can be represented by Fig. 20 , the 3D model is projected in the sagittal direction as shown in , and the data points located in front of the segmentation line 2000 are programmatically determined by reducing or segmenting, wherein the segmentation line 2000 is parallel to the final tibial axis (not specifically shown), and the segmentation line 2000 intersects the anterior long edge 1802 of the tibial mesh 1808. Therefore, the tibial tuberosity 1106 can be programmatically determined as the maximum value or vertex of the segmented data points (e.g., the most anterior point 2002). With the tibial mesh 1820 programmatically placed and the tibial tuberosity 1106 programmatically determined, the clinician can plan the tibial tunnel positioning and send the information to the intraoperative device.

[0135] Still refer to Fig. 20 To further assist the surgeon in placing the tibial tunnel, and in other examples, additional scale-invariant information may be generated. Fig. 20 In the case of projecting the 3D model in the sagittal direction as shown, another example finds the most posterior point 2030. The most posterior point 2030 and a line 2032 parallel to the tibial mesh 1820 together define the anterior-posterior distance (AP in the figure, and hereinafter referred to as the AP distance). The AP distance can be as Fig. 20 The depictions are shown, for example, at 25% of the AP distance, 43% of the AP distance, and 62% of the AP distance. These depictions can help determine the placement of the tibial tunnel, or at least the location of the orifice of the tibial tunnel on the tibial plateau.

[0136] In the example discussed above, the splitting plane is selected to be a plane perpendicular to the first approximate tibial axis at a predetermined distance below or distal to the lowest point 1110 ( Fig.11 ). However, in other examples, the segmentation plane may be based on the posterior-most point of the tibia 2030. In particular, in these examples, the sagittal direction may be estimated in any suitable form, such as using a sagittal direction determined from analysis of a 3D model of the femur from the same patient (imaging acquired with the patient's leg extended). Based on the estimated sagittal direction, the segmentation plane may be as follows Fig. 20 The projected 3D model is shown, and the posterior-most point 2030 is determined. Using the posterior-most point 2030, the segmentation plane can again be a plane perpendicular to the first approximate tibial axis and containing the posterior-most point 2030. In other cases, the segmentation plane can be a predetermined distance (e.g., 3 mm) below or distal to the posterior-most point 2030, and again the segmentation plane is perpendicular to the first approximate tibial axis. Based on the segmentation data determined based on the alternative segmentation plane, the various exemplary steps discussed above can be used to place the tibial mesh.

[0137] Fig.21 Rendering of the 3D curvature values ​​projected onto the axial plane is shown. As described above, in alternative embodiments, the tibial plateau may be determined without the use of prior information such as the SSM. In particular, in alternative embodiments, a 3D model or at least segmented data of the 3D model may be evaluated to determine 3D curvature values, such as regions 2100, 2102, and the location of the tibial spine in region 2104. The 3D curvature values ​​may be used to generate a two-dimensional image by projecting the 3D curvature values ​​onto the axial plane, such as the projection of the tibial plateau onto the axial plane. Fig.21 Any suitable morphological operation can be applied to the 2D projection to find a closed contour, wherein the closed contour thus defines an outer perimeter. If necessary, the data defining the outer perimeter can be back-projected to the segmented data to identify the portion of the segmented data located within the outer perimeter.

[0138] Fig. 22 is a block diagram illustrating a computing device 2200 (eg, UE 106 as discussed above), which illustrates an example of a client device or a server device used in various embodiments of the present disclosure.

[0139] Depending on the deployment or use of the device 2200, the computing device 2200 may include: Fig. 222254, keyboard 2256, illuminator 2258, tactile interface 2262, GPS receiver 2264, or camera / sensor 2266. Some devices may include additional components not shown, such as a GPU device, a cryptographic coprocessor, an AI accelerator, or other peripheral devices.

[0140] like Fig. 22 As shown, the device 2200 includes a central processing unit (CPU) 2222, which communicates with a mass storage 2230 via a bus 2224. The computing device 2200 also includes one or more network interfaces 2250, an audio interface 2252, a display 2254, a keyboard 2256, an illuminator 2258, an input / output interface 2260, a tactile interface 2262, an optional GPS receiver 2264 (and / or an interchangeable or additional GNSS receiver), and (one or more) cameras or other optical, thermal or electromagnetic sensors 2266. The device 2200 may include one camera / sensor 2266 or multiple cameras / sensors 2266. The positioning of (one or more) cameras / (one or more) sensors 2266 on the device 2200 may vary per device 2200 model, per device 2200 capabilities, etc., or some combination thereof.

[0141] In some embodiments, CPU 2222 may include a general-purpose CPU. CPU 2222 may include a single-core or multi-core CPU. CPU 2222 may include a system on chip (SoC) or a similar embedded system. In some embodiments, a GPU may replace CPU 2222 or be used in combination with it. Mass storage 2230 may include a dynamic random access memory (DRAM) device, a static random access memory device (SRAM) or a flash memory (e.g., NAND flash memory) memory device. In some embodiments, mass storage 2230 may include a combination of such memory types. In one embodiment, bus 2224 may include a peripheral component express interconnect (PCIe) bus. In some embodiments, bus 2224 may include multiple buses instead of a single bus.

[0142] The mass storage 2230 illustrates another example of computer storage media for storing information such as computer readable instructions, data structures, program modules or other data. The mass storage 2230 stores a basic input / output system (“BIOS”) 2240 for controlling the low-level operation of the computing device 2200. The mass storage also stores an operating system 2241 for controlling the operation of the computing device 2200.

[0143] The application 2242 may include computer executable instructions that, when executed by the computing device 2200, perform any of the methods (or portions of methods) previously described in the description of the preceding figures. In some embodiments, software or programs implementing method embodiments may be read from a hard drive (not shown) and temporarily stored in the RAM 2232 by the CPU 2222. The CPU 2222 may then read the software or data from the RAM 2232, process them, and store them to the RAM 2232 again.

[0144] Computing device 2200 may optionally communicate with a base station (not shown) or directly with another computing device.Network interface 2250 is sometimes called a transceiver, a transceiver device, or a network interface card (NIC).

[0145] Audio interface 2252 generates and receives audio signals, such as the sound of human speech. For example, audio interface 2252 can be coupled to a speaker and microphone (not shown) to enable remote communication with others or to generate an audio confirmation for an action. Display 2254 can be a liquid crystal display (LCD), gas plasma, light emitting diode (LED), or any other type of display used with a computing device. Display 2254 can also include a touch-sensitive screen that is arranged to receive input from an object such as a stylus or a finger of a human hand.

[0146] Keyboard 2256 may include any input device arranged to receive input from a user. Illuminator 2258 may provide a status indication or provide light.

[0147] The computing device 2200 also includes an input / output interface 2260 for using devices such as USB, infrared, Bluetooth, etc. TM The haptic interface 2262 provides haptic feedback to the user of the client device.

[0148] The GPS transceiver 2264 can determine the physical coordinates of the computing device 2200 on the surface of the earth, which typically outputs the location as a latitude value and a longitude value. The GPS transceiver 2264 can also use other geolocation mechanisms, including but not limited to triangulation, assisted GPS (AGPS), E-OTD, CI, SAI, ETA, BSS, etc., to further determine the physical location of the computing device 2200 on the surface of the earth. However, in one embodiment, the computing device 2200 can communicate through other components to provide other information that can be used to determine the physical location of the device, including, for example, a MAC address, an IP address, etc.

[0149] For purposes of this disclosure, a module is a software, hardware or firmware (or a combination thereof) system, process or function or component thereof that performs or facilitates the processes, features and / or functions described herein (with or without human interaction or enhancement). A module may include submodules. The software components of a module may be stored on a computer-readable medium for execution by a processor. A module may be integrated with one or more servers, or loaded and executed by one or more servers. One or more modules may be grouped into engines or applications.

[0150] Those skilled in the art will recognize that the methods and systems of the present disclosure can be implemented in many ways and are therefore not limited by the foregoing exemplary embodiments and examples. In other words, functional elements performed by single or multiple components in various combinations of hardware and software or firmware and individual functions can be distributed between software applications at the client level or the server level or both. In this regard, any number of features of the different embodiments described herein can be combined into a single or multiple embodiments, and alternative embodiments with fewer or more than all of the features described herein are possible.

[0151] Functionality may also be distributed, in whole or in part, among multiple components in ways now known or to become known. Thus, a variety of software / hardware / firmware combinations are possible in implementing the functions, features, interfaces, and preferences described herein. Furthermore, the scope of the present disclosure encompasses conventionally known ways of performing the described features and functions and interfaces, as well as those changes and modifications that may be made to the hardware or software or firmware components described herein, as will now and hereafter be understood by those skilled in the art.

[0152] In addition, the embodiments of the methods presented and described as flow charts in the present disclosure are provided as examples to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of the various operations is changed and in which sub-operations described as part of a larger operation are performed independently.

[0153] Although various embodiments have been described for the purposes of this disclosure, such embodiments should not be construed as limiting the teachings of this disclosure to those embodiments. Various changes and modifications may be made to the above elements and operations to obtain results that remain within the scope of the systems and processes described in this disclosure.

[0154] The following clauses define various non-limiting examples.

[0155] Clause 1. A computer-implemented method comprising: a device identifying a three-dimensional (3D) model of a femur; a device analyzing the 3D model and determining a pair of points on the femur, the pair of points having a normal vector orthogonal to a vector connecting the pair of points; a device analyzing the vector and determining a sagittal direction; a device further analyzing the 3D model based on the determined sagittal direction and generating an intersection graph of the 3D model, the intersection graph including information related to curve values ​​of an intercondylar region; a device determining a Blumensaat line based on the intersection graph; a device analyzing the intersection graph based on the Blumensaat line and determining a Bernard-Hertel (BH) mesh, the BH mesh including an area surrounding the condyle when the femur is in a side view; and a device placing a digital representation of the BH mesh as a covering of the 3D model.

[0156] Clause 2. The computer-implemented method of clause 1 further comprises: determining the value of the pair of points and each normal vector; clustering the determined values; and determining a median based on the clustering, wherein the determined sagittal direction corresponds to the median.

[0157] Clause 3. A computer-implemented method according to any preceding clause, further comprising: executing an estimation model by a device; and determining the sagittal direction based on an output of the estimation model.

[0158] Clause 4. The computer-implemented method of any preceding clause, further comprising: identifying a lateral condyle and a medial condyle; and determining a lateral-medial orientation of the sagittal direction based on the identified lateral condyle and medial condyle.

[0159] Clause 5. The computer-implemented method of Clause 4, further comprising refining the sagittal direction based on adjusting outer boundaries of the lateral condyle and the medial condyle.

[0160] Clause 6. The computer-implemented method of any preceding clause, further comprising back-projecting the BH mesh onto the 3D model, the BH mesh being a two-dimensional (2D) model, wherein the back-projection is based on a cutting plane defined by the sagittal direction and a Blumensaat line.

[0161] Clause 7. The computer-implemented method according to any of the preceding clauses further includes: back-projecting points on the intersection graph to points on the 3D model, wherein the points on the intersection graph are two-dimensional (2D); and obtaining a cutting plane of the 3D model based on the sagittal direction; and determining the axial direction by circle fitting.

[0162] Clause 8. The computer-implemented method according to any of the preceding clauses further includes: determining the contour of the femur based on a sagittal view of the femur, the sagittal view being obtained by 2D projection along the sagittal direction; searching for a circle that is tangent to the contour at two points; and estimating a line connecting the respective centers of the circles, wherein the estimated line provides information related to the axial direction.

[0163] Clause 9. The computer-implemented method according to any of the preceding clauses further includes: analyzing the femur based on the sagittal direction and determining a sagittal view of the femur; determining an axial direction based on the sagittal view; and determining a coronal direction based on a cross product of the sagittal direction and the axial direction.

[0164] Clause 10. The computer-implemented method of Clause 9, further comprising determining an anatomical reference frame (ARF) of the femur based on the sagittal direction, the axial direction, and the coronal direction.

[0165] Clause 11. The computer-implemented method according to any of the preceding clauses further includes: identifying an orthogonal projection of the femur; identifying a set of projection rays associated with the femur within the 3D model based on the orthogonal projection; and determining a set of intersection points corresponding to the set of projection rays, wherein the intersection graph includes information associated with the set of intersection points.

[0166] Clause 12. The computer-implemented method according to Clause 12 further includes: analyzing the set of intersection points by edge detection analysis; and determining a curve value of the intercondylar region based on the edge detection analysis, wherein the Blumensaat line is tangent to a curve associated with the curve value.

[0167] Item 13. A non-transitory computer-readable storage medium tangibly encoded with computer-executable instructions that, when executed by one or more devices, cause the one or more devices to: identify a three-dimensional (3D) model of a femur; analyze the 3D model and determine a pair of points on the femur, the pair of points having a normal vector orthogonal to a vector connecting the pair of points; analyze the vector and determine a sagittal direction; further analyze the 3D model based on the sagittal direction and generate an intersection graph of the 3D model, the intersection graph including information related to curve values ​​of an intercondylar region; determine a Blumensaat line based on the intersection graph; analyze the intersection graph based on the Blumensaat line and determine a Bernard-Hertel (BH) mesh, the BH mesh including an area surrounding the condyle when the femur is in a side view; and place a digital representation of the BH mesh as an overlay on the 3D model.

[0168] Clause 14. A non-transitory computer-readable storage medium according to clause 13, wherein the instructions further cause the one or more devices to: determine the value of the pair of points and each normal vector; cluster the values; and determine a median based on the clusters, wherein the sagittal direction corresponds to the median.

[0169] Clause 15. A non-transitory computer-readable storage medium according to any one of clauses 13-14, wherein the instructions further cause the one or more devices to: identify the lateral condyle and the medial condyle; determine the lateral-medial orientation of the sagittal direction based on the identified lateral condyle and the medial condyle; and refine the sagittal direction based on adjustments to the outer boundaries of the lateral condyle and the medial condyle.

[0170] Clause 16. A non-temporary computer-readable storage medium according to any one of clauses 13-15, wherein the instructions also cause the one or more devices to: back-project points on the intersection graph to points on the 3D model, wherein the points on the intersection graph are two-dimensional (2D); obtain a cutting plane of the 3D model based on the sagittal direction; and determine the axial direction by circle fitting.

[0171] Clause 17. A non-transitory computer-readable storage medium according to any one of clauses 13-16, wherein the instructions further cause the one or more devices to: determine the contour of the femur based on a sagittal view of the femur, wherein the sagittal view is obtained by 2D projection along the sagittal direction; search for a circle that is tangent to the contour at two points; and estimate a line connecting the respective centers of the circles, wherein the estimated line provides information related to the axial direction.

[0172] Clause 18. A non-transitory computer-readable storage medium according to any one of clauses 13-17, wherein the instructions further cause the one or more devices to: analyze the femur based on the sagittal direction and determine a sagittal view of the femur; determine an axial direction based on the sagittal view; determine a coronal direction based on a cross product of the sagittal direction and the axial direction; and determine an anatomical reference system (ARF) of the femur based on the sagittal direction, the axial direction, and the coronal direction.

[0173] Clause 19. An apparatus comprising: one or more processors configured to implement the method of clauses 1-12 and / or to implement the instructions of clauses 13-18. Clause 20 is intentionally skipped.

[0174] Item 21. A computer-implemented method, comprising: receiving a three-dimensional model (3D model) of the proximal tibia by a device; identifying a first approximate tibial axis by the device; segmenting the 3D model by the device using the first approximate tibial axis to create segmented data; depicting the outer periphery of the tibial plateau by the device according to the segmented data; fitting a tibial plane by the device based on the outer periphery and at least a portion of the segmented data, the normal of the tibial plane defining a final tibial axis; finding a bitangent on the posterior portion based on the outer periphery and a portion of the segmented data by the device; forming a tibial mesh by the device, the tibial mesh comprising a rectangle having: a first long edge coaxial with the bitangent; a second long edge parallel to the first long edge and identifying the outermost edge of the anterior portion of the outer periphery; a first short edge perpendicular to the first long edge and identifying the innermost portion of the outer periphery; and a second short edge perpendicular to the first long edge and identifying the outermost portion of the outer periphery; and overlaying and displaying the tibial mesh by the device relative to the 3D model.

[0175] Clause 22. A computer-implemented method according to claim 21, wherein identifying the first approximate tibial axis also includes deforming a statistical shape model (SSM) to correspond to the 3D model thereby producing a deformed SSM, and identifying the first approximate tibial axis from the deformed SSM.

[0176] Clause 23. A computer-implemented method according to any one of clauses 21-22, further comprising: identifying a lowest point of the tibial plateau from the SSM; wherein segmentation further comprises segmenting the 3D model using the first approximate tibial axis and the lowest point to create segmented data.

[0177] Clause 24. A computer-implemented method according to clause 23, wherein segmenting the 3D model further comprises selecting data points from the 3D model that are located in and above a segmentation plane, wherein the segmentation plane is perpendicular to the first approximate tibial axis and the segmentation plane is a predetermined distance distal to the lowest point of the tibial plateau.

[0178] Clause 25. The computer-implemented method of Clause 24, wherein the lowest point of the tibial plateau is the most distal point of the medial plateau.

[0179] Clause 26. A computer-implemented method according to any one of Clauses 21-25, further comprising: estimating a sagittal direction of the 3D model; projecting the 3D model in the sagittal direction to generate a sagittal projection; finding the posteriormost point of the tibial plateau from the sagittal projection; and wherein segmentation further comprises segmenting the 3D model using the posteriormost point and the first approximate tibial axis.

[0180] Clause 27. A computer-implemented method according to clause 26, wherein segmenting the 3D model further comprises selecting data points from the 3D model that are located in and above a segmentation plane, wherein the segmentation plane is perpendicular to the first approximate tibial axis and the segmentation plane is a predetermined distance distal to the most posterior point of the tibial plateau.

[0181] Clause 28. The computer-implemented method of any of Clauses 21-27, wherein delineating the outer perimeter of the tibial plateau further comprises identifying a contour region within the segmented data, and assigning the outer perimeter based on the contour region.

[0182] Clause 29. A computer-implemented method according to any one of Clauses 21-28, wherein finding the bitangent further comprises: projecting data of the outer periphery and segmented data within the outer periphery onto the tibial plane to generate projection data; and finding the bitangent in the posterior portion of the projection data.

[0183] Clause 30. The computer-implemented method of any of Clauses 21-29, further comprising identifying, by a device, a tibial tuberosity within the 3D model and displaying an indication of a location of the tibial tuberosity within the 3D model.

[0184] Clause 31. A computer-implemented method according to clause 30, wherein identifying the tibial tuberosity further comprises: projecting the 3D model onto a sagittal plane; segmenting the 3D model along a line perpendicular to the second long edge, thereby producing a tuberosity segmentation; and identifying the anterior-most portion of the tuberosity segmentation as the tibial tuberosity.

[0185] Clause 32. The computer-implemented method of any of Clauses 21-31, wherein segmenting to create the segmented data further comprises selecting data proximal to a predetermined point distal to the lowest point.

[0186] Clause 33. The computer-implemented method of any of Clauses 21-32, wherein fitting the tibial plane further comprises fitting based on the outer perimeter and segmented data within the outer perimeter.

[0187] Item 34. A non-transitory computer-readable storage medium, the non-transitory computer-readable storage medium tangibly encoded with computer-executable instructions, the computer-executable instructions, when executed by one or more devices, cause the one or more devices to: receive a three-dimensional model (3D model) of the proximal tibia; identify a first approximate tibial axis; segment the 3D model using the first approximate tibial axis to create segmented data; delineate the outer periphery of the tibial plateau according to the segmented data; fit a tibial plane based on the outer periphery and at least a portion of the segmented data, the normal of the tibial plane defining a final tibial axis; find a bitangent on the posterior portion based on the outer periphery and a portion of the segmented data; form a tibial mesh, the tibial mesh comprising a rectangle having: a first long edge coaxial with the bitangent; a second long edge parallel to the first long edge and identifying the outermost edge of the anterior portion of the outer periphery; a first short edge perpendicular to the first long edge and identifying the innermost portion of the outer periphery; and a second short edge perpendicular to the first long edge and identifying the outermost portion of the outer periphery; and overlay and display the tibial mesh relative to the 3D model.

[0188] Item 35. A non-transitory computer-readable storage medium according to Item 34, wherein when the one or more devices identify the first approximate tibial axis and the lowest point of the tibial plateau, the instructions also cause the one or more devices to deform a statistical shape model (SSM) to correspond to the 3D model to produce a deformed SSM, and identify the first approximate tibial axis from the deformed SSM.

[0189] Clause 36. A non-transitory computer-readable storage medium according to any one of clauses 34-35, wherein the instructions further cause the one or more devices to identify the lowest point of the tibial plateau from the SSM, and wherein when the one or more devices segment the 3D model, the instructions further cause the one or more devices to segment the 3D model using the first approximate tibial axis and the lowest point of the tibial plateau.

[0190] Clause 37. A non-transitory computer-readable storage medium according to clause 36, wherein when the one or more devices segment the 3D model, the instructions cause the one or more devices to select data points from the 3D model that are located in and above a segmentation plane, wherein the segmentation plane is perpendicular to the first approximate tibial axis and the segmentation plane is a predetermined distance distal to the lowest point of the tibial plateau.

[0191] Clause 38. A non-transitory computer-readable storage medium according to any one of clauses 34-37, wherein the instructions further cause the one or more devices to: estimate the sagittal direction of the 3D model; project the 3D model in the sagittal direction to generate a sagittal projection; and find the posteriormost point of the tibial plateau from the sagittal projection; when the one or more devices segment the 3D model, the instructions cause the one or more devices to segment the 3D model using the posteriormost point and the first approximate tibial axis.

[0192] Clause 39. A computer-implemented method according to any one of clauses 34-38, wherein when the one or more devices segment the 3D model, the instructions also cause the one or more devices to select data points from the 3D model that are located in and above a segmentation plane, wherein the segmentation plane is perpendicular to the first approximate tibial axis and the segmentation plane is a predetermined distance distal to the most posterior point of the tibial plateau.

[0193] Clause 40. A non-transitory computer-readable storage medium according to any one of clauses 34-39, wherein when the one or more devices depict the outer perimeter of the tibial plateau, the instructions also cause the one or more devices to identify a contour area within the segmented data and assign the outer perimeter based on the contour area.

[0194] Clause 41. A non-transitory computer-readable storage medium according to any one of clauses 34-40, wherein when the one or more devices search for the bitangent, the instructions also cause the one or more devices to: project data points of the outer periphery and the segmented data within the outer periphery onto the tibial plane to generate projection data; and search for the bitangent on the posterior portion of the projection data.

[0195] Clause 42. The non-transitory computer-readable storage medium of any of clauses 34-41, wherein the instructions further cause the one or more devices to identify a tibial tuberosity within the 3D model and display an indication of a positioning of a tibial tuberosity mesh within the 3D model.

[0196] Clause 43. A non-transitory computer-readable storage medium according to clause 42, wherein when the one or more devices identify the tibial tuberosity, the instructions cause the one or more devices to: project the 3D model onto a sagittal plane; segment the 3D model along a line perpendicular to the second long edge, thereby producing a tuberosity segmentation; and identify the anterior-most portion of the tuberosity segmentation as the tibial tuberosity.

[0197] Clause 44. A non-transitory computer-readable storage medium according to any one of clauses 34-43, wherein when the one or more devices segment to create the segmented data, the instructions also cause the one or more devices to select data proximal to a predetermined point distal to the lowest point.

[0198] Clause 45. A non-transitory computer-readable storage medium according to any one of clauses 34-44, wherein when the one or more devices fit the tibial plane, the instructions cause the one or more devices to fit based on the outer perimeter and the segmented data located within the outer perimeter.

[0199] Clause 46. An apparatus comprising: one or more processors configured to implement the method of clauses 21-33 and / or to implement the instructions of clauses 34-45.

Claims

1. A computer-implemented method comprising: identifying, by the device, a three-dimensional (3D) model of the femur; analyzing, by a device, the 3D model and determining a pair of points on the femur, the pair of points having a normal vector orthogonal to a vector connecting the pair of points; analyzing, by the device, the vector and determining a sagittal direction; Further analyzing the 3D model based on the determined sagittal direction by the device, and generating an intersection map of the 3D model, the intersection map including information related to the curve value of the intercondylar region; Determining, by a device, a Blumensaat line based on the intersection graph; Analyzing, by the device, the intersection graph according to the Blumensaat lines and determining a Bernard-Hertel (BH) mesh including a region surrounding the condyle when the femur is in a lateral view; as well as A digital representation of the BH grid is placed by a device as an overlay on the 3D model.

2. The computer-implemented method of claim 1 , further comprising: determining the value of the pair of points and each normal vector; Clustering the determined values; as well as A median value is determined based on the clustering, wherein the determined sagittal direction corresponds to the median value.

3. The computer-implemented method of claim 1 , further comprising: executing, by the device, the estimation model; as well as The sagittal direction is determined based on an output of the estimation model.

4. The computer-implemented method of claim 1 , further comprising: Identify the lateral and medial condyles; as well as A lateral-medial orientation of the sagittal direction is determined based on the identified lateral and medial condyles.

5. The computer-implemented method of claim 4, further comprising refining the sagittal direction based on adjusting outer boundaries of the lateral condyle and the medial condyle.

6. The computer-implemented method of claim 1, further comprising back-projecting the BH mesh to the 3D model, the BH mesh being a two-dimensional (2D) model, wherein the back-projection is based on a cutting plane defined by the sagittal direction and the Blumensaat line.

7. The computer-implemented method of claim 1 , further comprising: Back-projecting points on the intersection graph to points on the 3D model, wherein the points on the intersection graph are two-dimensional (2D); as well as Obtaining a cutting plane of the 3D model based on the sagittal direction; and The axial direction was determined by circle fitting.

8. The computer-implemented method of claim 1 , further comprising: determining a contour of the femur based on a sagittal view of the femur, the sagittal view being obtained by 2D projection along the sagittal direction; Searching for a circle that is tangent to the contour at two points; as well as A line connecting the respective centers of the circles is estimated, wherein the estimated line provides information about the axial direction.

9. The computer-implemented method of claim 1 , further comprising: analyzing the femur based on the sagittal direction and determining a sagittal view of the femur; determining an axial direction based on the sagittal view; as well as A coronal direction is determined based on a cross product of the sagittal direction and the axial direction.

10. The computer-implemented method of claim 9, further comprising determining an anatomical reference frame (ARF) of the femur based on the sagittal direction, the axial direction, and the coronal direction.

11. The computer-implemented method of claim 1 , further comprising: identifying an orthogonal projection of the femur; identifying a set of projection rays associated with the femur within the 3D model based on the orthogonal projection; A set of intersection points corresponding to the set of projection rays is determined, wherein the intersection map includes information associated with the set of intersection points.

12. The computer-implemented method of claim 11 , further comprising: analyzing the set of intersection points by edge detection analysis; as well as A curve value for the intercondylar region is determined based on the edge detection analysis, wherein the Blumensaat line is tangent to a curve associated with the curve value.

13. A non-transitory computer-readable storage medium tangibly encoded with computer-executable instructions that, when executed by one or more devices, cause the one or more devices to: Identify a three-dimensional (3D) model of the femur; analyzing the 3D model and determining a pair of points on the femur, the pair of points having a normal vector orthogonal to a vector connecting the pair of points; analyzing the vector and determining a sagittal direction; further analyzing the 3D model based on the sagittal direction and generating an intersection map of the 3D model, the intersection map including information related to curve values ​​of an intercondylar region; Determining a Blumensaat line based on the intersection graph; analyzing the intersection graph according to the Blumensaat lines and determining a Bernard-Hertel (BH) mesh including a region surrounding a condyle when the femur is in a lateral view; as well as A digital representation of the BH mesh is placed as an overlay on the 3D model.

14. The non-transitory computer-readable storage medium of claim 13, wherein the instructions further cause the one or more devices to: determining the value of the pair of points and each normal vector; clustering the values; and A median value is determined based on the clustering, wherein the sagittal direction corresponds to the median value.

15. The non-transitory computer-readable storage medium of claim 13, wherein the instructions further cause the one or more devices to: Identify the lateral and medial condyles; determining a lateral-medial orientation of the sagittal direction based on the identified lateral and medial condyles; and The sagittal direction is refined based on the adjustment of outer boundaries of the lateral condyle and the medial condyle.

16. The non-transitory computer-readable storage medium of claim 13, wherein the instructions further cause the one or more devices to: Back-projecting points on the intersection graph to points on the 3D model, wherein the points on the intersection graph are two-dimensional (2D); Obtaining a cutting plane of the 3D model based on the sagittal direction; and The axial direction was determined by circle fitting.

17. The non-transitory computer-readable storage medium of claim 13, wherein the instructions further cause the one or more devices to: determining a contour of the femur based on a sagittal view of the femur, the sagittal view being obtained by 2D projection along the sagittal direction; Searching for a circle that is tangent to the contour at two points; and A line connecting the respective centers of the circles is estimated, wherein the estimated line provides information about the axial direction.

18. The non-transitory computer-readable storage medium of claim 13, wherein the instructions further cause the one or more devices to: analyzing the femur based on the sagittal direction and determining a sagittal view of the femur; determining an axial direction based on the sagittal view; determining a coronal direction based on a cross product of the sagittal direction and the axial direction; as well as An anatomical reference frame (ARF) of the femur is determined based on the sagittal direction, the axial direction, and the coronal direction.

19. An apparatus comprising: One or more processors configured to: Identify a three-dimensional (3D) model of the femur; analyzing the 3D model and determining a pair of points on the femur, the pair of points having a normal vector orthogonal to a vector connecting the pair of points; analyzing the vector and determining a sagittal direction; further analyzing the 3D model based on the determined sagittal direction and generating an intersection map of the 3D model, the intersection map including information related to curve values ​​of the intercondylar region; Determining a Blumensaat line based on the intersection graph; analyzing the intersection graph according to the Blumensaat lines and determining a Bernard-Hertel (BH) mesh including a region surrounding a condyle when the femur is in a lateral view; as well as A digital representation of the BH mesh is placed as an overlay on the 3D model.

20. The apparatus of claim 19, wherein the one or more processors are further configured to: analyzing the femur based on the sagittal direction and determining a sagittal view of the femur; determining an axial direction based on the sagittal view; determining a coronal direction based on a cross product of the sagittal direction and the axial direction; as well as An anatomical reference frame (ARF) of the femur is determined based on the sagittal direction, the axial direction, and the coronal direction.

21. A computer-implemented method comprising: receiving, by the device, a three-dimensional model (3D model) of the proximal tibia; identifying, by the device, a first approximate tibial axis; segmenting, by a device, the 3D model using the first approximate tibial axis to create segmented data; Delineating, by a device, the outer periphery of the tibial plateau from the segmented data; fitting, by a device, a tibial plane based on the outer perimeter and at least a portion of the segmented data, a normal to the tibial plane defining a final tibial axis; Finding, by the device, a bitangent line on the rear portion based on the outer perimeter and a portion of the segmented data; A tibial mesh is formed by the apparatus, the tibial mesh comprising a rectangle having: a first long edge coaxial with the bitangent; a second long edge parallel to the first long edge and identifying an outermost edge of a front portion of the outer perimeter; a first short edge perpendicular to the first long edge and identifying an innermost portion of the outer perimeter; and a second short edge perpendicular to the first long edge and identifying an outermost portion of the outer perimeter; as well as The tibia mesh is overlaid and displayed by the device relative to the 3D model.

22. The computer-implemented method of claim 21, wherein identifying the first approximate tibial axis further comprises deforming a statistical shape model (SSM) to correspond to the 3D model thereby generating a deformed SSM, and identifying the first approximate tibial axis from the deformed SSM.

23. The computer-implemented method of claim 21 , further comprising: identifying the lowest point of the tibial plateau from the SSM; Wherein segmenting further comprises segmenting the 3D model using the first approximate tibial axis and the lowest point to create segmented data.

24. The computer-implemented method of claim 23, wherein segmenting the 3D model further comprises selecting data points from the 3D model that are located in and above a segmentation plane, the segmentation plane being perpendicular to the first approximate tibial axis and the segmentation plane being a predetermined distance distal to the lowest point of the tibial plateau.

25. The computer-implemented method of claim 24, wherein the lowest point of the tibial plateau is the most distal point of the medial plateau.

26. The computer-implemented method of claim 21, further comprising: estimating a sagittal orientation of the 3D model; projecting the 3D model in the sagittal direction to generate a sagittal projection; Finding the most posterior point of the tibial plateau from the sagittal projection; and Wherein segmenting further comprises segmenting the 3D model using the posterior-most point and the first approximate tibial axis.

27. The computer-implemented method of claim 26, wherein segmenting the 3D model further comprises selecting data points from the 3D model that are located in and above a segmentation plane, the segmentation plane being perpendicular to the first approximate tibial axis and the segmentation plane being a predetermined distance distal to a posterior-most point of the tibial plateau.

28. The computer-implemented method of claim 21, wherein delineating an outer perimeter of the tibial plateau further comprises identifying a contour region within the segmented data, and assigning the outer perimeter based on the contour region.

29. The computer-implemented method of claim 21 , wherein finding the bitangent further comprises: projecting the data of the outer perimeter and the segmented data within the outer perimeter onto the tibial plane to generate projection data; as well as The bitangent line is found at the rear of the projection data.

30. The computer-implemented method of claim 21, further comprising identifying, by a device, a tibial tuberosity within the 3D model and displaying an indication of a location of the tibial tuberosity within the 3D model.

31. The computer-implemented method of claim 30, wherein identifying the tibial tuberosity further comprises: projecting the 3D model onto the sagittal plane; segmenting the 3D model along a line perpendicular to the second long edge, thereby producing a trochanter segmentation; as well as The most anterior portion of the tuberosity segmentation is identified as the tibial tuberosity.

32. The computer-implemented method of claim 21, wherein segmenting to create the segmented data further comprises selecting data proximal to a predetermined point distal to the lowest point.

33. The computer-implemented method of claim 21, wherein fitting the tibial plane further comprises fitting based on the outer perimeter and segmented data within the outer perimeter.

34. A non-transitory computer-readable storage medium tangibly encoded with computer-executable instructions that, when executed by one or more devices, cause the one or more devices to: receiving a three-dimensional model (3D model) of the proximal tibia; identifying a first approximate tibial axis; segmenting the 3D model using the first approximate tibial axis to create segmented data; delineating the outer periphery of the tibial plateau from the segmented data; fitting a tibial plane based on the outer perimeter and at least a portion of the segmented data, a normal to the tibial plane defining a final tibial axis; finding a bitangent line on the posterior portion based on the outer perimeter and a portion of the segmented data; forming a tibial mesh, the tibial mesh comprising a rectangle having: a first long edge coaxial with the bitangent; a second long edge parallel to the first long edge and identifying an outermost edge of a front portion of the outer perimeter; a first short edge perpendicular to the first long edge and identifying an innermost portion of the outer perimeter; and a second short edge perpendicular to the first long edge and identifying an outermost portion of the outer perimeter; as well as The tibia mesh is overlaid and displayed relative to the 3D model.

35. A non-transitory computer-readable storage medium according to claim 34, wherein when the one or more devices identify the first approximate tibial axis and the lowest point of the tibial plateau, the instructions also cause the one or more devices to deform a statistical shape model (SSM) to correspond to the 3D model to generate a deformed SSM, and identify the first approximate tibial axis from the deformed SSM.

36. A non-transitory computer-readable storage medium according to claim 34, wherein the instructions further cause the one or more devices to identify the lowest point of the tibial plateau from the SSM, and wherein when the one or more devices segment the 3D model, the instructions further cause the one or more devices to segment the 3D model using the first approximate tibial axis and the lowest point of the tibial plateau.

37. A non-transitory computer-readable storage medium according to claim 36, wherein when the one or more devices segment the 3D model, the instructions cause the one or more devices to select data points from the 3D model that are located in and above a segmentation plane, the segmentation plane being perpendicular to the first approximate tibial axis, and the segmentation plane being a predetermined distance distal to the lowest point of the tibial plateau.

38. The non-transitory computer-readable storage medium of claim 36, wherein the instructions further cause the one or more devices to: estimating a sagittal orientation of the 3D model; projecting the 3D model in the sagittal direction to generate a sagittal projection; as well as Find the most posterior point of the tibial plateau from the sagittal projection; When the one or more devices segment the 3D model, the instructions cause the one or more devices to segment the 3D model using the posterior-most point and the first approximate tibial axis.

39. A computer-implemented method according to claim 34, wherein when the one or more devices segment the 3D model, the instructions also cause the one or more devices to select data points from the 3D model that are located in and above a segmentation plane, wherein the segmentation plane is perpendicular to the first approximate tibial axis and the segmentation plane is a predetermined distance distal to the most posterior point of the tibial plateau.

40. The non-transitory computer-readable storage medium of claim 34, wherein when the one or more devices delineate the outer perimeter of the tibial plateau, the instructions further cause the one or more devices to identify a contour region within the segmented data and assign the outer perimeter based on the contour region.

41. The non-transitory computer-readable storage medium of claim 34, wherein when the one or more devices search for the bitangent, the instructions further cause the one or more devices to: projecting the outer perimeter and data points of the segmented data within the outer perimeter onto the tibial plane to generate projection data; and The bitangent line is found on the rear portion of the projection data.

42. The non-transitory computer-readable storage medium of claim 34, wherein the instructions further cause the one or more devices to identify a tibial tuberosity within the 3D model and display an indication of a positioning of a tibial tuberosity mesh within the 3D model.

43. The non-transitory computer-readable storage medium of claim 42, wherein when the one or more devices identify the tibial tuberosity, the instructions cause the one or more devices to: projecting the 3D model onto the sagittal plane; segmenting the 3D model along a line perpendicular to the second long edge, thereby producing a trochanter segmentation; as well as The most anterior portion of the tuberosity segmentation is identified as the tibial tuberosity.

44. The non-transitory computer-readable storage medium of claim 34, wherein when the one or more devices segment to create the segmented data, the instructions further cause the one or more devices to select data proximal to a predetermined point distal to the lowest point.

45. The non-transitory computer-readable storage medium of claim 34, wherein when the one or more devices fit the tibial plane, the instructions cause the one or more devices to fit based on the outer perimeter and segmented data located within the outer perimeter.

46. ​​An apparatus comprising: One or more processors configured to: receiving a three-dimensional model (3D model) of the proximal tibia; identifying a first approximate tibial axis; segmenting the 3D model using the first approximate tibial axis to create segmented data; delineating the outer periphery of the tibial plateau from the segmented data; fitting a tibial plane based on the outer perimeter and at least a portion of the segmented data, a normal to the tibial plane defining a final tibial axis; finding a bitangent line on the posterior portion based on the outer perimeter and a portion of the segmented data; forming a tibial mesh, the tibial mesh comprising a rectangle having: a first long edge coaxial with the bitangent; a second long edge parallel to the first long edge and identifying an outermost edge of a front portion of the outer perimeter; a first short edge perpendicular to the first long edge and identifying an innermost portion of the outer perimeter; and a second short edge perpendicular to the first long edge and identifying an outermost portion of the outer perimeter; as well as The tibia mesh is overlaid and displayed relative to the 3D model.

Images