Orthopedic surgery pre-operative planning system

Abstract

1. A method for determining one or more of the selection, positioning, or placement of a surgical implant, the method comprising: predicting the function of impaired anatomical structures in their intact state; predicting post-operative function of one or more implant structures; Selecting one or more of an implant, an implant position, or an implant location to improve predicted post-operative function.

Description

Technical Field

[0001] The present disclosure relates to the general field of planning surgical procedures (particularly orthopedic surgery). Also disclosed are methods, systems, and devices for preoperatively selecting and positioning implant components for use in surgical procedures.

[0002] The present disclosure also relates to methods, systems, and devices for preoperatively planning total joint replacement surgery.

[0003] The present disclosure also relates to methods, systems, and devices for measuring, predicting, and comparing a patient's joint function before and after total joint replacement surgery.

Background Art

[0004] In orthopedics, damaged or worn joints can be replaced with prostheses or implants (e.g., particularly hip implants, knee implants, or shoulder implants). The primary goal of total joint replacement surgery is to restore the patient's joint function. However, considering the patient's bone shape, muscle geometry and function, and changes in movement styles for specific functional tasks, it is difficult to predict the functional impact of implant selection and positioning.

[0005] To perform a functional task (e.g., lifting an object using one hand), a joint (e.g., the shoulder) needs to rotate over an angular range. A measure of the ability to perform the task is the magnitude of that angular range. For example, a healthy person may be able to bend and rotate their shoulder 180 degrees, while a patient who requires shoulder arthroplasty may only be able to bend it 90 degrees somehow. The goal of shoulder arthroplasty is to restore an angular range up to 180 degrees or the largest possible angle by implant selection and placement.

[0006] This example is complicated by variations in how each individual performs tasks. For example, due to differences in muscle strength and innate abilities, two individuals may perform lifting tasks using different ranges of elbow and shoulder rotation. It may be impractical and inconvenient for a patient to attempt to restore their function based on the behavior and abilities of another person.

[0007] In many cases, preoperative surgical planning is used to attempt to model the selection and positioning of implants. This can be combined with biomechanical models to help predict joint function. However, several issues need to be addressed. Currently, the tools used by physicians have limitations in their ability to model the patient's anatomical structure, and thus also in their ability to select and position the appropriate implant or prosthesis, which may differ from one another.

[0008] Furthermore, without knowledge of the patient's pre-operative joint function and the expected post-operative state of that function, it is difficult to grasp what should be the goals when selecting and placing implants during the pre-operative planning stage. [Overview of the project] [Problems that the invention aims to solve]

[0009] The object of the disclosure of the present invention is to provide a method, process, or system for pre-planning or scheduling orthopedic procedures (specifically, implant surgery procedures) that provides a viable or useful alternative to existing methods, processes, or systems. [Means for solving the problem]

[0010] Implants may include, but are not limited to, permanent implants (e.g., artificial joint replacement), temporary implants used during surgery (e.g., surgical cutting guides), or implants that are bioresorbed by the human body over time.

[0011] A system and method for preoperative surgical planning is disclosed for collecting, predicting, and analyzing patient function in order to optimize implant selection and placement and maximize postoperative joint function. This is done in a patient-specific manner. In one example, function is measured using a functional measure that may be joint angle range, but other quantitative values ​​that capture the patient's joint range of motion may also be used. Although joints are mentioned, it is recognized that this disclosure is more generally applicable to anatomical structures including bones, ligaments, tendons, and joints.

[0012] In one embodiment, the present disclosure generally provides a method for determining one or more of the selection, positioning, or placement of a surgical implant, and the method is Steps to predict the function of an anatomical structure with impairment while it is not damaged, Steps to predict the postoperative function of one or more implant structures, The procedure includes a step of selecting one or more of the following: an implant, an implant location, or an implant site, in order to improve the predicted postoperative function.

[0013] In one embodiment, the selection involves minimizing one or more differences between the predicted postoperative function and the predicted unimpaired function.

[0014] In one embodiment, preoperative data of the affected structure are obtained. Subject or patient data may also be obtained in such a way that it can become data for the subject population.

[0015] In one embodiment, postoperative data may be acquired to improve predictive capabilities.

[0016] In one embodiment, the method includes generating an anatomical model of a patient. Preferably, the model includes a three-dimensional model.

[0017] In one embodiment, the model is generated from medical images of one or more patients.

[0018] In one embodiment, medical images of one or more patients are processed to generate an anatomical model of the patient using a statistical shape model.

[0019] In one embodiment, the method may include machine learning methods, such as artificial neural networks or deep neural networks, for classifying and / or filtering anatomical or medical images of a patient, for performing one or more steps of the method.

[0020] In one embodiment, a statistical shape model is used to identify or generate one or more of the following: anatomical landmarks, anatomical features, or anatomical regions, one or more geometric models, or one or more morphological measurements.

[0021] Anatomical landmarks, features, or regions may be landmarks, features, or regions relevant to the surgery. Landmarks of surgical features may include implant fixation points, regions, or locations.

[0022] Anatomical landmarks, features, or regions may be relevant landmarks, features, or regions for determining the patient's function before or after surgery (e.g., range of motion before or after surgery).

[0023] In another aspect, the disclosure generally provides a method for determining one or more of the selection, positioning, or placement of a surgical implant, and the method is Steps include obtaining preoperative data from patients with impaired anatomical structures, A step to predict the postoperative function of the anatomical structures of one or more implants, The procedure includes a step of selecting one or more of the following: an implant, an implant location, or an implant site, to improve the predicted postoperative function of the structure.

[0024] The above method can be applied to determine one or more of the implant type, implant shape, and implant fixation points.

[0025] In certain embodiments, the method includes generating an anatomical model of the patient. Preferably, the model includes a three-dimensional model.

[0026] In certain embodiments, the model is generated from one or more medical images of the patient.

[0027] In certain embodiments, one or more medical images of the patient are processed to generate an anatomical model of the patient using a statistical shape model.

[0028] In certain embodiments, the method may include a machine learning method, such as an artificial neural network or a deep neural network, for performing one or more steps of the method, for example, classifying and / or filtering anatomical or medical images of the patient.

[0029] In certain embodiments, a statistical shape model is used to identify or generate one or more of anatomical landmarks, anatomical features, or anatomical regions, one or more geometric models, and one or more morphological measurements.

[0030] Anatomical landmark(s), anatomical feature(s), or anatomical region(s) can be landmarks, features, or regions related to surgery. Landmarks of features related to surgery can include implant fixation points, or regions or locations.

[0031] Anatomical landmark(s), anatomical feature(s), or anatomical region(s) can be related landmarks, features, or regions for determining the function of the patient before or after surgery (e.g., range of motion before or after surgery).

[0032] In another aspect, the disclosure provides a method or system for generating medical images that predict the function of anatomical structures that are not impaired or that occur after surgery.

[0033] In one embodiment, the method includes generating an anatomical model of a patient. Preferably, the model includes a three-dimensional model.

[0034] In one embodiment, the model is generated from medical images of one or more patients.

[0035] In one embodiment, medical images of one or more patients are processed to generate an anatomical model of the patient using a statistical shape model.

[0036] In one embodiment, the method may include machine learning methods, such as artificial neural networks or deep neural networks, for classifying and / or filtering anatomical or medical images of a patient, for performing one or more steps of the method.

[0037] In one embodiment, a statistical shape model is used to identify or generate one or more of the following: anatomical landmarks, anatomical features, or anatomical regions, one or more geometric models, or one or more morphological measurements.

[0038] Anatomical landmarks, features, or regions may be landmarks, features, or regions relevant to the surgery. Landmarks of surgical features may include implant fixation points, regions, or locations.

[0039] Anatomical landmarks, features, or regions may be relevant landmarks, features, or regions for determining the patient's function before or after surgery (e.g., range of motion before or after surgery).

[0040] In another embodiment, the present disclosure provides a graphical user interface for facilitating one or more of the methods described above.

[0041] In one embodiment, the interface includes a three-dimensional display of the patient's anatomical structure and a presentation of the implant superimposed on the patient's anatomical structure.

[0042] In one embodiment, the 3D display is operable to provide multiple viewpoints.

[0043] In one embodiment, the interface indicates the joint orientation of the implant or patient in multiple different planes. Preferably, the planes are orthogonal to each other.

[0044] In another aspect, the disclosure provides an apparatus for carrying out the method described above.

[0045] In one embodiment, the device includes a client-server system.

[0046] In another aspect, the disclosure provides a system for carrying out the method described above.

[0047] Further details will become clear from the attached explanation.

[0048] One or more examples of methods and systems for determining one or more of the selection, positioning, or placement of surgical implants are described below with reference to the attached diagrams. [Brief explanation of the drawing]

[0049] [Figure 1A] This figure shows an apparatus including a processing environment for implementing the methods and systems disclosed herein. [Figure 1B] This figure shows an apparatus including a processing environment for implementing the methods and systems disclosed herein. [Figure 2] This is an overview diagram of the system. [Figure 3]This figure shows the generation of a machine learning / biodynamic hybrid model to predict a patient's functional scale. [Figure 4] This is a diagram of the implant and its placement subsystem. [Figure 5] This is a diagram of the image processing subsystem. [Figure 6] This flowchart shows an example or embodiment of generating and simulating a 3D model of implant fit and patient function. [Figure 7] This is a schematic diagram of a client-server system providing an example of an embodiment of the present invention. [Figure 8] This is a schematic diagram showing examples of landmarks or geometric features that can be identified to form an anatomical model. [Figure 9A] This is a schematic diagram illustrating an example of morphological landmarks for target surgical features for implant integration of bone (or similar structures) in a model. [Figure 9B] This is a schematic diagram illustrating an example of morphological landmarks for target surgical features for implant integration of bone (or similar structures) in a model. [Figure 10] This shows an example of a graphical user interface that facilitates the use of the system. [Figure 11] This shows an example of a graphical user interface that facilitates the use of the system. [Figure 12] This shows an example of a graphical user interface that facilitates the use of the system. [Figure 13] This shows an example of a graphical user interface that facilitates the use of the system. [Figure 14] This shows an example of a graphical user interface that facilitates the use of the system. [Figure 15] This shows an example of a graphical user interface that facilitates the use of the system. [Figure 16] This shows an example of a graphical user interface that facilitates the use of the system. [Figure 17]This shows an example of a graphical user interface that facilitates the use of the system. [Modes for carrying out the invention]

[0050] Here, specific examples or embodiments are described in detail with reference to the accompanying drawings. However, the present invention can be embodied in many different forms and should not be construed as being limited to the embodiments described herein. Rather, these embodiments are provided so as to make this disclosure complete and comprehensive and to fully convey the scope of the invention to those skilled in the art. The technical terms used in the detailed description of the embodiments shown in the accompanying drawings are not intended to limit the invention. In the drawings, similar figures refer to similar elements.

[0051] The following description focuses on embodiments of the present invention applicable to planning orthopedic surgery. The method involves positioning virtual implant components relative to a digital model of a patient's anatomical structure. Embodiments of the present invention are described below with respect to planning total hip replacement surgery using a hip implant comprising an acetabular cup component and a femoral stem component. However, it is recognized that the present invention may be applicable to many other orthopedic surgeries, such as joint implant surgery (e.g., knee implant surgery, ankle implant surgery, etc.), but is not limited to this application. One or more implant components may be included in the procedure. For example, positioning a virtual implant component may involve defining positional information for at least one of the following: affected femoral head, affected femoral shaft axis, unaffected femoral head, unaffected femoral shaft axis, the cup of the virtual implant, and the stem of the virtual implant. Those skilled in the art will understand that the modeling of anatomical structures as disclosed herein is not limited to (although this is used as a primary example) the modeling of bone, but may include, but is not limited to, other structures, including connective tissue, ligaments, tendons, cartilage, muscles, and vascular structures.

[0052] The tools used to perform preoperative planning as disclosed herein are performed by computer. Therefore, aspects of the disclosure of the present invention are implemented in a data processing environment.

[0053] One or more aspects of the disclosure of the present invention are intended for use in a data processing environment and are first described in broad terms with reference to Figures 1A and 1B of the drawings. With reference to Figure 1A, a data processing network environment in which one or more embodiments of the present invention may be used is shown. The data processing environment 10 may include a plurality of individual networks, such as wireless and wired networks. A plurality of wired and / or wireless devices 11 may communicate through the network 12 with a third-party information source 13, a data processing service 14, and an information management system 15 (which may include a data store 16 and a data processing system or computing device 20).

[0054] Computing device 20 is shown in detail in Figure 1B. Device 20 may be implemented as a microprocessor capable of processing data accessed from additional network-based resources such as websites or other supplemental content delivery.

[0055] Figure 1B shows one embodiment of the architecture of an exemplary computing device 20 for implementing various aspects of a data processing system or environment 10 according to aspects of the present application. The data processing system 10 may be part of an instantiation of a set of virtual machine instances. The computing device 20 may be a standalone device that functions as the data processing system 10.

[0056] The overall architecture of device 20 shown in Figure 1B includes an array of computer hardware and software components that may be used to implement aspects of the disclosure of the present invention. As shown, device 20 includes a processing unit 24, a network interface 26, a computer-readable media drive 28, and an input / output device interface 29, all of which can communicate with each other by a communication bus. The components of computing device 20 may be physical hardware components or may be implemented in a virtual environment.

[0057] The network interface 26 can provide connectivity to one or more networks or computing systems. Therefore, the processing unit 24 can receive information and instructions from other computing systems or services via the network. The processing unit 24 can also communicate by traveling back and forth to the memory 30 and further provide output information.

[0058] Memory 30 may contain computer program instructions executed by the processing unit 24 to implement one or more embodiments. Memory generally includes RAM, ROM, or other persistent or non-temporary memory. Memory may store an operating system 34 that provides computer program instructions for use by the processing unit 24 in the general management and operation of the device. Memory may further contain computer program instructions and other information for implementing aspects of the disclosure of the present invention. For example, in one embodiment, memory includes interface software 32 for receiving and processing requests from a client device 11. Memory 30 includes an information matching processing component 36 for processing user interaction and creating a graphical interface as described herein.

[0059] The aspects of this application should not be interpreted as requiring physical, hypothetical, or logical embodiments unless specifically indicated otherwise.

[0060] Referring to Figure 2, an overall schematic diagram of one embodiment of System 100 for pre-operative planning is shown. Although this embodiment is described in reference to implants, those skilled in the art will recognize that the system is applicable to other surgical procedures.

[0061] The system has a biomechanical model 104 structure at its core, and in at least some embodiments, machine learning is implemented or extended as described further below. Model 104 receives a patient anatomical model 110 and an initial surgical plan 113, along with preoperative patient motion data. This motion data includes patient motion data 111 (derived from preoperative assessment 101) and patient motion data 112 (derived from postoperative patient function assessment 106).

[0062] The patient's anatomical model is derived from preoperative images 117, which are processed in 102 to provide model 110.

[0063] The output from model 104 includes a functional scale 104a for performing the surgical plan, preoperative functional scales 104b,c, and postoperative functional scale 104d. Using the preoperative scales 104b,c, a preoperative range of motion analysis 105 can be developed, which can be used to determine the selection and placement of implants, as shown in 103. Using the postoperative scale 104d, a postoperative range of motion analysis 107 can be developed, which can be compared with the preoperative analysis data 105 for review of options by a system user, such as a physician, in 114 before determining the surgical outcome 115, which can be provided to model 104 as data to improve future models and processing, for example, by using machine learning. In some embodiments, the system can automatically process data with minimal input from a physician. This may depend on the nature or complexity of the surgical procedure. In some embodiments, for example, the surgical procedure can be planned without specific decisions that need to be made by a physician. In some embodiments, the surgical plan may be provided in a machine-readable format that allows a machine, such as a robot, to perform the surgical procedure.

[0064] In other embodiments, physicians may be able to make manual selections based on data such as preoperative or postoperative outcomes determined by Model 104.

[0065] The implant model 118 is provided, enabling the system to perform necessary modeling regarding implant placement and postoperative outcomes as part of the procedure. Thus, the implant model 118 allows model 104 to generate data for implant placement relative to the patient's anatomical structure and, upon request, enables visualization of the implant.

[0066] In some embodiments, the results can be automatically optimized. For example, implant selection and placement can be automatically optimized. This can occur, for example, through an iterative process. Thus, initial implant selection and placement data can be input into the surgical plan 113 and processed again according to the model 104, and this process can continue until it is determined that the selection and placement fall within the range of one or more threshold parameters. Threshold parameters may include, for example, a portion of the preoperative range of motion analysis data 105 and / or the postoperative range of motion analysis data 107.

[0067] In other embodiments, the physician may, for example, use data from a preoperative range of motion analysis 105 to attempt the use of a completely different form of implant in 103. Thus, in the example shown in Figure 2, the physician may have the option to approve or revise the preoperative range of motion analysis 105 in 116, and consequently, the option to provide manual adjustment, repetition, or override of the implant selection and placement 103.

[0068] The finalized surgical plan 119 can be generated as an output, and as can be seen in Figure 2, the data from the plan is fed back as input to model 104, allowing the system to iterate through all inputs until the time taken for optimization falls within one or more required thresholds or parameters, or until an output is manually selected.

[0069] The surgical plan and other data generated by the system can be visualized to provide human users (such as physicians) with images that can assist in the surgical process, and / or enable users to visualize implant placement and the potential impact of the implant's postoperative range of motion or other effects the patient may experience.

[0070] In some embodiments, Model 104 may use machine learning to assist the system's predictive capabilities. For example, postoperative functional assessment may be performed by Model 104 based on postoperative data obtained from past patients. Thus, the predicted postoperative assessment can be used as another input when deciding on the selection and placement of implants.

[0071] Broadly speaking, System 100 provides a digitally implemented surgical planning system that generally includes the following 1-7: 1. Preoperative functional assessment of patients using wireless inertial motion / measurement unit (IMU) sensors.101 2. An image processing subsystem 102 that uses a deep neural network to generate a digital model of the patient's anatomical structure. 3. The following a-d implant selection and placement subsystem 103. a. Digitally adapt the 118 implant library to the 105 output. b. Display the patient's anatomical structure and implants. c. Allows the user to adjust the selection and placement of implants. d. Display the patient's preoperative functional scale, its predicted normal functional scale, and its predicted postoperative functional scale. 4. A machine learning / biomechanical hybrid model (hybrid model) 104 that performs the following a-d. a. Considering the initial surgical plan or operational surgical plan113, predict the patient's functional scale. Considering the motion data from b.101, estimate the patient's current functional scale. Considering the behavioral data from c.101, predict the patient's normal functional scale. d. Improve predictions 4(a), (b), and (c) by learning from pre- and post-operative functional data and patient anatomical data. Range of motion analysis subsystem 105 provides feedback to 103 by comparing the outputs of 5.104 to visualize and optimize implant selection and placement. 6.4(b) and 4(c) are verified, and postoperative patient functional assessment using a wireless inertial motion unit (IMU) sensor used to improve 4.106. 7. Postoperative range of motion analysis subsystem 107 for estimating actual postoperative functional scales.

[0072] Furthermore, the embodiments of system 100 are described in more detail below.

[0073] Referring to Figure 3, several embodiments provide more detail on how patient motor scales are acquired and processed. Model 104 includes a motor model generator 201 that generates a digital functional model 204 of the required or subject's anatomical structures from a patient anatomical model 110 generated by an image processing subsystem (described further below). This model consists of anatomical structures including bones, joints, muscles, and other features, with geometric constraints adapted to the patient's anatomical structures that model the function of the patient's musculoskeletal system, but is not limited to. The model generator 201 can optionally acquire a surgical plan 113 including one or more implants as input. In this example, the generator modifies the joint shape to replace the patient's own joints with artificial joints.

[0074] The functional scale estimator 202 generates functional scales from IMU data and generates patient-specific motor models 204 to provide preoperative functional scale estimates 205.

[0075] The functional scale predictor 203 predicts the patient's functional scale 207 (in the case of artificial joint replacement surgery) when the patient's joint is normal, using a kinematic model, the patient's medical images, a functional scale estimated from 205, raw patient motion data, and a population model 206 of anatomical structure and function. This predictor combines various input data types for prediction using a combination of biomechanical models and machine learning methods.

[0076] Referring to Figure 4, in some embodiments, how the selection and placement of implants are determined is provided in more detail. The implant selection and placement subsystem 103 may include an implant fitting simulator 401 that fits each implant in the implant library 118 to a patient's anatomical model 110, taking surgical constraints into account. In step 406, the fitting simulator 401 scores and then ranks each implant based on how well a-c below are performed. a. To what extent it conforms to the patient's anatomical structure. b. The degree to which the patient's anatomical structure has been restored. c. The degree to which the patient's function has recovered.

[0077] The output of 401 is sent to a graphical user interface 402 which is displayed to the user (e.g., a physician). In some embodiments, this allows the physician to do one or more of the following: - Visualizing the patient's anatomical structure and implants. In some embodiments, this allows the physician to visualize the implant and / or the implant and anatomical structure in one or more selected directions and / or distances, cross-sections, and / or graphic and text data overlays. - Visualize one or more graphical and textual representations of the measured patient function, predicted normal function, predicted postoperative function, and any of the measured postoperative functions. - Real-time graphical feedback on changes and effects on the patient's anatomical structure and function after surgery allows for adjustments to implant selection and placement. - Approve and finalize the selection and placement of implants to prevent further modifications and to generate documentation for the surgical plan, intraoperative guidance, and postoperative review of implant procurement.

[0078] After user approval or modification, the selected implant(s) and their positioning are entered into the hybrid biomechanical model 403, and postoperative function (see above) is predicted.

[0079] Joint range of motion analysis 405 is performed in the normal function prediction unit 2067 and the postoperative function prediction unit, and the difference is calculated in 408. In at least some embodiments, the system 100 is configured to minimize the difference 408. This difference is then sent to the fitting simulator 401 to adjust the implant positioning and scoring, and to the user interface 402 to provide the user with feedback on the performance of the selected implant(s).

[0080] Referring here to Figure 5, one or more embodiments of the image processing subsystem 102 are described in more detail here. The subsystem 102 may include a deep neural network (DNN) and a set of image filters 301 that generate three-dimensional models of anatomical structures such as bones and muscles, and other related anatomical structures, from one or more medical images 117 of a patient. The images or a set of images may include two-dimensional X-ray, three-dimensional X-ray CT, three-dimensional MRI, or other modalities. The DNN is trained to associate the input image texture with output three-dimensional voxel volumes of various anatomical structures. A set of image filters, including thresholding, region expansion, Gaussian smoothing, and marching cubes, then convert the three-dimensional voxel volumes into a three-dimensional triangular mesh. In some embodiments, these raw geometric models or meshes 302 include an arbitrary ordering of triangles and do not include information about anatomical regions or anatomical landmarks.

[0081] Subsystem 102 also includes a statistical shape model (SSM) 303 that can be fitted to the raw mesh. The SSM obtains a mesh of the patient's anatomical structures with uniform triangulations, as shown in 304, by morphing each standard triangulation of the anatomical object onto the raw mesh. This allows the system to map anatomical regions and anatomical landmarks of geometric shapes, as shown in 305, and to automatically obtain morphological measurements such as length, angle, area, and volume, as shown in 306.

[0082] Here, with reference to Figure 6, the overall process flow for obtaining an anatomical model and simulating implant fit according to an embodiment of the present invention is described. The process begins at 640, where, in 641, the first step is the acquisition or uploading of anatomical medical images. A three-dimensional model of the patient's anatomical structure (e.g., the patient's anatomical structure such as bone or multiple bones) relating to the procedure is generated at 642, where then, in 643, landmarks of the morphology of target surgical features for implant integration are identified in the model in bone or similar structures. Next, a digital model of the implant can be selected automatically or manually from the implant library. The implant model may have its own target surgical features already identified, or may be adjusted or identified according to the anatomical digital model. In step 644, the fit between the anatomical model and the implant model is simulated. Assuming the fit is appropriate, in step 645, the patient's function (e.g., range of motion of the joint) is simulated using the model. If the function is appropriate, in 646, the plan can be completed. If it is inappropriate, another implant can be selected and simulated as indicated by route 647.

[0083] Here, with reference to Figure 7, an overview of an embodiment of the pre-operative planning system is shown, and reference numeral 750 is disclosed in relation to an example of an embodiment of a client-server environment. In this embodiment, an information management system 15, represented as a server 752, communicates with a client-side application 754, which may be run by a machine 11, via a network 12.

[0084] The client-side application 754 is used by a user (e.g., a physician planning an orthopedic surgery) and may open a new surgical case and upload anatomical images of the patient, as shown in block 755. In some embodiments, the anatomical images may be sourced from various different imaging modalities (e.g., X-ray, CT, or MRI). Some modalities may provide two-dimensional images (e.g., X-ray source images). Other modalities may be three-dimensional, with CT or MRI as the source (or may consist of a stack of two-dimensional images that can be represented as a three-dimensional image, for example). The client-side application provides the images to the server 742 as two-dimensional images 756 or three-dimensional images 757.

[0085] Next, an image processing application launched on the server performs a three-dimensional reconstruction of the patient's anatomical structure from images 756 and 757, as shown in block 758, and automatically generates a three-dimensional model of the patient's anatomical structure. The anatomical structure to be modeled includes the anatomical region that is the target of the procedure (e.g., the buttocks, shoulder, or knee).

[0086] The 3D model generated in block 758 is provided as a digital model in a certain format (STL, PLY, OBJ, etc., or other format) and can be easily provided back to the client-side application 754 as shown in block 760, enabling the user to easily visualize the patient's anatomical structure and manipulate the representation that appears on the client-side device, thereby enabling the user to achieve proper visualization of all points of the patient's anatomical structure related to the intended procedure.

[0087] To generate a 3D model, the application represented by block 758 may, in some embodiments, utilize additional tools such as an artificial neural network 759, which may include one or more deep neural networks.

[0088] Server 752 may also include a database 761 containing a collection of statistical shape models (SSMs) of patient anatomical structures (e.g., bones, or other tissues and structures) that can be used to generate or reconstruct three-dimensional models.

[0089] In some embodiments, a three-dimensional anatomical model is generated or reconstructed from an input two-dimensional anatomical medical image 756 by first using a deep neural network 759 to identify selected landmarks that may include geometric features such as the volume, region, contour, or discrete points of the image belonging to the anatomical object.

[0090] Examples of landmarks or geometric features can be seen by referring to Figures 8A to 8D. Figures 8A to 8D are schematic diagrams using the hip joint (specifically, the femoral head 801 located next to or inside the acetabular fossa 802) as an example. Figure 8A shows the geometric features including the volume 803 (shaded) of the femoral head. Figure 8B shows the region 804 (shaded) of the femoral head occupied by the planar section. Figure 8C shows the contour 805 (indicated by the dashed contour line) of the femoral head in planar section. Figure 8D shows the identified point 806 of the femoral head.

[0091] The next step is to reconstruct a three-dimensional model of the bone by fitting the SSM of the relevant anatomical structures to landmarks or contours.

[0092] In some embodiments, a three-dimensional anatomical model is generated or reconstructed from input two-dimensional anatomical medical images 756 by first using a deep neural network 759, and the parameters of the bone SSM are directly predicted from one or more medical images. Then, the predicted parameters can be used to generate a three-dimensional model of the bone from the SSM.

[0093] In some embodiments, a three-dimensional anatomical model is generated or reconstructed from a three-dimensional image volume (e.g., consisting of a set of two-dimensional CT or MRI images) such as input anatomical medical images 757 by using a deep neural network 759, and relevant regions of bone of interest are identified and classified from the three-dimensional image volume.

[0094] When using input two-dimensional or three-dimensional images, identified volumes, regions, contours, or points may enclose or lie on a single connection point of an object (e.g., part of a bone), multiple unconnected regions of an object (e.g., different parts of a fractured bone), or all the bones that make up a joint (e.g., the femur, tibia, and patella of the knee) or a larger structure (e.g., multiple vertebrae that make up the spine).

[0095] After generating a 3D anatomical model of the patient, the next step is to identify morphological landmarks of target surgical features for implant integration of bone (or similar structures) in Model 3. These target surgical features or target regions are mapped onto the patient's 3D model using SSM. This can be achieved by the following i-iii. i. To generate a canonical representation of a three-dimensional geometric shape (e.g., a triangular mesh of bones) that includes the average shape observed in a population (e.g., changes in bone shape across an entire human population) and a description of the deformation modes of that average shape, or ii. The canonical representation of SSM can be customized to the shape of a specific individual by morphing the average shape according to deformation modes, each weighted by different scores, or iii. SSM can be adapted to individual shapes by doing the following: - (For example, from image segmentation) describe the individual shapes as a point cloud. -By optimizing the deformation mode score, the average shape of the SSM was morphed according to that deformation mode to minimize several cost functions (e.g., the sum of the squared distances between the point cloud and the morphed shape). This generated approximations of individual shapes that showed further significant differences in some areas. -By using high-quality scaling deformation functions, the pre-morphed SSM mesh is further morphed to further minimize the cost function from the above. An example of such high-quality scaling deformation functions is a series of radial basis functions. The final morphed mesh is within a 1mm RMS range of the individual shapes.

[0096] The SSM of each bone (or other structure) includes anatomical points, anatomical regions, anatomical axes, and other geometric features of standard geometric shapes (e.g., triangular meshes), such as additional information regarding spheres, cylinders, and cones that best fit the platform. Figures 9A and 9B show examples, and a schematic diagram of femur 900 is marked with reference to the landmarks, regions, and features described below. i. Anatomical landmark 901 can be described by the marking of the mesh vertex closest to the landmark. ii. The anatomical region 902 is described by a set of markers for mesh vertices and mesh faces contained within the region. iii. Additional features are anatomical axes 903 defined by lines between two landmarks, lines fitting through three or more landmarks, lines fitting through regions or axes 904, or lines fitting through combinations of landmarks and regions. iv. Additional features include a circle whose center and radius fit three or more anatomical landmarks and / or regions. v. An additional feature is a sphere 907 whose center 905 and radius 906 fit four or more anatomical landmarks and / or regions. vi. An additional feature is a plane 908 through which points and normal vectors in the plane are calculated from two landmarks or fitted through the region. vii. An additional feature is a local Cartesian coordinate system, in which the origin and three perpendicular vectors are calculated using at least three landmarks, or a combination of landmarks, axes, and planes. viii. Other geometric features can also be considered. c. When the SSM is morphed onto the divided bone surface, anatomical landmarks and anatomical regions on the SSM mean mesh are morphed along the division surface. d. Here, the morphed mesh becomes an accurate representation of the patient's bone shape, annotated with the locations of anatomical landmarks, anatomical regions, and anatomical features. These landmarks, regions, and features provide targets and constraints for implant fitting.

[0097] After generating an anatomical digital model of the patient's anatomical structure that identifies the target landmark or target area of ​​the surgery, the next step is to select an implant from a library of implant shapes and sizes, simulate implant fit from the selected implant, or easily perform a simulation across the entire library of implant shapes and sizes. a. The simulation includes optimizing the fit between a given region of the implant geometry and a region of the patient's anatomical structure. b. Following the above steps, the implant 3D model is fitted with the morphed structure model (e.g., bone). c. Similar to the bone model, the implant model is also annotated with landmark points and landmark regions. d. Depending on the type, brand, size, or shape of the bone and implant, different areas and landmarks are used as objects or constraints in the fitting simulation. For example, i. Contact optimization area, ii. Areas where contactless operation is optimized, iii. The point that minimizes the distance, iv. The point that maximizes the distance, v. Simulate the regions, planes, spheres, or other geometric features used when fitting the object or constraints. e. For example, in total hip replacement, to fit a cementless femoral stem into the femoral canal, i. Maximize the contact area between the intermediate and outer lateral regions of the stem and the femoral canal. ii. Minimize the distance between the tip of the femoral stem and the center point of the femoral canal, which is located in the middle along the shape of the femur. iii. Minimize the angle between the angle of the femoral neck and the angle of the stem neck. iv. Other rules are added to i-iii above. f. After the simulation, the quality of the fit is quantified by calculating a score based on one or more geometric and / or functional measurements of the implant and bone (e.g., change in leg length or range of joint movement before and after implant fitting). g. Define a geometric shape (e.g., a plane, sphere, or cuboid) using landmarks and other features of the fitted implant, and then use that geometric shape to simulate the bone resection (cutting) required to deliver the implant in an operational manner.

[0098] System 100 enables the simulation of patient function with unique and transplanted anatomical structures. a. Construct an articular coordinate system between adjacent bones (e.g., femur and pelvis) using anatomical landmarks and regions of morphed bone and implant models. b. Intrinsic function relates to the relative motion of the bone model that was not transplanted. c. The transplanted function relates to the relative motion of the model, which is a combination of a fitted implant and excised bone. d. The function of the patient's anatomical structures is determined by the ability of their bone and implant models to move freely relative to each other, as regulated by their articular coordinate system. e. Free movement is defined as follows: f. Set the 3D model so that it does not touch other 3D models, or so that it is at a defined distance from other 3D models. - A three-dimensional model has the ability to move relative to another model, modulated by a biomechanical model of passive and active forces provided by muscles, tendons, ligaments, and other anatomical structures, as well as mass properties, inertial properties, and other physical properties. f. Furthermore, function can be defined by the joints that can move freely to perform the movements required for functional tasks (e.g., standing, walking, stretching, grasping, swinging the arms).

[0099] As described above, system 100 includes the user interface shown in Figures 10 to 17. As can be seen in Figure 10, the graphical user interface 1000 has a surgical status field 1001, below which a surgical status tracker 1002 is provided. The surgical status tracker allows the user to quickly recognize the surgical status, including, but not limited to, whether pre-operative planning is complete, whether surgery is being performed, and whether post-operative evaluation is complete.

[0100] The control unit 1005 includes control elements configured to allow the user to make manual adjustments at various stages of the planning process, or to allow the system to perform steps automatically. Summary information for each step is provided in fields 1006-1009, which may include graphical control elements that allow the user to navigate to a process involving several or more steps, and / or use the control unit 1005 to perform, for example, changes in implant selection or positioning. The approve or sign "off" button 1010 allows for user or supervisor approval of the plan generated by the system, or, instead, approval of selected steps in the process.

[0101] To allow for verification, a field or window 1003 is provided, which represents a display of the patient's anatomical structure fitted in a 3D model and a 3D model of the simulated implant (in this example). The display or visualization in window 1003 is user-operable, as shown, for example, in Figures 11–17. The display shows the patient's function before and after implantation, allowing the user to select an implant based on the simulation results and to adjust the implant's position and orientation. Furthermore, when adjustments are made, the quantitative changes to the implant fit are communicated to the user in real time.

[0102] Postoperative measurements, intrinsic measurements, implant-specific visualizations, and image overlay control are provided in the field or window 1004. Importantly, this window provides multi-axis visualization of articular center offset in multiple planes, as shown in 1004A and 1004B of Figure 11, with one plane showing articular offset in the anterior-posterior and posterior-posterior axes, and the other plane showing articular offset in the intermediate transverse and anterior-posterior axes. For clarity, reference numerals are omitted in Figures 12–17.

[0103] The processes and systems described herein may be performed on or include various types of hardware, such as computer systems. In some embodiments, a computer, a display, and / or an input device may each be a separate computer system, application, or process, or may be invoked as part of the same computer system, application, or process. Alternatively, one or more computers, displays, and / or input devices may be combined to be invoked as part of a single application or process, and / or each or one or more of the computers, displays, and / or input devices may be part of a computer system or may be invoked by a computer system. A computer system may include a bus or other communication mechanism for communicating information and a processor coupled to the bus for processing information. A computer system may have main memory, such as random access memory or other dynamic storage devices coupled to the bus. Instructions and temporary variables may be stored using the main memory. A computer system may also include read-only memory or other static storage devices coupled to the bus for storing static information and instructions. A computer system may also be coupled to a display, such as a CRT monitor or LCD monitor. Input devices may also be coupled to a computer system. These input devices may include a mouse, trackball, or cursor directional keys.

[0104] Each computer system may be implemented using one or more physical computers or computer systems or parts thereof. Instructions executed by a computer system may also be read from a computer-readable medium. The computer-readable medium may be a CD, DVD, optical or magnetic disk, laserdisc®, carrier wave, or any other medium readable by a computer system. In some embodiments, wiring may be used in place of or in combination with software instructions executed by a processor. Communication between modules, systems, devices, and elements may occur through direct or switching connections, and through wired or wireless networks or connections, via directly connected wires, or via any other suitable communication mechanism. Communication between modules, systems, devices, and elements may include handshakes, notifications, coordination, encapsulation, encryption, headers (such as routing headers or error detection headers), or any other suitable communication protocol or communication attribute. The communication may also involve messages related to HTTP, HTTPS, FTP, TCP, IP, ebMS OASIS / ebXML, secure sockets, VPN, encrypted or unencrypted pipes, MIME, SMTP, MIME multipart / related content types, SQL, etc.

[0105] Any suitable 3D graphics processing, including processing based on WebGL, OpenGL, Direct3D, Java3D, etc., may be used for display or rendering. Whole, partial, or modified 3D graphics packages may also be used, such packages including 3DS Max, SolidWorks, Maya, Form Z, Cybermotion 3D, Blender, or any other package. In some embodiments, various parts of the required rendering may occur on conventional or specialized graphics hardware. Rendering may also occur on a general-purpose CPU, on programmable hardware, on separate processors, across multiple processors, across multiple dedicated graphics cards, or using any other suitable combination of hardware or technology.

[0106] As will be apparent, the features and attributes of the specific embodiments disclosed above may be combined in different ways to form additional embodiments, all of which fall within the scope of the disclosure of the present invention. Generally, conditional language used herein, particularly “can,” “could,” “might,” “may,” and “e.g.,” is intended to convey that a particular embodiment includes certain features, elements, and / or states, while other embodiments do not, unless otherwise specifically stated, or as understood within the context in which they are used. Accordingly, such conditional language is not generally intended to imply that features, elements, and / or states are required in any way to one or more embodiments, or that one or more embodiments necessarily include logic for determining whether these features, elements, and / or states are included in or performed in any particular embodiment, with or without the author’s information or instructions.

[0107] Any process description, element, or block in the flowcharts described herein and / or shown in the accompanying figures should be understood to potentially represent a portion of code containing one or more executable instructions for performing a specific logical function or step in a process, such as a module, segment, or function mentioned above. Alternative embodiments are included within the scope of the embodiments described herein, and elements or functions may be removed and executed in any order other than that shown or described, depending on the function involved, as will be understood by those skilled in the art, substantially simultaneously or in reverse order.

[0108] All of the methods and processes described above can be embodied in software code modules executed by one or more general-purpose computers or general-purpose processors, such as those computer systems described above, thereby enabling complete automation. The code modules can be stored in any type of computer-readable medium or other computer storage device. Some or all of these methods can, instead, be embodied in specialized computer hardware.

[0109] It should be emphasized that many variations and modifications can be made to the embodiments described above. These elements should be understood to be within the scope of other acceptable examples. All such modifications and variations are, in this specification, included within the scope of this disclosure and intended to be protected by the following claims.

[0110] The present invention has been described above with reference to specific embodiments. However, other embodiments different from those described above are equally possible within the scope of the invention. Steps of methods different from those described above, which carry out the method by hardware or software, may be provided within the scope of the invention. Different features and steps of the invention may be combined in other combinations different from those described. The scope of the invention is limited only by the appended claims.

Claims

1. A method for operating a device for determining one or more of the following: selection, positioning, or placement of a surgical implant, wherein the method is: The device includes the steps of acquiring data on the range of motion of joints before surgery for patients with impaired anatomical structures, The apparatus includes the steps of generating an anatomical model of the patient from pre-operative images of the patient, The device includes the steps of predicting the functional scale of a patient when the joints are normal, using a kinematic model obtained from an anatomical model of the patient, medical images of the patient, patient motion data, and estimated preoperative functional scales obtained from the kinematic model, raw patient motion data, and population models of anatomical structure and function. The device creates a model of the patient's anatomical model with the surgical implant in place, and uses this model to predict the function of the anatomical structure with the surgical implant in place after surgery. The device includes the steps of determining one or more of the following: selecting the surgical implant, positioning the surgical implant, or placing the surgical implant; Includes, The device performs range of motion analysis on the function of the anatomical structure predicted in the step of predicting the function and on the functional scale of the patient predicted in the step of predicting the functional scale, and calculates one or more differences between the results of each analysis. A method comprising minimizing the one or more differences between the function of the predicted anatomical structure and the predicted functional scale of the patient.

2. The method according to claim 1, wherein preoperative range of motion data of subjects of the surgical implant or patients having the impaired anatomical structure can also be obtained so as to be part of the data constituting the population of subjects.

3. The method according to claim 1 or 2, wherein postoperative range of motion data of the patient may be obtained to improve the prediction of function in the step of predicting function.

4. The method according to any one of claims 1 to 3, wherein the anatomical model of the patient includes a three-dimensional model.

5. The method according to any one of claims 1 to 4, wherein the anatomical model of the patient is generated from medical images of one or more patients.

6. The method according to claim 5, wherein the apparatus processes medical images of one or more patients and generates an anatomical model of the patients using a statistical shape model.

7. The method according to any one of claims 1 to 6, wherein the method may include a machine learning method of an artificial neural network or a deep neural network for classifying and / or filtering a patient's medical images for performing one or more steps of the method.

8. The method according to claim 1, which is applied to determine one or more of the type of surgical implant, the shape of the surgical implant, and the fixed surface of the surgical implant.

9. The method according to any one of claims 1 to 8, comprising a machine learning method including an artificial neural network or a deep neural network for performing one or more steps of the method.

10. The method according to claim 9, wherein the apparatus includes classifying and / or filtering a patient's medical images using the machine learning method.

11. The method according to claim 6, wherein the apparatus uses the statistical shape model to identify or generate one or more of the following: anatomical landmarks, anatomical features, or anatomical regions, one or more geometric models, or one or more morphological measurements.

12. The method according to claim 11, wherein the anatomical landmark(s), anatomical feature(s), or anatomical region(s) may be a landmark, feature, or region related to surgery.

13. The method according to claim 12, wherein the landmarks of the features related to the surgery may include the fixed surface, region, or location of the surgical implant.

Images