Systems, devices, and methods for predicting total knee arthroplasty, partial knee arthroplasty, and other medical procedures

Abstract

A method for predicting the type of surgery required for a patient, the method comprising: receiving image data including at least one image of a patient's anatomical structure acquired; determining at least one parameter of the patient's anatomical structure based on the image data, the at least one parameter including at least one of a B-score and a C-score; receiving patient data about the patient; generating a prediction program for the patient based at least in part on the at least one parameter and the patient data; and outputting the prediction program for display within a graphical user interface (GUI).

Description

Cross-references to related applications

[0001] This patent application is a continuation of U.S. Provisional Application No. 63,725,156, filed November 26, 2024; U.S. Provisional Application No. 63 / 709,057, filed October 18, 2024; U.S. Provisional Application No. 63,624,932, filed January 25, 2024; and U.S. Provisional Application No. 63 / 623,553, filed January 22, 2024, and claims priority to these U.S. Provisional Applications, all of which are incorporated herein by reference in their entirety. Technical Field

[0002] This disclosure relates to systems and methods for optimizing medical procedures, and more particularly to a system and method for processing and displaying images to provide clinical decision intelligence and optimize outcomes following joint replacement procedures. Background Technology

[0003] Musculoskeletal disorders present unique challenges for healthcare practitioners. Surgeries incorporating prostheses and / or implants (such as joint replacement procedures) often require careful consideration of various factors. Improved systems and methods for performing, collecting, analyzing, or processing image acquisition data are desired. Summary of the Invention

[0004] In one aspect of this disclosure, a method for predicting the type of surgery required for a patient may include: receiving image data including at least one image of acquired patient anatomy; determining at least one parameter of the patient anatomy based on the image data, the at least one parameter including at least one of a B-score and a C-score; receiving patient data about the patient; generating a predictive procedure for the patient based at least in part on the at least one parameter and the patient data; and outputting the predictive procedure for display within a graphical user interface (GUI). In some instances, the predictive procedure is partial knee arthroplasty or total knee arthroplasty. In some instances, generating the predictive procedure includes determining the type of implant for the patient. In some instances, the predictive procedure is generated at least in part using a machine learning model trained using multiple predictive procedures generated for previous patients and corresponding multiple validated procedures received for those previous patients. In some instances, the method further includes receiving validated procedures for the patient and using the predictive procedure and the validated procedures to further train the machine learning model. In some instances, the validated procedures are received via the GUI. In some instances, the at least one parameter includes a B-score and a C-score. In some instances, the at least one parameter includes multiple C-scores. In some instances, the multiple C-scores include two or more of the following: medial tibiofemoral C-score, lateral tibiofemoral C-score, medial patellofemoral C-score, and lateral patellofemoral C-score. In some instances, the method further includes normalizing the patient data.

[0005] In another aspect of this disclosure, a system for predicting the type of surgery required for a patient includes: an imaging device operable to generate image data including at least one image of acquired patient anatomy; and at least one processor executed on at least one computing device and operable to: receive image data generated by the imaging device including at least one image of the acquired patient anatomy; determine at least one parameter of the patient anatomy based on the image data, the at least one parameter including at least one of a B-score and a C-score; receive patient data about the patient; generate a predictive procedure for the patient based at least in part on the at least one parameter and the patient data; and output the predictive procedure for display within a graphical user interface (GUI). In some instances, the predictive procedure is partial knee arthroplasty or total knee arthroplasty. In some instances, the at least one processor is further operable to generate the predictive procedure by determining the type of implant for the patient. In some instances, the at least one processor is further operable to generate the predictive procedure at least in part using a machine learning model trained using multiple predictive procedures generated for previous patients and corresponding multiple validated procedures received for those previous patients. In some instances, the at least one processor is further operable to receive a validated procedure for the patient and use the validated procedure to further train the machine learning model. In some instances, the at least one processor is further operable to receive the validated procedure via the GUI. In some instances, the at least one parameter includes a B-score and a C-score. In some instances, the at least one parameter includes multiple C-scores. In some instances, the multiple C-scores include two or more of the following: medial tibiofemoral C-score, lateral tibiofemoral C-score, medial patellofemoral C-score, and lateral patellofemoral C-score. In some instances, the at least one processor is further operable to normalize the patient data. Attached Figure Description

[0006] A more complete understanding of the subject matter of this disclosure and its various advantages can be obtained by referring to the following detailed description, with reference to the following figures:

[0007] Figure 1 It is a schematic diagram depicting an electronic data processing system with an image analysis system according to various aspects of this disclosure.

[0008] Figure 2 yes Figure 1 A schematic diagram of an electronic data processing system is provided, depicting the interaction between the preoperative measurement system, preoperative data, image analysis system, output, and output system according to various aspects of this disclosure.

[0009] Figure 3A and Figure 3B Exemplary image data and an exemplary graphical user interface (GUI) for displaying image data are shown for an image analysis system according to various aspects of this disclosure.

[0010] Figures 4A to 4J Exemplary GUIs or user interfaces illustrating patient anatomy, implants, and associated determined parameters or analyses are shown in accordance with various aspects of this disclosure.

[0011] Figure 5 An exemplary GUI is shown illustrating the width of the joint space between two bones depicting a patient's anatomy according to various aspects of this disclosure.

[0012] Figures 6A to 6C An exemplary GUI is shown that depicts a prediction of cartilage loss based on joint space width, according to various aspects of this disclosure.

[0013] Figure 7 An exemplary GUI is shown, illustrating osteophytes on acquired images and / or representative models depicting patient anatomy according to various aspects of this disclosure.

[0014] Figure 8 An exemplary GUI is shown for depicting and segmenting osteophytes in relation to various aspects of this disclosure.

[0015] Figure 9 An exemplary GUI depicting osteophytes in different views is shown according to various aspects of this disclosure.

[0016] Figure 10 This is a flowchart depicting an exemplary method for determining osteophyte volume according to various aspects of this disclosure.

[0017] Figures 11A to 11E An exemplary GUI is shown that depicts anatomical divisions of one or more bones in a patient's anatomy according to various aspects of this disclosure.

[0018] Figure 12 This is a flowchart depicting an exemplary method for determining the volume of a osteophyte in a partition according to various aspects of this disclosure.

[0019] Figure 13 An exemplary GUI is shown that depicts osteophytes according to partitions based on various aspects of this disclosure.

[0020] Figure 14 An exemplary GUI is shown that depicts skeletal shape or B-score progression according to various aspects of this disclosure.

[0021] Figure 15 An exemplary GUI is shown, illustrating the depiction of skeletal shape based on B-scores and related predicted outcomes based on B-scores, according to various aspects of this disclosure.

[0022] Figures 16A to 16C An exemplary GUI is shown, depicting skeletal shape progression based on B-scores and related predictive outcomes based on B-scores, according to various aspects of this disclosure.

[0023] Figure 17 This is a flowchart depicting an exemplary method for determining the tissue-to-bone ratio according to various aspects of this disclosure.

[0024] Figure 18 An exemplary GUI is shown that relates to the segmentation and / or thresholding process for determining the tissue-to-bone ratio, according to various aspects of this disclosure.

[0025] Figure 19 An exemplary GUI is shown that visually depicts the tissue-skeletal ratio according to various aspects of this disclosure.

[0026] Figure 20 An exemplary GUI configured to allow the opening and closing of osteophytes is shown according to various aspects of this disclosure.

[0027] Figure 21 An exemplary GUI is shown that is configured to allow turning a portion of the skeleton on and off.

[0028] Figure 22 An exemplary GUI is shown that is configured to allow changes in the opacity of the osteophyte.

[0029] Figure 23 An exemplary GUI is shown that is configured to allow osteophytes and / or various bones to be turned on and off.

[0030] Figure 24 An exemplary GUI configured to demonstrate simulated motion of a patient's anatomical structures according to various aspects of this disclosure is shown.

[0031] Figure 25 An exemplary GUI is shown, illustrating a model of a planned implant overlaid on acquired images and a representative model, depicting various aspects of this disclosure.

[0032] Figure 26 An exemplary GUI depicting a bone resection plane according to various aspects of this disclosure is shown.

[0033] Figure 27 An exemplary GUI depicting a virtual skeletal model according to various aspects of this disclosure is shown.

[0034] Figure 28A and Figure 28B An exemplary GUI depicting a planned osteotomy on a virtual skeletal model is shown, according to various aspects of this disclosure.

[0035] Figure 29A yes Figure 1 A schematic diagram of an electronic data processing system is provided, depicting the interaction between the intraoperative measurement system, intraoperative data, image analysis system, output, and output system according to various aspects of this disclosure.

[0036] Figure 29B This is an exemplary method for generating a GUI based on acquired images, according to various aspects of this disclosure.

[0037] Figure 30 Exemplary methods for making decisions between surgical and non-surgical treatments are shown according to various aspects of this disclosure.

[0038] Figure 31 Exemplary methods for making treatment or surgical decisions based on various aspects of this disclosure are shown.

[0039] Figure 32 Exemplary methods for making treatment or surgical decisions based on various aspects of this disclosure are shown.

[0040] Figure 33 Exemplary methods for making treatment or surgical decisions based on various aspects of this disclosure are shown.

[0041] Figure 34 An exemplary GUI depicting a planned osteotomy on a virtual skeleton model with implants, according to various aspects of this disclosure, is shown.

[0042] Figure 35 An exemplary GUI is shown that depicts a prediction of cartilage loss based on joint space width, according to various aspects of this disclosure.

[0043] Figure 36 An exemplary GUI depicting bone volume density according to various aspects of this disclosure is shown.

[0044] Figure 37 A prediction algorithm for a decision tree model, implemented according to various aspects of this disclosure, is described.

[0045] Figure 38 An exemplary GUI of a depiction prediction program according to various aspects of this disclosure is shown.

[0046] Figure 39 This is an exemplary method for generating predictive procedures for patients according to various aspects of this disclosure. Detailed Implementation

[0047] Reference will now be made in detail to the various embodiments of this disclosure illustrated in the accompanying drawings. Wherever possible, the same or similar reference numerals will be used in all the drawings to denote the same or similar features. It should be noted that these drawings are simplified forms and not drawn to exact scale. Furthermore, as used herein, the term "a / a" means "at least one / a." This term includes the words expressly mentioned above, their derivatives, and words with similar meanings. Although at least two variations have been described herein, other variations may include combinations of the aspects described herein in any suitable manner, i.e., combinations having all or some of the described aspects.

[0048] As used herein, the terms “implant trial” and “trial” are to be used interchangeably, and therefore, unless otherwise stated, the explicit use of one term includes the other. In this disclosure, “user” is synonymous with “practitioner” and can be anyone who performs the described action (e.g., surgeon, technician, nurse, etc.).

[0049] An implant can be a device that is at least partially implanted in and / or placed within a patient's body. For example, an implant can be a sensor, artificial bone, or other medical device that is coupled to, implanted in, or at least partially implanted in bone, skin, tissue, organ, etc. A prosthesis or prosthesis can be a device configured to assist or replace a limb, bone, skin, tissue, etc., or a portion thereof. Many prostheses are implants, such as tibial prosthesis components. Some prostheses may be exposed outside the body and / or may be partially implanted, such as artificial forearms or legs. Some prostheses may not be considered implants and / or may otherwise be entirely located outside the body, such as knee braces. The systems and methods disclosed herein can be used in conjunction with implants, prostheses as implants, and prostheses that may not be strictly considered "implants." Therefore, the terms "implant" and "prosthesis" are to be used interchangeably, and thus, unless otherwise stated, explicit use of any term includes the other. Although the term "implant" is used in this disclosure, it should be included in prostheses that are not necessarily "implants" in the strict sense.

[0050] In describing preferred embodiments of this disclosure, reference will be made to the nomenclature used to describe the human body. It should be noted that this nomenclature is used for convenience only and is not intended to limit the scope of the invention. For example, as used herein, the term "proximal" means toward the human body and / or away from the operator, and the term "distal" means away from the human body and / or toward the operator. As used herein, the terms "comprises" or any other variation thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements may include not only those elements but also other elements not expressly listed or inherent to such a system, process, method, article, or apparatus. The term "exemplary" is used in the sense of "example" rather than "ideal." Further, relative terms (e.g., "about," "substantially," "approximately," etc.) are used to indicate that the numerical values ​​or ranges may vary by ±10%.

[0051] Figure 1 An electronic data processing system 1 is demonstrated for collecting, storing, processing, and outputting data during a patient's treatment process.

[0052] refer to Figure 1 The electronic data processing system 1 may include a diagnostic imaging device 110 (e.g., a computed tomography or CT scanner), an image analysis system 10, and an electronic display 210. The diagnostic imaging device 110 may first be used to image a current patient who is scheduled to undergo a procedure (e.g., surgery). The image analysis system 10 may analyze the images and / or information collected during imaging (which may be transmitted from or stored in the device 110) to determine various outputs 2000. Figure 2 The image analysis system 10 generates a graphical user interface (GUI) 250 for display on the monitor 210. The image analysis system 10 can further determine recommended procedures (e.g., predictive procedures), procedure logistics (e.g., procedure scheduling), and / or predicted outcomes (e.g., risk of complications during the procedure or risk of postoperative infection) based on the determined output 2000. As the treatment process continues, the image analysis system 10 can also update its predictions and / or make future predictions for future patients using actual outcomes and / or results 12. The image analysis system 10 can be implemented as one or more computer systems or cloud-based electronic processing systems. Details of the image analysis system 10 will be referenced in [reference needed]. Figure 2 discuss.

[0053] refer to Figure 2The electronic data processing system 1 may include one or more preoperative measurement systems 100 that collect and / or output (via arrow 102) preoperative data 1000 about the current patient and / or previous patients (e.g., similar previous patients). The image analysis system 10 may receive (via arrow 104) and analyze the preoperative data 1000 to generate one or more outputs or determinations 2000, which may be output (via arrow 106) to one or more output systems 200. Preoperative measurement system 100

[0054] The preoperative measurement system 100 may include an imaging device 110, an electronic device 120 for storing electronic medical records (EMR), a patient, practitioner, and / or user interface or application 130 (e.g., on a tablet, computer, or other mobile device), and a robotic and / or automated data system or platform 140 (e.g., MAKO robotic system or platform, MakoSuite, etc.) that may include a robotic device 142. The electronic data processing system 1 may collect current image data 1010 via the imaging device 110 and supplementary or additional information (e.g., patient data and medical history 1020, planned procedure data 1030, surgeon and / or staff data 1040, and / or previous procedure data 1050) via the EMR 120, interface 130, sensors and / or electronic medical devices, and / or robotic platform 140. Each device in the preoperative measurement system 100 (e.g., imaging device 110, EMR 120, user interface or application 130, sensors and / or electronic medical devices, and robotic platform 140) may include one or more communication modules (e.g., WiFi module, Bluetooth module, etc.) configured to transmit preoperative data 1000 to each other, to the image analysis system 10, and / or to one or more output systems 200.

[0055] Imaging device 110 can be configured to collect or acquire one or more images, videos, or scans of a patient's internal anatomical structures (such as bones, ligaments, soft tissues, brain tissue, etc.) to generate image data 1010, which will be described in more detail later. Imaging device 110 may include a computed tomography (CT) scanner (e.g., a supine CT scanner). In addition to a CT scanner, imaging device 110 may also include a magnetic resonance imaging (MRI) machine, an X-ray machine, a radiography system, an ultrasound system, a thermal imaging system, a tactile imaging system, an elastography system, a nuclear medicine functional imaging system, a positron emission tomography (PET) system, a single-photon emission computed tomography (SPECT) system, a camera, etc. The collected images, videos, or scans can be automatically or manually transmitted to image analysis system 10. In some examples, a user can select a specific image to be transmitted to image analysis system 10 from multiple images captured using imaging device 110.

[0056] The electronic data processing system 1 can use data previously collected from the EMR 120, which may include patient data and medical history 1020 in the form of past practitioner assessments, medical records, past patient report data, past imaging procedures, treatments, etc. For example, the EMR 120 may contain data on demographics, medical history, biometrics, past procedures, general observations about the patient (e.g., mental health), lifestyle information, data from physical therapy, etc. Patient data and medical history 1020 will be described in further detail below.

[0057] The electronic data processing system 1 can also collect on-site or current (e.g., real-time) patient data via patients, practitioners, and / or user interfaces or applications 130. These user interfaces 130 can be implemented on mobile applications and / or patient management websites or interfaces (such as OrthologIQ®). User interfaces 130 can present questionnaires, surveys, or other prompts for practitioners or patients to input assessments (e.g., in pre-operative rehabilitation procedures), observed psychosocial information and / or surgical preparation, comments, etc., as supplementary patient data 1020. Patients can also input psychosocial information (such as perceived or assessed pain, stress levels, anxiety levels, sensations, and other patient-reported outcome measures (PROMS)) into these user interfaces 130. Patients and / or practitioners can report lifestyle information via user interfaces 130. User interfaces 130 can also collect clinical data, such as planning procedure data 1030 and planning surgeon and / or staff data 1040, which will be described in more detail later. These user interfaces 130 can be executed on and / or combined with other devices disclosed herein (e.g., combined with robotic platform 140).

[0058] Electronic data processing system 1 can collect prior procedure data 1050 and / or other real-time data or observations (e.g., observed patient data 1020) from previous patients via robotic platform 140. Robotic platform 140 may include one or more robotic devices (e.g., surgical robot 142) used in prior procedures on different patients, computers, databases, etc. Surgical robot 142 may have been assisted in previous procedures via automated movement, surgeon-assisted movement, and / or sensing, and may be implemented as or include one or more automated or robotic surgical tools, robotic surgical or computer numerical control (CNC) robots, surgical tactile robots, surgical remote-controlled robots, surgical handheld robots, or any other surgical robots. Surgical robot 142 will refer to... Figure 27 To provide a more detailed description.

[0059] Although the preoperative measurement system 100 is described in conjunction with the imaging device 110, EMR 120, user interface 130, and robotic platform 140, other devices may be used preoperatively to collect preoperative data 1000. For example, mobile devices (such as mobile phones and / or smartwatches) may include various sensors (e.g., gyroscopes, accelerometers, temperature sensors, optical or light sensors, magnetometers, compasses, global positioning systems (GPS), etc.) to collect patient data 1020, such as location data, sleep patterns, movement data, heart rate data, lifestyle data, activity data, etc. As another example, wearable sensors with various sensors (e.g., cameras, optical sensors, barometers, GPS, accelerometers, temperature sensors, pressure sensors, magnetometers or compasses, MEM devices, inclinometers, acoustic ranging devices, etc.) such as heart rate monitors, motion sensors, and external cameras can be used during physical therapy or pre-rehabilitation programs to collect information about a patient's kinematics, alignment, movement, physical fitness, heart rate, electrocardiogram data, respiratory rate, body temperature, oxygenation, sleep patterns, activity frequency and intensity, sweating, perspiration, air circulation, stress, step pressure or push-off force, balance, heel strike, gait, fall risk, weakness, and overall function. Other types of systems or devices that can be used in the preoperative measurement system 100 may include electromyography or EMG systems or devices, motion capture (mocap) systems, sensors using machine vision (MV) technology, virtual reality (VR) or augmented reality (AR) systems, etc. Preoperative data 1000

[0060] Preoperative data (1000) can be data collected, received, and / or stored before the commencement of a medical treatment plan or procedure. For example... Figure 2As indicated by the arrows, preoperative measurement system 100 can collect preoperative data 1000 from memory system 20 (e.g., cloud storage system) of image analysis system 10 and from output system 200 (e.g., from previous procedures) for use in one or more continuous feedback loops. Some preoperative data 1000 may be directly sensed via one or more devices (e.g., wearable motion sensors or mobile devices) or may be manually entered by medical professionals, patients, or other parties. Other preoperative data 1000 may be determined based on directly sensed information, input information, and / or stored information from previous medical procedures (e.g., by image analysis system 10).

[0061] As previously described, preoperative data 1000 may include imaging data 1010, patient data and / or medical history 1020, information about the planned procedure 1030, surgeon data 1040, and previous procedure data 1050.

[0062] Image data 1010 may include one or more images (e.g., raw images), videos, or scans of patient anatomy collected and / or acquired by imaging device 110. Image analysis system 10 may receive and analyze one or more of these images to determine further image data 1010, which may be used as further input preoperative data 1000. In some examples, imaging device 110 may analyze and / or process one or more images and send any analyzed and / or processed image data to image analysis system 10 for further analysis.

[0063] One or more images of the image data 1010 may be displayed or indicated, and the image analysis system 10 may be configured to identify and / or recognize the following in the images: location or alignment of bone, cartilage, or soft tissue; composition or density; fracture or tear; bony landmarks (e.g., condylar surface, bone or epiphysis, neck or metaphysis, body or diaphysis, articular surface, epicondyle, lateral epicondyle, medial epicondyle, process, protuberance, tubercle and trochanter, tibial tuberosity, trochanter, spine, linear or linear, facet, ridge and crest, foramen and fissure, canal, fossa and pit, notch and groove, and sinus); geometry (e.g., diameter, slope, angle) and / or other anatomical geometry data, such as deformity or flaring (e.g., coronal plane deformity, sagittal plane deformity, flaring of the lateral metaphysis of the femur or flaring of the medial metaphysis of the femur). This geometry is not limited to an overall geometry and can include relative dimensions (e.g., the length or thickness of the tibia or femur).

[0064] One or more images of the image data 1010 may indicate (and / or the image analysis system 10 may determine based on one or more received images) morphological and / or anthropometric data (e.g., physical dimensions of internal organs, bones, etc.), fractures, slopes (e.g., anterior-posterior (AP) slope or medial-lateral (ML) slope) or angular data, tibial slope, posterior tibial slope or PTS, bone mass and / or density, or other measures of bone health (e.g., bone mineral or bone marrow density, bone flexibility or stiffness, or bone impact strength), etc. Bone density may be determined separately using the image analysis system 10 (as described further in detail below), and / or may be collected or supplemented using, for example, indentation testing or microindentation tools. The image data 1010 may not be limited to strictly skeletal data and may include other internal imaging data, such as data on cartilage, soft tissue, or ligaments.

[0065] Image data 1010 can indicate or be used to determine (e.g., via image analysis system 10) osteophyte size, volume, or location; bone loss; joint space; B-score; bone mass / density; skin-to-bone ratio; bone loss; hardware detection; anteroposterior (AP) and medial-lateral (ML) size and / or joint angle of the distal femur. The analysis and / or calculations that can be obtained from images or scans will be described in further detail below when describing image analysis system 10 and GUI 250.

[0066] Patient data and / or medical history 1020 may include information about the following: current patient identity (e.g., name or date of birth), demographics (e.g., patient age, sex, height, weight, nationality, body mass index (BMI), etc.), lifestyle (e.g., smoking habits, exercise habits, drinking habits, dietary habits, fitness level, activity level, frequency of incline activities such as climbing stairs, frequency of sit-to-stand or flexion movements (e.g., entering and exiting vehicles), daily steps, completed activities of daily living or ADLs, etc.), and medical history (e.g., history of allergies, disease progression, history of addiction, history of previous medication use, history of previous infections, frailty). Comorbidities, history of previous surgeries or treatments, history of injury, history of pregnancy, use of orthotics, braces, prostheses or other medical devices, etc.), assessment and / or evaluation (e.g., laboratory tests and / or blood tests, American Society of Anesthesiologists or ASA score and / or surgical or anesthesia tolerance), electromyography data (muscle responses or electrical activity in response to nerve stimulation), psychosocial information (e.g., perceived pain, stress level, anxiety level, mental health status), PROMS (e.g., Knee Disability and Osteoarthritis Outcome Score or KOOS, Hip Disability and Osteoarthritis Outcome Score or HOOS, Visual Analogue Scale for Pain or VAS, PROMIS). The list may include Global 10 or PROMIS-10, EQ-5D, comprehensive mental health score, satisfaction or expectation information, etc., past biometric data (e.g., heart rate or heart rate variability, electrocardiogram data, respiratory rate, body temperature (e.g., internal or skin temperature), fingerprints, DNA, etc.), past kinematic or alignment data, past imaging data, data from pre-rehabilitation programs or physical therapy (e.g., average weight-bearing time), etc. Medical history 1020 may include information on previous clinical or hospital visits, including type of visit, date of admission, hospital-reported comorbidity data (e.g., Elixirhauser and / or Charlson scores or selected comorbidities (e.g., ICD-10 POA)), previous anesthesia and / or reactions, etc. However, this list is not exhaustive, and preoperative data 1000 may include any other patient-specific information, clinical information, and / or surgeon or practitioner-specific information (e.g., level of experience).

[0067] Patient data 1020 may originate from EMR 120, user interface 130, memory system 20, and / or robotic platform 140, but the collection of patient data 1020 is not limited by the aspects disclosed herein. For example, other types of patient data 1020 or additional data may include data on: activity level; kinematics; muscle function or ability; range of motion data; strength and / or force measurements, push-off force, force, or acceleration; force, force, or acceleration at the toes during walking; angular range or axis of joint movement, or range of motion of the joint; flexion or extension data, including step count data (e.g., measured by a pedometer), gait data, or assessment; fall risk data; balance data; joint stiffness or laxity data; postural sway data; test data conducted in clinic or remotely; and so on.

[0068] Information regarding Plan 1030 may include logistical information and substantive information about the procedure. Logistical information regarding Plan 1030 may include: information about the planned location of the procedure, such as a hospital, outpatient surgical center (ASC), or operating room; the type of procedure or surgery to be performed (e.g., total or partial knee arthroplasty or replacement, total or partial hip arthroplasty or replacement, spinal surgery, patellar resurfacing, etc.); scheduling or appointment information, such as the date or time of the procedure or surgery, the planning or setup time, registration time, and / or bone preparation time; the surgeon's medical or infection status; the name of the attending surgeon or physician scheduled to perform the procedure; the equipment or tools required for the procedure; the medications or other substances required for the procedure (e.g., type of anesthesia); the type of insurance or billing information; consent and waiver information; and so on. Substantive information in the planning procedure 1030 may include the surgeon's surgical or other procedural or treatment plan, including planned steps or instructions regarding the incision, the patient's body side (e.g., left or right) and / or lateral information, osteotomy or resection depth, implant design, type and / or size, implant alignment, fixation or tooling information (e.g., implants, rods, plates, screws, wires, nails, bearings used), cemented versus uncemented techniques or implants, final or desired alignment, pose or orientation information (e.g., capture gap values ​​for flexion or extension, gap space or width between two or more bones, joint alignment), planned time, gap balancing time, use of extended tactile boundaries, etc. This initial planning procedure 1030 information may be prepared or entered manually by the surgeon, and / or prepared or determined in advance using one or more algorithms.

[0069] Surgeon data 1040 may include information about the surgeon or other staff members involved in the planned procedure 1030. Surgeon data 1040 may include identity (e.g., name), experience level, physical fitness level, height and / or weight, etc. Surgeon data 1040 may include the number of surgeries scheduled on a specific date, the number of complex surgeries scheduled on the day of the planned procedure, average surgery time, etc.

[0070] Previous procedure data 1050 may include information about previous procedures performed on the same patient or a previous patient. This information may include the same type of information as in the planned procedure data 1030 (e.g., instructions or steps of the procedure, osteotomy, implant design, implant alignment, etc.) and outcome and / or result information, which may include both immediate and long-term results, postoperative complications, length of hospital stay, revision surgery data, rehabilitation data, patient movement and / or mobility data, etc. Previous procedure data 1050 may include information about previous procedures performed on previous patients who share at least one of the same or similar characteristics (e.g., demographics, biometrics, disease state, etc.) with the current patient. Previous procedure data 1050 may additionally or alternatively include predictive procedures generated for previous patients and associated confirmed surgeries received by previous patients. Previous procedure data 1050 may include multiple predictive surgeries generated for corresponding multiple previous patients and corresponding multiple confirmed surgeries received by such multiple previous patients. Predictive procedures may be generated by the image analysis system 10, as described in further detail below.

[0071] Preoperative data 1000 may include any additional or supplementary information stored in memory system 20, and may also include known data and / or data from third parties, such as data from the Knee Society Clinical Scoring System (KSS) or data from the Western University and McMaster University Osteoarthritis Index (WOMAC). Image Analysis System 10

[0072] Image analysis system 10 may be an artificial intelligence (AI) and / or machine learning system, which is "trained" or can learn and refine preoperative data 1000, output 2000, and actual results 12. Figure 1The pattern between the two is determined. The image analysis system 10 can be implemented using one or more computing platforms, such as platforms including one or more computer systems and / or electronic cloud processing systems. Examples of one or more computing platforms may include, but are not limited to, smartphones, wearable devices, tablets, laptops, desktop computers, Internet of Things (IoT) devices, remote servers / cloud-based computing devices, or other mobile or fixed devices. The image analysis system 10 may also include one or more hosts or servers connected to a networked environment via wireless or wired connections. The remote platform may be implemented in or used as a base station (which may also be referred to as a Node B or Evolved Node B (eNB)). The remote platform may also include web servers, mail servers, application servers, etc.

[0073] The image analysis system 10 may include one or more communication modules (e.g., WiFi or Bluetooth modules) configured to communicate with the preoperative measurement system 100, the output system 200, and / or other third-party devices. For example, such a communication module may include an Ethernet card and / or port for sending and receiving data via an Ethernet-based communication link or network, or a Wi-Fi transceiver for communicating via a wireless communication network. Such a communication module may include wired or wireless interfaces (e.g., jacks, antennas, transmitters, receivers, transceivers, terminal blocks, etc.) for data communication with external sources via direct connections or network connections (e.g., internet connections, LAN, WAN, or WLAN connections, LTE, 4G, 5G, Bluetooth, near field communication (NFC), radio frequency identification (RFID), ultra-wideband (UWB), etc.). Such a communication module may include a radio interface including filters, converters (e.g., digital-to-analog converters, etc.), mappers, fast Fourier transform (FFT) modules, etc., to generate symbols for transmission via one or more downlinks and to receive symbols (e.g., via uplinks).

[0074] The image analysis system 10 may further include a memory system 20 and processing circuitry 40. The memory system 20 may have one or more memories or storage devices configured to store or maintain preoperative data 1000, outputs 2000, and stored data 30 from previous patients and / or previous procedures. Preoperative data 1000 and outputs 2000 from the current procedure may also serve as stored data 50. Although some information is described herein as preoperative data 1000 or outputs 2000, due to the continuous feedback loop of data (which may be anchored by the memory system 20), the preoperative data 1000 described herein may alternatively be determination or output 2000, and the determined output 2000 described herein may also be used as input to the image analysis system 10. For example, some preoperative data 1000 may be directly sensed or otherwise received, while other preoperative data 1000 may be determined, processed, or output based on other preoperative data 1000. Although the memory system 20 is shown close to the processing circuitry 40, the memory system 20 may include a memory or storage device (such as a cloud storage system and other remote electronic storage systems) implemented in a separate circuit, housing, device and / or computing platform and communicating with the image analysis system 10.

[0075] Memory system 20 may include one or more external or internal devices (random access memory or RAM, read-only memory or ROM, flash memory, hard disk storage devices or HDDs, solid-state devices or SSDs, static storage devices (such as disks or optical discs), other types of non-transitory machine or computer-readable media, etc.) configured to store data and / or computer-readable code and / or instructions for performing, executing, or facilitating the various processes or instructions described herein. Memory system 20 may include volatile or non-volatile memory (e.g., semiconductor-based memory devices, magnetic memory devices and systems, optical memory devices and systems, fixed memory, or removable memory). Memory system 20 may include database components, object code components, scripting components, or any other type of information structure for supporting the various activities described herein. In some instances, memory system 20 may be communicatively connected to processing circuitry 40 and may include computer code for performing one or more processes described herein. Memory system 20 may contain various modules, each capable of storing data and / or computer code associated with a specific type of function.

[0076] Processing circuitry 40 may include processor 42, which is configured to implement or execute one or more algorithms 90 to determine output 2000 based on received data, which may include preoperative data 1000 and / or any data in memory system 20. Preoperative data 1000 may be received via manual input, retrieved from memory system 20, and / or directly from preoperative measurement system 100. Processor 42 may be configured to determine patterns based on the received data.

[0077] Processor 42 may be implemented as a general-purpose processor or computer, a special-purpose computer or processor, a microprocessor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), one or more field-programmable gate arrays (FPGAs), a set of processing units, a processor based on a multi-core processor architecture, or other suitable electronic processing units. Processor 42 may be configured to execute machine-readable instructions, which may include one or more modules implemented as one or more functional logic, hardware logic, electronic circuits, software modules, etc. In some cases, processor 42 may be located remotely from one or more computing platforms constituting image analysis system 10. Processor 42 may be configured to perform one or more functions associated with image analysis system 10, such as precoding antenna gain / phase parameters, encoding and decoding of individual bits forming communication messages, formatting information, and overall control of one or more computing platforms constituting image analysis system 10 (including processes associated with the management of communication resources and / or communication modules).

[0078] In some aspects, the processing circuitry 50 and / or the memory system 20 may include several modules related to the medical procedure, such as input modules, analysis modules, and output modules. The image analysis system 10 does not need to be contained in a single housing. Instead, the components of the image analysis system 10 can be located in various different locations, or even remotely. The components of the image analysis system 10 (including the components of the processing circuitry 40 and the memory system 20) can be located in components of different computers, robotic systems, devices, etc., used in surgical procedures.

[0079] The image analysis system 10 may use one or more algorithms 90 to perform intermediate determinations and determine one or more outputs 2000. One or more algorithms 90 may be configured to determine or collect data based on preoperative data 1000 (including image data 1010). For example, one or more algorithms 90 may be configured for bone identification, soft tissue identification, and / or to make determinations related to the previously described intermediate image data 1010. One or more algorithms 90 may operate simultaneously and / or individually to determine one or more outputs 2000, and / or display or represent one or more outputs 2000 via a GUI 250.

[0080] One or more algorithms 90 may be algorithms trained using, for example, linear regression, random forest regression, CatBoost regression, statistical shape modeling, or SSM. One or more algorithms 90 may be based on actual outcomes and / or results 12 ( Figure 1 One or more algorithms 90 can be configured to continuously modify and / or improve the received image data 1010, including images, videos, and / or scans, using segmentation and / or thresholding techniques to determine the previously described intermediate image data 1010 and / or one or more outputs 2000. For example, one or more algorithms 90 can be configured to segment images (e.g., CT scans), threshold soft tissues, generate one or more plain text (e.g., .txt) comparisons of certain identified bones or tissues (e.g., tibia and femur), and run code to extract values ​​(e.g., PPT or PTT) and populate a database. One or more algorithms 90 can be configured to automatically perform data extraction and / or collection upon receiving images from the imaging device 110.

[0081] One or more algorithms 90 may include joint space width algorithm 50, osteophyte detection algorithm 60, B-score algorithm 70, alignment / deformity algorithm 80, predictive algorithm 85, and bone mineral density (BMD) algorithm 87. Alternatively, one or more of these algorithms may be combined. For example, joint space width algorithm 50, osteophyte detection algorithm 60, B-score algorithm 70, alignment / deformity algorithm 80, predictive algorithm 85, and BMD algorithm 87 may be combined into a single or master algorithm. Each of joint space width algorithm 50, osteophyte detection algorithm 60, B-score algorithm 70, alignment / deformity algorithm 80, predictive algorithm 85, and BMD algorithm 87 may be configured to use not only preoperative data 1000 as input, but also determinations and / or outputs 2000 from each other. Preoperative data 1000 may be used to create various intelligent models. In some examples, the intelligent model may be a statistical model, a finite element model, a neural network, and / or a predictive artificial intelligence model (such as a basic learning model). The technique for determining BMD using medical images is described in the article “A Novel Method of Determining Bone Mineral Density from a Pre-surgical CT scans to aid in Surgical Planning” by Niall Maguire, Jared Weir, Sebastian Lustig, and Alan Brett, which is included here as Appendix A and is incorporated herein by reference in its entirety.

[0082] Each of one or more algorithms 90 (e.g., joint space width algorithm 50, osteophyte detection algorithm 60, B-score algorithm 70, alignment / deformity algorithm 80, prediction algorithm 85, and BMD algorithm 87) can be configured to identify or detect bones, tissues, bony landmarks, etc., and calculate or predict their size and / or location based on images acquired by the imaging device 110 using image processing techniques. One or more algorithms 90 are not limited to determinations related to joint space width, osteophyte volume, B-score, alignment / deformity, and bone mineral density (BMD), and may include and / or be configured to perform other procedural determinations, such as those related to joint laxity or stiffness, discharge or hospital stay, frailty, fall risk, balance assessment, patient readiness, etc. The joint space width algorithm 50, osteophyte detection algorithm 60, B-score algorithm 70, alignment / deformity algorithm 80, prediction algorithm 85, and BMD algorithm 87 will be described in more detail throughout the specification.

[0083] One or more algorithms 90 (e.g., joint space width algorithm 50, osteophyte detection algorithm 60, B-score algorithm 70, alignment / deformity algorithm 80, predictive algorithm 85, and BMD algorithm 87) can operate simultaneously (or alternatively, at different times throughout the preoperative and intraoperative periods) and exchange inputs and outputs. One or more algorithms 90 can be configured to determine other scores, values, and / or parameters, and are not limited to joint space width, osteophyte volume, B-score, alignment / deformity, and BMD. For example, one or more algorithms 90 can be configured to determine scores related to bone mineral density / quality (e.g., T-score), joint stiffness or laxity, patient readiness, bone-to-skin ratio, etc.

[0084] One or more outputs 2000 may include a predicted procedure 2005, a predicted procedure time or duration 2010, a procedure plan 2020, an operating room layout 2030, an operating room scheduling 2040, assigned or designated staff 2050, recommended surgeon ergonomics 2070, a predicted procedure outcome 2080, and a patient anatomy representation 2090, which may include identified and / or enhanced images displayed on the monitor 210. Each of these outputs 2000 (e.g., predicted procedure 2005, predicted procedure time or duration 2010, procedure plan 2020, operating room layout 2030, operating room scheduling 2040, assigned or designated staff 2050, recommended surgeon ergonomics 2070, predicted procedure outcome 2080, and patient anatomy representation 2090) may be used as input 1000 to determine the other outputs 2000. Thus, each of these outputs 2000 (e.g., predictive procedure 2005, predictive procedure time or duration 2010, procedure plan 2020, operating room layout 2030, operating room scheduling 2040, assigned or designated staff 2050, recommended surgeon ergonomics 2070, predicted procedure outcome 2080, and patient anatomy representation 2090) can be based in part on different outputs 2000 (e.g., predictive procedure 2005, predictive procedure time or duration 2010, procedure plan 2020, operating room layout 2030, operating room scheduling 2040, assigned or designated staff 2050, recommended surgeon ergonomics 2070, predicted procedure outcome 2080, and patient anatomy representation 2090). For example, operating room scheduling 2040, assigned or designated staff 2050, and predicted outcome 2080 can be based in part on predictive procedure 2005 and / or predictive procedure time or duration 2010. As another example, patient anatomy representation 2090 can be based on predicted outcome 2080, but the aspects disclosed herein are not limited to this.

[0085] The predicted procedure time 2010 may be the total time or duration of the procedure (e.g., as outlined in the procedure plan 2020) and may further include the time or duration of each step or process of the procedure. In some examples, the predicted procedure time 2010 may be the predicted time for completing a portion of the procedure. The predicted outcome 2080 may include the patient's predicted perceived pain level, the patient's predicted stress level, anxiety level and / or mental health status, predicted cartilage loss, predicted risk of infection, case difficulty rating, etc. The predicted outcome 2080 may also include predictions and / or risks if the time taken during the procedure exceeds (or alternatively is less than) the predicted procedure time 2010 (e.g., how the risk of complications and / or infection may increase based on the procedure taking longer than the predicted procedure time 2010).

[0086] The patient anatomical representation 2090 may be determined or calculated in relation to the imaging data 1010 and the patient's anatomy, and may be displayed on various GUIs 250 described in more detail later. The patient anatomical representation 2090 may include and / or be based on predicted outcomes 2080, such as predicted cartilage loss, joint space width, etc. The patient anatomical representation 2090 may be based on and / or superimposed on images acquired by the imaging device 110 and input as imaging data 1010. In some examples, some or all portions of the patient anatomical representation 2090 may be based on prior procedural data 1050 and / or simulated.

[0087] Outputs 2000 may be output electronically (e.g., on display 210, mobile device 220, or any other monitor or display that may be part of program system 240) or physically printed (e.g., via a printer on paper, canvas, or film 230 or other materials). Display 210 may display one or more GUIs 250 to output these outputs 2000. For ease of description, GUI 250 will be described in more detail below in conjunction with one or more algorithms 90 and outputs 2000 (e.g., predicted outcome 2080 and patient anatomical representation 2090). GUI 250, Algorithm 90, and Output 2000

[0088] As previously explained, the image analysis system 10 can use one or more algorithms 90 to determine a graphical user interface (GUI) 250, which can be displayed on any output system 210. When implemented on a touchscreen, the GUI 250 can be interactive. Although various GUIs 250 are described individually herein, various GUIs 250 can be displayed simultaneously and / or shown on the same screen of the display 210.

[0089] refer to Figure 2 as well as Figures 3A to 3BImage data 1010 may include images 302 of patient anatomy acquired using imaging device 110 (e.g., CT scanner 110) (e.g., CT scan). These acquired images 302 may show one or more bones and may indicate bone features such as osteophytes (or bone spurs forming on bones), and if joints are shown, indicate the joint space width (or the distance between two bones). One or more algorithms 90 may analyze the acquired images 302 using one or more image processing methods to determine various outputs 2000. One or more algorithms 90 may also use additional preoperative data 1000 (e.g., patient data 1020) to determine the outputs 2000 to be displayed on GUI 250.

[0090] One or more GUIs 250 may include a first or “raw image” GUI 252, which may display one or more acquired images 302. As an example, the raw image GUI 252 may include visual indicators 304 (e.g., circles, pointers, etc.) that may indicate osteophytes, bony landmarks, or specific joint space widths. The position of the visual indicators 304 may be determined manually (e.g., by a practitioner touching the screen) or by one or more algorithms 90. Additionally, the raw image GUI 252 may display text 306 describing what is indicated by the visual indicators 304. Figure 3A The example shown illustrates an acquired image 302 of the knee joint. The visual indicator 304 may be a circle that encircles one or more osteophytes detected by one or more algorithms 90 (e.g., osteophyte detection algorithm 60), and the text 360 may display words such as "medial osteophyte" to identify the location of the osteophyte. Figure 3B The example shown illustrates an acquired image 302 of the knee joint. A visual indicator 304 could be a circle outlining the width of the joint space at a location between the tibia and femur, which is narrower than the joint space width at other areas or compared to other images. The joint space width can be detected and / or calculated by one or more algorithms 90 (e.g., joint space width volume algorithm 50), and text 360 could display terms such as "medial joint space width loss" to identify the location of the indicated joint space width.

[0091] In some examples, the information displayed on each of the GUI 250 can be manipulated by user input to operate the GUI 250 and / or the program system 240. For example, to manipulate the positioning of images displayed on the GUI 250 and / or the program system 240, the user can make one or more commands (inputs, actuation of one or more buttons, gestures, or other inputs). In some examples, the user can execute one or more commands to perform steps of a surgical workflow on patient image data (such as two-dimensional images or three-dimensional images or models of patient anatomy), performing implant selection, osteotomy selection, and / or other aspects of the surgical workflow, such as manipulating surgical parameters (e.g., positioning, thickness, type, depth). In some examples, the input / command is gesture control. Gesture control can be facilitated by machine vision software that can utilize one or more cameras within the program system 240. Gesture control can be used to manipulate the display of bones, implants, surgical workflow planning, etc. Some examples of gesture control can be achieved by the user looking at the display and moving their eyes in a specific manner (e.g., eye-tracking software); the user moving their hand relative to the display (e.g., performing one or more gestures to actuate one or more commands of the program system 240); or other types of gesture movements that can be detected and received by the program system 240. In some examples, the user can use their eyes to gesture up, down, left, right, and / or blink to interact with the GUI / display / program system 240, such as moving an image left, right, up, or down, or rotating an image left / right / up / down. In some examples, these gestures can be moving the hand up and down, pinching, pulling, sliding, etc. In one example, pinching while moving the hand up and down can move the 3D skeleton display up and down within the GUI. In another example, pinching while moving the hand left and right will rotate the 3D skeleton display around the X-axis within the GUI. In a further example, pinching the fingers while moving the hand can move the 3D skeleton display in the z-direction, allowing the skeleton to be placed anywhere within the display screen or a virtual reality environment (e.g., a 3D display of a virtual model, etc.). In some examples, gesture control can depend on which hand is moving (right or left) and what movements each hand is performing. In one example, to change the angle of the implant within the displayed 3D model, a user could make a fist with their left hand and squeeze and move their right hand up and down to change only the display angle of the implant's 3D model, without altering any other 3D models displayed alongside the implant on the GUI or other electronic display. In another example, to move the implant, a user could squeeze and move their right hand in the desired direction to adjust the implant's display in the same way, such as rotating the implant, moving the implant within the display, etc. 3D Model and Overall Analysis GUI

[0092] Figures 4A to 4J Examples of second GUI 254, third GUI 255, fourth GUI 256, and fifth GUI 257 are shown, illustrating an overall representation of the patient's anatomy (e.g., second GUI 254) and certain parameters that can be determined by the image analysis system 10 (e.g., fourth GUI 256). Second GUI 254 and third GUI 255 may display the patient's anatomy, fourth GUI 256 may display the patient's anatomy with one or more implants superimposed, and fifth GUI 257 may display analysis and / or include acquired images 302. Second GUI 254 and third GUI 255 may alternatively be referred to as anatomical or pre-op GUIs, and fourth GUI 256 and fifth GUI 257 may alternatively be referred to as predictive or post-op GUIs; however, second GUI 254 and third GUI 255 are not limited to displaying pre-determined parameters, and fourth GUI 256 is not limited to displaying post-determined parameters. Fifth GUI 257 may also be displayed pre-operatively, intra-operatively, and / or post-operatively. Algorithm 80 for Linear / Distortion

[0093] refer to Figures 2 to 4J Alignment / deformity algorithm 80 and / or one or more other algorithms 90 can analyze the acquired image 302 to determine output 2000 to be displayed on the second GUI 254, third GUI 255, and / or fourth GUI 256. Alignment and / or deformity can refer to how two or more bones are positioned and / or moved compared to a healthy patient with healthy alignment at those two or more bones. Alignment / deformity algorithm 80 can be configured to detect or identify one or more target bones or joints in the acquired image 302, detect the relative positioning and / or size of the one or more target bones or joints, detect one or more bony landmarks on the detected bones or joints, and determine or calculate one or more alignment / deformity parameters related to the size (e.g., volume) of one or more osteophytes detected or in one or more target joints based on preoperative data 1000 (e.g., image data 1010).

[0094] One or more alignment / deformity parameters may include alignment and / or relative positioning data at certain locations (e.g., joint locations), across different directions (e.g., medial or lateral), mean or average alignment and / or alignment scores, alignment in change or progression, alignment based on predicted or determined implants, etc. Alignment / deformity algorithm 80 may evaluate one or more of these alignment / deformity parameters on various anatomical regions (e.g., anterolateral, anteromedial, central lateral, central medial, posterolateral, posteromedial) of one or more bones (e.g., femur or tibia) and / or one or more bones (e.g., tibia and femur). Alignment / deformity algorithm 80 may also be configured to predict alignment or progression based on other preoperative data 1000 (e.g., kinematic data or activity level data).

[0095] One or more alignment / deformity parameters may include alignment and / or relative positioning (e.g., relative to anatomical and / or mechanical axes), such as lower limb mechanical alignment, lower limb anatomical alignment, femoral articular facet angle, tibial articular facet angle, mechanical axis alignment strategy, anatomical alignment strategy, natural knee alignment strategy, femoral arching, varus-valgus deformity and / or angle, tibial arching, patellofemoral alignment, coronal plane deformity, sagittal plane deformity, extension movement, flexion movement, anterior cruciate ligament (ACL) integrity, posterior cruciate ligament (PCL) integrity, and range of active and passive movements of the joint. Knee joint motion and / or range of motion data in all three planes during the period (e.g., collected using markers presented in the original images, videos, or scans), three-dimensional size, quantitative data indicating the proportions and relationships of joint anatomy in both static and dynamic states, quantitative data indicating joint line height, metaphyseal flare, medial femoral metaphyseal flare, proximal tibiofibular joint, coronal tibial diameter, intercondylar femoral diameter, intermetrophyseal femoral diameter, sagittal tibial diameter, posterior femoral condyle offset—medial and lateral, distance from the lateral epicondyle to the joint line, and / or distance from the tibial tuberosity to the joint line. However, the aspects disclosed herein are not limited to these alignment parameters.

[0096] The one or more alignment / deformity parameters may include data on: bony landmarks (e.g., condylar surfaces, bones or epiphysis, necks or metaphysis, body or diaphysis, articular surfaces, epicondyles, processes, protuberances, tubercles and trochanters, trochanters, spines, linear or linear structures, facets, ridges and crests, foramina and clefts, canals, fossae and pits, notches and grooves, and sinuses) and / or skeletal geometry (e.g., diameter, slope, angle) and other anatomical geometry data. This geometry is not limited to overall geometry and may include specific lengths or thicknesses (e.g., the length or thickness of the tibia or femur). Imaging data 1010 may also include data on soft tissue used for ligament attachment points and / or data used to determine ligament attachment sites.

[0097] Alignment / deformity algorithm 80 can determine, based on image data 1010 and / or supplementary patient data 1020, whether a particular alignment malformation, deformity, distance between certain bones, and / or angle between different bones is increased or decreased, by comparing previously measured alignment / deformity parameters and / or by comparing image data from previously acquired images. Alignment / deformity algorithm 80 can further determine, predict, or diagnose disease states or disease progression (e.g., osteoarthritis or OA) based on the determined alignment / deformity parameters.

[0098] Based on the determined alignment / deformation parameters, the image analysis system 10 can use alignment / deformation algorithm 80 and / or one or more algorithms 90 in combination to determine a patient anatomical representation 2090. The determined anatomical representation 2090 can be displayed and / or expressed on one of the GUIs 250 as an artificial or representative model (such as representative model 402 in the second GUI 254) of the current anatomy of the current patient (e.g., bone). In some examples, some or all of the representative model (e.g., features that may not be acquired through certain imaging modalities) can be simulated and / or based on prior procedural data 1050. For example, some X-ray scans may provide more information about bone and cartilage but less about soft tissue, and therefore ligaments can be simulated in the representative model. (See also: Regarding...) Figure 24 In more detail, one or more algorithms 90 can determine the motion of a representative model, such as flexion or extension, to demonstrate how the various parts of the anatomical structures (e.g., ligaments, fibula, etc.) interact with each other or other bones (e.g., tibia and / or femur) during the motion.

[0099] Some or all of the artificial model 402 may be based on one or more acquired images 302 taken intraoperatively using imaging device 110, either preoperatively or postoperatively, or even during the medical procedure. One or more algorithms 90 may use previously stored models or standard anatomical models, which may be included in memory system 20 as stored data 30. One or more algorithms 90 may detect or identify bony landmarks, osteophytes, joint space widths, and other features in the acquired images 302 of the current patient and modify previously stored models to reflect the current patient's anatomy, thereby determining the artificial model 402. One or more algorithms 90 may determine colors or other indicators to mark or identify the identified features and / or other determinations, such as impact points. The artificial model 402 may be a three-dimensional representation and different views may be selected via a touchscreen or mouse-operated GUI 254. For example, the artificial model 402 may show one or more bones that can be rotated, moved, or spun around various axes (to change the perspective of the one or more bones) using a mouse, touchscreen, or other user input device.

[0100] like Figure 4A As illustrated, the second GUI 254 may include an artificial model 402 of the knee joint determined by the image analysis system 10 using various knee joint (e.g., tibia, femur, and patella) models stored in the memory system 20, and shows the tibia and femur. Relative positioning (e.g., joint space width) may indicate the determined joint space width, but is not necessarily so. As illustrated, the second GUI 254 may use colored or shaded indicators 404 to display osteophytes. The shaded indicators 404 may be turned on and / or off, which will be described and shown in more detail later. "Toggling" may refer to showing or hiding certain features.

[0101] The second GUI 254 may also include multiple widgets 406 related to the determinations or outputs 2000 of the image analysis system 10 (e.g., predicted outcome 2080). In some examples, the multiple widgets 406 may be indicators of statistical rankings of diseased versus normal / healthy anatomy. The multiple widgets 406 may display the statistical rankings or other determinations and outputs 2000 as 3D volume measurements, 2D area or cross-sectional measurements, or 1D thickness or orientation measurements. For example, the multiple widgets 406 may include charts, graphs, text, and other indicators that predict perceived pain that the patient may perceive after medical procedures associated with the anatomy depicted in the artificial model 402. Multiple components 406 may also visually indicate other parameters determined by one or more algorithms 90, such as joint space width, osteophyte volume, B-score, deformity / alignment data, prediction procedure 2005, steps in the procedure plan 2020 (e.g., implant type or design), prediction procedure time 2010, operating room (OR) layout 2030 or operating room (OR) scheduling 2040, assigned staff 2050 or surgeon ergonomics 2070. Multiple components 406 may include graphs comparing certain parameters to parameters (such as B-score, joint space width, or osteophyte volume) of healthy patients with similar characteristics to the current patient (e.g., sex, age, medical history). Multiple components 406 may be or include selectable icons that, when clicked, display expanded and / or additional information (e.g., more textual information about perceived pain and recommended steps to alleviate patient pain).

[0102] like Figures 4B to 4E As shown, the main GUI 255 can be an alternative to or supplement to the second GUI 254. The third GUI 255 may include an artificial model 402, one or more indicators 404, multiple widgets or cards 409, and a menu 418, which are described in more detail in the explanation of one or more algorithms 90.

[0103] Artificial model 402 can be a simulation model or a model based on the patient's skeleton (e.g., from the acquired image 302). Artificial model 402 can be a joint (e.g., knee joint) model determined by image analysis system 10 using various joint or bone (e.g., tibia, femur, and patella) models stored in memory system 20 (and / or based on the acquired image 302), and shows the tibia and femur. Relative positioning (e.g., joint space width) can indicate the determined joint space width, but does not necessarily have to be. Artificial model 402 can depict the preoperative condition of the patient's anatomy, preoperative predictions of the patient's anatomy after various treatments (including predictions of the patient's anatomy after no treatment), intraoperative condition and / or predictions based on intraoperative data, and / or postoperative condition of the patient's anatomy and / or predictions of long-term anatomy or movement based on intraoperative and / or postoperative data, etc.

[0104] Indicator 404 can highlight areas of interest, such as osteophytes. As illustrated, the third GUI 255 can use colored or shaded indicator 404 to display osteophytes. The shaded indicator 404 can be turned on and / or off, as described in more detail in reference menu 418.

[0105] The third GUI 255 may also include multiple widgets or cards 409 related to the determination or output 2000 of the image analysis system 10 (such as prediction procedure 2005 or prediction outcome 2080), and may include charts, graphs, text, metrics, etc., as described in conjunction with widget 406 on the second GUI 254. As an example, the multiple widgets 409 may include a prediction procedure time widget 412, a B-score widget 414, and / or a C-score or prediction cartilage loss widget 416. Practitioners or users may click on one of these cards or widgets 409 to display a magnified view of widget 409, a pop-up, a box, a screen, or a new GUI based on one or more GUIs 250 described below.

[0106] The predicted procedure time widget 412 can display information related to the predicted procedure time or duration 2010. The procedure time widget 412 can display the predicted procedure time in minutes, hours, etc. (e.g., according to procedure plan 2020 and / or planned procedure 1030). The procedure time widget 412 can also display a visual indication of how long the procedure time 2010 lasts compared to other procedures and / or similar procedures (e.g., the average time of similar procedures for patients with similar characteristics). For example, the procedure time widget 412 can include a gradient bar or a semicircle or radial gradient to indicate the severity of the procedure time 2010. Longer procedure times 412 can be visualized by indicators further to the right on the gradient bar and / or by highlighted colors on the gradient bar, such as green for indicating a procedure time at or below a threshold procedure time (e.g., average procedure time), orange for indicating a procedure time within a first time period above the threshold procedure time, and / or red for indicating a procedure time above the first time period and / or a prediction that increases the risk and / or complications. Users can click the predicted procedure time widget 412 to display a magnified view and / or pop-up of metrics related to procedure time. For example, it can display scheduling information and / or availability, case difficulty, recommended staff assignments, surgical tools, etc.

[0107] B-score widget 414 can display information related to the B-score, which will be described in more detail with reference to B-score algorithm 70. B-score widget 414 can display the patient's B-score (e.g., determined by B-score algorithm 70), a skeletal image representing the B-score (e.g.,...). Figure 14 The femur and / or representative model 1402 shown), and videos or simulations of bone evolution used to represent changes in bone shape (e.g., see later reference) Figure 16A The description of B-score video 1606), and the gradient bar and / or indicators used to indicate the B-score (e.g., see later). Figures 16A to 16C The scale described is 1602. Indicators on the gradient scale can indicate the severity of the B-score and / or the likelihood of complications associated with the B-score. Users can click on the B-score widget 414 to display a magnified view and / or pop-up of metrics related to procedure time. For example, B-score video 1606 and / or other B-score GUIs described below.

[0108] The C-score widget 416 can display information related to the probability of cartilage loss and / or the C-score, which will be described in more detail with reference to the joint space width algorithm 50. The C-score widget 416 can display the patient's C-score and / or joint space width parameters or determination (e.g., determined by the joint space width algorithm 50), a bone image representing the C-score and / or joint space width, and gradient bars and / or indicators for indicating the C-score (e.g., as described later). Figures 6A to 6C The scale described is 608. As an example, the C-score widget 416 can display C-score values ​​corresponding to various sections of the skeleton, such as reference... Figures 6A to 6C The values ​​602 and / or 604 described are displayed on the artificial model 402. Users can click the C-scoring widget 416 to display a magnified view and / or pop-up of the metric related to the joint space width. For example, refer to... Figure 6A The gradient 608 and / or other joint space widths described below are GUIs.

[0109] although Figure 4B The plurality of components 409 shown illustrate a procedure timing component 412, a B-score component 414, and a C-score component 416, but the plurality of components 409 may include alternative or additional components, such as components displaying scheduling information, surgeon ergonomics, simulated motion, implant design, and / or depicting the output 2000 of the image analysis system 10 and other identified components.

[0110] Menu 418 may provide a user interface to allow a user (e.g., a practitioner) to change the view or orientation, toggle or highlight certain areas, features, or bones (e.g., hide or show osteophytes on the femur and / or tibia), show or hide certain bones (e.g., the tibia or fibula), and / or show or simulate specific movements (e.g., flexion or extension). The practitioner may also be able to change the opacity of certain highlighted features (e.g., osteophytes) by moving an indicator along a bar to change the opacity level. Menu 418 may provide various menus and / or submenus and may be provided as a panel or column separate from or otherwise distinguished from the frame showing the artificial model 402. For example, as... Figure 4B As shown, menu 418 may be a panel located on the left side of the screen displaying the artificial model 402. In some examples, menu 418 may be movable. For example, a user may interact with menu 418 (e.g., click and drag the menu) to change the menu's positioning (e.g., so as not to interfere with the display of the artificial model 402). In some examples, the positioning and / or orientation of menu 418 may change automatically so as not to interfere with the artificial model 402 (e.g., during simulated motion).

[0111] Menu 418 can display case-related identifiers, such as case number or other patient or case IDs. Menu 418 may include drop-down menus, buttons, or other user input configured to display patient information. For example, ... Figure 4B As illustrated, menu 418 may include a "Patient Details" button. When the user clicks the "Patient Details" button, the third GUI 255 may display a screen, pop-up, or submenu or box that displays patient-related information, such as patient data 1020, schedule data 1030, and / or other information. Figures 4F to 4J The information shown. Alternatively or in addition, menu 418 may include drop-down menus, buttons, or other user inputs configured to display information about surgeon or staffing, case difficulty, procedures, etc., such as surgeon data 1040, assigned staff 2050, OR layout 2030, OR scheduling 2040, procedure planning 1030, procedure plan 2020, or preoperative data 1000 and / or output 2000.

[0112] As previously mentioned, menu 418 may include various user inputs (e.g., switches, buttons, sliders) to turn certain features on and / or off. These user inputs for turning features on and / or off may be provided under submenus that can be hidden or shown. For example, as... Figure 4B As shown, the third GUI 255 can display a submenu titled "Surface". When the user clicks this submenu (and / or an arrow displayed on the submenu), user input (e.g., a switch) can be displayed (e.g., as an extension of menu 418). When the user clicks the submenu while the user input is displayed, the user input can be omitted and / or hidden (e.g., the size of menu 418 can be shortened or reduced).

[0113] Menu 418 may be included in the section for features intended to be turned on and / or off (e.g., Figure 4B The "bones" section provides switches or other user inputs (e.g., buttons, sliders, submenus, or dropdown menus). Within this section, the third GUI 255 can provide toggles, switches, sliders, etc., for each area or skeleton, allowing the user to selectively turn features on / off in that area. For example, in... Figure 4BUnder the "Osteophyte" section, a toggle or switch can be provided for the "Femur" and another toggle or switch can be provided for the "Tibia". When the user clicks or slides the "Femur" switch in the first direction, an indicator 404 of the osteophytes on the femur of the artificial model 402 can be displayed, while when the user clicks or slides the "Femur" switch in the second direction, the indicator 404 of the osteophytes on the femur of the artificial model 402 can be omitted or hidden. Similarly, when the user clicks or slides the "Tibia" switch in the first direction, an indicator 404 of the osteophytes on the tibia of the artificial model 402 can be displayed, while when the user clicks or slides the "Tibia" switch in the second direction, the indicator 404 of the osteophytes on the tibia of the artificial model 402 can be omitted or hidden. Although "femur" and "tibia" are used as examples, menu 418 can refer to different sections and / or different bones and include different labels to turn osteophytes on / off in these different sections or bones.

[0114] Menu 418 may also include sections for bones or bone segments intended to be turned on and / or off (e.g., Figure 4B The third GUI 255 can provide switches or other user inputs (e.g., buttons, sliders, submenus, or drop-down menus) under the "skeleton" in the GUI. Within this section, the third GUI 255 can provide toggles, switches, sliders, etc., for each section or skeleton, allowing the user to selectively turn that area or skeleton on / off. For example, in... Figure 4B Under the "Skeleton" section, a toggle or switch can be provided for the "Femur," another toggle or switch for the "Tibia," and yet another toggle or switch for the "Fibula." When the user clicks or slides the "Femur" switch in the first direction, the femur of the artificial model 402 can be displayed; when the user clicks or slides the "Femur" switch in the second direction, the femur of the artificial model 402 can be omitted or hidden. Similarly, when the user clicks or slides the "Tibia" switch in the first direction, the tibia of the artificial model 402 can be displayed; when the user clicks or slides the "Tibia" switch in the second direction, the tibia of the artificial model 402 can be omitted or hidden. When the user clicks or slides the "Fibula" switch in the first direction, the fibula of the artificial model 402 can be displayed; when the user clicks or slides the "Fibula" switch in the second direction, the fibula of the artificial model 402 can be omitted or hidden. Although “femur,” “tibia,” and “fibula” are used as examples, menu 418 can refer to different segments, bones, tissues, ligaments, etc., and includes corresponding labels to turn these features on / off.

[0115] Menu 418 may include a section for displaying motion (e.g., simulated motion) of a bone or bone segment intended to be turned on and / or off. Figure 4BThe third GUI 255 can provide switches or other user inputs (e.g., buttons, sliders, submenus, or drop-down menus) under "bending" in the context of movement. It can also provide toggles, switches, sliders, etc., for each type of movement (e.g., "bending," "extending," etc.) or alternatively for specific positioning (e.g., angular positioning, such as 45 degrees, 90 degrees, etc.), allowing the user to selectively display movement of the artificial model 402 to a particular positioning arrangement. For example, in... Figure 4B Under the "buckling" section, a toggle or switch can be provided for "Show buckling". Alternatively, a toggle or switch can be provided for a specific degree or amount of buckling (e.g., various angle values ​​or percentages, such as 50% or 100%). When the user clicks or slides the "Show buckling" switch in the first direction, the mannequin 402 can be displayed as buckling, while when the user clicks or slides the "Show buckling" switch in the second direction, the movement of the mannequin 402 can be paused and / or the mannequin 402 can be displayed in the opposite (e.g., extended) arrangement.

[0116] Menu 418 may include sliders, switches, or other user inputs (e.g., buttons, submenus, or drop-down menus) to change the opacity of certain features. For example, Figure 4B The diagram shows a section titled "Bone Spur Opacity," a slider below the section label, and an indicator for the percentage of opacity. The opacity of the indicator 404 depicting the bone spur changes depending on the location when the user clicks at a position on the slider or moves a button along the slider. For example, the leftmost position of the slider could correspond to 0% opacity, the rightmost position could correspond to 100% opacity, and the degree of button movement or its position along the right side of the slider could correspond to the percentage of opacity. Although... Figure 4B The text shows "osteophyte opacity," but alternatively or otherwise, the third GUI 255 may enable the modification of other opacities, such as the opacity of bones or segments of the artificial model 402 (e.g., to provide a view of implants superimposed on the artificial model 402), tissues, or ligaments.

[0117] Menu 418 may also include buttons, switches, etc., for turning widget 409 on and / or off and / or for hiding or showing different widgets 409 to individually turn widget 409 on and / or off. For example, Figure 4BAs shown, the third GUI 255 can display a submenu titled "Widgets". When the user clicks this submenu (and / or the arrow displayed on the submenu), all widgets 409 can be displayed, and when the user clicks the submenu again, these widgets can be hidden or omitted. Alternatively, when the user clicks the submenu, a list of individual widgets with toggle switches or other user input (e.g., program time widget 412, B-rating widget 414, and / or C-rating widget 416) can be displayed, allowing the user to selectively show or hide the individual widgets 409 independently of each other.

[0118] Figure 4B This is an example of the third GUI 255 showing the tibia, femur, and fibula with highlighted or colored osteophytes. Figure 4C The third GUI 255 shows a top view of the tibia and fibula, with the femur closed. Figure 4D The third GUI 255 is shown, displaying flexion of the tibia and femur. Figure 4E The third GUI 255 shows the osteophyte displayed at 60% opacity via indicator 404.

[0119] refer to Figure 4F and 4G The fourth GUI 256 may include an artificial model 402 of the current patient's current anatomy (e.g., bones), such as the artificial model 402 shown in the second GUI 254, overlaid with planned or recommended implants or prosthetic components 408. The fourth GUI 256 may show a portion of the artificial model 402 (e.g., femur or tibia) and the associated implant 408 (e.g., a femoral implant and a tibial implant as part of a knee joint implant) in various views. The implant 408 may be shown in a different color and / or shade compared to the artificial model 402. Figure 4C As illustrated, the fourth GUI 256 can display side, top, and perspective views of the femur 402 with the femoral implant 408 superimposed, and side, top, and perspective views of the tibia 402 with the tibial implant 408 superimposed. As another example, the fourth GUI 256 can display a portion of the pelvic bone, hip bone, iliac crest, and femur and other associated bones 402 with one or more hip implants 408 (e.g., femoral head with acetabular components and / or stems) superimposed. The implants 408 can be opened and / or closed based on user input, and the view of the artificial model 402 (along with the implants 408, if desired) can be manipulated, rotated, spun, etc., around various axes by manipulating the fourth GUI 256 with a mouse, touchscreen, or other user input.

[0120] The fourth GUI 256 may also include one or more components 410. One or more components 410 may include components and / or information similar to components 406 of the second GUI 254 and / or components 409 of the third GUI 255, but the aspects disclosed herein are not limited thereto. One or more components 410 may include patient anatomy or the size, alignment, or other geometric information of the implant 408, or parameters to be used for implant placement in the procedure plan 2020. For example, when used preoperatively or intraoperatively, one or more components 410 may display the recommended thickness, location, type (e.g., stable implant), brand, material, etc., of the implant 408, the recommended osteotomy or inclination or other preparation for implant placement, the number or thickness of pads or supplements, etc. Component 410 may display alignment and / or deformity information (e.g., as determined by alignment and / or deformity algorithm 80), patient data 1020, or other inputs 1000 (e.g., range of motion data), etc.

[0121] Widget 410 can display predicted outcome 2080 as well as desired outcome, and widget 410 can be interactive, allowing other predicted outcomes 2080 to change when the practitioner manipulates certain parameters of implant 408 (e.g., positioning, thickness, type), osteotomy, etc. (this can be done by manipulating information in widget 410 and / or by manipulating the displayed implant 408 or representative model 402), enabling the practitioner to assess whether at least some of the predicted outcomes 2080 can get closer to the desired outcome. When used postoperatively, widget 410 in the fourth GUI 256 can display the actual parameters used during the procedure, and widget 410 can also display patient outcomes (which can be reported by the patient or practitioner, or updated using sensors in implant 408), further predictions during recovery, recommendations for revision surgery, etc.

[0122] refer to Figures 4H to 4J The fifth GUI 257 can display a classification or analysis 415 of the patient's condition determined by one or more algorithms 90 (e.g., alignment / deformity algorithm 80), such as "severe varus" or "mild valgus". The classification 415 of the patient's condition can be based on a B-score determined by a B-score algorithm 70, a C-score determined by a joint space width algorithm 50 and / or one or more algorithms 90, etc.

[0123] The fifth GUI 257 may display an artificial model 402, metrics or other measurements 415 related to alignment or deformity (e.g., determined by alignment / deformity algorithm 80), and metrics 420 and / or gradient maps 417 and / or 419 related to B-scores and / or C-scores (e.g., determined by B-score algorithm 70, and C-scores determined by joint space width algorithm 50 and / or one or more algorithms 90). For example, metric 415 may include scores, points, or positioning values ​​(e.g., degrees) corresponding to motion or positioning parameters (such as flexion contractures and / or coronal malalignment), and may display the sum or total of these points or values. Metric 415 may also include tables or scales to help users assess the severity of a patient's condition based on the total number of points (e.g., mild is less than a first predetermined number of points, such as 10; moderate is between the first and second predetermined number of points, such as 20; and severe is greater than the second predetermined number of points). Metric 420 may include the determined B-score and the determined C-score. Regarding the C-score, metric 420 may display the C-score for each of multiple partitions. For example, metric 420 may include the C-scores for the medial tibiofemoral (MT) partition, the lateral tibiofemoral (LT) partition, the medial patellofemoral (MP) partition, and / or the lateral patellofemoral (LP) partition. Gradient plots 417 and / or 419 may include B-score gradient bars or scales 417 and C-score gradient bars or scales 419. The B-score gradient bar 417 may resemble a reference... Figure 16A The scale 1602 is described to visually depict the severity or value of the determined B-rating (e.g., using color and / or grayscale and indicators). Similar to the B-rating gradient bar 417, the C-rating gradient bar or scale 419 may resemble the reference... Figure 6A The scale 608 is described to visually depict the determined C-score and / or predicted severity or value of cartilage loss in a manner that is, for example, using color and / or grayscale and indicators. The C-score gradient bar 419 may refer to the total C-score and / or the C-score of individual anatomical regions (e.g., medial region).

[0124] The fifth GUI 257 may also display one or more of the acquired images 302, and may further display the implant 408 within the acquired images 302. For example, the fifth GUI 257 may display side views and / or lateral views (e.g., left and right sides), front views and / or rear views, top views and / or bottom views of the patient's anatomy, with and without the implant 408. The implant 408 may be a predicted implant model or simulation superimposed on the acquired images 302, or the acquired images 302 may be postoperative images showing the implant 408 installed. Figure 4I and Figure 4JIn addition to or as a supplement to metric 414, the fifth GUI 257 may display classification scores 422 determined by one or more algorithms 90 for classifying or describing patient conditions. Joint space width, cartilage loss, and / or C-score GUI

[0125] refer to Figure 2 and Figure 5 The image analysis system 10 can determine the joint space width and related parameters between two or more bones of a joint, and determine or generate one or more GUIs 250 to display the joint space width and related parameters.

[0126] Joint space width (JSW) can be the distance between two or more bones at a joint. Joint space width algorithm 50 can be configured to determine one or more JSW parameters based on images in image data 1010. JSW parameters can be associated with the joint space width of one or more target joints. One or more JSW parameters can include joint space width at a predetermined location, joint space width across different directions (e.g., medial JSW or lateral JSW), average or mean joint space width (e.g., mean three-dimensional or 3D joint space width), varying joint space (e.g., joint space narrowing), average or mean joint space narrowing (e.g., mean 3D joint space narrowing), impact data, impact angle, impact data based on a predicted or determined implant, etc. Joint space width algorithm 50 can detect and / or reference multiple (e.g., hundreds) bony landmarks to determine the joint space width at each location.

[0127] Joint space width algorithm 50 can evaluate one or more of these JSW parameters across various anatomical regions (e.g., anterolateral, anteromedial, lateral medial, medial medial, posterolateral, posteromedial, or for the knee joint, medial tibiofemoral (MT), lateral tibiofemoral (LT), medial patellofemoral (MP), and / or lateral patellofemoral (LP)) of one or more bones (e.g., tibia and femur). For example, joint space width algorithm 60 can determine four JSW parameters in the knee joint (e.g., joint space width of four regions). Joint space width algorithm 50 can also be configured to predict joint space based on weight-bearing and / or non-weight-bearing conditions using other preoperative data 1000 (e.g., kinematic data or activity level data). For example, Figure 5 The joint space widths of the medial tibiofemoral (MT), lateral tibiofemoral (LT), medial patellofemoral (MP), and lateral patellofemoral (LP) regions are shown in GUI 258. Figures 6A to 6B The measurements for each of the MT, LT, MP, and LP partitions are shown.

[0128] The joint space width algorithm 50 can determine whether the joint space width has decreased or narrowed (and / or increased or widened) based on supplemental patient data 1030, a comparison of previously measured joint space widths, and / or a comparison of image data from previously acquired images. The joint space width algorithm 50 can also determine, estimate, or predict one or more cartilage parameters, such as cartilage thickness or the probability of cartilage loss during the procedure (e.g., using a Z-score or other statistical measure). The determined cartilage parameter can be based on the determined joint space width or other JSW parameters determined by the joint space width algorithm 50. Prediction of cartilage loss can be performed on a per-partition or per-bone basis.

[0129] For example, the joint space width algorithm 50 can determine the mean three-dimensional joint space narrowing (3DJSN) of the medial and lateral sections of a bone (such as the tibia and / or femur). The joint space width algorithm 50 can determine the mean 3D joint space width (3DJSW) at the center of each section. For each section, the joint space width algorithm 50 can compare the parameters with those of a healthy patient with similar characteristics to the current patient, and the image analysis system 10 can use the determinations from the joint space width algorithm 50, along with other preoperative data 1000 or determinations from one or more other algorithms 90, to determine the disease state or other outputs 2000.

[0130] Image analysis system 10 can use JSW parameters determined by joint space width algorithm 50 to determine, estimate, or predict cartilage loss (e.g., the amount or probability of cartilage loss). Joint space width algorithm 50 can also be used to determine scores or values ​​for multiple (e.g., four) anatomical regions (e.g., the knee joint) based on joint space width or cartilage loss, and to determine a composite score or C-score based on the determined score for each region. The scores and / or C-scores for each region can also be based in part on patient data 1020, such as sex, since men and women typically have different cartilage widths. Joint space width algorithm 50 can alternatively be referred to as C-score algorithm 50. C-scores can be correlated with or proportional to the predicted cartilage loss, such that a higher C-score may indicate a higher probability of cartilage loss and / or a higher predicted severity or amount of cartilage loss.

[0131] The joint space width algorithm 50 can identify or select, among multiple partitions, the partitions that should be resurfaced during the procedure, and determine that the procedure plan 2020 should include one or more steps of surface replacement for the selected partitions. The joint space width algorithm 50 can determine cartilage thickness or loss based on a determined C-score and can take into account patient data 1020 (e.g., sex). The joint space width algorithm 50 can convert the joint space width (e.g., in mm) into a Z-score or other score. A Z-score can describe the relationship between a particular value (e.g., joint space width) and the mean or average of a set of values. For example, a Z-score can be measured by the standard deviation from the mean, such that a Z-score of 0 can indicate the same value as the mean score. In some examples, the joint space width algorithm 50 can determine patient data 1020 (e.g., sex) based on determined JSW parameters (e.g., C-score or Z-score). In some examples, the joint space width algorithm 50 can determine whether the procedure plan 2020 should include total arthroplasty or partial arthroplasty (e.g., total knee arthroplasty or partial knee arthroplasty).

[0132] Based on the determined JSW parameters, joint space width algorithm 50 and / or one or more algorithms 90 can be used together to determine one or more of the outputs 2000. In some examples, joint space width algorithm 50 can determine and / or predict (or be used to determine and / or predict) the program time or duration 2010 for executing program plan 2020. For example, joint space width algorithm 50 can determine that the patient's joint space width is outside a predetermined range, narrows over time and / or is less than a first predetermined threshold, or widens over time and / or is greater than a second predetermined threshold. Image analysis system 10 can at least in part predict longer or shorter program times 2010, recommended implants to be used in program plan 2020, predicted outcomes such as cartilage loss, etc., 2080, and patient anatomical representations 2090 based on these determinations of JSW algorithm 50. Other factors (e.g., from patient data 1020) can alter the analysis and / or relationships, allowing the image analysis system 10 and / or the osteophyte joint space width algorithm 50 to determine certain relationships when higher or lower JSW parameters are combined with certain patient data 1020. Additionally, the image analysis system 10 can learn other relationships between JSW parameters and predicted outcomes 2080 other than cartilage loss, for example, by analyzing previous JSW parameters from previous procedural data 1050.

[0133] GUI 250 may include a sixth GUI 258 and a seventh GUI 260, which may display JSW parameters determined by the joint gap width algorithm 50 that are related to the artificial model 402 (as in the sixth GUI 258) and / or related to the acquired image 302 (as in the seventh GUI 260).

[0134] The sixth GUI 258 can display one or more views of the artificial model 402 of a joint (e.g., a knee joint) in a manner that shows the gap between one or more bones. The sixth GUI 258 can depict the joint gap width determined by the joint gap width algorithm 50, which is achieved using JSW lines, arrows, or other symbols 502, 504 extending between one or more bones within the joint gap width. The JSW lines 502, 504 can be color-coded according to, for example, the partitions or sides of the bones or the directions they involve.

[0135] like Figure 5 As illustrated, the sixth GUI 258 can display an artificial model 402 of the knee joint, which may include one or more views of the patellofemoral and tibiofemoral joints. Like the other GUIs 250, each of these displayed joints (e.g., the patellofemoral and tibiofemoral joints) can be manipulated via user input to rotate, spin, or rotate about various axes, thereby changing the view. The sixth GUI 258 may include a first set of JSW lines 502 indicating the joint space width at a first side (e.g., lateral) of each of the patellofemoral and tibiofemoral joints, and may further include a second set of JSW lines 504 indicating the joint space width at a second side (e.g., medial) of each of the patellofemoral and tibiofemoral joints. The sixth GUI 258 may include a joint name or label 506 for each joint (patellofemoral and tibiofemoral joints), and side or partition labels 508 indicating which side or partition the JSW lines 502, 504 correspond to. Figure 5 As illustrated, the side label 508 can indicate which side is the outer side and which side is the inner side. Although combined Figure 5 The examples use outer and inner labels, but front and back labels can be used alternatively. The aspects disclosed herein are not limited to the information provided in labels 506 and 508. For clarity, the outer JSW line 502 may be displayed in a different color than the inner JSW line 504.

[0136] The seventh GUI 260 can display information similar to that of the sixth GUI 258, but as an alternative to or supplement to the artificial model 402, JSW lines 502, 504 can be overlaid on the acquired image 302. As illustrated in the seventh GUI 260, the seventh GUI 260 can display the acquired image 302 of the knee joint, including the femur and tibia of the current patient. The seventh GUI 260 can overlay JSW lines 502, 504. JSW lines 502 in one region or partition (e.g., lateral) can be presented in a different color than JSW lines 504 in another region or partition (e.g., medial). Additionally, the density of JSW lines 504 can be proportional to the determined joint space width. The seventh GUI 260 can also display a view of the corresponding artificial model 402 generated from the acquired image 302 on the same screen or a separate screen. Figure 5 As illustrated, the seventh GUI 260 may show a top view of a portion of the artificial model (e.g., the tibia) and color-coded lateral JSW lines 502 and medial JSW lines 504, but the aspects disclosed herein are not limited thereto. The artificial model 402 may be capable of spinning, rotating, and so on.

[0137] Although not shown, the sixth GUI 258 and the seventh GUI 260 may include widgets, tables, charts, or other information that can indicate (e.g., numerically) the JSW parameters determined by the joint space width algorithm 50, such as C-scores or Z-scores. The sixth GUI 258 and the seventh GUI 260 may indicate accurate or immediate parameters (e.g., parameters of the actual bone geometry of the current patient) and / or may indicate predicted parameters or recovery status (e.g., joint space width after implantation or in the later stages of recovery). The sixth GUI 258 and the seventh GUI 260 may be used preoperatively, intraoperatively, or postoperatively. Alternatively or additionally, the sixth GUI 258 and the seventh GUI 260 may be implemented as a reference. Figures 4A to 4E The components described are 406, 409 and / or 410.

[0138] refer to Figure 2 as well as Figures 6A to 6C The GUI may include an eighth GUI 262, which may indicate the predicted outcome 2080 related to the joint space width. Figure 6A In the example shown, the eighth GUI 262 indicates the predicted cartilage loss determined by the joint space width algorithm 50. For ease of description, the predicted cartilage loss will be described as an exemplary parameter, but the aspects disclosed herein are not limited thereto, as the image analysis system 10 may determine new or different outputs 2000 and / or predicted outcomes 2080 based on the joint space width.

[0139] The eighth GUI 262 may display a view (e.g., a top view) of an artificial model 402 of two or more bones of a joint (e.g., the tibia and femur). One or more values ​​602, 604 determined by the joint space width algorithm 50 may be superimposed on the top view of the artificial model 402. One or more values ​​602, 604 may include a first value 602 corresponding to a first partition or side (e.g., medial) and a second value 604 corresponding to a second partition or side (e.g., lateral). These values ​​602, 604 may indicate joint space width (e.g., mm), a score (e.g., C-score or Z-score), or a number or score corresponding to a predicted amount of cartilage loss, or a prediction or percentage of cartilage loss that will occur. The artificial model 402 may be colored in a manner corresponding to the values ​​602, 604.

[0140] The eighth GUI 262 may include a cartilage loss display 606 corresponding to each value 602, 604 in each of the displayed artificial models 402 of the joint. The cartilage loss display 606 may include a scale or axis 608. The scale 608 may be a gradient bar that is color-coded such that: numbers or values ​​indicating healthy cartilage loss (or, as another example, a low probability of cartilage loss) are presented in green, values ​​indicating extensive, severe, or unhealthy cartilage loss (or, as another example, a high probability of cartilage loss) are presented in red, and intermediate values ​​are presented in yellow or orange. The scale or axis 608 may have periodic numerical indicators. The cartilage loss display 606 may include indicators (e.g., lines) 610 presented on the scale 608 at locations corresponding to values ​​602, 604. The cartilage loss display 606 may display predicted cartilage loss for each section of the bone (e.g., four sections). The cartilage loss display 606 may include partition labels 610 indicating the partitions or locations (e.g., medial patellofemoral, lateral patellofemoral, medial tibiofemoral, or lateral tibiofemoral) corresponding to values ​​602, 604. The cartilage loss display 606 may include parameter labels 614 indicating the displayed parameters (e.g., high probability of cartilage loss), and may include a legend 616 indicating the importance of colors or numbers presented in the scale 608.

[0141] refer to Figure 6B The eighth GUI 262 can be implemented as one of the cards or widgets 409 displayed on the third GUI 255. For example, the plurality of widgets 409 may include references Figure 4B The description refers to cartilage loss or C-score card 416. Users can click on cartilage loss card 416 to display the eighth GUI 262. Alternatively or otherwise, clicking on C-score card 416 can bring up a simplified version of the eighth GUI 262, which displays some (but perhaps not all) features of the eighth GUI 262. Figure 6BExamples are shown of the eighth GUI 262 as a pop-up, an enlarged frame of the eighth GUI 262 and / or the C-score card 416, or a separate frame of the third GUI 255, but the aspects disclosed herein are not limited thereto. For example, clicking the cartilage loss card 416 can bring up the eighth GUI 262 as a full-screen GUI, as... Figure 6A As shown. Reference Figure 6C The eighth GUI 262 can be rotated or flipped upon clicking to display the “back” of the cartilage loss card 416, which may display text information, video, etc., or other additional information or analysis (e.g., determination by one or more algorithms 90) about cartilage loss, C-score, joint space width, etc., and / or gradient bars 608 or other display features related to C-score and / or joint space width as described herein.

[0142] Figure 35 Another example of a GUI 3600 displaying potential cartilage loss or C-score as described above is shown. In this example, on the left side of the GUI 3600, the femur 3602 and tibia 3604 are shown with C-scores superimposed on corresponding locations 3606, 3608, 3610, and 3612 on the femur 3602 and tibia 3604, respectively, to indicate cartilage loss in a specific area for a particular surgical plan. Each location corresponds to a partition, such as medial patellofemoral partition 3606, lateral patellofemoral partition 3608, medial tibiofemoral partition 3610, and lateral tibiofemoral partition 3612. The GUI 3600 indicates the predicted cartilage loss, which can be determined by the joint space width algorithm 50. The predicted cartilage loss and C-score may include visual indicators for indicating the likelihood of cartilage loss. For example, the left side (page-facing) of GUI 3600 includes C-scores at each location 3606, 3608, 3610, 3612; however, other visual indicators are envisioned. In some examples, in addition to the numerical C-score, each zone 3606, 3608, 3610, 3612 may include visual indicators of the likelihood of cartilage loss, such as color gradients or patterns. In one example, the visual indicator could be multiple colors, such as green, yellow, and red, where green indicates the lowest likelihood of cartilage loss and red indicates the highest likelihood. Further, it is envisioned that additional information related to alignment and cartilage loss, as determined by the joint space width algorithm 50, could be displayed on GUI 3600 along with the C-scores. In some examples, GUI 3600 may have multiple displays for different zones of a single bone 3602, 3604, which could be displayed within a single electronic screen, multiple electronic displays, and / or other display methods known in the art, such as virtual reality displays or other display devices.

[0143] The GUI 3600 may include multiple displays and images for each area of ​​a single skeleton 3602, 3604. In some examples, the GUI 3600 may display 2D images of defined view planes or sections of skeletons 3602, 3604. In another example, the GUI 3600 may display a 3D model that can be repositioned by the user to obtain a preferred view.

[0144] For example Figure 36 In some of the examples shown, additional information related to the C-score and the likelihood of cartilage loss can be displayed. The right side (page-facing) of the GUI 3600 displays indicators 3614, 3616, 3618, and 3620, respectively, corresponding to each of the medial patellofemoral region 3606, lateral patellofemoral region 3608, medial tibiofemoral region 3610, and lateral tibiofemoral region 3612. Figure 36 In the example shown, indicator bar 3614 for the medial patellofemoral region 3606 indicates possible cartilage loss, as indicated by the score of region 3606 on the femur 3602, while indicator bar 3616 for the lateral patellofemoral region 3608 has a higher score and is indicated as having severe cartilage loss. Similarly, on the tibia 3604, both the medial tibiofemoral region 3610 and the lateral tibiofemoral region 3612 have high cartilage loss scores, as indicated by indicators 3618 and 3620. In this example, indicators 3614, 3616, 3618, and 3620 (e.g., using color and / or grayscale and indicators) are displayed as gradient bars with discrete locations corresponding to a series of potential cartilage losses to depict the determined C-score and / or predicted severity or value of cartilage loss. Other graphical indicators are envisioned in other examples. Osteophyte GUI

[0145] refer to Figure 2 and Figure 7 Osteophytes can be bone spurs that form on bones. Osteophyte volume can refer to the total volume of osteophytes on a bone or a specific portion of a bone. Osteophyte detection algorithm 60 can be configured to detect or identify one or more osteophytes located at a target bone, joint, or portion of a bone in the acquired images of image data 1010, and to determine or calculate one or more osteophyte parameters based on preoperative data 1000 (including the acquired images in image data 1010). Osteophyte parameters can be related to the size or geometry (e.g., location, volume, thickness, occupied area, or partition) of the osteophyte detected or one or more detected osteophytes in one or more target joints. For example, osteophyte thickness can be a cross-sectional distance perpendicular to the bone surface.

[0146] One or more osteophyte parameters may include osteophyte location, number of osteophytes, volume of osteophytes at a predetermined location, area of ​​osteophytes across different directions (e.g., medial or lateral), average or mean osteophyte volume, volume of osteophytes changing or progressing, impingement data, impingement angle, impingement data based on predicted or determined implants, etc. For example, osteophyte detection algorithm 60 may determine an osteophyte volume, value, or parameter for each relevant bone (e.g., three in the knee joint). Osteophyte detection algorithm 60 may evaluate one or more of these osteophyte parameters on various anatomical partitions (e.g., anterolateral, anteromedial, central lateral, central medial, posterolateral, posteromedial, medial tibiofemoral (MT), lateral tibiofemoral (LT), medial patellofemoral (MP), and / or lateral patellofemoral (LP)) of one or more bones (e.g., femur or tibia) and / or one or more bones (e.g., tibia and femur). Osteophyte detection algorithm 60 may also be configured to predict osteophyte volume or progression based on other preoperative data 1000 (e.g., kinematic data or activity level data).

[0147] The osteophyte detection algorithm 60 can determine whether the osteophyte volume (e.g., total osteophyte volume, or osteophyte volume in a specific area or osteophyte) is increasing or decreasing based on supplemental patient data 1030, a comparison of previously measured osteophyte volumes, and / or a comparison of image data from previously acquired images. The osteophyte detection algorithm 60 can further determine, predict, or diagnose disease states or disease progression (e.g., osteoarthritis or OA) based on the determined osteophyte parameters.

[0148] Based on the determined osteophyte parameters, osteophyte detection algorithm 60 and / or one or more algorithms 90 can be used together to determine one or more of the outputs 2000. For example, osteophyte detection algorithm 60 can determine that the patient's osteophyte volume is progressing over time and / or exceeds a predetermined threshold, and thereby predict a certain (e.g., a longer) procedure time 2010, certain steps in the procedure plan 2020, etc. Additionally, osteophyte detection algorithm 60 can determine a predicted outcome 2080 (e.g., cartilage loss) and a patient anatomical representation 2090 that includes detected osteophytes or otherwise indicates osteophyte parameters.

[0149] GUI 250 may include a ninth GUI 264, which includes multiple screens 702, 704, 706 that display the osteophyte detection process performed by osteophyte detection algorithm 60, allowing users (e.g., practitioners) to supervise the detection.

[0150] The multiple screens 702, 704, and 706 may include one or more first screens 702, which may display at least the acquired image 302 of the target skeleton and / or the artificial model 402, as well as the outer boundary of the target skeleton. The one or more first screens 702 may include, for example, four screens displayed simultaneously or on different screens.

[0151] like Figure 7 As illustrated, the first screen 702 can display knee joints (e.g., including the tibia, femur, and / or patella) shown in multiple acquired images 302, presented in various views (top view, side view, perspective view, enlarged / magnified view, etc.). The osteophyte detection algorithm 60 can determine the outer boundary 708 of a target bone, such as the outer cortical bone of the target bone (e.g., the femur). The first screen 702 can depict the determined outer boundary, which can be in a bright color (e.g., green) to be visible on the acquired images 302, which may be presented in black and white or grayscale. The first screen 702 can also depict an artificial model 402 of the bone visually indicating the outer boundary 708, which can also be presented in the same color as in the acquired images 302.

[0152] The second screen 704 can display the same acquired image 302 and artificial model 402 as the first screen 702, except that in addition to continuing to display the outer boundary 708, it can further display the osteophyte-free boundary or surface 710 determined by the osteophyte detection algorithm 60. The osteophyte-free boundary 710 can be displayed in a different color than the outer boundary 708 (e.g., yellow).

[0153] The third screen 706 can display detected osteophytes 712, which can be determined as a function of the osteophyte-free boundary 710 and the outer boundary 708 (e.g., by subtraction or determining the difference). The osteophytes 712 can be displayed on the same acquired image 302 and artificial model 402 as the first screen 702 and the second screen 704, but may not necessarily display the outer boundary 708 and the osteophyte-free boundary 710. The osteophytes 712 can be presented in a different color (e.g., red) than the outer boundary 708 and the osteophyte-free boundary 710.

[0154] Figure 8 The segmentation process that the osteophyte detection algorithm 60 can use and / or execute is demonstrated. Alternatively, some segmentation steps can be performed manually by the practitioner (e.g., by interacting with the ninth GUI 264).

[0155] refer to Figure 2 and Figure 8The ninth GUI 264 may include a fourth screen 703 that displays features (e.g., bony landmarks) identified or detected by the osteophyte detection algorithm 60 in preparation for segmentation. The fourth screen 703 may include arrows or other indicators 802 superimposed on the acquired image 302 and indicating bony landmarks and / or other locations that may mark boundary positions. The image analysis system 10 may determine the location of these indicators 802 via the osteophyte detection algorithm 60, and / or practitioners may interact with and / or engage with the fourth screen 703 and / or the ninth GUI 264 to manually input (e.g., via a touchscreen and stylus, keyboard and mouse, or other input devices) the indicators 802.

[0156] Similar to the first screen 702, the osteophyte detection algorithm 60 can determine one or more outer boundaries 708 of one or more target bones based on indicators 802. The osteophyte detection algorithm 60 can, for example, use statistical modeling, machine learning, automatic segmentation techniques, etc. The osteophyte detection algorithm 60 can be a machine learning or artificial intelligence model trained on manually segmented images that do not include osteophytes. Therefore, the osteophyte detection algorithm 60 can be referred to as an osteophyte-free model. The osteophyte detection algorithm 60 may have learned image features characterizing osteophytes to distinguish between osteophyte-bone and non-osteophyte-bone structures, thereby identifying one or more osteophyte-free bone surfaces in an image (e.g., a CT image). The osteophyte detection algorithm 60 can be configured to automatically segment osteophyte-bone surfaces (e.g., cortical bone including any osteophytes) and automatically segment osteophyte-free bone surfaces (which may include the same cortical bone but not osteophytes). When comparing these two automatically segmented surfaces, the osteophyte-bone surface may overlap with or be larger than the osteophyte-free bone surface.

[0157] The fourth screen 703 can depict the determined outer boundary 708, which can be in a bright color (e.g., yellow) so that it is visible on the acquired image 302, which may be presented in black and white or grayscale. The fourth screen 703 can supplement or replace the first screen 702. As the osteophyte detection algorithm 60 progressively performs a segmentation method to display the osteophyte 712 (e.g., by subtracting the osteophyte-free boundary 710 from the outer boundary 708), the second screen 704 and the third screen 706 can subsequently be displayed.

[0158] The first screen 702 and / or the fourth screen 703, the second screen 704, and the third screen 706 can be repeated for various views of the target skeleton and / or various legs. For example, as Figure 9 As shown, a third screen 706' can be displayed for the right knee joint, and a third screen 706'' can be displayed for the left knee joint.

[0159] Figure 10An exemplary method or algorithm 1001 is shown that can be executed by an image analysis system 10 (e.g., via an osteophyte detection algorithm 60) to determine the volume of an osteophyte in a target bone. Reference Figure 7 and Figure 10 Method 1001 may include step 1002 of segmenting the osteophyte-free surface or boundary from the intact surface (or outer boundary) of a received or acquired image using a trained machine learning model. This segmentation step 1002 may be visualized in a second screen 704, which may show the outer boundary 708 (or intact surface) in a different color than the osteophyte-free boundary 710. Segmentation step 1002 may be performed by comparing the acquired image with multiple previously acquired images of a previous patient and / or with a previous model generated for a previous patient. The previous patient may be an example of a health condition or a patient. Segmentation step 1002 may be performed using an Active Appearance Model (AAM) and / or other image processing techniques configured for accurate cartilage and bone segmentation using large datasets. Segmentation step 1002 may include a second level of refinement using machine learning leveraging convolutional neural networks. Segmentation step 1002 may be performed using a machine learning model trained on manual segmentation of a separate set of many (e.g., more than 1,000) preoperative acquired images (e.g., CT images) of a target bone or joint (e.g., the knee joint). AAM can inherently contain a dense set of anatomically corresponding landmarks that can be used to consistently align surfaces in 3D, thereby correcting pose (translation and rotation), size, and shape to create spatially consistent osteophyte areas.

[0160] In step 1002, against the backdrop of the knee joint, the osteophyte-free surfaces of the femur and tibia can be segmented using a separate osteophyte-free AAM. The original "pre-disease" surfaces can be viewed in CT images (e.g., as shown in...). Figure 8 The osteophyte-free surface can be manually segmented by visual interpolation of the osteophyte-free skeletal surface at certain points (e.g., as indicated by indicator 802 on screen 703). The osteophyte-free model used may have been trained on many knees (e.g., more than 100 or between 100 and 150) selected to include a wide range of knee OA pathologies, and the manual segmentation is supervised by practitioners with many years (e.g., 10 to 15 years) of relevant segmentation experience.

[0161] Method 1001 may include steps 1004 of determining or calculating the volume of the segmented osteophyte-free surface and 1006 of determining the volume of the segmented intact surface. Steps 1004 and 1006 may be determined or approximated, for example, based on computational regions within the outer boundary 708 (for step 1006) and the osteophyte-free boundary 710 (for step 1004) of multiple different acquired images for different views of the target bone, but the aspects disclosed herein are not limited thereto. Steps 1004 and 1006 may be determined based on previous volumes from previously acquired images, and the osteophyte detection algorithm 60 may refine its determination in steps 1004 and 1006 to improve accuracy. Method 1001 may include step 1008 of determining the original volume by subtracting the determined volume of the osteophyte-free surface from the determined volume of the intact surface. Method 1001 may include step 1012 of normalizing the original volume to take into account the size of the patient's anatomy. The size of the patient's anatomical structures can be a single input 1000 (e.g., patient data 1020) and / or can be inferred from other acquired images or models.

[0162] refer to Figures 11A to 11E One or more GUIs 250 may include a tenth GUI 266, which displays a three-dimensional model of a target bone or joint (including osteophytes) divided into different anatomical partitions. An osteophyte detection algorithm 60 can detect osteophytes relative to these different anatomical partitions. Figures 11A to 11C Various views of a representative model of the femur 1100 are shown, and Figure 11D and Figure 11E Various views of a representative model of the tibia 1150 are shown. Although all views... Figures 11A to 11E They are presented on the same screen, but each view can be expanded and / or occupy a different screen.

[0163] Figure 11A A bottom view of a representative model (or “femoral model”) of the femur 1100 is shown. The femoral model 1100 can be divided into anterior lateral section 1102, anterior medial section 1104, central lateral section 1106, central medial section 1108, posterolateral section 1110, and posteromedial section 1112. Figure 11A In this design, the anterolateral partition 1102, anteromedial partition 1104, central lateral partition 1106, central medial partition 1108, posterolateral partition 1110, and posteromedial partition 1112 can be divided by color-coded lines 1114, which can be dashed or dotted lines. In some examples, labels or text 1116 identifying these partitions can be presented. In some examples, the color-coded lines 1114 can visually indicate (e.g., via color or density) the defined joint space width of each partition (e.g., determined by joint space width algorithm 50).

[0164] Figure 11B The medial side of the femoral model 1100 is shown. Figure 11B In the middle, the front inner partition 1104, the central inner partition 1108, and the rear inner partition 1112 can be color-coded with different colors for visibility. Figure 11C The lateral aspect of the femoral model 1100 is shown. Figure 11C In the middle, the front outer partition 1102, the central outer partition 1106, and the rear outer partition 1110 can be color-coded with different colors for visibility.

[0165] Figure 11D A top view of a representative model (or “tibial model”) of the tibia 1150 is shown. The tibial model 1150 can be divided into posterolateral section 1152, posteromedial section 1154, anteromedial section 1156, and anteromedial section 1158. Figure 11D In the diagram, the rear outer partition 1152, rear inner partition 1154, front inner partition 1156, and front inner partition 1158 can be divided by color-coded lines 1160, which can be dashed or dotted lines. In some examples, labels or text 1162 identifying these partitions can be displayed. Figure 11E A top view of a tibial model 1150 is shown, in which the posterolateral region 1152, posteromedial region 1154, anteromedial region 1156, and anteromedial region 1158 are color-coded instead of using color-coded lines 1160. Figure 11E In the middle, the medial and lateral menisci can be stained, while the central lateral surface or recess 1164 and the central medial surface or recess 1164 configured to support the articular cartilage can remain neutral in color or bone color.

[0166] refer to Figure 2 and Figure 12 The osteophyte detection algorithm 60 and / or image analysis system 10 can execute methods or algorithms 1200 to determine different anatomical regions (e.g., Figures 11A to 11EThe location and / or volume of osteophytes in the anatomical divisions shown for the knee joint. Method 1200 may include step 1202 of separating the identified surfaces in the acquired images of the target bone or joint into two or more partitions (e.g., medial and lateral partitions, or anterior and posterior partitions). Step 1202 may include comparing the acquired images with multiple images or models of the target bone or joint from previous or healthy patients. Step 1202 may also be performed using the semi-quantitative MRI osteoarthritis knee scoring (MOAKS) scoring system. In the context of the knee joint, in step 1202, the femur may be divided into three partitions—anterior, central, and posterior—using a bone and meniscus model of a healthy individual by referencing the anterior and posterior meniscus margins. The tibia may be separated into anterior and posterior partitions of equal size. These partitions may be displayed (e.g., displayed on GUI 266 as shown in Figure 11). When segmenting these partitions, the separators or dividing lines between the partitions may be derived from AAM segmentation (e.g., from... Figure 10 The anatomically corresponding landmarks are aligned using the AAM segmentation performed in method 1001.

[0167] Method 1200 may include step 1204 of determining the original partition volume for each anatomical partition. The original partition volume may be based on an original volume previously determined according to method 1001 and / or may be calculated using a segmentation process. The original partition volume of a partition may be the volume of all osteophytes in that partition.

[0168] Method 1200 may include a step 1206 of normalizing each raw partition volume to account for the size of the patient's anatomy (e.g., bone size). The size of the patient's anatomy may be a separate input 1000 (e.g., patient data 1020) and / or may be inferred from other acquired images or models. For example, in the context of a knee joint, the partition volume may be normalized for bone size by multiplying the raw value by a ratio: R b = Test bone volume / mean volume of all bones, using the distal femur volume and proximal tibia volume with no osteophyte surfaces.

[0169] refer to Figure 13One or more GUIs 250 may include an eleventh GUI 268 configured to display a model of osteophytes color-coded based on partitions (e.g., medial and lateral). For example, the eleventh GUI 268 may include a first screen 1302 for a first target bone (e.g., femur) of a joint and a second screen 1303 for a second target bone (e.g., tibia) of the joint. Each of the first screen 1302 and the second screen 1303 may respectively include one or more acquired images 302 of one or more views of the first or second target bone, showing one or more boundary lines 1304. The boundary lines 1304 may be determined using a segmentation process and represent outer boundaries and osteophyte-free boundaries. The boundary lines 1304 may be color-coded according to anatomical partitions (e.g., medial or lateral, or anterior or posterior). Each of the first screen 1302 and the second screen 1303 may include a representative model 1306 of an osteophyte, which is separated from the osteophyte-free surface or the underlying bone and is color-coded according to anatomical divisions (e.g., medial or lateral, or anterior or posterior).

[0170] The osteophyte detection algorithm 60 can determine features within the bone and, based on the detected features within the bone, determine osteophytes and other relevant parameters outside the bone. For example, the osteophyte detection algorithm 60 can detect transmissible regions in the original image and / or image data 1010 and determine osteophytes outside the bone based on the detected regions. B-Rating GUI

[0171] refer to Figure 14A B-score can be a rating or scoring system based on the shape of the femur or knee joint and / or a quantification of that shape. A B-score can be a composite score, an average score, or a holistic score that indicates an overall assessment of the femur and / or knee joint; however, knees with different specific complications or deformities may result in similar B-scores. A B-score can be based on a comparison of the shape of the femur with the knee joint shape of a person with OA and a person without OA, and can be determined using, for example, statistical shape modeling (SSM) or other processes. A B-score can be a continuous quantitative variable that can be used to quantify the total number of knee OA injuries and to measure progression in longitudinal studies. In other examples, such as for procedures related to any anatomical joint, similar measurement and scoring systems (based on a quantification and / or other measurement of the shape of one or more bones of the joint) are similar to B-scores. Some procedures that can utilize similar systems (e.g., those displaying associated B-scores or other metrics to quantify osteophyte volume, ligament laxity, range of motion, joint health, or other parameters described herein) are hip procedures, ankle procedures, spine procedures, shoulder procedures, elbow procedures, hand procedures, etc. Any measurement and scoring system described in this application, except for knee procedures, can also be used in other surgical procedures, such as other orthopedic surgeries or any other surgical procedure.

[0172] As osteophytes (OA) progress, each bone may exhibit characteristic shape changes, involving osteophyte growth around the cartilaginous plate, as well as the spread and flattening of the subchondral bone. Femoral shape changes may increase regardless of the affected anatomical region and may be more sensitive to changes than those in the tibia and patella. The B-score can represent the distance along the shape changes of the "OA" in the femur. The B-score can be correlated with the total osteophyte volume.

[0173] In some examples, the B-score can be recorded as a z-score, similar to the T-score in osteoporosis, which can represent the standard deviation (SD) of a healthy population in units, where 0 is defined as the mean of the healthy population. Values ​​from -2 to +2 can represent the healthy population, while values ​​above +2 may be outside the range of healthy individuals.

[0174] The B-score algorithm 70 can be configured to determine a B-score based on the acquired image 302. The B-score can be based in part on or related to OA progression, wherein a B-score of 0 can be correlated with and / or indicate the mean femoral shape of people without OA. Further details on how to calculate the B-score can be found in “Machine-learning, MRI bone shape and important clinical outcomes in osteoarthritis: data from the Osteoarthritis Initiative”, published November 13, 2020 by Michael A. Bowes, Katherine Kacena, Oras A. Alabas, Alan D. Brett, Bright Dube, Neil Bodick, and Philip G. Conaghan, and “3D Bone Shape from CT-Scans Provides an Objective Measure of Osteoarthritis Severity: data from the IMI-APPROACH study”, published December 2024 by James M. Burlison, Michael A. Bowes, Philip G. Conaghan, and Alan D. Brett. [3D Bone Shapes from CT Scans Provide an Objective Measurement of Osteoarthritis Severity: Data from the IMI-APPROACH Study], both of which are incorporated herein by reference in their entirety. However, the aspects disclosed herein are not limited to this B-score. For example, the B-score algorithm 70 may additionally and / or alternatively calculate other scores or quantifications of other bone shapes based on comparisons of other bone shapes with those of individuals with a specific disease.

[0175] B-score algorithm 70 can be configured to detect or identify one or more target bones or joints (e.g., femur), detect or identify the shape of the target bone or joint, and / or determine or calculate one or more shape scoring parameters based on preoperative data 1000 (e.g., image data 1010) associated with the shape of the target bone and / or a comparison of that shape with previous patients with a specific disease. For ease of description, an example of B-score algorithm 70 calculating one or more B-score parameters associated with the knee joint and / or femur will be described. One or more B-score parameters may include B-scores at different times or on different images, average or mean B-scores, and / or B-scores in variation or progression. B-score algorithm 70 can also be configured to predict future B-scores or B-score progression based on other preoperative data 1000 (e.g., kinematic data or activity level data).

[0176] The B-score algorithm 70 can determine whether the B-score of a specific femur (e.g., the left femur) or both femurs is increasing or decreasing based on supplemental patient data 1030, a comparison of previously measured B-scores, and / or a comparison of image data from previously acquired images. The B-score algorithm 70 can further determine, predict, or diagnose a disease state or disease progression (e.g., osteoarthritis or OA) based on the determined B-score and / or B-score progression.

[0177] like Figure 14 As shown, one or more GUIs 250 may include a twelfth GUI 270. The twelfth GUI 270 may include multiple screens or boxes displaying the patient's OA progression. Each screen may include a representative model 1402 of the patient's femur determined by the B-score algorithm 70 and / or the image analysis system 10, and text 1404 or another indicator indicating the B-score of that model. The multiple screens or boxes may be implemented as video progression, such that the twelfth GUI 270 displays the OA progression in video format. Some boxes in later frames may show, for example, predicted progression when the patient is not receiving treatment, or alternatively, predicted improvement when the patient receives treatment (e.g., program plan 2020).

[0178] refer to Figure 15Based on the determined B-score, B-score algorithm 70 and / or one or more algorithms 90 can be used together to determine one or more of the outputs 2000, such as a predicted outcome 2080 that can be displayed on the thirteenth GUI 272. For example, B-score algorithm 70 can calculate the patient's current B-score and display the current B-score as text or another indicator 1502 on the thirteenth GUI 272. B-score algorithm 70 and / or the image analysis system can use the determined B-score and / or other parameters or inputs (e.g., acquired images 302 and / or stored models) to generate a representative model 1504 of the patient's femur, and can also generate a healthy femur showing a B-score of 0 and / or a comparative representative model 1506 showing predictions about how the patient's femur might look when the B-score is 0.

[0179] The B-score algorithm 70 can determine predictive outcomes 2080, such as the predicted level of perceived pain and / or the probability of severe or moderate pain if the patient continues without treatment (or alternatively, based on the improvement prediction of the predictive plan 2020), the predicted loss of function and / or the probability of severe or moderate loss of function if the patient continues without treatment (or alternatively, based on the improvement prediction of the predictive plan 2020), and the prediction and / or probability of needing total joint replacement surgery and / or total arthroplasty (e.g., total knee arthroplasty or TKA) at a predetermined future time (e.g., within the next 5 years). Such predictive outcomes 2080 can be described and / or explained in text section 1508 of the thirteenth GUI 272. Predictive outcomes 2080 can also be displayed in one or more graphs or plots 1510. For example, when predictive outcomes 2080 are expressed as a probability percentage (e.g., the probability of severe or moderate pain), these predictions can be plotted as a function of the B-score.

[0180] refer to Figure 16A One or more GUIs 250 may include a fourteenth GUI 274 configured to display a predicted outcome 2080. The fourteenth GUI 274 may include a scale 1602 or gradient bar having an indicator 1604 to indicate a B-score calculated by the B-score algorithm 70. The scale 1602 may resemble scale 608 in terms of using colors and / or shading corresponding to the severity of OA progress or the health level associated with the B-score. Figure 6A Similar to GUI 270 (). Figure 14The fourteenth GUI 274 may also include a B-score evolution video 1606 showing the patient’s actual and / or predicted OA progression or femoral shape. The B-score evolution video 1606 may include a representative model 1608 of the patient’s femur determined by the B-score algorithm 70 and user input 1610 for controlling the progression of the B-score evolution video 1606 (e.g., play, pause, fast forward, rewind, zoom, volume and / or timeline buttons).

[0181] The fourteenth GUI 274 may also include predictive outcomes 2080, such as a predicted level of perceived pain and / or a prediction of the likelihood of severe or moderate pain if the patient continues without treatment (or alternatively, based on the improved predictions of the predictive plan 2020), a predicted loss of function and / or a prediction of the likelihood of severe or moderate loss of function if the patient continues without treatment (or alternatively, based on the improved predictions of the predictive plan 2020), and a prediction and / or likelihood that total joint replacement surgery and / or total arthroplasty (e.g., total knee arthroplasty or TKA) will be required at a predetermined future time (e.g., within the next 5 years). For example, the fourteenth GUI 274 may display the probability 1612 of perceived pain (e.g., using a visual analog scale (out of 10), or other values ​​or scoring systems). The probability 1612 of perceived pain may be expressed as a percentage, such as a probability of moderate pain 1614 and a probability of severe pain 1616. The B-score algorithm 70 can calculate the probability 1614 of moderate pain based on the predicted perceived pain being greater than a first predetermined pain score (e.g., a VAS score of 4). The B-score algorithm 70 can calculate the probability 1616 of severe pain based on the predicted perceived pain being greater than a second predetermined pain score (e.g., a VAS score of 8). The probability 1614 of moderate pain can be displayed and / or outlined with a first color associated with moderate pain (e.g., yellow), and the probability 1616 of severe pain can be displayed and / or outlined with a second color associated with severe pain (e.g., red).

[0182] The fourteenth GUI 274 can display the probability 1618 of functional loss (e.g., using the Knee Impairment or Osteoarthritis Outcome Scale or KOOS, or the Western University and McMaster University Osteoarthritis Index or the WOMAC scale (out of 64)). The probability 1618 of functional loss can be expressed as a percentage, such as the probability 1620 of moderate functional loss and the probability 1622 of severe functional loss. The B-score algorithm 70 can calculate the probability 1620 of moderate functional loss based on the predicted perceived functional loss being greater than a first predetermined functional loss score (e.g., a WOMAC score of 20). The B-score algorithm 70 can calculate the probability 1622 of severe functional loss based on the predicted perceived pain being greater than a second predetermined functional loss score (e.g., a WOMAC score of 8). The probability 1618 of moderate functional loss can be displayed and / or outlined with a first color associated with moderate functional loss (e.g., yellow), and the probability 1622 of severe functional loss can be displayed and / or outlined with a second color associated with severe functional loss (e.g., red).

[0183] B-score algorithm 70 can determine and / or predict (or be used to determine and / or predict) other outcomes 2000, such as the procedure time 2010 of the execution procedure plan 2020. B-score algorithm 70 can use both the determined B-score and other patient data 1020, and can determine different relationships based on different characteristics of patients in patient data 1020. For example, a patient with a higher B-score belonging to the US population may be associated with a longer procedure time 2010, while a patient with a higher B-score belonging to the EU population may be associated with a shorter procedure time 2010. Therefore, B-score algorithm 70 and / or image analysis system 10 can determine a longer procedure time 2010 based on a higher B-score and a patient's nationality being US, and a shorter procedure time 2010 based on a higher B-score and a patient's nationality being EU. Other factors (e.g., from patient data 1020) can alter certain relationships, allowing image analysis system 10 and / or B-score algorithm 70 to determine certain relationships when higher or lower B-scores are combined with certain patient data 1020.

[0184] refer to Figure 16B The fourteenth GUI 274 can be implemented as one of the cards or widgets 409 displayed on the third GUI 255. For example, the plurality of widgets 409 may include a B-rating card 414 as described with reference to FIG. 4. The user can click on the B-rating card 414 to display the fourteenth GUI 274. Figure 16BExamples are shown of the fourteenth GUI 274 being presented as a pop-up or a separate box of the third GUI 255, but the aspects disclosed herein are not limited to these. For example, clicking the B-rating card 414 can bring up the fourteenth GUI 274 as a full-screen GUI (such as...). Figure 16A (as shown), and / or a magnified or enlarged view of the B-score card 414 is displayed on the third GUI 255 before clicking. Alternatively or in addition, clicking the B-score card 414 may bring up a simplified version of the fourteenth GUI 274, which displays some (but perhaps not all) features of the fourteenth GUI 274. For example, when clicked, the B-score card 414 may display a gradient bar or scale 1602, the determined B-score (e.g., 5.1), and representations of bones (e.g., femur) under various B-scores for comparison, such as below the determined B-score (e.g., 0 and 3) and above the determined B-score (e.g., 7).

[0185] refer to Figure 16C The fourteenth GUI 274 can be rotated or flipped upon clicking to display the "back side" of the B-score card 414, which can display text information, videos, etc., or other additional information or analysis regarding the B-score or bone shape (e.g., determinations made by one or more algorithms 90). For example, the back side of the B-score card 414 can display a B-score video 1606 and / or an analysis of changes in the B-score. The B-score card 414 can provide an assessment of shape changes based on a 3D model (e.g., an artificial model 402): for example, a 3D femur with shape changes consistent with knee osteoarthritis, and the shape has been recorded as a B-score. The B-score card 414 can describe patient-specific changes in bone shape and / or curvature that may not be reflected in the overall B-score.

[0186] although Figure 15 as well as Figures 16A to 16CExamples of predictions related to pain and functional loss are illustrated, but the predicted outcomes 2080 shown are not limited to pain and functional loss. For example, the image analysis system 10 may base its predictions on a comparison of the joint space width determined by the joint space width algorithm 50 with the planned implant size in the determined procedure plan 2020; a narrower (or wider) joint space width determined by the joint space width algorithm 50; a joint space narrowing over time as determined by the joint space width algorithm 50; the volume or number of osteophytes determined by the osteophyte detection algorithm 60; the volume of osteophytes in progression determined by the osteophyte detection algorithm 60; or the B-score or the B-score in progression determined by the B-score algorithm 70 (or alternatively, the B-score). Outside the predetermined range), based on severe deformities detected by the alignment / deformity algorithm 80, based on OA progression determined using one or more algorithms 90, based on impact data calculated using parameters determined from the joint space width algorithm 50, osteophyte detection algorithm 60 and / or alignment / deformity algorithm 80, a large bone-tissue ratio, PPT and / or PTT, etc., to predict the patient's stress level, anxiety level and / or mental health status, recovery time, risk of complications during the procedure (e.g., dyspnea and / or blood flow or heart rate complications), risk of infection, likelihood of revision surgery, and / or increased difficulty rating. Regarding cartilage loss, the image analysis system 10 can determine a Z-score or other statistical measure to determine the risk of cartilage loss. The determined predicted cartilage loss can be based on joint space width. These predicted outcomes 2080 can be displayed on any GUI 250, such as the twelfth GUI 270, the fourth GUI 256, etc. Figure 4B ), second GUI 254 and third GUI 255 ( Figures 4A to 4E ) etc. Tissue-Skeleton GUI

[0187] refer to Figure 2 Figure 3 and Figures 17 to 19 One or more algorithms 90 can be configured to detect or determine, from one or more acquired images 302, prepatellar thickness (PPT) and / or pretibial tuberosity thickness (PTT), minimum distance from bone to skin, tissue-to-bone ratio, bone-to-tissue distance or value, and / or bone-to-tissue distance, bone-to-skin ratio, etc. of PPT and / or PTT.

[0188] PPT and / or PTT can be measurements of the distance between bone and skin determined using acquired image 302 (e.g., a CT scan) and can be used as a proxy or alternative to manually inputting BMI. In some examples, PPT and / or PTT at joints (e.g., the knee joint) can provide more accurate information than BMI (which can be a whole-body measurement). Image analysis system 10 can determine certain tissue-bone parameters (such as bone-tissue ratio, PPT, PTT, and / or BMI) and / or use some of these parameters as input (e.g., as patient data 1020 or from previous outputs of one or more algorithms). Image analysis system 10 can determine one or more outputs 2000 based on certain determined tissue-bone parameters. For example, one or more algorithms 90 can determine a larger procedure time 2010 based on a larger determined tissue-bone ratio, because practitioners may need more time to process (e.g., dissect) a larger amount of tissue. Additionally, the image analysis system 10 can determine a higher level of case difficulty based on a larger bone-to-tissue ratio, PPT, and / or PTT determined by one or more algorithms 90, because joints (e.g., the knee joint) may be more difficult to balance due to more tissue.

[0189] Figure 17 An image processing method 1700, which describes one or more algorithms 90 that can be used to determine tissue-bone parameters, is described. Figure 18 A fifteenth GUI 276 is shown, which can display certain steps during the execution of image processing method 1700 by one or more algorithms 90. Image processing method 1700 may include a step 1702 of segmenting acquired image 302 (e.g., a CT scan). Based on segmentation step 1702, fifteenth GUI 276 may display box 1802, which shows a representative model 1804 of the patient's anatomy. Image processing method 1700 may include a step 1704 of thresholding soft tissue, which can be visualized in box 1806 showing the representative model 1804 and the boundary 1808 surrounding the soft tissue. Image processing method 1700 may include a step 1706 of determining the minimum distance from bone to skin. For example, step 1706 may include determining the minimum distance from bone to skin at multiple locations on the bone, and these distances may be expressed according to a color gradient scale that can be superimposed on the representative model 1804. Box 1806 shows a color-coded representation 1812 of a portion of the representative model 1804 (e.g., the tibia).

[0190] As a replacement or supplement to the fifteenth GUI 276, one or more GUIs 250 may include the sixteenth GUI 278. Figure 19The sixteenth GUI 278 is configured to display a representative model 1902 of at least a portion of the patient's skeleton adjacent to or on the same screen as the corresponding color-coded representation 1904. The sixteenth GUI 278 may include one or more targets or regions 1906 that can be predefined and / or identified by one or more algorithms 90. For example, each region 1906 may represent a clinically significant region (e.g., typically associated with one or more specific bones, such as the knee joint, and / or specific to a particular patient). Each target region 1906 may include multiple smaller regions or points 1908. Each point 1908 may indicate a key anatomical area or point on a bone (e.g., femur, tibia, and patella). One or more algorithms 90 may calculate bone-skin distances to all these points 1908 in these target regions 1906 to provide bone-skin or soft tissue thickness values, which may be output to a storage device and / or displayed (e.g., on the GUI 278). Switching features

[0191] refer to Figures 20 to 23 Any one of the one or more GUIs 250 may include toggle functionality, wherein certain display features can be selectively shown, turned off, or modified. (See reference) Figure 20 One or more GUIs 250 may include a seventeenth or osteophyte-to-give GUI 280 having a first screen or frame 2102 and a second screen or frame 2104. The first frame 2102 may display a representative model 2106 and one or more osteophytes 2108 determined by the image analysis system 10 using one or more algorithms 90. A user can enter a command to close the osteophytes to reveal a second frame 2104, which may display the representative model 2106 and may omit one or more osteophytes 2108. The user can enter commands by tapping an osteophyte, pressing a button, touching an osteophyte with a touchscreen, or swiping on an osteophyte. Although the first frame 2102 and the second frame 2104 only show the representative model 2106 in the seventeenth GUI 280, Figure 20 The illustrated switching functionality can be applied to any GUI that displays representative models and osteophytes, such as the second GUI 254 and the third GUI 255. Figures 4A to 4E ), Ninth GUI 264 ( Figures 7 to 9 )wait.

[0192] refer to Figure 21One or more GUIs 250 may include an eighteenth or skeletal portion switching GUI 282 having a first screen or frame 2202 and a second screen or frame 2204. Similar to the seventeenth GUI 280, the first frame 2202 may display a representative model 2206 and one or more osteophytes 2208 determined by the image analysis system 10 using one or more algorithms 90. A portion 2210 of the representative model 2206 may be closed in the second frame 2204. Figure 21 As illustrated, portion 2210 of representative model 2206 can represent the fibula. A user can enter a command to close portion 2210 to reveal a second frame 2204, which can display representative model 2206 and may omit portion 2210 (e.g., the fibula), but may retain osteophyte 2208. Although portion 2206 is... Figure 21 The fibula is exemplified here, but other parts of the bone, such as the patella, certain anatomical regions, and certain bony landmarks (e.g., condylar surfaces), can also be opened and / or closed. Users can input commands by clicking on section 2210, pressing a button, touching the osteophyte using a touchscreen, or swiping on the osteophyte. Although only a representative model 2206 is shown in the eighteenth GUI 282, in the first box 2202 and the second box 2204, Figure 21 The illustrated switching functionality can be applied to any GUI that displays a representative model, such as the second GUI 254 and the third GUI 255. Figures 4A to 4E ), Ninth GUI 264 ( Figures 7 to 9 )wait.

[0193] refer to Figure 22 One or more GUIs 250 may include a nineteenth or opaque GUI 284 having a first screen or frame 2302 and a second screen or frame 2304. The first frame 2302 may display a representative model 2306 and one or more osteophytes 2308 determined by the image analysis system 10 using one or more algorithms 90. The osteophytes 2308 may be displayed with a first color and / or opacity. A user can enter a command to change the color and / or opacity of the osteophytes 2308 to reveal a second frame 2304, which may display the representative model 2306 and the osteophytes 2308 with a selected opacity. The user can enter commands by tapping the osteophytes 2308, pressing a button, touching the osteophytes with a touchscreen, or swiping on the osteophytes. For example, the first frame 2302 may display opaque osteophytes 2308, and the user can switch to the second frame 2304, which may display more transparent osteophytes 2308, allowing the portion of the representative model 2306 located below the osteophytes 2308 to be seen. Although the first box 2302 and the second box 2304 only show the representative model 2306 in the nineteenth GUI 284, Figure 22The illustrated switching functionality can be applied to any GUI that displays representative models and osteophytes, such as the second GUI 254 and the third GUI 255. Figures 4A to 4E ), Ninth GUI 264 ( Figures 7 to 9 )wait.

[0194] refer to Figure 23 One or more GUIs 250 may include a twentieth or skeleton and osteophyte switching GUI 286 having a first screen or frame 2402, a second screen or frame 2404, and a third screen or frame 2406. The first frame 2402 may display a representative model determined by the image analysis system 10 and having a first bone 2408 (e.g., tibia), a second bone 2410 (e.g., femur), one or more first osteophytes 2412 on the first bone 2408, and one or more second osteophytes 2414 on the second bone 2410. A user can enter a command to turn off any of the first bone 2408, the second bone 2410, one or more first osteophytes 2412, and one or more second osteophytes 2414. For example, the second frame 2404 may turn off the second bone 2410, such that the second frame 2404 displays the first bone 2408, one or more first osteophytes 2412, and one or more second osteophytes 2414, and omits the second bone 2410. Alternatively, the first bone 2408 may be turned off. The third frame 2406 can disable the second osteophyte 2414, causing the third frame 2406 to display the first bone 2408 and one or more first osteophytes 2412, while omitting the second bone 2410 and the second osteophyte 2414. Users can select features to disable and / or enable by entering commands, such as by clicking on the feature to be disabled (e.g., the first bone 2408, the second bone 2410, one or more first osteophytes 2412, and one or more second osteophytes 2414), pressing a button, touching the feature using a touchscreen, or swiping on the feature. Figure 23 The switching functionality illustrated in the example can be applied to any GUI that displays representative models and osteophytes, such as the second GUI 254 and the third GUI 255. Figures 4A to 4E ), Ninth GUI 264 ( Figures 7 to 9 )wait.

[0195] refer to Figures 20 to 23 And any of the GUIs or functions described in GUIs 17 through GUI 280 may be in GUI 255 ( Figure 4B Implemented in ) and / or via menu 418 ( Figure 4BAccess can be made via other interactive features. For example, a user can click the displayed indicator 404 on or off osteophytes and / or bones (e.g., femur, tibia, or fibula), and / or use menu 418 to click the switch. Menu 418 may also include a slider for the opacity of indicator 404 (e.g., osteophyte). Simulated Motion GUI

[0196] refer to Figure 24 Multiple GUIs 250 may include a twenty-first GUI 288 to display the movement of one or more bones and one or more ligaments. The twenty-first GUI 288 can allow practitioners to assess 3D interactions in situations where the imaging modality may produce 2D images (e.g., X-rays) and certain areas (e.g., the tibial plateau) may not be visible. A procedure plan (e.g., Procedure Plan 2020) may require the removal of large pieces of bone to make room for an implant, but may also require leaving some osteophytes. The twenty-first GUI 288 can allow practitioners to assess how the remaining osteophytes might cause problems with the soft tissue capsule and joint balance (e.g., knee balance). For example, if a ligament constantly interacts with the remaining osteophytes, the ligament may remain stretched. Osteophytes near a ligament may stretch the ligament and may create gaps in certain areas (e.g., between the ligament and bone). As another example, a tight positioning relationship between a ligament and the remaining osteophytes may cause pain.

[0197] The GUI 288 may display an artificial model 402 of one or more bones (e.g., the knee joint) and one or more indicators 404 of patient osteophytes (e.g., determined by one or more algorithms 90). The GUI 288 may display simulated movements of the artificial model 402, such as simulated flexion and / or extension. Simulated movements may be determined by one or more algorithms 90 based on prior procedural data 1050 from multiple patients and / or available simulation or statistical models. In some examples, simulated movements may be determined by one or more algorithms 90 using patient data 1020 (e.g., alignment data, range of motion data, etc.) and / or imaging data 1010. In some examples, the GUI 288 may display patient data 1020 and / or other data 1000 used to determine simulated movements.

[0198] The GUI 288 can also display ligaments 2502 (e.g., medial collateral ligament or MCL) on artificial model 402. The ligament 2502 itself and its motion during joint movement can be simulated (e.g., based on available models, statistical models, and / or prior procedural data 1050 from multiple patients). For example, ligament 2502 can be based on a known model and located on a known region of the bone where the average ligament (e.g., average MCL) is located. In some examples, ligament 2502 can be modeled using image analysis system 10 and / or based on the patient's own anatomy (e.g., using patient data 1020 from previous surgeries or image data 1010 obtained using a modality capable of imaging ligaments). Ligament 2502 can rotate and / or translate during movement (e.g., flexion and extension) with joint movement.

[0199] Image analysis system 10 can correlate the surfaces or features of a skeletal model with the surfaces or features of an artificial model 402. For example, artificial model 402 may include the same number of vertices, faces, triangles, bases, etc., as a patient's skeleton or other statistical skeletal model. These features may shift with slight changes in the shape of the skeleton. These features may define a set of points or locations within the artificial model 402 of the skeleton. Image analysis system 10 can create a mask over these points based on known locations. For example, the mask may include ligament representations, and the mask may be superimposed on these points or features based on the known locations of ligaments (e.g., MCL, ACL, etc.). The mask may be displayed over the skeleton of artificial model 402.

[0200] The 21st GUI 288 can simulate the movement of both the ligament 2502 and the joint, allowing practitioners to assess how the ligament 2502 will interact with the osteophytes (indicated by indicator 404) during movement. In some examples, the displayed osteophytes and ligament 2502 can represent the preoperative state. In other examples, the displayed osteophytes and ligament 2502 can represent the predicted postoperative state based on the current procedural plan 2020 and / or procedural data 1030. In still other examples, the displayed osteophytes and ligament 2502 can represent the predicted state when the patient has not received treatment. The 21st GUI 288 allows practitioners to evaluate and modify or adjust the procedural plan 2020 based on the assessment of the ligament 2502 relative to the osteophytes during movement. When the 21st GUI 288 displays simulated movement, the attachment points of the simulated ligament 2502 can remain the same. In some examples, the osteophytes and ligaments 2502 shown may represent the intraoperative status (e.g., new and / or intraoperative data may be received during the procedure) and / or the postoperative status after the procedure using intraoperative and / or postoperative data. In still other examples, the osteophytes and ligaments 2502 shown may represent the predicted long-term status after the procedure, allowing practitioners to assess the need for revision surgery and / or further treatment based on patient outcomes.

[0201] In some examples, the twenty-first GUI 288 may display the determined or predicted movement, rather than a simulation based on a statistical model or available model. For example, the image analysis system 10 may determine, via one or more algorithms 90 (e.g., the alignment / deformity algorithm 80), how the patient's anatomy is currently moving, how the patient's anatomy is predicted to move when the patient is not receiving treatment, how the patient's anatomy is predicted to move when the patient is receiving treatment (e.g., program plan 2020), and / or the desired or ideal movement. The image analysis system 10 may generate one or more simulations of the determined movement. For example, the image analysis system 10 may generate images of the movement of the tibia and femur relative to each other throughout the entire range of motion of the knee joint. The display of osteophytes on the tibia and femur and their relative positioning throughout the entire range of motion of the knee joint may facilitate the identification of osteophytes that may impede the patient's range of motion and / or cause pain during the patient's knee joint movement.

[0202] The GUI 288 can further display relevant metrics corresponding to the simulated movement of the ligament 2502 and / or determinations made by the image analysis system 10. For example, the image analysis system 10 can determine the perceived pain associated with the simulated movement of the ligament 2502, the measurement and / or size of the gap between the ligament 2502 and the surrounding bones, the range of extension and / or stretching of the ligament 2502, the range of stretching of the ligament 2502 and / or stretching exceeding a predetermined threshold and / or average value, etc. Simulated Implant GUI

[0203] refer to Figure 25 The image analysis system 10 can determine, based on parameters determined by one or more algorithms 90, that the procedure plan 2020 should include a specific implant design or size. For example, based on a joint space width or narrowing determined by the joint space width algorithm 50, the image analysis system 10 can determine that the implant width should be reduced and / or determine the type of implant (e.g., restraint type). Based on a joint width determined by the joint space width algorithm 50 and / or an increased joint space width, and / or a looser or less stable joint determined by the alignment / deformity algorithm 70, the image analysis system 10 can determine that the implant width should be increased (e.g., using an extension or pad) and / or that the type of implant should be a stable or restraint type, and that the procedure type or extent in the procedure plan 2020 should include more corrective surgery, such as from partial joint (e.g., knee, hip, or shoulder) replacement to total joint replacement.

[0204] like Figure 25 As illustrated, the image analysis system 10 can visually depict the determined implant design as a model implant 2602 superimposed or overlaid on the acquired image 302 (e.g., CT scan) and / or representative model 2604 in the twenty-second GUI 289. Figure 25 Images 302 of the knee joint and model implant 2602, which are configured to be attached to at least a portion of the knee joint, are shown. Bone resection GUI

[0205] The aspects disclosed herein can be used to determine the geometry and / or dimensions of osteotomy or resection and / or implant design. Figure 26 and Figure 27 An exemplary GUI is shown that depicts the bone resection plane and / or a virtual bone model.

[0206] refer to Figure 26 One or more GUIs 250 may include a 23rd or osteotomy GUI 290 having at least one screen or frame 2702, 2704, 2706 and / or 2708. At least one frame 2702, 2704, 2706 and / or 2708 may display a representative model determined by the image analysis system 10 and having at least one bone 2710 (e.g., tibia or femur).

[0207] Image analysis system 10 may use one or more algorithms 90 (e.g., osteophyte detection algorithm 60) to determine a recommended or planned excision region or volume 2712 (e.g., as part of program plan 2020). As an example, image analysis system 10 may determine the value of the excision region or volume 2712 based on the determined osteophyte volume, and may determine the location of the excision region or volume 2712 based on one or more detected osteophyte locations. Bone excision GUI 290 may display the determined excision region or volume 2712 superimposed on at least one bone 2710. At least one box 2702, 2704, 2706 and / or 2708 may include multiple boxes 2702, 2704, 2706 and / or 2708 illustrating various orientations and / or angles of the determined excision region or volume 2712 on at least one bone 2710.

[0208] Image analysis system 10 may use one or more algorithms 90 to determine a recommended or desired cutting start line 2714, where a practitioner (e.g., a surgeon) should position surgical tools (e.g., a drill or other cutting tool) to produce the displayed, determined resection area or volume 2712. Bone resection GUI 290 may display the recommended or desired cutting start line 2714 superimposed on at least one bone 2710 and / or the determined resection area or volume 2712. In some examples, bone resection GUI 290 may determine multiple cutting start lines 2714, which may be displayed in separate boxes 2704, 2706, and / or 2708. In some examples, bone resection GUI 290 may determine updated or adjusted start lines 2714 based on procedural progress, cutting, or other newly received information.

[0209] refer to Figure 27 One or more GUIs 250 may include a 24th or virtual skeleton GUI 292 having at least one screen or frame 2802, 2804, 2806, 2810 and / or 2812. At least one frame 2802, 2804, 2806, 2810 and / or 2812 may display a representative model determined by the image analysis system 10 and having at least one bone 2814 (e.g., tibia or femur).

[0210] Image analysis system 10 may use one or more algorithms 90 (e.g., osteophyte detection algorithm 60) to determine one or more osteotomies or planes 2816, 2818, 2820, 2822, 2824, 2826 (e.g., according to program plan 2020). Virtual skeleton GUI 292 may display the determined osteotomies 2816, 2818, 2820, 2822, 2824, 2826, 2828, and 2830 superimposed on at least one bone 2814. For example, at least one frame 2802, 2804, 2806, 2810 and / or 2812 may display a posterior condylar osteotomy 2816, a posterior oblique osteotomy 2818, a distal osteotomy 2820, an anterior oblique osteotomy 2822, anterior condylar osteotomy 2824, a plateau osteotomy 2826 (e.g., a tibial plateau osteotomy), a post-hole osteotomy 2828, and / or a lateral wall osteotomy 2830. At least one frame 2802, 2804, 2806, 2810 and / or 2812 may include multiple frames 2802, 2804, 2806, 2810 and / or 2812 that display bone 2814 in various orientations to optimally display the determined osteotomies 2816, 2818, 2820, 2822, 2824, 2826, 2828 and 2830. The image analysis system 10 can also use one or more algorithms 90 to determine a desired or recommended implant design 2832, and the virtual skeleton GUI 292 can display the determined implant design 2832 (with or without a skeleton 2814). The image analysis system 10 can determine certain planes or lines 2834 corresponding to the geometry of the skeleton 2814 and / or the determined osteotomies 2816, 2818, 2820, 2822, 2824, 2826, 2828 and 2830.

[0211] refer to Figure 28A One or more GUIs 250 may include a twenty-fifth GUI or osteotomy GUI 294 having at least one screen or frame 2902, 2904 and / or 2906. At least one frame 2902, 2904 and / or 2906 may display a representative model determined by the image analysis system 10 and having at least one bone 2908 (e.g., tibia or femur) and implant 2910 (e.g., as determined by the image analysis system 10). Bone 2908 may include at least one osteophyte 2914. Bone 2908 and / or osteophyte 2914 may be translucent or transparent, allowing the complete implant 1910 to be seen.

[0212] Image analysis system 10 may use one or more algorithms 90 (e.g., osteophyte detection algorithm 60) to determine one or more osteotomies 2912 (e.g., according to program plan 2020). Osteotomies 2912 may be configured to remove osteophytes 2914. Osteotomy GUI 294 may display the determined osteotomies 2912 superimposed on at least one bone 2908 and / or implant 2910.

[0213] For example, at least one frame 2902, 2904, and / or 2906 may include a first frame 2902, a second frame 2904, and a third frame 2906. The first frame 2902 may be configured to display a segmented bone 2908 (e.g., a femur). The second frame 2904 may be configured to display an implant 2910 superimposed on the bone 2908 (e.g., according to program plan 2020). The third frame 2906 may be configured to display a planned or determined osteotomy 2912 superimposed on the bone 2908 to show how much bone tissue and / or osteophytes 2914 will be removed using the osteotomy 2912. For example, the third frame 2906 may display a view showing a planar cross-section of the osteotomy 2912. The third frame 2906 may also display the implant 2910 superimposed on the bone 2908. In some examples, the third frame 2906 may display a reference axis, plane, or grid 2913 relative to the osteotomy 2912.

[0214] The image analysis system 10 can also use one or more algorithms 90 to determine the desired or recommended implant design of the implant 2910, and the osteotomy GUI 294 can display the determined implant design of the implant 2910 (with or without bone 2908).

[0215] Figure 28B An enlarged view of box 2906 is shown. (Reference) Figure 28B One or more osteophytes 2914 may include a first portion 2916 removed from osteotomy 2912 and a second portion 2918 retained after bone 2908 is cut according to osteotomy 2912. Figure 28B As shown, the first portion 2916 that was removed is shown on the outer side of the osteotomy 2912, while the remaining second portion 2918 is shown on the inner side of the osteotomy 2912.

[0216] refer to Figure 34The GUI 3500 includes an exemplary postoperative model with virtual tibia 3504 and femur 3506 generated by the image analysis system 10. Virtual models of implants 3508 and 3510 are shown as being mounted on the tibia 3504 and femur 3506. A resection line 3512 is shown superimposed on the femur 3506, and the resection line 3512 can indicate the cutting path on the femur 3504 in the surgical plan. Bone resection, resection line 3512, and implants 3508 and 3510 can be determined by the image analysis system 10. The tibia 3504 and femur 3506 may include at least one osteophyte 3516 and 3518, and the osteophytes 3516 and 3518 can be determined by the image analysis system 10. In some examples, bones 3504, 3506, wireframes, and / or osteophytes 3516, 3518 may be displayed as translucent or transparent, allowing the entirety of each implant 3508, 3510 to be visible. In other examples, implants 3508, 3510 may be transparent or translucent to reveal other features, such as the bone to be removed. Transparency and / or translucency can be turned on and off as needed by the user. In some examples, bones 3504, 3506 may be removed from the display on a control command and reinserted into the display upon a second actuation of the control command. Implant 3508 is positioned above the proposed resection area of ​​the tibia 3502, and implant 3510 is positioned above the proposed resection area of ​​the femur 3506. In this example, the display of the resection line 3512 and the surgical implant 3510 facilitates determining the fit of the implant 3510 after the cut is made under this particular surgical plan, and allows the surgeon to adjust the resection line 3512 based on the relative positioning of the resection line 3512 and the implant 3510. The surgeon will be able to examine the proposed cut outlined by the resection line 3512 with reference to the implant 3510, and in some examples, the positioning of the resection line 3512 can be adjusted using one or more controls of the GUI 3500. For example, the user can select the resection line 3512 and “drag and drop” it to a new location on the femur 3506. In some examples, osteophytes 3516, 3518 on the tibia 3506 and femur 3504 are also cut so that implants 3508, 3510 can be installed. Although not shown in this figure, the image analysis system 10 can also create and display resection lines corresponding to the proposed resection of the femur. Users can manipulate the positioning of the tibia 3504, femur 3506, and implants 3508 and 3510 to visualize the resection line 3512 in three dimensions and view it from any angle. In some examples, multiple resection lines 3512 can be displayed on bones such as the femur 3506 and / or tibia 3504 to allow users to compare different potential resection lines 3512.In some examples, users can actuate controls (such as buttons or other control commands for GUI 3500) to remove osteophytes 3516 and 3518 from the display of GUI 3500. Bone mineral density (BMD) algorithm and graphical user interface (GUI)

[0217] refer to Figure 2 and Figure 43 Bone mineral density (BMD) algorithm 87 and / or one or more other algorithms 90 can analyze image data 1010 to determine one or more outputs 2000 (such as the BMD of a patient's bones) to be displayed within the GUI (e.g., GUI 4200). BMD can refer to the amount of bone minerals present in the bone tissue and can be used as a consideration in various medical treatments and / or as an indicator of various medical conditions (such as osteoporosis or osteopenia). For example, BMD can be used for bone health assessment, risk prediction, monitoring treatment and disease, and preoperative planning for surgeries such as knee and hip replacements.

[0218] refer to Figures 40A to 40B as well as Figure 41 In clinical practice, bone marrow diameter (BMD) is typically measured indirectly through medical image analysis. However, BMD may not be directly obtainable from medical images and may require additional processing and / or calibration. For example, in many instances, the BMD of bone 4112 can be determined using images generated from a computed tomography (CT) scan that includes bone 4112 and a calibration device (such as a calibration phantom 4120). The calibration phantom 4120 may include a variety of different reference materials (e.g., reference materials 4122, 4124, 4126), where the BMD equivalent value of each reference material is known and can be placed near bone 4112 during the CT scan. The medical image 4102 generated by the CT scan represents the attenuation of X-rays as they pass through different materials, indicated by the different intensities shown in the medical image 4102 (in this example, higher intensities appear brighter, and lower intensities appear darker). At least one of several different reference materials included in the calibration phantom 4120 is configured to cause different X-ray attenuation than bone 4112, thereby producing a different display intensity in medical image 4102. Since the BMD equivalent value of this at least one reference material is known, its intensity and BMD equivalent value can be used as reference points for determining the BMD of bone 4112. However, such a calibration phantom 4120 is typically unavailable and / or uncomfortable for patients. Therefore, a method for determining bone BMD from medical images that is more readily available, reliable, accurate, and comfortable than using a calibration phantom is desired.

[0219] refer to Figure 40A and Figure 42 BMD algorithm 87 and / or one or more algorithms 90 can be configured to determine the BMD of the patient's bones from the acquired image 302 using BMD equivalents derived solely from a single reference material included in the calibration device. For example, the calibration device may be a motion bar 4110. The motion bar 4110 may be a straight or bar physically attached to the patient's anatomy or limb (e.g., the patient's leg) during a medical imaging procedure (e.g., a CT scan or MRI scan) to keep the patient's anatomy stationary or extended. Alternatively or additionally, to ensure the accuracy of the medical imaging procedure, the motion bar 4110 may be used to determine whether the patient's anatomy remains stationary or extended during the medical imaging procedure. For example, any movement of the motion bar 4110 detected during the medical imaging procedure may indicate that the patient's anatomy has moved during the medical imaging procedure, which may necessitate rejection or repetition of the medical imaging procedure. The motion bar 4110 may be included as a standard component of the medical imaging procedure protocol. The motion bar 4110 may be radiopaque and may be made of aluminum or any other suitable material.

[0220] Still referencing Figure 42In one example, BMD algorithm 87 can be configured to determine the BMD of the patient's skeleton from the acquired image 302 by first identifying the intensity of at least one of a single reference material included in the calibration device, the bone 4112 of the patient's anatomy, and the fat 4114 and air 4116 from the patient's anatomy within the acquired image 302, using the BMD equivalent value derived solely from the single reference material included in the calibration device (e.g., the movement bar 4110). In this example, BMD algorithm 87 then receives, accesses, retrieves, or otherwise obtains predetermined BMD equivalent values ​​for the single reference material included in the calibration device and at least one of the fat 4114 and air 4116. BMD algorithm 87 then uses the intensity and BMD equivalent value of the single reference material included in the calibration device and at least one of the fat 4114 and air 4116 to determine the BMD calibration factor (BCF) of the acquired image 302. The BCF can be determined using linear regression, a best-fit line, or any other suitable method. Since (i) a single reference material included in the calibration device and (ii) at least one of fat 4114 and air 4116 are typically present in medical images, determining the BCF using the intensity and BMD equivalent values ​​of the single reference material included in the calibration device and at least one of fat 4114 and air 4116 can be more reliable than using the calibration phantom 4120 (as described above). Furthermore, the single reference material included in the calibration device and at least one of fat 4114 and air 4116 can produce a wider intensity range than the calibration phantom 4120, resulting in a more accurate BCF. After determining the BCF, the BMD algorithm 87 can apply the BCF to the intensity of bone 4112 (i.e., CT image data) to determine the BMD of bone 4112. The BMD of bone 4112 can then be used to generate one or more GUIs. For example, the BMD of bone 4112 can be used to generate an artificial or virtual (e.g., two-dimensional or three-dimensional) model of a patient's anatomy and display that model within a GUI, which exhibits the BMD of bone 4112.

[0221] In some instances where the BCF is generated using only the strength of one of fat 4114 and air 4116, the BMD algorithm 87 can be configured to compare the strength and / or BMD equivalent of the one of fat 4114 and air 4116 not used to determine the BCF with one or more relationships between a single reference material included in the calibration apparatus and the strength and BMD equivalent of the one of fat 4114 and air 4116 used to determine the BCF to determine whether the accuracy of the BCF is acceptable. For example, if the strength and / or BMD equivalent of the one of fat 4114 and air 4116 not used to determine the BCF does not fit within an acceptable threshold to a linear regression or best-fit line generated using the strength and BMD equivalent of the other of fat 4114 and air 4116 included in the calibration apparatus, the BMD algorithm 87 can determine that the accuracy of the BCF is unacceptable. In some instances, the BCF is determined using the strength and BMD equivalent of a single reference material included in the calibration apparatus, fat 4114, and air 4116. In some instances, shape models can be used to identify bones 4112, a single reference material included in the calibration device, fat 4114 and / or air 4116, and / or their respective strengths from within the acquired image 302.

[0222] In some instances, instead of using a calibration device (e.g., a movement rod 4110) to determine the BMD of the patient's bones, the BMD algorithm 87 can use cortical bone as a reference material. For example, the BMD algorithm 87 can identify the strength of cortical bone from the patient's anatomy and the strength of at least one of fat 4114 and air 4116 from the patient's anatomy from the acquired image 302. The BMD algorithm 87 can then receive, access, retrieve, or otherwise obtain the BMD equivalent values ​​of cortical bone and at least one of fat 4114 and air 4116. The BMD algorithm 87 can then use the strength of cortical bone and at least one of fat 4114 and air 4116 to determine the BCF of the acquired image 302. Since cortical bone, fat 4114, and air 4116 are often present in medical images, using the strength and BMD equivalent values ​​of cortical bone, fat 4114, and / or air 4116 to determine the BCF of a medical image can be more reliable, feasible, or cost-effective than using a calibration phantom 4120 (as described above) or a motion bar 4110 (as described above).

[0223] Figure 43 A GUI 4200, provided, generated, or otherwise communicatively coupled to the image analysis system 10, is shown, similar to... Figure 4EGUI 255. GUI 4200 displays an artificial model 402 of the patient's anatomical structures (e.g., the patient's knee joint). In this example, GUI 4200 includes a bone mineral density (BMD) component 4202 in addition to the B-scoring component 414 (as described above) and the C-scoring component 416 (as described above). Figure 43 As shown, the B-score widget 414 can display various information about the B-score determined for the patient (e.g., via the B-score algorithm 70), such as a skeletal image representing the B-score or a gradient bar including the B-score in a scaled background, as described above. Figure 43 As shown, the C-score widget 416 can display various information about the C-score determined for the patient (e.g., via the joint space width algorithm 50), such as a skeletal image representing the C-score or a gradient bar indicating the C-score against a scale background, as described above. Similarly, as Figure 43 As shown, the BMD widget 4204 displays various information about the BMD determined for the patient, such as gradient bars indicating the BMD against a scale background. The GUI 4200 allows a user of the GUI 4200 (e.g., a surgeon or medical practitioner) to select any point or region on an artificial model 402 of the patient's anatomy and view the BMD determined for the corresponding point or region of the patient's anatomy within the BMD widget 4202. If the user selects a region in the GUI 4200, the determined BMD can correspond to the average BMD over the selected region. Other outputs 2000

[0224] Return to reference Figure 2One or more algorithms 90 may also determine (or be determined by the image analysis system 10) other outputs 2000 to be displayed on one or more GUIs 250, such as aspects of the procedure plan 2020, including steps, instructions, tools, etc., for preparing and / or performing the procedure (e.g., surgery). The procedure plan 2020 may include: the number, location, length, slope, angle, orientation, etc. of one or more planned tissue incisions or osteotomies; the planned type of implant; the planned implant design (e.g., shape and material); the planned or target location or alignment of the implant; the planned or target fit or tightness of the implant (e.g., based on gaps and / or ligament balance); the expected outcome (e.g., joint or bone alignment, bone inclination such as tibial inclination, level of activity, or expected value of postoperative output 2000); a list of steps to be performed by the surgeon; a list of tools that may be used, etc. The image analysis system 10 can determine the type or extent of the procedure in the procedure plan 2020, such as whether it should include more corrective surgery (e.g., from partial joint (e.g., knee, hip, or shoulder) replacement to total joint replacement), whether certain fixation or other techniques should be used, and whether cemented or uncemented techniques or implants should be used.

[0225] The procedure plan 2020 may include, for example, instructions on how to prepare the proximal tibia for receiving a tibial implant, how to prepare the distal femur for receiving a femoral implant, how to prepare the glenoid or humeral bone for receiving a glenoid head and / or a humeral prosthesis component, how to prepare the acetabular region or acetabulum for receiving a ball joint, etc. Bone surfaces may be cut, drilled, or scraped relative to a reference (e.g., via the epicondyle axis). The procedure plan 2020 may include the location, length, and other dimensions and / or slope values ​​of the surfaces used for bone preparation. As described later, the procedure plan 2020 may be updated and / or modified based on intraoperative data 3000. One or more GUIs 250 may include GUIs configured to display the procedure plan 2020 and / or related steps.

[0226] The program plan 2020 may also include predicted or target outcomes and / or parameters, such as target postoperative range of motion and alignment parameters, and target scores (e.g., stability, fall risk, joint stiffness or laxity, or OA progression). One or more GUIs 250 may include GUIs configured to display these target parameters and / or predicted parameters. These target parameters can ultimately be compared postoperatively with corresponding postoperative measurements or results to determine whether an optimized outcome has been achieved for the patient. The image analysis system 10 may be configured to update the program plan 2020 based on practitioner manual input and / or feedback input, newly acquired preoperative data 1000, or patient feedback.

[0227] The image analysis system 10 can also identify, assign, and / or designate assigned staff 2050 to assist in the execution of the procedure. For example, the image analysis system 10 can determine that the assigned staff 2050 should include surgeons, nurses, or other individuals with more experience in the type of surgery planned in the procedure plan 2020 (e.g., knee surgery or total knee arthroplasty) and / or with more experience in patients with similar characteristics to the current patient (e.g., narrower joint space width, patient history, specific type of deformity, etc.). The image analysis system 10 can determine that the assigned staff 2050 should include surgeons, nurses, or other individuals with procedure experience as long as the predicted procedure time 2010. The image analysis system 10 can store or determine the experience score or level of each staff member and can determine the average of composite procedures or staff teams and / or use a rolling average to determine the assigned staff 2050.

[0228] Image analysis system 10 may determine that the assigned personnel 2050 should have more experience individually and / or collectively based on: a certain type or more complex implantation plan; a narrower (or increasingly narrower) joint space width determined by joint space width algorithm 50; a larger volume or number of osteophytes (or an increasing volume or number of osteophytes over time, or an osteophyte volume exceeding a predetermined range) determined by osteophyte detection algorithm 60; a higher (or increased) B-score determined by B-score algorithm 70; severe or complex deformities detected by alignment / deformity algorithm 80; OA progression determined using one or more algorithms 90; and impact data calculated using parameters determined from joint space width algorithm 50, osteophyte detection algorithm 60, and / or alignment / deformity algorithm 80, etc. One or more GUIs 250 may include GUIs configured to display the assigned personnel 2050.

[0229] The image analysis system 10 can also determine the operating room layout 2030 and operating room scheduling 2040 based on the joint space width parameter determined by the joint space width algorithm 50, the osteophyte volume parameter determined by the osteophyte detection algorithm 60, the B-score determined by the B-score algorithm 70, the bone-tissue ratio, PPT and / or PTT, and / or based on the predicted procedure time 2010 or other determined or output 2000 (e.g., assigned staff 2050). The OR layout 2030 may include the room size, the setting, orientation, starting position, positioning, and / or movement or movement path of specific objects or personnel (such as robotic device 142, practitioners, surgeons or other staff members, operating table, camera, monitor 210, other equipment, sensors, or patients). The image analysis system 10 can determine a series of alarms, warnings, and / or alerts to be sent to practitioners, hospital staff, and / or patients during surgery preparation and / or surgery. The image analysis system 10 can determine or output new alerts to practitioners, hospital staff, and / or patients based on changes to any of the previously determined outputs 2000 (which could be based on newly acquired preoperative data 1000 and / or intraoperative data 3000, described later). In some examples, the alerts may be messages or indications displayed on a graphical user interface before or during surgery. One or more GUIs 250 may include GUIs configured to display OR layout 2030.

[0230] Image analysis system 10 can also determine or be used to determine surgeon ergonomics 2070 guidance. For example, image analysis system 10 can recommend certain postures or positions for assigned staff 2050 based on longer predicted procedure time 2010 (and / or parameters associated with longer procedure time 2010, such as narrower joint space width, larger osteophyte volume, larger B-score, more severe deformity, larger bone-to-tissue ratio, PPT and / or PTT, etc.), past experience of assigned staff 2050, and / or tools used as part of the determined procedure plan 2020. Image analysis system 10 can optimize surgeon ergonomics 2070 to reduce and / or optimize predicted procedure time 2010. One or more GUIs 250 may include GUIs configured to display steps or recommendations based on the determined surgeon ergonomics 2070.

[0231] refer to Figure 36 The image analysis system 10 can determine the density of the tibia 3704 and femur 3702 and display it on the GUI 3700. For example... Figure 36As shown, a virtual model of the tibia 3704 is displayed, with portions of the tibia 3704 outlined with shading and / or contours to indicate two bone density display portions 3706, 3708. In some examples, the bone density display portions 3706, 3708 may be shown centered below one or more contact points between the femoral condyle and the tibia 3704 when the leg is extended, and / or may be centered below the deepest points (points of highest load / pressure) on the medial and lateral tibial surfaces. Although the bone density display portions 3706, 3708 are shown positioned in this manner, the user has the ability to adjust the positioning of the bone density display portions 3706, 3708 to any location within any displayed bone, adjust the size of the bone density display portions 3706, 3708, and adjust the number of bone density display portions 3706, 3708. Alternatively, for example, when determining and / or displaying the bone density display portion, the image analysis system 10 can extract and / or remove cortical bone from the tibia 3704 and / or femur 3702. The bone density display portion can then be displayed above the trabeculae or cancellous bone of the tibia 3704 and / or femur 3702. In some examples, the cancellous bone of the tibia 3704 or any other bone can be segmented into lateral and medial epiphyseal and / or metaphyseal regions, and such regions can be displayed along with associated bone density and / or other parameters described herein. The 3D image model 3710 (including bone density display portions 3706, 3708) can determine the exact depth, width, height, and position of the bone density display portions 3706, 3708. In other examples, these dimensions can be user-selectable, and / or the user can adjust the determined depth, width, height, and position of the bone density display portions 3706, 3708. The location, size, and / or outer surface angle of the bone mineral density display portions 3706 and 3708 can be determined using a planned tibial cut (e.g., 3 mm below the original tibial surface). The volume can have a fixed size (e.g., approximately the size of a human thumb). Typically, after performing the tibial cut, the surgeon can test the excised area to assess the bone stability / strength of the tibial surface and feel with their thumb whether the cut bone is soft, deformable under pressure, and / or spongy. When the bone is soft, deformable under pressure, and / or spongy, the surgeon can adjust the surgical plan to cut more tibial bone or choose a cemented implant to achieve stability. Figure 36 In the example shown, a 3D model 3710 of the target bone 3704 can be rendered to display bone density display portions 3706, 3708 of a bone portion (e.g., a portion of the tibia 3704) specified by the user or system (e.g., specified by the image analysis system 10). This displays the density gradient of the bone density display portions 3706, 3708 to the user and shows the surgeon whether there are soft or spongy areas within the bone before cutting.

[0232] The average density values ​​of the bone mineral density display portions 3706 and 3708 can be determined, as well as density thresholds determined based on previous patient data or user-defined criteria. Figure 36 A GUI 3700 is shown, in which bone density display portions 3706 and 3708 are positioned within or within a portion of bone 3704. However, in some examples, the density of bone density display portions 3706 and 3708 may be displayed alongside bone density display portions 3706 and 3708, for example, on the exterior of bone density display portions 3706 and 3708 and / or on the exterior of bone 3704. In some examples, graphs of maximum and minimum density in bone density display portions 3706 and 3708 may be provided to indicate to the surgeon the density distribution within bone density display portions 3706 and 3708, and this may be displayed separately from each bone density display portion 3706 and 3708, with each associated bone density display portion 3706 and 3708 appropriately labeled. In some examples, a cross-section of model 3710 may be created to allow the surgeon to view density variations in a planar cross-section through bone density display portions 3706 and 3708. In some examples, users can define two different regions of bones 3702 and 3704 to compare the differences in density measurements, and the density measurements can be displayed for each defined region to allow users to quickly compare these measurements.

[0233] The GUI 3700 displaying the bone density display portions 3706 and 3708 can be based on direct volumetric rendering. The image analysis system 10 can determine a threshold density of the bone and display target bone portions with densities below the threshold (e.g., bone density display portions 3706 and 3708) as transparent. For target bone portions with densities above the threshold density, the higher-density target bone portions will be displayed using visual indicators (such as colors, patterns, combinations thereof, e.g., grayscale or color maps) based on the determined density values ​​from the image analysis system 10. In this example, the tibia 3704 is shown as transparent, and the bone density display portions 3706 and 3708 are indicated as patterns. The 3D image of the target bone (e.g., tibia 3704; femur 3702) generated by the image analysis system 10 can then be rendered using a monitor repositioned by a surgeon or other user. In some examples, the monitor can be a surgical monitor 210. In other examples, the monitor can be an augmented reality display on the monitor 210 of the program system 240. In some other examples, the display can be a mobile device 220, such as a mobile phone, tablet computer, or other type of portable display. A color map can be included on the display to indicate hard and soft bone based on a defined threshold density. The color map can use color gradients to indicate hard and soft regions of the bone density display portions 3706, 3708. In some examples, after the image analysis system 10 has rendered a 3D image 3710 with the bone density display portions 3706, 3708, a finite element model can be generated, allowing another numerical color map to be applied to the 3D model 3710.

[0234] Continue to refer to Figure 36 Each of the bone density display portions 3706, 3708 may include a numerical value associated with bone density. In some examples, the numerical value may be displayed within the bone density display portions 3706, 3708. In some examples, in addition to, or as an alternative to, the bone density-associated numerical value, it may be converted into a percentile of bone density relative to a group of other patients (e.g., a group of previous patients and / or other patients with one or more similar attributes (e.g., sex, BMI, weight, injury location / required surgery, and / or age)). In other examples, the average density across each bone density display portion 3706, 3708 may be displayed together with the corresponding bone density display portion 3706, 3708 determined by the image analysis system 10. In some other examples, the numerical value may be displayed alongside the 3D model 3710 on the GUI 3700. In some examples, the bone density display portions 3706, 3708 may include a display of the estimated load-bearing capacity of the bone density display portions 3706, 3708 prior to compression or collapse. Intraoperative system

[0235] refer to Figure 29A One or more intraoperative measurement systems 300 may collect (via arrow 303) intraoperative data 3000 during the procedure. During medical treatment planning or procedures, image analysis system 10 may collect, receive (e.g., from intraoperative measurement systems 300 via arrow 305) and / or store intraoperative data 3000. Image analysis system 10 may determine intraoperative output 4000 and output or transmit (via arrow 307) intraoperative output 4000 to output system 200.

[0236] Although the term "intraoperative" is used, the word "operation" should not be interpreted as requiring surgical intervention. Postoperative data may also be collected, received, and / or stored after the completion of medical treatment or procedures to serve as prior procedure data 1050 for subsequent procedures and / or to enable the refinement of one or more algorithms 90. Intraoperative output 4000 may be an updated or refined form of preoperatively determined output 2000. Figure 2 Intraoperative determined output 4000 may also be newly generated. Intraoperative determined output 4000 may also be referred to as secondary output 4000. Because many devices in one or more intraoperative measurement systems 300 are similar to devices in one or more preoperative measurement systems 100, many types of intraoperative data 3000 are similar to preoperative data 2000, and many procedures and included information in intraoperative output 4000 are similar to those procedures and information relating to preoperative determined output 2000. Any of the preoperative measurement systems 100 and data described herein may also be used and / or collected during surgery. Although some information is described herein as intraoperative data 3000 or intraoperative determined output 4000 and / or postoperative data or postoperative determined output, the intraoperative data 3000 described herein may alternatively be determined or output 4000 due to the continuous feedback loop of the data (which may be anchored by the memory system 20), and the intraoperative determined output 4000 described herein may also be used as input to the image analysis system 10. For example, some intraoperative data 3000 can be directly sensed or otherwise received, while other intraoperative data 3000 can be determined, processed, or output based on other intraoperative data 3000, preoperative data 1000, and / or stored data 30.

[0237] Like the preoperative measurement system 100, the intraoperative measurement system 300 may include an electronic medical record and / or user interface or application 340 and an imaging device 350 (e.g., an intraoperative X-ray device or a fluoroscopic device configured for intraoperative use). The intraoperative measurement system 300 may also include a robotic system 310 comprising a robotic device 142 (e.g., a surgical robot), sensors and / or devices 320 for performing intraoperative tests (e.g., range of motion tests), and a sensor-equipped implant 330 (e.g., a trial implant). Intraoperative determined outputs 4000 may include intraoperatively determined (e.g., updated) or secondary procedure times or durations 4010, procedure plans 4020, OR layouts 4030, OR scheduling 4040, assigned staff 4050, surgeon ergonomics 4070, and / or predicted outcomes 4080.

[0238] The user interface or application 340 can be used to input or update procedural information 3030, surgeon data 3040, and staff-collected data 3050 (e.g., observations during the procedure and / or other data from sensors that may not have wireless communication modules, such as conventional thermometers). Updated procedural information 3030, surgeon data 3040, and staff-collected data 3050 can be updates or improvements to preoperative data 1000 and / or newly generated. The imaging device 350 can collect image data 3080, which can be similar to the preoperative image data 1010.

[0239] Robotic device 142 may be a surgical robot, a robotic tool manipulated or held by a surgeon and / or surgical robot, or other device configured to facilitate the performance of at least a portion of a surgical procedure (such as a joint replacement procedure involving implant placement). In some examples, the surgical robot may be configured to automatically perform one or more steps of the procedure. Robotic device refers to a surgical robot system and / or robotic tool system, and is not limited to mobile or movable surgical robots. For example, robotic device may refer to a handheld robotic cutting tool, jig, drill, etc.

[0240] For ease of description, the robotic device 142 will be described as a robot configured to move in the operating room and assist staff in performing at least some of the steps of the preoperatively determined procedural plan 2020 and / or the newly generated, improved or updated procedural plan 4040 (hereinafter referred to as "intraoperatively determined procedural plan 4040").

[0241] The robotic device 142 may include or be configured to hold (e.g., via a robotic arm), move, and / or manipulate surgical tools and / or robotic instruments, such as cutting devices or blades, jigs, drills, scalpels, scissors, knives, implants, prostheses, etc. The robotic device 142 may be configured to move the robotic arm, cut tissue, cut bone, prepare tissue or bone for surgery, and / or be guided by a practitioner via the robotic arm to execute procedure plan 2020 and / or intraoperatively determined procedure plan 4040. The determined procedure plan 2020 and / or intraoperatively determined procedure plan 4040 may include instructions and / or algorithms for execution by the robotic device 142.

[0242] The robot device 142 may include and / or use various sensors (pressure sensors, temperature sensors, load sensors, strain gauge sensors, force sensors, weight sensors, current sensors, voltage sensors, positioning sensors, IMU, accelerometers, gyroscopes, positioning sensors, optical sensors, light sensors, ultrasonic sensors, acoustic sensors, infrared or IR sensors, cameras, etc.), sensor-equipped tools, cameras, or other sensors (e.g., timers, temperature, etc.) to record and / or collect robot data 3010.

[0243] The robotic system 310 and / or robotic device 142 may include one or more wheels for movement within the operating room and may include one or more motors configured to cause the wheels to spin and also manipulate surgical limbs (e.g., robotic arms, robotic hands, etc.) to manipulate surgical or robotic tools or sensors. Robotic device 142 may be a Mako SmartRobotics™ surgical robot, a ROBODOC® surgical robot, etc. However, the aspects disclosed herein are not limited to mobile robotic device 142.

[0244] The robotic device 142 can be controlled automatically and / or manually (e.g., via remote control or physical movement of the robotic device 142 or robotic arm by an operator). For example, program plan 2020 and / or intraoperatively determined program plan 4040 may include instructions configured to be executed by the processor, computer, etc., of the robotic device 142. The robotic device 142 may use machine vision (MV) technology for process control and / or guidance. The robotic device 142 may have one or more communication modules (WiFi module, Bluetooth module, NFC, etc.) and can receive updates to program plan 2020 and / or intraoperatively determined program plan 4040. Alternatively or additionally, the robotic device 142 may be configured to update program plan 2020 and / or generate new and / or intraoperatively determined program plans 4040 for execution.

[0245] Robot data 3010 may include data relating to the movement of the operating room, staff, and / or robotic device 142, the actual time spent on steps of the procedure plan 2020 and / or the procedure plan 4040 determined intraoperatively, and the actual total procedure time (e.g., compared to the determined procedure time 2010). Robot system 310 (via robotic device 142) may also collect or sense information about the performed procedure steps, such as incision length or depth, osteotomy or resection depth, or implant positioning or alignment. Robot system 310 (via robotic device 142) may also collect or sense information from the patient, such as biometric pressure, body temperature, heart rate or pulse, blood pressure, respiratory information, etc. Robot system 310 may monitor and / or store information collected using robotic device 142, and may transmit some of this information after the procedure is completed rather than during the procedure.

[0246] Other sensors and / or devices 320 may include one or more sensored surgical tools (e.g., sensored markers), wearable tools, sensors, or pads, etc. Sensors and / or devices 320 may be applied to or worn by the patient during the execution of the procedure plan 2020 and / or the procedure plan 4040 determined intraoperatively, such as wearable sensors, surgical markers, temporary surgical implants, etc. While some sensors and / or devices 320 may also be sensored implants 330 or robotic devices 142 (e.g., robotic surgical tools configured to execute instructions and / or use feedback from sensors using power tool heads), other sensors and / or devices 320 may not be strictly considered implants or robotic devices. For example, sensors and / or devices 320 may be or include tools used by medical personnel (e.g., probes, scalpels, drills, etc.) and include one or more optical sensors, load sensors, weighing sensors, strain gauge sensors, weight sensors, force sensors, temperature sensors, pressure sensors, etc.

[0247] The image analysis system 10 can use sensors and / or devices 320 to collect sensed data 3100, which may include pressure, incision length and / or location, soft tissue integrity, biometrics, etc. Additionally, the sensed data 3100 may include alignment data 3020, range of motion data (e.g., collected by the practitioner manipulating movements at or around the joint during intraoperative range of motion testing), and / or kinematic data.

[0248] One or more sensor-equipped implants 320 may include temporary or trial implants applied during the procedure and removed from the patient later in the procedure, and / or permanent implants configured to be retained for postoperative use. One or more sensor-equipped implants 320 may include implantation systems for the knee (e.g., femoral and tibial implants with a tibial stem, sensors configured to embed in the tibia and / or femur), hip (e.g., femoral implants with a femoral head having an acetabular component and / or stem), shoulder (e.g., humeral or humeral implants), spine (e.g., spinal rods or spinal screws), or other joint or limb implants, replacements, prostheses (e.g., fingers, forearms, etc.). Sensor-equipped implants 320 may include one or more load sensors, weighing sensors, force sensors, weight sensors, current sensors, voltage sensors, positioning sensors, IMUs, accelerometers, gyroscopes, optical sensors, light sensors, ultrasonic sensors, acoustic sensors, infrared or IR sensors, cameras, pressure sensors, temperature sensors, etc.

[0249] The sensor-equipped implant 320 can collect sensed data 3100 and / or alignment data 3020, such as range of motion, pressure, biometrics, implant positioning or alignment, implant type, design, or materials. The sensor-equipped implant 320 can also be configured to sense and / or monitor infection information (e.g., by sensing synovial fluid color or temperature).

[0250] The intraoperative measurement system 300 is not limited to the sensors discussed herein. For example, intraoperative data 3000 may also be collected using cameras or motion sensors installed in the operating room (e.g., cameras above the operating table, high on a wall, or on the ceiling) or sensor-equipped beds or operating tables (e.g., equipped with temperature sensors, weighing sensors, pressure sensors, positioning sensors, accelerometers, IMUs, timers, clocks, etc., to collect information about the patient's orientation or positioning and biometrics, heart rate, respiratory rate, skin temperature, skin moisture, pressure applied to the patient's skin, patient movement / activity, etc., and via wheel sensors to collect information about the movement or positioning of the bed or operating table, and / or the duration of the procedure). Additionally, intraoperative data 3000 may include prior procedure data 3090 from previous procedures with similar patients and / or similar intraoperative data 3000. Intraoperative data 3000 may include data of the same type as those in preoperative data 1000 and / or data such as operating room efficiency and / or performance, tourniquet time, blood loss, biometrics, incision length, resection depth, soft tissue integrity, pressure, range of motion or other kinematics, implant positioning or alignment, and implant type or design, but this list is not exhaustive.

[0251] As another example, cameras and / or navigation systems can be used to track operating room efficiency, pace, layout information, information about staff and / or surgeons performing the procedure plan 2020 and / or the procedure plan 4020 determined intraoperatively, and / or movement and posture patterns (e.g., measured via wearable sensors, external sensors, cameras and / or navigation systems, surgical robot 142, etc.). Based on the data 3000 collected intraoperatively, the image analysis system 10 can determine that the operating table is too high for the surgeon when determining the surgeon's ergonomics 4070, and determine a lower height for the operating table in the updated operating room layout 4030. This can improve operating room efficiency and thus reduce the determined procedure duration 4010, and can reduce fatigue for the surgeon working on the operating table.

[0252] Image analysis system 10 can execute one or more algorithms 90 to determine intraoperative output 4000 based on intraoperative data 3000, in a manner similar to how the one or more algorithms determine output 2000 based on preoperative data 1000. One or more algorithms 90 can also determine intraoperative output 4000 based on previously collected and / or stored intraoperative data 1000 and any other stored data 30 (such as previous procedural data 3090). For example, joint space width algorithm 50 can use intraoperative data 3000 to determine joint space width dimensions intraoperatively, such as determining an updated joint space width between two bones based on intraoperative data 3000, and / or determining a new joint space width when an implant (e.g., trial implant 330 and / or permanent implant 330) is applied or other corrective steps are performed in the procedure. Osteophyte detection algorithm 60 can determine osteophyte location, size, and volume, such as determining updated location and volume based on intraoperative data 3000 and / or determining new location and volume after certain steps in the procedure (e.g., when osteotomy is performed). B-score algorithm 70 can determine an updated B-score based on intraoperative data 3000, and / or a new B-score based on when the implant is applied or when other corrective steps are performed in the procedure. Alignment / deformity algorithm 80 can determine updated alignment and deformity information of the patient's bones based on intraoperative data 3000 and / or determine new alignment and deformity information after the implant is applied or after certain steps of the procedure.

[0253] Like the preoperative outputs 2000, intraoperative outputs 4000 may include surgical time 4010, procedure plan 4020, operating room layout 4030, operating room scheduling 4040, assigned staff 4050, surgeon ergonomics 4070, predicted outcome 4080, and patient anatomy representation 4090. As an example, based on complications during the procedure or due to certain information that is more readily apparent intraoperatively once tissue cutting has been performed (e.g., sutures, deformities, or infections), the image analysis system 10 may intraoperatively determine new implant designs and / or new predicted outcomes 4080 as part of the procedure plan 4020 (e.g., postoperative infection, perceived pain, stress level, anxiety level, mental health status, higher or lower risk or likelihood of cartilage loss, and / or increased case difficulty). The image analysis system 10 may update one or more GUIs 250 to accommodate the new implant model based on the newly determined implant designs and / or new predicted outcomes 4080. These intraoperative outputs 4000 may be output on the previously described output system 200.

[0254] As another example, the image analysis system 10 can determine that the procedure plan 4020 should include adjusted or additional steps, the operating room layout should be adjusted 4030, the operating room scheduling should be adjusted 4040 (and / or other appointments using some of the same staff or the same room should be adjusted), the assigned staff 4050 should include more or fewer staff, and / or the surgeon's ergonomics 4070 should include positioning suitable for longer durations.

[0255] In some cases, the image analysis system 10 can determine whether a procedure should be stopped and / or postponed to a later date based on the patient’s extreme complications and / or external factors (e.g., other emergencies in the facility, weather emergencies, etc.).

[0256] The intraoperative measurement system 300 can periodically and / or continuously sense or collect intraoperative data 3000 (arrow 303), some or all of which can be periodically and / or continuously sent to the procedure time prediction system (arrow 305). The image analysis system 10 can periodically or continuously determine the intraoperative determined output 4000 to update information, and can periodically and / or continuously send the intraoperative determined output 4000 to the output system (arrow 307).

[0257] Image analysis system 10 can periodically and / or continuously compare predicted outcome data 4080 with target or desired outcomes, and based on the comparison, further determine, update, or refine procedure duration 4010, procedure plan 4020, and / or other outputs 4000 (e.g., OR layout 4030, OR scheduling 4040, assigned staff 4050, and surgeon ergonomics 4070). Image analysis system 10 can be configured to output the comparison (e.g., via information and / or visually) to output system 200, such as one or more GUIs 250 of display 210. method

[0258] refer to Figure 2 Figure 3 and Figure 29B The exemplary method 3001 according to the embodiments can be used to determine and / or generate one or more GUIs. Method 3001 is to be exemplary only and not a complete explanation of all aspects disclosed herein. Method 3001 may include step 3002 of receiving one or more acquired images 302 of a patient's anatomical structures (e.g., legs or knees) from an imaging system having imaging device 110. Imaging device 110 may be a CT scanner, MRI scanner, X-ray machine, etc., and acquired images 302 may be CT scans, MRI scans, X-ray images, etc. Acquired images 302 may visualize the internal structures (e.g., bones and / or tissues) of the current patient. In step 3002, image analysis system 10 may receive the acquired images 302 into memory system 20.

[0259] Method 3001 may further include step 3004 of receiving patient-specific data about the current patient. Patient-specific data may include patient data and medical history 1020. For example, step 3004 may include receiving information about patient demographics, biometrics, treatment history, observations, etc., from EMR 120 and / or from input by a practitioner via interface 130 (e.g., at the time of admission). Step 3004 may also include receiving patient information directly from the current patient using, for example, an application on interface 130 on a mobile device. In step 3004, image analysis system 10 may store patient-specific data in memory system 20.

[0260] Method 3001 may further include step 3006 of receiving clinical data (such as information about the planning procedure 1030 and / or surgeon or staff data 1040). The clinical data may be entered by the practitioner or other staff into the user interface or application 130 for reception by the image analysis system 10. In step 3006, the image analysis system 10 may receive the clinical data into the storage system 20.

[0261] Method 3001 may include step 3008 of receiving prior procedure data 1050 from one or more previous patients. The prior procedure data 1050 may be input by the practitioner and received in the memory system 20, or may have already been incorporated into stored data 30 of the memory system 20. Previous patients may share at least one physical characteristic with the current patient (e.g., demographics, biometrics, disease or disease state, etc.) and may have received similar procedures to the current patient.

[0262] Method 3001 may include step 3010 of identifying, receiving, and / or selecting one or more prior models. The one or more prior models may be standard models or models obtained from healthy patients representing the same type of anatomical structure (e.g., leg or knee) as shown in the acquired image 302. Step 3010 may include identifying one or more bony landmarks in one or more received images and determining a model that includes the identified bony landmarks. Determining a prior model may also be based on received supplemental patient data, received clinical data, and / or received prior procedural data.

[0263] Method 3001 may include step 3012, which involves determining at least one of B-score, joint space width, osteophyte volume, and / or alignment or deformity data based on the acquired image 302. In step 3012, the image analysis system 10 may use one or more algorithms 90 to determine parameters associated with the B-score, joint space width, osteophyte volume, and / or alignment or deformity of at least one bone of interest. For example, the image analysis system 10 may execute B-score algorithm 70 to determine the B-score and associated parameters of the femur, execute joint space width algorithm 50 to determine the medial and / or lateral joint space width between the femur and tibia, execute osteophyte detection algorithm 60 to determine the total osteophyte volume and / or number of osteophytes detected on the femur and tibia, and execute alignment / deformity algorithm 80 to determine or detect alignment and / or deformity (e.g., varus-valgus deformity and / or tilt) at the knee joint. In step 3012, the image analysis system 10 may also use received supplemental patient data, received clinical data, and / or received prior procedural data.

[0264] Method 3001 may include step 3014 of generating an artificial model of the patient's anatomy. The artificial model may visually display the determined osteophyte volume, joint space width, B-score, and / or alignment. In step 3014, the image analysis system 10 may determine one or more modifications to be made to the determined prior model to visually depict the determined osteophyte volume, joint space width, B-score, and / or alignment, and represent the patient's anatomy shown in one or more received images.

[0265] Method 3001 may include step 3016 of generating an artificial model of a planned implant to be attached to a patient's anatomy. In step 3016, the image analysis system 10 may determine the planned implant design (e.g., size, thickness, type), and the generated artificial model may visually depict the planned implant design. The generated artificial model may be displayed separately and / or superimposed on an artificial model of the generated patient anatomy.

[0266] Method 3001 may further include step 3018 of overlaying an artificial model of the implant onto one or more acquired images (e.g., CT scans). The acquired images with the artificial model of the implant overlaid may also be displayed.

[0267] Method 3001 may include step 3019 of determining the severity of osteoarthritis progression in a patient based on a determined B-score, joint space width, osteophyte volume, and / or alignment based on imaging data. Step 3019 may include displaying the severity of OA.

[0268] One or more steps of method 3001 can be repeated (e.g., intraoperatively). For example, step 3002 can be repeated based on images acquired intraoperatively, the determinations in step 3012 can be newly determined and / or updated, and the artificial models generated in steps 3014 and 3016 can be newly determined and / or modified. Additionally, the determinations in step 3012 and the artificial models generated in steps 3014 and 3016 can be saved to a memory system (e.g., memory system 20) as prior procedural data for future patients.

[0269] The aspects disclosed in this article can be used to decide whether to proceed with surgery or pursue less invasive treatment. Figure 30 An exemplary method 3101 for determining whether to continue surgery can be shown. However, method 3101 is merely exemplary and not a comprehensive explanation of all aspects disclosed herein. For example, method 3101 does not include all possible decisions and does not include all possible image-based measurements. References Figure 30 Method 3101 for determining whether to continue surgery may include step 3102 of determining (e.g., using one or more algorithms 90 and / or image analysis system 10) whether there are signs of bone or cartilage damage. Alternatively, step 3102 may include determining whether the bone or cartilage damage exceeds a predetermined damage threshold.

[0270] If it is determined in step 3102 that there is no bone or cartilage damage and / or such damage does not exceed a predetermined damage threshold ("No" after step 3102), then method 3101 may include step 3104 of determining and / or evaluating non-surgical treatment (such as physical therapy).

[0271] If it is determined in step 3102 that there is bone or cartilage damage and / or such damage does indeed exceed a predetermined damage threshold ("Yes" after step 3102), then method 3101 may include step 3106 of determining whether the probability of surgical complications or a negative outcome is low and / or below a predetermined probability. If it is determined in step 3106 that there is no low probability and / or that probability is not less than (or alternatively, higher than) a predetermined probability ("No" after step 3106), then method 3101 may include step 3104 of proceeding to step 3104 of determining and / or evaluating non-surgical treatment. If it is determined in step 3106 that there is a low probability and / or that probability is less than (or alternatively, not higher than) a predetermined probability ("Yes" after step 3106), then method 3101 may include step 3108 of determining and / or evaluating surgical treatment options.

[0272] The aspects disclosed in this article can be used to make treatment decisions. Figure 32 An exemplary method 3200 for determining a treatment may be shown. However, this method 3200 is merely exemplary and not a comprehensive explanation of all aspects disclosed herein. For example, method 3200 does not include all possible decisions and does not include all possible image-based measurements. References Figure 31 The aspects disclosed herein can provide a method 3200 for determining treatment. Method 3200 can be performed together with or after method 3101. Method 3200 may include step 3202 of determining (e.g., using one or more algorithms 90 and / or image analysis system 10) whether bone or cartilage damage (e.g., tibia and femoral bone or cartilage damage in the context of a knee joint) exists in a specific region. Alternatively, step 3202 may include determining whether the bone or cartilage damage in the specific region exceeds a predetermined damage threshold. Step 3202 may be performed after step 3108 in method 3101. For ease of description, tibia and femoral bone or cartilage damage will be used to describe the knee joint context.

[0273] If it is determined that there is a tibiofemoral bone or cartilage injury and / or the tibiofemoral bone or cartilage injury exceeds (or is not lower than) a predetermined injury threshold ("Yes" after step 3202), then method 3200 may proceed to step 3204 to determine whether the bone or cartilage injury is limited to one region. If it is determined that the bone or cartilage injury is limited to one region ("Yes" after step 3204), then method 3200 may include step 3206 to determine that partial arthroplasty (e.g., partial knee arthroplasty) should be performed. If it is determined that the bone or cartilage injury is not limited to one region ("No" after step 3204), then method 3200 may include step 3208 to determine that total arthroplasty (e.g., total knee arthroplasty) should be performed.

[0274] If it is determined that there is no tibiofemoral bone or cartilage injury and / or the tibiofemoral bone or cartilage injury does not exceed (or is below) a predetermined injury threshold ("No" after step 3202), then method 3200 may include step 3210 of determining whether there is significant osteophyte growth and / or whether the osteophyte growth exceeds a predetermined osteophyte threshold. If it is determined in step 3210 that there is significant osteophyte growth or that the osteophyte growth exceeds the predetermined osteophyte threshold ("Yes" after step 3210), then method 3200 may include step 3212 of determining that osteotomy should be performed. If it is determined in step 3210 that there is no significant osteophyte growth or that the osteophyte growth does not exceed the predetermined osteophyte threshold ("No" after step 3210), then method 3200 may include step 3214 of reconsidering non-surgical treatment.

[0275] The aspects disclosed in this article can be used to make surgical decisions. Figure 32 An exemplary method 3300 for determining surgical decisions can be shown. However, this method 3300 is merely exemplary and not a comprehensive explanation of all aspects disclosed herein. For example, method 3300 does not include all possible decisions and does not include all possible image-based measurements. References Figure 32 The image analysis system 10 can perform various methods or decisions 3300 to arrive at a surgical decision 3302. For example, the image analysis system 10 can use one or more algorithms 90 to determine one or more decisions 3304 related to implant planning 3306. For example, one or more decisions 3304 related to implant planning 3306 may include determining the optimal size and / or location of one or more implants to be used during the procedure, but the aspects disclosed herein are not limited to size and location. For example, one or more decisions 3304 may include determining the type, material, thickness, etc., of one or more implants. The image analysis system 10 can use one or more algorithms 90 to determine one or more decisions 3308 related to surgical scheduling 3310. For example, one or more decisions 3308 related to implant planning 3310 may include determining the predicted procedure or surgical time or duration, but the aspects disclosed herein are not limited to size and location. For example, one or more decisions 3308 may include determining room assignment, date and time, location or place, etc. The image analysis system 10 can use one or more algorithms 90 to determine one or more decisions 3312 related to staff and equipment management 3314. For example, one or more decisions 3308 related to implantation planning 3310 may include determining the complexity of the procedure or surgery, but the aspects disclosed herein are not limited to size and location. For example, one or more decisions 3308 may include determining staff, practitioners, surgical instruments, etc.

[0276] refer to Figure 33The aspects disclosed herein can be used to make treatment, surgical, and / or clinical decisions based on the determinations made by the image analysis system 10 and / or multiple GUIs 250. For example, the determination and display of B-scores and C-scores can be used to make clinical decisions 3402 that compare treatment options. Figure 33 An exemplary clinical decision 3402 may be presented, rather than a comprehensive explanation of all aspects disclosed herein.

[0277] Image analysis system 10 (e.g., via B-score algorithm 70) can determine the B-score and display of a patient's femur (e.g., 12th GUI 270, 13th GUI 272, 14th GUI 274, and / or B-score video 1606). The determined B-score, along with the associated GUI and / or display, can help practitioners assess the severity 3406 of osteoarthritis. In some examples, image analysis system 10 may use one or more algorithms 90 to determine the severity 3406 of osteoarthritis.

[0278] Additionally, the image analysis system 10 (e.g., via joint space width algorithm 50) can determine the C-score and display of the patient's femur (e.g., sixth GUI 258, seventh GUI 260, eighth GUI 262, and / or a display of cartilage loss 606 with a gradient scale 608). The determined C-score, along with the associated GUI and / or display, can help practitioners assess the severity and / or location of cartilage loss 3410 (and / or predict cartilage loss). In some examples, the image analysis system 10 may use one or more algorithms 90 to determine cartilage loss 3410. The assessment performed by the practitioner can be used to make clinical decisions 3402, such as whether the patient will benefit more from total knee arthroplasty or partial knee arthroplasty. In some examples, the image analysis system 10 may automatically make clinical decisions 3402 based on the determination of osteoarthritis severity 3406 and / or cartilage loss 3410. Prediction Program

[0279] Refer again Figure 2As described above, the electronic data processing system 1 may include one or more preoperative measurement systems 100 operable to collect and / or output preoperative data 1000 about current and / or previous patients. The image analysis system 10 may receive and analyze the preoperative data 1000 to generate one or more outputs or determinations 2000, which may then be output to one or more output systems 200. For example, as described above, image data 1010 may be generated or otherwise collected by the imaging device 110, and patient and / or medical history data 1020 may be generated or otherwise collected by the patient or practitioner. The image analysis system 10 may then use one or more algorithms 90 (e.g., joint space width algorithm 50, osteophyte detection algorithm 60, B-score algorithm 70, or alignment / deformity algorithm 80) to process the image data 1010 and / or patient and / or medical history data 1020 to generate one or more outputs 2000, such as procedure time 2010, procedure plan 2020, or predicted outcome 2080. The image analysis system 10 can generate preoperative outputs, intraoperative outputs, and / or postoperative outputs, which patients and / or practitioners can use to improve patient outcomes.

[0280] like Figure 2 As depicted, in some instances, one or more algorithms 90 include a predictive procedure algorithm 85, using which the image analysis system 10 can generate a predictive procedure 2005 based on image data 1010, patient and / or medical history data 1020, and / or previous procedure data 1050 (e.g., predictive procedures and / or confirmed procedures). This predictive procedure can be output to an output system 200 and used by a patient or practitioner to determine which type of procedure the patient should receive. The predictive procedure 2005 can be a predicted or recommended procedure type for the patient. The predictive procedure 2005 can be generated based on image data 1010, one or more outputs of one or more algorithms 90, patient data and / or medical history 1020, preoperative data 1000, previous procedure data 1050, one or more outputs 2000, or any combination thereof. In some instances, the predictive procedure 2005 may be selected from a set of similar or related procedures, such as partial knee arthroplasty and total knee arthroplasty, and / or medial partial knee arthroplasty (mPKA) and lateral partial knee arthroplasty (lPKA). However, the predictive procedure 2005 may be selected from any suitable set of procedures, or from an unrestricted set of procedures. The predictive procedure 2005 may include the procedure plan 2020 as described above.

[0281] refer to Figure 2 and Figure 37The prediction algorithm 85 can generate a prediction program 2005 for the patient based on various inputs. The prediction algorithm 85 can take image data 1010 as input, which includes one or more acquired images 302 (as described above), patient and / or medical history data 1020, and / or one or more outputs 2000 of one or more algorithms 90. For example, patient and / or medical history data 1020 may include identity information (e.g., name, date of birth, nationality, etc.), demographic information (e.g., patient age, sex, height, weight, BMI, etc.), lifestyle information (e.g., smoking habits, exercise habits, drinking habits, dietary habits, etc.), medical history information (e.g., history of allergies, disease progression, history of addiction, history of previous medication use, history of previous drug use, history of previous infections, frailty, comorbidities, history of previous surgeries or treatments, history of previous injuries, history of previous pregnancies, use of orthotics, braces, prostheses or other medical devices, etc.), or any other suitable information as described above. One or more outputs 2000 of one or more algorithms 90 may include one or more B-scores, one or more C-scores, etc., which may be determined or generated by one or more algorithms 90 using image data 1010, as described above. In some instances, one or more algorithms 90 may be able to determine or generate a patient's calibrated bone mineral density (BMD), and the predictive algorithm 85 may take this BMD into account when generating the predictive algorithm 85. For example, a patient's BMD may influence whether partial knee arthroplasty (PKA) or total knee arthroplasty (TKA) is more suitable for the patient, or whether the implant used for the patient in PKA or TKA should be a cemented or uncemented implant.

[0282] The predictor algorithm 85 can be a machine learning algorithm trained using, for example, linear regression, random forest regression, CatBoost regression, statistical shape modeling (SSM), etc. Alternatively, for example, the predictor algorithm 85 can be a model generated or otherwise produced by a machine learning or artificial intelligence algorithm or model. Figure 37 The prediction algorithm 85 is described as a decision tree model 3800 generated by a machine learning algorithm. Figure 37In the depicted example, the first node 3801 of the decision tree model 3800 considers the patient's medial tibiofemoral C-score (“MED TF C-score”). In this example, if the patient's medial tibiofemoral C-score is greater than -1, the final output of the decision tree model 3800 indicates a predictive procedure 2005 indicating that the patient is very likely to require total knee arthroplasty (TKA). However, in this example, if the patient's medial tibiofemoral C-score is less than or equal to -1, the decision tree model 3800 continues to the second node 3802, where the patient's B-score is considered. If the patient's B-score is less than or equal to 1, the decision tree model 3800 continues to the third node 3803, where the patient's BMI is considered. If the patient's BMI is less than or equal to 40, the decision tree model 3800 ultimately outputs a predictive procedure 2005 indicating that the patient is very likely to require partial knee arthroplasty (PKA); if the patient's BMI is greater than 40, the decision tree model 3800 ultimately outputs a predictive procedure 2005 indicating that the patient is very likely to require total knee arthroplasty (TKA). However, in this example, if the patient's B-score is greater than 1, the decision tree model 3800 continues to the fourth node 3804, where the patient's age is considered. If the patient's age is less than or equal to 80 years, the decision tree model 3800 ultimately outputs a predictive procedure 2005 indicating that the patient is very likely to require PKA; if the patient's age is greater than 80 years, the decision tree model 3800 ultimately outputs a predictive procedure 2005 indicating that the patient is very likely to require TKA.

[0283] Figure 37 The decision tree model 3800 depicted herein is merely one example of a predictor algorithm 85 and should not be considered as limiting in any way. For example, a predictor algorithm 85 implemented as a decision tree model may include any number of nodes that consider any type of information. For example, a predictor algorithm 85 implemented as a decision tree model may additionally or alternatively include nodes that consider a patient's height or weight, a patient's gender, a patient's nationality, a patient's lateral tibiofemoral C-score, a patient's medial patellofemoral C-score, a patient's lateral patellofemoral C-score, or any other node that considers any other type of information. For example, a single node in a predictor algorithm 85 implemented as a decision tree model may simultaneously consider two or more types of information. Although... Figure 37 The decision tree model 3800 is constrained to output PKA or TKA, but the predictor algorithm 85 implemented as the decision tree model can be constrained to output one of any number of programs, or can be unconstrained, such that the decision tree model can output any appropriate program.

[0284] In some instances, the predictor algorithm 85 is implemented as a logistic regression model generated by or otherwise produced by a machine learning or artificial intelligence algorithm or model. For example, it is implemented as 1 / (1+e - func The predictor algorithm 85 is a logistic regression model in the form of a formula or function that includes one or more variables. These variables may include the patient's age, height or weight, sex, B-score, one or more C-scores, etc. The variables included in the logistic regression model may be normalized over a dataset of similar values, for example, such that the mean of any normalized variable is zero, with a standard deviation of one. Each of the variables may be assigned a weight (e.g., a multiplier) such that some variables have a greater influence in the logistic regression model than others. For example, the B-score or C-score included in one or more variables may be given a greater weight than any other variable included in one or more variables. In one example, if the result of the logistic regression model is rounded to one, the predictor algorithm 85 outputs a predictor 2005 indicating that the current patient is very likely to require PKA (e.g., mPKA); if the result of the logistic regression model is rounded to zero, the predictor algorithm 85 outputs a predictor 2005 indicating that the current patient is very likely to require TKA. In some instances, the predictor algorithm 85, implemented as a logistic regression model, includes a B-score and at least one C-score as variables. In some instances, the predictor algorithm 85, implemented as a logistic regression model, includes multiple C-scores (e.g., two or more of the following: medial tibiofemoral C-score, lateral tibiofemoral C-score, medial patellofemoral C-score, and lateral patellofemoral C-score) as variables.

[0285] In some instances, when generating prediction program 2005, prediction program algorithm 85 can be configured to select a program from a set of programs that include only two programs. For example, in some instances, when generating prediction program 2005, prediction program algorithm 85 can be configured to select a program from a set of programs that include only TKA and mPKA. In such instances, prediction program algorithm 85 can prioritize factors more closely associated with mPKA and / or de-prioritize factors less closely associated with mPKA, such as by prioritizing one or more C-scores over B-scores, or by prioritizing one or more C-scores over one or more other C-scores. For example, if prediction program algorithm 85 is implemented as a decision tree model (as described above), the first node of the decision tree model can consider a C-score (e.g., medial tibiofemoral C-score) before any subsequent node considers a B-score, or the first node of the decision tree model can consider a medial tibiofemoral C-score before any subsequent node considers any other C-score. Alternatively, for example, if the predictor algorithm 85 is implemented as a logistic regression model (as described above), the logistic regression model may assign greater weight to the C-score (e.g., medial tibiofemoral C-score) than to the B-score, or assign greater weight to the medial tibiofemoral C-score and medial patellofemoral C-score than to the lateral tibiofemoral C-score and lateral patellofemoral C-score. However, the predictor algorithm 85 may prioritize factors and / or de-prioritize factors in any other manner.

[0286] In some instances, when generating a predictive procedure 2005 for a patient, the predictive procedure algorithm 85 may consider one or more predictive procedures 2005 generated for one or more previous patients and / or one or more corresponding verified procedures 1055 received for those one or more previous patients. For example, the predictive procedure algorithm 85 may be a machine learning algorithm, or be generated by a machine learning algorithm trained on a program database including multiple predictive procedures 2005 generated for previous patients and multiple corresponding verified procedures 1055 received for those previous patients. Each entry in the program database may include a predictive procedure 2005 generated for a previous patient, any information used in generating the predictive procedure 2005, and a verified procedure 1055 received for the previous patient. In some instances, after generating and outputting the predictive procedure 2005 for a patient, the image analysis system 10 may receive the verified procedure 1055 for the patient, for example, through a GUI in which the predictive procedure 2005 is displayed or previously displayed, as described in further detail below. The predictor 2005 and the corresponding validated procedure 1055 can then be fed back into the procedure database to further train the predictor algorithm 85. For example, by training the predictor algorithm 85 on such a procedure database, the predictor algorithm 85 can establish various correlations or associations between individual types or grouped information or variables and various types of procedures. The predictor algorithm 85 can then use these correlations and / or associations to generate a more accurate predictor 2005. For example, based on the information included in the procedure database, different or additional nodes can be included in the predictor algorithm 85, which is implemented as a decision tree model as described above. For example, based on the information included in the procedure database, different or additional variables can be included in the predictor algorithm 85, which is implemented as a logistic regression model, or different weights can be assigned to one or more variables included in the logistic regression model.

[0287] Figure 38 A GUI 3900, which is provided by or otherwise communicatively coupled to the image analysis system 10, is shown, similar to... Figure 4E GUI 255. GUI 3900 displays an artificial knee model 402 with osteophyte indicator 404, as described above. In this example, GUI 3900 includes, in addition to the B-scoring component 414 (as described above) and the C-scoring component 416 (as described above), a prediction component 3902 and a confirmation component 3904. Figure 38 As shown, the B-score widget 414 can display various information about the B-score determined for the patient (e.g., determined by the B-score algorithm 70), such as a skeletal image representing the B-score or a gradient bar indicating the B-score against a scale background, as described above. Figure 38 As shown, the C-score widget 416 can display various information about the C-score determined for the patient (e.g., determined by the joint space width algorithm 50), such as a skeletal image representing the C-score or a gradient bar indicating the C-score against a scale background, as described above. Similarly, as Figure 38 As shown, the prediction program component 3902 displays various information about the prediction program 2005 generated for the patient, such as one or more aspects of the program plan 2020 (e.g., recommended implants). Selecting the prediction program component 3902 can allow a user of the GUI 3900 (e.g., a practitioner) to view additional information about the prediction program 2005, such as one or more factors (e.g., variables) that contribute most significantly to the generation of the prediction program 2005. The selection of the prediction program component 3902 can also, alternatively, allow a user of the GUI 3900 to manipulate the prediction program 2005, such as by increasing / decreasing or turning on / off one or more factors (e.g., variables) used in generating the prediction program 2005. Doing so allows the image analysis system 10 to generate new or updated prediction programs 2005 for the patient and for the new or updated prediction programs 2005 to be displayed within the GUI 3900.

[0288] like Figure 38 As shown, the verified procedure component 3904 allows a user of the GUI 3900 (e.g., a practitioner) to submit a verified procedure 1055 for a patient to the image analysis system 10. For example, after the image analysis system 10 generates a predictive procedure 2005 for the patient (e.g., using predictive procedure algorithm 85, as described above) and outputs the predictive procedure 2005 for display within the GUI (e.g., GUI 3900), the GUI user (e.g., a practitioner) can refer to the predictive procedure 2005 to determine the procedure that should be performed for the patient. After determining the procedure to be performed for the patient, regardless of whether the procedure is the same as the predictive procedure 2005, the GUI user can submit the procedure as a verified procedure 1055 to the image analysis system 10, for example, through the verified procedure component 3904. The image analysis system 10 can then use the predictive procedure 2005 and the corresponding verified procedure 1055 to further train the predictive procedure algorithm 85, as described above. For example, as... Figure 38As shown, the image analysis system 10 has generated a predictive procedure 2005 indicating that the patient is likely to require partial knee arthroplasty (PKA), and outputs the predictive procedure 2005 for display within the predictive procedure widget 3902 of the GUI 3900. In this example, after determining the procedure to be performed for the patient (e.g., PKA or total knee arthroplasty (TKA)), the user of the GUI 3900 can submit a confirmed procedure 1055 to the image analysis system 10 by selecting PKA or TKA accordingly within the confirmed procedure widget 3904.

[0289] refer to Figure 2 and Figure 39 According to some embodiments, method 4000 is an exemplary method for generating prediction program 2005. Method 4000 is intended to be exemplary only and not a comprehensive explanation of all aspects disclosed herein. Although the steps of method 4000 are depicted in a specific order, it should be understood that the steps of method 4000 can occur in any order, and not just in a specific order. Figure 39 The order described in the document. Similarly, one or more steps of method 4000 can occur simultaneously.

[0290] like Figure 39 As depicted, method 4000 may begin at step 4002, in which image analysis system 10 receives image data 1010, which includes one or more acquired images 302 of patient anatomy, such as one or more images of the patient's knee joint acquired by imaging device 110. Imaging device 110 may be a CT scanner, MRI scanner, X-ray machine, etc., and the acquired images 302 may be CT scans, MRI scans, X-ray images, etc. The acquired images 302 can visualize the internal structures (e.g., bones and / or tissues) of the current patient. Image analysis system 10 may store image data 1010 in memory system 20.

[0291] like Figure 39 As described, after the image analysis system 10 receives the image data 1010, method 4000 may continue to step 4004, in which the image analysis system 10 determines at least one parameter of the patient's anatomy based on the image data. The at least one parameter may include a B-score or a C-score as described above. The at least one parameter may include both a B-score and a C-score. The at least one parameter may include multiple C-scores, such as two or more of the following: medial tibiofemoral C-score, lateral tibiofemoral C-score, medial patellofemoral C-score, and lateral patellofemoral C-score. The image analysis system 10 may use one or more algorithms 90 (such as B-score algorithm 70 or joint space width algorithm 50) to determine the at least one parameter, as described above.

[0292] like Figure 39 As described, after the image analysis system 10 determines at least one parameter of the patient's anatomy, method 4000 can proceed to step 4006, in which the image analysis system 10 receives patient and / or medical history data 1020. For example, patient and / or medical history data 1020 may include patient demographics, biometrics, treatment history, observations, etc., from EMR 120 and / or from input by a practitioner via interface 130 (e.g., at the time of admission). Step 4006 may also include receiving patient information directly from the current patient using, for example, an application on interface 130 on a mobile device. In step 4006, the image analysis system 10 may store patient and / or medical history data 1020 in memory system 20. However, in some instances, method 4000 may be performed without step 4006 (e.g., method 4000 may proceed directly from step 4004 to step 4008, as described below). For example, the prediction program can be derived solely from data obtained via one or more medical images (e.g., acquired image 302) and associated parameters obtained from one or more medical images.

[0293] like Figure 39 As described, after the image analysis system 10 receives patient and / or medical history data 1020, method 4000 may continue to step 4008, in which the image analysis system 10 generates a predictive procedure 2005 for the patient based on at least one parameter of the patient's anatomy and the patient and / or medical history data 1020. For example, the image analysis system 10 may use a predictive procedure algorithm 85 to generate the predictive procedure 2005 for the patient, as described above. The predictive procedure algorithm 85 may be generated by a machine learning model, as described above. In some instances, the predictive procedure algorithm 85 is a decision tree model or a logistic regression model. In some instances, when generating the predictive procedure 2005, the image analysis system 10 normalizes at least one parameter and / or the patient and / or medical history data 1020. In some instances, the predictive procedure 2005 may be partial knee arthroplasty (PKA) or total knee arthroplasty (TKA). In some instances, prediction program 2005 is generated at least in part using multiple predictive procedures generated for previous patients and corresponding multiple validated procedures 1055 received by image analysis system 10 for previous patients, as described above. In some instances, prediction program 2005 includes a procedure plan 2020, such as implant type, as described above. In some instances, in step 4008, image analysis system 10 may generate prediction program 2005 without using patient and / or medical history data 1020 (e.g., in instances where method 4000 proceeds directly from step 4004 to step 4008).

[0294] like Figure 39As described, after the image analysis system 10 generates the prediction program 2005, method 4000 can continue to step 4010, in which the image analysis system 10 outputs the prediction program 2005 generated for the patient for display within a graphical user interface (GUI). In some instances, the image analysis system 10 can receive a verified program 1055 corresponding to the prediction program 2005 via the GUI, as described above. The image analysis system 10 can then use the prediction program 2005 and the corresponding verified program to further train the prediction program algorithm 85, as described above.

[0295] refer to Figure 2 and Figure 44 According to some embodiments, method 4500 is an exemplary method for generating prediction program 2005. Method 4500 may be exemplary only and not a complete explanation of all aspects disclosed herein. Although the steps of method 4500 are depicted in a specific order, it should be understood that the steps of method 4500 can occur in any order, and not just in a specific order. Figure 44 The order described in the document. Similarly, one or more steps of method 4500 can occur simultaneously.

[0296] like Figure 44 As depicted, method 4500 may begin at step 4502, in which image analysis system 10 receives image data 1010, which includes one or more acquired images 302 of patient anatomy, such as one or more images of the patient's knee joint acquired by imaging device 110. Imaging device 110 may be a CT scanner, MRI scanner, X-ray machine, or any other suitable imaging device, and the acquired images 302 may be CT scans, MRI scans, X-ray images, and / or any other type of image. The acquired images 302 may visualize the internal structures (e.g., bones and / or tissues) of the current patient. Image analysis system 10 may store image data 1010 within memory system 20.

[0297] like Figure 44As depicted, after the image analysis system 10 receives the image data 1010, method 4500 may continue to step 4504, in which the image analysis system 10 uses the image data 1010 (e.g., from one or more acquired images 302 included in the image data 1010) to identify or determine the strength of at least one of the following: bone 4112, reference material included in the calibration device (e.g., a single reference material included in the movement bar 4110 as described above), and fat 4114 and air 4116. In some instances, identifying or determining the strength of one or more of the following: bone 4112, reference material included in the calibration device, and fat 4114 and air 4116 includes using a shape model to identify at least one of the following: bone 4112, reference material included in the calibration device, and / or fat 4114 and air 4116, as described above.

[0298] like Figure 44 As depicted, after the image analysis system 10 identifies or determines the strength of bone 4112, reference material included in the calibration device, and at least one of fat 4114 and air 4116, method 4500 may continue to step 4506, in which the image analysis system 10 determines the bone mineral density (BMD) of bone 4112 based on the strength of bone 4112, reference material included in the calibration device, and / or at least one of fat 4114 and air 4116. In some instances, determining the BMD of bone 4112 includes determining the BMD equivalent value of reference material included in the calibration device and at least one of fat 4114 and air 4116, determining the BMD calibration factor (BCF) using the BMD equivalent value of reference material included in the calibration device and at least one of fat 4114 and air 4116, and determining the BMD of bone 4112 using the BCF and the strength of bone 4112, as described above. The BMD equivalent value of the reference material included in the calibration device and at least one of fat 4114 and air 4116 may be predetermined or otherwise known to the image analysis system 10.

[0299] like Figure 44As depicted, after the image analysis system 10 determines the BMD of the bone 4112, method 4500 can proceed to step 4508, in which the image analysis system 10 receives patient and / or medical history data 1020. For example, patient and / or medical history data 1020 may include patient demographics, biometrics, treatment history, observations, etc., from inputs made by the practitioner via interface 130 (e.g., at the time of admission) from EMR 120 and / or from other sources. Step 4508 may also include receiving patient information directly from the current patient using, for example, an application on interface 130 on a mobile device. In step 4508, the image analysis system 10 may store the patient and / or medical history data 1020 in memory system 20. However, in some instances, method 4500 may be performed without step 4508 (e.g., method 4500 may proceed directly from step 4506 to step 4510, as described below).

[0300] like Figure 44As described, after the image analysis system 10 receives patient and / or medical history data 1020, method 4500 may continue to step 4510, in which the image analysis system 10 generates a predictive procedure 2005 for the patient based on the BMD of the bone 4112 and the patient and / or medical history data 1020. For example, the image analysis system 10 may use a predictive procedure algorithm 85 to generate the predictive procedure 2005 for the patient, as described above. The predictive procedure algorithm 85 may be generated by a machine learning model, as described above. In some instances, the predictive procedure algorithm 85 is a decision tree model or a logistic regression model. In some instances, when generating the predictive procedure 2005, the image analysis system 10 normalizes at least one parameter and / or the patient and / or medical history data 1020. In some instances, the predictive procedure 2005 may be partial knee arthroplasty (PKA) or total knee arthroplasty (TKA). In some instances, the predictive procedure 2005 may include implant fixation techniques, such as cemented, uncemented, or hybrid implants. In some examples, the prediction procedure 2005 may include an indication of the type and size of the implant stem, or an indication of whether an additional stem extension is needed or recommended based on BMD and / or other parameters discussed herein. For example, if a patient has a lower mean BMD compared to the mean of previous patients with one or more common attributes, a larger implant stem and / or an additional stem extension may be recommended for such a patient. In some instances, the prediction procedure 2005 is generated at least in part using multiple prediction procedures generated for previous patients and corresponding multiple validated procedures 1055 received by the image analysis system 10 for the previous patients, as described above. In some instances, the prediction procedure 2005 includes a procedure plan 2020, such as the implant type, as described above. In some instances, the method 4500 also includes determining at least one parameter of the patient's anatomy (including at least one of a B-score or a C-score (as described above)), and generating the prediction procedure 2005 at least in part based on the BMD of the bone 4112 and at least one parameter of the patient's anatomy. At least one parameter of the patient's anatomy may include B-score and C-score, or multiple C-scores (e.g., two or more of the medial tibiofemoral C-score, lateral tibiofemoral C-score, medial patellofemoral C-score, and lateral patellofemoral C-score). In some instances, in step 4510, the image analysis system 10 may generate a prediction procedure 2005 without using patient and / or medical history data 1020 (e.g., in instances where method 4500 proceeds directly from step 4506 to step 4510).

[0301] like Figure 44As described, after the image analysis system 10 generates the prediction program 2005, method 4500 can continue to step 4512, in which the image analysis system 10 outputs the prediction program 2005 generated for the patient for display within a graphical user interface (GUI). In some instances, the image analysis system 10 can receive a verified program 1055 corresponding to the prediction program 2005 via the GUI, as described above. The image analysis system 10 can then use the prediction program 2005 and the corresponding verified program to further train the prediction program algorithm 85, as described above. In some instances, method 4500 may also include generating an artificial (e.g., three-dimensional) model 402 of the patient's anatomical structure (e.g., skeleton 4112) and outputting the artificial model for display within the GUI. The artificial model 402 of the patient's anatomical structure may represent the bone marrow diameter (BMD) of the skeleton 4112.

[0302] The aspects disclosed herein can be used to sense or collect preoperative, intraoperative, and / or postoperative information about patients and / or procedures.

[0303] The aspects disclosed herein envision implants or prostheses and are not limited to the described context. For example, the implants disclosed herein may be implemented as another implant system for another joint or other part of the musculoskeletal system (e.g., hip, knee, spine, bone, ankle, wrist, finger, hand, toe, or elbow), and / or implemented as sensors configured for direct implantation into patient tissues, bones, muscles, ligaments, etc. Each of the implants or implant systems may include sensors, such as inertial measurement units, strain gauges, accelerometers, ultrasonic or acoustic sensors, etc., configured to measure positioning, velocity, acceleration, orientation, range of motion, etc. Additionally, each of the implants or implant systems may include sensors that detect changes in synovial fluid, blood glucose, temperature, or other biomarkers (e.g., color changes, pH changes, etc.), and / or may include electrodes that detect electrical information, ultrasonic or infrared sensors that detect other nearby structures to detect infection, invasion, nearby tumors, etc. In some examples, each of the implants and / or implant systems may include a transmissive region, such as a transparent window on the outer surface of a prosthesis system, configured to allow radiofrequency energy to pass through the transmissive region. The IMU may include three gyroscopes and three accelerometers. The IMU may include a microelectromechanical (MEMS) integrated circuit. The implants and / or implant systems disclosed herein may also be implemented as implantable navigation systems. For example, the implant may primarily have sensing capabilities rather than joint replacement capabilities. The implant may be, for example, a sensor or other measuring device configured to drill into bone, into another implant, or otherwise implanted in a patient.

[0304] The implants, implant systems, and / or measurement systems disclosed herein may include strain gauge sensors, optical sensors, pressure sensors, weighing / load sensors, ultrasonic sensors, acoustic sensors, resistive sensors (including electrical transducers that convert mechanical measurements or responses (e.g., displacement) into electrical signals), and / or sensors configured to sense synovial fluid, blood glucose, heart rate variability, sleep disorders, and / or detect infection. Measurement data from the IMU and / or other sensors may be transmitted to a computer or other device in the system for processing and / or display of alignment, range of motion, and / or other information from the IMU. For example, measurement data from the IMU and / or other sensors may be wirelessly transmitted to a computer or other electronic device external to the patient's body for processing (e.g., via one or more algorithms) and display on an electronic display.

[0305] The aspects and systems disclosed herein can be determined based on images or imaging data (e.g., from CT scans). The images disclosed herein can display or represent bones, tissues, or other anatomical structures, and the various systems and aspects disclosed herein can identify, recognize, classify, and / or determine portions of anatomical structures, such as bones, cartilage, tissues, and bony landmarks, such as each specific vertebra in the spine. The aspects and systems disclosed herein can determine the relative positioning, orientation, and / or angles between identified bones, such as the Cobb angle, the angle between the tibia and femur, and / or other alignment data.

[0306] The aspects and systems disclosed herein provide displays with graphical user interfaces configured to graphically display data, determinations, and / or procedural steps, objectives, instructions, or other parameters, including preoperatively, intraoperatively, and / or postoperatively. Graphics, diagrams, animations, and / or videos displayed via the user interface can be recorded and stored on a memory system.

[0307] The aspects and systems disclosed herein can be implemented using machine learning techniques. One or more algorithms can be configured to learn patterns and / or other relationships across multiple patients by combining preoperative information and outputs, intraoperative information and outputs, and postoperative information and outputs, or to be trained on these patterns and / or other relationships. The learned patterns and / or relationships can refine the determinations made by one or more algorithms, and / or further refine how the one or more algorithms are executed, configured, designed, or compiled. The refinement and / or updating of the one or more algorithms can further refine the display and / or graphical user interface (e.g., bone recognition and / or determination, target, identification and / or display of other conditions and / or bone displacements, etc.).

[0308] The aspects disclosed herein can be configured to optimize the “fit” or “tightness” of implants delivered to patients during medical procedures based on detections performed by one or more algorithms. A tighter fit can be achieved by aligning the implant with a shallower bone slope and / or determining a shallower, desired bone slope, by increasing the thickness or other dimensions of the implant, or by determining certain types of materials or implant or prosthesis types (e.g., stable implants, VVC implants, ADM implants, or MDM implants). Implant thickness can be achieved by increasing (or decreasing) the size or shape of the implant. Tightness may be affected by the gap and / or joint space width, which can be adjusted by padding that may vary depending on the implant type or due to movement. The gap may be affected by femoral and tibial cuts. Tightness may be further affected by the slope. The slope range can be based on implant selection, surgical approach, and patient anatomy. Implant thickness can also be achieved by adding or removing augmentations or pads. For example, the patch or shim can be stackable and removable, and the thickness can be increased by adding one or more patches or shims or by adding patches or shims with a predetermined thickness (e.g., above a certain threshold). Fit or tightness can also be achieved through certain types of osteotomy, bone preparation, or tissue cutting, which reduce the number of cuts and / or invasiveness performed during surgery.

[0309] The aspects disclosed herein can be implemented during robotic medical procedures using robotic devices. The aspects disclosed herein are not limited to the specific scores, thresholds, etc., described herein. For example, the outputs and / or scores disclosed herein may include other types of scores, such as hip disability and osteoarthritis scores or HOOS, KOOS, SF-12, SF-36, Harris hip scores, etc.

[0310] The aspects disclosed herein are not limited to specific types of surgery, but can be applied in the context of osteotomy procedures, computer-guided surgery, neurosurgery, spinal surgery, otolaryngology surgery, orthopedic surgery, general surgery, urology, ophthalmology, obstetrics and gynecology, plastic surgery, valve replacement surgery, endoscopic surgery, and / or laparoscopic surgery.

[0311] The aspects disclosed herein can improve or optimize surgical outcomes, implant design and / or preoperative analysis, prediction, or workflow. The aspects disclosed herein can enhance continuity of care to optimize postoperative patient outcomes. The aspects disclosed herein can identify or determine previously unknown relationships to help optimize care, predict cartilage loss or other future joint damage, and / or optimize prosthesis design.

Claims

1. A method for predicting the type of surgery for a patient, the method comprising: Receive image data including at least one image of a patient’s anatomical structure acquired from an image, the at least one image showing (i) at least one of air and / or fat, (ii) bone, and (iii) a movement rod; The image data is used to identify the strength of at least one of the bone, the motion bar, and the air and the fat; The bone mineral density (BMD) of the bone is determined based on the strength of the bone, the movement rod, and at least one of the air and the fat. Receive patient data about the patient; A predictive procedure is generated for the patient, based at least in part on the bone's BMD and the patient's data. as well as The prediction program is output for display within a graphical user interface (GUI).

2. The method as described in claim 1, wherein, Determining the BMD of the bone includes: Determine the BMD equivalent value of the movement rod and at least one of the air and the fat; A BMD calibration factor is determined based on (a) the strength of the bone, the motion rod, and at least one of the air and the fat, and (b) the BMD equivalent value of the motion rod and at least one of the air and the fat; and The BMD of the bone is determined based on the BMD calibration factor and the strength of the bone.

3. The method of claim 2, further comprising: The image data is used to identify the strength of the bones, the motion rod, and the air and / or the fat; Determine the BMD equivalent values ​​of the motion rod and the air and / or the fat; as well as The BMD calibration factor is determined based on the strength of the motion bar, the air, and / or the fat, and the BMD equivalent value.

4. The method of claim 2, wherein, The BMD equivalent value of the exercise bar and at least one of the air and the fat is predetermined.

5. The method of claim 1, further comprising automatically identifying from the at least one image the motion bar, the bone, and at least one of the air and the fat.

6. The method of claim 5, wherein, The movement rod, the bone, and at least one of the air and the fat are automatically identified from the at least one image using a shape model.

7. The method of claim 1, further comprising generating a three-dimensional model of the skeleton and outputting the three-dimensional model of the skeleton for display together with the prediction program.

8. The method of claim 7, wherein, The three-dimensional model of the skeleton shows the determined BMD of the skeleton.

9. The method of claim 1, further comprising using the imaging data to determine at least one parameter of the patient's anatomy, the at least one parameter including at least one of a B-score and a C-score, and generating the prediction procedure for the patient based at least in part on the determined BMD of the skeleton, the patient data, and the at least one parameter.

10. The method of claim 8, further comprising outputting the determined BMD of the bone and the at least one parameter for display together with the prediction program.

11. The method of claim 8, wherein, The at least one parameter includes a B-score and a C-score.

12. The method of claim 8, wherein, The at least one parameter includes multiple C-scores.

13. The method of claim 10, wherein, The multiple C-scores include two or more of the following: medial tibiofemoral C-score, lateral tibiofemoral C-score, medial patellofemoral C-score, and lateral patellofemoral C-score. The method of claim 1, wherein the prediction procedure is partial knee arthroplasty or total knee arthroplasty.

14. The method of claim 1, wherein, The at least one image was taken from a computed tomography (CT) scan or a magnetic resonance imaging (MRI) scan.

15. The method of claim 1, wherein, The movement bar is a bar connected to the patient's anatomical structure.

16. The method of claim 16, wherein, The moving rod is made of aluminum.

17. The method of claim 16, further comprising automatically identifying the motion rod from within the at least one image using a shape model.

18. The method of claim 1, wherein, The prediction procedure is generated at least in part using a machine learning model trained with multiple prediction procedures generated for previous patients and corresponding multiple validated procedures received by the previous patients.

19. The method of claim 19, further comprising receiving a validated procedure for the patient and using the prediction procedure and the validated procedure to further train the machine learning model.

Images