Patient specific instrumentation automation platform

Abstract

There is provided a computer implemented method, comprising: automatically detecting at least one target region of a surface of at least one bone and / or cartilage on a 3D representation of the subject, defined by a surgical plan defining a surgical indication, automatically computing a pose of at least one surgical tool for performing the surgical action defined by the surgical plan, automatically generating a graphical 3D representation of the PSI according to the surgical plan, the graphical 3D representation of the PSI including a footprint region having a surface corresponding to the at least one target region for contact with the surface of the at least one bone and / or cartilage corresponding to the at least one target region, and for performing the surgical action according to the pose, and automatically generating instructions ready for 3D printing of the PSI according to the graphical 3D representation.

Description

[0001] PATIENT SPECIFIC INSTRUMENTATION AUTOMATION PLATFORM

[0002] RELATED APPLICATION

[0003] This application claims the benefit of priority of U.S. Provisional Patent Application Nos. 63 / 472,392, filed on June 12, 2023; 63 / 472,397, filed June 12, 2023 and 63 / 472,602, filed on June 13, 2023, the contents of which are incorporated herein by reference in their entirety.

[0004] This application is related to International Patent Application Attorney Docket No. 100577 claiming priority from U.S. Provisional Patent Application No. 63 / 472,392 filed on June 12, 2023, and to International Patent Application Attorney Docket No. 100579 claiming priority from U.S. Provisional Patent Application No. 63 / 472,397 filed on June 12, 2023, the contents of which are incorporated herein by reference in their entirety.

[0005] BACKGROUND

[0006] The present invention, in some embodiments thereof, relates to medical imaging and surgical planning technology and, more specifically, but not exclusively, to system and methods for generating patient specific surgical instruments (PSIs).

[0007] Traditional surgical preparation often involves the use of standardized instruments and implants, which can lead to suboptimal alignment, improper fit, and increased risk of postoperative complications. In complex surgeries such as orthopedics, neurosurgery, and oncology, where precision is paramount for successful outcomes, personalized devices are increasingly being used.

[0008] SUMMARY

[0009] According to a first aspect, computer implemented method for generating a personalized surgical instrument (PSI) for a subject, comprises: automatically detecting at least one target region of a surface of at least one bone and / or cartilage on a 3D representation of the subject, defined by a surgical plan defining a surgical indication, automatically computing a pose of at least one surgical tool for performing the surgical action defined by the surgical plan, automatically generating a graphical 3D representation of the PSI according to the surgical plan, the graphical 3D representation of the PSI including a footprint region having a surface corresponding to the at least one target region for contact with the surface of the at least one bone and / or cartilage corresponding to the at least one target region, and for performing the surgical action according to the pose, and automatically generating instructions ready for 3D printing of the PSI according to the graphical 3D representation. In a further implementation form of the first aspect, further comprising within and via an interactive user interface, in at least one iteration: presenting the graphical 3D representation of the PSI, receiving via the interactive user interface, an adaptation by at least one user of at least one of: the at least one target region, the pose, the at least one surgical tool, and the surgical plan, dynamically re-computing the graphical representation of the PSI according to the adaption, and presenting the re-computed graphical 3D representation of the PSI, and in response to an indication of a finalized graphical 3D representation of the PSI received via the interactive user interface, generating instructions for 3D printing of the PSI according to the re-computed graphical 3D representation.

[0010] In a further implementation form of the first aspect, further comprising: in response to an indication of a finalized graphical 3D representation of the PSI received via the interactive user interface, updating a library comprising a plurality of surgical plans according to at least one adaptation made by the user.

[0011] In a further implementation form of the first aspect, further comprising automatically selecting the at least one surgical tool according to at least one of: the pose, the surgical action, and the at least one bone, wherein the graphical 3D representation of the PSI is computed according to the automatically selected at least one surgical tool.

[0012] In a further implementation form of the first aspect, further comprising generating instructions for assembling a kit comprising the PSI and the automatically selected at least one surgical tool.

[0013] In a further implementation form of the first aspect, the surgical plan represents a generic surgical plan for a specific surgical indication for a plurality of patients, wherein a customized PSI is generated for each patient of the plurality of patients by applying the same surgical plan to features extracted from the respective 3D representation of each patient.

[0014] In a further implementation form of the first aspect, further computing a size and / or shape of the footprint region as at least one of: (1) an optimization between a tradeoff of a larger size and a smaller size, wherein stability of the PSI is increased with the larger size and decreased with the smaller size, and risk of infection of the PSI is increased with the larger size and decreased with the smaller size, and (2) based on anatomical landmarks and / or coverage areas that are advantageous to anchor to or which are to be avoided.

[0015] In a further implementation form of the first aspect, the surgical plan is accessed from a plurality of surgical plans each for a surgical indication selected from a plurality of surgical indication, each surgical plan defining at least: a surgical procedure, the at least one bone and / or cartilage to be operated on, the target region, approach for computing the pose, and the at least one surgical tool.

[0016] In a further implementation form of the first aspect, further comprising: accessing a plurality of electronic health records (EHR) of a plurality of subjects, and a corresponding plurality of 3D representations of the plurality of subjects, for each subject: automatically extracting features from the EHR and / or the 3D representation of the subject, automatically selecting the surgical plan from a plurality of surgical plans according to the extracted features, and automatically generating the graphical 3D representation of the PSI for the subject according to the automatically selected surgical plan, and providing a plurality of graphical 3D representations of a plurality of PSIs for the plurality of subjects.

[0017] In a further implementation form of the first aspect, further comprising computing a score for each of the plurality of graphical 3D representations of the plurality of PSIs of the plurality of subjects, and ranking the plurality of graphical 3D representations according to the score and / or selecting a subset of graphical 3D representations having scores above a threshold.

[0018] In a further implementation form of the first aspect, further comprising: defining a case for each image of a plurality of images of a plurality of subjects, the case including the image and related data, automatically computing a score for the case, automatically ranking a plurality of cases according to respective scores, automatically providing a subset of cases having scores above a threshold, for each case, sending a notification to client terminal of a user assigned to the case, and in response to an interaction with the notification, automatically triggering presentation of an online 3D viewer on a display of the client terminal for review of the automatically generated graphical 3D representation of the PSI and / or for adapting parameters for the automatic generation of the graphical 3D representation of the PSI.

[0019] In a further implementation form of the first aspect, further comprising: scheduling 3D printing of the PSI by 3D printer according to at least one of: predicted amount of time for 3D printing of the PSI, scheduled operation time when the PSI is predicted to be used, current load scheduled for the 3D printer for 3D printing other PSIs for other subjects, and scheduled operation for the other subjects requiring the other PSIs.

[0020] In a further implementation form of the first aspect, further comprising: accessing a plurality of parameters associated with a user of a plurality of users, wherein the plurality of parameters are automatically learned and / or extracted based on historically generated PSIs of the user and / or are manually set, wherein the automatically generating of the graphical 3D representation is performed by conforming to the plurality of parameters. In a further implementation form of the first aspect, further comprising: generating a 3D model based on the at least one bone and / or cartilage of the 3D representation of the subject, and simulating placement of the graphical 3D representation of the PSI on the 3D model.

[0021] In a further implementation form of the first aspect, further comprising: generating an alert in response to intersection of the at least one bone and / or cartilage with the graphical 3D representation of the PSI, a deviation of the graphical 3D representation of the PSI from a predefined envelope by more than a defined threshold, and / or another detected error.

[0022] In a further implementation form of the first aspect, further comprising: checking the simulation of the placement for detection of at least one error, automatically adapting at least one of: the at least one target region, the pose, the at least one surgical tool, and / or another parameter, re-generating the graphical 3D representation of the PSI according to the automatic adaptation, for generating an adapted graphical 3D representation, performing the simulating placement of the adapted graphical 3D representation, iterating the checking, the automatically adapting, the regenerating, and the performing, until no error is detected during the checking, and providing the adapted graphical 3D representation for which no error is detected.

[0023] In a further implementation form of the first aspect, the iterating the checking, the automatically adapting, the re-generating, and the performing, until no error is detected during the checking is performed in a background, without presenting on a display graphical 3D representations for which the at least one error is detected.

[0024] In a further implementation form of the first aspect, further comprising: generating a 3D representation of the subject predicting a post-operative procedure state of the at least one bone and / or cartilage after surgical placement of the PSI, the 3D representation of the subject generated based on corresponding at least one bone and / or cartilage of subjects that did not undergo surgical procedures.

[0025] In a further implementation form of the first aspect, further comprising: creating a surgical animation video simulating a surgical procedure following the surgical plan and / or features derived for computation of the surgical placement of the PSI on the at least one bone, based on the 3D model and the graphical 3D representation.

[0026] In a further implementation form of the first aspect, further comprising: automatically generating an instruction for use (IFU) dataset for instructing a user for performing a surgical procedure following the surgical plan, using the 3D representation of the PSI, and the at least one surgical tool at the pose, wherein the IFU dataset is selected from: text instructions readable by a human, at least one image, and a surgical animation video. In a further implementation form of the first aspect, further comprising: automatically selecting according to the surgical plan, at least one measurement model of a plurality of measurement model each trained to measure a different at least one parameter of the at least one bone and / or cartilage, and automatically applying the at least one measurement model to the at least one bone and / or cartilage to obtain a plurality of different parameters, wherein at least one of the automatically detecting the target region, the automatically computing the pose, and the automatically generating, is based on the at least one measurement model.

[0027] In a further implementation form of the first aspect, further comprising: determining a unique code representing the unique graphical 3D representation of the PSI for the subject, including the unique code on the graphical 3D representation of the PSI for the subject, wherein the unique code is generated for at least one of: being visually distinguishable on the PSI by a human, and being visually distinguishable on a CT, MRI, and / or x-ray of the subject after surgical implantation of the PSI, and updating a dataset mapping between the subject and the unique code.

[0028] In a further implementation form of the first aspect, the 3D representation of the subject is a first 3D image of a first imaging modality, and further comprising: accessing a second 3D image of a second imaging modality, the second 3D image depicting at least one tissue type not sufficiently depicted in the first 3D image, registering the first 3D image with the second 3D image, analyzing the at least one tissue type depicted in the second 3D image according to the surgical plan, wherein the graphical 3D representation of the PSI is automatically rendered according to the analysis.

[0029] In a further implementation form of the first aspect, the surgical plan further defines at least one of: at least one anatomical landmark, at least one measurement, and at least one positioning parameter, wherein the graphical 3D representation of the PSI is automatically rendered according to the surgical plan that defines at least one of: at least one anatomical landmark, at least one measurement, and at least one positioning parameter.

[0030] In a further implementation form of the first aspect, the graphical representation of the PSI further comprises at least one guiding component designed for guiding the at least one surgical tool for performing the surgical action according to the pose.

[0031] In a further implementation form of the first aspect, further comprising interfacing with at least one external system for automatic generating graphical 3D representations of PSIs based on data obtained from the at least one external system.

[0032] In a further implementation form of the first aspect, further comprising generating a combination image comprising the 3D image of the subject and an overlay of the graphical 3D representation of the PSI positioned over the at least one bone and / or cartilage according to the surgical plan.

[0033] In a further implementation form of the first aspect, further comprising presenting at least one bone correction according to the surgical plan, receiving an adjustment from a user via an interactive interface, and in response to an approval, generating the graphical 3D representation of the PSI based on the adjustment.

[0034] In a further implementation form of the first aspect, further comprising adding a stamped measuring shape to the 3D printed PSI, the stamped measuring shape in a fixed predefined size and / or shape for matching to at least one measurable feature of the stamped measuring shape stored in association with the graphical 3D representation of the PSI.

[0035] In a further implementation form of the first aspect, further comprising: receiving a comment and / or question entered by at least one user, automatically identifying a location and / or feature of the graphical 3D representation of the PSI impacted by the comment and / or question; and automatically executing a task impacting the location and / or feature according to the comment and / or question.

[0036] In a further implementation form of the first aspect, further comprising: accessing a template from a plurality of templates for automatically generating the pose and / or the surgical tool(s) and / or for defining a component of the PSI, presenting the template over the 3D model within a common space on a display, testing and / or simulating the template over the 3D model; and wherein the graphical 3D representation of the PSI is generated according to the template.

[0037] According to a second aspect, a personalized kit for a subject comprises: a PSI manufactured based on instructions created according to the first aspect or any implementation forms of the first aspect, and at least one surgical tool for use with the PSI selected according to at least one of: the pose, the surgical action, and the at least one bone, wherein the graphical 3D representation of the PSI is computed according to the selected at least one surgical tool.

[0038] According to a third aspect, a system for 3D printing a PSI for a subject comprises: at least one processor executing a code for implementing the computer implemented method of the first aspect or any implementation forms of the first aspect for generating instructions for additive manufacturing of the PSI, and a 3D printer configured for additive manufacturing of the PSI while maintaining sterility of the PSI suitable for implantation in a body of the subject.

[0039] According to a fourth aspect, a computer implemented method for generating a personalized surgical instrument (PSI) for a subject, comprises: defining a case for each image of a plurality of images of a plurality of subjects, the case including the image and related data, automatically computing a score for the case, automatically ranking a plurality of cases according to respective scores, automatically providing a subset of cases having scores above a threshold, for each case of a plurality of cases: automatically extracting features from the case, automatically selecting a surgical plan from a plurality of surgical plans according to the extracted features, each surgical plan for a different surgical indication, automatically detecting at least one target region of a surface of at least one bone and / or cartilage on the 3D representation of the subject, defined by the surgical plan, automatically computing a pose of at least one surgical tool for performing the surgical action defined by the surgical plan, and automatically generating a graphical 3D representation of the PSI according to the surgical plan, the graphical 3D representation of the PSI including a footprint region having a surface corresponding to the at least one target region for contact with the surface of the at least one bone and / or cartilage corresponding to the at least one target region, and for performing the surgical action according to the pose.

[0040] In a further implementation form of the fourth aspect, further comprising filtering the plurality of images for selection images meeting availability and / or suitability criteria for automatic generation of a graphical 3D representation of a PSI, wherein the score is computed for each of the filtered images.

[0041] In a further implementation form of the fourth aspect, further comprising pulling DICOM metadata for each image, and including the DICOM in the case.

[0042] In a further implementation form of the fourth aspect, further comprising: for each case, sending a notification to client terminal of a user assigned to the case, and in response to an interaction with the notification, automatically triggering presentation of an online 3D viewer on a display of the client terminal for review of the graphical 3D representation of the PSI and / or for adapting parameters for automatic generation of the graphical 3D representation of the PSI.

[0043] In a further implementation form of the fourth aspect, the score is computed according to at least one of: historical analysis of use of the automatically generated graphical 3D representations by the corresponding user, an analysis of quality of the case indicating likelihood of generating an accurate graphical 3D representation of the PSI for the case, and correlation with availability and / or suitability criteria for automatic generation of a graphical 3D representation of the PSI.

[0044] In a further implementation form of the fourth aspect, further comprising: generating a unique code to each image of the plurality of images, instructing a central dataset storing the plurality of images to associated respective unique codes to respective images, and for each user associated with case, providing the unique code associated with the case to a client terminal, wherein the unique code is designed for remotely accessing the case associated with the unique code from the client terminal, wherein the unique code is sharable with other client terminals of other users for joint access to the case.

[0045] In a further implementation form of the fourth aspect, the score is computed according to at least one of: intersection of the at least one bone and / or cartilage with the graphical 3D representation of the PSI, a deviation of the graphical 3D representation of the PSI from a predefined envelope by more than a defined threshold.

[0046] In a further implementation form of the fourth aspect, further comprising for each 3D representation associated with a subject: accessing a plurality of parameters associated with a surgeon assigned for operating on the subject, wherein the plurality of parameters are automatically learned and / or extracted based on historically generated PSIs of the user and / or are manually set, wherein the automatic generating of the graphical 3D representation is performed by conforming to the plurality of parameters, and / or wherein the score is computed according to the plurality of parameters.

[0047] In a further implementation form of the fourth aspect, further comprising for each 3D representation associated with a subject: extracting features from the electronic health record of the subject, and wherein the surgical plan is automatically selected according to the features extracted from the electronic health record.

[0048] In a further implementation form of the fourth aspect, further comprising: scheduling 3D printing of a subset of a plurality of graphical 3D representations of the plurality of cases by an additive manufacturing device according to at least one of: predicted amount of time for 3D printing of each PSI, and scheduled operation time and / or date when each PSI is predicted to be used.

[0049] Unless otherwise defined, all technical and / or scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the invention pertains. Although methods and materials similar or equivalent to those described herein can be used in the practice or testing of embodiments of the invention, exemplary methods and / or materials are described below. In case of conflict, the patent specification, including definitions, will control. In addition, the materials, methods, and examples are illustrative only and are not intended to be necessarily limiting.

[0050] BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

[0051] Some embodiments of the invention are herein described, by way of example only, with reference to the accompanying drawings. With specific reference now to the drawings in detail, it is stressed that the particulars shown are by way of example and for purposes of illustrative discussion of embodiments of the invention. In this regard, the description taken with the drawings makes apparent to those skilled in the art how embodiments of the invention may be practiced.

[0052] In the drawings:

[0053] FIG. 1 is a block diagram of components of a system for automatically generating a personalized surgical instrument (PSI) for a subject, in accordance with some embodiments of the present invention;

[0054] FIG. 2A is flowchart of a method for automatically generating a PSI for a subject, in accordance with some embodiments of the present invention;

[0055] FIG. 2B is a flowchart of a method for automatically generating a set of candidate 3D representations of PSIs, in accordance with some embodiments of the present invention;

[0056] FIG. 2C is a flowchart of a method of an adaptation of the method for automatically generating a set of candidate 3D representations of PSIs of FIG. 2B, in accordance with some embodiments of the present invention;

[0057] FIG. 3 is a schematic of a GUI for uploading an image and / or providing additional data of a subject, in accordance with some embodiments of the present invention;

[0058] FIG. 4 is a schematic depicting an exemplary GUI for opening a new case for automatic generation of a 3D representation of a PSI, in accordance with some embodiments of the present invention;

[0059] FIG. 5 is a schematic of a GUI for viewing, creating, and / or adapting a surgical plan for computation of a graphical 3D representation of a PSI, in accordance with some embodiments of the present invention;

[0060] FIG. 6 is a schematic of a GUI for simulating placement of a 3D representation of a PSI on a 3D model of a bone of a patient, in accordance with some embodiments of the present invention;

[0061] FIG. 7 is a schematic of a GUI that includes a window for presenting the image corresponding to 3D model and / or bone segmentation, in accordance with some embodiments of the present invention;

[0062] FIG. 8 is a schematic of a GUI presenting an automatically computed 3D representation of a PSI, in accordance with some embodiments of the present invention;

[0063] FIG. 9 is a schematic of a GUI with different graphical elements for manual adjustment of different parameters for computing a 3D representation of a PSI, in accordance with some embodiments of the present invention; FIG. 10 is a schematic of a GUI presenting status of different steps for automatic computation of a 3D representation of a PSI, in accordance with some embodiments of the present invention;

[0064] FIG. 11 is a schematic of a GUI with an automatic computed measurement for automatic computation of a 3D representation of a PSI, in accordance with some embodiments of the present invention;

[0065] FIG. 12 depicts schematics of different examples of PSIs generated based on different surgical plans, in accordance with some embodiments of the present invention;

[0066] FIG. 13 is a flowchart of an exemplary method for manufacturing a personalized medical instrument for a subject based on data provided via a user interface, in accordance with some embodiments of the present invention;

[0067] FIG. 14 is a schematic of an exemplary personalized device for hip replacement, in accordance with some embodiments of the present invention;

[0068] FIG. 15 is a schematic of an exemplary personalized device for distal radius fractures, in accordance with some embodiments of the present invention;

[0069] FIG. 16 is a schematic of an exemplary personalized device for osteotomy and resection surgeries (e.g., orthopedic oncology), in accordance with some embodiments of the present invention;

[0070] FIG. 17 is a schematic of an exemplary personalized device for reverse shoulder arthroplasty, in accordance with some embodiments of the present invention;

[0071] FIG. 18 is a schematic of an exemplary personalized device for dysplasia of the hip (DDH) corrective surgery, in accordance with some embodiments of the present invention;

[0072] FIG. 19 is a flowchart depicting use cases for automatic generations of personalized medical instruments, in accordance with some embodiments of the present invention; and

[0073] FIG. 20 includes schematics depicting a flow for creating a personalized medical instrument for resection of a bone with tumor based on boundaries of a tumor, in accordance with some embodiments of the present invention.

[0074] DETAILED DESCRIPTION

[0075] The present invention, in some embodiments thereof, relates to medical imaging and surgical planning technology and, more specifically, but not exclusively, to system and methods for generating patient specific surgical instruments (PSIs). As used herein, the term patient specific surgical instruments (PSIs) may refer to a surgical jig, personalized guide, implant, and / or mold. The terms PSI and surgical jig may sometime be interchanged.

[0076] A surgical jig is a device or tool used in surgical procedures to guide the surgeon in accurately performing a specific task or procedure. Jigs are typically designed to provide precise measurements, angles, or templates to ensure consistency and accuracy during surgery. Surgical jigs are commonly used in orthopedic procedures, particularly in joint replacement surgeries such as knee or hip replacements. They help in achieving proper alignment and positioning of the prosthetic components within the patient's body. By using a surgical jig, the surgeon can accurately prepare the bone surfaces and guide the placement of implants, ensuring optimal fit and function.

[0077] A surgical implant is a medical device designed to replace a missing biological structure, support a damaged biological structure, or enhance an existing biological structure, in a body of a patient. Surgical implants are used in various surgical procedures to help improve function, provide stability, or enhance aesthetics. Surgical implants are designed to remain within the body of the patient after the surgery. For example, implant for hip replacement, implant for knee replacement, and bone screw.

[0078] A surgical mold is a specialized device or template used in various medical procedures to shape, guide, or support the formation of tissues or implants. Surgical molds may be used for ensuring precise outcomes in reconstructive and cosmetic surgeries, and / or in the manufacturing of custom implants.

[0079] As used herein, the term bone, or bone and / or cartilage is meant as a not necessarily limiting example, and may refer to other tissues and / or entities found in the body, for example, soft tissues, cartilage, tumor, regions of abnormally high calcification (e.g., due to cancer), fluid accumulation, infectious regions, previously installed implants, and the like.

[0080] As used herein, the term 3D representation (of the subject) may refer to, for example, to a 3D model created from imaging data, an interactive model, a video clip, one or more still images, and the like. The term 3D representation and 3D model may be interchanged. The term image or medical imaging data may refer to, for example, MRI, CT, EOS, x-ray, other type of scan format, and the like, which may be 3D or 2D. 2D imaging data may be converted to 3D imaging data, and / or the 3D model may be generated from the 2D imaging data. The 3D model, also referred to as an anatomical model (e.g., of bone and / or tissues) may be received as input and / or created by segmentation (e.g., of slices of layers of a CT and / or MRI scan) and / or other type of conversion from the medical imaging data to a 3D generated object (e.g., PSI file), for example, EOS / x-ray to 3D. The term 3D Template / Surgical Tool may refer to components that are placed in space over or in the anatomical model to create a surgical plan. The term 3D Generated Object File may refer to the PSI file generated based on the surgical plan using the system described herein. The file may be presented to the user, downloaded, opened by other software, or sent to print (STL, OBJ, 3DS, or other formats). The term 3D Rendering may refer to presentation of any imaging data on a screen, applicable to any of the aforementioned 3D elements.

[0081] As used herein, the term interactive user interface may refer, for example, to a graphical user interface (GUI), and / or other user interface in which a user communicates with the system (e.g., platform), for example, using spoken natural language, typed natural language, interaction with a device such as touchscreen, mouse, and the like, for performing comments and / or asking for modifications, while receiving visual and / or audio output to confirm. In another example, the interactive user interface may include implementations such as sending a video to a user for approval, receiving text-based confirmations, using voice or text commands for modifications and / or approvals, an interactive chat where changes are proposed and the user can specify desired adjustments, and the like. The term GUI is meant to be a not necessarily limiting example of the interactive interface. It is to be understood that other implementations of the interactive interface may be used instead of and / or in addition to the GUI.

[0082] As used herein, the terms additive manufacturing and 3D printing are used interchangeably, and the terms 3D printer and additive manufacturing machine are used interchangeably. The terms 3D printer and additive manufacturing machine may refer to a unified vending machine designed to receive instructions (e.g., files) the system described herein (e.g., over a network), and designed to implement processes for generating a ready to use, sterilized PSI. The vending machine manages a printing management workflow to decide when a part needs to be printed. The vending machine is designed to handle the steps of generating a ready to use, sterilized PSI, for example, printing, removal of the ready PSIs, washing, UV curing, sterilization and packaging. The vending machine is designed to be self-sustained and no intervention is necessarily required between processes.

[0083] An aspect of some embodiments of the present invention relates to systems, methods, computing devices, and / or code instructions for automatically generating a personalized surgical instrument (PSI) for a subject. One or more target regions of a surface of a bone(s) and / or cartilage may be detected on a 3D representation of the subject, optionally a 3D model of the bone(s) and / or cartilage computed from a 3D image(s) of the subject. The automatic detection of the target region is defined by a surgical plan defining a surgical indication, for example, location of femur to cut for a hip replacement, location for insertion of screws for securing a plate for reduction of a fracture, and the like. A pose (e.g., angle and / or location) of one or more surgical tools for performing the surgical action defined by the surgical plan is automatically computed. For example, angle and / or location of a plane of a saw for cutting the femur, and / or angles and / or locations for insertion of screws for placement of the plate for reduction of the fracture. A graphical 3D representation of the PSI (e.g., 3D generated object file) is automatically generated according to the surgical plan and / or the surgical action and / or the pose. The graphical 3D representation of the PSI includes a footprint region having a surface corresponding to the target region(s) for contact with the surface of the bone(s) and / or cartilage corresponding to the target region(s), and for performing the surgical action according to the pose. Instructions (e.g., a file) ready for 3D printing of the PSI according to the graphical 3D representation may be automatically generated.

[0084] An aspect of some embodiments of the present invention relates to systems, methods, computing devices, and / or code instructions for automatically generating graphical 3D representations (e.g., 3D generated object files) of multiple PSIs for different patients, for example, different patients that underwent imaging (e.g., CT scan, MRI scan) for different orthopedic indications, and which may be waiting for surgery. Images may be automatically pulled from a dataset (e.g., database) such a PACS server, as the images arrive and / or in batch (e.g., per day for the last 24 hours). A case may be defined for each image, where the case may include the image and related data such as DICOM metadata associated with the image. A score may be computed for each case. The score may indicate suitability of the case for automatically generating an accurate graphical 3D representation of the PSI and / or likelihood of the automatically generated PSI being 3D printed (e.g., based on historical analysis of which users 3D printed their PSIs). The cases may be ranked according to the score. A subset of highest ranking cases and / or cases having scores above a threshold are selected. Graphical 3D representations of PSIs may be automatically generated as described herein for the subset of cases. Notifications may be sent to client terminals used by the users (e.g., surgeons) assigned to the cases, indicating that the automatically computed graphical 3D representations are ready for viewing and / or modification and / or 3D printing, as described herein.

[0085] An aspect of some embodiments of the present invention relates to systems, methods, computing devices, and / or code instructions for automatically generating graphical 3D representations (e.g., 3D generated object files) of multiple PSIs for different patients, for example, different patients that underwent imaging (e.g., CT scan, MRI scan) for different orthopedic indications, and which may be waiting for surgery. The images of the different patients may be automatically obtained, for example, by accessing a PACS server. Surgical plans for the different patients may be automatically obtained and / or computed, for example, based on the surgical indication of each patient. A respective graphical 3D representation of a PSI may be automatically generated for each patient based on their surgical plan, as described herein. The automatically generated graphical 3D representations may be provided, for example, to each surgeon who will operate on each respective patient. Each surgeon may evaluate the automatically generated graphical 3D representation of the PSI, for example, to make adjustments and / or decide whether to 3D print the respective PSI. Optionally, a score is computed for each automatically generated graphical 3D representation. The score may be computed based on one or more parameters, for example, intersection of the bone and / or cartilage with the graphical 3D representation of the PSI, a deviation of the graphical 3D representation of the PSI from a predefined envelope by more than a defined threshold, and historical analysis of use of automatically generated graphical 3D representations for the respective surgical indication and / or by the corresponding surgeon. The automatically generated graphical 3D representations may be ranked according to score. The top scoring and / or automatically generated graphical 3D representations with scores above a threshold may be selected, for example, for providing to the physicians.

[0086] An aspect of some embodiments of the present invention relates to a personalized kit for a subject for surgery. The personalized kit includes a PSI manufactured (e.g., 3D printed) based on an automatically generated graphical 3D representation of a PSI, generated based on a surgical plan, as described herein. The personalized kit further includes one or more surgical tools for use with the PSI. The graphical 3D representation of the PSI is computed according to the selected surgical tool(s). The surgical tools are selected according to one or more of: the pose of the surgical tools to be user during surgery with the PSI, the surgical action to be performed during the surgery in association with the PSI, bone(s) on which the surgical tools are to be used and / or on which the PSI is to be placed.

[0087] An aspect of some embodiments of the present invention relates to a system for 3D printing a PSI for a subject for surgery. The system includes one or more processors executing a code for automatically generating a graphical 3D representation of a PSI of the subject, and a 3D printer for additive manufacturing of the PSI while maintaining sterility of the PSI suitable for implantation in a body of the subject.

[0088] At least some embodiments described herein address the technical problem of improving PSIs for subjects. At least some embodiments described herein improve the medical field of orthopedics, and / or the technical field of 3D representations for 3D printing, by automatic generation of PSIs for subjects. Using traditional approaches, the surgeon selects from one or more predefined designs, which may be unsuitable and / or inaccurate for specific patients. The physician may design their own PSI using manual approaches, which is time consuming and / or prone to error.

[0089] Existing processes for generating PSIs for subjects are manual, requiring multiple systems, skilled designers, and engineers to create each patient specific 3D instrument. This manual process takes a long time to complete and generally requires a surgeon and another professional to work together, which makes it inefficient.

[0090] At least some embodiments described herein address the technical problem(s) described herein, and / or improve the technology described herein, by automatically computing a graphical 3D representation of a PSI for a subject, which may be suitable for 3D printing. The graphical 3D representation of the PSI may be automatically computed based on a surgical plan, which may enable a fully automatic generation. In contrast, existing PSI design processes are done using CAD software based on digital design elements and manual sculpturing, not by simply placing surgical elements over an anatomy. Similarly for fine-tuning of the generated PSI, the fine-tuning based on at least some embodiments described herein may be done by adapting parameters and / or modifying the surgical plan, in contrast to standard approaches where the user manually changes the design itself. In at least some embodiments described herein, a semi-automatic approach may be implemented, where the user enters desired parameters, for example, which surgical tools to use, and / or preferred surgical approaches. A user may fine-tune to automatically generated 3D representation of the PSI. The automatically generated 3D representation of the PSI may be automatically evaluated for errors, for example, intersection with the bone(s) of the subject.

[0091] At least some embodiments described herein relates to a comprehensive platform and workflow for automating the creation of custom made surgical instruments based on patientspecific anatomical data derived from medical scans (e.g., CT, MRI, and X-rays), optionally at scale and / or with minimal friction in the point of care. The platform potentially streamline the workflow from image acquisition to instrument production, facilitating a more precise and efficient surgical process.

[0092] Potential advantages of one or more embodiments described herein include:

[0093] • Full Automation of PSI Design-. Automation of the entire process of converting surgical plans into PSIs, reducing the need for manual intervention and ensuring precision.

[0094] • User-Friendly Interface (e.g. GUI)'. Intuitive interface that allows users (e.g., surgeons) to make quick adjustments and receive real-time feedback, without requiring specialized skills for designing psi. • Broad Applicability. Versatility platform, which may be used for various types of surgeries, making it a comprehensive solution for hospitals.

[0095] • Cost-Effective 3D Printing Integration-. Enabling hospitals and / or other facilities to set up affordable in-house 3D labs, making advanced surgical planning technology accessible to more institutions.

[0096] The PSI design process described herein may be fully automated and / or designed for minimizing manual intervention. Surgical instructions may be customized to fit individual patient anatomy and surgical needs.

[0097] In at least some embodiments described herein, the platform is equipped with various preset applications (apps), where each app may be designed for a specific surgical indication. These apps may be designed to cover the entire workflow, for example, from initial medical data acquisition to the final PSI creation, for the specific surgical indication. Each indication may be implemented as an app within the platform. Each app may provide a comprehensive toolkit for conducting end-to-end surgical planning specific to the chosen indication.

[0098] Optionally, before an app is ready to be used, the app may go through a process of selection of suitable surgical indications and / or creation of design envelope(s). The design envelope(s) may be based on the surgical tools application to the surgical indication (for example, screws required to conform to a specific fixation element, etc...) and / or creation of the logic, parameters and / or rules, of how the envelope and / or app may be applied. Parameters that may be adapted by a user and / or fixe parameters may be defined. Each design envelop may be a combination of presets and / or functions that each is applied to a different process, for example, comparing bone to a normal bone library, finding anatomical landmarks, measuring areas of the anatomy (e.g., angle, distance, radius), selection of templates based on parameters such as size, placement of surgical tools (e.g., based on templates selected from a library) such as drills, and / or cutting planes. Additional exemplary details of the apps are described below. Potential benefits of one or more embodiments described herein include:

[0099] • Streamlined Process'. The app-based approach described herein may centralize and / or simplify the workflow, which may make it user-friendly and / or efficient for medical professionals.

[0100] • Customization and Precision'. Users (e.g., surgeons) may rely on the precision of the automated processes, and may be provided with the ability to make manual adjustments as needed, which may help ensure accuracy and / or adaptability in surgical preparation. • Scalability and Accessibility. New surgical indications may be added as separate apps, which may allow the platform to expand its offerings and / or adapt to new advancements in medical procedures and technologies.

[0101] Some potential applications of PS Is created using at least one embodiment described herein include:

[0102] • Orthopedic Surgeries: Custom jigs and bone alignment tools for precise cutting and placement (ORIF, Joints).

[0103] • Neurosurgical Procedures: Tailored cranial implants.

[0104] • Oncological Surgeries: Custom resection tools based on tumor- specific modeling.

[0105] • Reconstructive Surgeries: Implants and scaffolds perfectly matched to patient anatomy.

[0106] • Antibiotic-Infused Cement Molds: Molds that deliver localized antibiotic treatment to prevent post-operative infections.

[0107] Before explaining at least one embodiment of the invention in detail, it is to be understood that the invention is not necessarily limited in its application to the details of construction and the arrangement of the components and / or methods set forth in the following description and / or illustrated in the drawings and / or the Examples. The invention is capable of other embodiments or of being practiced or carried out in various ways.

[0108] The present invention may be a system, a method, and / or a computer program product. The computer program product may include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present invention.

[0109] The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium may be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of the computer readable storage medium includes the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, and any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted through a wire.

[0110] Computer readable program instructions described herein can be downloaded to respective computing / processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and / or a wireless network. The network may comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing / processing device.

[0111] Computer readable program instructions for carrying out operations of the present invention may be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) may execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present invention.

[0112] Aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer readable program instructions. These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, a programmable data processing apparatus, and / or other devices to function in a particular manner, such that the computer readable storage medium having instructions stored therein comprises an article of manufacture including instructions which implement aspects of the function / act specified in the flowchart and / or block diagram block or blocks.

[0113] The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process, such that the instructions which execute on the computer, other programmable apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0114] The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flowchart illustration, and combinations of blocks in the block diagrams and / or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts or carry out combinations of special purpose hardware and computer instructions.

[0115] Reference is now made to FIG. 1, which is a block diagram of components of a system 100 for automatically generating a personalized surgical instrument (PSI) for a subject, in accordance with some embodiments of the present invention. Reference is also made to FIG. 2A, which is flowchart of a method for automatically generating a PSI for a subject, in accordance with some embodiments of the present invention. Reference is also made to FIG. 2B, which is a flowchart of a method for automatically generating a set of candidate 3D representations of PS Is, in accordance with some embodiments of the present invention. Reference is also made to FIG. 2C, which is a flowchart of a method of an adaptation of the method for automatically generating a set of candidate 3D representations of PSIs of FIG. 2B, in accordance with some embodiments of the present invention. Reference is also made to FIG. 3, which is a schematic of a GUI 302 for uploading an image and / or providing additional data of a subject, in accordance with some embodiments of the present invention. Reference is also made to FIG. 4, which is a schematic depicting an exemplary GUI 402 for opening a new case for automatic generation of a 3D representation of a PSI, in accordance with some embodiments of the present invention. Reference is also made to FIG. 5, which is a schematic of a GUI 502 for viewing, creating, and / or adapting a surgical plan for computation of a graphical 3D representation of a PSI, in accordance with some embodiments of the present invention. Reference is also made to FIG. 6, which is a schematic of a GUI 602 for simulating placement of a 3D representation of a PSI 604 on a model of a bone 606 of a patient, in accordance with some embodiments of the present invention. Reference is also made to FIG. 7, which is a schematic of a GUI 702 that includes a window 704 for presenting the image corresponding to 3D model and / or bone segmentation 706, in accordance with some embodiments of the present invention. Reference is also made to FIG. 8, which is a schematic of a GUI 802 presenting an automatically computed 3D representation of a PSI 804, in accordance with some embodiments of the present invention. Reference is also made to FIG. 9, which is a schematic of a GUI 902 with different graphical elements for manual adjustment of different parameters for computing a 3D representation of a PSI, in accordance with some embodiments of the present invention. Reference is also made to FIG. 10, which is a schematic of a GUI 1002 presenting status of different steps for automatic computation of a 3D representation of a PSI, in accordance with some embodiments of the present invention. Reference is also made to FIG. 11, which is a schematic of a GUI 1102 with an automatic computed measurement 1104 for automatic computation of a 3D representation of a PSI, in accordance with some embodiments of the present invention. Reference is also made to FIG. 12, which depicts schematics of different examples of PSIs generated based on different surgical plans, in accordance with some embodiments of the present invention. Reference is also made to FIG. 13, which is a flowchart of an exemplary method for manufacturing a personalized medical instrument for a subject based on data provided via a user interface, in accordance with some embodiments of the present invention. Reference is also made to FIG. 14, which is a schematic of an exemplary personalized device for hip replacement, in accordance with some embodiments of the present invention. Reference is also made to FIG. 15, which is a schematic of an exemplary personalized device for distal radius fractures, in accordance with some embodiments of the present invention. Reference is also made to FIG. 16, which is a schematic of an exemplary personalized device for osteotomy and resection surgeries (e.g., orthopedic oncology), in accordance with some embodiments of the present invention. Reference is also made to FIG. 17, which is a schematic of an exemplary personalized device for reverse shoulder arthroplasty, in accordance with some embodiments of the present invention. Reference is also made to FIG. 18, which is a schematic of an exemplary personalized device for dysplasia of the hip (DDH) corrective surgery, in accordance with some embodiments of the present invention. Reference is also made to FIG. 19, which is a flowchart depicting use cases for automatic generations of personalized medical instruments, in accordance with some embodiments of the present invention. Reference is also made to FIG. 20, which includes schematics depicting a flow for creating a personalized medical instrument for resection of a bone with tumor based on boundaries of a tumor, in accordance with some embodiments of the present invention.

[0116] As used herein, the term platform may refer to system 100 with one or more components, such as a computing environment 104 and / or processor(s) 102 optionally executing code 106A stored on a memory 106. The terms platform and system are used interchangeably.

[0117] System 100 described with reference to FIG. 1 may implement the features of the method described with reference to FIGs. 2-20, by one or more processors 102 of computing environment 104 executing code instructions 106 A stored on memory 106.

[0118] Computing environment 104 may be implemented as, for example, a client terminal, a server, a virtual machine, a virtual server, a computing cloud, a mobile device, a desktop computer, a thin client, a Smartphone, a Tablet computer, a laptop computer, and an augmented reality device. Computing environment 104 may include an advanced visualization workstation that sometimes is add-on to a surgical workstation and / or other devices.

[0119] Computing environment 104 may receive a 3D image(s) (e.g., 3D images such as CT) from image (imaging data) repository 118A, access a surgical plan from a surgical plan repository 120A, process the 3D image using one or more models, compute a graphical 3D representation of a PSI, present a 3D rendering of the graphical 3D representation within a GUI 108A running on a client terminal 108, and create a 3D generated object file based on the graphical 3D representation for manufacturing by a manufacturing device 130.

[0120] Multiple architectures of system 100 based on computing environment 104 may be implemented:

[0121] In an exemplary implementation of a centralized architecture, computing environment 104 may be implemented as one or more servers (e.g., network server, web server, a computing cloud, a virtual server) that provides centralized services (e.g., one or more of the acts described with reference to FIGs. 2-20) to one or more client terminals 108 in communication with computing environment 104 over a network 110, for example, providing software as a service (SaaS), software services accessible using a software interface (e.g., application programming interface (API), software development kit (SDK)), an application for local download, and / or providing functions using a remote access session, such as through a web browser and / or viewing application. Computing environment 104 may be accessible from external devices (e.g., any device), for example, client terminal(s) 108, such as smartphones and / or tablets, and may be used via the web, making it accessible from different locations (e.g., anywhere). For example, computing environment 104 may be implemented as a server in a hospital, providing services to multiple client terminals 108 implemented as surgical work stations located in multiple operating rooms and / or surgical planning rooms. The server implementation of computing environment 104 may access images of different subjects from one or more image repositories 118A, which may be located in different geographical regions, and / or image may be provided by different client terminals 108. Different surgical plans for different surgical indications of the different subjects may be centrally accessed, for example, from surgical plan repository 120A. Computing environment 104 may centrally compute the different graphical 3D representation of different PSIs for the different subjects. Each PSI may be provided to a different client terminal 108, for example, located in different surgical planning rooms. In each room, a user may use a respective client terminal 108 presenting GUI 108A to view and / or manipulate the computed graphical 3D representation of the PSI computed for a specific subject for which the user is performing surgical pre-planning.

[0122] In an exemplary implementation of a localized architecture, computing environment 104 may provide dedicated and / or localized services (e.g., one or more of the acts described with reference to FIGs. 2-20), for example, to an individual user, such as a surgeon performing preplanning of a surgical procedure. Computing environment 104 may be implemented, for example, as a surgical workstation, a laptop, a desktop, and / or a server in a hospital. Computing environment 104 may access an image of a subject from image repository 118A. The surgical plan for the surgical indication of the subject may be obtained. Computing environment 104 may compute the graphical 3D representation of the PSI for the subject. The PSI may be presented within a GUI 108 A running on a display connected to computing environment 104.

[0123] Image repository 118A may store images captured by an image modality, including 2D and / or 3D and / or 4D images, for example, CT, MRI, x-ray, and ultrasound. Image repository 118A may be hosted by server(s) 118, for example, a PACS server. Server(s) 118 may host electronic health records (EHR) 118B of subjects. The EHRs may be analyzed for automatic selection of the surgical plan, and / or to obtain features for computing the graphical 3D representation of the PSI.

[0124] Alternatively or additionally, server(s) 118 may include third-party surgical planning systems. Computing environment 104 may integrate with other system, such as the third-party surgical planning systems, for automatically generating the graphical 3D representations of PSIs (e.g., personalized guides, implants, and / or molds) based on data obtained from the third-party surgical planning systems and / or other systems.

[0125] Manufacturing device 130 may be implemented as, for example, a 3D printer, additive manufacturing device, and the like.

[0126] Processor(s) 102 of computing environment 104 may be hardware processors, which may be implemented, for example, as a central processing unit(s) (CPU), a graphics processing unit(s) (GPU), field programmable gate array(s) (FPGA), digital signal processor(s) (DSP), and application specific integrated circuit(s) (ASIC). Processor(s) 102 may include a single processor, or multiple processors (homogenous or heterogeneous) arranged for parallel processing, as clusters and / or as one or more multi core processing devices.

[0127] Memory 106 stores code instructions 106A executable by hardware processor(s) 102. Exemplary memories 106 include a random access memory (RAM), read-only memory (ROM), a storage device, non-volatile memory, magnetic media, semiconductor memory devices, hard drive, removable storage, and optical media (e.g., DVD, CD-ROM). For example, memory 106 stores code 106A that execute one or more acts of the method described with reference to FIGs. 2-20.

[0128] Computing environment 104 may include a data storage device 120 for storing data, for example, a surgical plan repository 120A storing different surgical plans for different indications, one or more models 120B such as for segmentation, measurement, extraction of other features, computation of the graphical 3D representation of the PSI, and other data described herein such as parameters of different users. Data storage device 120 may be implemented as, for example, a memory, a local hard-drive, a removable storage unit, an optical disk, a storage device, a virtual memory and / or as a remote server 118 and / or computing cloud (e.g., accessed over network 110).

[0129] Network interface 122 may be implemented as, for example, a wire connection (e.g., physical port), a wireless connection (e.g., antenna), a network interface card, a wireless interface to connect to a wireless network, a physical interface for connecting to a cable for network connectivity, and / or virtual interfaces (e.g., software interface, application programming interface (API), software development kit (SDK), virtual network connection, a virtual interface implemented in software, network communication software providing higher layers of network connectivity).

[0130] Computing environment 104 and / or client terminal(s) 108 include and / or are in communication with one or more user interfaces 124 that include a mechanism for presenting GUI 108A, and / or for inputting data and / or for viewing data, optionally via GUI 108A. For example, one or more of, a touchscreen, a display, a keyboard, a mouse, and voice activated software using speakers and microphone.

[0131] Referring now back to FIG. 2A, at 202, an image(s), i.e. imaging data, of the subject is accessed. The imaging data may be a 3D image, for example, CT and / or MRI. The imaging data may be a 2D image, for example, a slice of a CT scan, an x-ray, and the like.

[0132] The computing environment and / or the processor(s) may be designed to interface and / or integrate with one or more external systems, for example, via one or more interfaces over a network, such as via a virtual interface (e.g., API, SDK) and / or physical interface (e.g., network interface). The computing device may automatically generate graphical 3D representations of PSIs based on data obtained from external system(s) interfacing with computing device 104. The computing environment use the interface(s) (e.g., APIs) for receiving file formats from other planning software. The surgical plan may be received from another planning software, and used to automatically generate the PSI, later allowing fine-tuning, as described herein.

[0133] Optionally, different representations generated by different medical imaging modalities may be converted to a format of a 3D model suitable for being processed as described herein. For example, a 3D model may be computed from one or more 2D images such as x-rays.

[0134] Other data may be accessed, for example, an electronic health record of the subject, preferences of the surgeon assigned to perform surgery on the subject, and the like.

[0135] Exemplary features of the system described herein for accessing image(s) and / or other data include:

[0136] • System Integration: The platform may be designed for integrating with existing hospital (or other facility) information systems. This integration may enable seamless access to patientspecific imaging data and / or other medical data such as health records.

[0137] • Data Acquisition Methods: The platform may be designed to support automatic and / or manual data acquisition methods. For example, for automatically pulling imaging data such as CT, MRI, and X-ray directly from hospital (or other facility) systems such as a PACS server and / or EHR server, and / or to allow users to manually upload such data. • Data Format’. The acquired imaging data may be primarily in DICOM format, which is the standard for handling, storing, and transmitting information in medical imaging. This may include data from CT scans, MRI scans, X-rays, and other relevant imaging technologies. It is noted that other formats may be supported.

[0138] Images and / or other data may be provided, for example by one or more of the following not necessarily limiting examples:

[0139] • Manual Upload:

[0140] • Users log in, select surgery type, and upload DICOM files (CT, MRI, X-ray).

[0141] • System validates and processes the files.

[0142] • Software Integration:

[0143] • Integration with PACS systems and other software.

[0144] • Surgeons or hospital staff can create cases with a click in the hospital system and our system can pull the right imaging and create a case.

[0145] • Automated Crawling:

[0146] • Code may crawl hospital systems to identify potential cases.

[0147] • Suggests relevant cases for planning and PSI creation.

[0148] Referring now back to FIG. 3, GUI 302 may include graphical elements designed for uploading an image 304 and / or providing additional data of a subject such as patient details 306 and / or surgery details 308. The data provided via GUI 302 may be used for automatic selection of the surgery plan and / or for automatic computation of the 3D graphical representation of the PSI.

[0149] Referring now back to FIG. 2A, at 204, a surgical plan may be accessed.

[0150] The surgical plan may be implemented as, for example, one or more of: a data structure storing values in fields, instructions in human readable format (e.g., text), instructions in machine format translatable to human readable format, a sequence of images, a simulation animation, a video, and the like.

[0151] There may be multiple surgical plans, defined according to surgical indication, for example, fracture reduction, tumor resection, knee replacement, and hip replacement. The surgical plan may be accessed from multiple surgical plans each for a different surgical indication according to the surgical indication of the current patient.

[0152] More specialized surgical plans may be automatically and / or manually created for example, different surgeons may customize their own surgical plans for the same surgical indication. Specialized surgical plans may be automatically created by one or more models that learn how users have modified more general surgical plans and / or 3D representations of PSIs created by the more general surgical plan, for example, creating specialized surgical plans for different surgeons. In another example, surgical plans may be automatically and / or manually created for different configurations of bone fragments (e.g. fracture), such that for a new patient with a certain pattern of bone fragments the closest matching surgical plan is selected. In yet another example, surgical plans may be automatically and / or manually created for patients with different bone conditions, for example, bone in a cancer patient, bone in an old person with osteoporosis, bone in in a young healthy person, and the like.

[0153] The surgical may plan represent a generic surgical plan for a specific surgical indication for multiple patients having the same surgical indication. A customized PSI may be generated for each patient by applying the same surgical plan according to each patient, for example, to features extracted from the respective 3D model of each patient and / or according to patient specific data and / or surgeon specific data, and the like.

[0154] Each surgical plan may define at least one or more of: a surgical procedure to be performed, bone and / or cartilage to be operated on, a target region where the PSI is to be positioned, one or more surgical tools to be used (e.g., saw, screw), approach for computing a pose of the surgical tools, anatomical landmarks and optionally how they are used (e.g., for measurement, target of surgical tools, to avoid by PSI, for positioning of PSI), one or more measurements (e.g., relative to anatomical landmarks), extrusion angle to allow easy placement and removal of the PSI, and at least one positioning parameter indicting where and / or how the PSI is to be positioned. The graphical 3D representation of the PSI is automatically rendered according to the surgical plan defining one or more of: the anatomical landmark(s), the measurement(s), and the positioning parameter(s).

[0155] Optionally, an initial surgical plan is accessed and / or computed based on a selected and / or automatically detected surgical indication. Indications of relevant surgical tools may be automatically presented, for example, 3D templates of the surgical tools may be presented in a space over or in the 3D model (e.g. anatomical model) of the bone(s) of the subject. The user may fine-tune the 3D templates of the surgical tools via the GUI, for example, selecting specific tools and / or adjusting the pose of the surgical tools. The user may approve the surgical plan, which is used for automatic generation of the 3D representation of the PSI, as described herein.

[0156] The surgical plan may be accessed, optionally automatically, based on one or more details about the medical case, for example: • Surgery type (the indication), for example, reverse should arthroscopy (RSA), cranioplasty (CRP), distal radius fracture (DRF), hip resurfacing (RSF), total hip replacement (THR), total knee replacement (TKR), osteotomy for deformity correction, and tumor resection.

[0157] • Surgery date.

[0158] • Side of the surgery, such as left or right, in particular when the surgery is being performed on a limb or anatomy which is symmetric (e.g., shoulder, hip).

[0159] • Surgical approach.

[0160] • Patient demographics, for example, initials, gender, age, year of birth.

[0161] • Medical history, for example, related to the condition leading to the surgery and / or background chronic medical illnesses, and / or previous surgeries.

[0162] Once the relevant details are provided, the new case file may be created. The file may be used to store and / or manage the medical and / or procedural data associated with the case. The medical data uploaded and / or pulled from the hospital information systems may be associated (e.g., attached) to the newly created case file.

[0163] Optionally, the medical data is automatically anonymized. The anonymization may be performed to ensure compliance with privacy regulations and hospital standards. Exemplary features include:

[0164] • Removing all personally identifiable information (PIT) from the data, such as patient names, specific dates, and any other identifiers not necessary for the surgical planning.

[0165] • Deleting any non-relevant data that does not contribute to the system's performance or the surgical outcome.

[0166] After anonymization, the data may be organized into a predetermined structure and / or converted into a format that is optimized for use within the platform. The structured approach may help ensure that the data is easily accessible and manageable throughout the surgical planning and instrument design process.

[0167] Each case may be assigned a unique identifier (e.g., code) that represents the case. The case number may be engraved onto the PSI. The unique identifier (e.g., code) may be used for tracking communication with the case requestor (e.g., surgeon, external entity).

[0168] In at least some embodiments described herein, the platform is equipped with various preset applications (apps), where each app may be designed for a specific surgical indication and / or to perform a certain task. These apps may be designed to cover the entire workflow, for example, from initial medical data acquisition to the final PSI creation, for the specific surgical indication.

[0169] The apps may be automatically accessed from a common pool based on the surgical plan. The combination of the apps from the common pool is defined based on the surgical plan. The same app may be used for different surgical plans, for example, the same measurement app may be used to make measurements for a surgical app for resection of tumor and for fracture reduction.

[0170] The design using apps enables modifying and / or adding new apps, which may then be used in associated with different surgical plans as needed.

[0171] Examples of apps for different phases of the workflow (e.g., as described with reference to FIG. 2A and / or FIG. 2B and / or FIG. 2C) include:

[0172] • Anatomy Recognition and Segmentation: Each app may be tailored to recognize and / or segment specific anatomical structures relevant to the corresponding surgical indication. This preset knowledge may improve the model(s) for accurately automatically identifying and / or delineating the relevant anatomy in medical imaging data.

[0173] • Surgical Plan Presets: These apps may include predefined surgical plans which outline the steps, procedures, and / or considerations unique to each type of surgery. The surgical plans may include, for example, where cuts should be made, the positioning of implants, and / or other important surgical features.

[0174] • Integrated Elements and Templates: Incorporating a library of surgical tools and / or templates, each app may facilitate the selection and / or customization of tools and / or implants predicted to be used during the surgery. For example, drills, blades, screws, and / or other necessary surgical apparatus, which may be automatically suggested based on the surgical plan.

[0175] • PSI Design Envelope: The design parameters for creating patient- specific instruments may be predefined within each app. The PSI design envelope includes specifications, for example, size, shape, and / or fitting requirements, which may ensure that the instruments are tailored to the patient’s anatomy and / or the surgical requirements.

[0176] Referring now back to FIG. 4, GUI 402 may include graphical elements opening a new case for automatic generation of a 3D representation of a PSI. Graphical elements 404 may be designed for a user to select the surgical indication, which may define the surgical plan. For example, cranioplasty, distal radius fracture, reverse should arthroscopy, hip resurfacing, total hip replacement, and the like. Graphical elements 406 may be designed for viewing recently viewed previous cases of the same user and / or by different users. Graphical elements 408 may be designed for viewing all previous cases, by the same user and / or different users. Referring now back to FIG. 2A, at 206, the imaging data is automatically processed, optionally for creating a 3D model. The imaging data may be automatically segmented, and / or the imaging may be automatically analyzed for detecting of one or more target areas, for example, for placement of the PSI and / or for performing a surgical action defined by the surgical plan.

[0177] Optionally, one or more target regions are automatically detected. The target regions are defined by the surgical plan. The target regions may include a surface of a bone and / or cartilage which is to be operated on by one or more surgical tools and / or which is to contact the PSI (also referred to herein as a footprint region).

[0178] Optionally, a size and / or shape of the footprint region is computed as an optimization considering a tradeoff between a larger size and a smaller size. Stability of the PSI is increased with the larger size and decreased with the smaller size. Risk of infection of the PSI is increased with the larger size and decreased with the smaller size. There may be other factors with tradeoffs which are considered. Alternatively or additionally, the size and / or shape of the footprint region is computed based on anatomical landmarks and / or coverage areas that are advantageous to anchor to, or which are to be avoided such as areas with soft tissue and / or unexposed areas during surgery. A footprint quality measure may be computed, indicating how definitive the footprint is on the anatomy of the current patient.

[0179] Optionally, images of the subject are accessed, such as different images of different types. The multiple images may be different images captured by different imaging modalities. One imaging modality may depict one or more tissue types are not sufficiently depicted in another imaging modality. For example, a CT image is better for bone and an MRI is better for soft tissues. The two images of different types may be registered and / or aligned to one another. Different tissues may be analyzed in the registered images according to the surgical plan. For example, soft issue overlaying a bone to be operated on is analyzed in the MRI using the surgical plan applied to the CT, according to the registration between the CT and MRI. The graphical 3D representation of the PSI may be automatically rendered according to the analysis of the registered images. The alignment and / or registration of the image data collected from different 3D models and / or 3D images (e.g., MR scan and CT scan) may be used for example, for determining the margins of tumor, which may provide high accuracy for of the generated PSI.

[0180] A series picking process may be applied to the accessed imaging data. The series picking process may be designed to automatically select the most relevant imaging series from a set of available scans. The selection may be based on one or multiple criteria, for example, image quality, slice thickness, number of slices, and coverage of relevant anatomy. Automatic segmentation of anatomical structures from the imaging data (e.g., the selected imaging data) may be performed. The automatic segmentation may be performed, for example, by one or more machine learning models trained to segment certain anatomical structures.

[0181] Alternatively or additionally, relevant anatomical features for surgical planning may be detected and / or isolated. The anatomical features may be used, for example, for computing the footprint, for determining where surgical tools are to be positioned, for measurement of parameter used to compute the 3D representation of the PSI, and the like.

[0182] The GUI may present interactive tools for the user (e.g., surgeon) to manually review and / or adjust the automatically generated segmentation(s). The manual review may help ensure that the segmentation accuracy meets the surgical requirements and / or allows for personalized adjustments. Alternatively, the manual review may be skipped, entirely, or may be presented in the GUI after the graphical 3D representation has been generated and / or presented with respect to other features of the method.

[0183] Optionally, a 3D anatomical model is automatically generated according to the image, and / or fused images. The 3D anatomical model may be based on the segmentation and / or analysis. The 3D model may depict the anatomy relevant to the surgery. The 3D model may provide a detailed representation of the patient's anatomical structure. The 3D model may be used for computing a surgical simulation and / or for computing the 3D representation of the PSI. The 3D model may be implemented as, for example, a 3D image with associated values for different anatomical regions which may model physical properties, for example, strength of bone, areas to avoid, areas selected as footprint, and / or other data determined as described herein.

[0184] The GUI may present the 3D model, optionally alongside the original imaging data (e.g., CT scans). The generated presentation may include marked areas of the segmented anatomy for highlighting the focus areas. The user (e.g., surgeon) may interact with the 3D model within the GUI, for example, rotating, zooming, and examining the 3D model from different angles and / or poses. Using the GUI, the user may compare the segmented areas directly with corresponding slices from the original imaging data to validate the 3D model's accuracy.

[0185] In cases where images from multiple imaging modalities are accessed (e.g., CT and MRI), these different image data sources may be integrated (e.g., fused), for example, into a single coherent model. The platform may adjust and / or aligns input from different image data sources, for example, by modifying parameters such as coverage area, angle, and / or position to ensure that the images align accurately while maintaining scale proportions. A computed merged view may be presented in the GUI. The merged may combine the multiple images from different sources, which may provide a comprehensive visualization, for example, including hard tissues (bones) and soft tissues (ligaments, tumors, etc...). The integration may be performed for surgeries where multiple tissue types must be considered, optionally as defined by the surgical plan.

[0186] The platform may be designed to enable adjustment and / or fine-tuning, to allow for automatic and / or manual adjustments of the fused data. Users may manipulate the combined model to better suit the surgical objectives, adjusting size, position, and rotation based on segmented elements common across multiple data inputs.

[0187] At 208, automatic surgical planning and / or surgical simulation may be performed according to the surgical plan.

[0188] Optionally, a pose of one or more surgical tools for performing one or more surgical actions defined by the surgical plan is automatically computed. For example, a plane of a saw for cutting bone is automatically computed. In another examples, locations for insertion of screws to secure the PSI to the bone are automatically computed.

[0189] Optionally, the surgical tool(s) are automatically selected. The type of surgical tool may be selected, for example, saw, screw, and the like. The model and / or parameters of the surgical tool may be selected, for example, diameter of saw blade, length of screw, diameter of screws, and the like. The surgical tool(s) may be selected, for example, according to at least one of: the pose, the surgical action, and the bone which is to be operated on. The surgical planning simulation and / or graphical 3D representation of the PSI is computed according to the automatically selected surgical tool. The surgical tool may be selected, for example, by a machine learning model that is previously trained on different parameters (e.g., pose, action, bone) and ground truth of selected surgical tools. The machine learning model may automatically learn the surgical tools manually selected and / or adapted (from an initial automatic selection) by users, in order to perform the automatic selection.

[0190] Optionally, instructions for assembling a kit are automatically created. The kit includes the PSI (after 3D printing) and the automatically selected surgical tools to be used with the PSI. The instructions may be, for example, manual instructions for a user to assemble the kit (e.g., text, image, animation, video) and / or for a 3D printer to automatically print the PSI and / or surgical tools, and / or for a robot to automatically assemble the kit (e.g., to maintain sterility).

[0191] The platform may automate the creation of a post-surgery anatomical structure, which may be presented within the GUI for being fine-tuned by the user (e.g., surgeon). The fine-turning may help ensure the surgical process is as precise and effective as possible. An exemplary process for creating the post-surgery anatomical structure is now described:

[0192] • Automated Surgical Proposals’. A proposed anatomical structure based on the preoperative plan may be automatically generated, optionally based on the surgical plan, and may be presented within the GUI. The presentation within the GUI may allow the user to visualize the expected and / or predicted outcome and / or make adjustments before the procedure.

[0193] • User Input and Fine-Tuning’. Users may interact with the surgical plan via the GUI to make detailed adjustments such as where to cut, the positioning of drilling, and / or the selection of surgical tools. Examples of available options include drill diameter, saw blade thickness, and choosing the appropriate off-the-shelf implants.

[0194] • Integrated Surgical Steps’. The surgical plan may define and / or include the steps for the actual surgical process, which may streamline the workflow and / or potentially reduce the chance for errors during the operation.

[0195] Optionally, the GUI presents a range of surgical tools tailored to the specific surgical plan (based on the surgical indication), including for example, drills, cutting planes, templates, implants, and / or shapes.

[0196] • User Manipulation: Surgeons can adjust these elements for better precision. Adjustments can include changing the positioning, measurements, angles, types, and shapes.

[0197] • Element Customization and Exclusion: Each surgical tool can be fine-tuned or even disabled to ensure it does not appear in the final patient- specific instrument (PSI) if it's not needed.

[0198] Optionally, an anatomy reduction process is performed, optionally according to the surgical plan. The anatomy reduction process may be used in surgical planning for cases involving fractured bones. An exemplary flow of the anatomy reduction process is may include one or more of the following features:

[0199] • Reduction Proposal'. Initially, the platform may automatically generate a suggestion for a configuration for the realignment of bone fragments. The suggestion may be generated based on the pre-operative positioning of the bones and / or may be aimed at restoring the anatomical structure as closely as possible to its original state. The suggestion may be made, for example, by a model trained on images of other patients (e.g., similar age, size, measurements of anatomical landmarks, bone state), which are normal / healthy without reduction, and / or of other patients in a post-op state after reduction for a similar fracture.

[0200] • Visualization and Comparison: The GUI may present a visual side-by-side (or other arrangement) comparison between the pre-operative (pre-op) and proposed post-operative (post-op) positions of the bone fragments. The side-by-side comparison may help in assessing the effectiveness of the suggested reduction.

[0201] • Interactive Adjustments'. To help achieve precise alignment, the GUI may present one or more interactive tools designed to enable the user (e.g., surgeon) to make fine adjustments to the bone fragments. The fine adjustments may be facilitated by a feature of the GUI referred to as a “Gumball,” which may enable translation and / or rotation, optionally in all directions. Using the GUI, the user may drag the fragments and / or make step-by-step adjustments to ensure optimal alignment.

[0202] • Surgeon Approval'. After making the necessary adjustments and achieving satisfactory alignment, using the GUI, the user may review and / or approves the final positioning of the bone fragments. The approval may confirm that the suggested reduction is suitable for moving forward into the surgical procedure.

[0203] Optionally, a library of pre-existing templates is accessed. The pre-existing template may represent physical tools for selection, and / or may virtually represent the underlying tool for presentation on a display within a common space with the 3D model of the bone(s) and / or cartilage. The templates may include off-the-shelf templates, which may be predefined, such as by the manufacturer. The templates may be included automatically and / or manually in the surgical plan. For example, a 3D template of a certain tool may be moved by the user relative to the 3D model, such as for computing parameters for generating the graphical 3D representation of the PSI. The template may for example, for implants, screws, and / or tools that are commonly used in surgeries. The template may include one or more predefined shapes, for example, connectors and / or other ports that allow the combination of external component such as metal lots and / or laser markers. The templates may be automatically selected according to the surgical plan, and / or manually chosen by the user. Each template in the library may include a 3D model (e.g., STL file format), and / or may be associated with addition data, for example, manufacturer details, sizes, serial numbers, and / or materials. The additional data may help ensure a realistic representation. Templates may be previewed in the GUI. In certain cases or all cases, the templates may be modified and / or combined (e.g., using the GUI), which may be useful such as in complex procedures like joint surgeries. These templates may be used to automatically adjust the surgical plan, for example, predefined holes in an implant plate are shown as drill locations defined by the surgical plan. Alternatively or additionally, the templates may be automatically selected by the surgical plan, for example, the implant plate with predefined holes is automatically selected according to the drill locations defined by the surgical plan. Alternatively or additionally, the template may be used for automatically generating the pose (e.g., drilling position, cutting area) and / or the surgical tool(s) and / or for defining a component of the PSI (e.g., consider the template in the overall shape of the PSI), for example, a pocket for holding the template within the PSI. The template may be download for obtaining a complete kit and / / or set for testing and / or practicing before manufacturing. The template(s) may be presented over the 3D model within a common space on a display. The template(s) may be tested and / or simulated over the 3D model.

[0204] Measurement tools that automatically calculate predefined features, which may be defined by the surgical plan, may be applied. The measurement tools may measure define distances and / or sizes, for example, screw length, the distance between anatomical points, and / or required angles.

[0205] Optionally, one or more measurement tools are selected from multiple available measurement tools. Each measurement tool may be trained to measure a different parameter of the bone and / or cartilage, for example, defined relative to one or more anatomical landmarks identified on the bone and / or cartilage. For example, distance between two landmarks, angle between landmarks, depth of a cortical bone at a certain landmark, and the like. The measurement tools may be implemented as measurement models that have been trained to accurately perform the needed measurement such as on anatomical variations. For example, a measurement between two anatomical landmarks may vary between patients due to variations of the bone, location of a fracture, thickness of cortical bone, strength of the bone, size and / or shape of the bone, and the like.

[0206] The selection may be according to the surgical plan, and / or by a machine learning model trained to select measurement tools. The machine learning model may be trained, for example, on different surgical plans and / or images of patients and / or other data described herein with a ground truth of what measurement was made. The machine learning model may automatically learn to select the measurement tool based on manual user selections of historical cases.

[0207] The platform may automatically apply the measurement model(s) to the bone and / or cartilage to obtain different parameters.

[0208] The measurements made by the measurement model(s) may feed into different processes described herein, for example, for automatically detecting the target region, automatically computing the pose of surgical tools, the automatic generation of the graphical 3D representation of the PSI, and the like.

[0209] In an example, based on a surgical plan for hip arthroplasty (e.g., THR, resurfacing), when deciding on the position of the femoral component, the two main measurements are a head neck ratio, and a leg length discrepancy. The measurements made by the measurement models may be presented within the interactive user interface, for example, during the planning process, enabling the user to adapt the measurements and / or select different measurement tools. Anatomical landmarks used by the measurement tools to compute the measurements may be modified by the user via the GUI. The computed measurement may dynamically update accordingly.

[0210] The measurement tools may aid in selecting the correct template and / or customizing the template to fit the specific anatomical dimensions accurately. The measurement tools may be automatically selected according to the surgical plan. The automatically selected measurement tools may be automatically applied.

[0211] Additional exemplary features of the GUI for viewing, creating and / or adapting the surgical plan include:

[0212] • Viewing a list of relevant elements, for example, surgical tools, anatomy, templates, implants, guide, etc...

[0213] • Ability to hide / show each element or element group.

[0214] • Select one or more element from the menu and / or by clicking within the GUI.

[0215] • When an element is selected, the other elements may be made transparent to allow easy understanding of the element intersection with others (e.g., view screw trajectory within the anatomy).

[0216] • Hide / show measurements in one click.

[0217] • One button to generate PSI based on the surgical plan.

[0218] • Pre / post anatomy switch.

[0219] • Switch between multiple views to focus on different areas of anatomy, or surgical steps.

[0220] Referring now back to FIG. 5, GUI 502 is may be designed for viewing, creating, and / or adapting a surgical plan for computation of a graphical 3D representation of a PSI. GUI 502 may include graphical elements for generating the surgical plan, and / or for manually adapting the surgical plan. For example, GUI presents a femur bone 504 segmented from a CT of a patient, and a cutting plane 506 indicating where to cute femur 504 for hip replacement surgery. Cutting plane 506 may be adapted by a user using GUI 502. Additional examples of graphical elements include: menu 508 for selecting the PSI, menu 510 for selecting the surgical tool, and menu 512 for selecting the surgical indication. Button 514 may confirm the surgical plan.

[0221] Referring now back to FIG. 2A, at 210, a graphical 3D representation of the PSI is automatically generated according to the surgical plan, surgical simulation, and / or other determined data such as measurements, computed footprint, and the like. The automatic generation may be, for example, for one or more of: surgical guides, implants, molds, and other specialized tools tailored to each surgery's requirements, optionally according to the surgical plan.

[0222] The graphical 3D representation of the PSI may be automatically generated, for example, by a machine learning model fed the surgical plan, surgical simulation, and / or other determined data such as measurements, footprint, and the like. The machine learning model may be implemented as, for example, a 3D neural network. The machine learning model may be trained on records, where a record may include a sample surgical plan, sample surgical simulation, and / or other determined sample data such as sample measurements, sample, footprint, and the like, for a sample case of a sample patient. The record may include a ground truth of a sample graphical 3D representation of the PSI for the sample case. Alternatively or additionally, other non-machine learning approaches may be used for generating the graphical 3D representation of the PSI, for example, 3D graphical rendering packages which may follow a set of rules applied to the input, and the like.

[0223] The graphical 3D representation of the PSI may include a footprint region having a surface corresponding to the identified target region for contact with the surface of the bone and / or cartilage corresponding to the target region, and for performing the surgical action by the surgical tool(s) according to the computed pose. The user may modify the automatically generated footprint area via the interactive user interface.

[0224] The graphical 3D representation of the PSI may be automatically computed according to parameters associated with multiple user, and / or based on historical parameters of the current user. The parameters may be automatically learned and / or extracted based on historically generated PSIs of the user and / or are manually set. For example, if a certain modification is manually made to the automatically generated graphical 3D representation of the PSI by users 60% of the time, the platform learns to generate the graphical 3D representation of the PSI that already includes the predicted modification.

[0225] In some cases there may be multiple options, for example, different surgical approaches and / or different PSI designs. Each approach and / or design may be generated based on a design envelope which creates a baseline of how the graphical 3D representation of the PSI is to be generated, which may allow for training and / or standardization in the customized design process described herein.

[0226] Different PSIs may be generated in response to the same surgical plan according to one or more parameters which may be specific to the patient. For example, in cranioplasty there are 3 types of possible PSI’s: implant, cement mold or titanium mesh press mold. No modification may be made to the PSI, only to the surgical plan. In contrast, in reverse shoulder arthroplasty, there are few modification that may be done to the PSI and one configuration eventually is generated.

[0227] Optionally, a 3D model is generated based on the bone and / or cartilage of the 3D image (and / or other 3D representation) of the subject. For example, the bone is segmented from a CT scan. Alternatively, the 3D model is based on the 3D image, such as by creating a version of the 3D image that enables placement of the PSI therein. For example, the 3D image and the PSI are mapped to a common space that retains accurate physical dimensions. Placement of the graphical 3D representation of the PSI on the 3D model is simulated, for example, by 3D rendering of a combined image. The combined image may be presented within the GUI. The user may move the PSI within the 3D model, and / or may adapt the PSI and / or may adapt parameters that contributed to computing the PSI for re-computing the PSI and re-simulating the location of the PSI.

[0228] The simulation of the placement of the graphical 3D representation of the PSI may be evaluated, for example, to detect errors. The evaluation may be performed, for example, based on a set of rules, error detection models, and the like.

[0229] Examples of errors that may be automatically detected include:

[0230] • An intersection of the bone and / or cartilage with the graphical 3D representation of the PSI. The intersection may indicate that the PSI does not accurately “fit” the surface of the bone.

[0231] • A deviation of the graphical 3D representation of the PSI from a predefined envelope by more than a defined threshold, for example, about 5%, or 10%, about 5 millimeters (mm), or about 10 mm, or other values. The deviation may indicate for example, risk of damage to the patient, risk that the PSI will not accurate “fit” with the surgical procedure, and the like. For example, a screw to hold the PSI will likely fail since the thickness of the cortical bone where the screw is to be inserted is too small according to the envelope. A region of the graphical 3D representation of the PSI is too thick according to the envelope. A region of the graphical 3D representation of the PSI is too sharp according to the envelope.

[0232] An alert may be generated and presented in response to detection of one of more errors, for example, a visual message, an audio message, a visual marking, and the like.

[0233] Alternatively or additionally, the platform may attempt to automatically correct the detected errors generated based on the simulation of the placement of the graphical 3D representation of the PSI on the 3D model of the bone and / or cartilage. The error may be analyzed to determine the cause, for example, the source of a portion of the design of the graphical 3D representation of the PSI that triggered the error, a model trained to map errors to causes, and the like. Examples of causes of errors include: the target region, the pose, the surgical tool, and / or another parameter. The cause of the error may be automatically adapted in an attempt to eliminate the error, for example, random adaptation, trial and error, another value within the envelop associated with the parameter, by a model trained to make adjustments, and the like. An adapted graphical 3D representation of the PSI may be created by re-generation according to the automatic adaptation. The simulation of placement of the adapted graphical 3D representation, is performed on the 3D model. The placed adapted graphical 3D representation may be evaluated. When another error is found, the process automatic adaption, re-generation of the adapted graphical 3D representation of the PSI, and of checking for errors, may be iterated until no error is detected. The adapted graphical 3D representation of the PSI for which no error is detected may be provided, optionally presented within the GUI. The iterations may be performed as a background process, without presenting on the display the versions of the graphical 3D representations for which errors is detected.

[0234] Optionally, a 3D model of the subject predicting a post-operative procedure state of the bone and / or cartilage after surgical placement of the PSI, is generated. The 3D model of the subject may be generated based on images and / or 3D models of the corresponding bone and / or cartilage in other subjects that did not undergo surgical procedures, and / or using 3D post-op images from other subjects that underwent similar surgical procedures.

[0235] Optionally, a surgery simulation animation (e.g., video) simulating a surgical procedure using the graphical 3D representation of the PSI is automatically generated. The surgery simulation may generate a detailed, step-by-step animation of the planned surgery utilizing the PSI, optionally as defined by the surgical plan. The animation may provide a dynamic, time-sequenced portrayal of how the surgery is predicted to unfold. The animation may show the designated placement and / or use of the PSI within the context of the patient's specific surgical needs and / or patient’s specific anatomy. The simulated preview may help the surgeon visualize the procedure ahead of time, such as for ensuring a deeper understanding of the operational steps and / or the role of the PSI. The animation may be presented on a display in real time during the surgery, for acting a practical guide during the actual surgery, for example, by offering visual cues and / or deployment strategies to improve precision and outcomes. The surgical animation video may be generated following the surgical plan and / or features derived for computation of the surgical placement of the PSI on the bone, based on the 3D model and the graphical 3D representation of the PSI. The surgical animation video may be created, for example, by a generative adversarial network (GAN), animation creation model using the existing data, and the like. Optionally, an instruction for Use (IFU) dataset may be automatically generated, for example, in human readable format (e.g., text file), schematics, cartoons, images, animations, and / or videos. The IFU is for instructing a user for performing a surgical procedure following the surgical plan, using the graphical 3D representation of the PSI, and the selected surgical tool(s) at the selected pose. The IFU may be presented alongside the 3D model within the GUI. The IFU document, may include, for example, usage instructions, surgical steps, and / or safety guidelines for the PSI. The IFU may include instructions outlining how to effectively integrate the PSI into the surgical procedure. The IFU may be designed for example, to be printed with the manufactured PSI, and / or presented in electronic format on a display, such as during a surgical procedure for implanting the manufactured PSI. The IFU may be automatically generated, for example, by taking screenshots of different angles from the surgical planning step, taking measurements of location and / or angles of the surgical tools (e.g., screw lengths), and combining into a single file (e.g., document, animation video) summarizing the surgical plan and / or surgical flow. The IFU may be automatically created by a model (e.g., GAN) that is fed the surgical plan, the graphical 3D representation of the PSI, the selected surgical tool(s) at the selected pose, and / or other data.

[0236] Optionally, a unique code representing the unique graphical 3D representation of the PSI for the subject is accessed. The unique code may correspond to the patient, providing an anonymous identifier. The unique number may include a case number. The unique code may be included within the graphical 3D representation of the PSI for the subject, for example, on the surface of the PSI and / or in an interior of the PSI. The unique code may be generated for being visually distinguishable on the PSI by a human, and / or for being visually distinguishable on a CT, MRI, and / or x-ray of the subject after surgical implantation of the PSI. A dataset mapping between the subject and the unique code may be updated. The unique code (e.g., case number) may be engraved on the PSI, to allow identification of the case / patient. In case of multiple parts, numbering for each part may be provided for easy identification. The unique number may include an automatically generated QR code and / or another shape designed to be readable by digital devices to allow synergy between the physical and digital layers, for example, as marker(s) for robotic navigation, VR / AR, and additional digital data.

[0237] Optionally, a measuring shape is added to each 3D printed PSI. The measuring shape may be stamped to the printed PSI. The measuring shape may be in a fixed and / or predefined shape and / or and size, allowing precise measuring, for example, diameter, length, thickness, and the like. The measurable features of the measuring shape that are measurable may be stored in association with the graphical 3D representation of the PSI. The measuring shape may be used as a quality control add-on, such as for determining whether the printed PSI is identical to the digital file design of the PSI (e.g., the graphical 3D representation of the PSI) before using it, by matching the measurements of the measureable features of the physical measuring shape to the stored values.

[0238] Optionally, the graphical representation of the PSI includes one or more guiding components designed for guiding the surgical tool(s) for performing the surgical action according to the selected pose. For example, a guide for a saw blade to cut the bone at a selected angle, and / or bores designed to accept screws at selected angles.

[0239] An exemplary process for generating the 3D representation of the PSI includes one or more of the following features:

[0240] • Parametric Design Approach'. Each PSI may be created using a parametric design framework, which may utilize a predefined design envelope for each surgical plan (e.g., surgical indication). Using the design envelope may help ensures consistency across similar cases while allowing for necessary customization.

[0241] • Customization of PSP. Despite the uniformity in shape and size with PSIs from the same category, each instrument may be uniquely adapted to the patient’s specific anatomical features. Examples of adjustments include: minor modifications to the footprint, drill positions, cutting guides, and / or overall shape to precisely match the surgical site.

[0242] • Integration of Surgical tools'. The design process may integrate the predefined surgical tools and / or poses for the surgical tools (e.g., drill angles, positions, diameters), optionally defined based on the surgical plan, to affect corresponding features in the PSI, such as guide sleeve angles and positions.

[0243] • Creation of Unique Footprint'. The design process may include generating a unique footprint for the PSI, by considering various anatomical landmarks, surfaces, and / or predetermined exposure areas to avoid unnecessary bone exposure. The computation of the footprint may help ensures that the PSI fits precisely and / or securely in one specific location on the bone, which may enhance safety and / or efficacy.

[0244] • Modular and Integrative Features'. PSIs may include a single or multiple parts, including bridging connections between parts for complex assemblies. The PSIs may incorporate off- the-shelf components and / or include custom-fabricated parts like metal sleeves to ensure no debris is left post-surgery.

[0245] PSI Modification:

[0246] • Custom Adjustments: After initial design, PSIs can be further modified to better meet the specific needs and preferences of the operating surgeon. These modifications include adjustments to the thickness of the footprint, the area covered by the footprint, and the length or thickness of guides and sleeves.

[0247] • Impact of Modifications: These modifications are designed to refine the fit and function of the PSI without altering the underlying surgical plan or its intended outcome, ensuring that the tool performs optimally during the procedure.

[0248] Referring now back to FIG. 6, GUI 602 presents a simulation of placement of a 3D representation of a PSI 604 on a 3D model of a bone 606 of a patient. The placement may be automatically computed, for example, as an initial suggestions. A message 608 indicating detected errors may be presented, for example, indicating that 3D representation of PSI 604 intersects bone 606, meaning that the generated PSI is predicted to be misaligned with the bone. The user may use GUI 602 to re-design the PSI and / or re-align the PSI, and / or GUI 602 may automatically suggest a re-design and / or re-alignment. Graphical elements 610 may be designed for selecting a specific drill type, which may be included and / or excluded from the PSI. A calculated length of screw to be used with the computed PSI may be automatically selected.

[0249] Referring now back to FIG. 2A, at 212, one or more features may be adapted by the user via the GUI.

[0250] The graphical 3D representation of the PSI may be presented in the interactive user interface. In one or more iterations, an adaptation by the user may be received via the interactive user interface. The user may adapted, for example, the target region, the pose, and / or the surgical tools. The platform may dynamically re-computing the graphical representation of the PSI according to the adaption. The re-computed graphical 3D representation of the PSI may be presented. The user may approve the re-computed graphical 3D representation of the PSI. In response to an indication of a finalized graphical 3D representation of the PSI received via the interactive user interface, instructions for 3D printing of the PSI may be generated according to the re-computed graphical 3D representation.

[0251] Optionally, in response to an indication of a finalized graphical 3D representation of the PSI received via the interactive user interface, a library of surgical plans may be updated according to adaptation made by the user and / or other automatically detected adaptations. Alternatively or additionally, other model described herein may be automatically updated and / or may learn based on collected information.

[0252] Optionally, as part of the surgical planning and execution workflow, the GUI may generate presentations for conducting a final review of the PSI, such as to ensure that the PSI meets surgical requirements. Optionally, the 3D representation of the PSI is generated and presented within the GUI. The 3D representation may be presented along with positioning relative to the patient's anatomy. The visualization aids in confirming that the instrument aligns (e.g., perfectly) with the surgical site and / or with the specific anatomical features for which it is designed. The alignment may be automatically computed. The automated alignment may be used for automatic evaluation of the generated 3D representation of the PSI, as described herein.

[0253] Optionally, multiple users may use different client terminals 108 (or the same client terminal using different logins) to access the same case (e.g., for creating the PSI as described herein). The multiple users may simultaneously access the same case, and / or may access the same case at different times. Enabling multiple user access allows for collaboration by multiple users (e.g., surgeons) on the same PSI design and / or to obtain feedback from the different users regarding the same PSI design.

[0254] The multiple users may leave comments (e.g., textual, voice, image, video, or other formats), ask questions on the actual 3D rendering, allowing easy collaboration and communication about specific areas by pointing to locations in the 3D space. Users may make modifications and / or perform tasks by interaction with the computing environment, for example, by talking (e.g., speaking into a microphone) or writing (e.g., typing) instructions. The commenting feature may be applied to point to specific locations and / or elements in the 3D model and / or the graphical 3D representation of the PSI. Tasks related to the comments may be executed. For example, a comment and / or question may be entered by one or more users. A location and / or feature of the graphical 3D representation of the PSI impacted by the comment and / or question may be automatically identified. A task impacting the location and / or feature may be executed according to the comment and / or question. For example, a user may say “this footprint region should not be used, it may cause pain to the patient”, triggering an automatic selection of a different footprint region, and a re-computation of the PSI.

[0255] Referring back to FIG. 7, GUI 702 includes window 704 for presenting the image corresponding to 3D model and / or bone segmentation 706. GUI 702 may include control elements for scrolling between 2D slices of the CT and / or MRI images. Surgical tools and / or poses of the surgical tools for performing surgical actions may be presented on the CT and / or MRI images, for example, for simulating post-op outcomes such as after hip replacement as shown. Each of the 2D slices may be expanded to full screen view.

[0256] Referring back to FIG. 8, GUI 802 presents automatically computed 3D presentation of PSI 804. PSI 804 as shown, is designed as a surgical guide for cutting a femur according to a defined angle of a cutting plane, for example, as shown with respect to cutting plane 506 described with reference to FIG. 5.

[0257] Referring now back to FIG. 9, GUI 902 includes different graphical elements for manual adjustment of different parameters for computing a 3D representation of a PSI. For example, elements for selecting the surgical indication, the surgical action, associated surgical tools to be used with the PSI, user interface components, measurements automatically computed using the image (e.g., segmentations) used to compute the PSI, and the like.

[0258] Referring now back to FIG. 10, GUI 1002 presents status of different steps for automatic computation of a 3D representation of a PSI, for example, loading data (e.g., images, patient data) 1004, generating the 3D representation of the PSI 1006, completion of the 3D representation of the PSI 1008, sending to manual review 1010, detection of errors in the creation of the 3D representation of the PSI 1012, and a message 1012 indicating the error 1014.

[0259] Referring now back to FIG. 11, GUI 1102 presents automatic computed measurement 1104 for automatic computation of the 3D representation of the PSI, for example, for cutting of a femur 1106 according to a cutting plane 1108.

[0260] Referring now back to FIG. 12, schematics of different examples of PSIs generated based on different surgical plans are depicted, for example, for total hip replacement 1202, hip resurfacing 1204, RS A 1206, TKR 1208, and CRP 1212.

[0261] Referring now back to FIG. 2A, at 214, instructions for manufacturing the PSI based on the graphical 3D representation, are automatically generated. For example, a 3D generated object file of the PSI is created. The 3D generated object file may be, for example, presented to the user, downloaded, opened by other software, and / or sent for 3D printing (e.g., STL, OBJ, 3DS, or other formats). The instructions are provided, for example, to a manufacturing device, such as a 3D printer and / or additive manufacturing device.

[0262] Graphical 3D representations of the PSI and / or graphical 3D representations of other related components (e.g., tools, parts of the PSI) and / or templates of the PSI and / or related components, may be downloaded to a computer (e.g., client terminal) using the 3D generated object file, on any supported devices and / or software.

[0263] Optionally, 3D printing of the PSI by the 3D printer may be scheduled. Scheduling may be according to at least one of: predicted amount of time for 3D printing of the PSI, scheduled operation time when the PSI is predicted to be used, current load scheduled for the 3D printer for 3D printing other PSIs for other subjects, and scheduled operation for the other subjects requiring the other PSIs. For example, each PSI is scheduled to be printed in time to be ready for the surgical procedure. As used herein, the term manufacturing device, printer, 3D printer, and additive manufacturing device, as used interchangeably.

[0264] Files that are ready for 3D printing may be automatically generated. The files may be automatically formatted to be compatible with the manufacturing device, for example, a medicalgrade printer located within the hospital and / or at an external fabricating facility.

[0265] An interface, optionally the GUI, may display information about the printer, for example, printer availability, print times, and / or material requirements, which may allow for efficient management of production resources.

[0266] Optionally, a queue of printing tasks for multiple PSIs is managed, automatically and / or manually. Manufacturing may be scheduled for each PSI, for example, based on the urgency dictated by the surgery times and / or the estimated print durations. A prioritization in the scheduling may help ensure that the PSIs are printed and available precisely when needed, which may minimize the need for storage and handling.

[0267] At 216, the PSI is automatically manufactured by the manufacturing device according to the instructions.

[0268] The manufacturing device may be implemented as a specialized and / or dedicated PSI printer. Such specialized and / or dedicated PSI printer may be installed within hospital facilities, designed to be connected seamlessly to the system described herein.

[0269] The manufacturing device may be designed to provide a comprehensive printing solution, designed to manufacturing the PSI and to automatically perform one or more post-processing stages, for example, cleaning, curing, sterilization, and / or packaging. The manufacturing device may be designed to provide an all-in-one approach to streamline the production process from start to finish.

[0270] The manufacturing device may include bio-degradable printing technology, which may enhance the sustainability of the production process.

[0271] The manufacturing device may be designed for automatic operation, requiring minimal to no human intervention. Surgeons or hospital staff may simply retrieve the ready-to-use PSI and / or accessory surgical instruments directly from the manufacturing device, in a sterile and / or ready to use state, prepared according to the specifications (e.g., based on the surgical plan).

[0272] At 218, one or more features described with reference to 202-216 may be iterated.

[0273] Optionally, the iterations are for automated learning by respective models. Each segmentation, generated surgical plan, generated 3D representation, and other features, that may be accepted or rejected (e.g., adapted) by the user may be used for teaching and / or updating the corresponding models how to do better next time. User (e.g., surgeon) preferences for the next time the user will use the system may be learned. The teaching and / or learning may be based on the anatomy and / or surgical indication, from user inputs and / or feedback via the interactive interface. The learning process may be automated and / or manual.

[0274] Referring now back to FIG. 2B, one or more features described with reference to FIG. 2B may be implemented in addition to, as alternatives to, and / or in combination with, one or more features described with reference to FIG. 2A. One or more features described with reference to FIG. 2B may relate to automatic selection of images and automatic generation of candidate graphical 3D representations of PSIs for review and / or adaptation by a user.

[0275] At 250, image(s) are automatically added to a dataset (e.g., database). For example, medical imaging scans such as CT scans, MRI scans, and / or x-rays may be added to the PACS server for storage. A DICOM file may be automatically created for each image, and stored in association with the image in the dataset.

[0276] Additional exemplary details of image(s) are described, for example, with reference to 202 of FIG. 2A.

[0277] At 252, a subset of one or more images is automatically pulled from the datasets.

[0278] A ranking and / or picking process may automatically filter the images in the dataset to obtain a subset of relevant cases. The subset may be selected according to available data in the dataset, such as available images, and / or available text which may meet criteria indication relevancy to supported applications and / or indications. For example, CT scans of patients designated for total knee replacement, multiple x-rays of patients with detected fractures which are suitable for computation of a 3D model of the broken bone, and the like may be automatically pulled. The filter may be done, for example, based on a set of rules, by a trained machine learning model, and the like.

[0279] Is it noted that the images may be selected for the subset without regards to a surgical plan and / or design result.

[0280] The DICOM metadata related to the images selected for inclusion in the subset may be automatically pulled from the dataset.

[0281] Cases are created automatically according to the pulled images and / or DICOM metadata and / or other data.

[0282] A case refers to the data related to the image in the subset, such as including the image(s), DICOM metadata, other data, created graphical 3D representation of the PSI, and the like.

[0283] A case may be created for each image of the dataset prior to the selection of the subset, where cases are selected for inclusion in the subset, or a case may be created for each image of the subset. Cases and / or images may be filtered for selection of cases and / or images meeting availability and / or suitability criteria for automatic generation of a graphical 3D representation of a PSI.

[0284] Filtering may be done according to a score computed for each case (e.g., when cases are created for images of the dataset) and / or image (e.g., when cases are create for the images in the subset). The score may be computed according to one or more of:

[0285] • Historical analysis of use of the automatically generated graphical 3D representations by the corresponding user (e.g., surgeon). For example, users that have historically 3D printed their PSIs are assigned a higher score. Surgical indication of PSIs that have historically been 3D printed over other surgical indications are assigned a higher score.

[0286] • An analysis of quality of the case indicating likelihood of generating an accurate graphical 3D representation of the PSI for the case. Good quality images and / or images captured with certain protocols which are can be segmented with higher accuracy and / or result in a higher accuracy of the automatically generated graphical 3D representation of the PSI are assigned higher scores.

[0287] • Correlation with availability and / or suitability criteria for automatic generation of a graphical 3D representation of the PSI. Images of fractures which are not suitable for surgical correction, or images depicting other medical conditions not relevant to PSIs such as pneumonia, are assigned low scores.

[0288] At 254, automatic anonymization may be performed on the case. For example, unique codes and / or identifiers are selected. The anonymization enables generating the candidate graphical 3D representation of the PSI without identifying the subject, as described herein. At 256, the image(s) may be automatically segmented as described herein. For example, bone(s) which will be operated on are automatically segmented. A 3D model may be automatically generated based on the automated segmentation, as described herein. A candidate graphical representation of the PSI may be automatically generated according to the image(s), related surgical plan, and / or other data, as described herein. Additional exemplary details of segmentation, processing the image, surgical plans, automatic surgical planning, and / or automatic generation of the graphical 3D representation of the PSI image(s) are described, for example, with reference to 204-210 of FIG. 2A.

[0289] At 258, a notification may be sent to client terminal of a user associated with the case, for example, to the physician assigned to operate on the subject. The notification may indicate that a relevant case is available for viewing by the user. The notification may be sent, for example, to an email of the user, as a SMS message and / or via a messenger application to a mobile device of a user, and / or via a pop-up on a display of a computer when the user logs in (e.g., hospital client terminal).

[0290] The graphical 3D representation of the PSI may be generated in response to sending of the notification. The graphical 3D representation of the PSI may be generated prior to sending of the notification, such that the notification is sent in response to the graphical 3D representation of the PSI being ready. Alternatively, the graphical 3D representation of the PSI may be generated in response to sending the notification, and / or in response to the user interacting with the notification (e.g., pressing an icon or a link therein) for viewing the automatically generated graphical 3D representation of the PSI.

[0291] The graphical 3D representation of each PSI may be automatically generated as described herein.

[0292] At 260, an online 3D viewer may be automatically activated and presented on a display, such as of a desktop and / or mobile device being used by the user. The online 3D viewer may be automatically triggered in response to a user interacting with the notification, for example, clicking on an icon in the notification for online viewing.

[0293] The online 3D viewer may present details of the case, for example, patient data (e.g., anonymized), raw image, segmentation, proposed automatically generated graphical 3D representation of the PSI, and the like.

[0294] At 262, the proposed automatically generated graphical 3D representation of the PSI may be adapted, for example, as described with reference to 212 of FIG. 2A.

[0295] At 264, the PSI may be 3D printed, for example, as described with reference to 214-216 of FIG. 2A.

[0296] At 266, one or more features described with reference to 250-264 may be automatically iterated, for example, for each new image added to the dataset, and / or for batches of images added to the dataset (e.g., each day for the images added over the last 24 hours, or other time intervals). The iterations may be performed for automatically generating multiple graphical 3D representations of PSIs for multiple subjects.

[0297] Referring now back to FIG. 2C, the flowchart depicted in FIG. 2C may be an adaption of the method for automatically generating a set of candidate 3D representations of PSIs of FIG. 2B. The following are additional features of FIG. 2C in comparison to FIG. 2B:

[0298] At 257A, a unique code may be generated for each image of the images of the dataset (e.g., PACS server) and / or of the generated cases. The unique code may be, for example, a QR code or other type of scannable visualization pattern and / or link which may be selected (e.g., clicked). At 257B, each unique code may associated with its respective image and / or case.

[0299] The dataset, which may be centrally managed (e.g., PACS server) storing the images of the cases may be instructed to associated respective unique codes to respective images and / or cases. The unique codes may be uploaded to the dataset, such as to the PACS server.

[0300] At 258, the notification sent to the client terminal of each user associated with each case may include the unique code.

[0301] The unique code may be designed for remotely accessing the case associated with the unique code from the client terminal, for example, by scanning the QR code and / or pressing the link.

[0302] The unique code may be sharable with other client terminals of other users for joint access to the case, such as to enable other surgeons to collaborate on the same case, as described herein.

[0303] Alternatively or additionally, in other embodiments which may represent other adaptations of the methods described with reference FIG. 2A-2C, features 250-264 may be iterated for automatically generating multiple graphical 3D representations of PS Is for multiple subjects based on the images and / or electronic health records of the subjects and / or an automatically selected surgical plan. Features may be automatically extracted from the electronic health record and / or the image. The features may denote the surgical indication. The surgical plan for each graphical 3D representation may be automatically selected from multiple surgical plans according to the extracted features. For example, multiple graphical 3D representations of PSIs are generated each day for each patient likely to undergo surgery, by automatically accessing the PACS server to obtain images and the EHR server to obtain the electronic health record for each patient, and automatically generating the graphical 3D representation of the PSI for the patient.

[0304] The multiple created graphical 3D representations may be provided, for example, for review by different surgeons scheduled to operate on the different patients to determine whether to print the PSI (optionally after adapting the PSI). Each surgeon may access the previously automatically generated graphical 3D representation of the PSI for their patient, without needed to make a manual request. The graphical 3D representations may be automatically generated in response to the platform detecting that the required data (e.g., image, electronic record) is available for the patient, rather than in response to a manual request by the surgeon.

[0305] Optionally, a score is computed for each graphical 3D representation of each subject. The score may be computed according to at least one of: intersection of the bone and / or cartilage with the graphical 3D representation of the PSI, a deviation of the graphical 3D representation of the PSI from a predefined envelope by more than a defined threshold, and historical analysis of use of automatically generated graphical 3D representations for the respective surgical indication and / or by the corresponding surgeon, such as PSIs more likely to be 3D printed.

[0306] The graphical 3D representations may be ranked according to the score. A subset of graphical 3D representations having scores above a threshold and / or the highest ranked graphical 3D representations may be provided, for example, for access by the surgeons. The subset may represent the PSIs most likely to be 3D printed. Other PSI which are unlikely to be 3D printed may be ignored, since providing them to the surgeon may be unnecessary.

[0307] Optionally, parameters associated with the surgeon assigned for operating on the subject are accessed for each 3D representation. The parameters may be automatically learned and / or extracted based on historically generated PSIs of the surgeon and / or are manually set, for example, whether the surgeon 3D printed the PSI, and / or whether the surgeon adapted the graphical 3D representation or not, and if so, what the adaptation was. The graphical 3D representation is automatically generated by conforming to the parameters, and / or the score may be computed according to the parameters.

[0308] The multiple graphical 3D representations which are approved by the different surgeons may be scheduled for 3D printing by a 3D printed according to predicted amount of time for 3D printing of each PSI, and / or scheduled operation time and / or date when each PSI is predicted to be used.

[0309] The following are examples of adaptations and / or implementations of the flow described with reference to FIG. 2A and / or FIG. 2B, for different potential applications:

[0310] Inclusion of Osteotomies / Deformity Correction in the Patent:

[0311] Application-. Custom surgical plans for correcting deformities such as varus or valgus deformities of the knee.

[0312] • Data Acquisition-. Automatic retrieval of high-resolution imaging data like CT scans from hospital databases or manual uploads.

[0313] • Auto Segmentation: Al tools segment the bone structure and the deformity area, with options for surgeon-led adjustments.

[0314] • 3D Anatomical Model: Generates a detailed 3D model of the pre-surgery anatomy, highlighting areas needing correction.

[0315] • Surgical Simulation and Planning: Proposes corrective osteotomy cuts and allows for surgeon customization to finalize the surgical plan.

[0316] • PSI Generation: Designs specific surgical guides and instruments for performing the osteotomy and correcting deformities. • Manufacturing Interface. Produces a ready-to -print file that can be directly sent to a printer for creating the required surgical instruments.

[0317] Oncology - Using MRI and CT Scans to Determine Resection Margins:

[0318] Application-. Precision surgical instruments for tumor resections in oncology, integrating MRI and CT scans to define exact cutting margins.

[0319] • Data Acquisition-. Combines MRI and CT scans to gather detailed imaging of the tumor and surrounding tissues.

[0320] • Auto Segmentation: Precisely segments the tumor from surrounding tissues using Al, with manual refinement capabilities for the surgeon.

[0321] • 3D Anatomical Model: Creates a detailed 3D model of the tumor area to aid in planning the resection margins.

[0322] • Surgical Simulation and Planning: Simulates the resection process to determine optimal tumor removal pathways while preserving healthy tissue.

[0323] • PSI Generation: Develops specialized surgical guides that help accurately resect the tumor according to the simulation.

[0324] • Manufacturing Interface: Sends a ready-to-print file for the surgical guides to be printed, facilitating immediate application in surgeries.

[0325] Antibiotic-Infused Cement Molds - (From Scanning, Library, or Dynamic Model):

[0326] Application: Custom cement molds for use as temporary spacers or for filling bone defects.

[0327] • Data Acquisition: Captures detailed scans of the infected or damaged bone / implant or choose from the library to design a mold that fits perfectly.

[0328] • Auto Segmentation: Uses Al to delineate the bone structure and the area needing the cement spacer.

[0329] • 3D Anatomical Model: Models the bone / template defect and designs a mold that will create a cement spacer infused with antibiotics.

[0330] • Surgical Simulation and Planning: Simulates the placement of the cement spacer to ensure it fits well and delivers antibiotics effectively.

[0331] • PSI Generation: Designs molds that can be used to create antibiotic-infused cement spacers tailored to the patient's specific needs.

[0332] • Manufacturing Interface: Provides a ready-to-print file for the mold, which can be used to produce the spacer on-site or at a specialized fabricator. Reconstructive Surgeries (Using Bone Defect Correction as an example):

[0333] Application-. Custom surgical molds and implants for reconstructing significant bone defects and ensuring anatomical correctness. These molds can be used as press molds for material reshaping or cement molds for injecting material.

[0334] • Data Acquisition-. Gathers comprehensive imaging data, typically CT scans, from the affected skeletal region to capture all aspects of the bone defect.

[0335] • Auto Segmentation: Utilizes Al to segment the bone and the defect area accurately, with options for manual adjustments by the surgeon for greater precision.

[0336] • 3D Anatomical Model: Develops a precise 3D model of the existing bone structure including the defect, providing a basis for the design of custom implants or guides.

[0337] • Surgical Simulation and Planning: Simulates the reconstruction surgery using the 3D model, allowing for meticulous planning of implant placement and fixation strategies.

[0338] • PSI Generation: Designs patient-specific implants and surgical guides tailored to the unique contours and needs of the defect site, optimizing surgical outcomes.

[0339] • Manufacturing Interface: Outputs a ready-to-print file for both the mold / implant and any required spacers, facilitating on-site or external 3D printing.

[0340] Guides for Surgeries with Reduction (ORIF - Open Reduction Internal Fixation):

[0341] Application: Custom jigs and spacers for precise reduction and fixation of complex fractures, enhancing surgical outcomes and reducing recovery time during orthopedic reconstructive surgeries.

[0342] • Data Acquisition: Automatically retrieves or manually uploads detailed CT scans of the fracture site for precise imaging.

[0343] • Auto Segmentation: Employs Al to accurately segment and delineate the fractured bone structures, offering manual correction features for surgical precision.

[0344] • 3D Anatomical Model: Constructs a 3D representation of the fractured distal radius, enabling pre-surgical assessment and planning of the bone reduction.

[0345] • Surgical Simulation and Planning: Simulates the reduction process and fixation method, allowing surgeons to plan and adjust the surgical approach interactively.

[0346] • PSI Generation: Designs patient-specific surgical guides (jigs) that assist in the accurate realignment and fixation of bone fragments. Also designs molds for creating bone spacers if required.

[0347] • Manufacturing Interface: Outputs a ready-to-print file for both the jig and any required spacers, facilitating on-site or external 3D printing. Patient Specific Plates ( ORIF):

[0348] Application-. Customized fixation plates for orthopedic surgeries, specifically designed to conform to the patient's bone anatomy. These plates facilitate easier positioning, fixation, and effective bone reduction in open reduction and internal fixation (ORIF) surgeries.

[0349] Potential benefits'.

[0350] • Custom plates reduce surgery time by eliminating the need for intra-operative adjustments.

[0351] • Enhances stability and promotes better healing by ensuring perfect alignment and fixation.

[0352] • Decreases the likelihood of complications and the need for revision surgeries.

[0353] Patient Specific Implants

[0354] Application'. Customized implants designed for individual patients, catering specifically to their unique anatomical needs. These implants are used in a variety of surgical fields including orthopedics, craniofacial reconstruction, and spinal surgery.

[0355] Potential benefits'.

[0356] • Enhanced Compatibility and Comfort'. Implants tailored to the patient’s own body significantly improve comfort and functionality, as they are designed to fit perfectly without the need for extensive modification during surgery.

[0357] • Reduced Surgical Time and Complications: By minimizing the need for intraoperative adjustments and ensuring a precise fit, patient- specific implants can reduce overall surgery time and potential complications.

[0358] • Improved Clinical Outcomes: Custom implants are often associated with better integration with the body, leading to faster recovery times and improved long-term outcomes.

[0359] • Innovative Solutions for Complex Cases: Enables the treatment of complex cases where standard implants may not be feasible, providing solutions tailored to specific medical challenges.

[0360] Patient Specific Robotic Position Markers

[0361] Application: This application involves the use of 3D printed surgical guides that incorporate a specific footprint, designed to attach precisely to a defined area on a patient's bone. These guides may be equipped with embedded QR codes or similar markers that are recognizable by surgical robots.

[0362] Benefits: • Enhances the precision of robotic surgeries by allowing real-time, dynamic tracking of the bone's position.

[0363] • Reduces the time needed for manual adjustments during surgery.

[0364] • Increases the safety and accuracy of robotic-assisted procedures.

[0365] Some additional exemplary embodiments are now described.

[0366] In some embodiments, a computer implemented method for manufacturing a personalized medical instrument for a subject, comprises: within and via a user interface presented on a display: presenting a segmentation of at least one bone of the subject depicted in 3D, receiving an indication of a non-linear boundary demarcating a target region of a surface of the at least one bone, presenting a graphical element indicating a surgical action for being performed on the at least one bone, receiving a pose of the graphical element with respect to the at least one bone, rendering a graphical 3D representation of the personalized medical instrument including a footprint region having a surface corresponding to the target region for contact with the surface of the at least one bone corresponding to the target region, and a guiding component designed for guiding the surgical tool for performing the surgical action according to the pose of the graphical element, and generating instructions for additive manufacturing of the personalized medical instrument.

[0367] In some embodiments, a computer implemented method for manufacturing a personalized medical instrument for a subject, comprises: feeding into a 3D image generation machine learning model, a segmentation of at least one bone of a subject, an indication of a target region on a surface of the at least one bone, and a pose of a graphical element representing a surgical action for being performed on the at least one bone by a surgical tool, obtaining a personalized medical instrument as an outcome of the 3D image generation machine learning model, the personalized medical instrument including a footprint region having a surface corresponding to the target region for contact with the surface of the bone corresponding to the target region, and a guiding component designed for guiding the surgical tool for performing the surgical action according to the pose of the graphical element, and generating instructions for additive manufacturing of the personalized medical instrument.

[0368] In some embodiments, a computer implemented method for training a 3D image generation machine learning model, comprises: creating a training dataset comprising a plurality of records, wherein a record includes: a segmentation of a sample bone of a sample subject, an indication of a sample target region on a surface of the sample bone, a sample pose of a graphical element representing a surgical action for being performed on the sample bone by a surgical tool, and a ground truth of a sample 3D personalized medical instrument, and training the 3D image generation machine learning model on the training dataset.

[0369] Optionally, further comprising receiving, via the user interface, a selection of a surgical action for performance, and selecting the graphical element from a plurality of graphical elements according to the selected surgical action.

[0370] Optionally, further comprising, in response to the received pose of the graphical element representing the surgical action, presenting on the user interface, a recommendation of a plurality of candidate surgical tools suitable for performing the selected surgical action according to the pose, and receiving via the user interface, a selection of one of the candidate surgical tools.

[0371] Optionally, further comprising generating a recommendation of at least one of: the target region and the pose of the graphical element, based on an analysis of records of historical surgical procedures performed on individuals, the records selected according to a correlation of the at least one bone, the surgical action, and clinical procedure, between the subject and the individuals, the recommendation for obtaining a predicted a target post-op outcome.

[0372] Optionally, further comprising receiving, via the user interface, a selection of at least one surgical tools from a plurality of surgical tools, wherein the surgical action corresponds to the selected at least one surgical tool, and selecting the graphical element from a plurality of graphical elements according to the selected at least one surgical tool.

[0373] Optionally, further comprising presenting in the user interface a 3D rendering of the selected at least one surgical tool, and further comprising dynamical adapting the 3D rendering according to an adaptation of a dimension of the selected at least one surgical tool.

[0374] Optionally, further comprising generating instructions for generating a surgical kit comprising the personalized medical instrument and the selected at least one surgical tool.

[0375] Optionally, the plurality of surgical tools are defined by an industry catalogue.

[0376] Optionally, further comprising simulating: application of the personalized medical instrument at the footprint region to the target region of the surface of the at least one bone, performing the surgical action according to the pose of the graphical element to obtain a post-op state of the at least one bone, and computing a 3D rendering of the post-op state of the at least one bone.

[0377] Optionally, further comprising 3D printing of the at least one bone, and performing at least one of: quality assurance that the personalized medical instrument corresponds to the at least one bone, and simulating surgery on the 3D printed at least one bone using the personalized medical instrument. Optionally, rending the graphical representation comprises feeding the segmentation of the at least one bone, the indication of the target region, and the pose of the graphical element, into a 3D image generation machine learning model, and obtaining the graphical representation as an outcome of the 3D image generation machine learning model.

[0378] Optionally, the 3D image generation machine learning model is trained on a training data of a plurality of records, each record including a segmentation of a sample bone of a sample subject, an indication of a sample target region, a sample pose of the graphical element, and a ground truth of a sample 3D personalized medical instrument.

[0379] Optionally, the 3D image generation machine learning model is implemented based on a generative adversarial network (GAN) architecture.

[0380] Optionally, the personalized medical instrument comprises a personalized surgical jig.

[0381] Optionally, further comprising manufacturing the personalized medical instrument by an additive manufacturing device executing the instructions.

[0382] Optionally, the guiding component is designed for fitting an auxiliary element sized and shaped according to the surgical tool.

[0383] Optionally, the personalized medical instrument comprises at least one of: a single all-in- one design that incorporates at least one guiding component and at least one footprint region, a plurality of sub -components that sequentially assemble into the personalized medical instrument according to an operating sequence, and a plurality of sub-components that are sequentially placed in contact with the at least one bone according to the operating sequence.

[0384] Optionally, further comprising performing a bone density analysis of the at least one bone and at least one of: visually marking regions of the at least one bone having a bone density above a threshold recommended as being suitable for the target region and / or for the pose of the graphical element, and presenting an error message when the location of the target region and / or the graphical element is unsuitable due to bone density being less than a threshold.

[0385] Optionally, when the surgical action comprises sawing, and the surgical tool comprises a saw, the graphical element represents a sawing plane, and the guiding component comprises at least one slot having a pose corresponding to pose of the sawing plane.

[0386] Optionally, further comprising obtaining dimensions of an implant for insertion into the at least one bone and / or of a plate for fixation to the at least one bone, and further comprising evaluating the pose of the sawing plane for correspondence with the dimensions, and in response to a mismatch in the correspondence with the dimensions, further comprising recommending a new pose of the sawing plane that corresponds with the dimensions. Optionally, when the surgical action comprises insertion of an elongated tool into the bone, the surgical tool comprises the elongated tool, the graphical element represents a pose of the elongated tool within the bone, and guiding component comprises a lumen having a pose for guiding the elongated tool into the bone at the pose.

[0387] Optionally, the elongated tool and surgical action are selected from: screw for screwing into bone, nail for hammering into bone, drill for drilling a lumen in the bone, and rod for insertion into the drilled lumen.

[0388] Optionally, the surgical action comprises adjusting the at least one bone from a current pose to a target pose, the graphical element comprises the adjusted target pose of the at least one bone, and the guiding component includes surfaces for connecting to the at least one bone for maintaining the at least one bone in the adjusted target pose.

[0389] Optionally, further comprising: receiving via the user interface a pose of a fixation device for fixation to the adjusted target pose of the at least one bone via a plurality of attachment components, determining target poses of the plurality of attachment components for fixation of the fixation device to the adjusted target pose of the at least one bone, computing a transformation for transforming the at least one bone from the adjusted target pose to the current pose, applying the transformation to the target poses of the plurality of attachment components from the adjusted target pose to the current pose, and including the current pose of the plurality of attachment components in the personalized medical instrument for drilling the at least one bone at the current pose and applying the plurality of attachment components to the at least one bone at the adjusted target pose using the drilling when the at least one bone was at the current pose.

[0390] Optionally, the target pose is obtained by analyzing a current pose of the at least one bone of the subject from an opposite side of the subject denoted as healthy.

[0391] Optionally, further comprising analyzing the received pose in view of the received target region and the segmentation of the bone for determining a safety profile of the surgical action performed by the pose, and generating a recommendation for another pose and / or another target region when the safety profile is below a safety threshold indicating risk to the subject.

[0392] Optionally, further comprising receiving the safety profile via the user interface, the safety profile includes relationships to blood vessels and / or nerves.

[0393] Optionally, further comprising generating instructions for at least one of: a robot to automatically apply the personalized medical instrument to a bone of the subject, and the robot to automatically perform the surgical action using the surgical tool via the guiding component. Optionally, further comprising receiving via the user interface, an indication of a plurality of anchoring points for anchoring the personalized medical instrument to the target region, wherein the personalized medical instruments includes the plurality of anchoring points.

[0394] Optionally, further comprising: automatically detecting boundaries of a tumor within the at least one bone: defining the surgical action as cutting by a saw, and defining the graphical element as denoting a pose of a cutting plane, and automatically selecting the target region and the pose of the cutting plane for resection of the at least one bone including the tumor.

[0395] Optionally, the segmentation of the at least one bone is generated by projecting a first image of the at least one bone obtained by a first imaging device on a second image of the at least one bone obtained by a second imaging device, and segmenting the projection.

[0396] Optionally, the at least one bone comprises a source bone and a target bone, and further comprising: automatically analyzing angulation of the source bone relative to the target bone, defining the surgical action as cutting by a saw, and defining the graphical element as denoting a pose of a cutting plane, and automatically selecting a plurality of poses of the cutting plane for wedge resection of the source bone for adjusting an angulation of the source bone for fitting to target bone.

[0397] Optionally, the source bone comprises a head of a femur and the target bone comprises an acetabulum of a pelvis.

[0398] Optionally, further comprising obtaining at least one 2D image of the at least one bone, generating a 3D reconstruction of the at least one bone from the at least one 2D image, and segmenting the 3D reconstruction.

[0399] As used herein, the terms personalized surgical instrument (PSI), personalized medical device, personalized surgical guidance device, and jig, are used interchangeably.

[0400] As used herein, the term bone may be interchanged with the term bone and / or cartilage. For example, the footprint of the personalized medical device may be designed to be placed in contact with the surface of the bone and / or cartilage.

[0401] As used herein, the segmentation of the bone(s) of the subject may include other entities, for example, soft tissues, cartilage, tumor, regions of abnormally high calcification (e.g., due to cancer), fluid accumulation, infectious regions, previously installed implants, and the like.

[0402] An aspect of some embodiments of the present invention relates to systems, methods, computing device, and code instructions (e.g., stored on a data storage device and executable by one or more processors) for manufacturing a personalized medical instrument for a subject. The following are implemented within and via a user interface (UI) (e.g., graphical user interface (GUI) presented on a display: A segmentation of at least one bone of the subject depicted in 3D is presented in the UI. An indication of a non-linear boundary demarcating a target region of a surface of the at least one bone is received via the UI. A graphical element indicating a surgical action for being performed on the at least one bone is presented in the UI. A pose of the graphical element with respect to the at least one bone is received via the UI. A graphical 3D representation of the personalized medical instrument is rendered. The personalized medical instrument includes a footprint region having a surface corresponding to the target region for contact with the surface of the bone corresponding to the target region, and a guiding component designed for guiding the surgical tool for performing the surgical action according to the pose of the graphical element. Instructions for additive manufacturing of the personalized medical instrument are created.

[0403] An aspect of some embodiments of the present invention relates to systems, methods, computing device, and code instructions (e.g., stored on a data storage device and executable by one or more processors) for manufacturing a personalized medical instrument for a subject. A processor feeds into a 3D image generation machine learning model, a segmentation of at least one bone of a subject, an indication of a target region on a surface of the at least one bone, and a pose of a graphical element representing a surgical action for being performed on the at least one bone by a surgical tool. The processor obtains a personalized medical instrument as an outcome of the 3D image generation machine learning model. The personalized medical instrument includes a footprint region having a surface corresponding to the target region for contact with the surface of the bone and / or cartilage corresponding to the target region, and a guiding component designed for guiding the surgical tool for performing the surgical action according to the pose of the graphical element. The processor generates instructions for additive manufacturing of the personalized medical instrument.

[0404] An aspect of some embodiments of the present invention relates to systems, methods, computing device, and code instructions (e.g., stored on a data storage device and executable by one or more processors) for training a 3D image generation machine learning model, for example, a generative adversarial network (GAN). A processor creates a training dataset of multiple records. A record includes: a segmentation of a sample bone of a sample subject, an indication of a sample target region on a surface of the sample bone, a sample pose of a graphical element representing a surgical action for being performed on the sample bone by a surgical tool, and a ground truth of a sample 3D personalized medical instrument. The 3D image generation machine learning model is trained on the training dataset.

[0405] At least some embodiments described herein relate to a platform for surgical jig automation. By allowing surgeons to accurately position personalized surgical instruments and implants, surgical jigs have the potential to revolutionize orthopedic surgeries and to substantially reduce O.R. time, and / or minimize risks and / or complications.

[0406] At least some embodiments descried herein improve upon current use of personalized surgical jigs, which is rather scarce due to limited accessibility and extensive logistic resources that are currently required. The state of the art solutions offers a surgical jig that is manually designed and is provided within 2-5 weeks at a minimal cost of IK USD per unit.

[0407] At least some embodiments descried herein enable scaling and / or democratizing surgical jigs engineering by providing a personalized jig design optionally within minutes, based on the patients' imaging data (e.g., a CT scan) and a series of the surgeon's preferences (e.g., one or more of: surgical approach, desired surgical set, desired clinical outcome, and other inputs described herein). The platform automatically generates instructions (e.g., an STL file) for 3D printing of the jig, which may be done at the point of care.

[0408] At least some embodiments descried herein relate to a platform that may disrupt not only the current clinical landscape by allowing any surgeon to achieve the speed and precision of the most proficient, experienced surgeon, but may also pave the way for surgical robots by significantly easing their way into the O.Rs.

[0409] At least some embodiments descried herein relate to a fully automated, end-to-end solution for some cases, and semi-automated solutions for other cases, for example, requiring one or two inputs from a user.

[0410] Customized surgical jigs created using embodiments described herein may improve upon current approaches, for example, by reducing orthopedic surgery duration (in minutes) for example, from about 60-180 to 45-150. Surgeon time (hours) may be reduced by up to about 0.25 hours from a total of about 2-5 hours.

[0411] At least some embodiments described herein may provide interfaces for surgical workflows through Al-powered digitalization. The preoperative planning platform described herein may democratize the process by providing surgeons with a streamlined interface, seamless integrated with EMRs, and / or state-of-the-art tools that make planning and executing surgical procedures simpler (e.g., 3D- printed surgical guides / AR / mixed reality / Robotics). Patient outcomes may be improved by using embodiments described herein for improving surgical precision and / or efficiency.

[0412] At least some implementations of the systems, methods, computing devices, and / or code instructions (stored on a data storage device and executable by one or more hardware processors) described herein address the technical problem of generating customized medical devices for guiding surgery for a subject, such as personalized surgical jigs. Existing engineering and / or design processes are cumbersome, requiring highly-trained personnel working simultaneously, and thus limited in used.

[0413] At least some implementations of the systems, methods, computing devices, and / or code instructions described herein provide one or more of the following potential advantages: enhancing accuracy, reducing O.R. time, minimize risks such as infections and blood loss, reducing requiring imaging, and improve logistics.

[0414] At least some implementations of the systems, methods, computing devices, and / or code instructions described herein address the technical problem of accurately planning orthopedic surgeries (e.g., to remove tumors and / or correct deformity) in order to achieve optimal surgical results. For the purpose of planning the surgery at the required level of detail, expensive means are used, and appropriate training for their operation is required .Besides dedicated software for converting the anatomy into high-resolution 3D representations, significant practical knowledge in the field of design and engineering, anatomical knowledge and familiarity with the surgical procedure is also required.

[0415] At least some embodiments described herein address the aforementioned technical problems and / or improve upon the aforementioned prior art by creating customized surgical guidance tools (e.g., Surgical Guides, PSI, jig) to guide surgical tools to perform the various surgical operations (e.g., drilling, sawing, positioning) in the most precise manner and / or according to preliminary planning.

[0416] At least some embodiments described herein address the aforementioned technical problems and / or improve upon the aforementioned prior art by providing an interface that facilitates pre-surgical planning, and / or automatic creation of personalized surgical guidance tools (Jigs, Surgical Guides, PSI) optionally based on an anatomical database, is provided.

[0417] In deformity correction and / or tumor removal surgeries in the world of orthopedics, preoperative planning with height resolution is extremely important in order to choose the optimal treatment technique. During the planning of the surgery, a certain order of operations may be followed, in which the definition of the deformity, the determination of the cutting planes, and the decision on the fixation component are made, sometimes also including the planning and creation of a customized implant. In this order of operations there are several weak points that may lead to a suboptimal result of the operation. The use of 3D technology based on large databases makes it possible to receive suggestions for the anatomical structure after the repair and thus - plan in advance the placement of drills to fix the fixation component, pre-planning of cutting planes and repositioning bony structures, all this automatically and based on a suitable database. In addition to saving time and the problem of accessibility for planning such procedures, such a solution has the possibility of reducing the chance of making a mistake, shortening the analysis time and / or the number of reflections and / or enabling predictable results in this challenging analysis.

[0418] At least some embodiments described herein address the aforementioned technical problems and / or improve upon the aforementioned prior art by a user (e.g., surgeon) entering a patient's imaging data into the system (e.g., CT MRI...) and optionally a set of preferences (for example, angle of correction of the bone, surgical approach, preferred set of tools, number of docking points, safety considerations such as distance from blood vessels or nerves). Instructions (e.g., a file) for 3D printing a personalized medical instrument (e.g., Patient specific instrument (PSI) or surgical jig) may be generated.

[0419] In cases where 3D technology is used to produce customized medical surgical guidance tools (e.g., jigs - PSI) to perform the surgical procedure (e.g., osteotomy and / or) fixation, the production process of the jig may include one or several defined steps, for example:

[0420] 1. Acquisition of the anatomy based on a two-dimensional imaging file.

[0421] 2. Separating the relevant anatomy by a segmentation process.

[0422] 3. Creating a 3D file of the anatomy selected in the segmentation process.

[0423] 4. Planning the course of surgery and / or creating a post position based on a mirror image of the healthy side / comparison with the database.

[0424] 5. Designing the appropriate jigs based on the anatomy and actions of the surgery (drilling / sawing / positioning) .

[0425] 6. The production of tools using 3D printing.

[0426] 7. Performing a trial (the course of surgery on an anatomical model) before entering the operating room.

[0427] In the aforementioned method, beyond the precision of performing the surgery (e.g., osteotomy or other) at the degree and millimeter level in relation to the planning through the use of the customized surgical guidance tool, the operation time may be shortened and / or the number of x-ray images captured may be reduced. An ideal and / or predictable surgical result may be ensured according to the pre-operative planning.

[0428] Referring now back to FIG. 13, features described with reference to 1302-1318 may be performed within a user interface, optionally a graphical user interface (GUI) presented on a display, for example, via a user interacting with the UI, such as by touching a touch screen, verbal commands, a mouse, and the like.

[0429] At 1302, a segmentation of one or more bones of the subject is presented in the UI. The segmentation may be a 3D model of the bone. Bone tumor and / or tumor that invaded the bone may be depicted. The segmentation may a 3D reconstruction of the bone generated from a 3D image, such as CT and / or MRI.

[0430] Optionally, the segmentation is generated from different types of imaging devices, for example, CT, MRI, ultrasound, x-ray, and the like. A first image of depicting the bone obtained by a first imaging device may be projected on a second image of the bone obtained by a second imaging device, to obtain a projected image. The projected image may be segmented.

[0431] At 1304, an indication of a non-linear boundary demarcating a target region of a surface of the bone is received from the user. For example, the user draws a free-form boundary on the surface of the bone by touching the screen, using a mouse, using an electronic pen, and the like. The boundary defines the footprint, i.e., contract region of the manufactured custom medical instrument.

[0432] At 1306, a surgical action may be selected by the user (e.g., via the UI) from multiple candidate surgical actions. Exemplary surgical actions include: cutting, fixation, positioning, drilling, re-orientation, injection, and reduction. Some surgical actions may be represented by a common graphical element. For example, an elongated cylinder may represent, for example, drilling, injection, and insertion of screws and / or nails for reduction, re-orientation, and / or positioning. Cutting may refer to sawing. Fixation may refer to placing the bone in a fixed location, such as by screws, nails, fixation plate, and frictional pressure applied by an implant. Positioning may refer to temporary placement of the bone. Drilling ma refer to making a hole in the bone during a drill. Re-orientation may refer to changing pose of a bone to another pose which is not the original pose of the bone. Injection may refer to insertion of an element into the bone. Reduction may refer to changing pose of a bone to another pose which is the same or similar to the original pose of the bone, such as for fracture reduction.

[0433] The surgical action represents a basic building block of the orthopedic surgery. The graphical element is presented according to the selected surgical action. A guiding component of the customized medical instrument is created for guiding a surgical tool to perform the surgical action, as described herein.

[0434] Some examples of surgical actions, and corresponding tools, graphical elements, and guiding components, are now described.

[0435] Optionally, when the surgical action is sawing, the surgical tool is a saw. The graphical element represents a sawing plane, for example, depicted as a 2D (or 3D with a certain thickness) plane which may have a width corresponding to the dimension of the saw. The guiding component may be a slot having a pose corresponding to selected pose of the sawing plane. Optionally, when the surgical action is insertion of an elongated tool into the bone, the surgical tool is the elongated tool, for example, screw for screwing into bone, nail for hammering into bone, drill for drilling a lumen in the bone, and rod for insertion into the drilled lumen. The graphical element represents a pose of the elongated tool within the bone, for example the graphical element is an elongated cylinder that can be positioned within the bone, showing the pose of drilling and / or insertion of the tool within the bone. The guiding component includes a lumen (e.g., sleeve) having a pose for guiding the elongated tool into the bone at the pose.

[0436] Optionally, when the surgical action is adjusting the bone from a current pose to a target pose, the graphical element is the adjusted target pose of the bone, for example, re-orientation, reduction, and fixation. For example, the user is able to rotate and / or move the bone or piece thereof (e.g., in a broken bone) to a new pose. The guiding component includes surfaces for connecting to the bone for maintaining the bone in the adjusted target pose.

[0437] It is noted that the order of 1304 and 1306 may be reversed, performed substantially simultaneously, and / or iterated in different sequences.

[0438] At 1308, the graphical element indicating the surgical action for being performed on the bone is presented in the UI. The graphical element corresponds to the selected surgical action.

[0439] Optionally, a selection of a surgical tool from multiple different available surgical tools for performing the surgical action, may be received via the UI. The surgical tools may be, for example, defined by an industry catalogue. For example, different sizes and / or designs of drill bits. A 3D rendering of the surgical tool, with dimensions, may be presented within the UI, to enable the user to dynamically adapt the pose of the 3D surgical tool. The user may change one or more dimensions of the surgical tool, triggering dynamic update of the 3D rendering of the surgical tool in the UI. For example, changing diameter of a drill.

[0440] The graphical element may be selected from multiple different graphical elements according to the selected surgical tool. For example, when the user selects a certain saw, the 3D plane is defined according to the dimensions of the selected saw.

[0441] At 1310, a pose of the graphical element with respect to the bone is received by the UI. The user may adjust the pose to the graphical element with respect to the bone, and / or place multiple different graphical elements. The user may click an icon to indicate when done.

[0442] At 1312, other data may be received from the user, and / or automatically generated.

[0443] For example, the user may be placed anchoring points on the bone for anchoring the personalized medical instrument to the target region. The personalized medical instruments includes the defined anchoring points. In another example, the user may enter details of the implant and / or fixation plate which will be used, or select from a list of available implants and / or fixation plates.

[0444] In yet another example, the user may enter details of a safety profile, such as distance from blood vessels and / or nerves.

[0445] Optionally, pose of attachment components are computed. The pose indicates how to insert (e.g., drill) the attachment components into a current pose of the bone(s) to obtain a target pose of the attachment components when the bone(s) is adjusted to a target pose. For example, holes are drilled in the bone at the current pose. The bone is the adjusted to the target pose. A fixation plate is used to fix the bone at the adjusted target pose, by inserting the attachment components into the drilled holes. The pose of the drilled holes at the adjusted target pose is designed to correspond to the fixation plate at the adjusted target pose. The process may include: receiving via the user interface a pose of a fixation device for fixation to the adjusted target pose of the bone(s) via attachment components. Determining target poses of the attachment components for fixation of the fixation device to the adjusted target pose of the bone(s). Computing a transformation for transforming the bone(s) from the adjusted target pose to the current pose. The transformation may be a transformation matrix, and / or an adjustment to the 3D representation of the bone(s). The transformation is applied to the target poses of the attachment components, from the adjusted target pose of the bone to the current pose of the bone. The transformation may be applied by applying the matrix, and / or rotating the 3D bone with attachment components therein. The current pose of the attachment components are include in the rendering of the personalized medical instrument for drilling the bone(s) at the current pose, then during the procedure moving the bone to the target pose, and applying the attachment components to the bone(s) at the adjusted target pose using the drilling that was done when the bone(s) was at the current pose.

[0446] At 1314, one or more automated recommendations may be generated.

[0447] Optionally, in response to the received pose of the graphical element representing the surgical action, presenting on the user interface, a recommendation of candidate surgical tools suitable for performing the selected surgical action according to the pose, may be presented on the UI. A selection of one of the candidate surgical tools may be obtained via the UI.

[0448] Optionally, dimensions of an implant (e.g., hip replacement surgery, knee replacement, filling in removed cancerous bone) and / or fixation plate for insertion into the bone are received, for example, according to the selected implant and / or fixation plate. The pose of the sawing plane (and / or other graphical elements based on corresponding surgical actions such as drilling) may be evaluated for correspondence with the dimensions of the implant and / or fixation plate. In response to a mismatch in the correspondence with the dimensions, an automated recommendation fora new pose that corresponds with the dimensions may be generated.

[0449] Optionally, a target pose for adjustment of the bone may be automatically obtained by analyzing a current pose of the corresponding bone of the subject from an opposite side of the subject which is healthy.

[0450] Optionally, the pose of the graphical element may be analyzed in view of the received target region and the segmentation of the bone for determining the safety profile of the surgical action performed by the pose. A recommendation for another pose and / or another target region may be generated when the safety profile is below a safety threshold indicating risk to the subject.

[0451] Optionally, boundaries of a tumor within the bone may be automatically detected. The surgical action may be automatically defined as cutting by a saw. The graphical element may be automatically defined as denoting a pose of a cutting plane. The target region and the pose of the cutting plane may be automatically selected for resection of the bone including the tumor.

[0452] Optionally, when the bones include a source bone (e.g., femur) and a target bone (e.g., pelvis), angulation of the source bone (e.g., a head of the femur) relative to the target bone (e.g., an acetabulum of the pelvis) may be automatically analyzed, and angulation for correction may be automatically computed. The surgical action may be automatically defined as cutting by a saw. The graphical element may be automatically defined as a pose of a cutting plane. Multiple poses of the cutting plane may be automatically selected for wedge resection of the source bone (e.g., femur) for adjusting an angulation of the source bone (e.g., femur head) for fitting to the target bone (e.g., acetabulum).

[0453] Optionally, the recommendation is generated based on an analysis of records (e.g., of a dataset) of historical surgical procedures performed on individuals. The records may be selected according to a correlation between the current subject and records of the individuals, using one or more of: corresponding bones being operated on, corresponding surgical actions, corresponding clinical procedures (e.g., similar operations), and the like. The recommendation may be selected for obtaining a predicted a target post-op outcome that corresponds to a post-op outcome of the records, for example, successfully inserted implant, successful wedge resection, and the like.

[0454] The recommendation may be for the target region and / or for the pose of the graphical element, optionally according to the analysis of the dataset of records.

[0455] Optionally, a bone density analysis of the bone(s) is performed according to the imaging data and / or segment. The recommendation may include regions of the bone(s) having a bone density above a threshold, which are recommended as being suitable for the target region and / or for the pose of the graphical element. The recommended regions may be visually marked on the bone segmentation in the UI, for example, color coded. The recommendation may include presenting an error message when the location of the target region and / or the graphical element is unsuitable due to bone density being less than a threshold.

[0456] At 1316, a graphical 3D representation of the personalized medical instrument may be automatically rendered. The personalized medical instrument includes a footprint region define by the user entered boundary, having a surface corresponding to the target region for contact with the surface of the bone corresponding to the target region. The personalized surgical guidance includes one or more guiding components designed for guiding the surgical tool for performing the surgical action according to the pose of the graphical element.

[0457] Optionally, the rendering of the graphical 3D representation is performed by feeding the segmentation of the bone, the indication of the target region (e.g., user entered boundary), and the pose of the graphical element, into a 3D image generation machine learning model. The graphical representation may be obtained as an outcome of the 3D image generation machine learning model.

[0458] The personalized medical instrument may include a personalized surgical jig.

[0459] Optionally, a simulation is performed. The simulation may include application of the personalized medical instrument at the footprint region to the target region of the surface of bone. The surgical action according to the pose of the graphical element is simulated to obtain a predicted post-op state of the bone(s). A 3D rendering of the post-op state of the bone may be presented in the UI. The user may determine that the post-op state is suitable, in which case the personalized medical instrument is 3D printed, or the user may determine that the post-op state is unsuitable in which case the user may adjust the target region and / or pose, and / or other parameters.

[0460] Optionally, the guiding component is designed for fitting an auxiliary element sized and / or shaped according to the surgical tool, for example, the auxiliary is designed to click into the guiding component. For example, the guiding component may be a slot having a width of 5 millimeters, and the auxiliary element may be an insert (e.g., reusable, made of metal) designed to fit into the slog for reducing the width of the original slot down to 1.6 mm, or 2 mm, or other value according to the saw that is actually being used. The auxiliary element may be selected and inserted during the actual procedure according to the actual saw blade being used.

[0461] At 1318, the 3D ML model may be accessed and / or trained. The 3D image generation machine learning model may be implemented based on a generative adversarial network (GAN) architecture.

[0462] Optionally, the 3D ML model is trained by creating a training dataset of multiple records.

[0463] A record includes: a segmentation of a sample bone of a sample subject, an indication of a sample target region on a surface of the sample bone, a sample pose of a graphical element representing a surgical action for being performed on the sample bone by a surgical tool, and a ground truth of a sample 3D personalized medical instrument. The 3D image generation machine learning model is trained on the training dataset.

[0464] At 1320, one or more features described with reference to 1304-1316 may be iterated, for example, by adapting different features and / or selecting different features, which may result in different versions of the 3D representation. The iterations may be performed until the user is satisfied with the rendered 3D representation.

[0465] At 1322, instructions for additive manufacturing (i.e., 3D printing) of the personalized medical instrument are generated.

[0466] The personalized medical instrument may be 3D printed by an additive manufacturing device executing the instructions.

[0467] The personalized medical instrument may be designed as, for example:

[0468] • A single all-in-one design that incorporates one or more guiding components and one or more footprint regions.

[0469] • Multiple sub-components that sequentially assemble into the personalized medical instrument according to an operating sequence. The sub-components may click into one another. For example, a first sub-component is a base with footprint. A second subcomponent is the guiding component that clicks into the first sub-component once in place.

[0470] • Multiple sub-components that are sequentially placed in contact with the bone according to the operating sequence. For example, a first sub-component is contact with the bone for making a first cut. The first sub-component is removed, and a second sub-component is placed in contact with the bone for making a second cut.

[0471] Optionally, instructions for generating a surgical kit that includes the personalized medical instrument and the selected surgical tools for use using the personalized medical instrument may be generated. The kit may be assembled and / or manufactured according to the instructions. For example, the kit is manually assembled and / or automatically assembled by a robot, by obtaining the selected surgical tools, such as from a storage of multiple different surgical tools. The kit represents a customized surgical tool kit for the surgery of the specific patient, which reduces the number of tools in the kit and / or includes accurate tool. In comparison, for example, to standard approaches in which multiple different versions of the same tool are provided for different sizes, none of which are necessarily the most accurate for the patient. Optionally, instructions are generated for 3D printing of the bone(s). The 3D printed bones and the 3D printed personalized medical instrument may be used for performing, for example, quality assurance that the personalized medical instrument corresponds to the bone(s) (e.g., placing the footprint of the personalized medical instrument on the target region of the surface of the bone), and / or simulating surgery on the 3D printed bone(s) using the personalized medical instrument (e.g., for training and / or pre-op planning and / or rehearsing the procedure).

[0472] At 1324, the subject may be treated using the personalized medical instrument. The treatment may be performed automatically by a robot, for example, by generating instructions for at least one of: a robot to automatically apply the personalized medical instrument to a bone of the subject, and for the robot to automatically perform the surgical action using the surgical tool via the guiding component.

[0473] Alternatively or additionally, a physician may treat the subject using the personalized medical instrument.

[0474] Referring now back to FIG. 14, the schematic of the exemplary personalized device for hip replacement is shown.

[0475] Hip replacement encompasses hip resurfacing, hip hemi arthroplasty, and / or total hip replacement

[0476] Exemplary clinical impacts and / or appeal include: high accuracy; Reducing LLD (Leg Length Discrepancy) and instability rate; Expanding the current use of robotic assisted hip replacement procedures; and Shorter operation correlates with reduced prosthetic joint infection.

[0477] Exemplary operational benefits include: Shorter surgeries; Reduction of O.R radiation; Independent of O.R imaging; Minimize operation errors and complications; and Reduced reoperation rate.

[0478] Estimated operation time reduction is about 15-20 minutes (e.g., about 25%).

[0479] Referring now back to FIG. 15, the schematic of the exemplary personalized device for distal radius fractures, is presented.

[0480] Exemplary clinical impacts and / or appeal include: High accuracy in an error prone procedure. Allows a precise positioning of a plate based on the fracture’s features.

[0481] Exemplary operational benefits include: A more efficient surgical flow; Shorter surgeries; Reduction of O.R radiation; Independent of O.R imaging; Minimize operation errors and complications; and Reduced re-operation rate.

[0482] Estimated operation time reduction is about 15-20 minutes (e.g., about 40%).

[0483] Referring now back to FIG. 16, the schematic of the exemplary personalized device for osteotomy and resection surgeries (e.g., orthopedic oncology) is presented. Exemplary clinical impacts and / or appeal include: Enables a precise, multiplane correction; Enabling correction and fixation in one phase; Safe and minimum oncological margins; Optimizing tumor removal is highly challenging; High degrees of freedom in terms of contour, anchoring elements etc; and The surgeon is actively engaged in the design.

[0484] Exemplary operational benefits include: Predicted better clinical outcome and reduced need for repeated surgery; Shorter surgeries; Reduction of O.R radiation; Independent of O.R imaging; Minimize operation errors and complications; and Reduced re operation rate.

[0485] Estimated operation time reduction is about 1-1.5 hours (e.g., about 33%).

[0486] Referring now back to FIG. 17, which is a schematic of an exemplary personalized device for reverse shoulder arthroplasty is shown.

[0487] Exemplary clinical impacts and / or appeal include: A fairly deterministic clinical outcome in a scenario where only 1 chance is given to obtain the desired outcome; and Targets this procedure's most common pitfail.

[0488] Exemplary operational benefits include: Rendering a highly common procedure more efficient; Shorter surgeries; Reduction of O.R radiation; Independent of O.R imaging; Minimize operation errors and complications; and Reduced re-operation rate.

[0489] Estimated operation time reduction is about 15-20 minutes (e.g., about 20%).

[0490] Referring now back to FIG. 18, a schematic of an exemplary personalized device for dysplasia of the hip (DDH) corrective surgery is shown.

[0491] Exemplary clinical impacts and / or appeal include: Rendering an extremely complicated surgical course to deterministic and smooth procedure; Enables surgeons to operate pediatrics; Minimize blood loss; Shorter fluoroscopy time; and Minimize growth plate damage.

[0492] Exemplary operational benefits include: Rendering a highly common procedure more efficient; Shorter surgeries; Reduction of O.R radiation; Independent of O.R imaging; Minimize operation errors and complications; and Reduced re-operation rate.

[0493] Estimated operation time reduction is about 1.5-2 hours (e.g., about 40%).

[0494] Referring now back to FIG. 19, the flowchart depicting use cases for automatic generations of personalized medical instruments is presented.

[0495] Optionally, the approach for creating the personalized medical instrument described herein may be used for different use cases, for example, generation osteotomy, oncological case in which large resection is necessary (optionally using a process for detecting boundaries of the growth), and / or for correction of deformity (optionally using a process for suggesting how to resect bone). Referring back to FIG. 20, schematics depicting the flow for creating the personalized medical instrument for resection of the bone with tumor based on boundaries of the tumor is shown.

[0496] Alternatively or additionally, the personalized medical instrument may be created based on a surgeon's decision. After the presentation of the anatomy, the surgeon may mark the location of the cutting plane on the bone within the user interface. The order of features may be the same as described herein, for example, with reference to FIG. 20.

[0497] In the case of a procedure to correct deformation of the bone, the degree of correction may be automatically determined. Cutting planes may be automatically placed according to the determined correction angle, for resection of the bone to obtain the correction. The user may confirm the position of the planes. The order of features may resume as described herein, for example, with reference to FIG. 20.

[0498] The descriptions of the various embodiments of the present invention have been presented for purposes of illustration, but are not intended to be exhaustive or limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein was chosen to best explain the principles of the embodiments, the practical application or technical improvement over technologies found in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.

[0499] It is expected that during the life of a patent maturing from this application many relevant PSIs will be developed and the scope of the term PSI is intended to include all such new technologies a priori.

[0500] As used herein the term “about” refers to ± 10 %.

[0501] The terms "comprises", "comprising", "includes", "including", “having” and their conjugates mean "including but not limited to". This term encompasses the terms "consisting of" and "consisting essentially of".

[0502] The phrase "consisting essentially of" means that the composition or method may include additional ingredients and / or steps, but only if the additional ingredients and / or steps do not materially alter the basic and novel characteristics of the claimed composition or method.

[0503] As used herein, the singular form "a", "an" and "the" include plural references unless the context clearly dictates otherwise. For example, the term "a compound" or "at least one compound" may include a plurality of compounds, including mixtures thereof.

[0504] The word “exemplary” is used herein to mean “serving as an example, instance or illustration”. Any embodiment described as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments and / or to exclude the incorporation of features from other embodiments.

[0505] The word “optionally” is used herein to mean “is provided in some embodiments and not provided in other embodiments”. Any particular embodiment of the invention may include a plurality of “optional” features unless such features conflict.

[0506] Throughout this application, various embodiments of this invention may be presented in a range format. It should be understood that the description in range format is merely for convenience and brevity and should not be construed as an inflexible limitation on the scope of the invention. Accordingly, the description of a range should be considered to have specifically disclosed all the possible subranges as well as individual numerical values within that range. For example, description of a range such as from 1 to 6 should be considered to have specifically disclosed subranges such as from 1 to 3, from 1 to 4, from 1 to 5, from 2 to 4, from 2 to 6, from 3 to 6 etc., as well as individual numbers within that range, for example, 1, 2, 3, 4, 5, and 6. This applies regardless of the breadth of the range.

[0507] Whenever a numerical range is indicated herein, it is meant to include any cited numeral (fractional or integral) within the indicated range. The phrases “ranging / ranges between” a first indicate number and a second indicate number and “ranging / ranges from” a first indicate number “to” a second indicate number are used herein interchangeably and are meant to include the first and second indicated numbers and all the fractional and integral numerals therebetween.

[0508] It is appreciated that certain features of the invention, which are, for clarity, described in the context of separate embodiments, may also be provided in combination in a single embodiment. Conversely, various features of the invention, which are, for brevity, described in the context of a single embodiment, may also be provided separately or in any suitable subcombination or as suitable in any other described embodiment of the invention. Certain features described in the context of various embodiments are not to be considered essential features of those embodiments, unless the embodiment is inoperative without those elements.

[0509] Although the invention has been described in conjunction with specific embodiments thereof, it is evident that many alternatives, modifications and variations will be apparent to those skilled in the art. Accordingly, it is intended to embrace all such alternatives, modifications and variations that fall within the spirit and broad scope of the appended claims.

[0510] It is the intent of the applicant(s) that all publications, patents and patent applications referred to in this specification are to be incorporated in their entirety by reference into the specification, as if each individual publication, patent or patent application was specifically and individually noted when referenced that it is to be incorporated herein by reference. In addition, citation or identification of any reference in this application shall not be construed as an admission that such reference is available as prior art to the present invention. To the extent that section headings are used, they should not be construed as necessarily limiting. In addition, any priority document(s) of this application is / are hereby incorporated herein by reference in its / their entirety.

Claims

WHAT IS CLAIMED IS:

1. A computer implemented method for generating a personalized surgical instrument (PSI) for a subject, comprising: automatically detecting at least one target region of a surface of at least one bone and / or cartilage on a 3D representation of the subject, defined by a surgical plan defining a surgical indication; automatically computing a pose of at least one surgical tool for performing the surgical action defined by the surgical plan; automatically generating a graphical 3D representation of the PSI according to the surgical plan, the graphical 3D representation of the PSI including a footprint region having a surface corresponding to the at least one target region for contact with the surface of the at least one bone and / or cartilage corresponding to the at least one target region, and for performing the surgical action according to the pose; and automatically generating instructions ready for 3D printing of the PSI according to the graphical 3D representation.

2. The computer implemented method of claim 1, further comprising within and via an interactive user interface, in at least one iteration: presenting the graphical 3D representation of the PSI; receiving via the interactive user interface, an adaptation by at least one user of at least one of: the at least one target region, the pose, the at least one surgical tool, and the surgical plan; dynamically re-computing the graphical representation of the PSI according to the adaption; and presenting the re-computed graphical 3D representation of the PSI; and in response to an indication of a finalized graphical 3D representation of the PSI received via the interactive user interface, generating instructions for 3D printing of the PSI according to the re-computed graphical 3D representation.

3. The computer implemented method of claim 2, further comprising: in response to an indication of a finalized graphical 3D representation of the PSI received via the interactive user interface, updating a library comprising a plurality of surgical plans according to at least one adaptation made by the user.

4. The computer implemented method of claim 1, further comprising automatically selecting the at least one surgical tool according to at least one of: the pose, the surgical action, and the at least one bone, wherein the graphical 3D representation of the PSI is computed according to the automatically selected at least one surgical tool.

5. The computer implemented method of claim 4, further comprising generating instructions for assembling a kit comprising the PSI and the automatically selected at least one surgical tool.

6. The computer implemented method of claim 1, wherein the surgical plan represents a generic surgical plan for a specific surgical indication for a plurality of patients, wherein a customized PSI is generated for each patient of the plurality of patients by applying the same surgical plan to features extracted from the respective 3D representation of each patient.

7. The computer implemented method of claim 1, further computing a size and / or shape of the footprint region as at least one of: (1) an optimization between a tradeoff of a larger size and a smaller size, wherein stability of the PSI is increased with the larger size and decreased with the smaller size, and risk of infection of the PSI is increased with the larger size and decreased with the smaller size, and (2) based on anatomical landmarks and / or coverage areas that are advantageous to anchor to or which are to be avoided.

8. The computer implemented method of claim 1, wherein the surgical plan is accessed from a plurality of surgical plans each for a surgical indication selected from a plurality of surgical indication, each surgical plan defining at least: a surgical procedure, the at least one bone and / or cartilage to be operated on, the target region, approach for computing the pose, and the at least one surgical tool.

9. The computer implemented method of claim 1, further comprising: accessing a plurality of electronic health records (EHR) of a plurality of subjects, and a corresponding plurality of 3D representations of the plurality of subjects; for each subject: automatically extracting features from the EHR and / or the 3D representation of the subject; automatically selecting the surgical plan from a plurality of surgical plans according to the extracted features; andautomatically generating the graphical 3D representation of the PSI for the subject according to the automatically selected surgical plan; and providing a plurality of graphical 3D representations of a plurality of PSIs for the plurality of subjects.

10. The computer implemented method of claim 9, further comprising computing a score for each of the plurality of graphical 3D representations of the plurality of PSIs of the plurality of subjects, and ranking the plurality of graphical 3D representations according to the score and / or selecting a subset of graphical 3D representations having scores above a threshold.

11. The computer implemented method of claim 1, further comprising: defining a case for each image of a plurality of images of a plurality of subjects, the case including the image and related data; automatically computing a score for the case; automatically ranking a plurality of cases according to respective scores; automatically providing a subset of cases having scores above a threshold; for each case, sending a notification to client terminal of a user assigned to the case; and in response to an interaction with the notification, automatically triggering presentation of an online 3D viewer on a display of the client terminal for review of the automatically generated graphical 3D representation of the PSI and / or for adapting parameters for the automatic generation of the graphical 3D representation of the PSI.

12. The computer implemented method of claim 1, further comprising: scheduling 3D printing of the PSI by 3D printer according to at least one of: predicted amount of time for 3D printing of the PSI, scheduled operation time when the PSI is predicted to be used, current load scheduled for the 3D printer for 3D printing other PSIs for other subjects, and scheduled operation for the other subjects requiring the other PSIs.

13. The computer implemented method of claim 1, further comprising: accessing a plurality of parameters associated with a user of a plurality of users, wherein the plurality of parameters are automatically learned and / or extracted based on historically generated PSIs of the user and / or are manually set, wherein the automatically generating of the graphical 3D representation is performed by conforming to the plurality of parameters.

14. The computer implemented method of claim 1, further comprising: generating a 3D model based on the at least one bone and / or cartilage of the 3D representation of the subject; and simulating placement of the graphical 3D representation of the PSI on the 3D model.

15. The computer implemented method of claim 14, further comprising: generating an alert in response to intersection of the at least one bone and / or cartilage with the graphical 3D representation of the PSI, a deviation of the graphical 3D representation of the PSI from a predefined envelope by more than a defined threshold, and / or another detected error.

16. The computer implemented method of claim 14, further comprising: checking the simulation of the placement for detection of at least one error, automatically adapting at least one of: the at least one target region, the pose, the at least one surgical tool, and / or another parameter, re-generating the graphical 3D representation of the PSI according to the automatic adaptation, for generating an adapted graphical 3D representation, performing the simulating placement of the adapted graphical 3D representation, iterating the checking, the automatically adapting, the re-generating, and the performing, until no error is detected during the checking, and providing the adapted graphical 3D representation for which no error is detected.

17. The computer implemented method of claim 16, wherein the iterating the checking, the automatically adapting, the re-generating, and the performing, until no error is detected during the checking is performed in a background, without presenting on a display graphical 3D representations for which the at least one error is detected.

18. The computer implemented method of claim 14, further comprising: generating a 3D representation of the subject predicting a post-operative procedure state of the at least one bone and / or cartilage after surgical placement of the PSI, the 3D representation of the subject generated based on corresponding at least one bone and / or cartilage of subjects that did not undergo surgical procedures.

19. The computer implemented method of claim 14, further comprising:creating a surgical animation video simulating a surgical procedure following the surgical plan and / or features derived for computation of the surgical placement of the PSI on the at least one bone, based on the 3D model and the graphical 3D representation.

20. The computer implemented method of claim 1, further comprising: automatically generating an instruction for use (IFU) dataset for instructing a user for performing a surgical procedure following the surgical plan, using the 3D representation of the PSI, and the at least one surgical tool at the pose, wherein the IFU dataset is selected from: text instructions readable by a human, at least one image, and a surgical animation video.

21. The computer implemented method of claim 1, further comprising: automatically selecting according to the surgical plan, at least one measurement model of a plurality of measurement model each trained to measure a different at least one parameter of the at least one bone and / or cartilage; and automatically applying the at least one measurement model to the at least one bone and / or cartilage to obtain a plurality of different parameters, wherein at least one of the automatically detecting the target region, the automatically computing the pose, and the automatically generating, is based on the at least one measurement model.

22. The computer implemented method of claim 1, further comprising: determining a unique code representing the unique graphical 3D representation of the PSI for the subject; including the unique code on the graphical 3D representation of the PSI for the subject, wherein the unique code is generated for at least one of: being visually distinguishable on the PSI by a human, and being visually distinguishable on a CT, MRI, and / or x-ray of the subject after surgical implantation of the PSI; and updating a dataset mapping between the subject and the unique code.

23. The computer implemented method of claim 1, wherein the 3D representation of the subject is a first 3D image of a first imaging modality, and further comprising: accessing a second 3D image of a second imaging modality, the second 3D image depicting at least one tissue type not sufficiently depicted in the first 3D image; registering the first 3D image with the second 3D image;analyzing the at least one tissue type depicted in the second 3D image according to the surgical plan, wherein the graphical 3D representation of the PSI is automatically rendered according to the analysis.

24. The computer implemented method of claim 1, wherein the surgical plan further defines at least one of: at least one anatomical landmark, at least one measurement, and at least one positioning parameter, wherein the graphical 3D representation of the PSI is automatically rendered according to the surgical plan that defines at least one of: at least one anatomical landmark, at least one measurement, and at least one positioning parameter.

25. The computer implemented method of claim 1, wherein the graphical representation of the PSI further comprises at least one guiding component designed for guiding the at least one surgical tool for performing the surgical action according to the pose.

26. The computer implemented method of claim 1, further comprising interfacing with at least one external system for automatic generating graphical 3D representations of PSIs based on data obtained from the at least one external system.

27. The computer implemented method of claim 1, further comprising generating a combination image comprising the 3D image of the subject and an overlay of the graphical 3D representation of the PSI positioned over the at least one bone and / or cartilage according to the surgical plan.

28. The computer implemented method of claim 1, further comprising presenting at least one bone correction according to the surgical plan, receiving an adjustment from a user via an interactive interface, and in response to an approval, generating the graphical 3D representation of the PSI based on the adjustment.

29. The computer implemented method of claim 1, further comprising adding a stamped measuring shape to the 3D printed PSI, the stamped measuring shape in a fixed predefined size and / or shape for matching to at least one measurable feature of the stamped measuring shape stored in association with the graphical 3D representation of the PSI.

30. The computer implemented method of claim 1, further comprising: receiving a comment and / or question entered by at least one user; automatically identifying a location and / or feature of the graphical 3D representation of the PSI impacted by the comment and / or question; and automatically executing a task impacting the location and / or feature according to the comment and / or question.

31. The computer implemented method of claim 14, further comprising: accessing a template from a plurality of templates for automatically generating the pose and / or the surgical tool(s) and / or for defining a component of the PSI; presenting the template over the 3D model within a common space on a display; testing and / or simulating the template over the 3D model; and wherein the graphical 3D representation of the PSI is generated according to the template.

32. A personalized kit for a subject comprising: a PSI manufactured based on instructions created according to claim 1; and at least one surgical tool for use with the PSI selected according to at least one of: the pose, the surgical action, and the at least one bone, wherein the graphical 3D representation of the PSI is computed according to the selected at least one surgical tool.

33. A system for 3D printing a PSI for a subject comprising: at least one processor executing a code for implementing the computer implemented method of claim 1 for generating instructions for additive manufacturing of the PSI; and a 3D printer configured for additive manufacturing of the PSI while maintaining sterility of the PSI suitable for implantation in a body of the subject.

34. A computer implemented method for generating a personalized surgical instrument (PSI) for a subject, comprising: defining a case for each image of a plurality of images of a plurality of subjects, the case including the image and related data; automatically computing a score for the case; automatically ranking a plurality of cases according to respective scores; automatically providing a subset of cases having scores above a threshold;for each case of a plurality of cases: automatically extracting features from the case; automatically selecting a surgical plan from a plurality of surgical plans according to the extracted features, each surgical plan for a different surgical indication; automatically detecting at least one target region of a surface of at least one bone and / or cartilage on the 3D representation of the subject, defined by the surgical plan; automatically computing a pose of at least one surgical tool for performing the surgical action defined by the surgical plan; and automatically generating a graphical 3D representation of the PSI according to the surgical plan, the graphical 3D representation of the PSI including a footprint region having a surface corresponding to the at least one target region for contact with the surface of the at least one bone and / or cartilage corresponding to the at least one target region, and for performing the surgical action according to the pose.

35. The computer implemented method of claim 34, further comprising filtering the plurality of images for selection images meeting availability and / or suitability criteria for automatic generation of a graphical 3D representation of a PSI, wherein the score is computed for each of the filtered images.

36. The computer implemented method of claim 34, further comprising pulling DICOM metadata for each image, and including the DICOM in the case.

37. The computer implemented method of claim 34, further comprising: for each case, sending a notification to client terminal of a user assigned to the case; and in response to an interaction with the notification, automatically triggering presentation of an online 3D viewer on a display of the client terminal for review of the graphical 3D representation of the PSI and / or for adapting parameters for automatic generation of the graphical 3D representation of the PSI.

38. The computer implemented method of claim 34, wherein the score is computed according to at least one of: historical analysis of use of the automatically generated graphical 3D representations by the corresponding user, an analysis of quality of the case indicating likelihood of generating an accurate graphical 3D representation of the PSI for the case, and correlation withavailability and / or suitability criteria for automatic generation of a graphical 3D representation of the PSI.

39. The computer implemented method of claim 34, further comprising: generating a unique code to each image of the plurality of images; instructing a central dataset storing the plurality of images to associated respective unique codes to respective images; and for each user associated with case, providing the unique code associated with the case to a client terminal, wherein the unique code is designed for remotely accessing the case associated with the unique code from the client terminal, wherein the unique code is sharable with other client terminals of other users for joint access to the case.

40. The computer implemented method of claim 34, wherein the score is computed according to at least one of: intersection of the at least one bone and / or cartilage with the graphical 3D representation of the PSI, a deviation of the graphical 3D representation of the PSI from a predefined envelope by more than a defined threshold.

41. The computer implemented method of claim 34, further comprising for each 3D representation associated with a subject: accessing a plurality of parameters associated with a surgeon assigned for operating on the subject; wherein the plurality of parameters are automatically learned and / or extracted based on historically generated PSIs of the user and / or are manually set, wherein the automatic generating of the graphical 3D representation is performed by conforming to the plurality of parameters, and / or wherein the score is computed according to the plurality of parameters.

42. The computer implemented method of claim 34, further comprising for each 3D representation associated with a subject: extracting features from the electronic health record of the subject, and wherein the surgical plan is automatically selected according to the features extracted from the electronic health record.

43. The computer implemented method of claim 34, further comprising: scheduling 3D printing of a subset of a plurality of graphical 3D representations of the plurality of cases by an additive manufacturing device according to at least one of: predicted amount of time for 3D printing of each PSI, and scheduled operation time and / or date when each PSI is predicted to be used.

Images