Spinal stress map creation using finite element analysis

Abstract

A system and technique for creating a spinal stress map is provided. The system may be configured to generate a multi-class segmentation of an anatomical element of a patient based on a plurality of magnetic resonance images of the anatomical element from a plurality of patients. Additionally, one or more stress maps may be generated based on simulating stress on the anatomical element. In some embodiments, the simulated stress may be simulated using finite element analysis based at least in part on the multi-class segmentation. Additionally, the system may be configured to display one or more stress maps via a user interface, where the one or more stress maps are determined based on one or more deep learning models configured to predict a multi-mark mask and / or a stress map of the anatomical element.

Description

Background Art

[0001] The present disclosure generally relates to spinal stress maps, and more particularly, to creating spinal stress maps using finite element (FE) analysis.

[0002] When a patient's spinal cord is stressed and that stress often causes pain to the patient, spinal stenosis disorders are caused. Additionally, spinal stenosis is a common cause of back surgery in patients. For example, a patient may undergo a laminectomy, which involves a surgical procedure to create space by removing bone spurs and spinal tissue. A laminectomy typically involves removing a small piece of the dorsal portion (e.g., the lamina) of a small bone (e.g., a vertebra) of the spine, which can widen the spinal canal to relieve pressure and stress on the spinal cord or nerves. Additionally or alternatively, a patient may undergo a laminotomy, which involves a less invasive surgical procedure in which a smaller incision is made to remove a smaller piece of the dorsal portion of a small bone of the spine compared to what is removed in a laminectomy. In some cases, back surgery may benefit from using spinal stress maps to identify specific regions of the spine that experience a higher amount of stress for more targeted and effective surgery. Summary of the Invention

[0003] Example aspects of the present disclosure include:

[0004] A system for creating a spinal stress map, the system comprising: a processor; and a memory that stores data for processing by the processor, the data when processed causes the processor to: generate a multi-class segmentation of the anatomical elements of the patient at least in part based on a plurality of magnetic resonance images of the anatomical elements from a plurality of patients; generate a plurality of stress maps at least in part based on simulating stress on the anatomical elements, the simulated stress being simulated using finite element analysis at least in part based on the multi-class segmentation; determine one or more of the plurality of stress maps to display at least in part based on one or more deep learning models, the one or more deep learning models being configured to predict a multi-label mask and / or stress map of the anatomical elements; and display the one or more stress maps via a user interface.

[0005] Any aspect herein, wherein the memory stores additional data for processing by the processor, the additional data when processed causes the processor to: train a deep learning model at least in part based on the plurality of magnetic resonance images of the anatomical elements from the plurality of patients; and generate the multi-class segmentation at least in part based on the deep learning model.

[0006] Any aspect herein, wherein the deep learning model is further trained at least in part based on a plurality of annotated soft tissue segmentation maps of the anatomical elements from the plurality of patients.

[0007] Any aspect of the present disclosure, wherein the plurality of magnetic resonance images includes a plurality of three-dimensional magnetic resonance images.

[0008] Any aspect of the present disclosure, wherein the memory stores additional data for processing by the processor, the additional data causing the processor, when processed, to: simulate a plurality of stresses on the anatomical element at least in part based on simulating a plurality of physiological motions and deformations that induce stress on the anatomical element.

[0009] Any aspect of the present disclosure, wherein the memory stores additional data for processing by the processor, the additional data causing the processor, when processed, to: generate individual stress maps for each of the plurality of simulated stresses, wherein the plurality of stress maps includes the individual stress maps.

[0010] Any aspect of the present disclosure, wherein the memory stores additional data for processing by the processor, the additional data causing the processor, when processed, to: train a deep learning model at least in part based on the plurality of stress maps of the anatomical element and the plurality of segmentations; and generate the one or more stress maps for display via the user interface at least in part based on the deep learning model.

[0011] Any aspect of the present disclosure, wherein the memory stores additional data for processing by the processor, the additional data causing the processor, when processed, to: generate a plurality of simulated stress relief maps at least in part based on the plurality of stress maps and simulating the removal of one or more portions of the anatomical element, wherein the one or more portions of the anatomical element are simulated for removal at least in part based on additional finite element analysis; and display via the user interface a proposed surgical plan generated at least in part based on the plurality of simulated stress relief maps.

[0012] Any aspect of the present disclosure, wherein the memory stores additional data for processing by the processor, the additional data causing the processor, when processed, to: train a deep learning model at least in part based on the plurality of simulated stress relief maps, wherein the proposed surgical plan is generated at least in part based on the deep learning model.

[0013] Any aspect of the present disclosure, wherein the plurality of stress maps includes three-dimensional stress maps of the anatomical element.

[0014] A system for creating spinal stress maps, the system comprising: a processor; and a memory that stores data for processing by the processor, the data when processed causing the processor to: generate a multi-class segmentation of an anatomical element of a patient at least in part based on a plurality of magnetic resonance images of anatomical elements from a plurality of patients; generate a plurality of stress maps of the anatomical element of the patient at least in part based on simulating stress on the multi-class segmentation of the anatomical element, the simulated stress being simulated using finite element analysis at least in part based on the multi-class segmentation; determine one or more of the plurality of stress maps for display at least in part based on a deep learning model, the deep learning model being configured to predict a multi-label mask and / or stress map of the anatomical element; and display one or more of the plurality of stress maps via a user interface.

[0015] Any aspect of this document, wherein the memory stores additional data for processing by the processor, the additional data when processed causing the processor to: train a first deep learning model at least in part based on the plurality of magnetic resonance images of the anatomical elements from the plurality of patients; and generate the multi-class segmentation at least in part based on the first deep learning model.

[0016] Any aspect of this document, wherein the memory stores additional data for processing by the processor, the additional data when processed causing the processor to: train a second deep learning model at least in part based on the plurality of stress maps and the multi-class segmentation of the anatomical element; and generate the one or more of the plurality of stress maps for display via the user interface at least in part based on the deep learning model.

[0017] Any aspect of this document, wherein the first deep learning model is further trained at least in part based on a plurality of annotated soft tissue segmentation maps of the anatomical element from the plurality of patients.

[0018] Any aspect of this document, wherein the memory stores additional data for processing by the processor, the additional data when processed causing the processor to: simulate a plurality of stresses on the anatomical element at least in part based on simulating a plurality of physiological motions and deformations that cause stress on the anatomical element.

[0019] Any aspect of this document, wherein the memory stores additional data for processing by the processor, the additional data when processed causing the processor to: generate a separate stress map for each of the plurality of simulated stresses, wherein the plurality of stress maps includes the separate stress map.

[0020] Any aspect of the present disclosure, in which the memory stores additional data for processing by the processor, and the additional data, when processed, causes the processor to: generate a plurality of simulated stress relief maps at least in part based on the plurality of stress maps and the simulation of the removal of one or more portions of the anatomical element, wherein the one or more portions of the anatomical element are simulated for removal at least in part based on additional finite element analysis; and display, via the user interface, a proposed surgical plan generated at least in part based on the plurality of simulated stress relief maps.

[0021] Any aspect of the present disclosure, in which the memory stores additional data for processing by the processor, and the additional data, when processed, causes the processor to: train a deep learning model at least in part based on the plurality of simulated stress relief maps, wherein the proposed surgical plan is generated at least in part based on the deep learning model.

[0022] A system for creating spinal stress maps, the system comprising: a processor; and a memory that stores data for processing by the processor, and the data, when processed, causes the processor to: generate a plurality of stress maps and multi-class segmentations of the spinal cord at least in part based on simulating stress on the spinal cord of a patient, the simulated stress being simulated using finite element analysis that is at least in part based on the multi-class segmentations; determine, at least in part based on one or more deep learning models, one or more of the plurality of stress maps to display, the one or more deep learning models being configured to predict multi-label masks and / or stress maps of the spinal cord; and display, via a user interface, one or more of the plurality of stress maps.

[0023] Any aspect of the present disclosure, in which the simulated stress includes moving the vertebrae of the spinal cord, compressing the discs of the spinal cord, sizing the ligamentum flavum of the spinal cord, deforming the spinal cord, additional physiological movements of the spinal cord, or a combination thereof.

[0024] Any one aspect in combination with any one or more other aspects.

[0025] Any one or more of the features disclosed herein.

[0026] Any one or more of the features generally disclosed herein.

[0027] Any one or more of the features generally disclosed herein in combination with any one or more other features generally disclosed herein.

[0028] Any one of the aspects / features / embodiments in combination with any one or more other aspects / features / embodiments.

[0029] Use any one or more of the aspects or features disclosed herein.

[0030] It should be understood that any feature described herein can be claimed in combination with any other feature described herein, regardless of whether the features are from the same described embodiment.

[0031] Details of one or more aspects of the present disclosure are set forth in the following drawings and the description. Other features, objects, and advantages of the technology described in this disclosure will be apparent from the description, the drawings, and the claims.

[0032] The phrases "at least one," "one or more," and "and / or" are open-ended expressions that are both conjunctive and disjunctive in operation. For example, each of the expressions "at least one of A, B, and C," "at least one of A, B, or C," "one or more of A, B, and C," "one or more of A, B, or C," and "A, B, and / or C" means only A, only B, only C, A and B together, A and C together, B and C together, or A, B, and C together. When each of A, B, and C in the above expressions refers to elements such as X, Y, and Z or element categories such as X1-Xn, Y1-Ym, and Z1-Zo, the phrase is intended to refer to a single element selected from X, Y, and Z, a combination of elements selected from the same category of elements (e.g., X1 and X2), and a combination of elements selected from two or more categories of elements (e.g., Y1 and Zo).

[0033] The term "a" entity means one or more of that entity. Thus, the terms "a," "one or more," and "at least one" may be used interchangeably herein. It should also be noted that the terms "comprising," "including," and "having" may be used interchangeably.

[0034] The foregoing is a simplified summary of the present disclosure to provide an understanding of some aspects of the present disclosure. This Summary of the Invention is neither an extensive overview nor an exhaustive overview of the present disclosure and its various aspects, embodiments, and configurations. It is neither intended to identify the key or important elements of the present disclosure nor to delineate the scope of the present disclosure, but rather to present selected concepts of the present disclosure in a simplified form as an introduction to the more detailed description presented below. As should be understood, other aspects, embodiments, and configurations of the present disclosure may make use, individually or in combination, of one or more of the features set forth above or described in detail below.

[0035] Many additional features and advantages of the present disclosure will become apparent to those skilled in the art upon consideration of the following description of the embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The accompanying drawings are incorporated in and form a part of this specification to illustrate several examples of the present disclosure. These drawings, together with the description, explain the principles of the present disclosure. The drawings merely illustrate how to implement and use the preferred and alternative examples of the present disclosure, and these examples should not be construed as limiting the present disclosure to only the illustrated and described examples. Additional features and advantages will become apparent from the following more detailed description of the various aspects, embodiments, and configurations of the present disclosure, as illustrated by the accompanying drawings referenced below.

[0037] Figure 1 is a block diagram of a system according to at least one embodiment of the present disclosure;

[0038] Figure 2 is a diagram of a training process according to at least one embodiment of the present disclosure;

[0039] Figure 3 is a diagram of a system according to at least one embodiment of the present disclosure;

[0040] Figure 4 is a flowchart of a method according to at least one embodiment of the present disclosure;

[0041] Figure 5 is a flowchart of a method according to at least one embodiment of the present disclosure; and

[0042] Figure 6 is a flowchart of a method according to at least one embodiment of the present disclosure. Detailed Description

[0043] It should be understood that the various aspects disclosed herein can be combined in combinations different from those specifically presented in the specification and the accompanying drawings. It should also be understood that, depending on the example or embodiment, certain actions or events of any of the processes or methods described herein can be performed in a different order and / or can be added, combined, or completely omitted (e.g., depending on different embodiments of the present disclosure, not all of the described actions or events may be required to implement the disclosed technology). Additionally, although certain aspects of the present disclosure are described for clarity as being performed by a single module or unit, it should be understood that the technology of the present disclosure can be performed by a combination of units or modules associated with, for example, a computing device and / or a medical device.

[0044] In one or more examples, the described methods, processes, and techniques may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored as one or more instructions or code on a computer-readable medium and executed by a hardware-based processing unit. Alternatively or additionally, the functions may be implemented using a machine learning model, a neural network, an artificial neural network, or a combination thereof (either alone or in combination with instructions). The computer-readable medium may include a non-transitory computer-readable medium, which corresponds to a tangible medium, such as a data storage medium (e.g., RAM, ROM, EEPROM, flash memory, or any other medium that can be used to store the desired program code in the form of instructions or data structures and can be accessed by a computer).

[0045] The instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors (e.g., Intel Core i3, i5, i7, or i9 processors; Intel Celeron processors; Intel Xeon processors; Intel Pentium processors; AMD Ryzen processors; AMD Athlon processors; AMD Phenom processors; Apple A10 or 10X Fusion processors; Apple A11, A12, A12X, A12Z, or A13 Bionic processors; or any other general-purpose microprocessor), graphics processing units (e.g., Nvidia GeForce RTX 2000 series processors, Nvidia GeForce RTX 3000 series processors, AMD Radeon RX 5000 series processors, AMD Radeon RX 6000 series processors, or any other graphics processing unit), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Thus, as used herein, the term "processor" may refer to any one of the foregoing structures or any other physical structure suitable for implementing the described techniques. Additionally, these techniques may be fully implemented in one or more circuits or logic elements.

[0046] Before explaining any embodiments of the present disclosure in detail, it is to be understood that the present disclosure is not limited in its application to the construction details and component arrangements set forth in the following description or illustrated in the drawings. The present disclosure is capable of other embodiments and of being practiced or carried out in various ways. Additionally, it is to be understood that the terminology and phrases used herein are for the purpose of description and should not be regarded as limiting. As used herein, the terms "comprising," "including," or "having," and variations thereof, are intended to cover the items listed thereafter and their equivalents, as well as additional items. Further, the present disclosure may use examples to illustrate one or more of its aspects. Unless otherwise expressly stated, the use or listing of one or more examples (which may be indicated by "for example," "by way of example," "such as," or similar language) is not intended and does not limit the scope of the present disclosure.

[0047] The terms proximal and distal are used in their conventional medical meanings in the present disclosure, with proximal being closer to the operator or user of the system and farther from the surgical area of concern within or on the patient's body, while distal is closer to the surgical area of concern within or on the patient's body and farther from the operator or user of the system.

[0048] When the patient's spinal cord is stressed and this stress often causes pain to the patient, a spinal stenosis condition is caused. Additionally, spinal stenosis is a common cause of back surgery in patients. As described herein, the spine can include multiple vertebrae (e.g., typically 33 vertebrae), multiple intervertebral discs (e.g., typically 23 intervertebral discs, which are pads located between vertebrae), the spinal cord, and connecting ribs. For example, a patient may undergo a laminectomy, which includes a surgical procedure to create space by removing bone spurs and spinal tissue. A laminectomy typically involves removing a small piece of the dorsal portion (e.g., the lamina) of a small bone (e.g., a vertebra) of the spine, which can expand the spinal canal to relieve pressure and stress on the spinal cord or nerves. Additionally or alternatively, a patient may undergo a laminotomy, which includes a less invasive surgical procedure in which a smaller incision is made to remove a smaller piece of the dorsal portion of a small bone of the spine compared to what is removed in a laminectomy.

[0049] To perform minimally invasive surgery, a surgeon may identify the stressed area of the patient. Thus, for performing spinal surgery (e.g., for stress relief) and identifying the stressed area, a surgeon would benefit if a magnetic resonance (MR) image (e.g., of the patient's spine) includes a three-dimensional (3D) stress map of the spine. The stress map can assist the surgeon in performing minimal and accurate bone cutting by identifying the bone portions to be removed that create stress on the nerves (e.g., "pushing on the nerves"). That is, back surgery can benefit from using a spinal stress map that identifies specific areas of the spine that experience a higher amount of stress for more targeted and effective surgery.

[0050] As described herein, techniques are provided for creating spinal stress maps based on a combination of finite element (FE) analysis and deep learning models. For example, a spinal stress map can be created based on one or more deep learning models and FE analysis. A first deep learning model is trained based on a plurality of 3D MR images of the spines of previous patients (e.g., an MR imaging (MRI) spine database) and is configured to create a 3D multi-class segmentation of the spine of a current patient. The 3D multi-class segmentation is then used as an input to an FE analysis that simulates various stresses on the patient's spine (e.g., physiological movements, deformations, and / or material changes such as disc degeneration that may cause stenosis) and creates a stress map for each simulation. The simulation end results can be generated based on the first deep learning model trained using the plurality of 3D MR images of the spine such that the spinal stress map is predicted from real images (e.g., 3D MR images of the spine) rather than simulated images. Based on the FE analysis, each simulation end result can be saved as a segmentation map and a corresponding stress map.

[0051] Then, a second deep learning model can take the simulation end results (e.g., the segmentation map and the stress map) as inputs to generate a cumulative stress map for the patient, which is displayed (e.g., via a user interface) for a surgeon to identify the stressed regions of the patient's spine for performing surgery. In some embodiments, the first deep learning model can predict one or more multi-label masks from the MR images, and the second deep learning model can predict a stress map from the multi-label masks. Additionally, the second deep learning model can also generate recommendations or suggestions for stress-relieving bone cuts to be made by the surgeon (e.g., a bone cut recommendation map), which can also be displayed for the surgeon to view.

[0052] The spinal stress map can help the surgeon identify the minimum bone regions to cut to create stress relief. Additionally, the described techniques can reduce the amount of bone removal and alleviate future pain for the patient. In some embodiments, the spinal stress map can help avoid surgeries such as laminectomy and favor laminotomy. The laminotomy can be fine-tuned with the spinal stress map such that only the bone adjacent to the stressed nerve will be removed. Additionally, the bone cut recommendations can set the optimal minimum bone cuts and stress relief for the surgeon.

[0053] Figure 1FIG. 0 is a block diagram of a system 100 according to at least one embodiment of the present disclosure. The system 100 may include one or more inputs 102 that are used by a processor 104 to generate one or more outputs 106. The processor 104 may be a computing device or part of a different device. Additionally, the processor 104 may be any processor described herein or any similar processor. The processor 104 may be configured to execute instructions or data stored in a memory that may cause the processor 104 to perform one or more computational steps using or based on the input 102 to generate the output 106.

[0054] As described herein, the input 102 may include one or more MR images 108. For example, the MR images 108 may include a plurality of 3D MR images of the spines of previous patients (e.g., an MRI spine database) who may or may not have undergone spinal surgery to relieve stress on their spines. In some examples, the MR images 108 may include MR images of spines that are considered “good” or healthy (e.g., the spines of patients without stenosis and / or without experiencing other types of back pain) and injured or unhealthy spines (e.g., the spines of patients who do have stenosis and / or who experience other types of back pain). For example, a “good” spine may include a spine that is considered healthy (e.g., not in pain or injury), and / or the spine of a patient who is in pain or injury but the pain is in a part of the spine (e.g., vertebrae) that is not in pain for a given patient (e.g., not the vertebrae or region that is in pain for a given patient).

[0055] In some embodiments, the processor 104 may take the input 102 (such as the MR image 108 of a “good” spine) to find the differences (e.g., increments) between the “good” spine and the degenerative simulation of the spine of a given patient, and may run a stress simulation on the spine of the given patient to identify the differences. For example, the processor 104 may perform a FE analysis 110 to mimic a “degenerative spine” representing the given patient based on simulating different deformations and / or degenerative conditions, and then simulate different stresses on the “degenerative spine” to identify the differences. As part of the FE analysis 110, parameters of each element of the spine of the patient may be input (e.g., based on the differences between the spine of the given patient and the “good” spine). For example, the parameters may include a plurality of parameters of each element of the spine (e.g., discs, vertebrae, canals, ligaments, etc.), such as elasticity, rigidity, mimicking different amounts of hydrogen, size, thickness, and / or other parameters characterizing each element of the spine. In some examples, the parameters of each element may be determined or adjusted based on a previously obtained scan of the spine of the given patient (e.g., a computed tomography (CT) scan).

[0056] After inputting different parameters of each element of a patient's spine for FE analysis 110, FE analysis 110 may then include simulating different stresses on a "degenerative spine" or a model representing the patient's spine. For example, the simulation may include physiological movements and deformations that may potentially cause stenosis to occur on the patient's spine, such as but not limited to moving vertebrae, compressing discs, resizing the ligamentum flavum (e.g., the ventral portion of the lamina connecting adjacent vertebrae or the ligament of the nerve), etc.

[0057] Based on the simulated stresses (e.g., partially based on one or more deep learning models as described with reference to Figure 2 and generated together with FE analysis 110), FE analysis 110 may generate one or more stress maps 112 as an output 106 of system 100. For example, a segmentation map and a corresponding stress map may be generated for each simulation end result (e.g., each simulated stress) during the training process. In some embodiments, stress maps 112 may be compiled at inference time to create or predict a single stress map of the patient's spine to determine which regions of the patient's spine may experience the highest amount of stress and are targets for surgery. The stress maps 112 and / or the single stress map may be displayed (e.g., via a user interface) for a surgeon to view and accordingly plan surgery to eliminate the predicted stress.

[0058] Additionally or alternatively, FE analysis 110 may be used to generate recommendations (e.g., bone cutting suggestions or recommendations) for stress-relieving bone cuts to be performed by a surgeon, which may also be displayed (e.g., via a user interface). For example, after different stresses have been simulated, as part of FE analysis 110, different surgical simulations may also be performed to identify which surgery is most likely to eliminate the simulated stress. That is, different surgical simulations may include simulations of removing different parts of the spine (e.g., as part of a laminectomy), and bone cutting suggestions or recommendations may be generated based on which surgical simulation in the different surgical simulations achieves elimination of the simulated stress on the patient's spine.

[0059] As described herein, in addition to FE analysis 110, processor 104 may also employ one or more deep learning models (e.g., artificial intelligence (AI) models, neural networks, etc.) to generate stress maps 112 as an output 106 of system 100 based on MR images 108 and / or other inputs 102 of system 100. Reference Figure 2 describes the deep learning model in more detail.

[0060] Figure 2 is a diagram of a training process 200 according to at least one embodiment of the present disclosure. In some examples, the training process 200 may implement Figure 1Aspects of or implemented by the various aspects. For example, the training process 200 may be a more detailed view of the system 100, where one or more computational steps of the processor 104 (e.g., including FE analysis 110) are implemented using, in part, multiple MR images 108 as input 102 to generate an output 106, including a stress map 112.

[0061] As previously referenced Figure 1 As described, multiple MR images 108 (e.g., 3D MR images) can be used as input 102 to the training process 200, where the multiple MR images 108 include MR images of the spine (e.g., an MRI spine database of a patient's spine). For example, the MR images 108 may include MR images of "good" spines (e.g., the spines of patients who are not experiencing pain or are not experiencing pain in the areas of their spine where the current patient is experiencing pain) and "bad" spines (e.g., the spines of patients with stenosis and / or experiencing other types of back pain). Additionally, the input 102 may include a multi-label mask 204 of the spine (e.g., a 3D MR multi-label mask). For example, the multi-label mask 204 may include segmented labeled or annotated images (e.g., masks) that together cover the entire spine, and these labeled or annotated images indicate different elements of the spine (e.g., discs, vertebrae, canals, ligaments, etc.).

[0062] The MR images 108 and the multi-label mask 204 can be used to train a first deep learning model 206 (e.g., model 1), which takes a 3D MRI (e.g., the MR image 108) to generate a multi-class segmentation 208 (e.g., a 3D multi-class segmentation) of the spine of a given patient. For example, the first deep learning model 206 can be trained, at least in part, based on inputs from an MRI spine database (e.g., the MR image 108) and an annotated soft tissue segmentation map (e.g., the multi-label mask 204) that includes all soft and bony elements of the patient's spine, to create a 3D multi-class segmentation (e.g., a 3D multi-segmentation mask, a 3D multi-label mask, etc.) as an inference or output. That is, the first deep learning model 206 can take one or more MR images and classify and mask the sub-anatomical elements of the patient's spine to create a mesh for each element (such as canals, vertebrae, discs, etc.). Additionally, based on training using the MR images 108 that include MR images of both "good" and "bad" spines, the first deep learning model 206 can be configured to segment any type of spine. In some examples, the elements can have different classes. Additionally, the first deep learning model 206 can output a multi-label mask 210.

[0063] Subsequently, the FE analysis 110 can create a simulated stenosis of the spine of a given patient from the multi-label mask 210 (e.g., the 3D mask output of the first deep learning model 206), thereby generating a stress map. For example, the FE analysis 110 can run multiple simulations 214, which include physiological movements and deformations that may cause stenosis in a given patient. In some examples, in the FE analysis 110, "good" candidates (e.g., MR images of patients with "good" spines) can be used to simulate the deformation and / or degenerative simulation of a given patient. The simulation 214 can include, but is not limited to, moving one or more vertebrae, squeezing the discs, resizing the ligamentum flavum (e.g., a thicker ligamentum flavum may push against the canal to cause stenosis), classifying degenerative elements, etc. In some embodiments, the FE analysis 110 can create a stress map (e.g., von Mises stress map) for each simulation, and can save the end result of each simulation as a segmentation map and the corresponding stress map. For example, the FE analysis 110 can create one or more multi-label masks 216 after each simulation, and create the corresponding stress map 218 after each simulation (e.g., the stress map 218 can include a regression model that predicts continuous values). In some embodiments, the FE analysis 110 can also save recommendations for stress-relieving bone cuts (e.g., bone cut recommendations 220) based on the simulation.

[0064] That is, the results of the first deep learning model 206 are input into the FE analysis 110, which takes the labeled elements (e.g., the mask) and performs different simulations 214 to create and mimic a degenerative back. Each element of the spine (e.g., discs, vertebrae, spinal canal, etc.) is assigned specific parameters, such as elasticity, rigidity, or additional parameters that characterize the element. For example, the FE analysis 110 can take a disc and change and characterize the soft tissue of the disc according to the parameters of the material itself (e.g., based on the parameters of the discs of the spine of a given patient, such as obtained from the patient's CT scan), such as simulating the disc as being harder based on having less hydrogen or less fluid. Thus, the FE analysis 110 can change the parameters of the disc (e.g., or other elements of the spine) as a simulation, and determine what happens to the surrounding elements of the spine (e.g., the vertebrae above and below the disc) and the stress generated as a result of such changes. Additionally or alternatively, the FE analysis can simulate a fracture in one or more of the elements of the spine (e.g., such as a vertebral fracture), and determine what stress is generated based on the fracture or break. Thus, the FE analysis 110 can create the stress map 218 based on different simulations 214. In some embodiments, the simulation 214 can be performed based on the difference or increment between the patient's current or initial condition and the simulated deformation of the patient's spine, which is partially based on the "good" spine included in the MR image 108.

[0065] Processor 104 may then employ a second deep learning model 222 (e.g., Model 2), which takes a 3D multi-class segmentation (e.g., at the end of simulation 214) to generate one or more stress maps 224 (e.g., 3D stress maps) and an optional bone cutting recommendation map (e.g., bone cutting recommendation 228). For example, the output of FE analysis 110 (such as the end-of-simulation multi-label segmentation mask 216 and stress map 218) may be used to train the second deep learning model 222 to predict the stress map 224 and optional bone cutting recommendation 228 (e.g., 3D bone cutting recommendation) from the multi-segmented 3D image. In some embodiments, the inference of the second deep learning model 222 may include predicting the stress map 224 and / or stress map 112 and optional bone cutting recommendation 228 from the multi-label mask 216 (e.g., 3D multi-class segmentation).

[0066] That is, the multi-label mask 216 and stress map 218 generated from FE analysis 110 are used to teach or train the second deep learning model 222, and the second deep learning model 222 may be configured to generate an inference including the stress map 224. For example, the second deep learning model 222 may generate a stress map based on the difference between a degenerative back and a "good" back. Thus, the spine of any given patient may be input into processor 104 (e.g., employing the first deep learning model 206, FE analysis 110, and second deep learning model 222) to identify the differences (e.g., increments) from a "good" back in the spine of the given patient and generate a stress map of the spine of the given patient.

[0067] Turning to Figure 3 , a block diagram of a system 300 in accordance with at least one embodiment of the present disclosure is shown. The system 300 may be used to generate a spinal stress map based at least in part on FE analysis, as referenced Figure 1 and Figure 2 described. The system 300 includes a computing device 302, one or more imaging devices 312, a robot 314, a navigation system 318, a database 330, and / or a cloud or other network 334. Systems in accordance with other embodiments of the present disclosure may include more or fewer components than the system 300. For example, the system 300 may not include one or more components of the imaging device 312, the robot 314, the navigation system 318, the computing device 302, the database 330, and / or the cloud 334.

[0068] The computing device 302 includes a processor 304, a memory 306, a communication interface 308, and a user interface 310. Computing devices in accordance with other embodiments of the present disclosure may include more or fewer components than the computing device 302.

[0069] The processor 304 of the computing device 302 can be any processor described herein or any similar processor. For example, the processor 304 can be represented by the processor 104 as described with reference to Figure 1 above. The processor 304 can be configured to execute instructions or data stored in the memory 306, which can cause the processor 304 to perform one or more computational steps using or based on data received from the imaging device 312, the robot 314, the navigation system 318, the database 330, and / or the cloud 334.

[0070] The memory 306 can be or include RAM, DRAM, SDRAM, other solid-state memory, any memory described herein, or any other tangible non-transitory memory for storing computer-readable data and / or instructions. The memory 306 can store information or data for completing any steps of, for example, the methods 400, 500, and / or 600 described herein or any other method. The memory 306 can store instructions and / or machine learning models that support one or more functions of the robot 314, for example. For example, the memory 306 can store content (e.g., instructions and / or machine learning models) that, when executed by the processor 304, implements stress simulation 320, stress map generation 322, deep learning model training 324, and / or stress map display 328.

[0071] The stress simulation 320 enables the processor 304 to simulate stress on anatomical elements of a patient (e.g., different elements of the spine). For example, FE analysis can be used to simulate the simulated stress. In some embodiments, the stress simulation 320 enables the processor 304 to simulate multiple stresses on anatomical elements at least in part based on multiple physiological motions and deformations that cause stress on the anatomical elements. For example, the simulated stresses can include moving vertebrae of the spinal cord, compressing discs of the spinal cord, sizing the ligamentum flavum of the spinal cord, deforming the spinal cord, additional physiological motions of the spinal cord, or combinations thereof.

[0072] The stress map generation 322 enables the processor 304 to generate multiple stress maps and / or multi-class segmentations of the anatomical elements of the patient at least in part based on the simulated stress on the anatomical elements. For example, the stress map generation 322 enables the processor 304 to generate a separate stress map for each of the multiple simulated stresses, where the multiple stress maps include the separate stress maps. In some embodiments, the multiple stress maps can include 3D stress maps of the anatomical elements.

[0073] Deep learning model training 324 enables the processor 304 to train a first deep learning model at least in part based on multiple MR images of anatomical elements from multiple patients (e.g., obtained from the imaging device 312 and / or the database 330) (e.g., an MRI spine database), where the multi-class segmentation is generated at least in part based on the first deep learning model. In some embodiments, the first deep learning model can be further trained at least in part based on multiple annotated soft tissue segmentation maps of anatomical elements from multiple patients. Additionally, the multiple MR images can include multiple 3D MR images.

[0074] Additionally or alternatively, deep learning model training 324 enables the processor 304 to train a second deep learning model at least in part based on multiple stress maps of anatomical elements and the multi-class segmentation (e.g., the output of a FE analysis simulating these stresses). Subsequently, deep learning model training 324 enables the processor 304 to generate one or more of the multiple stress maps for display (e.g., via the user interface 310) at least in part based on the second deep learning model.

[0075] In some embodiments, deep learning model training 324 can optionally enable the processor 304 to generate multiple simulated stress relief maps at least in part based on the multiple stress maps and the simulation of the removal of one or more parts of the anatomical element, where the removal of one or more parts of the anatomical element is simulated based at least in part on additional FE analysis. Additionally, deep learning model training 324 can enable the processor 304 to train the second deep learning model at least in part based on the multiple simulated stress relief maps and to generate a recommended surgical plan at least in part based on the second deep learning model.

[0076] Stress map display 328 enables the processor 304 to display one or more of the multiple stress maps (e.g., via the user interface 310). Additionally, stress map display 328 can optionally enable the processor 304 to display a recommended surgical plan (e.g., via the user interface 310), where the recommended surgical plan is generated at least in part based on the multiple simulated stress relief maps.

[0077] In some embodiments, if provided as instructions, the content stored in the memory 306 may be organized into one or more application software, modules, packages, layers, or engines. Alternatively or additionally, the memory 306 may store other types of content or data (e.g., machine learning patterns, artificial neural networks, deep neural networks, etc.) that can be processed by the processor 304 to implement the various methods and features described herein. Thus, although the various contents of the memory 306 may be described as instructions, it should be understood that the functions described herein may be implemented by using instructions, algorithms, and / or machine learning models. The data, algorithms, and / or instructions may cause the processor 304 to manipulate the data stored in the memory 306 and / or data received from or via the imaging device 312, the robot 314, the database 330, and / or the cloud 334.

[0078] The computing device 302 may also include a communication interface 308. The communication interface 308 may be used to receive image data or other information from external sources (such as the imaging device 312, the robot 314, the navigation system 318, the database 330, the cloud 334, and / or any other system or component that is not part of the system 300), and / or to send instructions, images, or other information to external systems or devices (e.g., another computing device 302, the imaging device 312, the robot 314, the navigation system 318, the database 330, the cloud 334, and / or any other system or component that is not part of the system 300). The communication interface 308 may include one or more wired interfaces (e.g., USB ports, Ethernet ports, FireWire ports) and / or one or more wireless transceivers or interfaces (configured to send and / or receive information via one or more wireless communication protocols such as 802.11a / b / g / n, Bluetooth, NFC, ZigBee, etc.). In some embodiments, the communication interface 308 may be used to enable the device 302 to communicate with one or more other processors 304 or computing devices 302, either to reduce the time required to complete computationally intensive tasks or for any other reason.

[0079] The computing device 302 may also include one or more user interfaces 310. The user interface 310 may be or include a keyboard, a mouse, a trackball, a monitor, a television, a screen, a touch screen, and / or any other device for receiving information from a user and / or for providing information to a user. The user interface 310 may be used, for example, to receive user selections or other user input regarding any step of any method described herein. Nevertheless, any required input for any step of any method described herein may be automatically generated by the system 300 (e.g., by the processor 304 or by another component of the system 300) or received by the system 300 from a source external to the system 300. In some embodiments, the user interface 310 may be used to allow a surgeon or other user to modify the instructions to be executed by the processor 304 in accordance with one or more embodiments of the present disclosure, and / or to modify or adjust the settings of other information displayed on or corresponding to the user interface 310.

[0080] Although the user interface 310 is shown as part of the computing device 302, in some embodiments, the computing device 302 may utilize a user interface 310 that is separately housed from one or more of the remaining components of the computing device 302. In some embodiments, the user interface 310 may be located near one or more of the other components of the computing device 302, while in other embodiments, the user interface 310 may be located remote from one or more of the other components of the computing device 302.

[0081] The imaging device 312 can be used to image anatomical features (e.g., bone, vein, tissue, etc.) and / or other aspects of the patient's anatomy to generate image data (e.g., image data depicting or corresponding to bone, vein, tissue, etc.). As used herein, "image data" refers to data generated or captured by the imaging device 312, including data in machine-readable form, in graphical / visual form, and in any other form. In different examples, the image data can include data corresponding to an anatomical feature or a part thereof of the patient. The image data can be or include preoperative images, intraoperative images, postoperative images, or images taken independently of any surgical procedure. In some embodiments, the first imaging device 312 can be used to obtain first image data (e.g., a first image) at a first time, and the second imaging device 312 can be used to obtain second image data (e.g., a second image) at a second time after the first time. The imaging device 312 may be capable of taking 2D images or 3D images to generate image data. The imaging device 312 can be or include, for example, an ultrasound scanner (which may include, for example, physically separate transducers and receivers, or a single ultrasound transceiver), an O-arm, a C-arm, a G-arm, or any other device utilizing X-ray-based imaging (e.g., a fluoroscope, a CT scanner, or other X-ray machine), a magnetic resonance imaging (MRI) scanner, an optical coherence tomography (OCT) scanner, an endoscope, a microscope, an optical camera, a thermal imaging camera (e.g., an infrared camera), a radar system (which may include, for example, a transmitter, a receiver, a processor, and one or more antennas), or any other imaging device 312 suitable for obtaining an image of an anatomical feature of the patient. The imaging device 312 can be fully contained within a single housing, or can include a transmitter / transmitter and a receiver / detector located in separate housings or otherwise physically separated.

[0082] In some embodiments, the imaging device 312 can include more than one imaging device 312. For example, the first imaging device can provide first image data and / or a first image, and the second imaging device can provide second image data and / or a second image. In still other embodiments, the same imaging device can be used to provide both first image data and second image data and / or any other image data described herein. The imaging device 312 can be used to generate an image data stream. For example, the imaging device 312 can be configured to operate with the shutter open, or with the shutter continuously alternating between open and closed to capture consecutive images. For the purposes of this disclosure, unless otherwise specified, image data can be considered continuous and / or provided as an image data stream if the image data represents two or more frames per second.

[0083] The robot 314 can be any surgical robot or surgical robot system. The robot 314 can be or include, for example, a Mazor XTM Stealth Edition robotic guidance system. The robot 314 can be configured to position the imaging device 312 at one or more precise positions and orientations, and / or to return the imaging device 312 to the same position and orientation at a later time point. The robot 314 can additionally or alternatively be configured to manipulate surgical tools (whether or not guided by the navigation system 318) to perform or assist in surgical tasks. In some embodiments, the robot 314 can be configured to hold and / or manipulate anatomical elements during or in conjunction with a surgical procedure. The robot 314 can include one or more robotic arms 316. In some embodiments, the robotic arm 316 can include a first robotic arm and a second robotic arm, but the robot 314 can include more than two robotic arms. In some embodiments, one or more of the robotic arms 316 can be used to hold and / or manipulate the imaging device 312. In embodiments where the imaging device 312 includes two or more physically separate components (e.g., a transmitter and a receiver), one robotic arm 316 can hold one such component, and another robotic arm 316 can hold another such component. Each robotic arm 316 can be capable of positioning independently of the other robotic arms. The robotic arms 316 can be controlled in a single shared coordinate space or in separate coordinate spaces.

[0084] The robot 314 together with the robotic arms 316 can have, for example, one, two, three, four, five, six, seven or more degrees of freedom. Additionally, the robotic arms 316 can be positioned or can be capable of being positioned in any pose, plane, and / or focus. The pose includes position and orientation. Thus, the imaging device 312, surgical tool, or other object held by the robot 314 (or more specifically, by the robotic arms 316) can be precisely positioned at one or more desired and specific positions and orientations.

[0085] The robotic arm 316 can include one or more sensors that enable the processor 304 (or the processor of the robot 314) to determine the precise pose of the robotic arm (and any object or element held or attached to the robotic arm) in space.

[0086] In some embodiments, reference markers (i.e., navigation markers) may be placed on the robot 314 (including, for example, on the robotic arm 316), on the imaging device 312, or on any other object in the surgical space. The reference markers may be tracked by the navigation system 318, and the results of the tracking may be used by the robot 314 and / or by an operator of the system 300 or any of its components. In some embodiments, the navigation system 318 may be used to track other components of the system (e.g., the imaging device 312), and the system may operate without using the robot 314 (e.g., where a surgeon manually manipulates the imaging device 312 and / or one or more surgical tools, for example, based on information and / or instructions generated by the navigation system A18).

[0087] During operation, the navigation system 318 may provide navigation for the surgeon and / or the surgical robot. The navigation system 318 may be any currently known or future-developed navigation system, including, for example, the Medtronic StealthStation TM S8 surgical navigation system or any successor thereof. The navigation system 318 may include one or more cameras or other sensors for tracking one or more reference markers, navigation trackers, or other objects within the operating room or other room in which part or all of the system 300 is located. The one or more cameras may be optical cameras, infrared cameras, or other cameras. In some embodiments, the navigation system 318 may include one or more electromagnetic sensors. In various embodiments, the navigation system 318 may be used to track the position and orientation (e.g., pose) of the imaging device 312, the robot 314, and / or the robotic arm 316 and / or one or more surgical tools (or more specifically, to track the pose of a navigation tracker directly or indirectly attached in a fixed relationship to one or more of the foregoing). The navigation system 318 may include a display for displaying one or more images from an external source (e.g., the computing device 302, the imaging device 312, or other source) or for displaying images and / or video streams from one or more cameras or other sensors of the navigation system 318. In some embodiments, the system 300 may operate without using the navigation system 318. The navigation system 318 may be configured to provide guidance to the surgeon or other users of the system 300 or its components, to the robot 314, or to any other element of the system 300 regarding, for example, the pose of one or more anatomical elements, whether a tool is in the proper trajectory, and / or how to move the tool into the proper trajectory to perform a surgical task according to a preoperative or other surgical plan.

[0088] In some embodiments, the robot 314, the robotic arm 316, and the navigation system 318 may operate based on the stress maps generated as described herein. For example, the stress maps and / or bone cutting recommendations described herein may be used as inputs to determine a surgical plan to be executed by components of the system 300.

[0089] The database 330 may store information associating one coordinate system to another (e.g., associating one or more robotic coordinate systems to a patient coordinate system and / or a navigation coordinate system). The database 330 may additionally or alternatively store, for example, one or more surgical plans (including, for example, pose information about a target and / or image information about the patient's anatomy at and / or near the surgical site for use by the robot 314, the navigation system 318, and / or the computing device 302 or a user of the system 300); one or more useful images of a surgery performed by or assisted by one or more other components of the system 300; and / or any other useful information. The database 330 may be configured to provide any such information to the computing device 302 or any other device of the system 300 or any other device external to the system 300, either directly or via the cloud 334. In some embodiments, the database 330 may be or include a part of a hospital image storage system, such as a Picture Archiving and Communication System (PACS), a Health Information System (HIS), and / or another system for collecting, storing, managing, and / or transmitting electronic medical records including image data.

[0090] The cloud 334 may be or represent the Internet or any other wide area network. The computing device 302 may be connected to the cloud 334 using a wired connection, a wireless connection, or both via the communication interface 308. In some embodiments, the computing device 302 may communicate with the database 330 and / or an external device (e.g., a computing device) via the cloud 334.

[0091] The system 300 or a similar system may be used, for example, to implement one or more aspects of any of the methods 400, 500, and / or 600 described herein. The system 300 or a similar system may also be used for other purposes.

[0092] Figure 4 A method 400 is depicted that may be used to generate and display one or more stress maps, for example, based in part on FE analysis.

[0093] Method 400 (and / or one or more of its steps) can be implemented, for example, by at least one processor or otherwise executed. The at least one processor can be the same as or similar to processor 304 of computing device 302 described above. The at least one processor can be part of a robot (such as robot 314) or part of a navigation system (such as navigation system 318). A processor other than any of the processors described herein can also be used to execute method 400. The at least one processor can execute method 400 by executing elements stored in a memory (such as memory 306). The elements stored in the memory and executed by the processor can cause the processor to execute one or more steps of the functions as shown in method 400. One or more parts of method 400 can be executed by the processor executing any of the contents in the memory (such as stress simulation 320, stress map generation 322, deep learning model training 324, and / or stress map display 328).

[0094] Method 400 includes: generating a multi-class segmentation of an anatomical element of a patient based at least in part on a plurality of magnetic resonance images of anatomical elements from a plurality of patients. Additionally, method 400 includes: generating a plurality of stress maps by simulating stress on the anatomical element of the patient based at least in part on at least some of the plurality of multi-class segmentations and their stress maps, the simulated stress being simulated using a finite element (FE) analysis based at least in part on the multi-class segmentation (step 404). For example, a plurality of stresses can be simulated on the anatomical element based at least in part on simulating various physiological motions, deformations, and / or material changes that cause stress on the anatomical element. In some embodiments, the simulated stress can include vertebrae moving the spinal cord, discs compressing the spinal cord, sizing the ligamentum flavum of the spinal cord, deformation of the spinal cord, additional physiological motions of the spinal cord, or combinations thereof. Additionally, a separate stress map can be generated for each of the plurality of simulated stresses, where the plurality of stress maps includes the separate stress map. In some embodiments, the plurality of stress maps can include 3D stress maps of the anatomical element.

[0095] Method 400 further includes: displaying one or more of the plurality of stress maps via a user interface (step 408).

[0096] The present disclosure encompasses embodiments of method 400 that include more or fewer steps than those described above and / or one or more steps different from those described above.

[0097] Figure 5 Method 500 is depicted that can be used to generate and display one or more stress maps, for example, based at least in part on FE analysis and one or more deep learning models.

[0098] Method 500 (and / or one or more of its steps) may be implemented or otherwise performed, for example, by at least one processor. The at least one processor may be the same as or similar to processor 304 of computing device 302 described above. The at least one processor may be part of a robot (such as robot 314) or part of a navigation system (such as navigation system 318). Processors other than any of the processors described herein may also be used to perform method 500. The at least one processor may perform method 500 by executing elements stored in a memory (such as memory 306). The elements stored in the memory and executed by the processor may cause the processor to perform one or more steps of the functions as shown in method 500. One or more portions of method 500 may be performed by the processor executing any of the content in the memory (such as stress simulation 320, stress map generation 322, deep learning model training 324, and / or stress map display 328).

[0099] Method 500 includes: training a first deep learning model (step 504) at least in part based on a plurality of MR images of anatomical elements from a plurality of patients. For example, the plurality of MR images may include a plurality of 3D MR images (e.g., from an MRI spine database of "good" and "bad" spines, as referenced Figure 1 and Figure 2 as described). In some embodiments, the first deep learning model may be further trained at least in part based on a plurality of annotated soft tissue segmentation maps of anatomical elements from a plurality of patients.

[0100] Method 500 further includes: generating a multi-class segmentation of a patient's anatomical elements at least in part based on a plurality of MR images of anatomical elements from a plurality of patients. Additionally, method 500 includes: generating a plurality of stress maps simulating stress on the anatomical elements, the simulated stress being simulated using FE analysis at least in part based on the multi-class segmentation (step 508). Step 508 may implement similar aspects of step 404 as referenced Figure 4 as described. Additionally, the multi-class segmentation may be generated at least in part based on the first deep learning model.

[0101] Method 500 further includes: training a second deep learning model (step 512) at least in part based on a plurality of stress maps and multi-class segmentations of anatomical elements. Method 500 further includes: generating one or more of the plurality of stress maps for display (step 516) at least in part based on the second deep learning model.

[0102] Method 500 further includes: displaying (e.g., the one or more stress maps generated in step 516) one or more of the plurality of stress maps via a user interface (step 520).

[0103] Embodiments of the present disclosure cover embodiments of method 500 that include more or fewer steps and / or one or more steps different from those described above.

[0104] Figure 6 Depicts a method 600 that can be used, for example, to generate bone cutting recommendations.

[0105] Method 600 (and / or one or more of its steps) can be implemented or otherwise executed, for example, by at least one processor. The at least one processor can be the same as or similar to the processor 304 of the computing device 302 described above. The at least one processor can be part of a robot (such as robot 314) or part of a navigation system (such as navigation system 318). Processors other than any of the processors described herein can also be used to execute method 600. The at least one processor can execute method 600 by executing elements stored in a memory (such as memory 306). The elements stored in the memory and executed by the processor can cause the processor to execute one or more steps of the functions as shown in method 600. One or more parts of method 600 can be executed by the processor executing any of the contents in the memory (such as stress simulation 320, stress map generation 322, deep learning model training 324, and / or stress map display 328).

[0106] Method 600 includes: generating a plurality of stress maps and multi-class segmentations of an anatomical element based at least in part on simulating stress on the anatomical element of a patient, the simulated stress being simulated using FE analysis based at least in part on the multi-class segmentation (step 604). Step 604 can implement similar aspects of steps 404 and 508 as described respectively with reference to Figure 4 and Figure 5 steps 404 and 508 described above.

[0107] Method 600 further includes: generating a plurality of simulated stress relief maps based at least in part on the plurality of stress maps and on simulating the removal of one or more parts of the anatomical element, wherein the one or more parts of the anatomical element are simulated to be removed based at least in part on additional FE analysis (step 608). In some embodiments, a deep learning model (e.g., the second deep learning model or an additional deep learning model described herein) is trained based at least in part on the plurality of simulated stress relief maps, and a recommended surgical plan can be generated based at least in part on the deep learning model. For example, the recommended surgical plan can include bone cutting recommendations or suggestions.

[0108] Method 600 further includes: displaying one or more of the plurality of stress maps (via a user interface) (step 612). Method 600 further includes: displaying the recommended surgical plan based at least in part on the plurality of simulated stress relief maps via the user interface (step 616).

[0109] The present disclosure encompasses embodiments of method 600 that include fewer or more steps than those described above and / or one or more steps different from those described above.

[0110] As described above, the present disclosure encompasses methods (and corresponding descriptions of methods 400, 500, and 600) having fewer steps than all of the steps identified in Figure 4 , Figure 5 and Figure 6 , and methods (and corresponding descriptions of methods 400, 500, and 600) that include additional steps beyond the steps identified in Figure 4 , Figure 5 and Figure 6 . The present disclosure also encompasses methods that include one or more steps from one method described herein and one or more steps from another method described herein. Any correlation described herein can be or include registration or any other correlation.

[0111] The foregoing is not intended to limit the present disclosure to one or more forms disclosed herein. In the foregoing detailed description, for example, for purposes of simplifying the present disclosure, various features of the present disclosure are grouped together in one or more aspects, embodiments, and / or configurations. Features of the aspects, embodiments, and / or configurations of the present disclosure can be combined in alternative aspects, embodiments, and / or configurations other than those discussed above. The methods of the present disclosure should not be construed as reflecting an intention that the claims require more features than those expressly recited in each claim. Rather, as reflected in the following claims, the aspects of the invention lie in less than all of the features of a single foregoing disclosed aspect, embodiment, and / or configuration. Accordingly, the following claims are hereby incorporated into this detailed description, where each claim stands on its own as a separate preferred embodiment of the present disclosure.

[0112] In addition, although the foregoing has included a description of one or more aspects, embodiments, and / or configurations and certain variations and modifications, other variations, combinations, and modifications are within the scope of the present disclosure after understanding the present disclosure, for example, within the skill and knowledge of those skilled in the art. It is intended to obtain rights to alternative aspects, embodiments, and / or configurations, including alternatives, interchangeables, and / or equivalents of those claimed, including structures, functions, scopes, or steps, whether or not such alternatives, interchangeables, and / or equivalents of structures, functions, scopes, or steps are disclosed herein, and without intending to disclose any patentable subject matter.

Claims

1. A system for creating a spinal stress map, the system comprising: a processor; and a memory storing data for processing by the processor, the data when processed causing the processor to: generate a multi-class segmentation of the anatomical elements of a patient based at least in part on multiple magnetic resonance images of the anatomical elements from multiple patients; generate multiple stress maps based at least in part on simulating stress on the anatomical elements of the multiple patients, the simulated stress being simulated using finite element analysis based at least in part on the multi-class segmentation; determine one or more of the multiple stress maps to display based at least in part on one or more deep learning models configured to predict a multi-label mask and / or stress map of the anatomical elements; and display the one or more stress maps via a user interface.

2. The system according to claim 1, wherein the memory stores additional data for processing by the processor, the additional data when processed causing the processor to: train a deep learning model based at least in part on the multiple magnetic resonance images of the anatomical elements from the multiple patients; and generate the multi-class segmentation based at least in part on the deep learning model.

3. The system according to claim 2, wherein the deep learning model is further trained based at least in part on multiple annotated soft tissue segmentation maps of the anatomical elements from the multiple patients.

4. The system according to claim 2, wherein the multiple magnetic resonance images include multiple three-dimensional magnetic resonance images.

5. The system according to claim 1, wherein the memory stores additional data for processing by the processor, the additional data when processed causing the processor to: simulate multiple stresses on the anatomical elements based at least in part on simulating multiple physiological movements, deformations, material changes, or combinations thereof that cause stress on the anatomical elements.

6. The system according to claim 5, wherein the memory stores additional data for processing by the processor, the additional data when processed causing the processor to: generate a separate stress map for each of the simulated multiple stresses.

7. The system according to claim 1, wherein the memory stores additional data for processing by the processor, the additional data when processed causing the processor to: train a deep learning model based at least in part on the multiple stress maps and the multi-class segmentation of the anatomical elements; and generate the one or more stress maps for display via the user interface based at least in part on the deep learning model.

8. The system according to claim 1, wherein the memory stores additional data for processing by the processor, the additional data when processed causing the processor to: Generate a plurality of simulated stress relief maps at least in part based on the plurality of stress maps and the removal of one or more parts of the anatomical element, wherein the removal of the one or more parts of the anatomical element is simulated at least in part based on additional finite element analysis; and Display, via the user interface, a proposed surgical plan generated at least in part based on the plurality of simulated stress relief maps.

9. The system of claim 8, wherein the memory stores additional data for processing by the processor, the additional data when processed causing the processor to:[[]] Train a deep learning model at least in part based on the plurality of simulated stress relief maps, wherein the proposed surgical plan is generated at least in part based on the deep learning model.

10. The system of claim 1, wherein the plurality of stress maps includes three-dimensional stress maps of the anatomical element.

11. A system for creating spinal stress maps, the system comprising: A processor; And A memory that stores data for processing by the processor, the data when processed causing the processor to:[[]] Generate a multi-class segmentation of the anatomical element of the patient at least in part based on a plurality of magnetic resonance images of the anatomical element from a plurality of patients; Generate a plurality of stress maps of the anatomical element of the patient at least in part based on simulated stress on the multi-class segmentation of the anatomical element from a plurality of patients, the simulated stress being simulated using finite element analysis at least in part based on the multi-class segmentation; Determine, at least in part based on a deep learning model, one or more of the plurality of stress maps for display, the deep learning model being configured to predict a multi-label mask and / or stress map of the anatomical element; And Display, via a user interface, one or more of the plurality of stress maps.

12. The system of claim 11, wherein the memory stores additional data for processing by the processor, the additional data when processed causing the processor to:[[]] Train a first deep learning model at least in part based on the plurality of magnetic resonance images of the anatomical element from the plurality of patients; and Generate the multi-class segmentation at least in part based on the first deep learning model.

13. The system of claim 12, wherein the memory stores additional data for processing by the processor, the additional data when processed causing the processor to:[[]] Train a second deep learning model at least in part based on the plurality of stress maps and the multi-class segmentation of the anatomical element; and Generate the one or more of the plurality of stress maps for display via the user interface at least in part based on the deep learning model.

14. The system of claim 12, wherein the first deep learning model is further trained at least in part based on a plurality of annotated soft tissue segmentation maps of the anatomical element from the plurality of patients.

15. The system according to claim 11, wherein the memory stores additional data for processing by the processor, and the additional data, when processed, causes the processor to: Simulate a plurality of stresses on the anatomical element at least in part based on simulating a plurality of physiological movements, deformations, material changes, or combinations thereof that induce stress on the anatomical element.

16. The system according to claim 15, wherein the memory stores additional data for processing by the processor, and the additional data, when processed, causes the processor to: Generate individual stress maps for each of the plurality of simulated stresses, wherein the plurality of stress maps includes the individual stress maps.

17. The system according to claim 11, wherein the memory stores additional data for processing by the processor, and the additional data, when processed, causes the processor to: Generate a plurality of simulated stress relief maps at least in part based on the plurality of stress maps and simulating the removal of one or more portions of the anatomical element, wherein the one or more portions of the anatomical element are simulated as being removed at least in part based on additional finite element analysis; and display a proposed surgical plan generated at least in part based on the plurality of simulated stress relief maps via the user interface.

18. The system according to claim 17, wherein the memory stores additional data for processing by the processor, and the additional data, when processed, causes the processor to: Train a deep learning model at least in part based on the plurality of simulated stress relief maps, wherein the proposed surgical plan is generated at least in part based on the deep learning model.

19. A system for creating spinal stress maps, the system comprising: A processor; And A memory that stores data for processing by the processor, and the data, when processed, causes the processor to: Generate a multi-class segmentation of the spinal cord of a patient at least in part based on a plurality of magnetic resonance images of the spinal cords of a plurality of patients; Generate a plurality of stress maps at least in part based on simulating stress on the spinal cord, the simulated stress being simulated using finite element analysis at least in part based on the multi-class segmentation; Determine one or more of the plurality of stress maps to display at least in part based on one or more deep learning models configured to predict a multi-label mask and / or stress map of the spinal cord; And Display one or more of the plurality of stress maps via a user interface.

20. The system according to claim 19, wherein the simulated stress includes moving the vertebrae of the spinal cord, compressing the discs of the spinal cord, sizing the ligamentum flavum of the spinal cord, deforming the spinal cord, additional physiological movements of the spinal cord, or combinations thereof.

Images