A technology for mapping 3D human body structure data onto 2D human body structure data.

Abstract

Disclosed are systems and methods for designing and implementing patient-specific surgical procedures and / or medical devices. An example of the method includes: acquiring first multidimensional image data of an anatomical region of a patient in a first loading state; determining regions corresponding to anatomical elements of the spine from the first multidimensional image data; identifying a first set of landmarks in each region; acquiring second multidimensional image data of an anatomical region including the spine of a patient in a second loading state; identifying a second set of landmarks corresponding to the first set of landmarks from the second multidimensional image data; acquiring aligned multidimensional image data of the patient's spine; generating a design for a medical implant based on spinal-pelvic parameters measured from the aligned multidimensional image data; and manufacturing a medical implant by transmitting the design for the medical implant to a manufacturing apparatus.

Description

Technical Field

[0001] 〔Cross - Reference to Related Applications〕 This application is a continuation - in - part of U.S. Patent Application Publication No. 18 / 102,444, filed on January 27, 2023, which is hereby incorporated by reference in its entirety.

[0002] The present disclosure generally relates to medical implant generation techniques and / or the design and implementation of techniques for mapping features of the human body structure captured in different types of multi - dimensional imaging, such as mapping features of the human body structure captured in three - dimensional (3D) to the positions of features captured in two - dimensional (2D), and / or mapping anatomical features captured in 3D to those captured in 2D.

Background Art

[0003] The human body structure can be imaged using different imaging techniques. For example, X - ray technology can be used to image a fractured arm to identify the location and / or severity of the fracture. In another example, a computed tomography (CT) scan or magnetic resonance imaging (MRI) can be performed on a part of the human body to determine problems or anatomical features of that part of the human body. Different imaging techniques have their own strengths and weaknesses, and physicians can use these imaging techniques to obtain specific details about a part of the human body as desired.

Brief Description of the Drawings

[0004] [Figure 1] A flowchart showing a method of mapping a 3D scan of at least a part of a human body structure to a 2D scan of at least the same part of the human body structure according to an embodiment.

[0005] [Figure 2] A network connection diagram showing a computer system for providing patient - specific medical care according to an embodiment.

[0006] [Figure 3]This figure shows a computer device suitable for use in relation to the system shown in Figure 2, according to one embodiment.

[0007] [Figure 4A] This figure shows an example flowchart of the process of mapping a part of the human body structure in 3D image analysis to a corresponding part of the same human body structure in a 2D image.

[0008] [Figure 4B] This diagram shows an example flowchart for mapping a portion of a human body structure in multidimensional image data to a corresponding portion of the same human body structure in another multidimensional image data.

[0009] [Figure 5] This figure shows an example of a 2D image of the anterior-posterior view of the spine obtained from an X-ray.

[0010] [Figure 6] This figure shows an example of a 2D image of a lateral view of the spine obtained from an X-ray.

[0011] [Figure 7] This figure shows an example of a 2D image of the coronal view of the spine region obtained from a CT scan.

[0012] [Figure 8] This figure shows an example of a 2D image of the coronal view of the spine region obtained from a CT scan.

[0013] [Figure 9] This figure shows an example of a 2D sagittal view image of the spinal region obtained from a CT scan.

[0014] [Figure 10] This figure shows an example flowchart of the process for generating a 3D spinal model that simulates vertebral movement and patient load.

[0015] [Figure 11] FIG. is a diagram showing an example of a 2D image of a view of the spine including sagittal vertical axis measurement values obtained from X-ray imaging.

[0016] [Figure 12] FIG. shows an exemplary surgical planning report that can be used and / or generated in connection with the methods described herein, according to one embodiment.

[0017] [Figure 13] FIG. shows an exemplary surgical planning report that can be used and / or generated in connection with the methods described herein, according to one embodiment.

[0018] [Figure 14] FIG. shows exemplary XYZ measurement values and rotational measurement values of a patient's biological structure, according to one embodiment.

[0019] [Figure 15] FIG. is a diagram showing an example of a flowchart of an operation for generating a 3D virtual model that simulates a patient's postoperative loading state, according to one embodiment.

[0020] [Figure 16] FIG. is a diagram showing an example of a flowchart of an operation for generating a multi-fidelity 3D virtual model of a patient's anatomical region, according to one embodiment.

[0021] [Figure 17] FIG. is a diagram showing an example of a flowchart of an operation for generating a 3D virtual model of a patient's biological structure, according to one embodiment.

MODE FOR CARRYING OUT THE INVENTION

[0022] The drawings illustrate various embodiments of the system, the method, and various other embodiments of the present disclosure. Those skilled in the art will understand that the element boundaries shown in the drawings (e.g., boxes, groups of boxes, or other shapes) represent examples of boundaries. In some examples, one element may be designed as multiple elements, or multiple elements may be designed as one element. In some examples, an element shown as an internal component of one element may be implemented as an external component of another element, and vice versa. Furthermore, the elements may not be to scale. The following description is non-limiting and non-exclusive, with reference to the drawings. The components in the drawings are not necessarily to scale, and the emphasis is rather on illustrating the principle.

[0023] This technology relates to a system, apparatus, and method for mapping image data from an image dataset. The image dataset may include different types of scans or images of a portion of a subject. The mapping can compensate for different types of multidimensional image data. For example, a three-dimensional (3D) scan or image of at least a portion of a human structure can be mapped to a position in a 2D plane of a two-dimensional (2D) scan or image of at least the same portion of the human structure. One of the technical advantages of this technology is its ability to generate 3D anatomical data (or 3D models or scans) that reflect the spatial relationships captured using 2D scans or images. For example, when imaging or scanning a portion of a patient's human structure using 3D imaging techniques (e.g., computed tomography (CT) scans or magnetic resonance imaging (MRI)), the 3D scanned biostructure of the patient is captured when the patient is lying down or in a horizontal (or unloaded) position. However, physicians or surgeons may want to view and understand such information of a patient in a vertical (or loaded or weight-bearing) position. In this example, the disclosed technology can map 3D scanned bio-structures of the same patient to the locations on 2D images or scans acquired by 2D imaging technology (e.g., X-ray) when the patient is in a load-bearing, vertical, or standing position. Generally, 3D imaging technology can produce higher-resolution images or scans that can provide more detail about the human structure compared to 2D imaging technology. Thus, one of the technical advantages of the disclosed technology is that it can display high-resolution 3D anatomical data of a patient in a load-bearing or weight-bearing posture. Based on this 3D anatomical data, surgical plans, medical devices, and instruments can be designed. The disclosed technology can also use 3D anatomical data in a weight-bearing posture to determine how to improve spinal orientation and / or vertebral positioning.

[0024] In some embodiments, the system can acquire images of a patient's biological structures. The system can identify dimensionally constant features to determine image acquisition information. Image acquisition information may include, for example, the patient's posture and load state at the time of image acquisition. Dimensionally constant features may include, for example, bone tissue or implants (e.g., rigid implants). The system can evaluate images of biological structures in load states (e.g., standing or upright), unloaded states (e.g., when the patient is lying down), and preset diagnostic positions. The system can correlate the positions of dimensionally constant features in images taken under various load conditions to determine, for example, anatomical changes associated with multiple load states. The system may have trained machine learning algorithms that predict or determine (single or multiple) load states in acquired images based on past images.

[0025] The system can generate an anatomical model of a patient using information from all or part of an image. The system can quantify and score image resolution based on, for example, pixel size, voxel size, or spatial resolution (e.g., the interval / number of slices in a tomographic image). Smaller pixel or voxel sizes correspond to higher resolution. Pixel size is determined by dividing the physical dimensions of the image detector (such as X-ray film or a digital sensor) by the number of pixels in the corresponding direction. The resolution evaluation method can be selected according to the specific imaging modality and analytical objectives.

[0026] The system can obtain the shape of anatomical elements from images that can provide high-definition images, high-resolution images, or other images that show the surface features of anatomical elements. The system can use images showing the load state to determine additional information about anatomical elements in a load state. For example, CT scans can be analyzed to obtain dimensional information of individual anatomical elements such as vertebrae or vertebral bodies. X-rays, such as scanning X-rays, can be used to determine dimensional information to pinpoint the location of these anatomical elements. As a result, the system can combine information from images taken when the patient is lying down (e.g., lying down for a CT scan) with information from images taken when the patient is in a load state (e.g., when the patient is standing during a standing X-ray). The system can automatically compare the patient's load state with the images using historical information or user input.

[0027] The system can identify abnormal load conditions and images and notify the user that additional images are needed. In some embodiments, the system can compare the captured anatomical features to the locations of predicted anatomical features and send an alert to the user if the captured anatomical elements are located within a threshold deviation.

[0028] The system can accurately predict and / or display the sagittal balance and / or coronal balance of vertebrae, including untreated vertebrae. The system can favorably predict un-imaged areas of a patient's spine. For example, suppose a patient undergoes a high-resolution CT scan of the lumbar region. The system can predict how the shape of other areas, including the thoracic and cervical regions, will change as a result of lumbar procedures. The system can determine how to construct a suitable 3D model to simulate the results of lumbar procedures, even if the thoracic and cervical regions are not imaged using a CT scan. For example, the system can generate a spinal model including the thoracic and cervical regions based on non-CT images of the entire spine. An example of a non-CT image is, for example, a radiograph. In some procedures, the system can acquire a complete spinal image to generate a 3D spinal model. A complete spinal image can be taken from different viewpoints. For example, a complete spinal image can be a transverse image, anterior-posterior (AP) image, or other images. The system can combine non-CT images with a CT scan of the lumbar region to generate a complete 3D model. The system can perform 3D patient matching of vertebral bodies at various levels using CT scans and non-CT scans. Non-CT scans can provide positional information to accurately determine the location of vertebral bodies. For example, a non-CT scan may be an upright radiograph. The system can obtain load information from the upright radiograph to accurately determine the location of vertebral bodies generated based on CT scans that may not show the desired load state.

[0029] The 3D model can include planning information to receive further image input from users, such as a surgical team. The 3D model can be used to generate a 3D plan. In some embodiments, the 3D model can receive user input for manual correction. One or more predictive models can be applied to simulate compensatory mechanisms for spinal corrections. Compensatory mechanisms may include, for example, lumbar corrections and how such corrections affect other parts of the spine. Predictive models can be generated using historical patient data. Examples of predictive models include, for example, (1) a compensatory response of pelvic tilt to lumbar correction, (2) a compensatory response of thoracic kyphosis to lumbar correction, and / or (3) a compensatory response of cervical lordosis to lumbar correction. Other predictive models can also be generated based on lumbar corrections. The system can enable the surgical team to plan local or global alignment with compensatory curvature for better patient outcomes. The system can also be used to generate surgical plans that provide access to the intervertebral disc space. For example, the system can generate a 3D model to assist in accessing the intervertebral disc space by identifying the positions of the iliac crest and ribs relative to the surgical route.

[0030] Figure 1 is a flowchart illustrating a method 100 of mapping a 3D scan of at least a portion of a human body structure to a 2D scan of at least the same portion of the human body structure, according to one embodiment. In operation 102, the image processing module can acquire a volumetric 3D image of at least the spine of the patient. The volumetric 3D image can be acquired by scanning or imaging the patient's spine using a 3D imaging technique (e.g., CT scan or MRI) when the patient is in a horizontal or supine position. For example, the image processing module can acquire a high-quality Digital Imaging and Communications in Medicine (DICOM) image of the 3D biostructure. The volumetric 3D image may include images acquired along multiple planes that bisect the spine.

[0031] In operation 104, the image processing module can segment the 3D skeletal biostructure of the spine. The spine includes multiple segments or multiple anatomical elements that can start at C1, C2, and C3 and end at L4, L5, the sacrum, and the coccyx. In operation 104, the image processing module can segment the lumbar vertebrae (L1-L5) and the sacrum (or S1) from the volumetric 3D image and perform image processing techniques to identify a series of planes or regions in the volumetric 3D image corresponding to each of the L1-L5 and sacral segments or anatomical elements. Thus, for example, the image processing module can identify a first series of planes or a first region in the 3D volumetric image associated with L1, a second series of planes or a second region in the 3D volumetric image associated with L2, and so on, until the image processing module can identify a series of planes or regions in the 3D volumetric image associated with S1. Therefore, by segmenting the biological structure, six 3D objects are obtained representing the L1, L2, L3, L4, L5, and S1 vertebrae, each still containing its original position in the volumetric 3D image. In some embodiments, one or more other sections of the spine (e.g., the thoracic vertebrae including vertebrae T1-T12) can also be segmented using the technique described for operation 104.

[0032] In operation 106, the image processing module can identify landmarks on lumbar and sacral segments or anatomical elements from a 3D volumetric image. For example, the image processing module can identify the anterior, posterior, left, and right ends of the vertebral endplate for each of the L1-L5 lumbar and sacral vertebrae. The vertebral endplates can be located on the superior and inferior surfaces of the vertebrae. The image processing module can identify additional landmarks for more accurate mapping. For example, the image processing module can identify the inferior surface of the spinous process, the superior surface of the spinous process, the left surface of the transverse process, and the right surface of the transverse process for each segment or anatomical element of the lumbar vertebrae. Each landmark identified by the image processing module can be associated with a 3D location (e.g., x, y, and z coordinates) of the landmark in 3D space.

[0033] In some embodiments, a graphical user interface (GUI) displayed by the image processing module can provide the user with an option to perform either faster mapping or more accurate mapping. If the image processing module receives an instruction from the GUI that the faster mapping option has been selected, it can identify the anterior, posterior, left, and right ends of the spinal endplate. If the image processing module receives an instruction from the GUI that the more accurate mapping option has been selected, it can identify, for each segment of the lumbar vertebra, the anterior, posterior, left, right ends of the spinal endplate, the inferior surface of the spinous process, the superior surface of the spinous process, the left surface of the transverse process, and the right surface of the transverse process.

[0034] In some embodiments, an image processing module can measure the height and angle between individual vertebrae using landmarks on lumbar vertebral segments or anatomical elements.

[0035] In operation 108, the image processing module can acquire a two-dimensional (2D) image including the lumbar spine and sacrum. The 2D images of the lumbar spine and sacrum belong to the same patient from whom spine-related information was acquired and analyzed in operations 102-106. The 2D images can be acquired by scanning or imaging the patient's spine using a 2D imaging technique (e.g., X-ray) when the patient is in a vertical or load-bearing position. In some embodiments, the image processing module can acquire different multi-dimensional images. The image processing module can analyze these images to determine, for example, whether the image set contains 2D data or 3D data, whether it contains load condition data, whether it contains patient posture data, etc. Load conditions, for example, may include whether the anatomical features are under natural load, whether the patient is standing or exercising, or whether it is in a different load-bearing position. Subject posture data may include, for example, whether the patient is lying down, or whether the patient is in a specific posture, etc.

[0036] In operation 110, the image processing module can identify landmarks on lumbar and sacral segments or anatomical elements from 2D images. Operation 110 can identify the same landmarks on the 2D images that were identified in the 3D volumetric images. For example, the image processing module can obtain sagittal views (e.g., median sagittal view) and coronal views of 2D planar images of the same biological structures acquired and analyzed in operations 102-106. From the sagittal and coronal views, the image processing module can identify the anterior, posterior, left, and right ends of the vertebral endplates for each of the L1-L5 lumbar and sacral vertebrae. The image processing module can identify further landmarks on the sagittal and coronal views for more accurate mapping. For example, the image processing module can identify the inferior surface of the spinous process, the superior surface of the spinous process, the left surface of the transverse process, and the right surface of the transverse process for each of the lumbar segments or anatomical elements. Each landmark identified by the image processing module can be associated with a 2D location (e.g., two coordinates) in 2D space. For example, in a sagittal view, since the image is captured in 2D, landmark points can be associated with x and y coordinates, and in a coronal view, since the image is captured in 2D, landmark points can include y and z coordinates. The coordinates in the two planes mentioned above are relative coordinates representing the location of the landmark on the coronal view plane and the sagittal view plane.

[0037] In operation 112, the image processing operation can map the 3D biostructure to 2D landmarks, as further described in operations 114-122, which can be performed as part of the execution of operation 112. Operation 112 may include the process of mapping the 3D biostructure to 2D landmarks, where the image processing module can scale, translate, and / or rotate the 3D biostructure to align identified 3D landmarks (from operation 106) with identified 2D landmarks (from operation 110). In some embodiments, in operation 112, the 3D biostructure can be loaded into 3D space such that segmented vertebrae appear relative to each other as in the original 3D image. Operations 114-122 can be performed for each view, including lateral and AP views, as further described below. In the lateral view, the image processing module can map anterior and posterior landmarks from the 3D and 2D images. In the AP view, the image processing module can map left and right landmarks from the 3D and 2D images.

[0038] In operation 114, the image processing module can form a first 2D plane (or 2D planar image) in a sagittal view from a 3D volumetric measurement of the biological structure approximately along the midline. Thus, in operation 114, the image processing module can form a first 2D planar image in a sagittal view from a 3D volumetric image, where the first 2D plane is a midline plane, such as a plane bisecting the sacrum along the sagittal view. The technical advantage of starting processing from the sagittal view is that the analysis can determine whether the lumbar vertebrae are aligned or in the correct position.

[0039] In operation 116, the image processing module projects a 2D image of a sagittal view (e.g., a 2D X-ray sagittal image) onto a first 2D plane (or first 2D planar image). The image processing module can scale and / or translate the first 2D plane so that the sacrum in the 2D sagittal image (or 2D image) overlaps with the 3D version of the sacrum. In some embodiments, the image processing module can scale, rotate, and / or translate the 2D sagittal image (or 2D image) so that the sacrum in the first 2D plane overlaps with the sacrum in the 2D sagittal image.

[0040] After performing operations 114-116, the image processing module determines that the sacrum from the 2D image and the sacrum from the 3D image have been aligned, and can then align one or more lumbar vertebrae as described in operations 118-122. The technical advantage of using the sacrum first to align the 2D and 3D images is that the sacrum does not move much between the loaded and unloaded positions, and therefore aligning the sacrum first allows for more efficient use of computational resources to align the lower region of the spine (e.g., the lumbar spine). In some embodiments, the lumbar spine can also be used in operation 116 to align the 2D sagittal image onto a first 2D plane that bisects the lumbar spine.

[0041] In operation 118, the image processing module can align common landmarks from the 3D analysis (or 3D image) to the 2D analysis (or 2D image). For example, in operation 118, the image processing module can align a lumbar vertebra (e.g., the L5 vertebra), moving the anterior and posterior landmarks of the lumbar vertebra (e.g., L5) in the 3D volumetric image to match the same landmarks in the sagittal plane image of the 2D image, and adjusting other landmarks by the same amount used to move the anterior and posterior landmarks.

[0042] In operation 120, the image processing module can form a second 2D plane from the 3D volumetric image that extends across the sacrum along the coronal view. The image processing module projects the 2D coronal image (e.g., a 2D X-ray coronal image) onto the second 2D plane. The image processing module can scale and / or translate the second 2D plane so that the sacrum in the 2D coronal image (or 2D image) overlaps with the sacrum in the 3D version. In some embodiments, the image processing module can scale and / or translate the 2D coronal image (or 2D image) so that the sacrum in the second 2D plane overlaps with the sacrum in the 2D coronal image.

[0043] In operation 122, the image processing module moves the anterior and posterior landmarks of the lumbar vertebrae (e.g., L5) in the 3D volumetric image to match the same landmarks on the coronal plane image of the 2D image, and can adjust other landmarks by the same amount used to move the anterior and posterior landmarks. The technical advantage of mapping the sacrum between the 2D plane of the 3D volumetric image and the 2D image is that it becomes possible to use common coordinates between the coronal and sagittal plane images in the mapping calculation more efficiently. In operations 118 and 122, by moving / adjusting the lumbar vertebral landmarks from the 3D volumetric image to match the same landmarks on the sagittal and coronal plane images of the 2D image, it is possible to obtain 3D anatomical data of at least a portion of the spine in a loading or weight-bearing posture.

[0044] As indicated by the arrow from action 122 to action 118, actions 118–122 can be repeated for one or more further vertebrae (e.g., L4, L3, L2, and / or L1). In some embodiments, actions 118–122 are performed for all of the remaining four vertebrae. Once actions 118–122 have been performed for one or more further lumbar vertebrae, the 3D biostructure can be aligned to the position of the vertebrae in the 2D planar image to form aligned 3D image data. The aligned 3D image data includes a 3D representation of at least the sacrum and lumbar vertebrae (e.g., L1–L5) in a loading or weight-bearing posture. In some embodiments, an image processing module can generate a corrective model of the spine based on the aligned 3D image data.

[0045] In operation 124, the image processing module can display a reorganized 3D analysis (or 3D image) by rearranging the biological structure based on the 2D analysis (or 2D image). In some embodiments, in operation 124, the image processing module can display aligned 3D biological structures of the lumbar region and sacrum aligned with the corresponding biological structures in the 2D image data on a GUI.

[0046] In some embodiments, an image processing module can measure spinopelvic parameters (e.g., lumbar lordosis, pelvic tilt, sagittal vertical axis (SVA), Cobb angle, coronal offset, etc.) from a reconstructed 3D image volume that can be load-bearing after mapping a 3D biological structure to a 2D image.

[0047] In some embodiments, an image processing module performs further analysis on a 2D-mapped 3D biostructure to determine whether any feature of the spine can be corrected. For example, the image processing module may determine the amount by which the 3D biostructure can be corrected and send instructions to another computer to design an implant corresponding to the determined amount and fit it to the corrected 3D biostructure.

[0048] The image processing module can generate a virtual model of a patient's biostructure using modified or aligned 3D biostructures. This virtual model can be multidimensional (e.g., 2D or 3D) and may include, for example, computer-aided design (CAD) data, material data, surface modeling, or manufacturing data. CAD data may include, for example, 3D modeling data (e.g., part files, assembly files, libraries, part / object identifiers, etc.), model geometry, object representations, parametric data, object representations, topology data, surface data, assembly data, metadata, etc. The image processing module can also use the virtual model of the patient's biostructure to generate predicted postoperative or corrective anatomical models, surgical plans, virtual implant models, implant design parameters, and instruments. The above examples are U.S. Patent Application Publications 16 / 048,167, 16 / 242,877, 16 / 207,116, 16 / 352,699, 16 / 383,215, 16 / 569,494, 16 / 699,447, 16 / 735,222, 16 / 987,113, 16 / 990,810, and 17 / 085 These are listed in issues 564, 17 / 100, 396, 17 / 342, 329, 17 / 518, 524, 17 / 531, 417, 17 / 835, 777, 17 / 851, 487, 17 / 867, 621, and 17 / 842, 242, each of which is incorporated herein by reference in its entirety.

[0049] Figure 2 is a network connection diagram showing a computer system 200 that provides patient-specific medical care according to one embodiment. As will be described in more detail herein, the system 200 is configured to generate a medical plan for a patient. In some embodiments, the system 200 is configured to generate a medical plan for a patient suffering from an orthopedic disease or disorder such as trauma (e.g., fracture), cancer, deformity, degeneration, pain (e.g., back pain, leg pain), abnormal spinal curvature (e.g., scoliosis, lordosis, kyphosis), abnormal spinal displacement (spondylolisthesis, lateral displacement, axial displacement, etc.), osteoarthritis, lumbar degenerative disc disease, cervical degenerative disc disease, lumbar spinal stenosis, cervical spinal stenosis, or a combination thereof. The medical plan may include surgical information, surgical plans, technical recommendations (e.g., recommendations for devices and / or instruments), and / or medical device designs. For example, a medical plan may include at least one treatment procedure (e.g., a surgical procedure or intervention) and / or at least one medical device (e.g., an implantable medical device (also referred herein as “implant” or “implantable device”) or implant delivery device).

[0050] In some embodiments, the system 200 generates a medical plan, also referred herein as a “patient-specific” or “personalized” treatment plan, that is customized to a particular patient or group of patients. A patient-specific treatment plan may include at least one patient-specific surgical procedure and / or at least one patient-specific medical device that is designed and / or optimized to the patient’s specific characteristics (e.g., condition, biostructure, pathological condition, state, medical history). For example, a patient-specific medical device may not be a ready-made device but may be specifically designed and manufactured for a particular patient. On the other hand, it should be understood that a patient-specific treatment plan may also include aspects that are not customized to a particular patient. For example, a patient-specific or personalized surgical procedure may include one or more instructions, parts, steps, etc. that are not patient-specific. Similarly, a patient-specific or personalized medical device may include one or more components that are not patient-specific and / or can be used with instruments or tools that are not patient-specific. Personalized implant designs can be used to manufacture or select patient-specific technologies, including medical devices, instruments, and / or surgical kits. For example, a personalized surgical kit may include one or more patient-specific devices, patient-specific instruments, non-patient-specific techniques (e.g., standard instruments, devices, etc.), usage instructions, patient-specific treatment plan information, or a combination of these.

[0051] System 200 includes a client computer device 202, which may be a user device such as a smartphone, mobile device, laptop, desktop, personal computer, tablet, phablet, or other such device well known in the art. As further described herein, the client computer device 202 may include one or more processors and memory for storing instructions that can be executed by one or more processors to perform the methods described herein. The client computer device 202 may be associated with a healthcare provider treating patients. Although Figure 2 shows a single client computer device 202, in another embodiment, the client computer device 202 may instead be implemented as a client computer system including multiple computer devices, and thus the operations described herein with respect to the client computer device 202 may instead be performed by the computer system and / or multiple computer devices.

[0052] The client computer device 202 is configured to receive a patient dataset 208 related to the patient to be treated. The patient dataset 208 may include data representing the patient's condition, bio-structure, pathological condition, medical history, preferences, and / or any other information or parameters related to the patient. For example, patient dataset 208 may include medical history, surgical intervention data, treatment outcome data, follow-up data (e.g., physician's notes), patient feedback (e.g., feedback obtained using quality of life questionnaires or surveys), clinical data, provider information (e.g., physician, hospital, surgical team), patient information (e.g., demographics, sex, age, height, weight, type of condition, occupation, activity level, tissue information, health assessment, comorbidities, health-related quality of life (HRQL)), vital signs, diagnostic results, medication information, allergies, imaging data (e.g., camera images, magnetic resonance imaging (MRI) images, ultrasound images, computed tomography (CAT) images, positron emission tomography (PET) images, X-ray images), or diagnostic equipment information (e.g., manufacturer, model number, specifications, user-selected settings / configuration, etc.). In some embodiments, the patient dataset 208 includes data representing one or more of the following: patient identification number (ID), age, sex, body mass index (BMI), lumbar lordosis, (single and double) Cobb angle, pelvic incidence, intervertebral disc height, segment flexibility, bone quality, rotational displacement, and / or spinal treatment level.

[0053] The client computer device 202 is operationally connected to the server 206 via the communication network 204, thereby enabling data transfer between the client computer device 202 and the server 206. The communication network 204 can be a wired and / or wireless network. If the communication network 204 is wireless, it can be implemented using communication technologies such as visible light communication (VLC), worldwide interoperability for microwave access (WiMAX), long-term evolution (LTE), wireless local area network (WLAN), infrared (IR) communication, public switched telephone network (PSTN), radio waves, and / or other communication technologies well known in the art.

[0054] Server 206, which may also be called a “treatment support network” or “prescriptive analytics network,” may include one or more computer devices and / or systems. As further described herein, Server 206 may include one or more processors and memory for storing instructions that can be executed by one or more processors to perform the methods described herein. In some embodiments, Server 206 is implemented as a distributed “cloud” computer system or facility across a preferred combination of either hardware and / or virtual computing resources.

[0055] The client computer device 202 and the server 206 can individually or collectively perform various methods for providing patient-specific medical care as described herein. For example, some or all steps of the methods described herein can be performed by the client computer device 202 alone, by the server 206 alone, or by a combination of the client computer device 202 and the server 206. Thus, although specific operations relating to the server 206 are described herein, these operations can also be performed by the client computer device 202, and vice versa. For example, the client computer device 202 and / or the server 206 may include (one or more) processors that can be configured to perform the operations described above for the image processing module.

[0056] Server 206 includes at least one database 210 configured to store reference data useful for the treatment planning methods described herein. The reference data may include historical and / or clinical data from the same or other patients, data collected from the patient's previous surgeries and / or other treatments by the same or other healthcare provider, data relating to medical device design, data collected from research groups or survey groups, data from clinical databases, data from academic institutions, data from implant manufacturers or other medical device manufacturers, data from image analysis, data from simulations, data from clinical trials, demographic data, treatment data, outcome data, or mortality data.

[0057] In some embodiments, database 210 includes multiple reference patient datasets, each associated with a corresponding reference patient. For example, a reference patient may be a patient who has previously received treatment or a patient currently receiving treatment. Each reference patient dataset may include data representing any other information or parameters related to the reference patient, such as the corresponding reference patient's condition, biomechanism, pathological condition, medical history, disease progression, preferences, and / or any of the data described herein with respect to patient dataset 208. In some embodiments, a reference patient dataset includes pre-operative data, intra-operative data, and / or post-operative data. For example, a reference patient dataset may include data representing one or more of the following: patient ID, age, sex, BMI, lumbar lordosis, (single and double) Cobb angle, pelvic incidence angle, intervertebral disc height, segment flexibility, bone quality, rotational displacement, and / or level of spinal treatment. As another example, a reference patient dataset may include treatment data relating to at least one treatment procedure performed on the reference patient, such as a description of a surgical procedure or intervention (e.g., surgical approach, osteotomy, surgical manipulation, orthodontic procedure, implant or other device placement). In some embodiments, treatment data may include medical device design data such as the physical properties (e.g., size, shape, volume, material, mass, weight), mechanical properties (e.g., stiffness, strength, modulus of elasticity, hardness), and / or biological properties (e.g., bone integration, cell adhesion, antibacterial properties, antiviral properties) of at least one medical device used to treat a reference patient. In yet another example, the reference patient dataset may include outcome data representing the treatment outcomes of the reference patient, such as corrected anatomical metrics, presence of fusion, HRQL, activity level, return to work, complications, recovery time, efficacy, mortality, and / or follow-up surgery.

[0058] In some embodiments, the server 206 receives at least a portion of reference patient datasets from multiple healthcare provider computer systems (e.g., systems 212a-212c, collectively 212). The server 206 can connect to the healthcare provider computer systems 212 via one or more communication networks (not shown). Each healthcare provider computer system 212 may be associated with a corresponding healthcare provider (e.g., a physician, surgeon, clinic, hospital, medical network, etc.). Each healthcare provider computer system 212 may contain at least one reference patient dataset (e.g., reference patient datasets 214a-214c, collectively 214) associated with reference patients treated by the corresponding healthcare provider. The reference patient dataset 214 may include, for example, electronic medical records, electronic health records, biomedical datasets, etc. The server 206 receives the reference patient dataset 214 from the healthcare provider computer systems 212 and can reformat it into a different format for storage in the database 210. Optionally, the reference patient dataset 214 may be processed (e.g., cleaned) to ensure that the represented patient parameters are more likely to be useful in the treatment planning methods described herein.

[0059] As will be described in more detail herein, the server 206 may be configured to include one or more algorithms that generate patient-specific treatment plan data (e.g., treatment procedures, medical devices) based on reference data. In some embodiments, patient-specific data is generated based on the correlation between the patient dataset 208 and the reference data. Optionally, the server 206 can predict outcomes including recovery time, efficacy based on clinical endpoints, likelihood of success, predicted mortality, or predicted related follow-up surgeries. In some embodiments, the server 206 can continuously or periodically analyze patient data (including patient data acquired during patient stay) to determine near real-time or real-time risk scores, mortality predictions, and the like.

[0060] In some embodiments, the server 206 includes one or more modules for performing one or more steps of the patient-specific treatment planning method described herein. For example, in the illustrated embodiment, the server 206 includes a data analysis module 216 and a treatment planning module 218. In other embodiments, one or more of these modules may be combined with each other or omitted. Thus, while some operations relating to a particular module or module are described herein, this is not intended to be limiting, and in other embodiments, such operations may be performed by different modules or modules.

[0061] The data analysis module 216 is configured to include one or more algorithms for identifying a subset of reference data from the database 210 that is likely to be useful in developing patient-specific treatment plans. For example, the data analysis module 216 can compare patient-specific data (e.g., patient dataset 208 received from client computer device 202) with reference data from the database 210 (e.g., reference patient dataset) to identify similar data (e.g., one or more similar patient datasets within the reference patient dataset). This comparison can be based on one or more parameters such as age, sex, BMI, lumbar lordosis, pelvic angle of incidence, and / or treatment level. These (one or more) parameters can be used to calculate a similarity score for each reference patient. The similarity score can represent a statistical correlation between the patient dataset 208 and the reference patient dataset. Thus, similar patients can be identified based on whether the similarity score is above, below, or equal to a specified threshold. For example, as will be described in more detail below, this comparison can be performed by assigning values ​​to each parameter to determine the aggregate difference between the target patient and each reference patient. Reference patients whose total difference is below the threshold can be considered similar patients.

[0062] The data analysis module 216 may be further configured to include one or more algorithms for selecting a subset of reference patient datasets based on, for example, similarity to patient dataset 208 and / or treatment outcomes of corresponding reference patients. For example, the data analysis module 216 may identify one or more similar patient datasets within the reference patient dataset and then select a subset of similar patient datasets based on whether the similar patient datasets contain data indicating favorable or desired treatment outcomes. Outcome data may include data representing one or more outcome parameters, such as corrected anatomical metrics, presence of fusion, HRQL, activity level, complications, recovery time, efficacy, mortality, or follow-up surgery. In some embodiments, as will be described in more detail below, the data analysis module 216 calculates an outcome score by assigning a value to each outcome parameter. A patient may be considered to have a good outcome if the outcome score is above, below, or equal to a specified threshold.

[0063] In some embodiments, the data analysis module 216 selects a subset of reference patient datasets based at least in part on user input (e.g., from a clinician, surgeon, physician, or healthcare provider). For example, user input can be used to identify similar patient datasets. In some embodiments, a healthcare provider or physician can select weights for similarity and / or outcome parameters to adjust similarity and / or outcome scores based on clinician input. In further embodiments, a healthcare provider or physician can select (or define) sets of similarity and / or outcome parameters to be used to generate similarity and / or outcome scores.

[0064] In some embodiments, the data analysis module 216 includes one or more algorithms used to select a set or subset of reference patient datasets based on criteria other than patient parameters. For example, these one or more algorithms can be used to select subsets based on healthcare provider parameters (e.g., healthcare provider rankings / scores such as hospital / physician expertise, number of surgeries performed, hospital rankings), and / or medical resource parameters (e.g., surgical equipment such as diagnostic equipment, facilities, surgical robots), or other non-patient-related information that can be used to predict outcomes and risk profiles for current healthcare provider procedures. For example, reference patient datasets containing images taken from similar diagnostic equipment can be aggregated to reduce or limit irregularities resulting from variability between diagnostic equipment. Alternatively, data from similar healthcare providers (e.g., healthcare providers with traditionally similar outcomes, physician expertise, surgical teams, etc.) can be used to develop patient-specific treatment plans for a particular healthcare provider. In some embodiments, reference healthcare provider datasets, hospital datasets, physician datasets, surgical team datasets, post-treatment datasets, and other datasets can be utilized. For example, a patient-specific treatment plan for battlefield surgery can be based on reference patient data from similar battlefield surgeries and / or datasets related to battlefield surgery. In another example, a patient-specific treatment plan can be generated based on the available robotic surgery system. A reference patient dataset can be selected based on patients who have undergone surgery using a comparable robotic surgery system under similar conditions (e.g., size and capabilities of the surgical team, hospital resources, etc.).

[0065] The treatment planning module 218 is configured to include one or more algorithms for generating at least one treatment plan (e.g., a preoperative plan, a surgical plan, a postoperative plan, etc.) based on the output from the data analysis module 216. In some embodiments, the treatment planning module 218 is configured to build and / or implement at least one predictive model for generating patient-specific treatment plans, also called “prescriptive models.” Predictive models (single or multiple) can be built using clinical knowledge, statistics, machine learning, artificial intelligence (AI), or neural networks, etc. In some embodiments, the output from the data analysis module 216 is analyzed (e.g., using statistics, machine learning, neural networks, or AI) to identify correlations between datasets, patient parameters, healthcare provider parameters, medical resource parameters, treatment procedures, medical device designs, and / or treatment outcomes. These correlations can be used to build at least one predictive model that predicts the likelihood that a given treatment plan will yield favorable outcomes for a particular patient. Predictive models (single or multiple) can be validated, for example, by inputting data into the models and comparing the model’s output to the predicted output.

[0066] In some embodiments, the treatment planning module 218 is configured to generate a treatment plan based on previous treatment data from a reference patient. For example, the treatment planning module 218 can receive a selected subset of a reference patient dataset and / or similar patient datasets from the data analysis module 216 and determine or identify treatment data from this selected subset. Treatment data may include, for example, treatment procedure data (e.g., surgical procedure or intervention data) and / or medical device design data (e.g., implant design data) related to the preferred or desired treatment outcome of the corresponding patient. The treatment planning module 218 can analyze the treatment procedure data and / or medical device design data to determine the optimal treatment protocol for the patient to be treated. For example, values ​​can be assigned to the treatment procedures and / or medical device designs, and these can be aggregated to generate a treatment score. Patient-specific treatment plans can be determined by selecting (single or multiple) treatment plans based on scores (e.g., higher or highest score, lower or lowest score, score above, below, or equivalent to a specified threshold). Personalized patient-specific treatment plans may be based at least in part on patient-specific techniques or patient-specific selected techniques. In some embodiments, the treatment planning module 218 is configured to generate a treatment plan (e.g., to produce medical device design data) based on spinal-pelvic parameters measured by the image processing module from aligned 3D image data of aligned vertebrae (e.g., lumbar and sacral vertebrae) with corresponding anatomical features from 2D image data.

[0067] Separately or in combination with this, the treatment planning module 218 can also generate treatment plans based on correlations between datasets. For example, the treatment planning module 218 can correlate treatment procedure data and / or medical device design data from similar patients with good outcomes (identified, for example, by the data analysis module 216). Correlation analysis may include converting correlation coefficient values ​​into values ​​or scores. These values / scores can be aggregated, filtered, or otherwise analyzed to determine a statistical significance of one or more. These correlations can be used to determine (single or multiple) treatment procedures and / or (single or multiple) medical device designs that are likely to produce the best or most favorable outcomes for the patient to be treated.

[0068] Separately or in combination with this, the treatment planning module 218 can also generate treatment plans using one or more AI technologies. Using AI technologies, it is possible to develop computer systems that can simulate aspects of human intelligence such as learning, reasoning, planning, problem solving, and decision-making. AI technologies include, but are not limited to, case-based reasoning, rule-based systems, artificial neural networks, decision trees, support vector machines, regression analysis, Bayesian networks (e.g., naive Bayes classifiers), genetic algorithms, cellular automata, fuzzy logic systems, multi-agent systems, swarm intelligence, data mining, machine learning (e.g., supervised learning, unsupervised learning, reinforcement learning), and hybrid systems.

[0069] In some embodiments, the treatment planning module 218 generates a treatment plan using one or more trained machine learning models. Various types of machine learning models, algorithms, and techniques are suitable for use with this technique. In some embodiments, the machine learning model is first trained on a training dataset, which is a set of examples used to fit the model's parameters (e.g., the weights of connections between "neurons" in an artificial neural network). For example, the training dataset may include any of the reference data stored in the database 210, such as multiple reference patient datasets or a selected subset thereof (e.g., multiple similar patient datasets).

[0070] In some embodiments, a supervised learning method (e.g., gradient descent or stochastic gradient descent) can be used to train a machine learning model (e.g., a neural network or a Naive Bayes classifier) ​​on a training dataset. The training dataset can include pairs of generated "input vectors" and their corresponding "answer vectors" (commonly called targets). The current model is run on the training dataset to produce results, which are then compared to the targets for each input vector in the training dataset. The model parameters are adjusted based on the results of the comparison and the specific learning algorithm being used. Model fitting can include both variable selection and parameter estimation. The fitted model can be used to predict the response for observation on a second dataset called a validation dataset. The validation dataset can provide an unbiased evaluation of the model fit on the training dataset while tuning the model parameters. The validation dataset can be used for regularization by early stopping, for example, by stopping training when errors on the validation dataset increase, which may be an indication of overfitting on the training dataset. In some embodiments, the error in the validation dataset may fluctuate during training, and therefore an ad-hoc rule can be used to determine when overfitting actually began. Finally, the test dataset can be used to fairly evaluate the final model fit to the training dataset.

[0071] To generate treatment plans, a trained machine learning model can be input with a patient dataset 208. The trained machine learning model can also be input with additional data, such as a reference patient dataset and / or a selected subset of similar patient datasets, and / or treatment data from the selected subset. As a result, the (single or multiple) trained machine learning models can calculate whether various treatment procedure candidates and / or medical device design candidates are likely to yield favorable outcomes for the patient. Based on these calculations, the (single or multiple) trained machine learning models can select at least one treatment plan for the patient. In embodiments using multiple trained machine learning models, the models can be run sequentially or concurrently to compare results, and the models can be periodically updated using the training dataset. The treatment plan module 218 can use one or more of these machine learning models based on their predicted accuracy scores.

[0072] The patient-specific treatment plan generated by the treatment planning module 218 may include at least one patient-specific treatment procedure (e.g., a surgical procedure or intervention) and / or at least one patient-specific medical device (e.g., an implant or implant delivery device). The patient-specific treatment plan may include the entire surgical procedure or a part thereof. Furthermore, one or more patient-specific medical devices may be specially selected or designed to match the corresponding surgical procedure, so that the patient can be treated using a combination of various components of the patient-specific technology.

[0073] In some embodiments, patient-specific treatment procedures include orthopedic surgical procedures such as spinal surgery, hip surgery, knee surgery, temporomandibular joint surgery, wrist surgery, shoulder surgery, elbow surgery, total joint reconstruction (arthroplasty), skull reconstruction, and ankle surgery. Spinal surgery may include spinal fusion procedures such as posterior lumbar interbody fusion (PLIF), anterior lumbar interbody fusion (ALIF), transverse or transvertebral interbody fusion (TLIF), lateral lumbar interbody fusion (LLIF), direct lateral lumbar interbody fusion (DLIF), or extreme lateral lumbar interbody fusion (XLIF). In some embodiments, patient-specific treatment procedures include instructions and / or commands for performing one or more aspects of a patient-specific surgical procedure. For example, patient-specific surgical procedures may include one or more of surgical approaches, corrective operations, osteotomy, or implant placement.

[0074] In some embodiments, patient-specific medical device design includes the design of orthopedic implants and / or instruments for delivering orthopedic implants. Examples of such implants include, but are not limited to, screws (e.g., bone screws, spinal screws, pedicle screws, facet screws), intervertebral implant devices (e.g., intervertebral implants), cages, plates, rods, discs, fusion devices, spacers, rods, expandable devices, stents, brackets, ties, scaffolds, fixation devices, anchors, nuts, bolts, rivets, connectors, tethers, fasteners, artificial joints, or hip implants. Examples of instruments include, but are not limited to, screw guides, cannulas, ports, catheters, or insertion tools.

[0075] A patient-specific medical device design may include data representing one or more of the following physical properties of the corresponding medical device: (e.g., size, shape, volume, material, mass, weight), mechanical properties (e.g., stiffness, strength, modulus of elasticity, hardness), and / or biological properties (e.g., bone integration, cell adhesion, antibacterial properties, antiviral properties). For example, the design of an orthopedic implant may include the shape, size, material, and / or effective stiffness of the implant (e.g., lattice density, number of struts, strut position, etc.). In some embodiments, the generated patient-specific medical device design is a design for the entire device. Alternatively, the generated design may be for one or more components of the device rather than the entire device.

[0076] In some embodiments, this design is for one or more patient-specific device components that can be used with standard off-the-shelf components. For example, in spinal surgery, a pedicle screw kit may include both standard components and patient-specific, customized components. In some embodiments, the resulting design is for a patient-specific medical device that can be used with standard off-the-shelf delivery devices. For example, implants (screws, screw holders, rods, etc.) can be designed and manufactured for the patient, and the implant delivery device can be a standard device. This approach allows for the design and manufacture of implantable components based on the patient's biostructure and / or the surgeon's preference to enhance treatment. The patient-specific devices described herein are expected to improve delivery into the patient's body, placement at the treatment site, and / or interaction with the patient's biostructure.

[0077] In embodiments where the treatment plan includes surgical procedures to implant medical devices into a patient-specific treatment plan, the treatment plan module 218 can also store various types of implant surgery information, such as implant parameters (e.g., type, dimensions), implant availability, preoperative planning aspects (e.g., initial implant configuration, detection and measurement of the patient's biostructure, etc.), and FDA requirements for the implant (e.g., specific implant parameters and / or characteristics that comply with FDA regulations). In some embodiments, the treatment plan module 218 can convert the implant surgery information into a format usable by machine learning-based models and algorithms. For example, the implant surgery information can be tagged with specific identifiers for mathematical formulas, or this information can be converted into a numerical representation suitable for feeding into (single or multiple) trained machine learning models. The treatment plan module 218 can also store information about the patient's biostructure, such as two-dimensional or three-dimensional images or models of the biostructure, and / or information about the biological, geometric, and / or mechanical properties of the biostructure. This anatomical information can be used to inform the design and / or placement of the implant.

[0078] The treatment plan generated by the treatment planning module 218 can be transmitted to a client computer device 202 via a communication network 204 for output to a user (e.g., a clinician, surgeon, healthcare provider, or patient). In some embodiments, the client computer device 202 includes, or is operably coupled to, a display 222 for outputting (one or multiple) treatment plans. The display 222 may include a GUI for visually depicting various aspects of the (one or multiple) treatment plan. For example, the display 222 may show various aspects of the surgical procedure performed on the patient, such as the surgical approach, treatment level, orthodontic operation, tissue resection, and / or implant placement. To facilitate visualization, a virtual model of the surgical procedure may be displayed. As another example, the display 222 may show the design of a device, such as a two-dimensional or three-dimensional model of the medical device design to be implanted in the patient. The display 222 may also show patient information, such as a two-dimensional or three-dimensional image or model of the patient's biostructure to which the surgical procedure should be performed and / or the device should be implanted. The client computer device 202 may further include one or more user input devices (not shown) that enable the user to modify, select, approve, and / or reject the displayed (single or multiple) treatment plans.

[0079] In some embodiments, the (single or multiple) medical device designs generated by the treatment planning module 218 can be transmitted from the client computer device 202 and / or server 206 to a manufacturing system 224 for manufacturing the corresponding medical devices. The manufacturing system 224 can be located on-site or off-site. On-site manufacturing can reduce the number of sessions with the patient and / or the waiting time required before performing surgery, while off-site manufacturing facilities can have specialized manufacturing equipment, making off-site manufacturing useful for manufacturing complex devices. In some embodiments, complex device components can be manufactured off-site, and simple device components can be manufactured on-site.

[0080] Various types of manufacturing systems are suitable for use according to the embodiments of this specification. For example, manufacturing system 224 can be configured for additive manufacturing such as three-dimensional (3D) printing, stereolithography (SLA), digital light processing (DLP), fused deposition modeling (FDM), selective laser sintering (SLS), selective laser melting (SLM), selective thermal sintering (SHM), electron beam melting (EBM), additive manufacturing (LOM), powder bed printing (PP), thermoplastic printing, direct material deposition (DMD), inkjet photoresist printing, or a combination thereof. Separately or in combination therewith, manufacturing system 224 can also be configured for (conventional) subtractive manufacturing such as CNC machining, electrical discharge machining (EDM), grinding, laser cutting, waterjet machining, manual machining (e.g., milling, turning), or similar techniques, or a combination thereof. The manufacturing system 224 can manufacture one or more patient-specific medical devices based on manufacturing instructions or data (e.g., CAD data, 3D data, digital blueprints, stereolithography data, or other data suitable for the various manufacturing techniques described herein). Different components of system 200 can generate at least some of the manufacturing data used by the manufacturing system 224. Manufacturing data may include, but is not limited to, manufacturing instructions (e.g., programs executable by additive manufacturing equipment, subtractive manufacturing equipment, etc.), 3D data, CAD data (e.g., CAD files), computer-aided manufacturing (CAM) data (e.g., CAM files), path data (e.g., printhead paths, toolpaths, etc.), material data, tolerance data, surface finish data (e.g., surface roughness data), or regulatory data (e.g., FDA requirements, reimbursement data, etc.). The manufacturing system 224 can analyze the manufacturability of an implant design based on the received manufacturing data. The implant design can be finalized by generating a manufacturing instruction after modifying the shape, surface, etc. In some embodiments, the server 206 generates at least a portion of the manufacturing data, which is then transmitted to the manufacturing system 224.

[0081] The manufacturing system 224 can generate CAM data, printing data (e.g., powder bed printing data, thermoplastic printing data, photoresin data, etc.), and may include additive manufacturing equipment, subtractive manufacturing equipment, or heat treatment equipment. Additive manufacturing equipment may be 3D printers, stereolithography equipment, digital light processing equipment, fused deposition modeling equipment, selective laser sintering equipment, selective laser melting equipment, electron beam melting equipment, additive manufacturing equipment, powder bed printers, thermoplastic printers, direct material deposition equipment, inkjet photoresin printers, or similar technologies. Subtractive manufacturing equipment may be CNC machines, electrical discharge machines, grinders, laser cutters, water jet machines, manual machines (milling machines, lathes, etc.), or similar technologies. Both additive and subtractive technologies can be used to manufacture implants with complex shapes, surface finishes, material properties, etc. The generated manufacturing instructions can be configured to cause the manufacturing system 224 to manufacture patient-specific orthopedic implants that match or are therapeutically identical to a patient-specific design. In some embodiments, patient-specific medical devices may include features, materials, and designs that are shared between designs to simplify manufacturing. For example, deployable patient-specific medical devices for different patients may have similar internal deployment mechanisms but different deployment configurations. In some embodiments, components of a patient-specific medical device may be selected from a set of available pre-fabricated components, and these selected pre-fabricated components may be modified based on manufacturing orders or data.

[0082] The treatment plans described herein can be performed by a surgeon, a surgical robot, or a combination thereof, thus allowing for treatment flexibility. In some embodiments, the surgical procedure can be performed entirely by a surgeon, entirely by a surgical robot, or a combination thereof. For example, one stage of the surgical procedure can be performed manually by a surgeon, and another stage of the procedure can be performed by a surgical robot. In some embodiments, the treatment planning module 218 generates control commands configured to cause a surgical robot (e.g., a robotic surgical system, a navigation system, etc.) to perform the surgical procedure partially or entirely. These control commands can be transmitted by a client computer device 202 and / or a server 206 to the robotic device.

[0083] After treating a patient according to a treatment plan, the treatment progress can be monitored for one or more periods to update the data analysis module 216 and / or the treatment planning module 218. Post-treatment data can be added to the reference data stored in the database 210. The post-treatment data can be used to train machine learning models to develop patient-specific treatment plans, patient-specific medical devices, or combinations thereof.

[0084] It should be understood that the components of system 200 can be configured in many different ways. For example, in another embodiment, the database 210, data analysis module 216, and / or treatment planning module 218 may be components of client computer device 202 rather than server 206. As another example, the database 210, data analysis module 216, and / or treatment planning module 218 may also be deployed across multiple different servers, computer systems, or other types of cloud computing resources rather than a single server 206 or client computer device 202.

[0085] Furthermore, in some embodiments, System 200 can operate with a number of other computer system environments or configurations. Examples of computer systems, environments, and / or configurations suitable for use with this technology include, but are not limited to, personal computers, server computers, handheld or laptop devices, mobile phones, wearable electronic devices, tablet devices, multiprocessor systems, microprocessor-based systems, programmable home appliances, network PCs, minicomputers, mainframe computers, or distributed computing environments including any of these systems or devices.

[0086] Figure 3 shows a computer device 300 suitable for use in connection with the system 200 of Figure 2, according to one embodiment. The computer device 300 can be incorporated into various components of the system 200 of Figure 2, such as a client computer device 202 or a server 206. The computer device 300 includes one or more processors 310 (e.g., a (single or duplicate) CPU, a (single or duplicate) GPU, a (single or duplicate) HPU, etc.). The (single or duplicate) processors 310 can be a single processing unit or multiple processing units, either within one device or distributed across multiple devices. The (single or duplicate) processors 310 can be coupled to other hardware devices using a bus, such as a PCI bus or a SCSI bus. The (single or duplicate) processors 310 can be configured to execute one or more computer-readable program instructions, such as program instructions that perform any of the methods described herein.

[0087] The computer device 300 may include one or more input devices 320 that provide input to one or more processors 310 that notify the computer device 300 of actions from a user. These actions can be performed via a hardware controller that interprets signals received from the input devices and transmits the information to the processors 310 using a communication protocol. The input devices 320 may include, for example, a mouse, keyboard, touch screen, infrared sensor, touchpad, wearable input device, camera or image-based input device, microphone, or other user input device.

[0088] The computer device 300 may include a display 330 used to display various types of output, such as text, models, virtual procedures, surgical plans, implants, graphics, and / or images (for example, images containing voxels showing radiation density units or Hounsfield units representing tissue density at a given location). In some embodiments, the display 330 provides the user with graphical and textual visual feedback. A (one or more) processor 310 can communicate with the display 330 via the device's hardware controller. In some embodiments, the display 330 includes the (one or more) input devices 320 as part of it, such as when the (one or more) input devices 320 include a touch screen or are equipped with a gaze direction monitoring system. In another embodiment, the display 330 is separated from the (one or more) input devices 320. Examples of display devices include LCD display screens, LED display screens, projection displays, holographic displays, or augmented reality displays (for example, head-up display devices or head-mounted devices).

[0089] Optionally, the (single or duplicate) processors 310 may also be coupled with other input / output (I / O) devices 340, such as network cards, video cards, audio cards, USB, FireWire or other external devices, cameras, printers, speakers, CD-ROM drives, DVD drives, disc drives, or Blu-ray devices. The other I / O devices 340 may also include input ports for information from directly connected medical equipment, such as imaging devices including MRI machines, X-ray machines, and CT scanners. The other I / O devices 340 may further include input ports for receiving data from these types of machines from other sources, for example, via a network, or from previously captured data stored in a database or the like.

[0090] In some embodiments, the computer device 300 also includes a communication device (not shown) capable of communicating wirelessly or via a wired connection with network nodes. The communication device can communicate with other devices or servers over the network, for example, using the TCP / IP protocol. The computer device 300 can use the communication device to distribute its operations across multiple network devices, including imaging equipment, manufacturing equipment, and so on.

[0091] The computer device 300 may include memory 350, which may reside within a single device or be distributed across multiple devices. Memory 350 may include one or more hardware devices for volatile and non-volatile storage, and may include both read-only and writable memory. For example, memory may include random access memory (RAM), various caches, CPU registers, read-only memory (ROM), and writable non-volatile memory such as flash memory, hard drives, floppy disks, CDs, DVDs, magnetic storage devices, tape drives, and device buffers. Memory is not a propagating signal independent of the underlying hardware and is therefore non-temporary. In some embodiments, memory 350 is a non-temporary computer-readable storage medium that stores, for example, programs, software, or data. In some embodiments, memory 350 may include program memory 360 that stores programs and software such as an operating system 362, one or more image processing modules 364, and other application programs 366. The image processing module 364 may include one or more modules configured to perform various methods described herein (for example, the data analysis module 216 and / or treatment planning module 218 described with respect to Figure 2). The memory 350 may also include a data memory 370 which may include reference data, configuration data, settings, user options or preferences, etc., which can be provided to, for example, the program memory 360 or any other element of the computer device 300.

[0092] Figure 4A shows an example flowchart 400 of the process of mapping a portion of a human anatomical structure in 3D image analysis to a corresponding portion of the same human anatomical structure in a 2D image. Process 402 includes acquiring three-dimensional (3D) image data of an anatomical region that includes a 3D representation of at least a portion of the spine of a patient in a non-loading posture. Process 404 includes determining multiple regions from the 3D image data that correspond to multiple segments or multiple anatomical elements of the spine, the multiple segments or multiple anatomical elements including the sacrum and multiple vertebrae. Process 406 includes identifying a first set of landmarks for the spine segments or anatomical elements in each of the multiple regions. Process 408 includes acquiring two-dimensional (2D) image data of the anatomical region that includes the patient's spine. Process 410 includes identifying a second set of landmarks on the sacrum and multiple vertebrae from the 2D image data, these landmarks should be common between the 2D and 3D imaging. Operation 412 includes obtaining aligned 3D image data of the sacrum and several vertebrae by aligning the sacrum and each vertebra between 3D image data and 2D image data, wherein the aligned 3D image data includes a 3D representation of the sacrum and several lumbar vertebrae in at least a loaded or weight-bearing posture, and the alignment of each lumbar vertebra is performed based on a first landmark set and a second landmark set. Operation 414 includes displaying the aligned 3D image data in a graphical user interface (GUI). The operations described in flowchart 400 can be performed by the image processing module described in this patent document.

[0093] In some embodiments, identifying a first set of landmarks of a vertebral segment or anatomical element in response to the segment or anatomical element being a lumbar vertebra includes identifying a first set of information that includes identifying the inferior surface of the lumbar spinous process, the superior surface of the lumbar spinous process, the left surface of the lumbar transverse process, and the right surface of the lumbar transverse process. In some embodiments, the first set of information is identified in response to receiving an instruction to select a fast mapping option from the GUI. In some embodiments, identifying a first set of landmarks of a vertebral segment or anatomical element includes identifying a second set of information that includes identifying anterior landmarks of the vertebra or vertebral endplate, posterior landmarks of the vertebra or vertebral endplate, lateral landmarks of the vertebra or vertebral endplate, and lateral landmarks of the vertebra or vertebral endplate.

[0094] In some embodiments, a first and second set of information are identified in response to receiving instructions from a GUI to select precise mapping options. In some embodiments, each of the first set of landmarks of a segment or vertebral anatomical element is associated with a 3D location in 3D space of 3D image data. In some embodiments, a second set of landmarks of the sacrum and multiple lumbar vertebrae is identified from 2D image data including sagittal and coronal views of the anatomical region. In some embodiments, identifying the second set of landmarks of the vertebrae includes identifying one or more of the following: the inferior surface of the lumbar spinous process, the superior surface of the lumbar spinous process, the left surface of the lumbar transverse process, the right surface of the lumbar transverse process, anterior landmarks of the spinal or vertebral endplate, posterior landmarks of the spinal or vertebral endplate, lateral landmarks of the spinal or vertebral endplate, and lateral landmarks of the spinal or vertebral endplate.

[0095] In some embodiments, each of the first and second landmarks of the spine is associated with a 2D position in 2D space of the 2D image data. In some embodiments, aligning the sacrum between 3D and 2D image data is performed by forming a first image data from the 3D image data along a sagittal view that bisects the sacrum, projecting a second image data containing the sacrum along a sagittal view from the 2D image data onto the first image data, and scaling, rotating, and / or translating the first image data so that the sacrum from the first image data overlaps with the sacrum in the second image data.

[0096] In some embodiments, aligning each lumbar vertebra between 3D and 2D image data is performed in response to aligning the sacrum by performing a first lumbar vertebral alignment operation by moving the anterior and posterior landmarks of the lumbar vertebrae in the 3D image data to coincide with the same landmarks on a second image data containing the lumbar vertebrae from the 2D image data; forming a third image data from the 3D image data along a coronal view extending across the sacrum; projecting a fourth image data containing the sacrum along the coronal view from the 2D image data onto the third image data; scaling or translating the first image data so that the sacrum from the third image data overlaps with the sacrum in the second image data; and performing a second lumbar vertebral alignment operation by moving the anterior and posterior landmarks of the lumbar vertebrae to coincide with the same landmarks on the fourth image data containing the lumbar vertebrae from the 2D image.

[0097] In some embodiments, the method further includes adjusting one or more other landmarks of the lumbar vertebrae by the same amount used to move the anterior and posterior landmarks of the lumbar vertebrae in the 3D image data for a first alignment operation, and adjusting one or more other landmarks of the lumbar vertebrae by the same amount used to move the anterior and posterior landmarks of the lumbar vertebrae in the 3D image data for a second alignment operation. In some embodiments, once multiple regions are determined, the positions of multiple segments of the vertebrae or multiple anatomical elements in the 3D image data are maintained. In some embodiments, the 3D image data is obtained from computed tomography (CT) scans or magnetic resonance imaging (MRI) scans. In some embodiments, the method further includes generating a design for a medical implant based on spinal-pelvic parameters measured from the aligned 3D image data, and having the medical implant manufactured by transmitting the design for the medical implant to a manufacturing apparatus.

[0098] Figure 4B shows an example flowchart 450 of the operation of mapping a portion of a human body structure in multidimensional image data to a corresponding portion of the same human body structure in another multidimensional image data. Flowchart 450 can be used to map common landmarks (e.g., anterior-posterior (AP) direction and lateral direction) from two 2D images to a 3D image for each of at least three landmarks per vertebra. For each vertebra, each of the one or more 3D landmarks includes three coordinates (e.g., x, y, z), each of the one or more 2D landmarks in the first view (e.g., AP view) has two coordinates (e.g., x, y), and each of the one or more 2D landmarks in the second view (e.g., lateral view) has two coordinates (e.g., y, z).

[0099] Operation 452 includes acquiring a first multidimensional image data of an anatomical region of a patient in a first loading state. In some embodiments, the first loading state may be a horizontal (or unloaded) posture. Operation 454 includes determining multiple regions from the first multidimensional image data that correspond to multiple anatomical elements of the spine. Operation 456 includes identifying a first set of landmarks in each of the multiple regions. Operation 458 includes acquiring a second multidimensional image data of an anatomical region including the spine of a patient in a second loading state. In some embodiments, the second loading state may be a vertical (or loaded or load-bearing) posture. Operation 460 includes identifying a second set of landmarks corresponding to the first set of landmarks from the second multidimensional image data. Operation 462 includes acquiring aligned multidimensional image data of the patient's spine by aligning the corresponding anatomical elements between the first and second multidimensional image data. Operation 464 includes generating a design for a medical implant based on spinal-pelvic parameters measured from aligned multidimensional image data. Operation 466 includes transmitting the design for the medical implant to a manufacturing apparatus so that the medical implant may be manufactured.

[0100] The operation of flowchart 450 can be performed by an image processing module using the technical examples described in this patent document. In some embodiments, a first multidimensional image data is acquired in three dimensions, and a second multidimensional image data is acquired in two dimensions. In some embodiments, the aligned multidimensional image data is a three-dimensional image. In some embodiments, acquiring the aligned multidimensional image data includes correcting the positions of multiple anatomical elements in the first multidimensional image data according to the positions of multiple anatomical elements in the second multidimensional image data. In some embodiments, the method further includes determining a load state mapping based on first and second load states, and performing a load state mapping between the first multidimensional image data and the second multidimensional image data to generate aligned multidimensional image data.

[0101] In some embodiments, identifying a first set of landmarks of a region in response to the region being the lumbar vertebrae includes identifying the inferior surface of the lumbar spinous process, the superior surface of the lumbar spinous process, the left surface of the lumbar transverse process, and the right surface of the lumbar transverse process. In some embodiments, aligning corresponding anatomical elements between a first multidimensional image data and a second multidimensional image data includes forming a first image data from 3D image data along a sagittal view that bisects the sacrum, projecting a second image data containing the sacrum along a sagittal view from 2D image data onto the first image data, and aligning the sacrum between 3D and 2D image data by scaling, rotating, and / or translating the first image data so that the sacrum from the first image data overlaps with the sacrum in the second image data.

[0102] In some embodiments, aligning the corresponding anatomical elements includes aligning each lumbar vertebra between 3D and 2D image data, which is performed in response to aligning the sacrum by performing a first lumbar vertebral alignment operation by moving the anterior and posterior landmarks of the lumbar vertebrae in 3D image data to coincide with the same landmarks on a second image data containing the lumbar vertebrae from 2D image data; forming a third image data from 3D image data along a coronal view extending across the sacrum; projecting a fourth image data containing the sacrum along the coronal view from 2D image data onto the third image data; scaling or translating the first image data so that the sacrum from the third image data overlaps with the sacrum in the second image data; and performing a second lumbar vertebral alignment operation by moving the anterior and posterior landmarks of the lumbar vertebrae to coincide with the same landmarks on the fourth image data containing the lumbar vertebrae from 2D image data.

[0103] In some embodiments, the method further includes adjusting one or more other landmarks of the lumbar vertebrae by the same amount used to move the anterior and posterior landmarks of the lumbar vertebrae in the 3D image data for a first alignment operation, and adjusting one or more other landmarks of the lumbar vertebrae by the same amount used to move the anterior and posterior landmarks of the lumbar vertebrae in the 3D image data for a second alignment operation. In some embodiments, the first multidimensional image data is obtained from a computed tomography (CT) scan or a magnetic resonance imaging (MRI) scan, and the second multidimensional image data is obtained from an X-ray scan.

[0104] Figure 5 shows an example 2D image 500 of an anterior-posterior (AP) view (or coronal view) of the spine acquired from an X-ray. The 2D image 500 shows the curvature of the spine in the AP view when the spine is in a vertical (or loaded or weight-bearing) position. The 2D image 500 also shows that multiple vertebrae have different 2D positions (e.g., two sets of coordinates such as y and z) in 2D space, and the image processing module can analyze each segment or anatomical element of the vertebrae as described above in this patent document to identify landmarks having 2D coordinates. The image processing module can determine the load state of the spine (e.g., whether the spine is unloaded, weight-bearing, or in another state) based on user input (e.g., the user indicating the load state), the anatomical configuration of the spine, the type of image data (e.g., X-ray, CT scan), machine learning algorithms, metadata, etc. For example, the image processing module can determine that image 500 is a 2D image taken in a weight-bearing state. Subsequently, the image processing module can map features based on the load state and predict the anatomical configuration of the load state.

[0105] Figure 6 shows an example 2D image 600 of a lateral view (or sagittal view) of the spine obtained from radiography. The 2D image 600 shows the curvature of the spine in a lateral view when the spine is in a vertical (or loaded or weight-bearing) position. The 2D image 600 also shows that multiple vertebrae have different 2D positions (e.g., two other sets of coordinates such as x and y) in 2D space, and the image processing module can analyze each segment or anatomical element of the vertebrae as described above in this patent document to identify landmarks having 2D coordinates.

[0106] Figure 7 shows example 2D image 700 of a coronal view of the spinal region acquired from a CT scan. 2D image 700 is shown with focus on the L4-L5 vertebrae. 2D image 700 shows the curvature of the spinal region from a coronal view when the spine is in a horizontal (or non-load-bearing) position. The image processing module can determine that the spine is in a non-load-bearing configuration based on the CT scan taken when the patient is in a non-load-bearing position, user input, or metadata associated with the image. 2D image 700 also shows the intervertebral spaces. Figure 8 shows example 2D image 800 of a coronal view of the spinal region acquired from a CT scan. 2D image 800 is shown with focus on the L5-S1 vertebrae. 2D image 800 shows a coronal view of the curvature of the L5-S1 region of the spine when the spine is in a horizontal (or non-load-bearing) position. The image processing module can analyze 2D images 700, 800 for both Figures 7 and 8 to identify landmarks within the 2D images 700, 800 as described in this patent document. Figure 9 shows an example of a 2D image 900 of a sagittal view of a region of the spine obtained from a CT scan. The 2D image 900 is shown to focus on the lumbar region of the spine. The 2D image 900 shows a sagittal view of the curvature of the spine and the gaps between segments or anatomical elements when the spine is in a horizontal (or unloaded) position.

[0107] Figure 10 shows an example flowchart 1000 of the actions to generate a 3D spine model that simulates the movement and loading of the vertebrae of a patient's spine according to an embodiment. Flowchart 1000 can generate a 3D spine model that can be used to perform simulations of the movement and / or loading of anatomical elements (e.g., vertebral bodies, implants, etc.) at various levels for different loading conditions. For example, loading information (e.g., how the loading affects the anatomical elements) can be determined by comparing images of a biological structure under different loading conditions. Using the determined loading information, a 3D anatomical model representing one or more loading conditions can be generated. Using these actions, it is possible to map parts of human structures in 3D image analysis to corresponding parts of the same human structures in 2D images, as described in relation to Figures 4A and 4B. For example, the steps described in relation to Figures 4A and 4B can be incorporated into or replaced with the steps described in relation to the actions described in relation to Figure 10.

[0108] The 3D spine model in flowchart 1000 may include 3D virtual vertebrae with precise size, orientation, and position in height in acquired imaging (e.g., anterior-posterior (AP) and lateral directions). The 3D spine model can enable healthcare providers (e.g., surgeons, therapists, etc.) or implant designers to perform 3D spine planning (e.g., SVA, CSVL, TK, LL, PI, PT) using patient image input (e.g., X-rays, CT scans, etc.). The 3D spine model can simulate reciprocal mechanisms in response to lumbar correction. For example, the 3D model can be used to predict compensatory responses of pelvic tilt, thoracic kyphosis, or cervical lordosis in response to lumbar correction. Planning global alignment with compensatory curvature increases surgical success rates and improves patient health outcomes. 3D spinal models can enable surgeons, sales representatives, medical decision-makers (MDMs), and patients to verify the target alignment of planned procedures (e.g., SVA, CSVL, TK, PT, etc.). For example, 3D placement of other bio-structures such as the iliac crest and ribs can visually show surgeons how they will access the intervertebral disc space. Planning an L45 LLI using 3D visualization of the iliac crest allows for flagging potential problems, such as difficulties in inserting inserters / implants into the iliac crest.

[0109] Flowchart 1000 can be used to map common landmarks from two 2D images (e.g., AP image and lateral image) to a 3D image for each of at least three landmarks per vertebra. For each vertebra, each of the one or more 3D landmarks includes three coordinates (e.g., x, y, z), each of the one or more 2D landmarks in the first view (e.g., AP view) has two coordinates (e.g., x, y), and each of the one or more 2D landmarks in the second view (e.g., lateral view) has two coordinates (e.g., y, z).

[0110] Operation 1002 includes acquiring first multidimensional image data (e.g., CT scan data) of an anatomical region of a patient in a first loading state. In some embodiments, the first loading state may be when the patient is in a horizontal (or non-load-bearing) position. For example, the patient may be lying down during a CT scan. The system (e.g., image processing module, system 200, image processing module 364, etc.) can determine multiple regions from the first multidimensional image data that correspond to multiple anatomical elements of the spine. Operation 1002 may include performing segmentation by identifying different structures or tissues in the image. This process may be performed manually by the user or automated using one or more segmentation routines. Segmentation routines can be used to extract specific regions of interest, such as vertebrae, vertebral bodies, or intervertebral discs, from the surrounding tissue.

[0111] Operation 1004 includes acquiring a second multidimensional image data (e.g., planar X-ray data, orthogonal contemporaneous X-ray data, or other X-ray data) of an anatomical region including the spine of a patient under a second load condition. The system can identify a second landmark set corresponding to a first landmark set from the second multidimensional image data. Operation 1004 includes acquiring aligned multidimensional image data of the patient's spine by aligning corresponding anatomical elements between the first and second multidimensional image data. In some embodiments, operation 1004 includes acquiring load effect data by comparing the positions of corresponding anatomical elements in the first and second multidimensional image data. The load effect data can be correlated with known loads at the time the first and second multidimensional image data were acquired. The load effect data can be used to predict the positions of anatomical elements under further load conditions.

[0112] Operation 1006 involves mapping the XYZ position coordinates and rotation parameters of vertebral bodies and other biostructures using AP and lateral X-rays. The XYZ position coordinates and rotation parameters of biostructures are determined by capturing the XYZ position coordinates of biostructure landmarks visible in the AP and / or lateral views of the images. These coordinates and parameters are captured with reference to a common origin, such as the posterior edge of S1. The system can use a biostructure sizing module to size a library of vertebral body models to reproduce the approximate size of each vertebra / auxiliary body by circumscribing bounding boxes around the biostructures of interest. The system can then position these sized bodies within the identified XYZ position coordinates and rotation parameters.

[0113] Operation 1008 may include forming a three-dimensional representation of a structure of interest. The three-dimensional reconstruction technique uses segmented data to generate a virtual model that captures the shape, size, and spatial relationships of the structure of interest. The resolution of the acquired images can affect the level of detail and accuracy of the generated three-dimensional representation. Operation 1008 may also include generating a vertebral model by analyzing the vertebrae in the images to identify the size of each vertebra. In some embodiments, the images are segmented to identify features (e.g., boundaries of anatomical elements such as vertebrae). The vertebral boundaries can be determined by analyzing the segmented features from the images. The three-dimensional boundaries of the vertebrae can be determined by combining images taken from different viewpoints. Landmarks can be identified and keyed to other images in order to perform image mapping between different types of images, including images acquired using different equipment (e.g., CT scanners, X-ray machines, MRI machines, etc.) or imaging modalities. The vertebrae can be measured by applying measurement routines to the images and vertebrae.

[0114] Action 1010 includes generating a 3D spine model that simulates vertebral movement and patient loading. The 3D spine model can simulate compensatory mechanisms for lumbar adjustments. For example, the 3D spine model can be used to predict other non-instrumented lumbar levels, pelvic tilt compensatory responses to lumbar adjustments, thoracic compensatory responses to lumbar adjustments, or cervical compensatory responses to lumbar adjustments. The technology for generating 3D models is shown in Figures 1-9, as well as in U.S. Patent Application Publication No. 16 / 569,494, filed on September 12, 2019, entitled "Systems and Methods for Orthopedic Implants," U.S. Patent Application Publication No. 16 / 735,222 (currently U.S. Patent No. 10,902,944), filed on January 6, 2020, entitled "Patient-Specific Medical Procedures and Devices, and Associated Systems and Methods," and in "Systems and Methods of Assisting a Surgeon with Screw Placement During Spinal Surgery," filed on January 8, 2019. This explanation is in relation to U.S. Patent Application Publication No. 16 / 242,877, titled "SURGERY," and these documents are incorporated in their entirety by citation.

[0115] Operation 1012 includes inserting a model of the vertebrae into a 3D spine model according to the XYZ position coordinates and translation, scaling, and / or rotation parameters determined through analysis. Insertion of auxiliary biomolecules can be performed using landmarks visible on AP and lateral images (e.g., iliac crests and ribs). Operation 1012 may include moving or generating an anatomical model for the 3D spine model. For example, vertebrae can be generated based on imaged vertebrae. The generation process can ensure a desired match between anatomical features by considering an existing virtual anatomical model within the 3D spine model. This match can be verified based on measurements of the 3D spine model against measurements obtained using images. In some embodiments, operation 1012 may include moving vertebrae to position them within the 3D spine model. For example, the 3D spine model can be generated at least partially based on images from a first dataset. The new position of the vertebrae can be determined by analyzing a second dataset. For example, the first dataset may represent an unloaded state, while the new position may represent a loaded state. Thus, multiple 3D spine models representing multiple states of a patient can be generated. Furthermore, it is possible to generate multiple load-bearing 3D spine models representing different load conditions, such as when the patient is standing or performing a task.

[0116] Action 1014 includes evaluating compensatory responses (e.g., compensatory curvature and / or intermodulation) resulting from a procedure such as a spinal fusion (e.g., lumbar spinal fusion, cervical spinal fusion, etc.).

[0117] The advantage of performing the operations of flowchart 1000 is that personalized surgical plans are constructed by adding the ability to measure, predict, and display the results to the user alignment (e.g., local alignment such as a user-selected region of interest, global alignment, etc.). The system can predict synthetic / hybrid upright CTs, plan surgical procedures, and simulate the shape of the spine at local and distant locations by determining the position of anatomical features (e.g., intervertebral discs, vertebral bodies, spinous processes, or joints (e.g., surface joints)) while weight is applied to the spine. In some procedures, flowchart 1000 is used to predict alignment based on lumbar fusion. The system can predict the shape of the patient's spine in the thoracic and / or cervical regions, for example. As a result, the user can evaluate predicted changes to different spinal segments that may not be directly modified during surgery. The system can identify far anatomical effects resulting from anatomical adjustments at the surgical site, and can determine one or more relationships between anatomical adjustments and far anatomical effects. In response to determining one or more relationships, the system sends a notification to the user. U.S. Patent Application Publication 63 / 437,975 discloses a method for calculating and displaying the interrelationship changes of neighboring biomolecules resulting from the correction of an index biomolecule. For example, the system can model anatomical changes in the thoracic and cervical spine based on lumbar spine correction. U.S. Patent Application Publication 63 / 437,975 is incorporated in its entirety by reference.

[0118] The operations of flowchart 1000 can be performed by an image processing module. First multidimensional image data is acquired in 3D, and second multidimensional image data is acquired in 2D. The aligned multidimensional image data is a 3D image. In some embodiments, obtaining aligned multidimensional image data includes correcting the positions of multiple anatomical elements in the first multidimensional image data according to the positions of multiple anatomical elements in the second multidimensional image data. In some embodiments, the method further includes determining a load state mapping based on first and second load states, and performing a load state mapping between the first and second multidimensional image data to generate aligned multidimensional image data.

[0119] In some embodiments, identifying a first set of landmarks of a region in response to the region being the lumbar vertebrae includes identifying the inferior surface of the lumbar spinous process, the superior surface of the lumbar spinous process, the left surface of the lumbar transverse process, and the right surface of the lumbar transverse process. In some embodiments, aligning corresponding anatomical elements between a first multidimensional image data and a second multidimensional image data includes forming a first image data from 3D image data along a sagittal view that bisects the sacrum, projecting a second image data containing the sacrum along a sagittal view from 2D image data onto the first image data, and aligning the sacrum between 3D and 2D image data by scaling, rotating, and / or translating the first image data so that the sacrum from the first image data overlaps with the sacrum in the second image data.

[0120] In some embodiments, aligning the corresponding anatomical elements includes aligning each lumbar vertebra between 3D and 2D image data, which is performed in response to aligning the sacrum by performing a first lumbar vertebral alignment operation by moving the anterior and posterior landmarks of the lumbar vertebrae in 3D image data to coincide with the same landmarks on a second image data containing the lumbar vertebrae from 2D image data; forming a third image data from 3D image data along a coronal view extending across the sacrum; projecting a fourth image data containing the sacrum along a coronal view from 2D image data onto the third image data; scaling, rotating, or translating the first image data so that the sacrum from the third image data overlaps with the sacrum in the second image data; and performing a second lumbar vertebral alignment operation by moving the anterior and posterior landmarks of the lumbar vertebrae to coincide with the same landmarks on the fourth image data containing the lumbar vertebrae from 2D image data.

[0121] In some embodiments, the method further includes adjusting one or more other landmarks of the lumbar vertebrae by the same amount used to move the anterior and posterior landmarks of the lumbar vertebrae in the 3D image data for a first alignment operation, and adjusting one or more other landmarks of the lumbar vertebrae by the same amount used to move the anterior and posterior landmarks of the lumbar vertebrae in the 3D image data for a second alignment operation. In some embodiments, the first multidimensional image data is obtained from a computed tomography (CT) scan or a magnetic resonance imaging (MRI) scan, and the second multidimensional image data is obtained from an X-ray examination or a series of X-ray examinations, etc.

[0122] Figure 11 shows an example 2D image 1100 of a view of the spine, including sagittal vertical axis measurements obtained from X-ray imaging.

[0123] Figure 12 shows an exemplary surgical planning report 1200 that can be used and / or generated in connection with the method described herein, according to one embodiment.

[0124] Figure 13 shows an exemplary surgical planning report 1300 that can be used and / or generated in connection with the method described herein, according to one embodiment.

[0125] Figure 14 shows exemplary XYZ and rotational measurements of a patient's biological structure according to one embodiment.

[0126] Figure 15 shows an example flowchart 1500 of an operation to generate a 3D virtual model for simulating a patient's postoperative loading state according to one embodiment. Flowchart 1500 can generate a 3D spinal model that can be used to perform simulations of the movement and / or loading of anatomical elements (e.g., vertebrae, implants, etc.) at various levels for different preoperative and postoperative loading states. For example, loading information (e.g., how the loading affects the anatomical elements or the position and tissue of the anatomical elements) can be determined by comparing images of the biomimetic structure under different loading states. Using the determined loading information, a 3D anatomical model representing one or more loading conditions can be generated. Using these operations, it is possible to map parts of human structures in 3D image analysis to corresponding parts of the same human structures in 2D images, as described in relation to Figures 4A and 4B. For example, the steps described in relation to Figures 4A and 4B can be incorporated into or replaced with the steps described in relation to the operation described in relation to Figure 15.

[0127] Operation 1510 includes receiving first multidimensional image data (e.g., CT scan data) of an anatomical region of a patient in a first loading state (e.g., a spinal region such as the thoracic and / or cervical regions of the patient's spine). Operation 1510 includes acquiring first multidimensional image data of an anatomical region of a patient in a first loading state. Based on the resolution level of the first multidimensional image data, the system can model virtual lumbar anatomical elements that correspond to the anatomical elements shown in the first multidimensional image data. In some embodiments, the first loading state may be when the patient is in a horizontal (or unloaded) position. For example, the patient may be lying down during a CT or MRI scan. The system (e.g., image processing module, system 200, image processing module 364, etc.) can determine from the first multidimensional image data multiple regions corresponding to multiple anatomical elements of the spine. Operation 1510 may include performing segmentation by identifying different structures or tissues in the image. This process can be performed manually by the user or automated using one or more segmentation routines. Segmentation routines can be used to extract specific regions of interest, such as vertebrae, vertebral bodies, and intervertebral discs, from the surrounding tissue.

[0128] Operation 1520 includes receiving a second multidimensional image data (e.g., X-ray data) of at least a portion of an anatomical region (e.g., a spinal region such as the lumbar region of the patient's spine) under a second load condition. Based on the resolution level of the second multidimensional image data, the system can model virtual anatomical elements of the thoracic and cervical regions that correspond to the anatomical elements shown in the second multidimensional image data. The system can identify a second set of landmarks from the second multidimensional image data that corresponds to a first set of landmarks. Operation 1520 includes obtaining aligned multidimensional image data of the patient's spine by aligning the corresponding anatomical elements between the first and second multidimensional image data. In some embodiments, operation 1520 includes obtaining load effect data by comparing the positions of the corresponding anatomical elements in the first and second multidimensional image data. The load effect data can be correlated with known loads at the time the first and second multidimensional image data were acquired. The load effect data can be used to predict the positions of anatomical elements under further load conditions.

[0129] Operation 1530 includes determining one or more loading effects on at least one anatomical element in an anatomical region based on first and second multidimensional image data. The system can identify a first image type (e.g., CT scan) of the first multidimensional image data and determine a first loading state based on the first image type of the first multidimensional image data. The system can determine one or more anatomical metrics affected by the first loading state and measure one or more anatomical metrics in an anatomical region using the first multidimensional image data. In some embodiments, the system generates a 3D model of the patient's spine using the measured one or more anatomical metrics. The system can select a mapping routine based on a first image type (e.g., CT scan), a second image type (e.g., X-ray), and / or anatomical elements in an anatomical region. The system can configure the mapping routine to map anatomical features in a CT scan to the same anatomical features in an X-ray and to determine one or more loading effects in an anatomical region based on the positional differences of the same anatomical features due to loading during the CT scan and X-ray.

[0130] Operation 1540 includes generating a 3D virtual model of the patient (e.g., a 3D multi-domain simulation model) to simulate the postoperative load state based on one or more load effects correlated with the preoperative load state. The technology for generating 3D models is shown in Figures 1-9, as well as in U.S. Patent Application Publication No. 16 / 569,494, filed on September 12, 2019, entitled "Systems and Methods for Orthopedic Implants," U.S. Patent Application Publication No. 16 / 735,222 (currently U.S. Patent No. 10,902,944), filed on January 6, 2020, entitled "Patient-Specific Medical Procedures and Devices, and Associated Systems and Methods," and in "Systems and Methods of Assisting a Surgeon with Screw Placement During Spinal Surgery," filed on January 8, 2019. This description is in reference to U.S. Patent Application Publication No. 16 / 242,877, entitled “SURGERY,” and these documents are incorporated in their entirety by citation. The 3D model may include a first set of anatomical element models having high-fidelity surface topology for designing one or more patient-specific implants. The 3D model may include a second set of anatomical element models for measuring spinal metrics that predict surgical outcomes associated with the implantation of one or more patient-specific implants. In some cases, one or more of the anatomical element models in the second set have less feature data than all or some of the anatomical element models in the first set. The system can generate the 3D model by positioning, oriented, and / or scaling the anatomical element models of the first set to be incorporated into the second set of anatomical element models.

[0131] The system can generate X-ray fidelity anatomical element models based on one or more upright X-ray images of a second multidimensional image data set. The system can also generate tomographic fidelity anatomical element models based on a set of tomographic images of a first multidimensional image data set, including tomographic fidelity (e.g., polygon count of the number of polygons or triangles used to represent the surface of the model, edge and curve smoothness, surface quality (surface roughness, continuity, and the presence of imperfections or artifacts that may affect the fidelity of the model), geometric detail level (including fine features, complex structures, and smaller elements), and matching with the image). X-ray fidelity anatomical element models have a resolution lower than threshold fidelity, while tomographic fidelity anatomical element models have a fidelity higher than threshold fidelity, a resolution higher than threshold resolution, etc.

[0132] The system can generate a 3D multi-region spine model of a patient (including, for example, the cervical and thoracic regions of the patient's spine) based on a first multi-dimensional image data and a 3D partial spine model that matches the corresponding region of the imaged spine in the second multi-dimensional image data (for example, including a lumbar region model with surface topology data for designing one or more implants). The system can generate a 3D multi-region simulation model by combining the anatomical elements of the 3D partial spine model with the 3D multi-region spine model. The system can replace low-fidelity anatomical element models (for example, based on X-ray images) of the 3D multi-region spine model with high-fidelity anatomical elements (for example, based on CT images or MRI analysis) of the anatomical elements of the 3D partial spine model. The system can collect further multi-dimensional image data in other load states and generate virtual models for replacing anatomical elements to be placed within the 3D virtual model.

[0133] Figure 16 shows an example flowchart 1600 of the operation for generating a multi-fidelity 3D virtual model of a patient's anatomical region according to an embodiment. Flowchart 1600 can generate a multi-fidelity 3D spinal model that can be used to perform simulations of the movement and / or loading of anatomical elements (e.g., vertebrae, implants, etc.) at various levels for different pre- and post-operative loading conditions. For example, loading information (e.g., how the loading affects the anatomical elements) can be determined by comparing images of the biomimetic structure under different loading conditions. Using the determined loading information, a 3D anatomical model representing one or more loading conditions can be generated. Using these operations, it is possible to map parts of human body structures in 3D image analysis to corresponding parts of the same human body structure in 2D images, as described in relation to Figures 4A and 4B. For example, the steps described in relation to Figures 4A and 4B can be incorporated into or replaced with the steps described in relation to the operation described in relation to Figure 16.

[0134] Operation 1610 includes receiving first multidimensional image data (e.g., CT scan data) of an anatomical region of a patient in a first loading state (e.g., a spinal region such as the thoracic and / or cervical regions of the patient's spine). Operation 1610 includes acquiring first multidimensional image data of an anatomical region of a patient in a first loading state. Based on the resolution level of the first multidimensional image data, the system can model virtual lumbar anatomical elements that correspond to the anatomical elements shown in the first multidimensional image data. In some embodiments, the first loading state may be when the patient is in a horizontal (or unloaded) position. For example, the patient may be lying down during a CT scan. The system (e.g., image processing module, system 200, image processing module 364, etc.) can determine from the first multidimensional image data multiple regions corresponding to multiple anatomical elements of the spine. Operation 1610 may include performing segmentation by identifying different structures or tissues in the image. This process may be performed manually by the user or automated using one or more segmentation routines. Segmentation routines can be used to extract specific regions of interest, such as vertebrae, vertebral bodies, and intervertebral discs, from the surrounding tissue.

[0135] Operation 1620 includes receiving a second multidimensional image data (e.g., X-ray data) of at least a portion of an anatomical region (e.g., a spinal region such as the lumbar region of the patient's spine) under a second load condition. Based on the resolution level of the second multidimensional image data, the system can model virtual anatomical elements of the thoracic and cervical regions that correspond to the anatomical elements shown in the second multidimensional image data. The system can identify a second set of landmarks from the second multidimensional image data that corresponds to a first set of landmarks. Operation 1620 includes obtaining aligned multidimensional image data of the patient's spine by aligning the corresponding anatomical elements between the first and second multidimensional image data. In some embodiments, operation 1620 includes obtaining load effect data by comparing the positions of the corresponding anatomical elements in the first and second multidimensional image data. The load effect data can be correlated with known loads at the time the first and second multidimensional image data were acquired. The load effect data can be used to predict the positions of anatomical elements under further load conditions.

[0136] Operation 1630 includes generating a multi-fidelity anatomical representation (e.g., a 3D virtual model) of a patient's biological structure or anatomical region (e.g., via a surgical planning platform). The multi-fidelity 3D virtual model may include a first set of anatomical elements (e.g., the patient's cervical, thoracic, or lumbar vertebrae) generated based on a first set of multidimensional image data. The first set of anatomical elements may have a first fidelity for obtaining one or more measurements of the anatomical region. The first fidelity may be above or below a threshold fidelity, and the second fidelity may be above or below a threshold fidelity for implant design. For example, fidelity can be based on the number of mesh elements / volume of the meshed anatomical element, the characteristics of the 3D information (e.g., vertices, edges, polygonal faces for surface representation, encoded curves and surfaces, material data, etc.), whether the model has surfaces represented by small planes or surfaces represented by curved surfaces, level of detail, type of source image data (e.g., number of images, pixel size, etc.), contour algorithm (e.g., algorithm for generating anatomical contours based on an image set using a convolutional neutral network, machine learning system, etc.), (single or multiple) reconstruction program, voxel parameters, and / or further fidelity metrics. A user or system can score fidelity parameters or metrics to determine the fidelity of an anatomical element. For example, an anatomical model formed by curved surfaces generated based on a series of CT scans (e.g., 1mm slices, 1.5mm slices, 128-slice CT scans, CT dose, etc.) may be assigned a higher fidelity than an anatomical model with flat polygonal surfaces that present contour surfaces based on X-rays. As another example, an anatomical model generated based on a 256-slice CT scan may be assigned a higher fidelity (e.g., 2, 3, or 4 times higher) than an anatomical model generated based on a 16-slice CT scan. Threshold fidelity can be selected based on the (single or multiple) tasks to be performed.For example, such tasks may include generating a patient-fitted surface contour, measuring anatomical values / metrics (e.g., metrics related to spinal curvature and / or alignment), performing one or more anatomical element analyses (e.g., load-bearing analysis, fracture analysis, etc.), and other tasks disclosed herein. For example, the implant design threshold fidelity for designing a patient-fitted implant surface contour may be higher than the measurement threshold fidelity for measuring anatomical features (e.g., intervertebral disc space height, spinal curvature, etc.).

[0137] In some embodiments, the high-fidelity 3D virtual model may include a second set of anatomical elements generated based on a second set of multidimensional image data (for example, representing the lumbar spine of a patient). The second set of anatomical elements may have a second fidelity for designing one or more implants that fit the second set of anatomical elements. The system can use the high-fidelity 3D virtual model to simulate one or more lumbar spine adjustments. In response to one or more simulated lumbar spine adjustments, the system can generate at least one of the following: a lumbar compensatory response, a pelvic tilt compensatory response, a thoracic compensatory response, or a cervical compensatory response.

[0138] The system can perform a first task (e.g., designing at least one implant surface that matches the patient surface) on a first region of a 3D virtual model (e.g., using a surgical planning platform) based on a first fidelity level of the first region. The first fidelity level may correspond to a first level of image detail of the patient images in a first image dataset. The system can also perform a second task on a second region of the 3D virtual model (e.g., using a surgical planning platform) based on a second fidelity level of the second region. The second fidelity level may correspond to a second level of image detail of the patient images in a second image dataset. The number and type of tasks can be selected based on the procedure to be performed. The surgical planning platform can limit the available tasks for each of several regions of the 3D multi-resolution virtual model based on the resolution level of that region appropriate for the available tasks. Based on the fidelity analysis of the 3D multi-resolution virtual model, the surgical planning platform can assign usage constraints (e.g., multi-element constraints such as grouped anatomical element constraints and individual element constraints) to the 3D multi-resolution virtual model.

[0139] Figure 17 shows an example flowchart 1700 of an operation to generate a 3D virtual model of a patient's biostructure according to one embodiment. Flowchart 1700 can generate a high-fidelity 3D spinal model that can be used to perform simulations of the movement and / or loading of anatomical elements (e.g., vertebrae, implants, etc.) at various levels for different pre- and post-operative loading conditions. For example, loading information (e.g., how the loading affects the anatomical elements) can be determined by comparing images of the biostructure under different loading conditions. Using the determined loading information, a 3D anatomical model representing one or more loading conditions can be generated. Using these operations, it is possible to map parts of the human structure in 3D image analysis to corresponding parts of the same human structure in 2D images, as described in relation to Figures 4A and 4B. For example, the steps described in relation to Figures 4A and 4B can be incorporated into or replaced with the steps described in relation to the operation described in relation to Figure 17.

[0140] Operation 1710 includes receiving a first set of image data (e.g., CT scan data or X-ray data) having a first resolution of an anatomical region (e.g., a spinal region such as the thoracic and / or cervical regions of the patient's spine). Based on the resolution level of the first image dataset, the system can model a virtual lumbar anatomical element that matches the anatomical element shown in the first image dataset. In some embodiments, the first loading state may be when the patient is in a horizontal (or unloaded) position. For example, the patient may be lying down during a CT scan. The system (e.g., image processing module, system 200, image processing module 364, etc.) can determine from the image data multiple regions corresponding to multiple anatomical elements of the spine. Operation 1710 may include performing segmentation by identifying different structures or tissues in the images. This process may be performed manually by the user or automated using one or more segmentation routines. Segmentation routines can be used to extract specific regions of interest, such as vertebrae, vertebral bodies, or intervertebral discs, from the surrounding tissue.

[0141] Operation 1710 includes receiving a second image dataset (e.g., tomographic scan data) having a second resolution for at least a portion of an anatomical region (e.g., a spinal region such as the lumbar region of the patient's spine). Based on the resolution level of the second image dataset, the system can model virtual anatomical elements of the thoracic and cervical regions that correspond to the anatomical elements shown in the image data. The system can identify a second set of landmarks from the second image dataset that corresponds to a first set of landmarks. Operation 1720 includes obtaining aligned image data of the patient's spine by aligning the corresponding anatomical elements between the first and second image data. In some embodiments, operation 1720 includes obtaining load effect data by comparing the positions of the corresponding anatomical elements in the first and second image data. The load effect data can be correlated with known loads at the time the first and second image datasets were acquired. The load effect data can be used to predict the positions of anatomical elements under further load conditions.

[0142] Operation 1720 includes obtaining surgical planning information from a surgical planning platform configured to generate a 3D virtual model of the patient's biostructure (e.g., a 3D spine model). The surgical planning platform may assign a set of available first tasks to a first region of the 3D virtual model based on a first resolution. The first region may be generated using a first set of images. In a first example, the set of available first tasks may include determining at least one of lumbar, thoracic, or cervical compensatory responses in response to one or more spinal corrections in a second region. In a second example, the set of available first tasks may include determining multilevel spinal correction metrics measured at heights above the lumbar region.

[0143] The surgical planning platform can assign a set of available second tasks to a second region of a three-dimensional virtual model based on a second resolution. The second region can be generated using a second set of images. In the first example, the set of available second tasks may include determining a pelvic tilt compensatory response in response to one or more spinal corrections at the lumbar level of the second region. In the second example, the set of available second tasks may include designing at least one of one or more spinal implants for implantation in the lumbar region, or measuring spinal metrics along the lumbar region.

[0144] Operation 1730 includes receiving a first output from one or more first tasks performed using the first region. The first region of the 3D virtual model may include virtual anatomical elements having X-ray feature details present in X-ray images in the first dataset. Operation 1740 includes receiving a second output from one or more first tasks in the second region. The second region of the 3D virtual model may include virtual anatomical elements having tomographic feature details present in tomographic images in the second dataset.

[0145] In some embodiments, the embodiments, features, systems, apparatus, materials, methods, and techniques described herein may be similar to one or more of the embodiments, features, systems, apparatus, materials, methods, and techniques described later. U.S. Patent Application Publication No. 16 / 048,167, filed on July 27, 2017, entitled "Systems and Methods for Assisting and Augmenting Surgical Procedures" U.S. Patent Application Publication No. 16 / 242,877, filed on January 8, 2019, entitled "Systems and Methods of Assisting a Surgeon with Screw Placement During Spinal Surgery," U.S. Patent Application Publication No. 16 / 207,116, filed on December 1, 2018, entitled "Systems and Methods for Multi-Planar Orthopedic Alignment" U.S. Patent Application Publication No. 16 / 352,699, filed on March 13, 2019, entitled "Systems and Methods for Orthopedic Implant Fixation" U.S. Patent Application Publication No. 16 / 383,215, filed on April 12, 2019, entitled "Systems and Methods for Orthopedic Implant Fixation" U.S. Patent Application Publication No. 16 / 569,494, titled "Systems and Methods for Orthopedic Implants," filed on September 12, 2019. U.S. Patent Application Publication No. 62 / 773,127, titled "Systems and Methods for Orthopedic Implants," filed on November 29, 2018. U.S. Patent Application Publication No. 62 / 928,909, filed on October 31, 2019, entitled "Systems and Methods for Designing Orthopedic Implants Based on Tissue Characteristics," U.S. Patent Application Publication No. 16 / 735,222 (currently U.S. Patent No. 10,902,944), filed on January 6, 2020, entitled "Patient-Specific Medical Procedures and Devices, and Associated Systems and Methods," U.S. Patent Application Publication No. 16 / 987,113, filed on August 6, 2020, entitled "Patient-Specific Artificial Discs, Implants, and Associated Systems and Methods," U.S. Patent Application Publication No. 16 / 990,810, filed on August 11, 2020, entitled "Linking Patient-Specific Medical Devices with Patient-Specific Data, and Associated Systems, Devices, and Methods," U.S. Patent Application Publication No. 17 / 085,564, filed on October 30, 2020, entitled "Systems and Methods for Designing Orthopedic Implants Based on Tissue Characteristics," U.S. Patent Application Publication No. 17 / 100,396, filed on November 20, 2020, entitled "PATIENT-SPECIFIC VERTEBRAL IMPLANTS WITH POSITIONING FEATURES" U.S. Patent Application Publication No. 17 / 124,822, filed on December 17, 2020, entitled "PATIENT-SPECIFIC MEDICAL PROCEDURES AND DEVICES, AND ASSOCIATED SYSTEMS AND METHODS," and, International patent application No. 2021 / 012065, filed on January 4, 2021, entitled "Patient-Specific Medical Procedures and Devices, and Associated Systems and Methods".

[0146] All of the above-mentioned patents and applications are incorporated herein by reference in their entirety. In addition, in certain embodiments, the embodiments, features, systems, apparatus, materials, methods and techniques described herein may be applied or used in connection with one or more of these embodiments, features, systems, apparatus or other matters.

[0147] As those skilled in the art will understand, any of the above-described software modules can be combined to form a single software module that performs the operations described herein. Similarly, the software modules can be distributed across any combination of computer systems and devices described herein, and are not limited to the specific arrangements described herein. Thus, the operations described herein can be performed by any of the computer devices or systems described herein unless otherwise expressly stated.

[0148] Examples

[0149] For example, the present technology will be illustrated according to various embodiments described later. For convenience, various embodiments of the present technology will be described as numbered embodiments (1, 2, 3, etc.). These are shown as examples and do not limit the present technology. Dependent embodiments can be combined in any preferred manner and incorporated into their respective independent embodiments. Other embodiments can be presented in the same manner. 1. A computer implementation method for simulating spinal load on a patient, Receiving first multidimensional image data of an anatomical region of a patient in a first load state, Determining the first position coordinates of a vertebra in a first multidimensional image data under a first load state, Determining the first rotation, translation, and scaling parameters of a vertebra in a first multidimensional image data under a first load state, Receiving a second multidimensional image data of an anatomical region of a patient in a second load state, Determining the second position coordinates of the vertebrae in the second multidimensional image data under a second load state, To determine the second rotation, translation, and scaling parameters of the vertebrae in the second multidimensional image data under a second load state, Based on a first position coordinate, first rotation, translation, and scaling parameters, a second position coordinate, and a second rotation parameter, a model of the patient's spine is generated to simulate spinal loading in a first and second loading state, Computer implementation methods including 2. A computer implementation method of Example 1, further comprising determining first and second position coordinates based on mapping the vertebrae to one or more landmarks of a patient. 3. A computer implementation method of any of Examples 1 to 2, further comprising obtaining aligned multidimensional image data of a patient's spine by aligning corresponding anatomical elements between a first multidimensional image data and a second multidimensional image data. 4. To generate one or more models of one or more vertebrae in the patient's spine, Inserting one or more models into a spine model according to a first position coordinate, a first rotation parameter, a second position coordinate, and second rotation, translation, and scaling parameters corresponding to one or more vertebrae, A computer implementation method, further comprising any of Examples 1 to 3. 5. Determine the load state mapping based on the first and second load states, Perform load state mapping between the first multidimensional image data and the second multidimensional image data to generate aligned multidimensional image data, A computer implementation method, further comprising any of Examples 1 to 4. 6. Further including simulating compensatory or reciprocal mechanisms for lumbar spine correction through a model, the compensatory mechanisms are: Lumbar compensatory reaction, Pelvic tilt compensatory response to lumbar spine correction, Compensatory reaction of thoracic kyphosis to lumbar spine correction, or, Cervical lordosis compensatory response to lumbar spine correction, A computer implementation method, including any of Examples 1 to 5. 7. A computer implementation method according to any of Examples 1 to 6, wherein the first multidimensional image data is acquired from a computed tomography (CT) scan or magnetic resonance imaging (MRI) scan, and the second multidimensional image data is acquired from an X-ray scan. 8. A computer implementation method for simulating spinal load on a patient, Receiving first multidimensional image data of an anatomical region of a patient in a first load state, Receiving a second multidimensional image data of at least a portion of an anatomical region under a second load state, Based on the first and second multidimensional image data, determine one or more loading effects on at least one anatomical element in an anatomical region, Based on a load effect of 1 or more that correlates with the preoperative load state, a 3D virtual model of the patient is generated to simulate the postoperative load state, Computer implementation methods including 9.3-dimensional virtual models are A first set of anatomical element models with high-fidelity surface topology for designing one or more patient-specific implants, A second set of anatomical element models for measuring spinal metrics that predict surgical outcomes associated with the implantation of one or more patient-specific implants, The computer implementation method of Example 8, comprising, wherein one or more of the anatomical element models in the second set of anatomical element models have less feature data than all or part of the anatomical element models in the first set of anatomical element models. 10. A computer implementation method of any of Examples 8-9, further comprising arranging, oriented, and / or scaling the anatomical element models of a first set of anatomical element models to be incorporated into a second set of anatomical element models in order to generate a three-dimensional virtual model. 11. To generate an X-ray fidelity anatomical element model based on one or more upright X-ray images of the second multidimensional image data, To generate a tomographic fidelity anatomical element model based on a series of tomographic images of the first multidimensional image data, A computer implementation method of any of Examples 8 to 10, further comprising the following: the X-ray fidelity anatomical element model has a resolution below threshold fidelity, and the tomographic fidelity anatomical element model has a fidelity above threshold fidelity. 12. The 3D virtual model is a 3D multi-domain simulation model of the patient's spine, and the computer implementation method is as follows: The first step is to generate a 3D multi-region spine model of the patient based on multidimensional image data, To generate a 3D partial spine model that matches the corresponding region of the spine captured in the second multidimensional image data, This involves generating a 3D multi-domain simulation model by combining the anatomical elements of a 3D partial spine model with a 3D multi-domain spine model, and A computer implementation method, further comprising any of Examples 8 to 11. 13. A computer implementation method according to any of Examples 8 to 12, further comprising replacing a low-fidelity anatomical element model of a three-dimensional multi-domain spine model with a corresponding high-fidelity anatomical element of a three-dimensional partial spine model. 14. A computer implementation method according to any of Examples 8 to 13, wherein the 3D multi-region spine model includes the cervical and thoracic regions of the patient's spine, and the 3D partial spine model includes a lumbar region model having surface topology data for designing one or more implants. 15. The first multidimensional image data shows at least the thoracic and cervical regions of the patient's spine, and the second multidimensional image data shows the lumbar region of the patient's spine. The computer implementation method is as follows: Based on the resolution level of the first multidimensional image data, a virtual lumbar anatomical element corresponding to the anatomical element shown in the first multidimensional image data is modeled. Based on the resolution level of the second multidimensional image data, the virtual anatomical elements of the thoracic and cervical regions corresponding to the anatomical elements shown in the second multidimensional image data are modeled. A computer implementation method, further comprising any of Examples 8 to 14. 16. The patient receives a third multidimensional image data in a third load state that corresponds to a second load state, To generate a virtual model for replacing anatomical elements to be placed within a 3D virtual model, A computer implementation method, further comprising any of Examples 8 to 15. A computer implementation method according to any of Examples 8 to 16, further comprising arranging anatomical elements in anatomical regions based on load effects of 17.1 or more. 18. Identifying a first image type of the first multidimensional image data, Determining a first load state based on a first image type of the first multidimensional image data, To determine one or more anatomical metrics affected by the first load state, The first multidimensional image data is used to measure one or more anatomical metrics of an anatomical region, and the measured one or more anatomical metrics are used to generate a three-dimensional virtual model of the spine. A computer implementation method, further comprising any of Examples 8 to 17. 19. Identifying a first image type of the first multidimensional image data, Identifying a second image type of the second multidimensional image data, Selecting a mapping routine based on the first image type, the second image type, and anatomical elements in the anatomical region, A computer implementation method, further comprising any of Examples 8 to 18. 20. The first image type is a CT scan. The second image type is X-ray imaging. A computer implementation method according to any of Examples 8 to 19, wherein the mapping routine is configured to map anatomical features in a CT scan to the same anatomical features in an X-ray image, and to determine one or more load effects in an anatomical region based on the positional differences of the same anatomical features due to load during the CT scan and X-ray image. 21. A computer implementation method of any of Examples 8 to 20, wherein a 3D virtual model of the spine has a rendered virtual surface, which is configured for designing one or more implant models whose contours match those of the rendered virtual surface, and the 3D virtual model is generated by combining 2D simultaneous orthogonal X-ray data and 3D MRI data to generate an anatomically loaded 3D spine model. 22. A computer implementation method for simulating spinal load on a patient, Receiving the first multidimensional image data of the patient's anatomical region, Receiving a second multidimensional image data of at least a portion of an anatomical region, To generate high-fidelity 3D virtual models of anatomical regions, The high-fidelity 3D virtual model includes, A first set of anatomical elements generated based on a first multidimensional image data, A second set of anatomical elements generated based on a second multidimensional image data, Includes, The first set of anatomical elements has a first fidelity for obtaining one or more measurements of an anatomical region. A computer implementation method having a second fidelity for designing one or more implants that fit the second set of anatomical elements. 23. The method of Example 22, in which the first fidelity is below the threshold fidelity, the second fidelity is above the threshold fidelity for implant design, and the second fidelity is below the threshold fidelity. 24. The first set of anatomical elements includes at least one of the patient's cervical, thoracic, or lumbar vertebrae, according to any method of Examples 22-23. 25. Any method of Examples 22-24, wherein the first set of anatomical elements represents the patient's cervical and thoracic vertebrae, and the second set of anatomical elements represents the patient's lumbar vertebrae. 26. Simulating one or more lumbar spine adjustments using a high-fidelity 3D virtual model, In response to one or more lumbar spine adjustments, it generates at least one of the following: a lumbar compensatory response, a pelvic tilt compensatory response, a thoracic compensatory response, or a cervical compensatory response. Any method of Examples 22 to 25, further including the above. 27. Any method of Examples 22-26, further comprising applying an imaging modality correction routine to the second set of anatomical elements in order to position the second set of anatomical elements within a high-fidelity 3D virtual model. 28. Prepare a first set of patient images, including images having a first resolution, and a second set of patient images, including images having a second resolution different from the first resolution. A surgical planning platform configured to generate a three-dimensional virtual model of a patient's biological structure, Based on a first resolution, a series of available first tasks are assigned to a first region of a three-dimensional virtual model generated using a first set of images. Based on the second resolution, a series of available second tasks are assigned to the second region of the 3D virtual model generated using the second set of images. Obtaining surgical planning information from the surgical planning platform, Receiving a first output from one or more first tasks that were executed using a first area of ​​the first task, Receiving a second output from one or more first tasks that were executed using the second area of ​​the first task, Methods that include... 29. The method of Example 28, wherein the images in the first image set are X-ray images and the images in the second image set are tomographic scans. 30. The method of any of Examples 28-29, wherein the 3D virtual model is a spine model, and the available first task is to determine at least one of a thoracic kyphosis compensatory response or a cervical lordosis compensatory response in response to one or more spinal corrections in a second region. 31. The 3D virtual model is a spine model, and the available second task is to determine the pelvic tilt compensatory response in response to one or more spinal corrections at the lumbar level of a second region, as described in any of the methods of Examples 28-30. 32. A series of available first tasks include determining multilevel spinal correction metrics measured at levels above the lumbar region, The series of available second tasks are: Designing at least one of one or more spinal implants to be implanted in the lumbar region, Measuring spinal metrics along the lumbar spine region, A method of any of Examples 28 to 31, comprising at least one of the above. 33. A 3D virtual model representing the patient's corrected spine, using any of the methods in Examples 28-32. 34. The first region of the 3-dimensional virtual model includes virtual anatomical elements that have radiographic feature details present in the X-ray images in the first set of images. The second region of the three-dimensional virtual model includes virtual anatomical elements having tomographic feature details present in the tomographic images in the second set of images, according to any of the methods of Examples 28 to 32. 35. Obtain the first patient image set and the second patient image set, Through the surgical planning platform, a three-dimensional high-fidelity virtual model of the patient's biological structure is generated based on the first and second sets of patient images, Using a surgical planning platform, perform a first task on a first region of a three-dimensional virtual model based on a first fidelity level corresponding to a first image detail level of patient images in a first set of patient images. Using a surgical planning platform, perform a second task on a second region of a 3D virtual model based on a second fidelity level corresponding to a second image detail level of patient images in a second set of patient images. A method that includes this. 36. The first task involves designing at least one implant surface that matches the patient surface. The surgical planning platform of the method of Example 35 limits the available tasks for each of multiple regions of a three-dimensional high-fidelity virtual model based on the resolution level of that region appropriate for the available tasks. 37. Any method of Examples 35 to 36, further comprising assigning usage constraints to the 3D multi-fidelity virtual model based on a fidelity analysis of the 3D multi-fidelity virtual model. 38. The first and second tasks are performed according to any of the methods of Examples 35-37, subject to usage constraints. 39. Any method of Examples 35–38, further comprising displaying usage constraints to the user and displaying fidelity-dependent tasks that can be performed using a 3D high-fidelity virtual model. 40. Any method of Examples 35 to 39, further comprising determining at least one of a first or second task based on at least one fidelity parameter of a 3-dimensional multi-fidelity virtual model. 41. A computer implementation method for medical imaging data, To acquire first multidimensional image data of an anatomical region of a patient in a first load state, From the first multidimensional image data, multiple regions corresponding to multiple anatomical elements of the spine are determined, Identifying a first set of landmarks in each of multiple regions, To acquire a second multidimensional image data of an anatomical region, including the spine, of a patient in a second load state, Identifying a second landmark set corresponding to a first landmark set from a second multidimensional image data, By aligning the corresponding anatomical elements between the first and second multidimensional image data, a aligned multidimensional image data of the patient's spine is obtained. To generate a medical implant design based on spinal and pelvic parameters measured from aligned multidimensional image data, By transmitting the design of a medical implant to a manufacturing device, a medical implant is manufactured. Computer implementation methods including 42. A computer implementation method of Example 41, in which the first multidimensional image data is collected in 3 dimensions and the second multidimensional image data is collected in 2 dimensions. 43. A computer implementation method according to any of Examples 41 to 42, wherein the aligned multidimensional image data is a three-dimensional image. 44. A computer implementation method of any of Examples 41 to 43, wherein acquiring aligned multidimensional image data includes correcting the positions of multiple anatomical elements in a first multidimensional image data according to the positions of multiple anatomical elements in a second multidimensional image data. 45. Determining load state mapping based on the first and second load states, The process involves performing load state mapping between the first multidimensional image data and the second multidimensional image data to generate aligned multidimensional image data, A computer implementation method, further comprising any of Examples 41 to 44. 46. ​​In response to the region being the lumbar spine, identifying the first set of landmarks in the region is: The inferior surface of the spinous process of the lumbar vertebrae, The upper surface of the spinous process of the lumbar vertebrae, The left side of the transverse process of the lumbar vertebrae, The right side of the transverse process of the lumbar vertebra, A computer implementation method of any of Examples 41 to 45, which includes identifying a 47. Aligning corresponding anatomical elements between the first multidimensional image data and the second multidimensional image data is From three-dimensional (3D) image data, a first image data is formed along a sagittal view that divides the sacrum into two equal parts. A second image data, including the sacrum, along the sagittal view from a two-dimensional (2D) image data, is projected onto the first image data. The first image data is scaled, rotated, and / or translated so that the sacrum from the first image data overlaps with the sacrum in the second image data. A computer implementation method according to any of Examples 41 to 46, which includes aligning the sacrum between 3D image data and 2D image data. 48. Aligning corresponding anatomical elements is The first alignment operation of the lumbar vertebrae is performed by moving the anterior and posterior landmarks of the lumbar vertebrae in the 3D image data to match the same landmarks in a second image data containing the lumbar vertebrae from the 2D image data. From the 3D image data, a third image data is formed along the coronal view extending across the sacrum. A fourth image data, including the sacrum along the coronal view from 2D image data, is projected onto the third image data. By scaling or translating the first image data, the sacrum from the third image data is made to overlap with the sacrum in the second image data. A second lumbar alignment operation is performed by moving the anterior and posterior landmarks of the lumbar vertebrae to match the same landmarks in a fourth image data, including the lumbar vertebrae, from a 2D image. A computer implementation method of any of Examples 41 to 47, which includes aligning each lumbar vertebra between 3D image data and 2D image data, performed in response to aligning the sacrum by a certain action. 49. Adjust one or more other landmarks of the lumbar vertebrae by the same amount used to move the anterior and posterior landmarks of the lumbar vertebrae in the 3D image data for the first alignment operation, Adjust one or more other landmarks of the lumbar vertebrae by the same amount used to move the anterior and posterior landmarks of the lumbar vertebrae in the 3D image data for the second alignment operation, A computer implementation method, further comprising any of Examples 41 to 48. 50. A computer implementation method according to any of Examples 41 to 49, wherein the first multidimensional image data is acquired from a computed tomography (CT) scan or magnetic resonance imaging (MRI) scan, and the second multidimensional image data is acquired from an X-ray scan. 51. A non-temporary computer-readable storage medium for storing instructions, wherein the instructions are stored when executed by a computer system. To acquire first multidimensional image data of an anatomical region of a patient in a first load state, From the first multidimensional image data, multiple regions corresponding to multiple anatomical elements of the spine are determined, Identifying a first set of landmarks in each of multiple regions, To acquire a second multidimensional image data of an anatomical region, including the spine, of a patient in a second load state, Identifying a second landmark set corresponding to a first landmark set from a second multidimensional image data, By aligning the corresponding anatomical elements between the first and second multidimensional image data, a aligned multidimensional image data of the patient's spine is obtained. To generate a medical implant design based on spinal and pelvic parameters measured from aligned multidimensional image data, By transmitting the design of a medical implant to a manufacturing device, a medical implant is manufactured. A non-temporary computer-readable storage medium that causes a computer system to execute a method including [a specific method]. 52. A non-temporary computer-readable storage medium of Example 51, in which the first multidimensional image data is collected in three dimensions and the second multidimensional image data is collected in two dimensions. 53. A non-temporary computer-readable storage medium of any of Examples 51 to 52, wherein the aligned multidimensional image data is a three-dimensional image. 54. A non-temporary computer-readable storage medium of any of Examples 51 to 53, wherein acquiring aligned multidimensional image data includes correcting the positions of multiple anatomical elements in a first multidimensional image data according to the positions of multiple anatomical elements in a second multidimensional image data. 55. Determining load state mapping based on the first and second load states, The process involves performing load state mapping between the first multidimensional image data and the second multidimensional image data to generate aligned multidimensional image data, A non-temporary computer-readable storage medium of any of Examples 51 to 54, further comprising the above. 56. In response to the region being the lumbar spine, identifying the first set of landmarks in the region is: The inferior surface of the spinous process of the lumbar vertebrae, The upper surface of the spinous process of the lumbar vertebrae, The left side of the transverse process of the lumbar vertebrae, The right side of the transverse process of the lumbar vertebra, A non-temporary computer-readable storage medium of any of Examples 51 to 55, including the identification of a 57. Aligning corresponding anatomical elements between the first multidimensional image data and the second multidimensional image data is From three-dimensional (3D) image data, a first image data is formed along a sagittal view that divides the sacrum into two equal parts. A second image data, including the sacrum, along the sagittal view from a two-dimensional (2D) image data, is projected onto the first image data. The first image data is scaled, rotated, and / or translated so that the sacrum from the first image data overlaps with the sacrum in the second image data. A non-temporary computer-readable storage medium according to any of Examples 51 to 56, which includes aligning the sacrum between 3D image data and 2D image data. 58. Aligning corresponding anatomical elements is The first alignment operation of the lumbar vertebrae is performed by moving the anterior and posterior landmarks of the lumbar vertebrae in the 3D image data to match the same landmarks in a second image data containing the lumbar vertebrae from the 2D image data. A fourth image data, including the sacrum along the coronal view from 2D image data, is projected onto the third image data. A fourth image data, including the sacrum along the coronal view from 2D image data, is projected onto the third image data. By scaling or translating the first image data, the sacrum from the third image data is made to overlap with the sacrum in the second image data. A second lumbar alignment operation is performed by moving the anterior and posterior landmarks of the lumbar vertebrae to match the same landmarks in a fourth image data, including the lumbar vertebrae, from a 2D image. A non-temporary computer-readable storage medium of any of Examples 51 to 57, which includes aligning each lumbar vertebra between 3D image data and 2D image data, performed in response to aligning the sacrum by such means. 59. The method is, Adjust one or more other landmarks of the lumbar vertebrae by the same amount used to move the anterior and posterior landmarks of the lumbar vertebrae in the 3D image data for the first alignment operation, A non-temporary computer-readable storage medium of any of Examples 51 to 58, further comprising adjusting one or more other landmarks of the lumbar vertebrae by the same amount used to move the anterior and posterior landmarks of the lumbar vertebrae in the 3D image data for a second alignment operation. 60. A non-temporary computer-readable storage medium according to any of Examples 51 to 59, wherein the first multidimensional image data is acquired from a computed tomography (CT) scan or magnetic resonance imaging (MRI) scan, and the second multidimensional image data is acquired from an X-ray scan. 61. A system for medical imaging data comprising one or more processors and a memory for storing instructions, wherein when an instruction is executed by one or more processors, To acquire first multidimensional image data of an anatomical region of a patient in a first load state, From the first multidimensional image data, multiple regions corresponding to multiple anatomical elements of the spine are determined, Identifying a first set of landmarks in each of multiple regions, To acquire a second multidimensional image data of an anatomical region, including the spine, of a patient in a second load state, Identifying a second landmark set corresponding to a first landmark set from a second multidimensional image data, By aligning the corresponding anatomical elements between the first and second multidimensional image data, a aligned multidimensional image data of the patient's spine is obtained. To generate a medical implant design based on spinal and pelvic parameters measured from aligned multidimensional image data, By transmitting the design of a medical implant to a manufacturing device, a medical implant is manufactured. A system that performs a task. 62. The system of Example 61, in which the first multidimensional image data is collected in 3D and the second multidimensional image data is collected in 2D. 63. The system of any of Examples 61-62, wherein the aligned multidimensional image data is a 3D image. 64. A system of any of Examples 61 to 63, wherein acquiring aligned multidimensional image data includes correcting the positions of multiple anatomical elements in a first multidimensional image data according to the positions of multiple anatomical elements in a second multidimensional image data. 65. Determining load state mapping based on the first and second load states, The process involves performing load state mapping between the first multidimensional image data and the second multidimensional image data to generate aligned multidimensional image data, A system further comprising any of Examples 61 to 64. 66. In response to the region being the lumbar spine, identifying the first set of landmarks in the region is: The inferior surface of the spinous process of the lumbar vertebrae, The upper surface of the spinous process of the lumbar vertebrae, The left side of the transverse process of the lumbar vertebrae, The right side of the transverse process of the lumbar vertebra, A system of any of Examples 61 to 65, which includes identifying [something]. 67. Aligning corresponding anatomical elements between the first multidimensional image data and the second multidimensional image data is From three-dimensional (3D) image data, a first image data is formed along a sagittal view that divides the sacrum into two equal parts. A second image data, including the sacrum, along the sagittal view from a two-dimensional (2D) image data, is projected onto the first image data. The first image data is scaled, rotated, and / or translated so that the sacrum from the first image data overlaps with the sacrum in the second image data. A system according to any of Examples 61 to 66, which includes aligning the sacrum between 3D image data and 2D image data. 68. Aligning corresponding anatomical elements is The first alignment operation of the lumbar vertebrae is performed by moving the anterior and posterior landmarks of the lumbar vertebrae in the 3D image data to match the same landmarks in a second image data containing the lumbar vertebrae from the 2D image data. From the 3D image data, a third image data is formed along the coronal view extending across the sacrum. A fourth image data, including the sacrum along the coronal view from 2D image data, is projected onto the third image data. By scaling or translating the first image data, the sacrum from the third image data is made to overlap with the sacrum in the second image data. A second lumbar alignment operation is performed by moving the anterior and posterior landmarks of the lumbar vertebrae to match the same landmarks in a fourth image data, including the lumbar vertebrae, from a 2D image. A system of any of Examples 61 to 67, which includes aligning each lumbar vertebra between 3D image data and 2D image data, performed in response to aligning the sacrum by a certain action. 69. Adjust one or more other landmarks of the lumbar vertebrae by the same amount used to move the anterior and posterior landmarks of the lumbar vertebrae in the 3D image data for the first alignment operation, Adjust one or more other landmarks of the lumbar vertebrae by the same amount used to move the anterior and posterior landmarks of the lumbar vertebrae in the 3D image data for the second alignment operation, A system from any of Examples 61 to 68, further including the above. 70. A system according to any of Examples 61 to 69, in which the first multidimensional image data is acquired from a computed tomography (CT) scan or magnetic resonance imaging (MRI) scan, and the second multidimensional image data is acquired from an X-ray scan. A processor with a 71.1 or higher processor, One or more memory locations that store instructions to cause a computer system to execute any of the processes described in Examples 1 to 70 when executed by one or more processors, A computer system equipped with the following features. 72. A non-temporary computer-readable medium that stores instructions causing a computer system to perform any of the actions described in Examples 1 to 70 when executed by the computer system.

[0150] The detailed description above illustrates various embodiments of the apparatus and / or process through block diagrams, flowcharts and / or examples. Those skilled in the art will understand that if such block diagrams, flowcharts and / or examples include one or more functions and / or operations, each function and / or operation in such block diagrams, flowcharts and / or examples can be implemented individually and / or collectively by a wide range of hardware, software, firmware or substantially all combinations thereof. In some embodiments, multiple parts of the subject matter described herein can be implemented by application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs) or other integrated formats. However, those skilled in the art will recognize that some aspects of the embodiments disclosed herein can be implemented equivalently in whole or in part within an integrated circuit as one or more computer programs running on one or more computers (e.g., as one or more programs running on one or more computer systems), as one or more programs running on one or more processors (e.g., as one or more programs running on one or more microprocessors), as firmware, or substantially all combinations thereof, and that designing the circuits and / or writing the software and / or firmware code falls well within the scope of the art in light of this disclosure. Furthermore, those skilled in the art will understand that the mechanisms of the subject matter described herein can be distributed as various forms of program products, and that the exemplary embodiments of the subject matter described herein apply regardless of the specific type of signaling medium actually used for distribution. Examples of signaling mediums include, but are not limited to, recordable media such as floppy disks, hard disk drives, CDs, DVDs, digital tapes, and computer memory, as well as transmitting media such as digital and / or analog communication media (e.g., fiber optic cables, waveguides, wired communication links, wireless communication links, etc.).

[0151] Those skilled in the art will recognize that it is common practice in the art to describe apparatus and / or processes as described herein and then, using engineering conventions, integrate such described apparatus and / or processes into a data processing system. That is, at least some of the apparatus and / or processes described herein can be integrated into a data processing system through reasonable experimental means. Those skilled in the art will recognize that a typical data processing system generally includes a system unit housing, video display devices, memory such as volatile and non-volatile memory, processors such as microprocessors and digital signal processors, computing entities such as operating systems, drivers, graphical user interfaces and application programs, one or more interacting devices such as touchpads or screens, and / or control systems including feedback loops and control motors (e.g., feedback for sensing position and / or velocity, control motors for moving and / or adjusting components and / or quantities). A typical data processing system can be implemented using any suitable commercially available components, such as those typically found in data computer / communication systems and / or network computer / communication systems.

[0152] The subjects described herein may sometimes show different components that are included in or connected to other different components. The architectures shown in this manner are merely illustrative, and it should be understood that many other architectures can be implemented to achieve the same functionality. Conceptually, any configuration of components that achieve the same functionality effectively "relates" to each other in such a way that the desired functionality is achieved. Therefore, any two components in this specification that combine to achieve a particular functionality can be considered "relates" to each other, regardless of the architecture or intervening components, in such a way that the desired functionality is achieved. Similarly, any two such related components can be considered "operably connected" or "operably coupled" to each other in such a way that the desired functionality is achieved, and any two components that can relate in such a way can be considered "operably coupled" to each other in such a way that the desired functionality is achieved. Specific examples of operatably coupled components include, but are not limited to, physically coupled and / or physically interacting components, and / or wirelessly interacting and / or wirelessly interacting components, and / or logically interacting and / or logically interactable components.

[0153] All of the above-mentioned patents and applications are incorporated herein by reference in their entirety. In addition, in certain embodiments, the embodiments, features, systems, apparatus, materials, methods and techniques described herein may be applied or used in connection with one or more of these embodiments, features, systems, apparatus or other matters.

[0154] The scope disclosed herein includes all overlaps, sub-scopes, and combinations thereof. Expressions such as “up to,” “at least,” “greater than,” “less than,” or “between” include the numbers stated. The numbers following terms such as “approximately,” “about,” and “substantially” as used herein include the numbers stated (for example, about 10% = 10%) and represent an amount close to the stated amount that performs the desired function or achieves the desired result. For example, the terms “approximately,” “about,” and “substantially” may mean an amount within the range of less than 10%, less than 5%, less than 1%, less than 0.1%, and less than 0.01% of the stated amount.

[0155] The above-described embodiments of the disclosure are illustrative and can be modified in various ways without departing from the scope and spirit of the disclosure. Accordingly, the embodiments disclosed herein are not intended to be limiting.

Claims

1. A computer implementation method for simulating spinal load on a patient, Receiving first multidimensional image data of an anatomical region of a patient in a first load state, Determining the first position coordinates of the vertebrae in the first multidimensional image data under the first load state, Determining the first rotation, translation, and scaling parameters of the vertebra in the first multidimensional image data under the first load state, Receiving a second multidimensional image data of the anatomical region of the patient in a second load state, Determining the second position coordinates of the vertebra in the second multidimensional image data under the second load state, Determining the second rotation, translation, and scaling parameters of the vertebra in the second multidimensional image data under the second load state, Based on the first position coordinates, the first rotation, translation, and scaling parameters, the second position coordinates, and the second rotation parameters, a model of the patient's spine is generated to simulate spinal loading in the first and second loading states, Computer implementation methods, including those mentioned above.

2. The further includes determining the first and second position coordinates based on mapping the vertebrae to one or more landmarks of the patient. The computer implementation method according to claim 1.

3. The method further includes obtaining aligned multidimensional image data of the patient's spine by aligning corresponding anatomical elements between the first multidimensional image data and the second multidimensional image data. The computer implementation method according to claim 1.

4. To generate one or more models of one or more vertebrae in the spine of the patient, Inserting the one or more models into the model of the spine according to the first position coordinates, first rotation parameter, second position coordinates, and second rotation, translation, and scaling parameters corresponding to the one or more vertebrae, The computer implementation method according to claim 1, further comprising:

5. Determining the load state mapping based on the first and second load states, Performing the load state mapping between the first multidimensional image data and the second multidimensional image data to generate aligned multidimensional image data, The computer implementation method according to claim 1, further comprising:

6. The model further includes simulating compensatory or reciprocal mechanisms for lumbar spine correction, wherein the compensatory mechanism is: Lumbar compensatory reaction, Pelvic tilt compensatory response to the aforementioned lumbar spine correction, The thoracic kyphosis compensatory response to the lumbar spine correction, or The cervical lordosis compensatory response to the aforementioned lumbar spine correction, The computer implementation method according to claim 1, including the method described in claim 1.

7. The first multidimensional image data is acquired from a computed tomography (CT) scan or a magnetic resonance imaging (MRI) scan, and the second multidimensional image data is acquired from an X-ray scan. The computer implementation method according to claim 1.

8. A computer implementation method for simulating spinal load on a patient, Receiving first multidimensional image data of an anatomical region of a patient in a first load state, Receiving a second multidimensional image data of at least a portion of the anatomical region in a second load state, Based on the first and second multidimensional image data, one or more loading effects on at least one anatomical element in the anatomical region are determined, Based on the one or more load effects that correlate with the preoperative load state, a three-dimensional virtual model of the patient is generated to simulate the postoperative load state, Computer implementation methods, including those mentioned above.

9. The aforementioned three-dimensional virtual model is A first set of anatomical element models having high-fidelity surface topology for designing one or more patient-specific implants, A second set of anatomical element models for measuring spinal metrics that predict surgical outcomes related to the implantation of one or more patient-specific implants, The second set of anatomical element models includes, and one or more of the anatomical element models have fewer feature data than all or part of the anatomical element models in the first set of anatomical element models. The computer implementation method according to claim 8.

10. The method further includes arranging, oriented, and / or scaling the anatomical element models of the first set of anatomical element models so as to be incorporated into the second set of anatomical element models to generate the three-dimensional virtual model. The computer implementation method according to claim 9.

11. To generate an X-ray fidelity anatomical element model based on one or more standing X-ray images of the second multidimensional image data, To generate a tomographic fidelity anatomical element model based on a series of tomographic images of the first multidimensional image data, The X-ray fidelity anatomical element model has a resolution below threshold fidelity, and the tomographic fidelity anatomical element model has a fidelity above threshold fidelity. The computer implementation method according to claim 9.

12. The three-dimensional virtual model is a three-dimensional multi-domain simulation model of the patient's spine, and the computer implementation method is Based on the first multidimensional image data, a three-dimensional multi-region spine model of the patient is generated, To generate a three-dimensional partial spine model that corresponds to the corresponding region of the spine captured in the second multidimensional image data, The three-dimensional multi-domain simulation model is generated by combining the anatomical elements of the three-dimensional partial spine model with the three-dimensional multi-domain spine model. The computer implementation method according to claim 8, further comprising:

13. The computer implementation method further includes replacing the low-fidelity anatomical element model of the three-dimensional multi-region spine model with the corresponding high-fidelity anatomical element of the three-dimensional partial spine model. The computer implementation method according to claim 8.

14. The three-dimensional multi-region spine model includes the cervical and thoracic regions of the patient's spine, and the three-dimensional partial spine model includes a lumbar region model having surface topology data for designing one or more implants. The computer implementation method according to claim 12.

15. The first multidimensional image data shows at least the thoracic and cervical regions of the patient's spine, the second multidimensional image data shows the lumbar region of the patient's spine, and the computer implementation method is Based on the resolution level of the first multidimensional image data, a virtual lumbar anatomical element corresponding to the anatomical element shown in the first multidimensional image data is modeled. Based on the resolution level of the second multidimensional image data, virtual anatomical elements of the thoracic region and the cervical region that correspond to the anatomical elements shown in the second multidimensional image data are modeled. The computer implementation method according to claim 8, further comprising:

16. The patient receives a third multidimensional image data in a third load state that corresponds to the second load state, To generate a virtual model for replacing anatomical elements to be placed within the aforementioned three-dimensional virtual model, The computer implementation method according to claim 8, further comprising:

17. This further includes arranging anatomical elements in the anatomical region based on the one or more load effects described above. The computer implementation method according to claim 8.

18. Identifying the first image type of the first multidimensional image data, The first load state is determined based on the first image type of the first multidimensional image data, To determine one or more anatomical metrics affected by the first load state, Using the first multidimensional image data, one or more anatomical metrics of the anatomical region are measured, and the measured one or more anatomical metrics are used to generate the three-dimensional virtual model of the spine. The computer implementation method according to claim 8, further comprising:

19. Identifying the first image type of the first multidimensional image data, Identifying the second image type of the second multidimensional image data, Selecting a mapping routine based on the first image type, the second image type, and the anatomical elements in the anatomical region, The computer implementation method according to claim 8, further comprising:

20. The first image type is a CT scan, The second image type is X-ray imaging. The mapping routine is configured to map anatomical features in a CT scan to the same anatomical features in an X-ray, and to determine one or more load effects in the anatomical region based on the differences in the position of the same anatomical features due to the load during the CT scan and X-ray. The computer implementation method according to claim 19.

21. The three-dimensional virtual model of the spine has a rendered virtual surface, and the rendered virtual surface is configured for designing one or more implant models whose contours match those of the rendered virtual surface, and the three-dimensional virtual model is generated by combining two-dimensional simultaneous orthogonal X-ray data and three-dimensional MRI data to generate an anatomically loaded 3D spine model. The computer implementation method according to claim 8.

22. A computer implementation method for simulating spinal load on a patient, Receiving the first multidimensional image data of the patient's anatomical region, Receiving a second multidimensional image data of at least a portion of the anatomical region, To generate a high-fidelity 3D virtual model of the aforementioned anatomical region, The high-fidelity 3D virtual model includes, A first set of anatomical elements generated based on the first multidimensional image data, A second set of anatomical elements generated based on the second multidimensional image data, Includes, The first set of anatomical elements has a first fidelity for obtaining one or more measurements of the anatomical region. The second set of anatomical elements has a second fidelity for designing one or more implants that fit the second set of anatomical elements. Computer implementation method.

23. The first fidelity is below the threshold fidelity, the second fidelity is above the threshold fidelity for implant design, and the second fidelity is below the threshold fidelity. The method according to claim 22.

24. The first set of anatomical elements includes at least one of the patient's cervical, thoracic, or lumbar vertebrae. The method according to claim 22.

25. The first set of anatomical elements represents the cervical and thoracic vertebrae of the patient, and the second set of anatomical elements represents the lumbar vertebrae of the patient. The method according to claim 22.

26. The above-mentioned high-fidelity 3D virtual model is used to simulate one or more lumbar spine corrections, In response to one or more lumbar spine adjustments, it generates at least one of the following: a lumbar compensatory reaction, a pelvic tilt compensatory reaction, a thoracic compensatory reaction, or a cervical compensatory reaction. The method according to claim 22, further comprising:

27. To arrange the second set of anatomical elements within the high-fidelity three-dimensional virtual model, the method further includes applying an imaging modality correction routine to the second set of anatomical elements. The method according to claim 22.

28. To prepare a first set of images of the patient, which includes images having a first resolution, and a second set of images of the patient, which includes images having a second resolution different from the first resolution, A surgical planning platform configured to generate a three-dimensional virtual model of the patient's biological structure, Based on the first resolution, a series of available first tasks are assigned to a first region of the three-dimensional virtual model generated using the first set of images, Based on the second resolution, a series of available second tasks are assigned to a second region of the three-dimensional virtual model generated using the second set of images. Obtaining surgical planning information from the surgical planning platform, Receiving a first output from one or more first tasks executed using the first area of ​​the first task, Receiving a second output from one or more first tasks executed using the second area of ​​the first task, Methods that include...

29. The image in the first image set is an X-ray image, and the image in the second image set is a tomographic scan. The method according to claim 28.

30. The three-dimensional virtual model is a spine model, and the available first task includes determining at least one of a thoracic kyphosis compensatory response or a cervical lordosis compensatory response in response to one or more spinal corrections in the second region. The method according to claim 28.

31. The three-dimensional virtual model is a spine model, and the available second task includes determining a pelvic tilt compensatory response in response to one or more spinal corrections at the lumbar level of the second region. The method according to claim 28.

32. The aforementioned series of available first tasks include determining multilevel spinal correction metrics measured at a level above the lumbar region, The aforementioned series of available second tasks are: Designing at least one of one or more spinal implants to be implanted in the lumbar region, To measure spinal metrics along the lumbar spine region, Including at least one of the following, The method according to claim 28.

33. The three-dimensional virtual model represents the corrected spine of the patient. The method according to claim 28.

34. The first region of the three-dimensional virtual model includes a virtual anatomical element having X-ray imaging feature details present in the X-ray image in the first image set, The second region of the three-dimensional virtual model includes a virtual anatomical element having tomographic feature details present in the tomographic images in the second set of images. The method according to claim 28.

35. To obtain a first patient image set and a second patient image set, The surgical planning platform generates a three-dimensional high-fidelity virtual model of the patient's biological structure based on the first and second sets of patient images, Using the surgical planning platform, a first task is performed on the first region of the three-dimensional virtual model based on a first fidelity level corresponding to a first image detail of the patient images in the first patient image set. Using the surgical planning platform, a second task is performed on the second region of the three-dimensional virtual model, based on a second fidelity level corresponding to a second image detail level of the patient images in the second patient image set. Methods that include...

36. The first task includes designing at least one implant surface that matches the patient surface, The surgical planning platform restricts the tasks available for each of the multiple regions of the three-dimensional high-fidelity virtual model based on the resolution level of that region appropriate for the available tasks. The method according to claim 35.

37. The further includes assigning usage constraints to the three-dimensional high-fidelity virtual model based on a fidelity analysis of the three-dimensional high-fidelity virtual model, The method according to claim 35.

38. The first and second tasks are subject to the usage constraints. The method according to claim 35.

39. The further includes displaying the usage constraints to the user and displaying the fidelity-dependent tasks that can be performed using the three-dimensional high-fidelity virtual model, The method according to claim 35.

40. The further includes determining at least one of the first or second tasks based on at least one fidelity parameter of the three-dimensional multi-fidelity virtual model, The method according to claim 35.

41. A computer implementation method for medical imaging data, To acquire first multidimensional image data of an anatomical region of a patient in a first load state, From the first multidimensional image data, multiple regions corresponding to multiple anatomical elements of the spine are determined, Identifying a first set of landmarks in each of the aforementioned multiple regions, To acquire a second multidimensional image data of the anatomical region, including the spine of the patient in a second load state, Identifying a second landmark set corresponding to the first landmark set from the second multidimensional image data, By aligning the corresponding anatomical elements between the first multidimensional image data and the second multidimensional image data, a aligned multidimensional image data of the patient's spine is obtained. To generate a design for a medical implant based on spinal and pelvic parameters measured from the aforementioned aligned multidimensional image data, The medical implant is manufactured by transmitting the design of the medical implant to a manufacturing device. Computer implementation methods, including those mentioned above.

42. The first multidimensional image data is collected in three dimensions, and the second multidimensional image data is collected in two dimensions. The computer implementation method according to claim 41.

43. The aforementioned aligned multidimensional image data is a three-dimensional image. The computer implementation method according to claim 41.

44. Acquiring the aligned multidimensional image data includes correcting the positions of the multiple anatomical elements in the first multidimensional image data according to the positions of the multiple anatomical elements in the second multidimensional image data. The computer implementation method according to claim 41.

45. Determining the load state mapping based on the first and second load states, The load state mapping is performed between the first multidimensional image data and the second multidimensional image data to generate the aligned multidimensional image data, The computer implementation method according to claim 41, further comprising:

46. Identifying the first set of landmarks in the region in response to the region being the lumbar spine is, The inferior surface of the spinous process of the lumbar vertebra, The upper surface of the spinous process of the lumbar vertebra, The left side of the transverse process of the lumbar vertebra, The right surface of the transverse process of the lumbar vertebra, A computer implementation method according to claim 41, which includes identifying a

47. Aligning the corresponding anatomical elements between the first multidimensional image data and the second multidimensional image data is: From three-dimensional (3D) image data, a first image data is formed along a sagittal view that divides the sacrum into two equal parts. A second image data including the sacrum along the sagittal view from the two-dimensional (2D) image data is projected onto the first image data. The first image data is scaled, rotated, and / or translated so that the sacrum from the first image data overlaps with the sacrum in the second image data. The computer implementation method according to claim 41, further comprising aligning the sacrum between the 3D image data and the 2D image data.

48. Aligning the corresponding anatomical elements is The first alignment operation of the lumbar vertebrae is performed by moving the anterior and posterior landmarks of the lumbar vertebrae in the 3D image data to match the same landmarks in the second image data, which includes the lumbar vertebrae, from the 2D image data. From the aforementioned 3D image data, a third image data is formed along a coronal view extending across the sacrum. A fourth image data including the sacrum along the coronal view from the 2D image data is projected onto the third image data. The first image data is scaled or translated so that the sacrum from the third image data overlaps with the sacrum in the second image data. The second alignment operation of the lumbar vertebrae is performed by moving the anterior and posterior landmarks of the lumbar vertebrae to match the same landmarks in the fourth image data, which includes the lumbar vertebrae, from the 2D image. This includes aligning each lumbar vertebra between the 3D image data and the 2D image data, which is performed in response to aligning the sacrum by the above, The computer implementation method according to claim 47.

49. Adjusting one or more other landmarks of the lumbar vertebrae by the same amount used to move the anterior and posterior landmarks of the lumbar vertebrae in the 3D image data for the first alignment operation, Adjusting one or more other landmarks of the lumbar vertebrae by the same amount used to move the anterior and posterior landmarks of the lumbar vertebrae in the 3D image data for the second alignment operation, The computer implementation method according to claim 48, further comprising:

50. The first multidimensional image data is acquired from a computed tomography (CT) scan or a magnetic resonance imaging (MRI) scan, and the second multidimensional image data is acquired from an X-ray scan. The computer implementation method according to claim 41.

51. A non-temporary computer-readable storage medium for storing instructions, wherein the instructions, when executed by a computer system, To acquire first multidimensional image data of an anatomical region of a patient in a first load state, From the first multidimensional image data, multiple regions corresponding to multiple anatomical elements of the spine are determined, Identifying a first set of landmarks in each of the aforementioned multiple regions, To acquire a second multidimensional image data of the anatomical region, including the spine of the patient in a second load state, Identifying a second landmark set corresponding to the first landmark set from the second multidimensional image data, By aligning the corresponding anatomical elements between the first multidimensional image data and the second multidimensional image data, a aligned multidimensional image data of the patient's spine is obtained. To generate a design for a medical implant based on spinal and pelvic parameters measured from the aforementioned aligned multidimensional image data, The medical implant is manufactured by transmitting the design of the medical implant to a manufacturing device. The computer system is made to perform a method including the following: Non-temporary computer-readable storage medium.

52. The first multidimensional image data is collected in three dimensions, and the second multidimensional image data is collected in two dimensions. The non-temporary computer-readable storage medium according to claim 51.

53. The aforementioned aligned multidimensional image data is a three-dimensional image. The non-temporary computer-readable storage medium according to claim 51.

54. Acquiring the aligned multidimensional image data includes correcting the positions of the multiple anatomical elements in the first multidimensional image data according to the positions of the multiple anatomical elements in the second multidimensional image data. The non-temporary computer-readable storage medium according to claim 51.

55. Determining the load state mapping based on the first and second load states, The load state mapping is performed between the first multidimensional image data and the second multidimensional image data to generate the aligned multidimensional image data, A non-temporary computer-readable storage medium according to claim 51, further comprising:

56. Identifying the first set of landmarks in the region in response to the region being the lumbar spine is, The inferior surface of the spinous process of the lumbar vertebra, The upper surface of the spinous process of the lumbar vertebra, The left side of the transverse process of the lumbar vertebra, The right surface of the transverse process of the lumbar vertebra, A non-temporary computer-readable storage medium according to claim 51, comprising identifying a

57. Aligning the corresponding anatomical elements between the first multidimensional image data and the second multidimensional image data is: From three-dimensional (3D) image data, a first image data is formed along a sagittal view that divides the sacrum into two equal parts. A second image data including the sacrum along the sagittal view from the two-dimensional (2D) image data is projected onto the first image data. The first image data is scaled, rotated, and / or translated so that the sacrum from the first image data overlaps with the sacrum in the second image data. The non-temporary computer-readable storage medium according to claim 51, comprising aligning the sacrum between the 3D image data and the 2D image data.

58. Aligning the corresponding anatomical elements is The first alignment operation of the lumbar vertebrae is performed by moving the anterior and posterior landmarks of the lumbar vertebrae in the 3D image data to match the same landmarks in the second image data, which includes the lumbar vertebrae, from the 2D image data. From the aforementioned 3D image data, a third image data is formed along a coronal view extending across the sacrum. A fourth image data including the sacrum along the coronal view from the 2D image data is projected onto the third image data. The first image data is scaled or translated so that the sacrum from the third image data overlaps with the sacrum in the second image data. The second alignment operation of the lumbar vertebrae is performed by moving the anterior and posterior landmarks of the lumbar vertebrae to match the same landmarks in the fourth image data, which includes the lumbar vertebrae, from the 2D image. This includes aligning each lumbar vertebra between the 3D image data and the 2D image data, which is performed in response to aligning the sacrum by the above, The non-temporary computer-readable storage medium according to claim 57.

59. The aforementioned method, Adjusting one or more other landmarks of the lumbar vertebrae by the same amount used to move the anterior and posterior landmarks of the lumbar vertebrae in the 3D image data for the first alignment operation, Adjusting one or more other landmarks of the lumbar vertebrae by the same amount used to move the anterior and posterior landmarks of the lumbar vertebrae in the 3D image data for the second alignment operation, A non-temporary computer-readable storage medium according to claim 58, further comprising:

60. The first multidimensional image data is acquired from a computed tomography (CT) scan or a magnetic resonance imaging (MRI) scan, and the second multidimensional image data is acquired from an X-ray scan. The non-temporary computer-readable storage medium according to claim 51.

61. A system for medical imaging data comprising one or more processors and a memory for storing instructions, wherein the instructions, when executed by the one or more processors, To acquire first multidimensional image data of an anatomical region of a patient in a first load state, From the first multidimensional image data, multiple regions corresponding to multiple anatomical elements of the spine are determined, Identifying a first set of landmarks in each of the aforementioned multiple regions, To acquire a second multidimensional image data of the anatomical region, including the spine of the patient in a second load state, Identifying a second landmark set corresponding to the first landmark set from the second multidimensional image data, By aligning the corresponding anatomical elements between the first multidimensional image data and the second multidimensional image data, a aligned multidimensional image data of the patient's spine is obtained. To generate a design for a medical implant based on spinal and pelvic parameters measured from the aforementioned aligned multidimensional image data, The medical implant is manufactured by transmitting the design of the medical implant to a manufacturing device. A system that causes the aforementioned system to perform the above action.

62. The first multidimensional image data is collected in three dimensions, and the second multidimensional image data is collected in two dimensions. The system according to claim 61.

63. The aforementioned aligned multidimensional image data is a three-dimensional image. The system according to claim 61.

64. Acquiring the aligned multidimensional image data includes correcting the positions of the multiple anatomical elements in the first multidimensional image data according to the positions of the multiple anatomical elements in the second multidimensional image data. The system according to claim 61.

65. Determining the load state mapping based on the first and second load states, The load state mapping is performed between the first multidimensional image data and the second multidimensional image data to generate the aligned multidimensional image data, The system according to claim 61, further comprising:

66. Identifying the first set of landmarks in the region in response to the region being the lumbar spine is, The inferior surface of the spinous process of the lumbar vertebra, The upper surface of the spinous process of the lumbar vertebra, The left side of the transverse process of the lumbar vertebra, The right surface of the transverse process of the lumbar vertebra, The system according to claim 61, which includes identifying a

67. Aligning the corresponding anatomical elements between the first multidimensional image data and the second multidimensional image data is: From three-dimensional (3D) image data, a first image data is formed along a sagittal view that divides the sacrum into two equal parts. A second image data including the sacrum along the sagittal view from the two-dimensional (2D) image data is projected onto the first image data. The first image data is scaled, rotated, and / or translated so that the sacrum from the first image data overlaps with the sacrum in the second image data. The system according to claim 61, further comprising aligning the sacrum between the 3D image data and the 2D image data.

68. Aligning the corresponding anatomical elements is The first alignment operation of the lumbar vertebrae is performed by moving the anterior and posterior landmarks of the lumbar vertebrae in the 3D image data to match the same landmarks in the second image data, which includes the lumbar vertebrae, from the 2D image data. From the aforementioned 3D image data, a third image data is formed along a coronal view extending across the sacrum. A fourth image data including the sacrum along the coronal view from the 2D image data is projected onto the third image data. The first image data is scaled or translated so that the sacrum from the third image data overlaps with the sacrum in the second image data. The second alignment operation of the lumbar vertebrae is performed by moving the anterior and posterior landmarks of the lumbar vertebrae to match the same landmarks in the fourth image data, which includes the lumbar vertebrae, from the 2D image. This includes aligning each lumbar vertebra between the 3D image data and the 2D image data, which is performed in response to aligning the sacrum by the above, The system according to claim 67.

69. Adjusting one or more other landmarks of the lumbar vertebrae by the same amount used to move the anterior and posterior landmarks of the lumbar vertebrae in the 3D image data for the first alignment operation, Adjusting one or more other landmarks of the lumbar vertebrae by the same amount used to move the anterior and posterior landmarks of the lumbar vertebrae in the 3D image data for the second alignment operation, The system according to claim 68, further comprising:

70. The first multidimensional image data is acquired from a computed tomography (CT) scan or a magnetic resonance imaging (MRI) scan, and the second multidimensional image data is acquired from an X-ray scan. The system according to claim 61.

Images