Method and system for spinal marking in MRI

Abstract

A method and system for processing magnetic resonance images of a spine includes receiving MR image data comprising a plurality of spine images of a patient, the plurality of spine images including a spine image for each of a plurality of sagittal slices; and then processing each spinal image with a deep learning (DL) model, where the DL model is trained to generate a plurality of labeled images for each spinal image, where the plurality of labeled images are each labeled with a different predetermined set of spinal segment labels. The plurality of marked images are combined into a marked spinal image for each of the plurality of slices, wherein each marked spinal image has a spinal segment tag for each of the predetermined plurality of spinal segments. A marked MR image is then generated based on the marked spinal image.

Description

Background Technology

[0001] This disclosure generally relates to systems and methods for magnetic resonance imaging (“MRI”). More specifically, this disclosure relates to systems and methods for performing calculations to automatically identify and label spinal segments in magnetic resonance (MR) images.

[0002] MRI is commonly used to obtain information about a patient’s internal physiology, including for imaging the brain, spine, heart, and other parts or tissues inside the patient’s body (or anywhere on the patient’s body).

[0003] MRI uses the nuclear magnetic resonance (“NMR”) phenomenon to produce images. When material, such as human tissue, is subjected to a uniform magnetic field (such as the so-called master magnetic field (polarization field B0) generated by an MRI system), the individual magnetic moments of the atomic nuclei in the tissue attempt to align with this B0 field, but precess around it in a random order at their characteristic Larmor frequencies. If the material or tissue is subjected to a magnetic field (excitation field B1) that is in the xy-plane and close to the Larmor frequency, the net alignment magnetic moment, or “longitudinal magnetization” Mz, can be rotated or “deflected” into the xy-plane to produce a net transverse magnetic moment Mt. After the excitation signal B1 terminates, a signal is emitted by the excitation spin, and this signal can be received and processed to form an image.

[0004] When these signals are used to generate images, the magnetic field gradient (G) is employed. x G y and G z Typically, the area to be imaged is scanned by a series of measurement cycles, where these gradients (sometimes called readout gradients) vary depending on the specific localization method used. The resulting set of received signals is digitized and processed to reconstruct the image using reconstruction techniques. Summary of the Invention

[0005] This summary is provided to introduce a series of concepts that will be further described in the detailed embodiments below. This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to help limit the scope of the claimed subject matter.

[0006] In one aspect of this disclosure, a method for processing magnetic resonance images of the spine includes receiving MR image data comprising multiple spine images of a patient, the multiple spine images including spine images for each of a plurality of slices; and processing each spine image with a deep learning (DL) model, wherein the DL model is trained to generate multiple labeled images for each spine image, wherein each of the multiple labeled images is labeled with a different predetermined set of spine segment labels. The multiple labeled images are combined into a labeled spine image for each of the multiple slices, wherein each labeled spine image has a spine segment label for each of the predetermined plurality of spine segments. A labeled MR image is then generated based on the labeled spine image.

[0007] In one implementation, the plurality of labeled spine images generated for each spine image include at least a first labeled image and a fully labeled image, wherein the first labeled image includes only a first spine segment label for a predetermined anchoring segment, and the fully labeled image includes a spine segment label for each of the predetermined plurality of spine segment labels.

[0008] In another implementation, each spinal segment label is located at the centroid of the vertebrae of each spinal segment in each spinal image for a predetermined set of spinal segments.

[0009] In another embodiment, the method further includes receiving either a neck name or a waist name. In response to receiving a neck name, the DL model generates at least a first labeled neck image with an S1 label and a fully labeled neck image with S1, L5, L4, L3, L2, L1, and T12 labels. In response to receiving a waist name, the DL model generates at least a first labeled waist image with a C1 label and a fully labeled waist image with C1, C2, C3, C4, C5, C6, and C7 labels.

[0010] In another embodiment, it further includes creating an ML model by training a convolutional neural network to receive a spine image and generate a labeled image for each of a predetermined plurality of spine segments between a first assigned segment and a last assigned segment, wherein the first labeled image includes only a first spine segment label, and wherein the complete labeled image includes spine segment labels for each of the plurality of spine segments.

[0011] In another implementation, the DL model includes a trained image segmentation model.

[0012] In another implementation, the DL model includes a first U-Net trained to locate and label the centroid of the vertebrae for each of a predetermined set of spinal segments.

[0013] In another implementation, the DL model is a W-Net and includes a second U-Net trained to refine the shape of the labels in each spine image.

[0014] In another embodiment, it further includes aligning multiple vertebral segment labels across multiple slices in a transected spinal image to identify a 3D label volume for each of the predetermined multiple vertebral segments.

[0015] In another embodiment, at least one spinal segment label in at least one of the multiple slices is adjusted such that the 3D label volume for each of the predetermined multiple spinal segments is continuous across the multiple slices.

[0016] In another embodiment, the method further includes aligning multiple spine segment labels across multiple slices in a labeled spine image to identify multiple 3D volumes covering the spine segment labels, and then determining the distance between each of the multiple 3D volumes. If the distance between two 3D volumes is less than a threshold distance, the method further includes combining the two 3D volumes together to form a 3D label volume for one of the multiple spine segments.

[0017] In another embodiment, the method further includes using an image segmentation model to process MR image data, the image segmentation model being trained to identify vertebral masks that label pixels associated with each vertebra in the MR image data, comparing the vertebral masks with multiple spinal segment labels to detect missing labels, and generating at least one spinal segment label to be added to the multiple spinal segment labels based on the vertebral masks.

[0018] In another implementation, the missing label in the spinal segment label is a spinal segment label for one of a predetermined plurality of spinal segments.

[0019] In another embodiment, the method further includes processing MR image data using an image segmentation model trained to identify vertebral masks labeled with pixels associated with each vertebra in the MR image data and / or intervertebral disc masks labeled with pixels associated with each intervertebral disc in the MR image data; comparing the vertebral masks and / or intervertebral disc masks with multiple spinal segment labels to detect fused vertebrae; and generating a user prompt requesting a clinician to confirm the fused vertebrae.

[0020] Various other features, objects, and advantages of the invention will become apparent from the following description taken in conjunction with the accompanying drawings. Attached Figure Description

[0021] This disclosure is described with reference to the following figures.

[0022] Figure 1 This is a schematic diagram of an MRI system according to an exemplary embodiment.

[0023] Figure 2 This is a flowchart illustrating an exemplary method for labeling MR images of the spine according to one embodiment of the present disclosure.

[0024] Figure 3 An exemplary trained DL model with input and output MR images of a spine is illustrated according to one embodiment of the present disclosure.

[0025] Figure 4 An exemplary model architecture of a trained DL model according to one embodiment of the present disclosure is illustrated.

[0026] Figures 5 to 6 Exemplary steps and functions for labeling MR images of the spine according to one embodiment of the present disclosure are illustrated.

[0027] Figure 7 This is a flowchart illustrating another exemplary method for labeling MR images of the spine according to an embodiment of the present disclosure.

[0028] Figure 8 These are exemplary labeled MR images illustrating the output of an embodiment according to this disclosure. Detailed Implementation

[0029] In this description, certain terms are used for the purpose of brevity, clarity, and ease of understanding. No unnecessary limitations should be inferred from these terms beyond the requirements of the prior art, as they are used for descriptive purposes only and are intended to be understood in a broad sense.

[0030] As used herein, unless otherwise limited or restricted, discussions of a particular orientation are provided by way of example only for a particular embodiment or related illustration. For example, discussions of “top,” “bottom,” “front,” “back,” “left,” “right,” “horizontal,” “vertical,” and “longitudinal” features and / or relative movements (e.g., “upward” and “downward” movement) are generally intended only to describe the orientation of such features relative to a reference frame for a particular example or illustration. Accordingly, for example, in some arrangements or embodiments, a “top” feature may sometimes be positioned below a “bottom” feature (etc.). Additionally or alternatively, embodiments can be arranged in different orientations such that the “top” and “bottom” features are arranged horizontally relative to each other, for example, in a “left-to-right” orientation.

[0031] The terms “comprising,” “including,” or “having,” as used herein, and variations thereof, are intended to cover the elements listed thereafter and their equivalents, as well as any additional elements. An embodiment described as “comprising,” “including,” or “having” certain elements is also considered to be “substantially composed of those certain elements” and “composed of those certain elements.”

[0032] The inventors have recognized that current methods and systems for the automatic identification and labeling of spinal segments in MR images are inaccurate and unreliable, often producing incorrect labels, where spinal segments are mislabeled, labeled multiple times, or vertebrae in the image are omitted and therefore not labeled at all. Sometimes artifacts or outliers are incorrectly labeled as part of the spinal image. Therefore, the inventors have strived to develop improved systems and methods for the automatic detection and labeling of spinal segments in MR spinal images.

[0033] The disclosed method and system are configured to receive MR image data and use a trained deep learning (DL) model to process spinal images for each of multiple slices (such as multiple 2D slice images) to generate multiple labeled images for each spinal image at each slice. The DL model is configured to generate at least two labeled images for each image at each slice, wherein each of the at least two labeled images has a different predetermined set of spinal segment labels. The multiple labeled images are then combined into a labeled spinal image for each of the multiple slices, wherein each labeled spinal image has a spinal segment label for each of the predetermined multiple spinal segments, and a labeled MR image is generated based on the labeled spinal image. For example, the labeled spinal image may be a 2D image slice. Alternatively or additionally, the disclosed method and system may be configured to receive image data, which is 3D MRI data or 3D computed tomography (CT) data. Thus, the DL model is configured to more robustly identify and generate spinal segment labels.

[0034] Multiple labeled images generated for each original slice image include at least two images with different spinal segment labels, such as a first labeled image with a first labeled segment (e.g., the bottom lumbar segment S1 or the top cervical segment C1) and a fully labeled image including the spinal segment labels of each of a predetermined set of spinal segment labels to be recognized by the DL model. In some embodiments, the DL model is configured to generate multiple labeled images, the number of which is equal to the number of spinal segments in a predetermined set of spinal segment labels. For example, the DL model can be trained to recognize cervical segments, such as including C1-C7. In one embodiment, the DL model can be trained to generate seven images, each of which contains a different predetermined set of spinal segment labels between C1 and C7, the seven images including a first image containing a C1 label, a second image containing C1 and C2 labels, a third image containing C1, C2, and C3 labels, and so on, up to a seventh image containing seven labels for each of C1 to C7. Alternatively or additionally, a DL model can be trained to identify lumbar segments. In one implementation, the lumbar segments may include S1 to T12, and seven images are generated, each of which contains a different predetermined set of spinal segment labels between S1 and T12 (e.g., a first image with only an S1 label, a second image with both an S1 label and an L1 label, and so on, up to a seventh image with labels for each of the S1 to T12 labels).

[0035] In some implementations, multiple spinal segment labels from multiple slices across a labeled spinal image are combined, and multiple 3D volumes covering each spinal segment label are identified. For example, the multiple 3D volumes may include a 3D volume for each of a predetermined plurality of spinal segments, wherein the depth of the 3D volume is based on the multiple slices. The 3D volumes can be used as processing tools to correct the labels, including merging broken labels, filling in missing labels, identifying fused vertebrae, propagating spinal segment labels to additional segments, and other functions. In one example, the system and method are configured to determine the distance between each of the multiple 3D volumes and evaluate the correctness of the labels based on that distance. If the distance between two 3D volumes in the multiple 3D volumes is less than a threshold distance, the two 3D volumes can be combined to form a 3D label volume for one spinal segment in the multiple spinal segments. For example, the threshold distance may be a predetermined number of pixels or a distance measurement, such as millimeters.

[0036] Alternatively or additionally, the system may include one or more additional DL models trained to process the same MR image data and generate vertebral masks and / or intervertebral disc masks. Information from the vertebral masks and / or intervertebral disc masks (such as those generated by the trained image segmentation models) can be used for further correction of spinal segment labels. For example, the system may be configured to perform a comparison between pixel locations in the masks and pixel locations of spinal segment labels to identify incorrect labels, fused vertebrae, and / or missing labels. In various specific embodiments of such implementations, missing segments may include unidentified segments among a predetermined plurality of spinal segments and / or may include additional segments above or below a predetermined plurality of spinal segments. Thus, the system is configured to self-correct errors generated in the spinal segment labeling performed by the aforementioned DL models.

[0037] refer to Figure 1 A schematic diagram of an exemplary MRI system 100 is shown according to an embodiment. The operation of the MRI system 100 is controlled by an operator workstation 110, which includes an input device 114, a control panel 116, and a display 118. The input device 114 may be a joystick, keyboard, mouse, trackball, touch-activated screen, voice control, or any similar or equivalent input device. The control panel 116 may include a keyboard, touch-activated screen, voice control, buttons, sliders, or any similar or equivalent control device. The operator workstation 110 is coupled to and communicates with a computer system 120, which enables the operator to control the generation and viewing of images on the display 118. The computer system 120 includes multiple components that communicate with each other via electrical and / or data connections 122. The computer system connection 122 may be a direct wired connection, a fiber optic connection, a wireless communication link, etc. The components of the computer system 120 include a central processing unit (CPU) 124, a memory 126, and an image processor 128, the memory of which may include frame buffers for storing image data. In an alternative implementation, image processor 128 may be replaced by image processing functions implemented in CPU 124. Computer system 120 may be connected to archival media devices, permanent or backup storage, or a network. Computer system 120 is coupled to and communicates with a separate MRI system controller 130.

[0038] The MRI system controller 130 includes a set of components that communicate with each other via electrical and / or data connections 132. The MRI system controller connection 132 can be a direct wired connection, a fiber optic connection, a wireless communication link, etc. The components of the MRI system controller 130 include a CPU 131, a pulse generator 133, a transceiver 135, a memory 137, and an array processor 139. The pulse generator is coupled to and communicates with the operator workstation 110. In an alternative embodiment, the pulse generator 133 may be integrated into the resonant assembly 140 of the MRI system 100. The MRI system controller 130 is coupled to and receives commands from the operator workstation 110 to indicate the MRI scan sequence to be performed during an MRI scan. The MRI system controller 130 is also coupled to and communicates with a gradient driver system 150, which is coupled to a gradient coil assembly 142 to generate a magnetic field gradient during an MRI scan.

[0039] The pulse generator 133 can also receive data from a physiological acquisition controller 155, which receives signals from multiple different sensors connected to the subject or patient 170 undergoing an MRI scan. These signals include electrocardiogram (ECG) signals from electrodes attached to the patient 170. Finally, the pulse generator 133 is coupled to and communicates with a scan room interface system 145, which receives signals from various sensors associated with the state of the resonant assembly 140. The scan room interface system 145 is also coupled to and communicates with a patient positioning system 147, which sends and receives signals to control the movement of the stage 171. The stage 171 can be controlled to move the patient into and out of the central section 146 and to move the patient to a desired position within the central section 146 for an MRI scan.

[0040] MRI system controller 130 provides gradient waveforms to gradient driver system 150, the gradient driver system including G X Amplifier, G Y Amplifier and G Z Amplifiers, etc. Each G X G Y and G ZA gradient amplifier excites a corresponding gradient coil in gradient coil assembly 142 to generate a magnetic field gradient for spatial encoding of MR signals during MRI scans. Gradient coil assembly 142 is included within resonant assembly 140, which also includes a superconducting magnet with a superconducting coil 144 that provides a uniform longitudinal magnetic field B0 in operation throughout the central portion 146 or the open cylindrical imaging volume surrounded by resonant assembly 140. Resonant assembly 140 also includes an RF body coil 148 that provides a transverse magnetic field B1 in operation, which is substantially perpendicular to B0 throughout the central portion 146. Resonant assembly 140 may also include an RF surface coil 149 for imaging different anatomical structures of a patient undergoing an MRI scan. RF body coil 148 and RF surface coil 149 can be configured to operate in transmit and receive modes, transmit mode, or receive mode.

[0041] The subject or patient undergoing an MRI scan 170 can be positioned within the central portion 146 of the resonance assembly 140. A transceiver 135 in the MRI system controller 130 generates an RF excitation pulse, which is amplified by an RF amplifier 162 and provided to the RF body coil 148 and the RF surface coil 149 via a transmit / receive switch (T / R switch) 164.

[0042] As described above, the RF body coil 148 and RF surface coil 149 can be used to transmit RF excitation pulses and / or receive derived MR signals from a patient undergoing an MRI scan. The derived MR signal emitted by the stimulated nuclei within the patient undergoing an MRI scan can be sensed and received by the RF body coil 148 or RF surface coil 149 and transmitted back to the preamplifier 166 via the T / R switch 164. The amplified MR signal is demodulated, filtered, and digitized in the receiver section of the transceiver 135. The T / R switch 164 is controlled by a signal from the pulse generator 133 to electrically connect the RF amplifier 162 to the RF body coil 148 during transmit mode and to the preamplifier 166 to the RF body coil 148 during receive mode. The T / R switch 164 also enables the RF surface coil 149 to be used in either transmit or receive mode.

[0043] The MR signal sensed and received by the RF coil 148 is digitized by the transceiver 135 and transmitted to the memory 137 in the MRI system controller 130.

[0044] The MR scan is complete when the array of raw k-space data corresponding to the received MR signal has been acquired and temporarily stored in memory 137 until the data is subsequently transformed to create an image. For each image to be reconstructed, the raw k-space data is rearranged into separate k-space data arrays, and each of these separate k-space data arrays is input to array processor 139, which operates to perform a Fourier transform on the data into an array of image data.

[0045] The array processor 139 uses known transformation methods, most commonly Fourier transform, to create images from the received MR signals. These images are transmitted to the computer system 120, where they are stored in memory 126. In response to commands received from the operator workstation 110, the image data may be archived in long-term storage or may be further processed by the image processor 128 and transmitted to the operator workstation 110 for presentation on the display 118.

[0046] In various implementations, components of the computer system 120 and the MRI system controller 130 may be implemented on the same computer system or multiple computer systems.

[0047] Figures 2 to 4 Examples illustrate the method steps and system architecture for identifying spinal segment labels according to embodiments of this disclosure. Figure 2 An exemplary method 200 is described for processing MR images of the spine to generate labeled MR images containing accurate spinal segment labels. At step 202, MR image data is received. The MR image data includes at least one 2D image representing at least one slice, as is standard in MR imaging. In many specific embodiments, the MR data includes multiple images, each representing one of multiple slices, such as sagittal 2D slices, each sagittal 2D slice including an image of the patient's spine along the sagittal plane. At step 204, the MR image data is processed using at least one DL model trained to generate labeled images, wherein one or more spinal segments are labeled within the sagittal slice images of the patient's spine. For example, the DL model is trained to receive each 2D sagittal slice image of the spine, wherein the multiple labeled images are each labeled with a different predetermined set of spinal segment labels. At step 206, the output labeled images are combined and processed using one or more correction algorithms, wherein the correction algorithms are configured to merge broken labels, identify and remove outliers, and / or repair missing labels.

[0048] In some implementations, the correction algorithm is also configured to identify and correct fused vertebrae and / or other physiological abnormalities captured in the images. For example, the correction algorithm may be configured to generate a 3D label volume for each label in the labeling and refine the spinal segment labels based on the 3D volume. Alternatively or additionally, the correction algorithm may include one or more trained DL models configured to process MR image data and / or labeled images to generate vertebral masks and / or intervertebral disc masks, and use those masks to correct and refine the spinal segment labels in the labeled images. The system may also be configured to prompt the user for confirmation and / or correction label input, and to utilize user input as additional input to the correction algorithm in addition to the labeled images.

[0049] In the depicted implementation, the correction algorithm is configured to identify and label fused vertebrae, and performs step 208 to identify whether the correction algorithm has detected any fused vertebrae. If a fused vertebra is detected, step 210 is performed to prompt the user to review and provide input for approval of automatically generated labels (including fusion identifiers), or input for correction of one or more spinal segment labels and / or fusion identifiers in the spinal segment labels. User input is received, which can be approval or correction. If correction user input is received, the correction input is provided to the correction algorithm at step 212. User input can be in any number of forms, such as moving labels and / or selection of pixels representing the location of fused vertebrae or representing the intervertebral disc or another point between two labels.

[0050] Steps 206 and 208 can be repeated using the corrected user input as additional input to the correction algorithm until user approval input is received at step 210. The corrected labeled image is then output at step 214. The corrected image, comprising multiple 2D sagittal images, is stored at step 216 and / or output as labeled MR images, each 2D sagittal image containing spinal segment labels for predetermined multiple spinal segments.

[0051] Figure 3This describes the input, output, and structure of an exemplary DL model configured according to this disclosure. The DL model is a trained segmentation model 310 trained to locate and label predetermined multiple spinal segments in MR images, which may be a predetermined set of lumbar segments and / or a predetermined number of cervical segments. The DL model 310 receives a 2D spinal image 301 for each sagittal slice in the MR image data and generates multiple labeled images 321-327 therefrom. It is well known that MR image data contains multiple images, each representing a different slice depth in a patient. Here, the MR image data is MR image data of the patient's spine, which is typically acquired at multiple sagittal slices across the patient's width. Thus, each slice image captures multiple vertebrae, which may be lumbar segments or cervical segments of the spine, or may be an image capturing both lumbar and cervical regions, such as the entire spine of the patient. Each of the multiple spinal images in the MR image data is provided to the trained DL model, which is trained to generate multiple labeled images for each input spinal image 301.

[0052] As shown in the example, the model is trained to generate multiple labeled images 321-327 for each input spine image including slices of MR image data. Each slice image in the multiple slice images is processed similarly, thus generating multiple sets of labeled images 321-327 for the input MR image data. Each of the multiple labeled images 321-327 is labeled with a different predetermined set of spine segment labels, including a first labeled image and a complete labeled image 327. The first labeled image includes only a first spine segment label, such as for a predetermined anchoring segment at the top or bottom of the spine, while the complete labeled image includes spine segment labels for each of the predetermined multiple spine segments for which the DL model 310 is configured to label.

[0053] Figure 3An embodiment in which the DL model 310 is configured to generate seven labeled images 321-327 is illustrated, including one labeled image for each of seven predetermined spinal segments (S1-T12 in this example) that the DL model is trained to identify. Each of the seven labeled images 321-327 contains a spinal segment label for an anchor segment, which could be S1 for the lumbar image and C1 for the cervical image. The first spinal segment image 321 includes a first spinal segment label 351, and then each subsequent image includes an additional spinal segment label until the last image, the complete labeled image, which contains spinal segment labels for all the multiple spinal segments that the DL model 310 is trained to identify. Thus, the second labeled image 322 includes the first spinal segment label 351 and the second spinal segment label 352; the third labeled image 323 includes the first spinal segment label 351, the second spinal segment label 352, and the third spinal segment label 353, and so on, until the complete labeled image. Here, the complete labeled image is the seventh labeled image 327, which includes first spinal segment labels 351, second spinal segment labels 352, third spinal segment labels 353, fourth spinal segment labels 354, fifth spinal segment labels 355, sixth spinal segment labels 356, and seventh spinal segment labels 357. Here, the spinal image captures the patient's lumbar region, and the predetermined multiple spinal segments are S1, L5, L4, L3, L2, L1, and T12, which correspond to the illustrated spinal segment labels 351-357, respectively.

[0054] The DL model 310 can be any type of image segmentation model trained to label predetermined multiple spinal segments in MR images, which may be a predetermined set of cervical segments and / or a predetermined set of lumbar segments. For example, the DL model 310 can be a U-Net architecture, which is a convolutional network containing consecutive layers utilizing upsampling operators. In some implementations, the DL model 310 may include multiple U-Net architectures cascaded together. In one example, the DL model is a W-Net architecture comprising two U-Net architectures, where the first U-Net acts as an encoder for segmenting the output spinal image, and the second U-Net acts as a decoder for reconstructing the image from the output of the first U-Net.

[0055] Figure 4 An example implementation is illustrated, which is an exemplary architecture of a trained DL model configured to generate multiple labeled spine images for each 2D sagittal slice spine image, as... Figure 3Those illustrated. Here, the trained DL model 301a is a W-Net architecture with size-weighted dice and a per-mask shape encoder. The first U-Net 405 is a U-Net trained to locate and label the centroids of the vertebrae for each vertebra in a predetermined set of spinal segments and generate multiple labeled images (e.g., 321-327) containing spinal segment labels. The first U-Net 405 provides a predicted label mask as output, which, such as containing Figure 2 Each of the predetermined multiple sets of spinal segment labels 351-357 shown and described.

[0056] The output from the first U-Net 405 is provided to the second U-Net 410, which is trained to receive the output of the first U-Net and acts as a decoder providing multiple masks 421-427, which include a mask for each spinal segment label of a predetermined set of spinal segment labels in each of the multiple spinal images 221-227. Therefore, the number of outputs from the second U-Net 410 or the decoder U-Net is consistent with the number of labeled images in the multiple labeled images generated by model 301a. The decoder U-Net 410 can be configured such that each output channel in the output channels for each mask 421-427 is equally weighted, regardless of the number of spinal segment labels in the channel. Therefore, the first channel of the first mask 421, configured to output for a first labeled image with only one spinal segment label, is more weighted on each labeled pixel than the last channel of the mask 427, configured to output for a complete labeled image, so that the focus of model 410 is uniformly balanced across all output channels. In other embodiments, the model 310a architecture may include multiple decoder U-Nets 410, such as one decoder U-Net for each mask 421-427, wherein each decoder U-Net is trained to generate one mask from the corresponding mask 421-427.

[0057] Each output mask 421-427 is provided to a corresponding shape encoder 441-447, which is configured to refine the shape of the label in each spine image. The shape encoders 421-427 may be an additional U-Net configured to refine the shape of the pixel groups identified in each spine segment label, such as adjusting each spine segment label to include a specific number of labels of a specific shape. The outputs of the shape encoders 441-447 are combined to generate the predicted final mask output by the DL model 301a.

[0058] In some embodiments, the system may include multiple DL models 310a, each trained to recognize different predetermined multiple spinal segments. For example, the system may include a lumbar model configured to recognize predetermined multiple lumbar segments and a cervical model configured to recognize a predetermined set of cervical segments. Other embodiments may include a thoracic model configured to recognize thoracic segments, which may be combined with cervical and / or lumbar segments. Alternatively, a model 310a may be configured to recognize all different groups of predetermined multiple spinal segments.

[0059] The output of DL model 310 is provided to at least one correction algorithm executed as part of a correction module configured to correct spinal segment labels across multiple labeled images, including merging broken labels, identifying and removing outliers, and / or repairing missing labels. The correction module includes software instructions stored on a non-transitory computer-readable medium and executable to process the various spinal image data described herein and correct the spinal segment label information generated by DL models 310, 410. For example, the correction module and / or DL ​​models 310, 410 may be stored and executed within the computer system 122 described herein. The correction module may include one or more trained DL models, such as image segmentation models or other image processing models described herein. In some embodiments, the correction module is also configured to identify and correct fused vertebrae and / or other physiological abnormalities captured in the images. For example, the correction module may be configured to generate a 3D label volume for each label in the labels and refine the spinal segment labels based on the 3D volumes. Figures 5 to 6 An exemplary step is illustrated, including generating a 3D label volume based on spinal segment labels across multiple labeled slice images generated from labeled spinal images. The correction module may include programming configured to merge some or all of the labeled images for each slice into a merged labeled image for each slice. In one embodiment, the first and last labeled images for each slice (e.g., ...) are combined... Figure 3In the example, labeled images 321 and 327 are merged together to create a merged labeled image for that slice. The merged labeled image includes all labels for the predetermined multiple spinal segments because the last image (i.e., the complete labeled image) includes all spinal segment labels. Since both the first and last labeled images contain spinal segment labels for the anchoring segment, the merged labeled image will have a double-weighted spinal segment label for the anchoring segment (e.g., S1 for the lumbar image and C1 for the cervical image). Therefore, the anchoring segment can be identified accordingly. In other embodiments, multiple labeled images from different subgroups (e.g., Figure 3 The labeled images 321-327 in the example are used to generate a merged labeled image for each slice. Since all labeled images contain a spinal segment label for the anchoring segment, this label will always have the highest weight in the merged labeled image.

[0060] Then, the merged labeled images (one merged labeled image for each slice in the MR image data slice) are used to generate a 3D label volume for each spinal segment in the labeled spinal segment. Figure 5 This concept is illustrated in which multiple merged labeled images 501a-501n are aligned such that 3D label volumes 551-558 can be identified for each of a predetermined plurality of spinal segments, wherein the 3D label volumes are preferably continuous across multiple slices. Here, each of the merged labeled images 501a-501n for multiple slices (e.g., n slices) contains eight spinal segment labels 541-548. The spinal segment labels 541-548 across all aligned merged labeled images 501a-501n are unified into a set of 3D label volumes 551-558, which includes the 3D label volume 551-558 for each of the spinal segment labels 541-548 in the merged images. Therefore, the shape of the 3D volumes 551-558 is determined by the spinal segment labels. For example, if each of the spinal segment labels 541-548 comprises a group of pixels arranged in a circle, the 3D volume 551-558 will be a roughly cylindrical shape, which roughly extends as the width of the corresponding vertebra captured in various slices of MR image data.

[0061] The 3D map 520 identifies 3D label volumes, which include x-dimensions (e.g., the frequency dimension of MR image data), y-dimensions (e.g., the phase dimension of MR image data), and z-dimensions (e.g., the slice dimension of MR image data). The drawn 3D volumes can be processed to identify and correct problems with spine segment labels generated by the DL model, such as merging broken labels, identifying and removing outliers, and / or filling in missing labels. Figure 6 Two different 3D figures 620a and 620b of the 3D tag volume are shown, illustrating exemplary problems that the correction module is configured to fix.

[0062] 3D labels are analyzed to identify broken labels and outliers and correct those errors. Figure 620a shows a set of 3D label volumes 651a-659a for each of the nine spinal segments. The lowest 3D label volume 651a is a broken label, where 3D label volume 651a was initially split into two parts, 651a' and 651a'". This may be due to missing spinal segment labels at the broken location in a merged labeled image for one or more slices. The correction module can be configured to identify broken labels, for example, based on the distance between 3D volumes 651a' and 651a' and / or the corresponding position of the volume relative to the x-axis and / or y-axis. For example, if the distance between two 3D volumes is less than a threshold distance, the 3D volumes can be combined into a single 3D label volume for a single spinal segment. The threshold distance can vary, for example, depending on the scale of the imaged segment and / or image. As just one example, the threshold distance could be 30 mm or 40 mm, or the corresponding number of pixels. In Figure 620a, the distance between 3D volumes 651a' and 651a" is less than a threshold distance, and therefore they are joined together as a single 3D label volume by padding with intermediate pixels (e.g., interpolation between the two 3D volumes 651a' and 651a"). This will add and / or adjust the spinal segment labels in each of the affected merged labeled images for the corresponding slice.

[0063] Alternatively or additionally, broken labels can be identified by comparing the 3D label volume with the vertebral mask volume generated based on the vertebral mask for each slice in the slice, as described in more detail herein. For example, in the case where two 3D volumes (such as 651' and 651a") appear in a single 3D vertebral mask volume, those 3D volumes can be merged into a single 3D label volume, as just described.

[0064] The same process can be performed on the 3D label volumes shown in Figure 620b, which illustrates a set of 3D label volumes 651b-659b for each of the nine spinal segments. 3D label volume 655b illustrates another broken label that can be repaired by correction algorithms such as threshold distance-based assessment and / or comparison with vertebral mask volumes. Figure 620b also illustrates segmentation outliers, where 3D volume 670b is not associated with any spinal segment and should not be identified. Segmentation outliers can be identified based on various parameters, such as the magnitude of the volume, its relative position and / or distance to other 3D volumes, etc. For example, if a 3D volume (e.g., 370b) meets certain criteria, such as its size being smaller than a threshold value (e.g., less than a threshold volume or a threshold number of pixels), and / or it is not aligned with other 3D labeled volumes 651b-659b along the x-axis and / or y-axis, and / or its distance from any other 3D spinal segment label is greater than a threshold distance, then the 3D volume can be marked as a segmentation outlier and thus removed from the 3D labeled volumes and the labeled image.

[0065] In addition to the exemplary broken and / or incorrect labels, the correction module can be configured to split incorrectly connected labels and / or propagate missing labels. In some embodiments, the correction algorithm may include one or more trained deep learning (DL) models configured to process MR image data and / or labeled images to generate vertebral masks and / or intervertebral disc masks, and use those masks to correct and refine the spinal segment labels in the labeled images. For example, an image segmentation model may be trained to generate vertebral masks that label pixels associated with each vertebra in the MR image data. Alternatively or additionally, the correction module may include an image segmentation module trained to generate intervertebral disc masks that label pixels associated with each intervertebral disc in the MR image data. The 3D label volume can be compared with the intervertebral disc mask and / or vertebral mask to detect split labels or missing labels, or to propagate spinal segment labels to additional spinal segments above or below the predetermined multiple spinal segments trained to be identified by the DL models 310, 410.

[0066] Figure 7Example of method 700, which includes steps for correcting spinal segment labels, including steps and processes for correcting labels using intervertebral disc masks and / or vertebral mask information, and for obtaining user input to modify and / or verify spinal segment labels. At step 702, multiple inputs are received, including a 3D label volume (and / or corrected labeled images for multiple slices), a vertebral mask, and an intervertebral disc mask. Step 704 is performed to clean the 3D label volume by comparing it with the vertebral mask and intervertebral disc mask, such as to identify and remove outliers, and otherwise remove segmentation errors outside the region of interest. In one embodiment, spinal regions can be identified in image data based on intervertebral disc masks and vertebral masks. Each spinal segment label in the spinal segment labels is then evaluated to ensure that each label is within the spinal region and / or within a threshold distance of the spinal region. If any label is not within the spinal region, it is determined to be an outlier and removed.

[0067] Step 706 is performed to cross-check the labels between the 3D labeled volumes (and / or corrected labeled images for multiple slices) and one or more of the intervertebral disc masks and vertebral masks, and then labels are corrected based on the comparison results, such as filling in missing labels. For example, spinal segment labels for 3D volumes may overlap with vertebral masks. If the 3D volumes are correctly labeled, each vertebral mask (or the 3D volume generated from it) should be aligned with or intersect a 3D labeled volume, and the corresponding label should target the same spinal segment. If the 3D volumes are not correctly labeled, discrepancies are corrected to correct errors in the spinal segment labels.

[0068] Step 708 identifies fused vertebrae based on a comparison of 3D label volumes with vertebral masks and / or intervertebral disc masks. For example, if each 3D label volume is not separated by the intervertebral disc based on the intervertebral disc mask, or if any 3D volume intersects with two vertebral masks, it can be determined that the 3D label volume spans two vertebrae that may be fused and need to be split into two labels. An output is generated at step 710 to notify the user of the detected fused vertebrae. In some embodiments, the correction module can be configured to prompt the user for input, such as approval / confirmation of the fused vertebra identification or to provide correction input. The correction module can then utilize the corrected user input to reprocess and modify the spinal segment labels, as described above.

[0069] Step 712 is performed to identify missing labels based on a comparison of the 3D label volume with the vertebral mask and / or intervertebral disc mask. For example, if the vertebral mask for a vertebra does not intersect with any 3D label volume, the missing label is identified and added. Missing labels may be higher or lower than a predetermined number of spinal segments that the DL models 310, 410 are trained to detect. Therefore, the intervertebral disc mask and / or vertebral mask information can be used to propagate spinal segment labels to additional segments that are not labeled by the model and are therefore not included in the predetermined number of segments that the model is trained to identify. These additional segments are counted and labeled sequentially, starting from the closest labeled spinal segment in the 3D label volume. Figure 8 An exemplary output of this method is illustrated, showing labeled 2D images in which each of a predetermined plurality of lumbar vertebral segments is labeled, and where segment labels are propagated up to additional thoracic vertebral segments labeled above the predetermined lumbar vertebral segments by the trained DL model. 2D images of each slice in the MR image data are labeled accordingly.

[0070] At step 714, an alert is provided to the user regarding additional detected and propagating tags. In some embodiments, this may include the location of the abnormality, such as the L6 vertebra (which may be detected in a minority of patients). The alert may also include notification of such an abnormality. The modified tagged image may be presented to the user as part of the alert. In some embodiments, user input may be requested or required, and such user input may be used to confirm or modify the tags, as described above. The final tagged MR image is then stored, output, and / or displayed at step 716. Figure 8 A slice of an exemplary labeled image is depicted.

[0071] In various embodiments, any suitable computer-readable medium may be used to store instructions for performing the functions and / or processes described herein. For example, in some embodiments, the computer-readable medium may be transient or non-transitory. For example, a non-transitory computer-readable medium may include media such as magnetic media (such as hard disks, floppy disks, etc.), optical media (such as compact discs, digital video discs, Blu-ray discs, etc.), semiconductor media (such as RAM, flash memory, electrically programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.), any suitable medium that is not transient during transmission or lacks any persistent appearance, and / or any suitable tangible medium. As another example, a transient computer-readable medium may include signals on a network, in wires, conductors, optical fibers, circuits, or in any suitable medium that is transient during transmission and lacks any persistent appearance, and / or any suitable intangible medium.

[0072] This written description uses examples to disclose the invention, including the best mode, and also enables any person skilled in the art to perform and use the invention. Certain terms are used for the purpose of brevity, clarity, and ease of understanding. Unnecessary limitations should not be inferred from this description beyond the requirements of the prior art, as such terms are used for descriptive purposes only and are intended to be understood broadly. The patent scope of this invention is defined by the claims and may include other examples that would occur to a person skilled in the art. These other examples are intended to be within the scope of the claims if they have features or structural elements that are not different from the literal language of the claims, or if they include equivalent features or structural elements that are not substantially different from the literal language of the claims.

Claims

1. A method for processing magnetic resonance images of the spine, the method comprising: Receive MR image data containing multiple spinal images of a patient, the multiple spinal images including spinal images for each sagittal slice in multiple sagittal slices; Each spine image is processed using a deep learning (DL) model, wherein the DL model is trained to generate multiple labeled images for each spine image, wherein each of the multiple labeled images is labeled with a different predetermined set of spine segment labels; The plurality of labeled images are combined into a labeled spine image for each of the plurality of sagittal slices, wherein each labeled spine image has a spine segment label for each of the predetermined plurality of spine segments. as well as A labeled MR image is generated based on the labeled spine image.

2. The method of claim 1, wherein the plurality of labeled images generated for each spinal image include at least a first labeled image and a complete labeled image, the first labeled image including only a first spinal segment label, and the complete labeled image including the first spinal segment label and additional spinal segment labels for each of the predetermined plurality of spinal segments.

3. The method of claim 1, wherein each spinal segment label is located at the centroid of the vertebra in the spinal image.

4. The method of claim 1, further comprising receiving either a neck name or a waist name; In response to receiving the neck name, the DL model generates at least a first labeled neck image with an S1 label and a complete labeled neck image with S1, L5, L4, L3, L2, L1, and T12 labels. and In response to receiving the waist name, the DL model generates at least a first labeled waist image with a C1 label and a complete labeled waist image with C1, C2, C3, C4, C5, C6 and C7 labels.

5. The method of claim 1, further comprising creating the DL model by training a convolutional neural network to receive the spinal image and generate a labeled image for each of the predetermined plurality of spinal segments between a first assigned segment and a last assigned segment, wherein the first labeled image includes only a first spinal segment label, and wherein the complete labeled image includes the first spinal segment label and additional spinal segment labels for each of the predetermined plurality of spinal segments.

6. The method of claim 1, wherein the DL model comprises at least a first U-Net trained to locate and label the centroid of the vertebrae for each of the predetermined multiple sets of spinal segments.

7. The method of claim 6, wherein the DL model is a W-Net and includes a second U-Net trained to refine the shape of the label in each spine image.

8. The method of claim 1, the method further comprising aligning the spinal segment labels across the plurality of sagittal slices in the labeled spinal image to identify a 3D label volume for each of the predetermined plurality of spinal segments.

9. The method of claim 8, wherein at least one spinal segment label in at least one of the spinal segment labels in at least one of the plurality of sagittal slices is adjusted such that the 3D label volume for each of the predetermined plurality of spinal segments is continuous across the plurality of sagittal slices.

10. The method according to claim 1, wherein the method further comprises: Align the spinal segment labels across the plurality of sagittal slices in the labeled spinal image to identify multiple 3D volumes encompassing the spinal segment labels; Determine the distance between each of the plurality of 3D volumes; If the distance between two 3D volumes among the plurality of 3D volumes is less than a threshold distance, the two 3D volumes are combined together to form a 3D label volume for one of the plurality of spinal segments.

11. The method according to claim 1, wherein the method further comprises: The MR image data is processed using an image segmentation model trained to identify vertebral masks that label pixels associated with each vertebra in the MR image data. The vertebral mask is compared with the spinal segment labels to detect missing labels; as well as At least one spinal segment label is generated based on the vertebral mask to be added to the predetermined plurality of spinal segment labels.

12. The method of claim 11, wherein the missing label in the spinal segment label is a spinal segment label for one of the predetermined plurality of spinal segments.

13. The method according to claim 1, wherein the method further comprises: The MR image data is processed using an image segmentation model trained to identify vertebral masks that label pixels associated with each vertebra in the MR image data and / or intervertebral disc masks that label pixels associated with each intervertebral disc in the MR image data. The vertebral mask and / or the intervertebral disc mask are compared with the spinal segment label to detect fused vertebrae; as well as Generate a user prompt requesting the clinician to confirm the fused vertebrae.

14. A magnetic resonance imaging (MRI) system, the magnetic resonance imaging (MRI) system comprising: A magnet system configured to generate a polarized magnetic field surrounding at least a portion of a subject arranged in the MRI system; Multiple gradient coils, the multiple gradient coils being configured to apply gradient pulses to the polarization magnetic field; A radio frequency (RF) system configured to apply an RF field to the subject and acquire magnetic resonance (MR) image data from the subject; Processing equipment; and A memory storage device, the memory storage device including instructions executable by the processing device to perform the following operations: Receive MR image data containing multiple spinal images of a patient, the multiple spinal images including spinal images for each of multiple slices; Each spine image is processed using a deep learning (DL) model, wherein the DL model is trained to generate multiple labeled images for each spine image, wherein each of the multiple labeled images is labeled with a different predetermined set of spine segment labels; The plurality of labeled images are combined into a labeled spine image for each of the plurality of slices, wherein each labeled spine image has a spine segment label for each of a predetermined plurality of spine segments. as well as A labeled MR image is generated based on the labeled spine image.

15. The system of claim 14, wherein the plurality of labeled spinal images generated for each spinal image include at least a first labeled image and a fully labeled image, the first labeled image including only a first spinal segment label for a predetermined anchoring segment, and the fully labeled image including a spinal segment label for each of the predetermined plurality of spinal segments.

16. The system of claim 14, wherein each spinal segment label is located at the centroid of the vertebrae of each spinal segment in each spinal image for the predetermined set of spinal segments.

17. The system of claim 14, wherein the instructions executable by the processing device are further executable to: The spinal segment labels are aligned across the plurality of slices in the labeled spinal image to identify a plurality of 3D volumes covering the spinal segment labels, and the plurality of 3D volumes are processed to correct the spinal segment labels.

18. The system of claim 17, wherein the instructions executable by the processing device are further executable to: Determine the distance between each of the plurality of 3D volumes; and If the distance between two 3D volumes among the plurality of 3D volumes is less than a threshold distance, the two 3D volumes are combined together to form a 3D label volume for one of the plurality of spinal segments.

19. The system of claim 14, wherein the instructions executable by the processing device are further executable to: The MR image data is processed using an image segmentation model trained to identify vertebral masks that label pixels associated with each vertebra in the MR image data. The vertebral mask is compared with the spinal segment labels to detect missing labels; as well as At least one spinal segment label is generated based on the vertebral mask to be added to the predetermined plurality of spinal segment labels.

20. The system of claim 14, wherein the instructions executable by the processing device are further executable to: The MR image data is processed using an image segmentation model trained to identify vertebral masks that label pixels associated with each vertebra in the MR image data and / or intervertebral disc masks that label pixels associated with each intervertebral disc in the MR image data. The vertebral mask and / or the intervertebral disc mask are compared with the spinal segment label to detect fused vertebrae; Generate a user prompt requesting the clinician to confirm the fused vertebrae.

Images