Detection of the spine in image data

Abstract

The vertebrae of the spine in the volumetric image are detected using multi-stage detection using trained artificial intelligence. In one embodiment, a trained neural network 116 is used in a first stage to detect individual vertebrae in the sagittal image. Two-dimensional bounding boxes around the detected vertebrae are combined to generate a three-dimensional model of the spine. A panoramic image of the spine is generated based on the three-dimensional model to generate a straight view of the spine. A trained neural network is used in a second stage to detect individual vertebrae in the panoramic image. The two-dimensional bounding boxes around the detected vertebrae in the panoramic image are transformed into three-dimensional space to generate three-dimensional image data having three-dimensional bounding boxes.

Description

Technical Field

[0001] The present invention generally relates to medical imaging informatics, and more particularly to the detection of the spine in image data.

Background Art

[0002] When a radiologist reads an axial spine image that has not been annotated (annotated) by identifying vertebrae, the radiologist can use a sagittal spine image and / or other images to cross-reference the position of the axial spine image relative to the spine. It is difficult to confirm which vertebra is represented in the axial spine image without reference.

Summary of the Invention

Problems to be Solved by the Invention

[0003] In one example, the radiologist is tasked with manually annotating and / or counting the vertebral centers. Unfortunately, this is tedious, time-consuming, and prone to human error. Automated vertebra detection algorithms are described in Payer et al., "Vertebrae Localization and Segmentation with Spatial Configuration-Net and U-Net," (2019) and Chen, "An Automatic Multi-stage System for Vertebra Segmentation and Labelling," (2019) and Lessmann et al., "Iterative fully convolutional neural networks," (2019). Unfortunately, these approaches require significant computational power. At least considering the above, there is an unsolved need for another approach to detect and label the spine in volumetric image data.

Means for Solving the Problems

[0004] The embodiments described herein address the above-mentioned problems and / or other problems.

[0005] The following describes a multi-stage approach for the automatic detection and labeling of vertebrae in volumetric image data. In one embodiment, an object detection algorithm is used in the first detection stage to detect individual vertebrae in a sagittal image and generate a 2D bounding box for the detected vertebrae. The sagittal image and the 2D bounding box are combined to generate a 3D model of the vertebrae with a 3D bounding box. A panoramic image of the vertebrae is then generated based on the 3D model so that a straight view of the vertebrae is produced. In the second stage, the object detection algorithm (or another algorithm) is used to detect individual vertebrae within the panoramic image and generate a 2D bounding box for the detected vertebrae. The 2D bounding boxes of the vertebrae in the panoramic image are converted into 3D space.

[0006] In one embodiment, the system is configured to detect vertebrae of the spine in volumetric image data. The system has a computing device. The computing device has a memory having instructions for a vertebra detection module. The computing device further has a processor configured to execute instructions for performing a two-stage vertebra detection (hereinafter referred to as "two-stage vertebra detection"), in the first stage of two-stage vertebra detection, a first set of vertebral boundary boxes is detected in a sagittal image and clustered into boxes in a volumetric image; in the second stage of two-stage vertebra detection, a panoramic image of the spine is generated based on the first set of detected boundary boxes, and a second set of vertebral boundary boxes is detected in the panoramic image. The computing device further has a display configured to display a 2D image of the detected vertebrae. In one example, all detected boundary boxes in both the first and second sets are labeled as either the sacrum, C2, or "other" vertebrae. This is because, at least, both the C2 and sacral vertebrae have a unique shape and act as anchors for vertebral annotation after the boundary box has been detected.

[0007] In another embodiment, the method is configured to detect vertebrae of the spine in volumetric image data. The method includes extracting a first set of vertebral bounding boxes in a sagittal image of the spine and labeling them, in one example, as either the sacrum, C2, or "other" vertebrae. The method then includes clustering the bounding boxes from the sagittal image into boxes in a volumetric image. The method further includes generating a panoramic image of the spine based on the first set of detected bounding boxes. The method further includes extracting a second set of vertebral bounding boxes in the panoramic image with the same labels as before.

[0008] In another embodiment, a computer-readable storage medium stores instructions for detecting vertebrae of the spine in volumetric image data. When executed by a computer processor, the instructions cause the processor to perform the following steps: extract a first set of vertebral bounding boxes in a sagittal image of the spine; label those bounding boxes, in one example, as either the sacrum, C2, or "other" vertebrae; cluster those bounding boxes into boxes in the volumetric data; generate a panoramic image of the spine based on the first set of detected volumetric bounding boxes; and extract a second set of vertebral bounding boxes in the panoramic image.

[0009] Those skilled in the art will recognize further aspects of this application by reading and understanding the accompanying description.

[0010] The present invention can take the form of various components and combinations of components, as well as various steps and combinations of steps.

[0011] The drawings are for illustrative purposes only and should not be construed as limiting the invention. [Brief explanation of the drawing]

[0012] [Figure 1] A schematic diagram illustrating an exemplary system according to embodiments of this specification, having a vertebral detection and labeling module configured to detect and label vertebrae of the spine in volumetric image data. [Figure 2] This figure schematically shows an example of a vertebra detection and labeling module of the system shown in Figure 1, according to an embodiment of this specification. [Figure 3] A schematic diagram illustrating an exemplary method for detecting and labeling vertebrae in volumetric image data according to embodiments of this specification. [Modes for carrying out the invention]

[0013] Figure 1 schematically shows an exemplary system 102 according to one embodiment of this specification. System 102 includes at least a data repository 104 and a computing device 106.

[0014] The data repository 104 has a physical storage medium configured to store at least digital medical images. In one example, the data repository 104 is for a healthcare entity and includes digital medical images of a subject acquired by the healthcare entity's imaging modality. The physical storage medium is local to the healthcare entity and / or remote from the healthcare entity, such as a portion of a "cloud" based resource. Examples of imaging modalities include magnetic resonance imaging (MR), computed tomography (CT), single-photon emission tomography (SPECT), positron emission tomography (PET), and X-ray. The digital medical images include a series of two-dimensional (2D) images (collectively providing a three-dimensional (3D) volumetric image dataset) and / or a 3D volumetric image dataset. The digital medical images include at least images of the vertebrae of the subject's spine.

[0015] The computing device 106 includes a processor 108 (e.g., a central processing unit (CPU), a microprocessor (CPU), a graphics processing unit (GPU), and / or other processors) and a computer-readable storage medium ("memory") 110 (excluding temporary media), such as a physical storage device like a hard disk drive, a solid-state drive, or an optical disc. The memory 110 includes at least computer-executable instructions 112 and data 114. The processor 108 is configured to execute the computer-executable instructions 112. In one example, the computing system 106 is configured to provide storage, access, and / or processing of medical information, including digital medical images and electronic reports. An example of the computing device 106 includes, but is not limited to, a picture archiving and communications system (PACS). If the computing system 106 is a PACS, digital medical images and / or other electronic information are stored and / or transmitted via the DICOM (Digital Imaging and Communications in Medicine) format and / or another format.

[0016] Instruction 112 includes at least instructions for the vertebra detection module 116. As will be described in more detail below, in one embodiment, the vertebra detection module 116 is configured to detect individual vertebrae in an image of the spine acquired in the sagittal plane ("sagittal image"), generate 2D bounding boxes for the detected vertebrae, combine the sagittal image and the 2D bounding boxes to generate a 3D model of the spine with a 3D bounding box, generate a panoramic image of the detected vertebrae based on the 3D model to generate a straight view of the spine, detect individual vertebrae in the panoramic image and generate 2D bounding boxes for the detected vertebrae in the panoramic image, convert the 2D bounding boxes into 3D space, and optionally annotate the displayed 2D image. As used herein, the bounding box borders or surrounds a vertebra with or without partial overlap of one or more adjacent vertebrae. In one example, the vertebra detection module 116 reduces the computational power required for detection and labeling 4 and / or improves the accuracy of vertebral visualization compared to a configuration without the vertebra detection module 116.

[0017] Input devices 118, such as keyboards, mice, and touchscreens, communicate electrically with the computing system 102. In one example, the input devices 118 are configured to allow a user to operate the computing system 102 via user input, which includes activating the vertebral detection module 116, selecting volumetric image data and / or sagittal images to load, etc. Human-readable output devices 120, such as displays, also communicate electrically with the computing system 106. In one example, the output device 120 is configured to display a 2D image of the vertebrae, prompt the user for input, present commands, etc. Input / output ("I / O") 122 are configured for communication (wired and / or wireless) with at least the data repository 104, which includes retrieving / receiving electronic data from the data repository 104, the input devices 118 and / or the output devices 120, and / or transmitting data to them.

[0018] Figure 2 schematically shows an example of a vertebral detection module 116 according to one embodiment of this specification. The illustrated vertebral detection module 116 includes a data preprocessor 200, a trained vertebral detector 202, a 3D model generator 204, a panoramic image generator 206, a 2D to 3D spatial converter 208, and an annotator 210.

[0019] The vertebral detection module 116 receives image data of a target spine scan as input, which includes one or more sets of a series of 2D images (collectively providing a 3D volumetric dataset) and / or a 3D volumetric dataset, or a series of 2D images (collectively providing a 3D volumetric dataset) and / or 3D volumetric datasets, and / or sagittal images generated from such a dataset. As briefly described herein, suitable datasets include MR, CT, SPECT, PET, X-ray, etc. In one example, specific image data is selected via user input from input device 118.

[0020] The data preprocessor 200 is configured to process input image data. In one example, the result of processing is one or more sets of sagittal slices. Parameters such as window width, window level, and slice thickness are determined via user input and / or pre-programmed settings. If multiple sets of sagittal slices are generated, in one example, the window width, window level, and / or other parameters are the same for all slices. In another example, at least two of the sets have at least one different parameter value. The window width refers to the range of CT numbers to display (Houndsfield units (HU)), and the window level refers to the CT number at the midpoint of the range. An example of window width and level (W / L) settings for viewing the spine in an image containing the spine is W=1800HU and L=400HU. If the input image data contains one or more sets of sagittal images, the data preprocessor 200 is not used to process the input image data to generate one or more sets of sagittal images.

[0021] The trained vertebral detector 202 is configured to process one or more sets of sagittal slices. In one example, this includes detecting whether a slice contains a vertebra and generating a 2D bounding box for the detected vertebrae. In one example, vertebral detection proceeds slice by slice, starting from the central slice and moving in both directions to the outermost slices, i.e., the first and last slices of the sagittal slices. Vertebral detection terminates after all slices have been processed and a predetermined stopping criterion is met (for example, after a predetermined number of consecutive slices have been processed, there is no indicia of a vertebra, i.e., the vertebrae are out of the dataset), or after other predetermined stopping criteria are met. In one example, the predetermined stopping criterion reduces processing time, for example, by limiting the number of slices processed.

[0022] Examples of suitable detectors include artificial intelligence-based detectors and / or other detectors, including neural network-based detectors such as the faster regional convolutional neural network described in Girshick et al., "Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks," 2015, and the You Only Look Once described in Redmon et al., "YOLOv3: An Incremental Improvement," 2018. For the sake of brevity and explanation, spine detection will be described herein using the YOLOv3 detector.

[0023] Using the YOLOv3 detector, a neural network is applied to an image, the image is divided into multiple regions, bounding boxes and probabilities are predicted for each region, and the bounding boxes are weighted by the predicted probabilities. Using the YOLOv3 detector, a predetermined minimum confidence threshold is utilized to determine whether to generate a 2D bounding box of an object that was probably detected as a vertebra. In one example, the minimum confidence threshold is 0.55. This means that 2D bounding boxes are generated only for vertebrae detected with a confidence of 0.55 or higher. In another example, the minimum confidence threshold is 0.50. In yet another example, the minimum confidence threshold is a different value. A minimum confidence threshold of 0 would result in generating 2D bounding boxes for all objects that were probably detected as vertebrae. Generally, a higher threshold improves specificity, while a lower threshold improves sensitivity.

[0024] The 3D model generator 204 is configured to process one or more sets of sagittal images and the 2D bounding boxes. In one example, this includes generating a 3D model with a 3D bounding box for each detected vertebra by combining the 2D bounding boxes across multiple sagittal slices. For the sake of brevity and explanation purposes, here, an application of density-based spatial clustering using the Density-Based Spatial Clustering of Applications with Noise (DBSCAN) algorithm described in Ester et al., "A density-based algorithm for discovering clusters in large spatial databases with noise. Proceedings of the Second International Conference on Knowledge Discovery and Data Mining (KDD-96)," AAAI Press. pp. 226-231 is used for illustration. DBSCAN is a density-based data clustering nonparametric algorithm that, given a set of points in a space, clusters / group points that are close to each other and considers points that are far from their nearest neighbors as outliers.

[0025] The panoramic image generator 206 is configured to process a 3D model and a 3D bounding box. In one example, this includes generating a panoramic image using a curve passing through the center of the 3D bounding box. In one example, the curve is extrapolated in front of the first vertebra and behind the last vertebra. Thereby, the panoramic image generation unit 206 can add vertebrae missing at the edges. The curve is interpolated and sampled at a predetermined interval (e.g., 0.1, 0.5, 1.0, 2.5 millimeters (mm), etc.). For each point on the curve, lines from the 3D model are sampled on a plane perpendicular to the curve at that point along the projection of a vector going from one side of the body to the opposite side of the body (e.g., from the front of the body to the back of the body). The result is a panoramic (quasi-sagittal) image including the entire vertically aligned spine.

[0026] The trained vertebra detector 202 is further configured to process the panoramic image. In one example, this includes detecting vertebrae within the panoramic image and generating a 2D bounding box around each detected vertebra. In one example, the minimum confidence threshold is 0.50. Similar to the detection in the sagittal slice, the minimum confidence threshold can be a different value. Generally, vertebra detection in the panoramic image should be more accurate than detection using the sagittal slice since at least the spine is straightened and displayed as a whole, and any vertebra missed using the sagittal image is detected here. In one example, this improves the sensitivity without reducing the specificity. In another embodiment, separate vertebra detectors are utilized for the detector vertebrae in the sagittal image and the panoramic image.

[0027] The 2D-to-3D spatial converter 208 is configured to process panoramic images having 2D bounding boxes. In one example, this involves converting the 2D bounding boxes into 3D space and adding depth to each bounding box. For example, in one example, each corner of each bounding box is converted into 3D space, and all four corner bounding boxes are set as vertebral bounding boxes, where the depth is set to be the same as the smaller edge of the 2D bounding box, i.e., the smaller edge of the width or height of the bounding box.

[0028] The trained vertebral detector 202 is further configured to determine the identification of at least a subset of the identified vertebrae. For example, in one example, the trained vertebral detector 202 identifies at least one cervical vertebra, such as C2, S1, S2, S3, S4, S5, several sacral vertebrae, and other vertebrae, as well as another symbol or word for all other vertebrae. Vertebrals other than C2 and S1-S5 (i.e., C3-C7, T1-T12, L1-L5, and / or coccygeal vertebrae) can be identified by counting posteriorly or anteriorly from the vertebrae identified as a reference. In another example, different combinations of subsets of vertebrae (e.g., C2, sacral, and other vertebrae) may be identified, or only one vertebra may be identified, or all of the vertebrae may be identified.

[0029] The annotator 210 is configured to annotate a displayed 2D image of the input volumetric image data. In one example, the annotation is a projection of the center of the bounding box onto the current slice. In a non-limiting example, for an off-center axial slice, the annotator 210 projects the center from the central axial slice. Alternatively, or additionally, the annotator marks the intersection of the centerline passing through the vertebrae with the current plane. In another example, detected vertebrae are annotated by displaying vertebral labels next to each vertebra without a bounding box or center projection.

[0030] Several variations are then attempted.

[0031] In one variation, the entire algorithm is executed iteratively. For this purpose, sagittal slices are processed, bounding boxes are found, a 3D model is generated, a panoramic image is generated, a bounding box is determined for the panoramic image, the bounding box is converted to 3D space, and the process is repeated until the stopping criteria are met.

[0032] In the above configuration, the trained vertebral detector 202 detects vertebrae in sagittal and panoramic images. In one modified configuration, separate trained vertebral detectors detect vertebrae in sagittal and panoramic images, respectively.

[0033] The following describes a non-limiting embodiment for training a vertebral detector to produce a vertebral detector 202 trained for a single imaging modality. For the sake of brevity and explanation, training using CT is described, and sagittal images of the lumbar, thoracic, and cervical spine from CT studies are annotated. All sagittal images are sampled at the same predetermined resolution with the same predetermined fixed interval. For example, in one example, the sagittal image is sampled at a resolution of 416 x 416 with a pixel interval of 1 mm. For larger images, the image is divided into multiple regions, and each region is sampled to cover the entirety of the image.

[0034] Sagittal images with 2D bounding boxes are supplied for training. These sagittal images with 2D bounding boxes are also augmented to generate additional training data. Examples of features to be augmented include one or more of the following: brightness, contrast, Gaussian noise, shift, scale, rotation, and inversion. For each feature, the probability and limit of augmentation are predetermined. For example, a probability of 0.80 and an augmentation limit of 0.03 for brightness would mean that the brightness of 80% of the image would be augmented within ±3% of the original value. Augmenting sagittal images in this way increases data diversity and / or reduces overfitting—memory—of the training data.

[0035] The training dataset is divided into training, testing, and validation subsets of a predetermined size. The CT training dataset is used to train a vertebral detector 202, which is trained for vertebral detection in CT image data. In a YOLOv3-based network, the vertebral detector is trained iteratively until a stopping criterion is met. In one example, the vertebral detector is trained iteratively until the error between the original bounding box and the generated bounding box meets a predetermined value. The vertebral detector is trained to detect all or part of a vertebra.

[0036] In one example, a validation dataset is used to determine the stopping criteria during training. In this example, the network weights are not affected by the validation phase, and the quality of training is determined by checking the metrics. For example, once training is complete, the network is run on a test dataset, and the metrics are therefore compared to those on the validation dataset. If the metrics deviate within a predetermined tolerance range, the training is considered valid and terminates.

[0037] Next, variations of the training are planned.

[0038] In one variation, the training images are based on annotated vertebral centers and include panoramic images with the same resolution and pixel spacing. Feature augmentation may differ between sagittal and panoramic images; for example, the panoramic images may not have shift and rotation augmentation.

[0039] In another variation, the vertebral detector is trained with respect to multiple imaging modalities. For the sake of brevity and explanation, the training is described using CT and MR image data. In this variation, the enhancement limit may vary across modalities; for example, the contrast limit for CT may be 0.30, while the contrast limit for MR may be 0.40. In one variation, the CT and MR datasets are used to train separate vertebral detectors, respectively, to detect vertebrae in CT and MR images. In another variation, the CT and MR datasets are used to train a CT+MR vertebral detector for vertebral detection in CT or MR image data.

[0040] Figure 3 schematically illustrates an exemplary method for detecting and labeling vertebrae of the spine in volumetric image data according to an embodiment of this specification.

[0041] It should be understood that the order of operation of one or more methods is not limited; therefore, other orders are contemplated in this specification. In addition, one or more operations may be omitted, and / or one or more additional operations may be included.

[0042] The image loading step 302 loads volumetric image data as described herein and / or otherwise.

[0043] The 2D image generation step 304 generates one or more sets of sagittal images from volumetric image data, as described herein and / or otherwise.

[0044] Alternatively, the loading step may involve loading one or more sets of sagittal images, and step 304 may be omitted as described herein and / or in other ways.

[0045] The vertebral detection step 306 detects vertebrae in one or more sets of sagittal images and generates 2D bounding boxes for those vertebrae, as described herein and / or otherwise.

[0046] Step 308 generates a 3D model of the spine having a 3D bounding box from a sagittal image and a 2D bounding box, as described herein and / or otherwise.

[0047] The panoramic image generation step 310 generates a panoramic image based on a 3D model as described herein and / or otherwise.

[0048] The vertebral detection step 312 detects (multiple) vertebrae in the panoramic image and generates 2D bounding boxes for those vertebrae as described herein and / or otherwise.

[0049] The 2D-to-3D conversion step 314 converts the 2D bounding boxes of the vertebrae in the panoramic image into 3D space as described herein and / or otherwise.

[0050] Annotation step 316 involves annotating the displayed 2D image of volumetric image data as described herein and / or otherwise.

[0051] The above method can be implemented by computer-readable instructions, which can be encoded or embedded on a computer-readable storage medium 110, causing the processor 108 to perform the described processing steps when executed by a computer processor. In addition, or alternatively, at least one of the computer-readable instructions may be executed by a signal, carrier wave, or other temporary medium that is not a computer-readable storage medium.

[0052] Although the present invention has been illustrated and described in detail in the drawings and the foregoing description, such illustrations and descriptions should be considered descriptive or illustrative and not limiting. The present invention is not limited to the disclosed embodiments. Other variations of the disclosed embodiments can be understood and practiced by those skilled in the art in carrying out the claimed invention, from the examination of the drawings, disclosure and appended claims.

[0053] The words "comprising" do not exclude other components or steps, and the indefinite articles "a" or "an" do not exclude plurality. A single processor or other unit may perform the functions of several items enumerated in the claims. The mere fact that certain means are described in different dependent claims does not imply that combinations of these means cannot be used advantageously.

[0054] Computer programs may be stored / distributed on suitable media such as optical storage media or solid-state media supplied together with or as part of other hardware, but they may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunications systems. No reference numeral in a claim should be construed as limiting the scope of that claim. The embodiments of the present invention are described below. (Note 1) A system configured to detect vertebrae of the spine in volumetric image data, comprising a computing device, wherein the computing device A memory containing instructions for a vertebral detection module, A processor configured to execute instructions for performing two-stage vertebral detection, wherein in the first stage of the two-stage vertebral detection, a first set of vertebral bounding boxes is detected in a sagittal image and clustered in volumetric image data, and in the second stage of the two-stage vertebral detection, a panoramic image of the vertebra is generated based on the first set of detected vertebral bounding boxes, and a second set of vertebral bounding boxes is detected in the panoramic image, A display that shows a 2D image of the detected spine from the volumetric image data, A system that has (Note 2) The system according to Appendix 1, wherein the vertebral detection module has a neural network trained to detect vertebrae. (Note 3) The system according to Appendix 1 or 2, wherein the vertebra detection module detects a first set of bounding boxes based on a first predetermined confidence level and generates a 2D bounding box for the detected vertebrae. (Note 4) The system according to any one of appendices 1 to 3, wherein the vertebral detection module detects a first set of bounding boxes, starting from the central image of the sagittal image and moving outward in both directions toward the first image of the sagittal image and the last image of the sagittal image, until a stopping criterion is met. (Note 5) The system as described in Appendix 4, wherein the aforementioned stopping criteria include the absence of detection of vertebrae in a predetermined number of consecutive images among the sagittal images. (Note 6) The system according to any one of appendices 3 to 5, wherein the vertebra detection module labels each vertebra of the first set of boundary boxes as a sacrum, C2, or other vertebra. (Note 7) The system according to any one of appendices 3 to 6, wherein the vertebral detection module combines the sagittal image and the 2D bounding box to generate a 3D model having a 3D bounding box. (Note 8) The vertebral detection module generates a curve passing through the center of the 3D bounding box, as described in Appendix 7. (Note 9) The vertebral detection module extrapolates the curve anterior to the first vertebra and posterior to the last vertebra to add the missing vertebrae, as described in Appendix 8. (Note 10) The system according to Appendix 8 or 9, wherein the spine detection module samples lines from the 3D model along the projection of a vector from the anterior to the posterior part of the spine for each point on the curve onto a plane perpendicular to the curve, thereby generating the panoramic image including a quasi-sagittal image including the entire spine aligned vertically. (Note 11) The system according to Appendix 10, wherein the vertebra detection module detects a second set of boundary boxes based on a second predetermined confidence level and generates a 2D boundary box for the detected vertebrae. (Note 12) The system as described in Appendix 11, wherein the vertebra detection module converts the 2D bounding box of the panoramic image into a 3D space to define the 3D bounding box of the vertebrae. (Note 13) The system according to appendix 11 or 12, wherein the vertebra detection module labels each vertebra in the second set of boundary boxes as a sacrum, C2, or other vertebra. (Note 14) The computing device is a picture archiving and communication system (PACS), as described in any one of the appendices 1 to 13. (Note 15) A computer implementation method for detecting vertebrae in volumetric image data, In a sagittal image of the spine, the first set of vertebral boundary boxes is extracted, The steps include generating a panoramic image of the spine based on the first set of detected boundary boxes, The steps include extracting a second set of boundary boxes for the vertebrae in the panoramic image, A method for realizing a computer having the following characteristics. (Note 16) The step of extracting the first set of bounding boxes is: The steps include: detecting the first set of bounding boxes, starting from the central image of the sagittal image and moving outward toward the first image of the sagittal image and the last image of the sagittal image; The step of terminating the detection in response to a predetermined number of consecutive sagittal images not containing vertebrae, The steps include generating a 2D bounding box for the detected vertebrae, The steps include identifying the center of the 2D bounding box, The steps include annotating the vertebrae in the 2D bounding box, A computer implementation method as described in Appendix 15, having the following characteristics. (Note 17) The steps include detecting a second set of 2D bounding boxes in the panoramic image based on a second predetermined confidence level, The steps include: converting the 2D bounding box in the panoramic image into a 3D space to define the 3D bounding box of the vertebrae; The process includes the step of annotating the vertebrae in the 3D bounding box, The computer implementation method described in Appendix 15. (Note 18) A computer-readable storage medium for storing computer-executable instructions for detecting vertebrae of the spine in volumetric image data, wherein, when the instructions are executed by a computer processor, the processor... In a sagittal image of the spine, the first set of vertebral boundary boxes is extracted, The steps include generating a panoramic image of the spine based on the first set of detected boundary boxes, The steps include extracting a second set of boundary boxes for the vertebrae in the panoramic image, A computer-readable medium that enables execution. (Note 19) The aforementioned computer executable instructions further provide the processor with: The steps include: detecting a first set of bounding boxes, starting from the central image of the sagittal image and moving outward toward the first image of the sagittal image and the last image of the sagittal image; The step of terminating the detection in response to a predetermined number of consecutive sagittal images not containing vertebrae, The steps include generating a 2D bounding box for the detected vertebrae, The steps include identifying the center of the 2D bounding box and annotating the vertebrae within the 2D bounding box, A computer-readable storage medium as described in Appendix 18, which enables the execution of the above. (Note 20) The aforementioned computer executable instructions further provide the processor with: Steps include detecting a second set of 2D bounding boxes in the panoramic image based on a second predetermined confidence level, The steps include: converting the 2D bounding box in the panoramic image into a 3D space to define the 3D bounding box of the vertebrae; The steps include annotating the vertebrae in the aforementioned 3D bounding box, A computer-readable storage medium as described in Appendix 18, which enables the execution of the above.

Claims

1. A system configured to detect vertebrae of the spine in volumetric image data, comprising a computing device, wherein the computing device Vertebral detection module, A memory having instructions for the vertebral detection module, A processor configured to cause the vertebral detection module to execute the command to perform two-stage vertebral detection, wherein in the first stage of the two-stage vertebral detection, a first set of vertebral boundary boxes is detected in the sagittal image and clustered in the volumetric image data, and in the second stage of the two-stage vertebral detection, a panoramic image of the spine is generated based on the first set of detected boundary boxes, and a second set of vertebral boundary boxes is detected in the panoramic image, A display that shows a 2D image of a vertebra generated from a second set of vertebral boundary boxes, It has, The vertebra detection module detects the first set of vertebral bounding boxes based on a first predetermined minimum confidence threshold, and generates a 2D bounding box for the detected vertebra. The vertebral detection module combines the sagittal image and the 2D bounding box to generate a 3D model having a 3D bounding box. The vertebra detection module generates a curve that passes through the center of the 3D bounding box, The vertebral detection module samples lines from the 3D model along the projection of a vector from the anterior to the posterior part of the vertebrae at each point on the curve, onto a plane perpendicular to the curve at that point, thereby generating the panoramic image, which is a quasi-sagittal image including the entire vertebrae aligned vertically. The vertebra detection module detects the second set of vertebral bounding boxes based on a second predetermined minimum confidence threshold, and generates a 2D bounding box for the detected vertebra. The vertebra detection module converts the 2D bounding box of the panoramic image into a 3D space to define the 3D bounding box of the vertebrae. A system wherein the vertebra detection module has a YOLO-based neural network trained to detect vertebrae based on the minimum confidence threshold.

2. The system according to claim 1, wherein the detection of a first set of vertebral boundary boxes in the sagittal image by the vertebral detection module moves outward in both directions toward the first image and the last image of the sagittal image, starting from the central image of the sagittal image and until a stopping criterion is met.

3. The system according to claim 2, wherein the stopping criterion includes the absence of detection of vertebrae in a predetermined number of consecutive images among the sagittal images.

4. The system according to any one of claims 1 to 3, wherein the vertebra detection module labels each vertebra of the first set of boundary boxes as a sacrum, C2, or other vertebra.

5. The system according to any one of claims 1 to 4, wherein the vertebral detection module extrapolates the curve to the anterior end of the first vertebra and the posterior end of the last vertebra to add the missing vertebra.

6. The system according to any one of claims 1 to 5, wherein the vertebra detection module labels each vertebra of the second set of boundary boxes as a sacrum, C2, or other vertebra.

7. The system according to any one of claims 1 to 6, wherein the computing device is an image storage and communication system (PACS).

Images