Method and apparatus for automatic determination of spinal deformities from images

Abstract

The present disclosure relates to a method and apparatus for automatically determining spinal deformity from an image. The invention describes a method for automatically determining spinal deformity from an image showing a plurality of vertebrae of a spine, the method comprising the steps of: a) detecting center points of the plurality of vertebrae shown in the image; b) constructing a center line based on the detected center points; c) calculating local tilt at points along the center line; d) determining a positive tilt maximum and a negative tilt maximum of the local tilt and selecting two reference vertebrae having center points closest to the determined tilt maximums; e) segmenting an upper endplate of the cranial reference vertebra; f) segmenting a lower endplate of the caudal reference vertebra; g) calculating an angle between the upper endplate of the reference vertebra and the lower endplate of the reference vertebra and outputting the angle. The invention also describes a related apparatus.

Description

Technical Field

[0001] This invention relates to a method and apparatus for automatically determining spinal deformities based on (medical) images, particularly X-ray images, wherein the images show multiple vertebrae of the spine. In particular, this invention relates to the automatic measurement of spinal curvature in images combining central line tilt and endplate orientation. Background Technology

[0002] Spinal deformities are common clinical cases. For example, adolescent idiopathic scoliosis (AIS) is a spinal deformity that occurs in growing children and has a prevalence of approximately 0.5% to 5% (depending on the study and country). Frequent measurements of spinal curvature on X-rays are performed for diagnosis and monitoring of treatment response.

[0003] The Cobb angle constitutes the gold standard and is the most frequently used curvature measurement in the coronal plane (see, for example, Takahashi et al. (2019), “Full-spine radiographs: what others are reporting—a survey of Society of Skeletal Radiology members,” *Skeleton Radiology*, Vol. 48, pp. 1759–1763). It was introduced in 1948 (see Cobb (1948), “Outline for the study of scoliosis,” American Academy of Orthopaedic Surgeons Teaching Course Lectures, Vol. 5, Ann Arbor, Michigan: Edwards), and has the advantage of being easily performed by humans on X-ray images.

[0004] However, the Cobb angle has been shown to be a good measure of spinal curvature. However, it depends on the local endplate orientation and ignores important parts of the curvature characteristics, such as overall spinal curvature and apical vertebral translation (see Bernstein et al., “Radiographic scoliosis angle estimation: spline based measurement reveals superior reliability compared to traditional COBB method”, European Journal of Spine (2021) 30: 676-685).

[0005] From a doctor's perspective, the following challenges exist in Cobb angle measurement:

[0006] a) Measurements are subjective and exhibit significant inter-reader / intra-reader variability. This is particularly important for subsequent imaging examinations. Systematic or random biases in measurements can influence clinical decisions.

[0007] b) Measurement errors may occur.

[0008] c) Conducting these measurements is both time-consuming and unattractive.

[0009] In clinical practice, Cobb angles are measured manually using standard measurement tools from a Picture Archiving and Communication System (PACS) or by utilizing dedicated Cobb angle software. This method is faster than using standard tools.

[0010] Besides manual measurement, there are two main ideas for automated Cobb angle / spinal curvature measurement described in the literature. However, none of these methods are known to be commercially available.

[0011] Concept 1 involves the automatic calculation of the Cobb angle based on the detected superior / inferior vertebral endplates. The idea behind this concept is to automatically locate the superior and inferior vertebral endplates and measure the Cobb angle. This concept reproduces traditional Cobb angle measurement. In recent years, (deep) machine learning has been used in research to locate endplates and calculate the Cobb angle in this manner (e.g., see Cai et al. (ed.) (2020), “Computational Methods and Clinical Applications for Spine Imaging”, https: / / doi.org / 10.1007 / 978-3-030-39752-4).

[0012] Idea 2 involves automatically calculating spinal curvature based on the detected spinal centerline. In this method, spinal curvature is determined by finding the center point of the vertebra, constructing a smooth centerline through the center point, and using the curvature of the centerline to determine an angle similar to the Cobb angle. Mathematically, this involves calculating an “angle function” (the inclination of the centerline relative to the horizontal at each elevation point) and taking the positive and negative maximum values ​​as reference points for the measurement. This idea is sometimes referred to as “analyzing the Cobb angle” (see Stokes (1994), “Three-dimensional Terminology of Spinal Deformity. A Report Presented to the Scoliosis Research Society by the Scoliosis Research Society Working Group on 3-D Terminology of Spinal Deformity”, Spine, 19(2): 236-48). Performing such measurements annually is not easy. However, it can be supported by using computer programs, as was proposed in the 1980s (Jeffries et al. (1980), “Computerized measurement and analysis of scoliosis: a more accurate representation of the shape of the curve”). Summary of the Invention

[0013] The object of this invention is to improve known systems, devices, and methods to facilitate improvements in determining spinal deformities based on images.

[0014] This objective is achieved by the method and apparatus according to the invention.

[0015] According to the present invention, a method for automatically determining (calculating / quantifying) spinal deformities based on an image (e.g., an X-ray image), wherein the image shows multiple vertebrae of the spine, particularly at least vertebrae S1 to C7. The method includes the following steps:

[0016] a) Detect the center point of the vertebrae shown in the image.

[0017] b) Construct a centerline based on the detected center point.

[0018] c) Calculate the local tilt (e.g., tilt angle) at points along the centerline.

[0019] d) Determine the maximum positive and negative tilt angles of the local tilt (angle), and select two reference vertebrae with a center point closest to the determined maximum tilt (angle) value.

[0020] e) Segment the superior endplate of the cranial reference vertebra.

[0021] f) Segment the inferior endplate of the caudal reference vertebra.

[0022] g) Calculate the angle between the superior endplate of the reference vertebra and the inferior endplate of the reference vertebra, and output the angle.

[0023] This method requires images. The (medical) image must show the vertebrae examined by this method, and in particular, images recorded using techniques that visualize bones in the body, such as X-ray images, CT images, or MRI images.

[0024] First, determine the center points of the vertebrae shown in the image. It's not necessary to detect the center points of all vertebrae, but rather at least several (especially adjacent) vertebrae. The more center points of different vertebrae you determine, the better the results. Since the physician knows the type of deformity and the associated vertebrae for those types are also known, at least the center points of the associated vertebrae for the predefined deformity type should be determined.

[0025] The determination of the center point is preferably done automatically. Preferably, this is achieved by segmenting the image by identifying vertebrae in the image as image objects and then calculating the center point of each identified image object (vertebra). This can be achieved by a programmed algorithm or by a trained machine learning algorithm trained to identify vertebrae in an image, and particularly for determining their center points. General machine learning algorithms capable of identifying vertebrae in an image and determining their center points are known in the art. Furthermore, the endplates of the vertebrae can be segmented using this algorithm. A preferred model can detect all four corner points of each vertebra and then define the endplate, wherein the upper endplate is defined by the line crossed by the two upper corner points, and the lower endplate by the line crossed by the two lower corner points. The center is defined as the center of all four corner points.

[0026] After the center points are known, a centerline is constructed based on the detected center points, preferably by using splines. Preferably, the centerline is constructed by guiding a smooth line through all the determined center points (along the spine). Here, it can be seen that it is advantageous to determine the center points of many (especially all) vertebrae shown in the image (at least all vertebrae between two selected vertebrae (e.g., S1 and C7), because otherwise the centerline would pass through some vertebrae with undetermined center points.

[0027] Then, for example, the local tilt of a point along the centerline can be calculated using the first derivative, particularly with respect to the Z-axis (the vertical length of the spine). Furthermore, the normal path of the spine can be compared to the centerline, and the deviation of the centerline from this normal path (as a reference) can be calculated. The tilt of a point relative to the normal path can then be determined, for example, by calculating the gradient. It should be noted that what is relevant here is not the deviation itself, but rather the tilt of the point along the centerline.

[0028] By examining the calculated local tilt, determine the maximum positive and negative tilt values ​​of the centerline, i.e., the points with the maximum tilt in one direction (positive) and the opposite direction (negative), related to the path of the centerline. Since the first deviation is a measure of tilt, the maximum positive and negative values ​​of the first deviation can be considered as the maximum tilt value. In the case of multiple positive or negative local maxima, the largest maximum value is preferably taken as the maximum tilt value.

[0029] Next, select two reference vertebrae. The reference vertebrae are the two vertebrae whose center points are closest to the determined maximum tilt value. It should be noted that one reference vertebra is closest to the positive tilt maximum, while the other reference vertebra is closest to the negative tilt maximum.

[0030] Now, the distal endplates of the reference vertebrae are segmented (the superior endplate of the cranial vertebrae and the inferior endplate of the caudal vertebrae). By segmentation, their lateral position and orientation (tilt angle) are determined.

[0031] The endplate can be defined by landmarks (e.g., the corners of the vertebra or points along the main contour of the vertebra). The algorithms described above (conventional or trained algorithms) can be used to segment the endplate of the vertebra; however, it is preferable to use a different algorithm.

[0032] Similarly, the determination of the endplate is preferably performed automatically. This is preferably achieved by defining the corresponding endplate distal to a reference vertebra (preferably by using the image object determined above) and then calculating the endplate plane. Likewise, this can be achieved by a programmed algorithm or by a trained machine learning algorithm that has been trained to identify the endplates of vertebrae in an image. General machine learning algorithms for identifying the endplates of vertebrae in an image are known in the art. Preferably, corner points are detected and the endplate is defined as a line passing through the corner points. Alternatively, it is preferable to find the vertebral contour and define the endplate as a portion of that contour (cranially or caudally).

[0033] A single algorithm can be used to identify / determine the endplate, or one algorithm can be used to segment the superior endplate of the cranial reference vertebra while another algorithm is used to segment the inferior endplate of the caudal reference vertebra.

[0034] Finally, the angle between the two endplates of the reference vertebra is calculated (automatically). Methods for automatically calculating the angle between the two endplates are generally known in the art.

[0035] This method demonstrates the possibility of automatically determining angles similar to the Cobb angle based on (e.g., X-ray) images to eliminate physician pain points indicated in the background section. In the first part of the method, the local tilt (angle) of the spinal centerline is used to determine the reference vertebra for angle measurement. In the second part of the method, the upper and lower vertebral endplates of the two vertebrae (reference vertebrae) are used to calculate the angle. Note that the Cobb angle is not technically determined here (in the conventional way) because not all endplates are considered here, but only the spinal centerline.

[0036] The device according to the invention for automatically determining spinal deformities based on images showing multiple vertebrae of the spine is preferably designed to perform the method according to the invention. The device includes the following components:

[0037] - Center point unit, which is designed to detect the center points of multiple vertebrae shown in the image.

[0038] - Centerline unit, which is designed to construct a centerline based on the detected center point.

[0039] - A line tilt element, designed to calculate the local tilt (e.g., tilt angle) at points along the centerline.

[0040] - The maximum and minimum units are designed to determine the maximum positive and negative tilt (angle) values ​​of a local tilt (angle), and select two reference vertebrae with a center point closest to the determined tilt (angle) values.

[0041] - A segmentation unit designed for segmenting the superior endplate of a cranial reference vertebra and for segmenting the inferior endplate of a caudal reference vertebra.

[0042] - An angle determination unit, which is designed to calculate and output the angle between the superior endplate and the inferior endplate of the reference vertebra.

[0043] Although the technical function of the unit can be derived from the description of the method, it should be noted that the segmentation unit can use a single algorithm to segment two end plates or use a separate algorithm for each end plate, wherein the algorithm is preferably a neural network.

[0044] It should be noted that, compared to existing technologies, the angle is determined based on the inclination of the endplates, wherein the reference vertebrae including these endplates are selected based on the local inclination (angle) of the centerline. Therefore, this invention combines the various advantages of existing methods.

[0045] A key advantage of using a centerline to determine spinal curvature angles is its greater robustness to limited X-ray image quality. Image quality can be limited by factors such as suboptimal exposure settings, patient movement, foreign bodies, and overlapping anatomy. In these cases, identifying endplate orientation can be challenging, especially under pathological spinal conditions. Using a centerline to determine curvature is generally robust to outliers of individual endplate orientations.

[0046] A key advantage of using vertebral endplates to measure spinal curvature angles is that these landmarks are used in the standard procedure for measuring the Cobb angle. This will improve the acceptance and practicality of routine clinical measurements.

[0047] Some units or modules of the aforementioned devices can be implemented, wholly or partially, as software modules running on the processor of a computing system. Implementation primarily as software modules offers the advantage of allowing applications already installed on existing systems to be updated with relatively little effort to install and run the units of this application. The object of the invention is also achieved by a computer program product having a computer program that can be directly loaded into the memory of a computing system, and the computer program including program units that perform the steps of the method of the invention when executed by the computing system. In addition to the computer program, such a computer program product may also include other parts, such as documentation and / or additional components, and hardware components, such as hardware keys (dongles, etc.), to facilitate access to the software.

[0048] Computer-readable media, such as memory sticks, hard disks, or other removable or permanently mounted carriers, can be used to transport and / or store executable portions of computer program products, enabling these executable portions to be read from the processor unit of a computing system. The processor unit may include a microprocessor or its equivalent.

[0049] Particularly advantageous embodiments and features of the invention are given by the dependent claims, as disclosed in the following description. Features from different claim classes may be suitably combined to provide other embodiments not described herein.

[0050] According to a preferred method, the images include coronal and / or sagittal views of the patient, and are preferably X-ray images, computed tomography (CT) images, ultrasound images, or magnetic resonance (MR) images. When the images are tomographic in nature, they particularly include multiplanar reformatted slices.

[0051] It should be noted that 3D images can also be used, and the angles between the endplates can be determined in 3D space. However, in clinical practice, angles in the coronal or sagittal planes are typically used for diagnosis. Therefore, in the case of 3D images such as CT or MR images, even if they consist of a series of slices, they are first reduced to 2D images showing the sagittal or coronal planes before the method according to the invention is applied. At least, when the method is applied to 3D images, angles are preferably calculated only in the coronal or sagittal planes, especially in the geometry of the endplates, and particularly in the local inclination of the centerline.

[0052] According to the preferred method, the detection of the center point (of the vertebrae) is performed using a trained machine learning algorithm, specifically a deep neural network. Such an algorithm can be trained on an image showing a spine with associated vertebrae as training data, where information about the segmented vertebrae, along with their center points, serves as the ground truth. The preferred algorithm is a neural network, particularly a convolutional network, commonly used in image processing.

[0053] According to the preferred method, the centerline is constructed as a smooth line or smoothed after the initial construction. This has the advantage that it is optimized to follow the centerline of the actual spine because its path is smooth. Suitable techniques for smoothing the line are well known in the art.

[0054] According to the preferred method, the segmentation of the endplate of a reference vertebra is determined by using a trained machine learning algorithm, particularly a deep neural network. Such an algorithm can be trained on images showing a spine with segmented vertebrae or only segmented vertebrae as training data, where information about the location of the endplate of the segmented vertebrae serves as ground truth. The preferred algorithm is a neural network, particularly a convolutional network, commonly used in image processing. Depending on the application, one algorithm can be used for both endplates, or one algorithm specifically trained to find the upper endplate and another specifically trained to find the lower endplate can be used.

[0055] According to a preferred method, the segmentation of the endplate is determined by detecting a certain number (e.g., six) point landmarks on the endplate and then by linear fitting. Alternatively or additionally, the segmentation of the endplate is determined by a landmark regression method that assigns a probability to each pixel in the image that it belongs to the endplate of that vertebra, and in particular, also by fitting. Preferably, during the preprocessing step, a region of interest is defined around a reference vertebra, which is used to crop the image prior to analysis.

[0056] Regarding the preferred equipment, the detection of the boundary markers can be implemented in the first module ("boundary marker module"), and the post-processing of the boundary markers can be implemented in the second module ("post-processing module").

[0057] Therefore, the center point unit, and especially the center line unit, can be implemented in the landmark module together with the segmentation unit or at least a portion of that unit. The purpose of these units is typically to determine landmarks, and they can be implemented using a single (deep) neural network or a neural network for each unit (and in the case of the segmentation unit, a separate (deep) neural network for each endplate).

[0058] The line tilt unit, the maxima / mina unit, and the unit for performing end-plate point fitting can be implemented in the post-processing module, where the end-plate point fitting unit can be part of the segmentation unit. Therefore, the segmentation unit can have two parts. The first part is a detector for identifying individual end plates (e.g., multiple points on the end plate), and the second part can be the unit performing end-plate point fitting. The first part can be part of a first module, and the second part can be part of a second module.

[0059] The third module may include an angle determination unit, and in particular, a unit designed for calculating coronal / sagittal balance.

[0060] According to the preferred method, steps d) to g) are repeated using additional reference vertebrae and preferably additional local positive and negative tilt maximum values ​​and / or different portions of the centerline. Preferably, the additional local positive or negative tilt maximum value closest to the centerline or the additional reference vertebra closest to the previous reference vertebra is selected. It should be noted that in the case of more than one positive or negative tilt maximum value, the largest tilt maximum value is selected first, and then, in another cycle, another (especially the next smaller) tilt maximum value is used. Preferably (to be considered), the Cobb angle of the spinal curve should be >10 degrees. If there are several peaks on the centerline tilt function, it is preferable to select adjacent peaks for minimum and maximum value determination.

[0061] According to the preferred method, the angle is calculated based on the segmentation of the endplate, and also based on the intermediate angle measured according to the local tilt of the centerline at the location of the endplate. This has the advantage of allowing for a health check of the vertebral tilt (“endplate geometry”). This tilt is related to the tilt height of the centerline. In the presence of high deviations, values ​​based on the centerline tilt are generally more reliable.

[0062] According to the preferred method, coronal or sagittal balance is further calculated based on the center point of the vertebra, particularly based on the centerline. Preferably, coronal balance is measured as the horizontal distance between the center points of C7 and S1 on a coronal spine image. Preferably, sagittal balance is measured as the horizontal distance between the center point of C7 and the posterosuperior angle of S1 on a sagittal spine image. The device preferably includes a distance determination unit designed to calculate coronal or sagittal balance.

[0063] According to the preferred method, in step g), a Cobb-like angle between the superior and inferior endplates of the reference vertebra is calculated. Alternatively, the angle of quantified thoracic kyphosis, the angle of quantified lumbar lordosis, or another angle in the sagittal plane, which is a common spinal measurement between the superior and inferior endplates of the reference vertebra, is calculated.

[0064] According to the preferred method, step g) includes sub-steps (for each final board):

[0065] - Determine the E angle using endplate orientation measurements.

[0066] - Determine the angle L as the intermediate angle measured based on the local centerline inclination at the intersection of the centerline and the end plate.

[0067] - Compare angle E with angle L.

[0068] The superior endplate (cranial vertebral body) and the inferior endplate (coccygeal vertebral body) were examined separately.

[0069] Preferably, when angle E deviates from angle L by a predetermined value, a corrected angle C is determined such that angle C is closer to angle L than angle E. Then, angle C is used as the determined angle by this method and that angle is output. This advantageously increases the robustness of the method.

[0070] Therefore, the angle measured using the end plate (E angle) is compared with the intermediate angle (L angle) measured based on the tilt of the local centerline. These two angles are highly correlated and should not deviate too much. If the E angle deviates from the L angle by a certain value, its value is adjusted here to make the resulting C angle closer to the L angle. It is assumed that the L angle can be determined more robustly compared to the E angle.

[0071] According to the preferred method, the corrected C angle is calculated based on the E angle and L angle using the following formula by using the weighting function w: C angle = w·L angle + (w–1)·E angle.

[0072] Preferably, w is determined using the absolute value of the difference between angle E and angle L, d = abs(angle E - angle L), and using predefined scalar values ​​a and b through a sigmoid function w = 1 / (1 + exp(-a·(db))) or a sigmoid function written in other ways below:

[0073]

[0074] The parameters a and b here are merely scalar values ​​that define the shape of the sigmoid function w. The value b can be interpreted as the acceptable deviation between the E angle and the L angle.

[0075] In a preferred embodiment of the invention, components of the device are part of a data network, wherein, preferably, the data network and a medical imaging system (i.e., an X-ray system providing image data) communicate with each other, wherein the data network preferably includes part of an Internet and / or a cloud-based computing system, wherein preferably, the device according to the invention or at least several components of the device are implemented in the cloud-based computing system. For example, components of a system are part of a data network, wherein, preferably, the data network and a medical imaging system providing image data communicate with each other. Such a networking solution can be implemented via an Internet platform and / or in a cloud-based computing system.

[0076] This method may also include elements of "cloud computing." In the field of cloud computing, IT infrastructure is provided via data networks, such as storage space or processing power and / or application software. Communication between the user and the "cloud" is achieved through data interfaces and / or data transmission protocols.

[0077] In a “cloud computing” environment, in a preferred embodiment of the method according to the invention, data is provided to the “cloud” via a data channel (e.g., a data network). This “cloud” includes (remote) computing systems, such as computer clusters that typically do not include the user’s local machine. In particular, this cloud can be used by medical facilities that also provide medical imaging systems. Specifically, image acquisition data is transmitted to the (remote) computer system (“cloud”) via a RIS (Radiology Information System) or PACS (Picture Archiving and Communication System).

[0078] Within the scope of a preferred embodiment of the system according to the invention, the aforementioned units (center point unit, center line unit, line tilt unit, maximum and minimum unit, segmentation unit, angle determination unit) exist on the "cloud" side. A preferred system further includes a local computing unit connected to the system via a data channel (e.g., particularly a data network configured as RIS or PACS). The local computing unit includes at least one data receiving interface for receiving data. Furthermore, preferably, the local computer also has a transmission interface for sending data to the system.

[0079] This invention provides a method for automatically determining angles similar to the Cobb angle, which is robust to local anomalies in endplate orientation and meets the need for reporting values ​​based on accepted anatomical landmark measurements. It should be noted that, formally, this method does not measure the traditional Cobb angle, but rather uses, as an intermediate step, a local tilt (angle) of a smoothed centerline to locate a reference vertebra. However, from a clinical perspective, this method considers global curvature in a natural way, which is beneficial for characterizing spinal curvature.

[0080] Other objects and features of the invention will become apparent from the following detailed description taken in conjunction with the accompanying drawings. However, it should be understood that the drawings are designed for illustrative purposes only and are not intended to define limitations on the invention. Attached Figure Description

[0081] Figure 1 The curvature of the spine in the coronal and sagittal planes and the definition of the Cobb angle are shown.

[0082] Figure 2 The curvature of the spine in the coronal plane is shown.

[0083] Figure 3 The curvature of the spine and the corresponding angle function in the coronal plane are shown.

[0084] Figure 4 It shows Figure 1 Cobb's angle and Figure 3 The analysis of the relationship between Cobb angles.

[0085] Figure 5 A block diagram illustrating the processing flow of a preferred method according to the present invention is shown.

[0086] Figure 6 A device according to the present invention is shown.

[0087] Figure 7 The workflow of the present invention is shown.

[0088] In the accompanying drawings, the same reference numerals always denote the same objects. The objects in the accompanying drawings are not necessarily drawn to scale. Detailed Implementation

[0089] Figure 1 The curvature of the spine S and its vertebrae V in the coronal plane are shown: a C-shaped spine (left) and an S-shaped spine (right). By conventional definition, the Cobb angle C is the angle between the most inclined vertebra above and below the apex of the curve (left). It should be noted that for the S-shaped spine, two Cobb angles can be measured. The angle is measured between the superior endplate of the upper vertebra and the inferior endplate of the lower vertebra. Therefore, the Cobb angle depends on the local endplate orientation and neglects important aspects of the curve's characteristics.

[0090] Figure 2 The curvature of the spine S in the coronal plane is shown. It can be seen that... Figure 1 The Cobb angle and the inclination T of the centerline L of the vertebral V of the spine are determined. It can be seen that the Cobb angle and the angle generated by the inclination T of the centerline at each endplate location are highly correlated with each other, but slightly different from each other.

[0091] Figure 3 The diagram shows the curvature of the spine S in the coronal plane (left) and the corresponding angle function (right). The angle function is the first derivative of the central line L of the spine S with respect to the Z-coordinate Z. The central line L of the spine is also designated as a "composite structural curve," and the angle function is calculated directly from this curve. The curvature of the spine can be determined by finding the center point of the vertebra, constructing a smooth central line through the center point, and using the curvature of the central line to determine an angle similar to a Cobb angle. Mathematically, this involves calculating the "angle function" (the inclination of the central line relative to the horizontal line at each elevation point) and taking the negative and positive maximum values ​​as reference points for measurement. This concept is also known as "analyzing the Cobb angle."

[0092] Figure 4 It shows Figure 1 (Traditional) Cobb angle C and Figure 3 The analysis examines the relationship between Cobb angles C. It can be seen that the relationship is close, but not 100% identical.

[0093] Figure 5 A block diagram of a preferred method according to the present invention is shown, which is used to process images IM of multiple vertebrae V of the spine S (e.g., see...). Figure 3 Automatically determines spinal deformities.

[0094] This method can be divided into two parts. The first part (steps I to IV) involves finding reference vertebrae R1 and R2 based on the local centerline tilt T. The second part (steps V and VI) involves calculating the angle based on the superior / inferior endplate tilt T of these reference vertebrae R1 and R2.

[0095] In step I, the center points P of multiple vertebrae V shown in image IM are detected. In practice, the algorithm (a trained deep neural network, "Model A") detects the center points P (from C1 to S1) of the vertebrae V in the coronal X-ray image IM shown. Where not all vertebrae V are visible, only the visible center points P are detected.

[0096] In step II, a (smoothed) centerline L is constructed based on the detected center point P. In this example, a smoothed spline is used to calculate the centerline.

[0097] In step III, the local tilt T is calculated at each point along the centerline L. This can be achieved by calculating the first derivative along the curve.

[0098] In step IV, the maximum positive tilt value M+ and the maximum negative tilt value M- of the local tilt T are determined, and two reference vertebrae R1 and R2 with center points P that are closest to the determined tilt values ​​M+ and M- are selected.

[0099] In step V, the segmentation of the endplates E1 and E2 of the reference vertebrae R1 and R2 is determined. This involves the segmentation of the superior endplate E1 of the cranial reference vertebra R1 and the segmentation of the inferior endplate E2 of the caudal reference vertebra R2.

[0100] More cranial reference vertebrae R1 can be analyzed using a second algorithm (a trained deep neural network, "Model B1") that detects the superior endplate E1 of the reference vertebrae R1. This can be achieved, for example, by detecting a certain number (e.g., six) point landmarks on the endplate E1 and then performing a linear fit. Another implementation could be a landmark regression method that assigns a probability (also through fitting) to each pixel in the image that belongs to the endplate E1 of the reference vertebrae R1. As a preprocessing step, a region of interest can be defined around the reference vertebrae R1, which is used to crop the image IM before analysis by the second algorithm.

[0101] The same action is performed on the reference vertebra R2 on the second caudal side using a third algorithm (a trained deep neural network, "model B2") to locate the inferior endplate E2. The second algorithm model B1 and the third algorithm model B2 can be separately trained networks or the same trained network.

[0102] The three models used to identify elements in the image IM (model A for identifying vertebra V and its center point P, and models B1 and B2 for identifying endplates E1 and E2) can be hosted in special “landmark modules”, such as in powerful computing units, especially in the cloud.

[0103] In step VI, the angle A between the superior endplate E1 of reference vertebra R1 and the inferior endplate E2 of reference vertebra R2 is calculated and output. Based on the orientation of the superior endplate E1 and the inferior endplate E2, a Cobb-like angle can be calculated from image IM.

[0104] If the spine has an S-shape, more than one angle A should usually be reported (usually two, sometimes three). In this case, repeat steps IV through VI for different segments along the curve or for different portions of the different maximum values ​​M+, M- and / or the centerline L.

[0105] Figure 6 A device 1 according to the present invention is shown, the device 1 being used for... Figure 5 The method shown automatically determines spinal deformities based on images IM of multiple vertebrae V of the spine S. Device 1 includes the following components:

[0106] Center point unit 2 is designed to detect the center point P of the multiple vertebrae V shown in image IM. This unit may include a (deep) neural network specifically trained for this purpose.

[0107] Centerline unit 3 is designed to construct centerline L based on detected center point P. This unit may include a conventional algorithm designed to construct a smooth line based on splines passing through the detected center point P.

[0108] Line tilt unit 4 is designed to calculate the local tilt T at points along the centerline L. This unit can calculate the first derivative of the centerline L with respect to its length or the Z-axis (patient height).

[0109] The maximum and minimum unit 5 is designed to determine the positive tilt maximum value M+ and the negative tilt maximum value M- of the local tilt T, and to select two reference vertebrae R1 and R2 with a center point P that is closest to the determined tilt maximum values ​​M+ and M-. This can be easily achieved by looking at the local maximum value of the first derivative or by calculating the second derivative and finding the zero value at the second derivative.

[0110] Segmentation unit 6 is designed to segment the superior endplate E1 of the cranial reference vertebra R1 and the inferior endplate E2 of the caudal reference vertebra R2. This can be achieved using the aforementioned models B1 and B2 (or a single model B). Therefore, the unit may also include a (deep) neural network or two (deep) neural networks specifically trained for this purpose.

[0111] Angle determination unit 7 is designed to calculate and output the angle A between the superior endplate E1 of reference vertebra R1 and the inferior endplate E2 of reference vertebra R2. Given the inclination and position of the endplates (i.e., geometric conditions), angle A can be easily calculated, for example, by forming 2D vectors with the inclinations of endplates E1 and E2 at their positions and calculating the angle A between these two vectors.

[0112] Figure 7 The workflow of the present invention is illustrated herein. Figure 6 The units of device 1 described herein are distributed in three modules: marker module 8, post-processing module 9, and measurement module 10.

[0113] Center point unit 2, and especially centerline unit 3, together with endplate unit 6a (which performs endplate point fitting to segment endplates E1, E2), are part of landmark module 8. The purpose of these units is typically to determine landmarks, and they can be implemented using a single (deep) neural network or a neural network for each unit (and in the case of endplate unit 6a, a separate (deep) neural network for each endplate E1, E2). Landmark module 8 can be arranged in a cloud that receives the image IM and provides the results (i.e., a segmented image IM with vertebrae V and center point P, and the identified endplates E1, E2). Endplate unit 6a can be a unit of segmentation unit 6 or center point unit 2. Therefore, segmentation unit 6 can be distributed in both module 8 and module 9. The first part of segmentation unit 6 is endplate unit 6a, designed to identify the corresponding endplate (e.g., multiple points of the endplate), and the second part of segmentation unit 6 is the unit that performs endplate point fitting after receiving the points generated by the first part.

[0114] According to the landmark module 8, the center points P of the identified vertebrae V (e.g., their coordinates) are sent to the post-processing module 9. Furthermore, landmarks, such as six points characterizing the geometry of the endplates E1 and E2, are also sent to the post-processing module 9. Therefore, in the case where the landmark module 8 is implemented in the cloud, the image IM must be uploaded to the cloud, which can be a large dataset, but the results received from the cloud can be a small dataset (coordinates only).

[0115] Line tilting unit 4, maximum and minimum unit 5, and segmentation unit 6 can be implemented in post-processing module 9.

[0116] The measurement module 10 includes an angle determination unit 7. The calculation of the coronal balance is also included in this diagram. The coronal balance is measured as the horizontal distance between the center points of C7 and S1. Therefore, this module may also include a distance determination unit 11.

[0117] All of these modules can be implemented in the cloud.

[0118] Although the present invention has been disclosed in the form of preferred embodiments and variations thereof, it should be understood that many other modifications and variations can be made to the invention without departing from its scope. For clarity, it should be understood that throughout this application, the use of "a" or "an" does not exclude a plurality, and "comprising" does not exclude other steps or elements. The expression "a plurality" means "at least one." References to "unit" or "device" do not exclude the use of more than one unit or device.

Claims

1. A method for automatic determination of a spinal deformity from an image (IM) showing a plurality of vertebrae (V) of the spine (S), the method comprising the steps of: a) detecting center points (P) of the plurality of vertebrae (V) shown in the image (IM), b) constructing a center line (L) based on the detected center points (P), c) calculating a local tilt (T) at points along the center line (L), d) determining a positive tilt maximum (M+) and a negative tilt maximum (M-) of the local tilt (T) and selecting two reference vertebrae having center points (P) closest to the determined tilt maximums (M+, M-), e) segmenting the upper endplate of the cranial reference vertebra, f) segmenting the lower endplate of the caudal reference vertebra, g) calculating an angle (A) between the upper endplate of the reference vertebra and the lower endplate of the reference vertebra and outputting the angle (A), wherein the angle (A) is calculated based on the segmentation of the upper endplate and the lower endplate and additionally from an intermediate angle measured from the local tilt (T) of the center line (L) at the location of the upper endplate and the lower endplate.

2. The method of claim 1, wherein, The image (IM) comprises a view on the patient in coronal or sagittal plane.

3. The method of claim 2, wherein, The image (IM) is an X-ray image, a computed tomography image, an ultrasound image or a magnetic resonance image.

4. The method of claim 2, wherein, In case of tomographic imaging data, the image (IM) comprises a multiplanar reformatted slice.

5. The method of any one of claims 1 to 4, wherein, The detection of the center points (P) is performed with a trained machine learning algorithm.

6. The method of claim 5, wherein, The trained machine learning algorithm is a deep neural network.

7. The method of any one of claims 1 to 4 and 6, wherein, The center line (L) is constructed as a smooth curve or is smoothed after an initial construction.

8. The method of any one of claims 1 to 4 and 6, wherein, The segmentation of the endplates of the reference vertebrae is determined by using a trained machine learning algorithm.

9. The method of claim 8, wherein, The trained machine learning algorithm is a deep neural network.

10. The method of any one of claims 1-4, 6, and 9, wherein, The segmentation of the endplates is determined by: - detecting a number of point landmarks on the endplate followed by a linear fitting, and / or - a landmark regression method assigning to each pixel in the image a probability of belonging to the endplate of the reference vertebra.

11. The method of claim 10, wherein, During a preprocessing step, a region of interest is defined around the reference vertebrae which is used to crop the image (IM) before analysis.

12. The method of any one of claims 1 to 4, 6, 9, and 11, wherein, Steps d) to g) are repeated with further reference vertebrae and / or different portions of the center line (L).

13. The method of claim 12, wherein, Steps d) to g) are repeated with the further reference vertebrae and further local positive tilt maximums (M+) and negative tilt maximums (M-) and / or different portions of the center line (L).

14. The method of any one of claims 1 to 4, 6, 9, 11, and 13, wherein, Additionally, a coronal balance or a sagittal balance is calculated based on the center points (P) of the vertebrae (V).

15. The method of claim 14, wherein, The coronal balance or the sagittal balance is calculated based on the center line (L).

16. The method of claim 14, wherein, The coronal balance is measured as a horizontal distance between the center points (P) of vertebra C7 and vertebra S1 on a coronal spine image and the sagittal balance is measured as a horizontal distance between the center point of vertebra C7 and the posterousuperior corner of vertebra S1 on a sagittal spine image.

17. The method of any one of claims 1 to 4, 6, 9, 11, 13, 15, and 16, wherein, In step g), the angle (A) is calculated as - an angle similar to Cobb, - quantifying the angle of thoracic kyphosis, - quantifying the angle of lumbar lordosis, or - another angle in the sagittal plane, which is a common spinal measurement between the upper and lower endplates of the reference vertebrae.

18. The method of any one of claims 1 to 4, 6, 9, 11, 13, 15, and 16, wherein, Step g) comprises the following sub-steps: - determining an E-angle measured by means of the upper endplate and the lower endplate, - determining an L-angle, which is an intermediate angle measured from the local centerline inclination, - comparing the E-angle with the L-angle.

19. The method of claim 18, wherein, In case the E-angle deviates from the L-angle by more than a predetermined value, a corrected C-angle is determined such that the C-angle is closer to the L-angle than the E-angle, then the C-angle is used as the determined angle and is output.

20. The method of claim 18, wherein, A corrected C-angle is calculated from the E-angle and the L-angle and a weight function w by: C-angle = w · L-angle + (w - 1) · E-angle.

21. The method of claim 20, wherein, w is determined with the absolute value d = abs(E-angle - L-angle) and the pre-defined scalar values a and b by w = 1 / (1+exp(-a-(d-b))).

22. A device (1) for automatically determining a spinal deformity from an image (IM) showing a plurality of vertebrae (V) of the spine (S), the device (1) automatically determining a spinal deformity from the image (IM) using the method according to any one of claims 1 to 21, the device (1) comprising: - a center point unit (2) designed to detect center points (P) of the plurality of vertebrae (V) shown in the image (IM), - a center line unit (3) designed to construct a center line (L) based on the detected center points (P), - a line inclination unit (4) designed to calculate local inclinations (T) at points along the center line (L), - a max-min unit (5) designed to determine a positive inclination maximum (M+) and a negative inclination maximum (M-) of the local inclinations (T) and to select two reference vertebrae having center points (P) closest to the determined inclination maximums (M+, M-), - a segmentation unit (6) designed to segment an upper endplate of the cranial reference vertebra and designed to segment a lower endplate of the caudal reference vertebra, - an angle determination unit (7) designed to calculate an angle (A) between the upper endplate of the reference vertebra and the lower endplate of the reference vertebra and to output the angle (A), wherein the angle (A) is calculated based on the segmentation of the upper endplate and the lower endplate and in addition from an intermediate angle measured from the local inclinations (T) of the center line (L) at the positions of the upper endplate and the lower endplate.

23. A computer program product comprising a computer program directly loadable into the internal memory of a computing system, said computer program comprising program elements for performing the steps of the method according to any one of claims 1 to 21 when said computer program is executed by the computing system.

24. A computer readable medium having stored thereon program elements which can be read and executed by a computer unit and cause performance of the steps of a method according to any one of claims 1 to 21 when the program elements are read and executed by the computer unit.

Images