Method and system for computer-aided diagnosis based on morphological features extracted from 3D medical images - Patents.com

Abstract

A method and a system are provided for generating a machine learning model for predicting disease severity risk from a plurality of three-dimensional medical images, the method comprising the steps of: providing a plurality of three-dimensional medical images, each image of the plurality of three-dimensional medical images comprising at least one anatomical structure of a patient; obtaining at least one three-dimensional segmentation mask from each image of the plurality of three-dimensional medical images, each mask of the at least one three-dimensional segmentation mask comprising at least one anatomical structure; reconstructing a mesh from each mask of the at least one three-dimensional segmentation mask; extracting morphological features from each reconstructed mesh; and training a machine learning model for performing a classification of disease severity risk, the training being performed based on the extracted morphological features. A method and a system are further provided for predicting disease severity risk from a plurality of three-dimensional medical images by applying the machine learning model generated for the same purpose.

Description

[Technical field]

[0001] The present disclosure relates generally to methods and systems for diagnosis of pathology based on morphological properties computed on medical images.

[0002] More specifically, the present disclosure relates to machine learning models for predicting disease severity based on morphological characteristics. [Background technology]

[0003] Since the beginning of medicine, morphology or morphological characteristics have provided symptom phenotypes that enrich the information available to physicians to establish clinical diagnoses. Morphology deals with the study of the form and structure of living organisms and their specific structural features, especially 2- or 3-dimensional anatomical structures (e.g. organs or sub-tissues).

[0004] According to the general principle of correspondence between structure and function in biology, malfunction of a biological process is always accompanied by a pathological deformation of the underlying biological structure.

[0005] To utilize morphology to facilitate clinical diagnosis, U.S. Patent No. 10130295 B2, entitled "Methods for the detection and staging of liver fibrosis from image acquired data," proposes using a quantitative method to measure liver surface nodularity and calculate a liver surface nodularity score. A morphological feature is a function, process, or observation that calculates a scalar from the approximated geometry or topology of a 2D image or 3D volume. Unfortunately, morphological features are often difficult to quantify.

[0006] Several publications, such as Choi W. et al., "Reproducible and interpretable spiculation quantification for lung cancer screening," Computer Methods and Programs in Biomedicine, Vol. 200 (2021), and China Patent No. 113066583A, entitled "Aneurysm rupture risk prediction method and device and storage medium," are directed toward characterizing the deformation of organs or subtissues by objectively quantifying morphological characteristics.

[0007] However, such quantification is limited to rough and coarse geometric quantifiers, such as functions of distance, area, or volume, which usually do not capture the complete information of the morphology.

[0008] Moreover, systems that use such quantities to perform diagnostic predictions are unable to achieve high prediction accuracy.On the other hand, since diseases often result in anisotropic or localized expansion, local deformations of organs or sub-tissues are usually good geometric and topological indicators of pathology. [Prior art documents] [Patent documents]

[0009] [Patent Document 1] U.S. Patent No. 10130295B2 [Patent Document 2] Chinese Patent No. 113066583A [Non-patent literature]

[0010] [Non-Patent Document 1] Choi W. et al., “Reproducible and interpretable spiculation quantification for lung cancer screening,” Computer Methods and Programs in Biomedicine, Volume 200 (2021) [Non-Patent Document 2] Leonardi V. et al., "Multiple reconstruction and dynamic modeling of 3d digital objects using a morphing approach," The Visual Computer, Vol. 31 (June 2014). Summary of the Invention

[0011] Therefore, a desirable approach to accurately predict disease is to appropriately utilize morphological features, in particular to optimize their performance with respect to predicting disease severity, as assessed, for example, by AUC-ROC score. [Means for solving the problem]

[0012] According to a first aspect, a method for generating a machine learning model for predicting risk of disease severity from a plurality of three-dimensional medical images is disclosed, the method comprising: - providing a plurality of three-dimensional medical images, each image of the plurality of three-dimensional medical images including at least one anatomical structure of a patient; - obtaining at least one 3-dimensional segmentation mask from each image of the plurality of 3-dimensional medical images, each mask of the at least one 3-dimensional segmentation mask including at least one anatomical structure; - reconstructing a mesh from each of the 3D segmentation masks; - extracting morphological features from each reconstructed mesh; - training a machine learning model to perform a classification of disease severity risk, the training being performed based on the extracted morphological features; Includes.

[0013] In some examples, reconstructing a mesh from each three-dimensional segmentation mask includes extracting a mesh from the three-dimensional segmentation mask by using marching cubes, preferably by using Ruiner marching cubes. Furthermore, reconstructing a mesh from each three-dimensional segmentation mask further includes simplifying each reconstructed mesh to remove inconsistencies. In addition, reconstructing a mesh from each mask of the at least one three-dimensional segmentation mask may further include smoothing each reconstructed mesh after simplifying it to detect and remove artifacts. The smoothing step may be determined based on anatomical structures included in the segmentation mask, but is preferably determined based on the anatomical structures included in the segmentation mask. - Anisotropic and / or Laplacian smoothing, or - Anisotropic and / or Taubin smoothing can be determined from among

[0014] In some examples, the morphological features may include at least a distribution of the curvature of the reconstructed mesh, preferably a distribution of discrete curvatures of the reconstructed mesh. Furthermore, extracting the morphological features may include calculating the distribution of the curvature of the reconstructed mesh and the associated mesh energy. More specifically, calculating the distribution of the curvature of the reconstructed mesh may include: - calculating the mean curvature or Gaussian curvature on the mesh, preferably by using an approximation of the Steiner formula; - calculating a probability distribution of the mean curvature or Gaussian curvature over the mesh; - binning said probability distribution into a distribution histogram; may include.

[0015] In addition to the distribution of curvature, the morphological features may further include baseline features, which include: - a volume of said at least one anatomical structure; - the elongation of said at least one anatomical structure; - a minimum and / or maximum size of an oriented bounding box of said at least one anatomical structure, and - average HPI value of said at least one anatomical structure; may include at least one of:

[0016] In some examples, the machine learning model that is trained is a supervised classifier model, preferably a decision tree-based classifier.

[0017] In some examples, the method according to the first aspect of the invention comprises: - providing clinical information about each patient; Training is further performed on the patient's clinical information.

[0018] In some examples, the method according to the first aspect of the invention comprises: - extracting radiomics features other than morphological features from the plurality of three-dimensional medical images; Training is further performed on the other radiomics features.

[0019] The other radiomics features may include at least one of the latest library of radiomics provided by Van Griethuysen et al., including standard statistics of Haralick GLCM (gray level co-occurrence matrix), GLRLM (gray level run length matrix), GLSZM (gray level size zone matrix), NGTDM (neighborhood gray tone difference matrix), GLDM (gray level dependency matrix) features, image gray level intensity features and classical morphological relationship features.

[0020] In some examples, training is performed based on said morphological features and at least one type of features among the patient's clinical information and said other radiomics features.

[0021] According to a second aspect, a method for predicting disease severity risk from a plurality of three-dimensional medical images by applying a machine learning model is disclosed, comprising: - receiving a plurality of three-dimensional medical images, each image of the plurality of three-dimensional medical images including at least one anatomical structure of a patient; - providing one of a plurality of medical images as a candidate image for prediction; - obtaining at least one candidate 3D segmentation mask from said candidate image, each mask of said at least one candidate 3D segmentation mask including at least one anatomical structure; - reconstructing a candidate mesh from each of said at least one candidate 3D segmentation mask; - extracting morphological features from each reconstructed candidate mesh; - inputting the extracted morphological features into a machine learning model to perform a classification of disease severity risk; Includes.

[0022] In some examples, the machine learning model has been generated by implementing a method for generating a machine learning model according to the first aspect of the present invention.

[0023] In some examples, reconstructing a candidate mesh from each mask of the at least one candidate 3D segmentation mask includes extracting a mesh from the 3D segmentation mask by using Marching Cubes, preferably by using Ruiner Marching Cubes. Furthermore, reconstructing a candidate mesh from each mask of the at least one candidate 3D segmentation mask may further include simplifying each reconstructed candidate mesh to remove inconsistencies. In addition, reconstructing a candidate mesh from each mask of the at least one candidate 3D segmentation mask may further include smoothing each reconstructed candidate mesh after simplifying it to detect and remove artifacts. The smoothing step may be determined based on candidate anatomical structures included in the candidate segmentation mask, but is preferably determined based on the following: - Anisotropic and / or Laplacian smoothing, or - Anisotropic and / or Taubin smoothing can be determined from among

[0024] In some examples, the morphological features may include at least a distribution of curvatures of the reconstructed candidate mesh, preferably a distribution of discrete curvatures of the reconstructed candidate mesh. Further, extracting the morphological features includes calculating the distribution of curvatures of the reconstructed candidate mesh and the associated energy of the candidate mesh. More specifically, calculating the distribution of curvatures of the reconstructed candidate mesh includes: - calculating the mean curvature or Gaussian curvature on the mesh, preferably by using an approximation of the Steiner formula; - calculating a probability distribution of the mean curvature or Gaussian curvature over the mesh; - binning said probability distribution into a distribution histogram; may include.

[0025] In addition to the distribution of curvature, the morphological features may further include baseline features, which include: - a volume of said at least one anatomical structure; - the elongation of said at least one anatomical structure; - a minimum and / or maximum size of an oriented bounding box of said at least one anatomical structure, and - average HPI value of said at least one anatomical structure; may include at least one of:

[0026] According to either the first or second aspect of the invention, the three-dimensional medical images may comprise computed tomography (CT) images and / or magnetic resonance imaging (MRI) images.

[0027] A further aspect of the present disclosure relates to a non-transitory computer-readable medium storing instructions which, when executed by at least one digital processing device, perform at least the steps of the method according to the first aspect of the invention and / or the steps of the method according to the second aspect of the invention.

[0028] A further aspect of the present disclosure relates to a computer program product comprising instructions which, when executed by at least one digital processing device, perform at least the steps of the method according to the first aspect of the invention and / or the steps of the method according to the second aspect of the invention.

[0029] Other objects, features, and advantages of the inventions disclosed herein, and various embodiments / aspects thereof, will become apparent in light of the following description of several exemplary embodiments.

[0030] Reference will now be made in detail to embodiments of the present disclosure, non-limiting examples of which may be illustrated in the figures of the accompanying drawings. Some of the figures may take the form of diagrams. Some elements in the figures may be shown exaggerated and other elements may be omitted for clarity of illustration.

[0031] The text appearing in the figures (legends, notes, reference numbers, and the like) are incorporated herein by reference. [Brief description of the drawings]

[0032] [Figure 1] 4 is a flowchart of some examples of a method according to a first aspect of the present disclosure. [Figure 2A] (On the right) illustrates the reconstructed mesh of a healthy liver (F0), where the curvature values ​​at each vertex are color-coded, and (on the left) illustrates the probability distribution of the mean curvature for a healthy liver. [Figure 2B] Illustrates (on the right) the reconstructed mesh of an advanced fibrosis stage liver (F4), where the curvature values ​​at each vertex are color-coded, and (on the left) the probability distribution of the mean curvature for an advanced fibrosis stage liver. [Figure 3A] FIG. 13 shows the performance of a predictive model using random forest classification for F0-F2 versus F3-F4 stages. [Figure 3B] FIG. 13 shows the predictive power for each feature using random forest classification for F0-F2 vs. F3-F4 stages. [Figure 4A] (On the right) illustrates a reconstructed mesh of a benign pulmonary nodule, where the curvature value of each vertex is color-coded, and (on the left) illustrates the probability distribution of the mean curvature for benign pulmonary nodules. [Figure 4B] Illustrates (on the right) a reconstructed mesh of a malignant pulmonary nodule, where the curvature value of each vertex is color-coded, and (on the left) the probability distribution of the mean curvature for malignant pulmonary nodules. [Diagram 5] FIG. 13 shows the performance of malignancy prediction using only curvature features. [Figure 6] 5 is a flowchart of some examples of a method according to a second aspect of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0033] Lesions systematically induce morphological changes that constitute a major, but not fully quantified, source of observables for diagnosis. In order to use morphological characteristics for disease prediction, several proposals have been made to employ quantitative methods to measure lesions, in particular pathological deformations of organs or subtissues on medical images.

[0034] Lesions may refer to pathological features considered collectively, i.e. typical observable symptoms of a disease. In the context of the present disclosure, pathological deformations of an organ or sub-tissue may be, for example, liver surface nodular formation, pulmonary spiculated nodules, and may be extended to other types of deformations such as ruptured aneurysms, exophytic renal tumors, among other examples.

[0035] The above mentioned US Patent No. 10130295 B2 proposes image segmentation of liver edge boundary (1D) on 2D image or 3D volume, and then fits a one-dimensional spline (polynomial curve) to the detected liver boundary. The distance between the detected liver boundary and the spline is measured pixel by pixel and is called liver surface nodularity score (LNS). The larger the distance, the correspondingly larger the surface nodularity score, thus quantifying the appearance of nodules on the surface (by LNS). However, considering that the spline is fitted on a 2D image, it does not utilize the 3D neighborhood information and the entire morphology to extract morphological features. Moreover, the use of direct correlation between LNS score and diagnosis of liver fibrosis or cirrhosis may not meet performance expectations for disease prediction.

[0036] In lung tumor grading, spiculation is an important predictor of lung cancer malignancy, which is a surface thorn, while benign nodules are ovoid. Choi W. et al., "Reproducible and interpretable spiculation quantification for lung cancer screening," Computer Methods and Programs in Biomedicine, Vol. 200 (2021), proposes to quantify spiculation by reconstructing a 3D mesh from a segmented mask of the nodule and conformally mapping it to a sphere. Then, after measuring the area distortion of the conformal mapping, the maximum negative change in area distortion is used as a metric to identify spicules. A machine learning model (SVM-LASSO) is then used to predict the malignancy of the nodule. However, both the metric used by Choi et al. to characterize local deformations that depend on a single scalar value, and the method of predicting the malignancy of lung nodules based on such metric, need to be improved.

[0037] In Chinese Patent No. 113066583A, it is proposed to predict the risk of aneurysm by reconstructing a 3D mesh from a segmented mask of the aneurysm and extracting morphological features from the mesh, which are then used as input data in a machine learning model for prediction. The morphological features are various distance ratios, such as between the maximum height value and the average neck size, between the maximum height value and the maximum width value, between the maximum width value and the average neck size, and between the maximum height value and the size of the parent vessel. However, the distance ratios can only quantify all possible local deformations or irregularities in shape in a rough, partial, approximate manner, which may result in a less accurate prediction.

[0038] The study by Leonardi V. et al., "Multiple reconstruction and dynamic modeling of 3d digital objects using a morphing approach," The Visual Computer, Vol. 31 (June 2014), aims to detect kidney cancer and proposes to localize kidney morphological nephromas using a curvature-based approach on surface meshes, either by maximum curvature or by recursive labeling of vertices. However, since the goal of Leonardi V. et al. is the application of 3D meshes to kidney animation and tumor tracking, the curvature is extracted only for visualization purposes.

[0039] Moreover, the few studies that have introduced 3D methods that use surface meshes of organs propose to extract a single conformal mapping deformation parameter, resulting in a single, potentially less discriminatory feature (Choi), or to perform a recursive curvature labeling of vertices that leads to high-dimensional features and therefore is not suitable for prediction by machine learning models (Leonardi).

[0040] The present disclosure proposes to characterize local pathological deformations as quantities and to perform accurate prediction of disease based on these quantities, thereby making it possible to optimize performance for prediction of disease severity, as assessed, for example, by AUC-ROC score.

[0041] AUC-ROC is one of the most important evaluation metrics for measuring classification performance. The AUC-ROC score is a value in the range of 0 to 1. The higher the AUC-ROC, the better the model is at distinguishing between two classes. In multi-class classification, by using the One-vs-All method, we can plot as many ROC curves as the number of classes. For example, for characterization of liver fibrosis stages F0, F1, F2, F3, F4, we can have one ROC curve for F0 pair (F1-F4), one ROC curve for F1 pair (F0, F2-F4), one ROC curve for F2 pair (F0-F1, F3-F4), one ROC curve for F3 pair (F0-F2, F4), and one ROC curve for F4 pair (F0-F3). In this case, the AUC-ROC score can be calculated by the one-vs-rest (OvR) algorithm, which calculates the average of the ROC-AUC scores of each class relative to all other classes.

[0042] The disclosure proposed here aims to exploit 3D morphological properties by extracting specific morphological features that quantify local deformations and developing a method for morphological classification based on the extracted features.

[0043] In geometry, curvature is the degree to which a curve deviates from a straight line, or a curved surface deviates from a plane. There are positive and negative curvatures. A surface has positive curvature if, at a point, the surface curves away from the point in the same direction with respect to the tangent to the surface, regardless of the cutting plane. A surface has negative curvature if, at a point, the surface curves away from the tangent plane in two different directions.

[0044] The Gauss-Bonnet theorem states that the Gaussian curvature of a surface, which is an intrinsic property of a surface, does not change when you bend the surface without cutting it or adding holes, and is therefore topologically invariant in this sense. As an example, if you take a sphere and deform the sphere to the shape of a liver, and calculate the integral over the curvature, the integral will be the same for both the sphere and the deformed version, which is the liver. Now, a liver, which has a surface with no boundary and no topological holes, is homeomorphic to a sphere (i.e. a closed surface with no topological holes), and according to the theorem, the integral over the curvature is equal to 2×π×Euler characteristic.

[0045] Advantageously, the method of the invention proposes to extract not only the sum of all local curvatures but also their distribution over the entire liver surface, so that the extracted features are not invariant but rather convey information of all deformations (except torsion).

[0046] In the same example, the integral over the curvature remains the same for both the sphere and the liver, but the distribution of curvatures for the sphere and liver is different due to the local deformation that "deformed" the sphere into a liver. In this example, a liver with bumps and saddles on its surface, i.e., local deformations, may have some negative and positive curvatures, respectively, while a sphere has zero negative curvature. By the same principle, the distribution of curvatures for a healthy liver and a cirrhotic liver should be different. The energy given by the integral of the absolute value of the curvatures for those two shapes is also different.

[0047] Advantageously, in the context of the present disclosure, those differences may be used to facilitate easy differentiation of anatomical structures having different pathological modifications.

[0048] It is therefore advantageous to use the distribution of curvature to characterize local deformations. Using the distribution of curvatures as a morphological feature is particularly advantageous for disease prediction in that using the distribution of curvatures to quantify local pathological deformations makes it possible to distinguish surfaces of regular shape from surfaces with irregularities in shape, e.g., the wavy morphology of liver fibrosis and spikes that appear as pulmonary nodules.

[0049] To exploit the 3D morphological characteristics obtained from the 3D medical images, this disclosure proposes to segment each 3D medical image to obtain at least one 3D segmentation mask, and to reconstruct a mesh from each 3D segmentation mask, each 3D segmentation mask including at least one anatomical structure.

[0050] The term "segmentation" refers to the process of dividing at least one three-dimensional medical image into anatomical regions (a collection of voxels, also known as image objects). In the context of this disclosure, segmentation refers to generating, for each three-dimensional medical image, a segmentation mask that includes at least one anatomical structure.

[0051] An "anatomical structure" is any biological entity that occupies space and is distinct from its surroundings. In the context of the present invention, an anatomical structure may be, for example, an organ, muscle, soft tissue, artery, vein, bone, accessory tissue, etc. Examples of organs may be the liver, spleen, lungs, etc. Examples of accessory tissues may be tumors, which are abnormal growths of body tissue, such as lung nodules. Tumors may be cancerous (malignant) or non-cancerous (benign). Additional examples of accessory tissues may be aneurysms, which are bulges that form in the thinned walls of arteries, such as cerebral aneurysms and thoracic aortic aneurysms. It is noted that the present invention is not limited to the anatomical structures to the listed examples.

[0052] It will be understood that in the context of the present invention, curvature as extracted from a reconstructed mesh refers to the degree to which a curve deviates from a straight line and is expressed as a quantity, for example the curvature at a point (vertex), i.e. the Gaussian curvature or the mean curvature, is a scalar value.

[0053] In other words, by reconstructing a mesh of the boundary of a 3D segmentation of an organ or sub-tissue and then extracting morphological features from the reconstructed mesh, the morphological features are calculated on the geometrized mesh and correspond to certain topological invariants such as Gaussian curvature (in the sense of the Bonnet-Gauss theorem or of isometric diffeomorphism invariance), or correspond to the mean curvature, or correspond to other functions for distinguishing depressions from vertices, or concaves from convexities, or ridges from valleys, thus making it possible to make the most of the 3D morphological information obtained from the 3D medical images to characterize local pathological deformations.

[0054] The present disclosure proposes to use curvature to characterize the local deformation of organ and sub-tissue boundaries, then calculate the distribution of these curvatures, which can be used to generate machine learning models for statistically quantified prediction of disease severity with high accuracy.Two examples of applying the proposed solution for clinical prediction are provided, namely liver fibrosis stage and lung nodule grade.

[0055] Furthermore, the present disclosure proposes to train a machine learning model on the extracted morphological features to perform disease severity risk classification from multiple 3D medical images.

[0056] Advantageously, the machine learning model is a supervised classifier model, preferably a decision tree-based classifier. In some examples, a random forest or an XGboost model may be used.

[0057] In the context of the present disclosure, the disease severity risk may correspond to a binary or multi-class classification.

[0058] In the case of binary classification, the present disclosure is applicable to diagnosing the stage of liver fibrosis, i.e., to distinguish between early (F0 to F2) and late (F3 to F4) stages.

[0059] In the case of binary classification, the present disclosure is applicable to the diagnosis of pulmonary nodules, i.e., to distinguish between benign and malignant.

[0060] In the case of multi-class classification, the present disclosure is applicable to the diagnosis of multiple stages of liver fibrosis (F0 to F4).

[0061] This disclosure proposes applying the generated machine learning model to perform classification on each candidate image of a plurality of 3D medical images.

[0062] FIG. 1 is a flowchart of some examples of a method according to a first aspect of the present disclosure.

[0063] Referring to FIG. 1, a method 100 for generating a machine learning model for predicting risk of disease severity from a plurality of three-dimensional medical images is shown.

[0064] As previously mentioned, in the context of the present invention, an anatomical structure may be, for example, an organ, a muscle, a soft tissue, an artery, a vein, a bone, a sub-tissue, etc. Although applications of the present disclosure are described in the context of the liver and the lungs, it will be understood that these applications may be extended to organs other than the liver and the lungs, such as the spleen.

[0065] While the present application is disclosed in the context of liver fibrosis and lung nodules, it will be understood that disease in the context of this disclosure may manifest in a defined anatomical structure, and the disease may not be limited to tumors, whether cancerous or not, and may not be limited to the liver or lungs. In the context of this disclosure, the anatomical changes resulting from the disease may take any shape, whether it be spikes or bumps, depressions or protrusions, so long as they have localized deformation. Furthermore, the anatomical changes may take the form of tissue breaks, such as ruptures or holes, and the like.

[0066] In the context of the present disclosure, it will be understood that "multiple" refers to at least three images as the minimum sampling size for training a successful machine learning model. That is, at least three three-dimensional medical images should be provided to generate a machine learning model for predicting disease severity risk. The term "multiple" is interchangeable with the term "set" (or "collection of") in the context of the present disclosure, provided that the latter meets the same requirement of the number of images. In the context of the present disclosure, it is assumed that there are at least three images among the multiple images that are different from each other.

[0067] The three-dimensional medical images may be CT and / or MRI images. The methods as proposed by this disclosure are not limited by how the three-dimensional medical images may be acquired.

[0068] The method for acquiring 3D medical images can be any MRI or CT scan, including with or without contrast, i.e. - Computed tomography (CT) scans of patients with and without the injection of contrast material prior to the scan; and - Whatever the sequence acquisition or reconstruction process, it may be applied to magnetic resonance imaging (MRI), with or without the injection of contrast agent prior to the scan.

[0069] In the context of this disclosure, the three-dimensional images may be referred to as “volumetric images.” In some examples, each three-dimensional medical image refers to all or a portion of a three-dimensional medical image.

[0070] 1, the method 100 includes providing 110 a plurality of three-dimensional medical images, each image of the plurality of three-dimensional medical images including at least one anatomical structure of a patient. It is noted that the at least one anatomical structure may be relevant to predicting a disease severity risk.

[0071] In the context of the present disclosure, it is necessary that at least three images that are different from one another are provided, but it is not necessary that two different images are from different patients, in other words, the present disclosure does not require that the image diversity be related to patient diversity.

[0072] As shown in FIG. 1, after step 110 of providing a plurality of three-dimensional medical images, method 100 may further include step 120 of obtaining at least one three-dimensional segmentation mask from each image of the plurality of three-dimensional medical images, each mask of the at least one three-dimensional segmentation mask may include at least one anatomical structure.

[0073] In the context of the present disclosure, it will be understood that when a three-dimensional segmentation mask "contains" at least one anatomical structure, the term "contains" is interchangeable with the term "delimits" and both may have the meaning of determining the boundary of at least one anatomical structure.

[0074] In some examples, obtaining 120 at least one three-dimensional segmentation mask from each image of the plurality of three-dimensional medical images may include segmenting 125 each image of the plurality of three-dimensional medical images such that at least one three-dimensional segmentation mask is generated for each three-dimensional medical image, each mask including at least one anatomical structure.

[0075] In some examples, one three-dimensional medical image includes or corresponds to at least one three-dimensional segmentation mask. In some examples, one three-dimensional segmentation mask includes or corresponds to at least one anatomical structure. In the context of the present disclosure, it is required that each image of the plurality of three-dimensional medical images provided to generate a machine learning model for predicting disease severity risk includes at least one anatomical structure. Thus, segmentation of the anatomical structure may enable prediction of disease severity risk to be performed on the segmented anatomical structure.

[0076] The segmentation mask may include, without limitation, at least one segmented anatomical structure. - at least one organ, such as the liver, lungs, and spleen; - at least one tumor, such as liver fibrosis, - A combination of organs and tumors, such as liver fibrosis, lungs with nodules.

[0077] In some examples, to facilitate post-processing, one segmentation mask may be further divided into at least one segmentation mask, so long as each segmentation mask contains at least one anatomical structure.

[0078] This disclosure does not limit how a three-dimensional medical image may be segmented. In some examples, the 3D segmentation of an organ or tissue may be obtained by automatically segmenting the organ or tissue using a supervised deep convolutional neural network. In some examples, the 3D segmentation may be obtained by semi-automatic or manual segmentation.

[0079] It should be understood that if an automatic segmentation method is deployed in the method of the present invention, this may advantageously lead to an automatic prediction. It should be noted that if manual or unsupervised segmentations are input, these may be compatible with the next steps of the method of the present invention. However, the prediction will not be performed automatically.

[0080] Still referring to FIG. 1, it can be seen that the method 100 may further include a step 130 of reconstructing a mesh from each of the at least one three-dimensional segmentation mask.

[0081] By "mesh" is meant a discrete surface of the boundary of a volume, which may be represented by voxels, i.e. a set of voxels in a region of interest. Said region of interest may for example be defined by a segmentation mask, which is typically a set of pixels in 2D or voxels in 3D, set to 1 to represent the region of interest, and other pixels or voxels outside the region of interest set to 0.

[0082] "Reconstructing" a mesh from a 3D mask means making an approximation of the surface of the volumetric mask. In the example where the liver is the organ of interest and the liver volume is represented in voxels, the liver surface may be reconstructed with a mesh. Based on the reconstructed mesh, morphological features are extracted and used to train a machine learning model.

[0083] In some examples, step 130 of reconstructing a mesh from each 3D segmentation mask may include step 133 of extracting a mesh from the 3D segmentation mask, then step 135 of simplifying each reconstructed mesh, and then step 137 of smoothing the simplified reconstructed mesh.

[0084] The step 130 of reconstructing a mesh from each 3D segmentation mask may further include a step 139 of performing optical smoothing after the reconstruction 130 and before the extraction 140 to close holes on the simplified mesh.

[0085] In some other examples, step 139 is performed after the smoothing step 137 that follows the simplification step 135. In still some examples, step 139 uses a Poisson reconstruction algorithm.

[0086] In fact, holes may be caused by previous improper segmentation. Due to those holes, the mesh will not be watertight, but such defects can be solved by step 139 of closing those holes. However, while closing the holes, the loss of physiological or morphological features (such as bumps and other diseases in the form of local deformations) may be unavoidable due to the Poisson reconstruction that allows to reconstruct the surface from a given point cloud. After balancing the pros and cons, the step 139 of optical smoothing may be optional. The optional aspect of this embodiment is depicted in FIG. 1 by the fact that the block is indicated with a dash. In some examples, step 139 may be omitted from the method 100. It is noted that omitting step 139 does not decrease the AUC-ROC score.

[0087] Step 133: Reconstruct mesh from 3D segmentation mask As mentioned above, in the context of this disclosure, extracting a mesh from a 3D segmentation mask (3D volume) step 133 refers to approximating a surface from a volume segmentation mask.

[0088] In some examples, step 133 may be accomplished by an algorithm that extracts a mesh from a 3D volumetric segmentation mask, such as marching cubes and its improvements.

[0089] The Marching Cubes algorithm is based on partitioning the input volume into a set of discrete cubes. In some examples of applying the Marching Cubes algorithm, the volume is given by a binary 3D segmentation mask of the organ. The algorithm then performs a triangulation according to a lookup table of 15 cube configurations for each cube intersected by the isosurface of the volume.

[0090] Additionally, in some examples, Ruiner Marching Cubes is used in step 133 to extract a mesh from the segmentation mask volume. Using Ruiner Marching Cubes in the method of the present invention is advantageous because it can resolve topological inconsistencies introduced by the Marching Cubes algorithm by reconstructing the volume with tetrahedrons. Using Ruiner Marching Cubes resolves ambiguities and ensures topologically correct results.

[0091] Step 135: Simplify each reconstructed mesh In some instances, the reconstructed mesh is simplified to remove inconsistencies or degeneracy such as duplicate vertices, duplicate faces, zero-area faces, etc.

[0092] In some further examples, the simplification step 135 may further improve the quality of the triangles by collapsing short edges and splitting long edges.

[0093] In some examples, when the Poisson reconstruction algorithm is used, the combination of the simplification step 135 and the optical smoothing step 139 may have the following advantages: Since the Poisson reconstruction algorithm is highly dependent on the vertex normals, pre-removing the degeneracy and ambiguity by step 135 helps to properly estimate the normals, and thus helps to achieve a geometrically more accurate mesh by closing holes in the Poisson reconstruction algorithm.

[0094] In some examples, simplifying the reconstructed mesh step 135 may further include tuning the number of iterations of the simplification 135 step 136. Step 136 may be optional. The optional aspect of this aspect is depicted in FIG. 1 by the fact that the block is indicated with a dash.

[0095] Indeed, a common result of using the marching cubes algorithm on medical images, step 133, is Z-sampling artifacts. Such artifacts are undesirable because they can introduce noise when calculating morphological features such as curvature, which in turn makes it difficult to detect physiological features. However, increasing the number of iterations reduces Z-sampling artifacts, but at the expense of preserving surface detail.

[0096] In some examples, the mesh is simplified n times, where n<=10, to slightly reduce artifacts while preserving surface detail.

[0097] Advantageously, tuning the number of simplification iterations to a diagnostic prediction metric (e.g. AUC-ROC) allows to provide an optimal signal-to-noise ratio while preserving some of the important physiological and / or pathological features present on the mesh, such as nodules, local deformations of the surface caused by disease, etc. Indeed, by smoothing the mesh once more with a Poisson reconstruction, those nodules can be reduced.

[0098] Step 137: Smoothing the simplified reconstructed mesh In some examples, the reconstructing step 130 may include a smoothing step 137 after a simplifying step 135 of the reconstructed mesh. The smoothing step 137 allows for detecting and removing artifacts. It is noted that the smoothing step 137 may be optional. The optional aspect of this aspect is depicted in FIG. 1 by the fact that the block is indicated with a dash.

[0099] In some examples, two smoothing methods that can preserve the simplified mesh topology have been proposed to attenuate high frequency components expected to be exclusively due to medical image acquisition artifacts: Laplacian smoothing or Taubin smoothing. Of these two methods, Taubin smoothing gives better results than Laplacian smoothing. This is because mesh shrinkage and mesh deformation occur after excessive iterations of Laplacian smoothing. In addition, excessive iterations may also reduce the AUC-ROC score. On the contrary, Taubin smoothing can preserve the global aspects of the mesh without the risk of mesh shrinkage. In the context of the present disclosure, either method may be adopted for the smoothing step 137.

[0100] In some other examples, anisotropic smoothing may be used, which has been tested and found to be more efficient in removing z-artifacts than Laplacian or Taubin smoothing. In the context of the present disclosure, anisotropic smoothing may be used to detect z-sampling artifacts and remove them by smoothing the mesh. More advantageously, anisotropic smoothing allows for the removal of z-sampling artifacts without erasing features.

[0101] In some further examples, the anisotropic smoothing may include other computational methods that allow for the detection and removal of artifacts.

[0102] In some further examples, the anisotropic smoothing may use Laplacian or Taubin smoothing, preferably Taubin smoothing, in combination with other computational methods to remove z-sampling artifacts.

[0103] In some examples, depending on the anatomical structures included in the segmentation mask, the smoothing method may be selected from among anisotropic and / or Laplacian smoothing, or anisotropic and / or Taubin smoothing.

[0104] If the segmentation mask contains liver fibrosis, anisotropic smoothing may be used since it removes Z-sampling artifacts and preserves physiological or pathological features. If the segmentation mask contains lung nodules, Laplacian or Taubin smoothing may be used since small size nodules may have artifacts in all directions rather than just in the Z direction.

[0105] It will be understood that in the context of the present disclosure, the "reconstructed mesh" may refer to the mesh after step 133 of extracting the mesh from the 3D segmentation mask, to the mesh after step 135 of simplifying the reconstructed mesh, i.e. the simplified reconstructed mesh, and to the mesh after step 137 of smoothing the simplified reconstructed mesh. If step 139 for closing holes is applied, the "reconstructed mesh" may also refer to the mesh after all steps 133, 135, 137 and 139.

[0106] In some examples, the "reconstructed meshes" refer to meshes that are prepared for the next step 140 of extracting morphological features from each reconstructed mesh as shown in Figure 1. Step 140 is described in detail below.

[0107] As mentioned above, in the context of this disclosure, "morphological features" refers to geometric or topological invariants. In some examples, the morphological features include the distribution of curvatures of the reconstructed mesh. As mentioned above, the curvature is a geometric or topological invariant that can be used to quantify local deformation. In some examples, the curvature is a discrete curvature of the reconstructed mesh. In those examples, the discrete curvature can be obtained by discretizing the continuous curvature. The advantage of using the discrete curvature compared to the continuous curvature is that computation time and resources can be significantly saved. In some further examples, the Gaussian curvature or the mean curvature is used as the discrete curvature.

[0108] In some examples, the step of extracting 140 morphological features may include the distribution of curvatures of the reconstructed mesh and the formula E(M)=∫ M |K G It may include a step 143 of calculating the energy of the associated mesh given by |dM, where E represents the energy, M represents the manifold mesh, and KG represents the Gaussian curvature. The energy is calculated as the sum of the absolute values ​​of the curvatures over the entire mesh.

[0109] In some further examples, the step 143 of calculating the curvature of the reconstructed mesh may include the following substeps. - substep 1431, which calculates the mean curvature or Gaussian curvature on the mesh; - a substep 1432 of computing a probability distribution of the mean curvature or Gaussian curvature over the mesh; and - A substep 1433 of binning the probability distribution into a distribution histogram.

[0110] Sub-step 1431 of computing the discrete mean or Gaussian curvature requires having a manifold mesh, since curvature is not defined for non-manifold vertices. In fact, a mesh is manifold if all edges in the mesh are either boundary edges or manifold edges. A manifold edge is part of exactly two faces for a surface. A boundary edge is part of exactly one face. A vertex is manifold if, starting from an adjacent face of the vertex, all other adjacent faces of the vertex can be accessed (by displacement of the faces). In some examples, an approximation of the Steiner formula is used to compute the mean or Gaussian curvature on a mesh.

[0111] In some examples, in sub-step 1433, the probability distribution of the mean curvature or Gaussian curvature is binned into range values, preferably of 10. It should be understood that in the context of the present disclosure, the bins are used to classify the curvature between negative and positive curvature.

[0112] Preferably, the number of bins may be set to 10, as it strikes a good balance between limiting the dimensionality of the analysis and retaining the curvature distribution information. Thus, 10 may be the number of bins used as an appropriate bin size to cover the entire range of the curvature data. In other examples, the number of bins may be arbitrarily larger or smaller than 10. The number of bins may be selected according to the sample size of the data, i.e., the larger the sample size, the larger the number of bins may be as a necessary effort to avoid the curse of dimensionality.

[0113] As explained before, the distribution of curvatures in the same organ with different pathologies (morphological deformations), such as a healthy liver and a cirrhotic liver, may appear different.

[0114] Figures 2A and 2B (right side) illustrate the output of the 3D mesh reconstruction, i.e., the reconstructed mesh of a healthy liver (F0) and an advanced fibrosis stage liver (F4), respectively. The curvature values ​​are color-coded, with blue representing local deformations with negative curvature, such as depressions and / or valleys, red representing local deformations with positive curvature, such as bumps and / or spikes, and green representing local deformations with curvature close to zero.

[0115] 2A and 2B (left side) illustrate the probability distribution of the mean curvature for a healthy liver and an advanced fibrosis stage liver.

[0116] In some further examples, the morphological features may further include baseline features, which may include at least one of the following features: - the volume of anatomical structures, - degree of elongation of anatomical structures, - the minimum and / or maximum size of the oriented bounding box of the anatomical structure, and - Mean HPI values ​​for anatomical structures.

[0117] In the above example, HPI stands for Hepatic Perfusion Index. It is defined as the ratio of hepatic artery blood flow to the sum of hepatic artery blood flow and portal vein blood flow. In the above example, HPI mean refers to the average value of HPI over the entire liver volume.

[0118] The advantages provided by using at least the curvature distribution as a morphological feature, and even by using both the curvature distribution and baseline features, are demonstrated in the following paragraphs based on an exemplary machine learning classifier and based on two exemplary applications, namely liver fibrosis stage prediction and lung nodule malignancy prediction, respectively.

[0119] 1, the method 100 further includes training 150 a machine learning model to perform the disease severity risk classification. In some examples, the training is performed based on the extracted morphological features.

[0120] In some examples, the machine learning model is a supervised classifier model. In some further examples, the machine learning model is a supervised decision tree-based classifier, such as a random forest or XGboost. The advantage of using a random forest or XGBoost classifier is the algorithm for selecting a function that separates the classes, which may be non-linear.

[0121] In some instances where small sample size datasets are used for training, the Random Forest classifier is preferred. In other instances where large sample size datasets are used for training, XGboost is preferred.

[0122] Although not shown in FIG. 1, the method 100 may further include providing clinical information about each patient. In some examples, the training step 150 is further performed on the patient's clinical information. In the context of the present disclosure, it should be understood that the phrase "patient's clinical information" refers to data about the patient obtained by clinical examination. This may include at least one of the following: age, sex, height, weight, BMI, ethnic origin, medical condition, and risk factors of each patient.

[0123] Alternatively or in addition to the clinical information, the method 100 of the present invention may further include extracting other radiomics features other than morphological features from the plurality of 3D medical images as provided, whereby training is further performed on the other radiomics features. The other radiomics features may include at least one of the latest library of radiomics provided by Van Griethuysen et al., including Haralick GLCM (gray-level co-occurrence matrix), GLRLM (gray-level run-length matrix), GLSZM (gray-level size zone matrix), NGTDM (neighborhood gray-tone difference matrix), GLDM (gray-level dependency matrix) features, standard statistics of image gray-level intensity features and classical morphological relationship features. EXAMPLES

[0124] Liver fibrosis stage prediction According to clinicians, a key task is to distinguish between early stages of fibrosis, from F0 to F2, and later stages (F3, F4). Moreover, these early and later stages of fibrosis show important differences in the level of nodular formation on the liver surface.

[0125] The dataset used for fibrosis stage characterization includes liver and spleen masks segmented from arterial phase CT scans using deep convolutional neural networks, and 112 patients with fibrosis stages F0 to F4 obtained by histology of biopsies. Within this dataset, 16% are F0, 4% are F1, 5% are F2, 12% are F3, and 63% are F4.

[0126] Despite these reasons, it is not appropriate to perform multi-class characterization of fibrosis stages, especially considering the small sample size and the disproportionate prevalence of fibrosis stages.Therefore, the prediction in Example 1 is a binary classification, pooling F0 to F2 livers (25%) on the one hand and F3, F4 livers (75%) on the other hand.The dataset is split into 50% for training and 50% for testing.

[0127] Figures 3A and 3B show the performance of the predictive model using random forest classification for F0-F2 and F3-F4 stages, respectively (shown in blue ROC curves), as well as the predictive power of each feature. In Figure 4B, baseline features 0-5 are shown above the solid black line, while liver surface features 6-16 (including curvature and mesh energy) are shown above the solid blue line. The predictive power of each feature can be calculated using the impurity-based feature importance method.

[0128] It is observed that when only baseline features are used, the performance of the Random Forest classifier is assessed by an AUC-ROC score of 0.88. In the examples selected to use baseline features, the Random Forest classifier has a minimum of 100 and a maximum of 300 trees. The depth is set to 3, 5, 15, or 25. In some examples, the number of features to consider when searching for the best split is in the range of 1 to 6. In further examples, the minimum number of samples required to split an internal node is 3, 5, or 10.

[0129] In some other examples, using morphological features combining baseline features with curvature distribution and mesh energy, the latter obtained on masks segmented using deep-CNN with the number of simplification iterations set to 10 and the number of Taubin smoothing iterations set to 5, improves the AUC-ROC score by 0.08, from 0.88 to 0.96. EXAMPLES

[0130] Pulmonary nodule malignancy prediction The dataset is from a small subset of NLST ("National Lung Screening Trial") cancer patients and healthy patients with nodules and contains 789 lung nodules that were annotated by clinicians and semi-automatically and manually segmented, of which 79% were benign and 21% were malignant.

[0131] As previously explained, for the diagnosis of pulmonary nodules, it is well known that local anomalies such as spiculations, which are spikes on the surface of the nodule, have significant predictive power for the malignancy of the nodule, whereas "roundness" and smooth borders usually indicate that the lesion is benign.

[0132] Advantageously, curvature is used to quantify these local deformations that allow distinguishing between regular and spiky surfaces. The method of the present invention replaces the usual qualitative and subjective clinician judgement with morphological features such as curvature.

[0133] 4A and 4B (right side) illustrate the output of the 3D mesh reconstruction, i.e., the reconstructed meshes of benign and malignant pulmonary nodules, respectively. The curvature values ​​are color coded, with blue representing local deformations with negative curvature, such as depressions and / or valleys, red representing local deformations with positive curvature, such as bumps and / or spikes, and green representing local deformations with curvature close to zero.

[0134] 4A and 4B (left side) illustrate the probability distributions of the mean curvatures of benign and malignant pulmonary nodules, respectively.

[0135] As illustrated in Figures 4A and 4B, malignant and benign lesions typically exhibit different curvature distributions, with an over-representation of high absolute values ​​of curvature in malignant nodules. In this example, the mesh is reconstructed without using Poisson reconstruction.

[0136] Similar to Figures 2A and 2B, Figures 4A and 4B (right side) display the mesh of all organs including their various probabilities of mean curvature or Gaussian curvature or derived morphological features color-coded on each surface, their probability distributions, and the patient's clinical information.

[0137] Figure 5 shows the performance of malignancy prediction using only curvature features. In Figure 6, an AUCROC score of 0.93 is achieved. In the example of lung malignancy prediction (benign: 0, malignant: 1 and vice versa), only the curvature distribution of lung nodules and the energy of the mesh are used as features for malignancy characterization with the number of smoothing iterations set to 0, and classification into benign or malignant nodules by random forest gives an AUCROC of 0.93.

[0138] 6 shows a flowchart according to some examples of the method 200 of the second aspect of the present disclosure, i.e., a method for predicting disease severity risk from a plurality of three-dimensional medical images by applying a machine learning model. The machine learning model may have been generated by implementing the method 100 according to any embodiment of the first aspect of the present disclosure.

[0139] It should be noted that in the context of the present disclosure, the descriptions given above for embodiments of method 100 should also be applicable to embodiments of method 200. For example, terms such as “organ”, “disease”, “multiple”, “3D medical images”, “anatomical structure”, “segmentation mask”, “reconstruction”, “morphological features”, “curvature”, “mean curvature”, “Gaussian curvature”, “baseline features”, “patient clinical information”, etc. have the same meaning in method 200 as in method 100.

[0140] 6, the method 200 includes receiving 205 a plurality of three-dimensional medical images, each image of the plurality of three-dimensional medical images including at least one anatomical structure of a patient. The method 200 further includes providing 210 one of the plurality of medical images as a candidate image for prediction.

[0141] 6, the method 200 further includes obtaining 220 at least one candidate 3D segmentation mask from the candidate image. In some examples, each mask of the at least one candidate 3D segmentation mask may include at least one anatomical structure.

[0142] Step 220 of obtaining at least one candidate 3D segmentation mask from each candidate image may include step 225 of segmenting each candidate image such that at least one candidate 3D segmentation mask is generated for said candidate image, each mask including at least one anatomical structure.

[0143] It will be appreciated that when an automatic segmentation method is deployed in the method of the present invention, this may advantageously result in an automatic prediction of disease severity risk.

[0144] As shown in Fig. 6, the method 200 may further comprise a step 230 of reconstructing a candidate mesh from each of said at least one candidate 3D segmentation mask. Step 230 may be adapted to step 130 detailed above. That is, step 230 of reconstructing a candidate mesh from each of the at least one candidate 3D segmentation mask may comprise a step 233 of extracting a candidate mesh from each candidate 3D segmentation mask, then a step 235 of simplifying each reconstructed candidate mesh, followed by a step 237 of smoothing the simplified reconstructed candidate mesh. Step 230 of reconstructing a candidate mesh from each candidate 3D segmentation mask may further comprise a step 239 of light smoothing to close holes on the simplified mesh, after reconstruction 230 and before extraction 240. Each of steps 233, 235, 237 and 239 may be adapted to each of steps 133, 135, 137 and 139, respectively, detailed above.

[0145] Still referring to Figure 6, the method 200 further comprises a step 240 of extracting morphological features from each candidate reconstructed mesh. Step 240 may be adapted to step 140 detailed above.

[0146] In some examples, the step of extracting 240 morphological features may include extracting the curvature distribution of the reconstructed candidate mesh and the associated energy of the candidate mesh E(M)=∫ M |K G This may include a step 243 of calculating |dM.

[0147] In some further examples, the step 243 of calculating the curvature of the reconstructed candidate mesh may include the following sub-steps. - substep 2431, which calculates the mean curvature or Gaussian curvature on the mesh; - a substep 2432 of computing a probability distribution of the mean curvature or Gaussian curvature over the mesh; and - A substep 2433 of binning said probability distribution into a distribution histogram.

[0148] It should be understood that each of the sub-steps 2431, 2432 and 2433 may be adapted to each of the sub-steps 1431, 1432 and 1433, respectively, as detailed above.

[0149] Still referring to FIG. 6, it can be seen that the method 200 may further include a step 250 of inputting the extracted morphological features into a machine learning model to perform disease severity risk classification.

[0150] Thus, prediction of disease severity risk from multiple 3D medical images is achieved.

[0151] The embodiments and aspects of the disclosure detailed herein may be described and illustrated in conjunction with systems, devices, and methods that are intended to be exemplary and explanatory, but not limiting in scope. Specific configurations and details may be described to aid in the understanding of the disclosure. However, it will be apparent to those skilled in the art that the disclosure may be practiced without some of the specific details presented herein. Furthermore, some well-known steps or components may be described only broadly or even omitted for illustrative clarity.

[0152] Some processes may be shown and described in a sequence of steps, it being understood that this sequence is exemplary and that the steps may be performed in an order different from that presented, some steps described may be omitted, and some additional steps may be omitted from the sequence or described elsewhere.

[0153] The disclosures of prior patents, publications, and applications may be referenced. Some text and figures from those sources may be presented herein, but may be modified, edited, or commented upon to more smoothly integrate with the disclosure of this application. Citation or identification of any document should not be construed as an admission that such document is available as prior art to the present disclosure.

[0154] The methods described herein may be implemented on one or more computers in any form. The components may be implemented as dedicated applications or in a client-server architecture, including a web-based architecture, and may include functional programs, codes, and code segments. Any computer may include a processor, a memory for storing and executing program data, a permanent storage device such as a disk drive, a communication port for handling communication with external devices, and a user interface device including a display, a keyboard, a mouse, and the like. When any software or algorithms are involved, they may be stored as program instructions or computer-readable code executable on a processor on a computer-readable medium such as a read-only memory (ROM), a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device. The computer-readable recording medium may be distributed across a network-coupled computer system such that the computer-readable code is stored and executed in a distributed manner. The medium may be read by the computer, stored in the memory, and executed by the processor.

[0155] For the purposes of promoting an understanding of the principles of the various embodiments of the present disclosure, reference has been made to a preferred embodiment illustrated in the drawings, and specific language has been used to describe the same. However, no limitation of the scope of the disclosure by this specific language is intended, and the disclosure should be construed to encompass all embodiments that would normally occur to one skilled in the art.

[0156] The present disclosure may be described in terms of functional block components and various processing steps. Such functional blocks may be realized by any number of hardware and / or software components configured to perform the specified functions. For example, the present disclosure may employ various integrated circuit components, such as memory elements, processing elements, logic elements, look-up tables, and the like, which may perform various functions under the control of one or more microprocessors or other control devices. Similarly, where elements of the present disclosure are implemented using software programming or software elements, the present disclosure may be implemented using any programming or scripting language, such as C, C++, Java, Python, Assembler, or the like, and various algorithms are implemented using any combination of data structures, objects, processes, routines, or other programming elements. Functional aspects may be implemented in algorithms executed on one or more processors. Additionally, the present disclosure could employ any number of conventional techniques for electronics configuration, signal processing and / or control, data processing, and the like. [Explanation of symbols]

[0157] 100 ways 200 ways

Claims

1. 1. A method (100) for generating a machine learning model for predicting risk of disease severity from a plurality of three-dimensional medical images, comprising: - providing (110) a plurality of three-dimensional medical images, each image of the plurality of three-dimensional medical images including at least one anatomical structure of a patient; - obtaining (120) at least one 3D segmentation mask from each image of said plurality of 3D medical images, each mask of said at least one 3D segmentation mask including at least one anatomical structure; - reconstructing (130) a mesh from each of the three-dimensional segmentation masks; - extracting (140) morphological features from each reconstructed mesh; - training (150) a machine learning model for performing said classification of said severity risk of a disease, said training being performed on the basis of said extracted morphological features; and A method (100).

2. 2. The method (100) of claim 1, wherein the step of reconstructing (130) a mesh from each 3D segmentation mask comprises a step of extracting (133) a mesh from the 3D segmentation mask by using Marching Cubes, preferably by using Ruiner Marching Cubes.

3. The method (100) of claim 2, wherein the step of reconstructing (130) a mesh from each 3D segmentation mask further comprises the step of simplifying (135) each reconstructed mesh to remove inconsistencies.

4. 4. The method (100) of claim 3, wherein the step of reconstructing (130) a mesh from each of the at least one three-dimensional segmentation mask further comprises a step of simplifying (135) each reconstructed mesh, followed by smoothing (137) to detect and remove artifacts.

5. The smoothing step (137) is based on the anatomical structures contained in the segmentation mask, and preferably comprises: - Anisotropic and / or Laplacian smoothing, or - Anisotropic and / or Taubin smoothing The method (100) of claim 4, wherein the determination is made from among:

6. The method (100) according to any one of claims 1 to 5, wherein morphological features comprise at least a distribution of curvatures of the reconstructed mesh, preferably a distribution of discrete curvatures of the reconstructed mesh.

7. The method (100) of claim 6, wherein the step of extracting (140) morphological features comprises the step of calculating (143) the distribution of curvature of the reconstructed mesh and an associated mesh energy.

8. The step of calculating (143) the distribution of curvatures of the reconstructed mesh comprises: - calculating (1431) the mean curvature or Gaussian curvature on said mesh, preferably by using an approximation of the Steiner rule; - calculating (1432) a probability distribution of said mean or Gaussian curvature over said mesh; - binning (1433) said probability distribution into a distribution histogram; 8. The method (100) of claim 7, comprising:

9. The morphological features further include baseline features, the baseline features being: - a volume of said at least one anatomical structure; - the elongation of said at least one anatomical structure; - a minimum and / or maximum size of an oriented bounding box of said at least one anatomical structure, and - the average HPI of said at least one anatomical structure; The method (100) according to any one of claims 6 to 8, comprising at least one of the following:

10. The method (100) according to any one of claims 1 to 9, wherein the machine learning model that is trained (150) is a supervised classifier model, preferably a decision tree based classifier.

11. - providing clinical information about each patient; The step of training (150) may include The method (100) according to any one of claims 1 to 10, further comprising:

12. The method (100) according to any one of claims 1 to 11, wherein the three-dimensional medical images comprise computed tomography (CT) images and / or magnetic resonance imaging (MRI) images.

13. 1. A method (200) for predicting said risk of disease severity from a plurality of three-dimensional medical images by applying a machine learning model, comprising: - receiving (205) a plurality of three-dimensional medical images, each image of the plurality of three-dimensional medical images including at least one anatomical structure of a patient; - providing (210) one of said plurality of medical images as a candidate image for prediction; - obtaining (220) at least one candidate 3D segmentation mask from said candidate image, each mask of said at least one candidate 3D segmentation mask including at least one anatomical structure; - reconstructing (230) a candidate mesh from each of said at least one candidate 3D segmentation mask; - extracting (240) morphological features from each reconstructed candidate mesh; - inputting (250) the extracted morphological features into the machine learning model to perform a classification of the severity risk of a disease; A method (200).

14. 14. The method (200) of claim 13, wherein the machine learning model is generated by implementing a method (100) for generating a machine learning model according to any one of claims 1 to 12.

15. 15. The method (200) according to claim 13 or 14, wherein the step of reconstructing (230) a candidate mesh from each of the at least one candidate 3D segmentation mask comprises a step of extracting (233) a mesh from the 3D segmentation mask by using Marching Cubes, preferably by using Ruiner Marching Cubes.

16. 16. The method (200) of claim 15, wherein the step of reconstructing (230) a candidate mesh from each of the at least one candidate 3D segmentation mask further comprises the step of simplifying (235) each reconstructed candidate mesh to remove inconsistencies.

17. 17. The method (200) of claim 16, wherein the step of reconstructing (230) a candidate mesh from each of the at least one candidate 3D segmentation mask further comprises a step of simplifying (235) each reconstructed candidate mesh, followed by smoothing (237) to detect and remove artifacts.

18. The step of smoothing (237) is based on the candidate anatomical structures contained in the candidate segmentation mask, preferably comprising: - Anisotropic and / or Laplacian smoothing, or - Anisotropic and / or Taubin smoothing The method (200) of claim 17, wherein the determination is made from among:

19. The method (200) according to any one of claims 13 to 18, wherein morphological features include at least a distribution of curvatures of the reconstructed candidate mesh, preferably a distribution of discrete curvatures of the reconstructed candidate mesh.

20. The method (200) of claim 19, wherein the step of extracting (140) morphological features comprises the step of calculating (143) the distribution of curvature of the reconstructed candidate mesh and the associated energy of the candidate mesh.

21. The step of calculating (143) the distribution of curvatures of the reconstructed candidate meshes comprises: - calculating (2431) the mean curvature or Gaussian curvature on said mesh, preferably by using an approximation of the Steiner rule; - calculating (2432) a probability distribution of the mean or Gaussian curvature over the mesh; - binning (2433) said probability distribution into a distribution histogram; 21. The method (200) of claim 20, comprising:

22. The morphological features further include baseline features, the baseline features being: - a volume of said at least one anatomical structure; - the elongation of said at least one anatomical structure; - a minimum and / or maximum size of an oriented bounding box of said at least one anatomical structure, and - the average HPI of said at least one anatomical structure; 22. The method (200) of any one of claims 19 to 21, comprising at least one of the following:

23. The method (200) according to any one of claims 13 to 22, wherein the three-dimensional medical images comprise computed tomography (CT) and / or magnetic resonance imaging (MRI) images.

24. A computer program product comprising instructions for performing at least some of the steps of the method (100) according to claims 1 to 12 and the method (200) according to claims 13 to 23 when executed by at least one digital processing device.

Images