Method and system for computer-aided diagnosis based on morphological characteristics extracted from three-dimensional medical images

By extracting morphological features and baseline features from three-dimensional medical images and training machine learning models, the problem of difficulty in quantifying morphological features in the prior art is solved, and the accuracy of prediction of disease severity is significantly improved.

CN119072713APending Publication Date: 2024-12-03MEDIAN TECH
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202380028642.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2022-02-18
Filing Date
2023-02-09
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

Existing medical image diagnosis methods are difficult to accurately quantify morphological characteristics, resulting in low accuracy in disease prediction, especially in the local deformation of organs or sub-organisms, which cannot effectively utilize three-dimensional morphological information.

Method used

By extracting morphological features from three-dimensional medical images, including reconstructing the curvature distribution and associated energy of the mesh, combined with baseline features, machine learning models are trained to predict the risk of disease severity.

Benefits of technology

Improve the prediction accuracy of disease severity, for example, the AUC-ROC score significantly improves the performance of diagnosis of liver fibrosis and pulmonary nodules, achieving higher classification performance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119072713A_ABST
    Figure CN119072713A_ABST
Patent Text Reader

Abstract

A method and system for generating a machine learning model for predicting a severity risk of a disease from a plurality of three-dimensional medical images is provided. The method comprises the steps of: providing a plurality of three-dimensional medical images, each image of the plurality of three-dimensional medical images comprising at least one anatomical structure of the patient; obtaining at least one three-dimensional segmentation mask from each of the plurality of three-dimensional medical images, each mask of the at least one three-dimensional segmentation mask comprising at least one anatomical structure; reconstructing a grid according to each mask of the at least one three-dimensional segmentation mask; extracting a morphological feature from each reconstructed grid; and training the machine learning model to classify the severity risk of the disease, the training being performed for the extracted morphological features. A method and system for predicting the severity risk of a disease from a plurality of three-dimensional medical images by applying a machine learning model generated for the same purpose is also provided.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure generally relates to methods and systems for pathology diagnosis based on morphological properties computed from medical images.

[0002] The present disclosure more particularly relates to machine learning models for predicting disease severity based on morphological characteristics. Background Art

[0003] Since the beginning of medicine, morphology or morphological properties have provided symptom phenotypes that enrich the information available to physicians to establish clinical diagnoses. Morphology involves the study of the form and structure of an organism and its specific structural features, especially two-dimensional or three-dimensional anatomical structures (such as organs or subtissues).

[0004] According to the general principle of the correspondence between structure and function in biology, dysfunction of biological processes is always accompanied by pathological deformation of basic biological structures.

[0005] In order to facilitate clinical diagnosis by using morphology, U.S. Patent No. 10130295B2 entitled "Methods for the detection and staging of liver fibrosis from image acquired data" proposes to use a quantitative method to measure the liver surface nodularity and calculate the liver surface nodularity score. A morphological feature is a function, process or observation that calculates a scalar from the approximate geometry or topology of a two-dimensional image or a three-dimensional volume. Unfortunately, morphological features are often difficult to quantify.

[0006] Some publications involve characterizing the deformation of organs or subtissues by objectively quantifying morphological properties, such as Choi W. et al., Reproducible and interpretable spiculation quantification for lung cancer screening, Computer Methods and Programs in Biomedicine. vol. 200 (2021); and Chinese Patent No. 113066583A entitled “Aneurysm rupture risk prediction method and device and storage medium”.

[0007] However, such quantization is limited to coarse and crude geometric quantifiers, such as functions of distance, area or volume, which usually fail to take into account the full information of the morphology.

[0008] Furthermore, systems that use such quantifiers to perform diagnostic prediction will not be able to achieve high prediction accuracy. Meanwhile, local deformation of organs or subtissues is usually a good indicator of pathological geometry and topology, since diseases often lead to anisotropy or local expansion.

[0009] Therefore, a desirable approach is to appropriately utilize morphological characteristics for accurate disease prediction, in particular to optimize the performance for prediction of disease severity, as assessed, for example, by AUC-ROC score. Summary of the invention

[0010] According to a first aspect, a method for generating a machine learning model for predicting the risk of severity of a disease based on a plurality of three-dimensional medical images is disclosed, the method comprising the following steps:

[0011] - providing a plurality of three-dimensional medical images, each of the plurality of three-dimensional medical images comprising at least one anatomical structure of a patient;

[0012] - obtaining at least one three-dimensional segmentation mask from each of a plurality of three-dimensional medical images, each mask of the at least one three-dimensional segmentation mask comprising at least one anatomical structure;

[0013] - Reconstructing a mesh based on each 3D segmentation mask;

[0014] - extracting morphological features from each reconstructed mesh; and

[0015] - Train a machine learning model to perform classification of disease severity risk, where:

[0016] The training is performed on the extracted morphological features.

[0017] In some examples, reconstructing a mesh according to each 3D segmentation mask comprises extracting a mesh from the 3D segmentation mask by using a Marching cube, and preferably, reconstructing a mesh according to each 3D segmentation mask comprises extracting a mesh from the 3D segmentation mask by using a Lewiner marching cube. In addition, reconstructing a mesh according to each 3D segmentation mask comprises a step of simplifying each reconstructed mesh to eliminate inconsistencies. In addition, reconstructing a mesh according to each mask of the at least one 3D segmentation mask comprises: after simplifying each reconstructed mesh, performing a smoothing step to detect artifacts and remove artifacts. The smoothing step may be determined according to the anatomical structure included in the segmentation mask, and preferably, the smoothing step is one of the following:

[0018] - anisotropic smoothing and / or Laplacian smoothing; or

[0019] - Anisotropic smoothing and / or Taubin smoothing.

[0020] In some examples, the morphological features may include at least a curvature distribution of the reconstructed mesh, and preferably, the morphological features may include a discrete curvature distribution of the reconstructed mesh. In addition, extracting the morphological features may include calculating the curvature distribution of the reconstructed mesh and the energy of the associated mesh. More specifically, calculating the curvature distribution of the reconstructed mesh may include the following steps:

[0021] - calculating the mean curvature or Gaussian curvature about the mesh, preferably by using an approximation of the Steiner formula,

[0022] - computing a probability distribution over the mean or Gaussian curvature of the grid, and

[0023] - Binning the probability distribution into a distribution histogram.

[0024] In addition to the curvature distribution, the morphological features may also include baseline features. The baseline features may include at least one of the following:

[0025] - the volume of the at least one anatomical structure;

[0026] - elongation of the at least one anatomical structure;

[0027] - a minimum size and / or a maximum size of the oriented bounding box of the at least one anatomical structure; and

[0028] - The HPI mean value of said at least one anatomical structure.

[0029] In some examples, the trained machine learning model is a supervised classifier model, and preferably, the trained machine learning model is a decision tree based classifier.

[0030] In some examples, the method according to the first aspect of the present invention may further include:

[0031] - provide clinical information about each patient, and

[0032] The training is also performed on the patient's clinical information.

[0033] In some examples, the method according to the first aspect of the present invention may further include:

[0034] - extracting other radiomics features other than morphological features from the plurality of three-dimensional medical images, and wherein the training is also performed on said other radiomics features.

[0035] The other imaging omics features may include at least one of the following: the most advanced imaging omics library provided by Van Griethuysen et al., including Haralick GLCM (gray level co-occurrence matrix), GLRLM (gray level run length matrix), GLSZM (gray level size zone matrix), NGTDM (neighboring gray level difference matrix), GLDM (gray level dependence matrix) features, standard statistics of image gray level intensity features, and typical morphology related features.

[0036] In some examples, the training is performed on morphological features and on at least one type of features among patient clinical information and other radiomics features.

[0037] According to a second aspect, a method for predicting the risk of severity of a disease based on a plurality of three-dimensional medical images by applying a machine learning model is disclosed, the method comprising the following steps:

[0038] - receiving a plurality of three-dimensional medical images, each of the plurality of three-dimensional medical images comprising at least one anatomical structure of a patient;

[0039] - providing one of the plurality of medical images as a candidate image for prediction;

[0040] - obtaining at least one candidate three-dimensional segmentation mask from the candidate image, each mask of the at least one candidate three-dimensional segmentation mask comprising at least one anatomical structure;

[0041] - reconstructing a candidate mesh based on each mask of the at least one candidate 3D segmentation mask;

[0042] - extracting morphological features from each reconstructed candidate mesh;

[0043] -The extracted morphological features are fed into a machine learning model to perform classification on the severity risk of the disease.

[0044] In some examples, the machine learning model has been generated by implementing the method for generating a machine learning model according to the first aspect of the present invention.

[0045] In some examples, reconstructing the candidate mesh according to each mask of the at least one candidate three-dimensional segmentation mask includes extracting a mesh from the three-dimensional segmentation mask by using marching cubes, and preferably, reconstructing the candidate mesh according to each mask of the at least one candidate three-dimensional segmentation mask includes extracting a mesh from the three-dimensional segmentation mask by using Lewinner marching cubes. In addition, reconstructing the candidate mesh according to each mask of the at least one candidate three-dimensional segmentation mask may also include the following steps: simplifying each reconstructed candidate mesh to eliminate inconsistencies. In addition, reconstructing the candidate mesh according to each mask of the at least one candidate three-dimensional segmentation mask also includes: after simplifying each reconstructed candidate mesh, performing a smoothing step to detect artifacts and remove artifacts. The smoothing step can be determined based on the candidate anatomical structure included in the candidate segmentation mask, and preferably the smoothing step is one of the following:

[0046] - anisotropic smoothing and / or Laplacian smoothing; or

[0047] - Anisotropic smoothing and / or Taubin smoothing.

[0048] In some examples, the morphological feature may include at least a curvature distribution of the reconstructed candidate mesh, and preferably, the morphological feature may include a discrete curvature distribution of the reconstructed candidate mesh. In addition, extracting the morphological feature may include calculating the curvature distribution of the reconstructed candidate mesh and the energy of the associated candidate mesh. More specifically, calculating the curvature distribution of the reconstructed candidate mesh may include the following steps:

[0049] - calculating the mean curvature or Gaussian curvature about the mesh, preferably by using an approximation of the Steiner formula,

[0050] - computing a probability distribution over the mean or Gaussian curvature of the grid, and

[0051] - Binning the probability distribution into a distribution histogram.

[0052] In addition to the curvature distribution, the morphological features may also include baseline features. The baseline features may include at least one of the following:

[0053] - the volume of the at least one anatomical structure;

[0054] - elongation of the at least one anatomical structure;

[0055] - a minimum size and / or a maximum size of the oriented bounding box of the at least one anatomical structure; and

[0056] - The HPI mean value of said at least one anatomical structure.

[0057] According to the first aspect or the second aspect of the present invention, the three-dimensional medical image may include a computed tomography (CT) image and / or a magnetic resonance imaging (MRI) image.

[0058] Another aspect of the present disclosure relates to a non-transitory computer-readable medium storing instructions which, when executed by at least one digital processing device, perform at least the steps of the method according to the first aspect of the present invention and / or the steps of the method according to the second aspect of the present invention.

[0059] Another aspect of the present disclosure relates to a computer program product comprising instructions which, when executed by at least one digital processing device, perform at least the steps of the method according to the first aspect of the present invention and / or the steps of the method according to the second aspect of the present invention.

[0060] Other objects, features and advantages of the present invention disclosed herein and its various embodiments / aspects may become apparent from the following description of some exemplary embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Embodiments of the present disclosure will be described in detail, and non-limiting examples of the embodiments may be shown in the accompanying drawings. Some of the drawings may be in the form of schematic diagrams. For clarity of explanation, some elements in the drawings may be exaggerated and other elements may be omitted.

[0062] Any text (legends, notes, reference numerals, etc.) appearing on the accompanying drawings is incorporated herein by reference.

[0063] Figure 1 are flowcharts of some examples of methods according to the first aspect of the present disclosure;

[0064] Figure 2A and Figure 2B (On the right) Reconstructed meshes of a healthy liver (F0) and an advanced fibrotic liver (F4) are shown, respectively, where the curvature value at each vertex is color-coded; Figure 2A and Figure 2B (On the left) Probability distributions of mean curvatures for healthy liver and advanced fibrosis liver are shown, respectively;

[0065] Figure 3A and Figure 3B The performance of the prediction model for the F0-F2 stage versus the F3-F4 stage using Random Forest classification is shown, as well as the predictive power of each feature;

[0066] Figure 4A and Figure 4B (On the right) Reconstructed meshes of benign and malignant pulmonary nodules are shown, respectively, where the curvature value at each vertex is color-coded. Figure 4A and Figure 4B (On the left) Probability distributions of mean curvature for benign and malignant pulmonary nodules, respectively;

[0067] Figure 5 The performance of malignancy prediction using only curvature features is shown;

[0068] Figure 6 are flow charts of some examples of methods according to the second aspect of the present disclosure. DETAILED DESCRIPTION

[0069] Pathology systematically induces morphological changes, constituting a major but under-quantified source of observations for diagnosis. To exploit morphological properties for disease prediction, quantitative methods have been proposed to measure pathology, especially pathological deformation of organs or subtissues in medical images.

[0070] Pathology can refer to pathological features that are collectively considered, i.e., typical observable symptoms of a disease. In the context of the present disclosure, pathological deformation of an organ or subtissue can be, for example, liver surface nodularity, spiculated lung nodules, and can be extended to other types of deformations, such as aneurysm rupture, exophytic renal tumors, and other examples.

[0071] As mentioned above, U.S. Patent No. 10130295B2 proposes image segmentation of the liver edge boundary (one-dimensional) on a two-dimensional image or a three-dimensional volume, and then fitting a one-dimensional spline curve (polynomial curve) to the detected liver boundary. The distance between the detected liver boundary and the spline is measured on a pixel-by-pixel basis and is called the liver surface nodule score (LNS). A larger distance corresponds to a larger surface nodule score - in this way, (via LNS) the presence of surface nodules can be quantified. However, given that the spline curve is fitted on a two-dimensional image, it is not possible to utilize all three-dimensional neighborhood information and morphology to extract morphological features. Moreover, making the LNS score directly related to the diagnosis of liver fibrosis or cirrhosis may not meet the performance expectations for disease prediction.

[0072] In the diagnosis of lung tumor malignancy, spinous processes are important predictors of lung cancer malignancy. Spinous processes are sharp objects on the surface, while benign nodules are egg-shaped. Choi W.et al., Reproducible and interpretable spiculation quantification for lung cancer screening. Computer Methods and Programs in Biomedicine. vol. 200 (2021) proposed to quantify spinous processes by reconstructing a three-dimensional mesh from the segmentation mask of the nodule and conformally mapping it to a sphere. Then, after measuring the area distortion of the conformal mapping, the maximum negative change in the area distortion is used as a quantifier to identify spikes. A machine learning model (SVM-LASSO) is then used to predict the malignancy of the nodule. However, the quantifiers used by Choi et al. that rely on a single scalar value to characterize local deformations and their method of predicting the malignancy of lung nodules based on such quantifiers need to be improved.

[0073] Chinese Patent No. 113066583A proposes to predict aneurysm risk by the following steps: reconstructing a three-dimensional mesh based on the segmentation mask of the aneurysm, and extracting morphological features from the mesh, and then using these features as input data for a machine learning model for prediction. The morphological features are various distance ratios, such as the distance ratio between the maximum height value and the average size of the neck, the distance ratio between the maximum height value and the maximum width value, the distance ratio between the maximum width and the average neck size, and the distance ratio between the maximum height and the parent blood vessel size. However, the distance ratio can only quantify all possible local deformations or shape irregularities in a rough, one-sided and approximate manner, thus resulting in low-accuracy predictions.

[0074] Leonardi V. et al.'s study: Multiple reconstruction and dynamic modeling of 3D digital objects using a morphing approach, The Visual Computer. Vol. 31 (June 2014) aims to detect cancer in the kidney and proposes to use a curvature-based method (by recursive labeling of maximum curvature or vertices) on the surface mesh to locate the renal exophytic tumor. However, the curvature is only extracted for visualization purposes, because the purpose of Leonardi V. et al.'s study is to apply 3D meshes to kidney animation and tumor tracking.

[0075] Furthermore, the few studies that have introduced 3D methods using organ surface meshes have proposed either extracting a single conformal mapping deformation parameter, resulting in a single and potentially poorly discriminative feature (Choi) or recursively curvature labeling of vertices, resulting in high-dimensional features that are therefore not suitable for prediction using machine learning models (Leonardi).

[0076] The present disclosure proposes to characterize local pathological deformations as quantifiers and then make accurate predictions of diseases based on these quantifiers, thereby allowing the performance of prediction of disease severity to be optimized, such as evaluated by AUC-ROC score.

[0077] AUC-ROC is one of the most important evaluation metrics used to measure classification performance. The values ​​of AUC-ROC score range from 0 to 1. The higher the AUC-ROC, the better the model distinguishes between the two classes. For multi-class classification, using the "one vs. all" approach, we can plot as many ROC curves as the number of classes. For example, for the characterization of liver fibrosis stages F0, F1, F2, F3, F4, we can plot a ROC curve for F0 vs. (F1-F4), a ROC curve for F1 vs. (F0, F2-F4), a ROC curve for F2 vs. (F0-F1, F3-F4), a ROC curve for F3 vs. (F0-F2, F4), and a ROC curve for F4 vs. (F0-F3). In this case, the AUC-ROC score can be calculated by a one-vs-rest (OvR) algorithm, which calculates the average of the ROC-AUC scores for each class relative to all other classes.

[0078] The present disclosure proposed herein aims to utilize three-dimensional morphological properties by extracting specific morphological features that quantify local deformation and developing a method for morphological classification based on the extracted features.

[0079] In geometry, curvature is the degree to which a curve deviates from a straight line or a curved surface deviates from a plane. There are positive and negative curvatures. If a surface curves away from a point in the same direction relative to a tangent to the surface, regardless of the cutting plane, then the surface has positive curvature at that point. If a surface curves away from a tangent plane in two different directions, then the surface has negative curvature at a point.

[0080] The Gauss-Bonnet theorem states that the Gaussian curvature of a surface is an intrinsic property of the surface, and that it does not change if the surface is bent without cutting it or adding holes to it, and in this sense, the Gaussian curvature of a surface is a topological invariant. For example, if we deform a sphere into a liver, and if we compute the integral over the curvature, the integral over the curvature of the sphere is the same as the integral over the curvature of its deformed version, the liver. In this case, the liver has a surface with no boundaries and no topological holes, and is isomorphic to the sphere (i.e., a closed surface with no topological holes), and the theorem states that the integral over the curvature is equal to 2 times pi times the Euler characteristic.

[0081] Advantageously, the present method proposes to extract not only the sum of all local curvatures but also their distribution over the liver surface, so that the extracted features convey information about all deformations (except torsion) rather than being invariant to them.

[0082] In the same example, although the integral of the curvature of the sphere and the liver is the same, the curvature distribution of the sphere and the liver is different due to the "conversion" of the sphere into a local deformation of the liver. In this example, the liver with bumps and saddles on the surface, i.e., local deformations, may have several negative and positive curvatures, respectively, while the sphere will have zero negative curvature. According to the same principle, the curvature distribution of a healthy liver and a cirrhotic liver should be different. The energy given by the integral of the absolute value of the curvature of these two shapes will also be different.

[0083] Advantageously, in the context of the present disclosure, these differences can be used to facilitate differentiation of anatomical structures having different pathological deformations.

[0084] Therefore, it is advantageous to use the curvature distribution to characterize the local deformation. Using the curvature distribution as a morphological feature is particularly advantageous for predicting disease, because quantifying the local pathological deformation using the curvature distribution allows distinguishing regular surfaces from surfaces with irregular shapes, such as the wavy shapes of liver fibrosis and the spikes present in lung nodules.

[0085] In order to utilize the 3D morphological characteristics obtained from the 3D medical images, the present disclosure proposes segmenting each 3D medical image to obtain at least one 3D segmentation mask, and reconstructing a mesh according to each 3D segmentation mask. Each 3D segmentation mask includes at least one anatomical structure.

[0086] The term "segmentation" refers to the process of partitioning at least one three-dimensional medical image into a plurality of anatomical regions (sets of voxels, also referred to as image objects). In the context of the present disclosure, segmentation refers to generating a segmentation mask comprising at least one anatomical structure for each three-dimensional medical image.

[0087] An "anatomical structure" is any biological entity that occupies space and is distinguishable from its surroundings. In the context of the present invention, an anatomical structure may be, for example, an organ, muscle, soft tissue, artery, vein, bone, subtissue, etc. Examples of organs may be the liver, spleen, lungs, etc. An example of a subtissue may be a tumor, which is an abnormal growth of body tissue, such as a lung nodule. A tumor may be cancerous (malignant) or non-cancerous (benign). An additional example of a subtissue may be an aneurysm, which is a bulge formed in the thin wall of an artery, such as a cerebral aneurysm and a thoracic aortic aneurysm. It should be noted that the present invention does not limit anatomical structures to the examples listed.

[0088] It is understood that, in the context of the present invention, the curvature extracted from the reconstructed mesh refers to the degree to which the curve deviates from a straight line and is expressed as a digital quantity. For example, curvature, i.e., Gaussian curvature or the average curvature of a point (vertex) is a scalar value.

[0089] In other words, by reconstructing a mesh of the three-dimensional segmentation boundaries of an organ or subtissue, and then extracting morphological features from the reconstructed mesh, the morphological features are calculated based on the geometric mesh and correspond to some topological invariants such as Gaussian curvature (in the sense of the Bonnet-Gauss theorem or isometric difference invariance), or to the mean curvature, or to other functions to distinguish pits from peaks, convexities from concavities, ridges from valleys, thereby allowing maximum utilization of the three-dimensional morphological information obtained from three-dimensional medical images to characterize local pathological deformations.

[0090] The present disclosure proposes to use curvature to characterize the local deformation of organ or subtissue boundaries, and then calculate the distribution of these curvatures, which can be used to generate machine learning models to statistically quantify the severity of diseases with high accuracy. Two examples of applying the proposed solution are provided for clinical prediction, namely, liver fibrosis stage and lung nodule malignancy.

[0091] In addition, the present disclosure proposes training a machine learning model based on the extracted morphological features to perform classification of disease severity risk based on multiple three-dimensional medical images.

[0092] Advantageously, the machine learning model is a supervised classifier model, preferably a decision tree based classifier. In some examples, a random forest or XGboost model may be used.

[0093] In the context of the present disclosure, disease severity risk may correspond to a binary classification or a multi-class classification.

[0094] In the case of binary classification, the present disclosure is applicable to the diagnosis of liver fibrosis stages, ie, distinguishing between early stages (F0 to F2) and late stages (F3 to F4).

[0095] In the case of binary classification, the present disclosure is applicable to the diagnosis of lung nodules, ie, distinguishing between benign and malignant.

[0096] In the case of multi-class classification, the present disclosure is applicable to the diagnosis of multiple stages (F0 to F4) of liver fibrosis.

[0097] The present disclosure proposes applying the generated machine learning model to perform classification on each candidate image in a plurality of three-dimensional medical images.

[0098] Figure 1 Flowcharts showing some examples of the method according to the first aspect of the present disclosure.

[0099] refer to Figure 1 , a method 100 for generating a machine learning model to predict disease severity risk from multiple three-dimensional medical images is shown.

[0100] As previously mentioned, in the context of the present disclosure, anatomical structures can be, for example, organs, muscles, soft tissues, arteries, veins, bones, subtissues, etc. Although the application of the present disclosure is discussed in the context of the liver and lungs, it is understood that the application can be extended to organs other than the liver and lungs, such as the spleen.

[0101] It should be understood that although the application is disclosed in the context of liver fibrosis and lung nodules, the disease in the context of the present disclosure can be manifested in a certain anatomical structure and may not be limited to a tumor (whether or not it is cancer) or to the liver or lung. In the context of the present disclosure, the anatomical change caused by the disease can be of any shape as long as it has a local deformation, whether it is a point or a bump, a concave or convex surface. In addition, the anatomical change can also be in the form of a tissue disconnection, such as a rupture or a hole.

[0102] It should be understood that, in the context of the present disclosure, "a plurality of" refers to at least three, which is the minimum number of images sampled for training a successful machine learning model. That is, at least three three-dimensional medical images are provided to generate a machine learning model for predicting the risk of disease severity. In the context of the present disclosure, the phrase "a plurality of" is interchangeable with the phrase "a set of", given that the latter satisfies the same image quantity requirement. In the context of the present disclosure, it is assumed that among the plurality of images, there are at least three images that are different from each other.

[0103] The three-dimensional medical image may be a CT and / or MRI image. The method proposed in the present disclosure is not limited to the manner in which the three-dimensional medical image may be acquired.

[0104] The method for acquiring 3D medical images can be applied indifferently to any MRI or CT imaging, including with or without contrast agents, i.e.:

[0105] - The patient undergoes a computed tomography (CT) scan with or without contrast injection prior to the scan, and

[0106] - Magnetic resonance imaging (MRI) with or without contrast injection prior to scanning, regardless of the sequence acquisition or reconstruction procedure.

[0107] In the context of the present disclosure, a three-dimensional image may be referred to as a “volume image.” In some examples, each three-dimensional medical image refers to a whole or partial three-dimensional medical image.

[0108] like Figure 1 As shown, the method 100 includes a step 110 of providing a plurality of three-dimensional medical images, and each of the plurality of three-dimensional medical images includes at least one anatomical structure of a patient. It should be noted that the at least one anatomical structure may be related to predicting the risk of disease severity.

[0109] In the context of the present disclosure, although it is required to provide at least three images that are different from each other, it is not required that the two different images come from different patients. In other words, the present disclosure does not require that various images are associated with various patients.

[0110] like Figure 1 As shown, after step 110 of providing a plurality of three-dimensional medical images, method 100 may further include step 120 of acquiring at least one three-dimensional segmentation mask from each of the plurality of three-dimensional medical images, and each mask in the at least one three-dimensional segmentation mask may include at least one anatomical structure.

[0111] It should be understood that in the context of the present disclosure, where a three-dimensional segmentation mask "includes" at least one anatomical structure, the term "comprise" is interchangeable with the term "delimit", both of which may have the meaning of determining the boundaries of at least one anatomical structure.

[0112] In some examples, step 120 of obtaining at least one three-dimensional segmentation mask from each of the plurality of three-dimensional medical images may include step 125 of segmenting each of the plurality of three-dimensional medical images such that at least one three-dimensional segmentation mask is generated for each three-dimensional medical image, and each mask includes at least one anatomical structure.

[0113] In some examples, a three-dimensional medical image includes or corresponds to at least one three-dimensional segmentation mask. In some examples, a three-dimensional segmentation mask includes or corresponds to at least one anatomical structure. In the context of the present disclosure, it is required that each of the multiple three-dimensional medical images used to generate a machine learning model for predicting disease severity risk includes at least one anatomical structure. Therefore, the segmentation of the anatomical structure can enable the prediction of the disease severity risk to be performed with respect to the segmented anatomical structure.

[0114] The segmentation mask may include at least one segmented anatomical structure, without limitation to the following examples:

[0115] - at least one organ, such as the liver, lungs and spleen;

[0116] - at least tumors, such as liver fibrosis;

[0117] - Combinations of organs and tumors, such as liver fibrosis or lung nodules.

[0118] In some examples, to facilitate post-processing, one segmentation mask may be further divided into at least one segmentation mask, as long as each segmentation mask includes at least one anatomical structure.

[0119] The present disclosure does not limit how a three-dimensional medical image can be segmented. In some examples, 3D segmentation of an organ or tissue can be obtained by automatically segmenting the organ or tissue using a deep supervised convolutional neural network. In some examples, 3D segmentation can be obtained by semi-automatic or manual segmentation.

[0120] It should be understood that if an automatic segmentation method is employed in the present method, this may advantageously result in automatic predictions. It should be noted that if manual or unsupervised segmentations are input, they may be compatible with the following steps of the present method. However, the predictions will not be automatic.

[0121] Still refer to Figure 1 The method 100 may further include a step 130 of reconstructing a mesh according to each of the at least one 3D segmentation mask.

[0122] By "mesh" is meant a discrete surface of a volume boundary, where the volume can be represented by voxels, i.e., a set of voxels of a region of interest. For example, the region of interest can be defined by a segmentation mask. A segmentation mask is typically a set of pixels in 2D (two-dimensional) or a set of voxels in 3D (three-dimensional), which are set to 1 to represent the region of interest, and other pixels or voxels outside the region of interest are set to 0.

[0123] By "reconstructing" a mesh from a 3D mask, we mean that we approximate the surface of the volumetric mask. In the example where the liver is the organ of interest, where the volume of the liver is represented by voxels, we can reconstruct the mesh surface of the liver. From the reconstructed mesh, morphological features are extracted and used to train a machine learning model.

[0124] In some examples, the step 130 of reconstructing a mesh from each 3D segmentation mask may include a step 133 of extracting a mesh from the 3D segmentation mask, followed by a step 135 of simplifying each reconstructed mesh, followed by a step 137 of smoothing the simplified reconstructed mesh.

[0125] The step 130 of reconstructing a mesh according to each three-dimensional segmentation mask may further include a step 139 of light smoothing after the reconstruction 130 and before the extraction 140 in order to close the holes on the simplified mesh.

[0126] In some other examples, this step 139 is performed after the smoothing step 137 after the simplification step 135. In some further examples, this step 139 uses a Poisson reconstruction algorithm.

[0127] In fact, the holes may be caused by an inappropriate previous segmentation. Although the mesh becomes less tight due to these holes, such defects can be solved by closing these holes in step 139. However, while closing the holes, physiological or morphological features (such as bumps and other diseases in the form of local deformations) may inevitably be lost through Poisson reconstruction, which allows the surface to be reconstructed from a given point cloud. After weighing the pros and cons, the optical smoothing step 139 may be optional. Figure 1 Optional aspects of this aspect are depicted by the relevant boxes with dashed lines. In some examples, step 139 can be omitted from method 100. It should be noted that omitting step 139 does not reduce the AUC-ROC score.

[0128] Step 133: Extracting the mesh from the 3D segmentation mask

[0129] As mentioned before, in the context of the present disclosure, the step 133 of extracting a mesh from a 3D segmentation mask (3D volume) refers to approximating a surface from a volume segmentation mask.

[0130] In some examples, this step 133 may be implemented by an algorithm for extracting a mesh from a 3D volume segmentation mask, such as the Marching Cube algorithm and its improvements.

[0131] The marching cubes algorithm is based on partitioning the input volume into a set of discrete cubes. In some examples of applying the marching cubes algorithm, the volume is given by a binary 3D segmentation mask of the organ. The algorithm then performs triangulation based on a lookup table of fifteen cubes for each cube intersected by the isosurfaces of the volume.

[0132] In some additional examples, Lewiner's Marching Cubes are used to extract a mesh from the segmentation mask volume in step 133. Using the Marching Cube of Lewiner in the present method is advantageous because it can resolve topological inconsistencies introduced by the Marching Cubes algorithm by reconstructing the volume with tetrahedrons. Thus, using the Marching Cube of Lewiner can resolve ambiguities and ensure topologically correct results.

[0133] Step 135: Simplify each reconstructed mesh

[0134] In some examples, the reconstructed Grid Simplify to remove inconsistencies or degeneracy, such as duplicate vertices, duplicate faces, and zero-area faces.

[0135] In some additional examples, the simplification step 135 may further improve triangle quality by collapsing short edges and splitting long edge sources.

[0136] In some examples where a Poisson reconstruction algorithm is used, the combination of simplification step 135 and light smoothing step 139 can have the following advantages: Since the Poisson reconstruction algorithm is highly dependent on vertex normals, removing degeneracy and ambiguity in advance through step 135 will help to better estimate the normals, thereby achieving a more geometrically accurate mesh by closing the holes with the Poisson reconstruction algorithm.

[0137] In some examples, the step 135 of simplifying the reconstructed mesh may also include a step 136 of adjusting the number of iterations of the simplification 135. This step 136 may be optional. Figure 1 Optional aspects of this aspect are depicted by the relevant boxes with dotted lines.

[0138] In fact, a common result of processing medical images using the marching cubes algorithm in step 133 is Z sampling artifacts. Such artifacts are undesirable because they may cause noise when calculating morphological features such as curvature, which in turn poses challenges in detecting physiological features. However, a higher number of iterations will reduce z sampling artifacts, but will not preserve surface details.

[0139] In some examples, the mesh is simplified n times, where n<=10, to slightly reduce these artifacts while preserving surface details.

[0140] Advantageously, adjusting the number of simplification iterations according to a diagnostic prediction metric (e.g. AUC-ROC) allows achieving an optimal signal-to-noise ratio while preserving some important physiological and / or pathological features present on the mesh, such as small bumps due to the disease, local deformations of the surface, etc. In fact, these small bumps can be attenuated by smoothing the mesh again through Poisson reconstruction.

[0141] Step 137: Smooth the simplified reconstructed mesh

[0142] In some examples, the reconstruction step 130 may include a smoothing step 137 after the step 135 of simplifying the reconstructed mesh. The smoothing step 137 allows for the detection and removal of artifacts. It should be noted that the smoothing step 137 may be optional. Figure 1 Optional aspects of this aspect are depicted by the relevant boxes with dotted lines.

[0143] In some examples, in order to attenuate the high-frequency content expected to be mainly caused by medical image acquisition artifacts, two smoothing methods are proposed that can preserve the topological structure of the simplified mesh, namely Laplace smoothing or Tobin smoothing. Of the two methods, Tobin smoothing gives better results than Laplace smoothing. This is because after too many iterations of Laplace smoothing, mesh shrinkage and mesh deformation will occur. In addition, too many iterations may also reduce the AUC-ROC score. In contrast, Tobin smoothing can preserve the global aspects of the mesh without the risk of mesh shrinkage. In the context of the present disclosure, the smoothing step 137 can adopt either of these two methods.

[0144] In some other examples, anisotropic smoothing can be used, which has been tested to be more effective than Laplace smoothing or Tobin smoothing in removing z artifacts. In the context of the present disclosure, anisotropic smoothing can be used to detect z sampling artifacts and remove them by smoothing the mesh. More advantageously, anisotropic smoothing allows z sampling artifacts to be removed without erasing features.

[0145] In some additional examples, anisotropic smoothing may include other computational methods to allow for detection and removal of artifacts.

[0146] In some further examples, anisotropic smoothing may use Laplace smoothing or Tobin smoothing, preferably Tobin smoothing, and may be combined with other computational methods to remove z-sampling artifacts.

[0147] In some examples, depending on the anatomical structures included in the segmentation mask, the smoothing method may be selected from: anisotropic smoothing and / or Laplace smoothing, or anisotropic smoothing and / or Tobin smoothing.

[0148] In cases where the segmentation mask includes liver fibrosis, anisotropic smoothing can be used because it removes Z sampling artifacts and preserves physiological or pathological features. In cases where the segmentation mask includes lung nodules, Laplace smoothing or Tobin smoothing can be used because small-sized nodules may have artifacts in all directions, not just in the Z direction.

[0149] It should be understood that in the context of the present disclosure, the "reconstructed mesh" may refer to the mesh after step 133 of extracting the mesh from the three-dimensional segmentation mask, may refer to the mesh after step 135 of simplifying the reconstructed mesh, i.e., the simplified reconstructed mesh, and may refer to the mesh after step 137 of smoothing the simplified reconstructed mesh. If step 139 for closing holes is applied, the "reconstructed mesh" may further refer to the mesh after all steps 133, 135, 137, and 139.

[0150] In some examples, a "reconstructed mesh" is a mesh that is Figure 1 Each reconstructed mesh shown is prepared for the next step 140 of extracting morphological features. Step 140 will be described in detail below.

[0151] As previously mentioned, in the context of the present disclosure, "morphological features" refer to geometric or topological invariants. In some examples, the morphological features include the curvature distribution of the reconstructed mesh. As previously mentioned, curvature is a geometric or topological invariant that can be used to quantify local deformation. In some examples, the curvature is the discrete curvature of the reconstructed mesh. In these examples, the discrete curvature can be obtained by discretizing the continuous curvature. Compared with the continuous curvature, the advantage of using discrete curvature is that it can greatly save computing time and resources. In some other examples, Gaussian curvature or mean curvature is used as the discrete curvature.

[0152] In some examples, the step 140 of extracting morphological features may include the step 143 of calculating the curvature distribution of the reconstructed mesh and the energy of the associated mesh, using the formula E(M)=∫ M |K G |dM, where E represents energy, M represents manifold mesh, and K G Represents Gaussian curvature. Energy is calculated as the sum of the absolute values ​​of the curvature of the entire mesh.

[0153] In some further examples, the step 143 of calculating the curvature of the reconstructed mesh may include the following sub-steps:

[0154] - Sub-step 1431 of calculating the mean curvature or Gaussian curvature of the mesh;

[0155] - a sub-step 1432 of calculating the probability distribution of the mean curvature or Gaussian curvature of the grid; and

[0156] - Sub-step 1433 of binning the probability distribution into distribution histograms.

[0157] Substep 1431, which calculates the discrete mean curvature or Gaussian curvature, requires having a manifold mesh because no curvature is defined for non-manifold vertices. In fact, if every edge in the mesh is a boundary edge or a manifold edge, then the mesh is manifold. For a surface, a manifold edge is part of exactly two faces. A boundary edge is part of exactly one face. If, starting from the adjacent faces of a vertex, all other adjacent faces of a vertex can be accessed (via face displacement), then the vertex is manifold. In some examples, an approximation to Steiner's formula is used to calculate the mean curvature or Gaussian curvature of a mesh.

[0158] In some examples, at sub-step 1433, the probability distribution of mean curvature or Gaussian curvature is binned into preferably 10 range values. It is to be understood that in the context of the present disclosure, binary is used to classify the curvature between negative curvature and positive curvature.

[0159] Preferably, the number of bins is set to 10, because it strikes a good balance between limiting the analysis dimension and preserving the curvature distribution information. Therefore, 10 can be used as a suitable bin size to cover the entire curvature data range. In other examples, the number of bins can be arbitrarily larger or smaller than 10. The number of bins can be selected based on the sample size of the data: the larger the sample size, the larger the number of bins, as an effort must be made to avoid the curse of dimensionality.

[0160] As mentioned before, the curvature distribution of the same organ with different pathologies (morphological deformations) may appear different, such as a healthy liver versus a cirrhotic liver.

[0161] Figure 2A and Figure 2B (On the right) the output of 3D mesh reconstruction is shown, i.e., the reconstructed meshes of a healthy liver (F0) and an advanced fibrotic liver (F4), respectively. The curvature values ​​are color-coded, with blue representing local deformations with negative curvature, such as pits and / or valleys; red representing local deformations with positive curvature, such as bumps and / or spikes; and green representing local deformations with near-zero curvature.

[0162] Figure 2A and Figure 2B (On the left) The probability distributions of their mean curvatures are shown for healthy liver and liver with advanced fibrosis, respectively.

[0163] In some further examples, the morphological features may further include baseline features, which may include at least one of the following features:

[0164] - Volume of anatomical structures;

[0165] - elongation of anatomical structures;

[0166] - a minimum size and / or a maximum size of an oriented bounding box of an anatomical structure; and

[0167] - Mean HPI values ​​for the anatomical structures.

[0168] In the above example, HPI stands for Hepatic Perfusion Index. It is defined as the ratio of the hepatic artery to the sum of the hepatic artery blood flow and the portal vein blood flow. In the above example, HPI mean refers to the mean value of the HPI over the entire liver volume.

[0169] The following paragraphs will demonstrate the advantages of using at least the curvature distribution as a morphological feature, and using both the curvature distribution and the baseline features, based on an exemplary machine learning classifier and with respect to two exemplary applications, namely, prediction of liver fibrosis stage and prediction of malignancy of lung nodules.

[0170] refer to Figure 1 , method 100 further includes a step 150 of training a machine learning model to perform classification of disease severity risk. In some examples, the training is performed based on the extracted morphological features.

[0171] In some examples, the machine learning model is a supervised classifier model; in some other examples, the machine learning model is a supervised classifier based on a decision tree, such as a random forest or XGboost. The advantage of using a random forest or XGBoost classifier is that it is an algorithm that selects a function that separates the categories, and the function can be nonlinear.

[0172] In some examples where a small sample size dataset is used for training, a random forest classifier is preferred. In some other examples where a large sample size dataset is used for training, XGboost is preferred.

[0173] Although in Figure 1 1, but method 100 may also include providing clinical information about each patient. In some examples, training step 150 is also performed based on the patient's clinical information. It should be understood that in the context of the present disclosure, the term "clinical information of a patient" refers to data about a patient that has been obtained through a clinical examination. It may include at least one of each patient's age, gender, height, weight, BMI, race, medical condition, and risk factor.

[0174] Alternatively or in addition to the clinical information, the method 100 may also include extracting other radiomic features other than morphological features from the provided plurality of three-dimensional medical images so as to further perform training based on the other radiomic features. The other radiomic features may include at least one of the following: the most advanced radiomic library provided by Van Griethuysen et al., including Haralick GLCM (grayscale co-occurrence matrix), GLRLM (grayscale run length matrix), GLSZM (grayscale size zone matrix), NGTDM (neighborhood grayscale difference matrix), GLDM (grayscale dependency matrix) features, standard statistics of image grayscale intensity features, and classical morphology related features.

[0175] Example 1: Liver fibrosis stage prediction

[0176] According to clinicians, an important task is to distinguish between the early stages of fibrosis from F0 to F2 and the late stages (F3, F4). Moreover, the early and late stages of fibrosis present important differences in the degree of liver surface nodularity.

[0177] The dataset for fibrosis stage characterization contains 112 patients, where masks of their liver and spleen were segmented from arterial phase CT scans using a deep convolutional neural network, and the fibrosis stage from F0 to F4 was obtained by biopsy histology. In this dataset, 16% were F0, 4% were F1, 5% were F2, 12% were F3, and 63% were F4.

[0178] For all these reasons, and especially in view of the small sample size and the uneven prevalence of fibrosis stages, a multi-class representation of fibrosis stage is inappropriate. Therefore, the prediction in Example 1 consists of a binary classification, pooling F0 to F2 livers (25%) on the one hand and F3 and F4 livers (75%) on the other hand. The dataset is split into 50% for training and 50% for testing.

[0179] Figure 3A and Figure 3B The performance of the prediction model using random forest classification for the F0-F2 stage versus the F3-F4 stage (indicated by the blue ROC curve) is shown, as well as the predictive power of each feature. Figure 4B In , baseline features 0-5 are shown above the black solid line, while liver surface features 6-16 (including curvature and mesh energy) are shown above the blue solid line. The predictive power of each feature can be calculated using the impurity-based feature importance method.

[0180] It is observed that using only the baseline features, the performance of the random forest classifier is 0.88 as assessed by the AUC-ROC score. In the examples using the baseline features, the random forest classifier has a minimum of 100 trees and a maximum of 300 trees. The depth is set to 3, 5, 15, or 25. In some examples, the number of features to consider when finding the best split is between 1 and 6. In some other examples, the minimum number of samples required to split an internal node is 3, 5, or 10.

[0181] In some other examples, morphological features are used, combining baseline features with curvature distribution and mesh energy, the latter obtained on masks segmented using deep-CNN, where the number of simplification iterations is set to 10 and the number of Tobin smoothing iterations is set to 5, and the AUC-ROC score is improved by 0.08, from 0.88 to 0.96.

[0182] Example 2: Prediction of malignancy of lung nodules

[0183] The dataset consists of a small subset of NLST ("National Lung Screening Trial") cancer patients and healthy patients with nodules, and contains 789 lung nodules annotated by clinicians and semi-automatically manually segmented, of which 79% are benign and 21% are malignant.

[0184] As mentioned above, for the diagnosis of pulmonary nodules, it is well known that local deformation of the nodule surface, such as its pointed spinous processes, has important predictive power for the malignancy of the nodule, while "round" and smooth borders usually indicate that the lesion is benign.

[0185] Advantageously, curvature can be used to quantify these local deformations, which will allow differentiation between regular and spinous surfaces.The present method replaces the typically qualitative and subjective clinician's judgement with morphological features such as curvature.

[0186] Figure 4A and Figure 4B (On the right) The output of 3D mesh reconstruction, i.e., the reconstructed meshes of benign and malignant pulmonary nodules, are shown, respectively. The curvature values ​​are color-coded, with blue representing local deformations with negative curvature, such as pits and / or valleys; red representing local deformations with positive curvature, such as bumps and / or spikes; and green representing local deformations with near-zero curvature.

[0187] Figure 4A and Figure 4B (On the left) The probability distributions of the mean curvatures of benign and malignant pulmonary nodules are shown, respectively.

[0188] like Figure 4B and Figure 4BAs shown, malignant and benign lesions often present different curvature distributions, with high absolute values ​​of curvature being overrepresented in malignant nodules. In this example, the mesh is reconstructed without using Poisson reconstruction.

[0189] Similar to Figure 2A and Figure 2B , Figure 4A and Figure 4B (On the right) A mesh of all organs is shown, with their mean or Gaussian curvature or various probabilities of derived morphological features, their probability distributions and the patient's clinical information color-coded on each facet.

[0190] Figure 5 The performance of malignancy prediction using only curvature features is shown. Figure 6 In the example of lung malignancy prediction (benign: 0, malignant: 1 or vice versa), only the curvature distribution and mesh energy of the lung nodules were used as features for malignancy characterization, with the number of smoothing iterations set to 0, and classification as benign or malignant nodules by random forests gave an AUCROC of 0.93.

[0191] Figure 6 Flowcharts showing some examples of the method 200 according to the second aspect of the present disclosure, i.e., a method for predicting disease severity risk from multiple three-dimensional medical images by applying a machine learning model. The machine learning model may have been generated by implementing the method 100 according to any embodiment of the first aspect of the present disclosure.

[0192] It should be noted that, in the context of the present disclosure, the previous description of the implementation of method 100 should be applicable to the implementation of method 200. For example, terms such as "organ", "disease", "plurality", "three-dimensional medical image", "anatomical structure", "segmentation mask", "reconstruction", "morphological feature", "curvature", "mean curvature", "Gaussian curvature", "baseline feature", "patient clinical information" and the like have the same meaning in method 200 as in method 100.

[0193] like Figure 6 As shown, the method 200 includes a step 205 of receiving a plurality of three-dimensional medical images, each of the plurality of three-dimensional medical images including at least one anatomical structure of a patient, and a step 210 of providing a medical image of the plurality of medical images as a candidate image for prediction.

[0194] like Figure 6As shown, the method 200 further includes a step 220 of acquiring at least one candidate 3D segmentation mask from the candidate image. In some examples, each mask of the at least one candidate 3D segmentation mask may include at least one anatomical structure.

[0195] The step 220 of obtaining at least one candidate three-dimensional segmentation mask from each candidate image may include a step 225 of segmenting each candidate image such that at least one candidate three-dimensional segmentation mask is generated for the candidate image and each mask includes at least one anatomical structure.

[0196] It will be appreciated that if an automated segmentation approach is employed in the present method, this may advantageously result in an automated classification of disease severity risk.

[0197] like Figure 6 As shown, the method 200 may further include a step 230 of reconstructing the candidate mesh according to each of the at least one candidate 3D segmentation mask. The step 230 may be consistent with the step 130 described in detail above. That is, the step 230 of reconstructing the candidate mesh according to each of the at least one candidate 3D segmentation mask may include a step 233 of extracting the candidate mesh from each candidate 3D segmentation mask, followed by a step 235 of simplifying each reconstructed candidate mesh, followed by a step 237 of smoothing the simplified reconstructed candidate mesh. The step 230 of reconstructing the candidate mesh according to each candidate 3D segmentation mask may further include a step 239 of light smoothing to close the holes on the simplified mesh after reconstruction 230 and before extraction 240. Each of the steps 233, 235, 237, and 239 may respectively be consistent with each of the steps 133, 135, 137, and 139 described in detail above.

[0198] Still reference Figure 6 The method 200 may further include a step 240 of extracting morphological features from each candidate reconstructed mesh. The step 240 may be consistent with the step 140 described in detail above.

[0199] In some examples, the step 240 of extracting morphological features may include a step 243 of calculating the curvature distribution of the reconstructed candidate mesh and the energy of the associated candidate mesh, E(M)=∫ M |K G |dM.

[0200] In some further examples, step 243 of calculating the curvature of the reconstructed candidate mesh may include the following sub-steps:

[0201] - Sub-step 2431 of calculating the mean curvature or Gaussian curvature of the mesh;

[0202] - a sub-step 2432 of calculating a probability distribution about the mean curvature or Gaussian curvature of the grid; and

[0203] - Sub-step 2433 of binning the probability distribution into distribution histograms.

[0204] It should be understood that each of sub-steps 2431, 2432 and 2433 may conform to each of sub-steps 1431, 1432 and 1433 described in detail above.

[0205] Still reference Figure 6 , method 200 may also include step 250 of inputting the extracted morphological features into a machine learning model to perform classification of disease severity risk.

[0206] This makes it possible to predict the risk of disease severity based on multiple three-dimensional medical images.

[0207] The disclosed embodiments and aspects described in detail herein may be described and illustrated in conjunction with systems, devices, and methods, which are intended to be exemplary and illustrative, rather than limiting in scope. Specific configurations and details may be listed to provide an understanding of the present disclosure. However, it will be apparent to those skilled in the art that the present disclosure may be implemented without some of the specific details presented herein. In addition, for clarity of description, some well-known steps or components may be described only in general terms, or even omitted.

[0208] Some processes may be presented and described in a series of steps (sequences). It should be understood that the sequence of steps is exemplary, and the steps may be performed in a different order than that presented, some of the described steps may be omitted, and some additional steps may be omitted from the sequence and may be described elsewhere.

[0209] Reference may be made to the disclosures of previous patents, publications, and applications. Some of the text and drawings in these sources may be presented herein, but may be modified, edited, or annotated to more smoothly blend with the disclosure of the present application. The citation or identification of any reference should not be construed as an admission that such reference may be used as prior art for the present disclosure.

[0210] The method described herein can be implemented on one or more computers of any form. Its components can be implemented as special applications or in client-server architectures (including network-based architectures), and can include functional programs, codes, and code segments. Any computer can include a processor, a memory for storing program data and executing data, a permanent storage device such as a disk drive, a communication port for processing communication with an external device, and a user interface device including a display, a keyboard, a mouse, etc. When it comes to some software or algorithms, they can be stored in computer-readable media such as read-only memory (ROM), random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage devices as program instructions or computer-readable codes that can be executed on a processor. Computer-readable recording media can also be distributed on a network-coupled computer system so that computer-readable codes are stored and executed in a distributed manner. The medium can be read by a computer, stored in a memory, and executed by a processor.

[0211] In order to promote the understanding of the principles of the various embodiments of the present disclosure, reference is made to the preferred embodiments shown in the accompanying drawings and specific language is used to describe the embodiments. However, this specific language is not intended to limit the scope of the present disclosure, and the present disclosure should be interpreted as including all embodiments that would normally occur to a person of ordinary skill in the art.

[0212] The present disclosure can be described in terms of functional block components and various processing steps. These functional blocks can be implemented by any number of hardware and / or software components configured to perform specified functions. For example, the present disclosure can adopt various integrated circuit components, such as memory elements, processing elements, logic elements, look-up tables, etc., which can perform various functions under the control of one or more microprocessors or other control devices. Similarly, in the case of using software programming or software elements to implement the elements of the present disclosure, the present disclosure can be implemented using any programming language or scripting language, such as C, C++, Java, Python, assembly language, etc., wherein various algorithms are implemented using any combination of data structures, objects, processes, routines or other programming elements. Functional aspects can be implemented by algorithms executed on one or more processors. In addition, the present disclosure can adopt any number of conventional techniques to carry out electronic configuration, signal processing and / or control, data processing, etc.

Claims

1. A method (100) for generating a machine learning model for predicting the risk of disease severity based on a plurality of three-dimensional medical images, the method comprising the following steps: - providing (110) a plurality of three-dimensional medical images, each image of the plurality of three-dimensional medical images comprising at least one anatomical structure of a patient; - acquiring (120) at least one three-dimensional segmentation mask from each image of the plurality of three-dimensional medical images, each mask of the at least one three-dimensional segmentation mask comprising at least one anatomical structure; - reconstructing (130) a mesh from each three-dimensional segmentation mask; - extracting (140) morphological features from each reconstructed mesh; as well as - Training (150) a machine learning model to perform classification of disease severity risk, wherein the training is performed on the extracted morphological features.

2. The method (100) according to claim 1, wherein: Reconstructing (130) a mesh from each three-dimensional segmentation mask comprises extracting (133) a mesh from the three-dimensional segmentation mask by using marching cubes, and preferably, reconstructing (130) a mesh from each three-dimensional segmentation mask comprises extracting (133) a mesh from the three-dimensional segmentation mask by using Lewinna marching cubes.

3. The method (100) according to claim 2, wherein: Reconstructing (130) a mesh based on each three-dimensional segmentation mask also includes the step of simplifying (135) each reconstructed mesh to eliminate inconsistencies.

4. The method (100) according to claim 3, wherein: Reconstructing (130) a mesh according to each mask of the at least one three-dimensional segmentation mask also comprises: after simplifying (135) each reconstructed mesh, performing a smoothing (137) step to detect artifacts and remove them.

5. The method (100) according to claim 4, wherein: The smoothing (137) step is determined based on the anatomical structure included in the segmentation mask, preferably, the smoothing (137) step is one of the following: - anisotropic smoothing and / or Laplace smoothing; or - Anisotropic smoothing and / or Tobin smoothing.

6. The method (100) according to any one of the preceding claims, wherein: The morphological feature at least comprises a curvature distribution of the reconstructed mesh, and preferably, the morphological feature comprises a discrete curvature distribution of the reconstructed mesh.

7. The method (100) according to claim 6, wherein: Extracting (140) morphological features includes calculating (143) the curvature distribution of the reconstructed mesh and the energy of the associated mesh.

8. The method (100) according to claim 7, wherein: Calculating (143) the curvature distribution of the reconstructed mesh comprises the following steps: - calculating (1431) a mean curvature or a Gaussian curvature about the grid, preferably by using an approximation of the Steiner formula; - calculating (1432) a probability distribution about the mean curvature or Gaussian curvature of the grid, and - Binning (1433) the probability distribution into a distribution histogram.

9. The method (100) according to any one of claims 6 to 8, wherein: The morphological features also include baseline features, and the baseline features include at least one of the following: - the volume of the at least one anatomical structure; - elongation of the at least one anatomical structure; - a minimum size and / or a maximum size of the oriented bounding box of the at least one anatomical structure; and - The HPI mean value of said at least one anatomical structure.

10. The method (100) according to any one of the preceding claims, wherein: The machine learning model being trained (150) is a supervised classifier model, preferably, the machine learning model being trained (150) is a decision tree based classifier.

11. The method (100) according to any one of the preceding claims, further comprising: - provide clinical information about each patient, and The training (150) is also performed on the clinical information of the patient.

12. The method (100) according to any one of the preceding claims, wherein: The three-dimensional medical images include computed tomography (CT) images and / or magnetic resonance imaging (MRI) images.

13. A method (200) for predicting the risk of disease severity based on a plurality of three-dimensional medical images by applying a machine learning model, the method comprising the following steps: - receiving (205) a plurality of three-dimensional medical images, each image of the plurality of three-dimensional medical images comprising at least one anatomical structure of a patient; - providing (210) a medical image from the plurality of medical images as a candidate image for prediction; - obtaining (220) at least one candidate three-dimensional segmentation mask from the candidate image, each mask of the at least one candidate three-dimensional segmentation mask comprising at least one anatomical structure; - reconstructing a candidate mesh based on each mask of the at least one candidate three-dimensional segmentation mask (230); - extracting (240) morphological features from each reconstructed candidate mesh; - Inputting (250) the extracted morphological features into the machine learning model to perform classification on the severity risk of the disease.

14. The method (200) of claim 13, wherein: The machine learning model has been generated by implementing the method (100) for generating a machine learning model according to any one of claims 1 to 12.

15. The method (200) according to any one of claims 13 to 14, wherein: Reconstructing (230) a candidate mesh from each mask of the at least one candidate three-dimensional segmentation mask comprises extracting (233) a mesh from the three-dimensional segmentation mask by using marching cubes, and preferably, reconstructing (230) a candidate mesh from each mask of the at least one candidate three-dimensional segmentation mask comprises extracting (233) a mesh from the three-dimensional segmentation mask by using Lewinna marching cubes.

16. The method (200) of claim 15, wherein: Reconstructing (230) a candidate mesh based on each mask of the at least one candidate three-dimensional segmentation mask further comprises the following steps: Each reconstructed candidate mesh is simplified (235) to eliminate inconsistencies.

17. The method (200) of claim 16, wherein: Reconstructing (230) a candidate mesh based on each mask of the at least one candidate three-dimensional segmentation mask further comprises: after simplifying (235) each reconstructed candidate mesh, performing a smoothing (237) step to detect artifacts and remove them.

18. The method (200) of claim 17, wherein: The smoothing (237) step is determined based on the candidate anatomical structure included in the candidate segmentation mask, preferably, the smoothing (237) step is one of the following: - anisotropic smoothing and / or Laplace smoothing; or - Anisotropic smoothing and / or Tobin smoothing.

19. The method (200) according to any one of claims 13 to 18, wherein: The morphological feature at least includes a curvature distribution of the reconstructed candidate mesh. Preferably, the morphological feature includes a discrete curvature distribution of the reconstructed candidate mesh.

20. The method (200) of claim 19, wherein: Extracting (140) morphological features includes calculating (143) the curvature distribution of the reconstructed candidate meshes and the energy of the associated candidate meshes.

21. The method (200) of claim 20, wherein: Calculating (143) the curvature distribution of the reconstructed candidate mesh comprises the following steps: - calculating (2431) a mean curvature or a Gaussian curvature about the grid, preferably by using an approximation of the Steiner formula; - calculating (2432) a probability distribution about the mean curvature or Gaussian curvature of the grid, and - Binning (2433) the probability distribution into a distribution histogram.

22. The method (200) according to any one of claims 19 to 21, wherein: The morphological features also include baseline features, and the baseline features include at least one of the following: - the volume of the at least one anatomical structure; - elongation of the at least one anatomical structure; - a minimum size and / or a maximum size of the oriented bounding box of the at least one anatomical structure; and - The HPI mean value of said at least one anatomical structure.

23. The method (200) according to any one of claims 13 to 22, wherein: The three-dimensional medical image includes a computed tomography (CT) image and / or a magnetic resonance imaging (MRI) image.

24. A computer program product comprising instructions which, when executed by at least one digital processing device, perform at least one of the steps of the method (100) according to claims 1 to 12 and at least one of the steps of the method (200) according to claims 13 to 23.

Citation Information

Patent Citations

  • Aneurysm rupture risk prediction method and device and storage medium

    CN113066583A

  • Method for the detection and staging of liver fibrosis from image acquired data

    US10130295B2