Method and system for processing images - Patents.com

Abstract

According to one aspect, a computer-implemented method is provided for processing an image of a subject including lymph nodes, the method comprising the steps of segmenting the lymph nodes in image data corresponding to the image, outlining the lymph nodes from the segmented image data, classifying the lymph nodes as belonging to a predefined region, evaluating the region of the image based on an assessment of a risk that the region includes a malignant lymph node, and indicating the evaluation of the region.

Description

[Technical field]

[0001] The present invention relates to a method for processing images of a subject, and more particularly to a method for processing images of a subject including lymph nodes. [Background technology]

[0002] TNM staging (T-tumor, N-lymph nodes, M-metastasis) based on computed tomography (CT) and / or positron emission tomography (PET) imaging is a task in oncology applied at diagnosis during treatment response evaluation after surgery. The T stage is defined by the size and location of the primary tumor. The classification of N and M stages requires a regional-based examination of the spread of the cancer to the lymphatic system.

[0003] Affected lymph nodes can be identified in PET or CT images by determining characteristics such as increased size, abnormal shape, non-homogeneous tissue, and high metabolic activity. However, assessment of lymph nodes from PET or CT is an extremely challenging and time-consuming task due to (1) complex and cancer-specific labeling schemes as visualized in Figure 1, where lymph nodes belonging to different regions have different cross-hatchings (taken from Mountain, CF, & Dresler, CM (1997). Regional lymph node classification for lung cancer staging. Chest, 111(6), 1718-1723, showing an example of lymph node labeling in the mediastinum), (2) risk boundaries according to regions, where risk indicators differ according to the region to which the lymph node belongs, (3) a large number of lymph nodes (approximately 300-700) spread widely in the human body, and (4) potentially unclear node boundaries due to small contrast differences between lymph nodes and surrounding tissues or adjacent lymph nodes forming clusters. Although important for treatment decisions, clinical workflows typically used to evaluate lymph nodes under high time pressure only allow assessment of a small set of selected lymph nodes, which can sometimes lead to inaccurate staging results. Examples of such suboptimal staging results are, for example, missed enlarged lymph nodes or missed lymph nodes that are not pathological by baseline but enlarge over time. Therefore, more advanced lymph node stratification is needed. Summary of the Invention [Problem to be solved by the invention]

[0004] Manual detection, measurement, and labeling of potentially malignant lymph nodes in PET or CT images is time-consuming and highly dependent on the radiologist's experience due to the aforementioned challenges (1)-(4). Accurate delineation of lymph nodes can take hours (especially in the case of lymphoma), limiting the assessment to a few selected lymph nodes. Furthermore, a particular challenge is the assessment of lymphoma, where most lymph nodes increase in size and often grow into large masses. [Means for solving the problem]

[0005] Rather than investigating only a few selected lymph nodes (as is typically done in clinical workflow), the methods disclosed herein outline an approach to automated risk assessment by nodal region to facilitate fast and accurate (T)NM staging and reduce dependency on the reader. Advantageously, the regional aggregation is less susceptible to minor imprecision for individual lymph nodes. The burden of transitioning lymphatic tissue nodes from per-node to per-region is further advantageous since N and M stages are defined by nodal involvement by region, and therefore results are directly related to these stages. Furthermore, such methods facilitate comparison of follow-up scans, as regional assessment is more likely to be quicker and more accurate than attempting to find the same previously determined lymph nodes as a comparison.

[0006] According to one aspect, a computer-implemented method is provided for processing an image of a subject including lymph nodes, the method comprising: segmenting lymph nodes in image data corresponding to the image, outlining the lymph nodes from the segmented image data, classifying the lymph nodes as belonging to a region, evaluating the region of the image based on an assessment of a risk that the region includes a malignant lymph node, and indicating the evaluation of the region. The region is an anatomical region and / or a nodal region, for example an anatomical nodal region (e.g., an anatomically defined region of a node (e.g., a lymph node)). The evaluated region of the image corresponds to or is the same as the region in which the lymph nodes are classified.

[0007] Thus, a method is provided in which assessment of the likelihood that a region contains malignant lymph nodes is made on a region-by-region rather than node-by-node basis, which advantageously improves the speed and accuracy with which an image of a subject can be determined.

[0008] According to a further aspect, there is provided a method of training a machine learning model for use in processing images of a subject including lymph nodes, the method comprising obtaining training data, the training data comprising images of the subject with at least one annotation of the lymph nodes, a set of rules for determining a risk of the lymph node being malignant or whether the lymph node is malignant, and annotations indicating at least one region, the method further comprising training a model based on the training data to classify regions of the image based on an assessment of a risk that the region includes a malignant lymph node.

[0009] According to a further aspect, there is provided a computer program product including a computer readable medium having computer readable code embodied therein, the computer readable code being configured to, upon execution by a suitable computer or processor, cause the computer or processor to perform the methods described herein.

[0010] According to a further aspect, there is provided a system for processing an image of a subject including a lymph node, the system comprising: a memory including instruction data representing a set of instructions; and a processor in communication with the memory and configured to execute the set of instructions, which when executed by the processor causes the processor to segment image data corresponding to the image, delineate lymph nodes from the segmented image data, classify the lymph nodes as belonging to a region, evaluate the region of the image based on an assessment of a risk that the region includes a malignant lymph node, and indicate the evaluation of the region.

[0011] These and other aspects will be apparent from and will be elucidated with reference to the embodiment(s) described hereinafter.

[0012] Example embodiments will now be described, by way of example only, with reference to the following drawings, in which: [Brief description of the drawings]

[0013] [Figure 1] FIG. 1 shows an example of lymph node labeling in the mediastinum. [Diagram 2] FIG. 1 illustrates an example of a system according to an example. [Diagram 3] FIG. 1 illustrates an example of a method by example. [Figure 4] FIG. 1 shows images relating to the method described herein. [Diagram 5] FIG. 2 illustrates image inputs and outputs in the method described herein. [Figure 6] FIG. 1 shows images resulting from the methods described herein. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Turning now to Figure 2, in some embodiments there is an apparatus 200 for use in a method of processing images of a subject including lymph nodes according to some embodiments herein. Typically the apparatus forms part of a computing device or system, such as, for example, a laptop, desktop computer, or other computing device. In some embodiments the apparatus 200 forms part of a distributed computing arrangement or cloud.

[0015] The apparatus comprises a memory 204 containing instruction data representing a set of instructions, and a processor 202 (e.g., processing circuitry or logic) in communication with the memory and configured to execute the set of instructions. In general, the set of instructions, when executed by the processor, causes the processor to perform any of the embodiments of a method for processing an image of a subject including lymph nodes, as described below.

[0016] An embodiment of the apparatus 200 is for use in processing an image of a subject that includes lymph nodes. More specifically, the set of instructions, when executed by a processor, cause the processor to segment lymph nodes in image data corresponding to the image, delineate the lymph nodes from the segmented image data, classify the lymph nodes as belonging to a region, evaluate the region of the image based on an assessment of a risk that the region contains a malignant lymph node, and indicate the evaluation of the region.

[0017] The processor 202 may comprise one or more processors, processing units, multi-core processors, or modules configured or programmed to control the device 200 in the manner described herein. In certain implementations, the processor 202 may comprise multiple software and / or hardware modules, each configured or intended to perform individual or multiple steps of the methods described herein. The processor 202 may comprise one or more processors, processing units, multi-core processors, and / or modules configured or programmed to control the device 200 in the manner described herein. In some implementations, for example, the processor 202 comprises multiple (e.g., used simultaneously) processors, processing units, multi-core processors, and / or modules configured for distributed processing. It will be understood by those skilled in the art that such processors, processing units, multi-core processors, and / or modules may be located in different locations and perform different steps of the methods described herein and / or different portions of a single step.

[0018] The memory 204 is configured to store program code that may be executed by the processor 202 to perform the methods described herein. Alternatively or additionally, the one or more memories 204 are external to the device 200 (i.e., separate from or remote from the device 200). For example, the one or more memories 204 are part of another device. The memory 204 may be used to store images, such as images of a subject, image data, segmented image data, results of a method, indications, images showing the indications, data indicative of an evaluation, and / or any other information or data received, calculated, or determined by the processor 202 of the device 200 or from any interface, memory, or device external to the device 200. The processor 202 is configured to control the memory 204 to store images, such as images of a subject, image data, segmented image data, results of a method, indications, images showing the indications, data indicative of an evaluation, and / or any other information or data received, calculated, or determined.

[0019] In some embodiments, memory 204 comprises multiple sub-memories, each capable of storing a portion of the instruction data, e.g., at least one sub-memory stores instruction data representing at least one instruction of the set of instructions, while at least one other sub-memory stores instruction data representing at least one other instruction of the set of instructions.

[0020] It will be understood that Fig. 2 only shows the components required to illustrate this aspect of the disclosure, and that in an actual implementation, the device 200 will comprise additional components to those shown. For example, the device 200 further comprises a display. The display may comprise, for example, a computer screen and / or a screen on a mobile phone or tablet. The device further comprises a user input device, such as a keyboard, mouse, or other input device that allows a user to interact with the device, for example to provide initial input parameters to be used in the methods described herein. The device 200 comprises a battery or other power source for powering the device 200, or a means for connecting the device 200 to a mains power source.

[0021] 3, there is a computer-implemented method 300 for use in processing images of a subject that includes lymph nodes. An embodiment of method 300 may be performed by an apparatus such as, for example, apparatus 200 described above.

[0022] Briefly, in a first step 302, the method 300 comprises segmenting lymph nodes in image data corresponding to the image. In a second step 304, the method comprises delineating lymph nodes from the segmented image data (e.g., isolating individual lymph nodes or clusters of lymph nodes from the segmented image data). In a third step 306, the method includes classifying lymph nodes as belonging to a region. In a fourth step 308, the method comprises evaluating a region of the image based on an assessment of the risk that the region contains a malignant lymph node. In a fifth step 310, the method comprises indicating the evaluation of the region.

[0023] The methods herein apply to a wide variety of images, such as, for example, medical images. Image segmentation involves extracting shape / morphological information about objects or shapes captured in an image. This is accomplished by converting the image into building blocks or "segments" that represent different features in the image. In some methods, image segmentation involves fitting a model to one or more features in the image.

[0024] One method of image segmentation is model-based segmentation (MBS), whereby a triangulated mesh of a structure of interest (e.g., heart, brain, lungs, lymph nodes, etc.) is fitted to features in an image in an iterative manner. Segmentation models typically encode population-based appearance features and shape information, and the model-based segmentation process both segments and outlines (individual) lymph nodes in one step. Such information describes allowed shape variations based on the actual shapes of the structures of interest in the members of the population. Shape variations are encoded, for example, in the form of eigenmodes that describe the manner in which changes in one part of the model are forced or dependent on the shape of other parts of the model.

[0025] Model-based segmentation has been used in a variety of applications to segment one or more organs of interest from medical images, see, for example, the 2008 paper by Ecabert, O. et al., entitled "Automatic Model-Based Segmentation of the Heart in CT Images"; IEEE Trans. Med. Imaging 27(9), 1189-1201. The use of triangulated surface meshes has resulted in MBS achieving generally smooth segmentation results. Furthermore, MBS is generally considered to be robust to image artifacts, such as variations in image quality.

[0026] Another segmentation method uses machine learning (ML) models to convert an image into multiple constituent shapes (e.g., block shapes or block volumes) based on similar pixel / voxel values ​​and image gradients, as described, for example, in the paper by Long et al., entitled "Fully Convolutional Networks for Semantic Segmentation."

[0027] Voxel-wise segmentation (performed by a voxel-wise segmentation network) is also used to perform the methods described herein. In the voxel-wise segmentation used in the methods described herein, for each voxel of the image data, it is determined whether the voxel belongs to a lymph node or not. The image data is first segmented (e.g., segmenting lymph nodes in the image data, where it is determined for each voxel whether the voxel belongs to a lymph node or to a node), and then from the segmented image data, the (individual) lymph nodes are delineated (e.g., determining the location of the individual lymph nodes). Voxel-wise segmentation is beneficial in lymph node segmentation, for example, as opposed to model-based segmentation, due to the high variability of lymph node shape.

[0028] Any suitable segmentation method may be used to segment the lymph nodes in the image. More particularly, the image (e.g., the image to be segmented) may be any type of image. In some embodiments, the image comprises a scientific image. In some embodiments, for example, the image comprises a medical image.

[0029] Medical images include images captured using any imaging modality. Examples of medical images include, but are not limited to, computed tomography (CT) images (e.g., from a CT scan), such as a C-arm CT image, a spectral CT image, or a phase contrast CT image, an X-ray image (e.g., from an X-ray scan), a magnetic resonance (MR) image (e.g., from an MR scan), an ultrasound (US) image (e.g., from an ultrasound scan), a fluoroscopic image, a nuclear medicine image, or any other three-dimensional medical image.

[0030] More generally, the images include images captured using a charge-coupled device, CCD, such as in a camera. Those skilled in the art will appreciate that the embodiments herein may be applied to other types of segmented images and / or other data sets.

[0031] Generally, an image comprises a two-dimensional image or a three-dimensional image. An image comprises a plurality (or set) of image elements. For example, in embodiments where the image comprises a two-dimensional image, the image elements comprise pixels. In embodiments where the image comprises a three-dimensional image, the image elements comprise voxels.

[0032] Image features include any object (e.g., real or simulated), shape or portion of an object, or its shape that is visible (e.g., discernable) in the image. In embodiments where the image includes a medical image, features include anatomical features, or portions thereof, such as parts of the lungs, heart, brain, or any other anatomical feature. Lymph nodes are included in the image.

[0033] The region is evaluated for at least one risk factor. The risk factors include one of lymph node volume, lymph node short axis length, lymph node long axis length, quantitative tissue size, positron emission tomography (PET) intensity, increased lymph node size, high metabolic activity, formation of connected lymph node clusters, regional lymphatic tissue volume, and gray value statistics. The risk factors are region dependent. For example, lymph node size may vary more in some (anatomical) regions than in others. An example of this would be lymph nodes under the ear, where swelling is not a sign of cancer. Thus, in an example, swollen lymph nodes in one region would not be a risk factor, while swollen lymph nodes in another region would be a risk factor. Risk factors in different regions are given different weights.

[0034] The evaluating includes evaluating the plurality of lymph nodes for at least one risk factor and evaluating the region based on evaluation of at least one lymph node of the plurality of lymph nodes corresponding to the region, the region being an anatomical region and / or a predefined region, where the region is used in a manner dependent on the type of cancer being determined.

[0035] Presenting the assessment includes outputting at least one of risk statistics for the region, a color overlay corresponding to the image showing the risk associated with the region, a heat map corresponding to the image showing the risk associated with the region, and an overlay on the potentially malignant lymph nodes.

[0036] The evaluating further includes comparing the assessment of risk of the region to a previously obtained assessment of risk of the region. The evaluating further includes determining whether the region has at least one of an increased risk as compared to a previously determined risk and a high variance between the previous risk assessment and the current risk assessment as compared to the previously determined risk.

[0037] The evaluating is performed for a plurality of regions of the image, and the evaluation indication includes at least one of: region risk statistics, a list of regions with an indication of a classification for each region, a list of regions ordered based on an evaluation of each region, and a heat map corresponding to the evaluation of each region.

[0038] The method (or a part / parts of the method, such as evaluation and / or classification) is performed by a trained model. The model includes any type of model, such as, for example, a model trained using a machine learning process. Examples of models used herein include, but are not limited to, neural network models, such as deep learning neural networks, and random forest models.

[0039] Those skilled in the art will be familiar with neural networks, but briefly, a neural network is a type of supervised machine learning model that can be trained to predict a desired output given input data. A neural network is trained by providing training data that includes example input data and the corresponding "correct" or ground truth outcomes that are desired. A neural network includes multiple layers of neurons, with each neuron representing a mathematical operation that is applied to the input data. The output of each layer in the neural network is fed into the next layer to generate an output. For each portion of the training data, the weights associated with the neurons are adjusted until an optimal weighting is found that produces predictions on the training examples that reflect the corresponding ground truth.

[0040] While examples of neural networks are provided herein, one of ordinary skill in the art will understand that a trained model generally includes any model trained to take in an image product as input and output (e.g., predict) an indication of a shape descriptor for one or more features of the image, such as a lymph node, a classification of the lymph node as belonging to a region, an assessment of the region, etc.

[0041] In some embodiments, the product associated with an image includes the image itself. For example, the image product associated with an image includes a portion of the image (e.g., all or a portion of the original image). In other words, in some embodiments, a model is used to predict shape descriptors of features in an image based on the image itself (e.g., taking the image as input). In other examples, the image product includes a portion of the image (e.g., all or a portion of the original image) that has been subjected to a pre-processing process. Examples of pre-processing processes include, but are not limited to, smoothing or filtering the image, for example, to accentuate features (or smooth background noise).

[0042] The trained model is trained to perform segmentation, contouring, classification, and / or evaluation based on training data having images of a subject with at least one annotation of a lymph node, a set of rules for determining the risk of the lymph node being malignant or whether the lymph node is malignant, and an annotation indicating at least one region.

[0043] There is also provided a method of training a machine learning model for use in processing images of a subject including lymph nodes, the method comprising obtaining training data, the training data comprising images of the subject with at least one annotation of lymph nodes, a set of rules for determining a risk of the lymph node being malignant or whether the lymph node is malignant, and annotations indicating at least one region, the method further comprising training a model based on the training data to evaluate regions of the image based on an assessment of the risk that the region includes a malignant lymph node.

[0044] There is further provided a computer program product including a computer readable medium having computer readable code embodied therein, the computer readable code being configured, upon execution by a suitable computer or processor, to cause the computer or processor to perform the methods described herein.

[0045] There is also provided a system for processing an image of a subject including a lymph node, the system comprising: a memory including instruction data representing a set of instructions; and a processor in communication with the memory and configured to execute the set of instructions, which when executed by the processor causes the processor to segment image data corresponding to the image, delineate lymph nodes from the segmented image data, classify the lymph nodes as belonging to a region, evaluate the region of the image based on an assessment of a risk that the region includes a malignant lymph node, and indicate the evaluation of the region.

[0046] Thus, by way of example, the present invention uses images such as volumetric PET and / or CT images as primary input data. The first part of the method comprises (automatic) lymph node segmentation, contouring, and region classification. A predictive model is trained on a set of data annotated for a specific task as shown in FIG.

[0047] FIG. 4 shows images of voxel-wise segmentation and contouring and region classification of lymph nodes using a trained model. In this example, the model is trained in a data-driven manner. The trained model takes a CT image volume as input and delivers as output a set of detected lymph nodes with associated voxel locations and the corresponding region class of each lymph node (visualized as a color-coded volume rendering, shown here as different cross-hatching). As shown in this figure, 4(a) shows an example of image data showing a subject's chest area (e.g., an area containing lymph nodes). 4(b) shows an example of a volume rendering of a subject voxel mask (e.g., an annotated image of lymph nodes and corresponding regions, where in this figure different regions are shown with different cross-hatching). FIG. 4(c) shows an example of a corresponding predicted mask rendering produced in this example by a volumetric neural network (a neural network trained with images such as the one in FIG. 4(b)). The legend shown in FIG. 4 indicates the different regions into which lymph nodes may be classified (e.g., level 1r, which corresponds to the regions shown in FIG. 1), and the color associated with each region (here shown as cross-hatching).

[0048] As discussed above, in a first part of the method, lymph nodes are segmented from image data corresponding to an image, lymph nodes are delineated from the segmented image data, and lymph nodes are classified as belonging to a region. The method is performed by a trained model. A single trained model may perform the entire method, or parts of the method may be performed by one or more trained models and other parts of the method may be performed by one or more trained models.

[0049] The trained model is trained using a set of annotated training examples that include pairs of CT image volumes of a subject / multiple subjects and corresponding lymph node (LN) annotations (see Fig. 4a, b). For example, the training data comprises images of a subject with at least one annotation of a lymph node, a set of rules for determining the risk of the lymph node being malignant or whether the lymph node is malignant, and an annotation indicating at least one region.

[0050] Using this training data, standard volumetric image segmentation or instance segmentation networks such as foveal networks (such as Brosch, T., & Saalbach, A., 2018, March Foveal fully convolutional nets for multi-organ segmentation, In Medical Imaging 2018: Image Processing (Vol. 10574, p. 105740U) International Society for Optics and Photonics), or Mask R-CNN (such as that presented in He, K., Gkioxari, G., Dollar, P., & Girshick, R. (2017), Mask R-CNN in Proceedings of the IEEE international conference on computer vision (p. 2961-2969)) are utilized as predictors. The tasks of lymph node segmentation, contouring, and region classification can be tackled by separate models or can be trained together via multi-task learning. The trained model or population of models assigns to each voxel in the image volume a probability that the voxel belongs to a particular lymph node and that it is within a particular nodal region (see FIG. 5).

[0051] Figure 5 shows a CT image volume (5(a)) used as input for a region classification model, where the model then delivers a probabilistic mask as output (5(b)). Thus, for each region (e.g., from "background" to "ax r" shown in Figure 5, which corresponds to the regions in Figure 4), the figure shows the probability that a voxel belongs to a particular lymph node, as well as the probability that the voxel is within the region. The lymph node probability map is thresholded to obtain a list of detected lymph nodes with associated voxel locations.

[0052] In a further part of the method, the method comprises evaluating regions of the image based on an assessment of the risk that the region contains a malignant lymph node. In particular, each detected lymph node within a certain region is evaluated with respect to at least one risk factor (e.g., risk feature) (e.g., lymph node volume, lymph node short axis length, lymph node long axis length, quantitative tissue size, positron emission tomography (PET) intensity, lymph node increased size, high metabolic activity, formation of connected lymph node clusters, regional lymphatic tissue volume, gray value statistics, etc.). The risk factors are region-dependent, whereby corresponding region-dependent risk boundaries are taken into account. For example, the corresponding region-dependent risk boundaries are obtained from the literature (or measurements on groups of healthy and unhealthy patients) and stored in a dictionary or look-up table. For example, for lung cancer, Mountain, CF, & Dresler, CM (1997). Regional lymph node classification for lung cancer staging. Chest, 111(6), 1718-1723 outlines regions for classification of lung cancer staging. The region(s) determined are predefined and depend on the type of cancer to be staged. Based on this dictionary and risk factors, a regional risk score is calculated for the lymph nodes determined to belong to the region (e.g., based on the number of increased lymph nodes in the region / the number of metabolically active lymph nodes in the region). The evaluating may further include comparing the regional risk assessment with a previously obtained regional risk assessment, for example, to determine whether the region has at least one of an increased risk compared to a previously determined risk and / or a high variability between the previous risk assessment and the current risk assessment compared to the previously determined risk.

[0053] The assessment of the region (or regions) is presented (e.g., to a user). This may take the form of per-region statistics (e.g., regional risk statistics, a sorted list starting with regions of increased risk, etc.). For example, regional statistics of risk scores across multiple exams may be provided to the user in the form of a sorted list (e.g., starting with regions of highest risk or the largest score change compared to previous exams). Such regions may alternatively or additionally be highlighted in the CT image volume using an overlaid heat map or color overlay (where different colors are shown as different cross-hatching), as shown in FIG. 6(b).

[0054] FIG. 6(a) shows detected lymph nodes (visualized via color coding, here shown as different cross-hatched areas (where the different cross-hatching indicates the different regions to which the lymph nodes belong)) that are assigned to region classes, where individual lymph nodes are further evaluated with respect to region-dependent risk markers such as nodal volume. As shown in FIG. 6(b), a heat map is used to highlight regions of increased risk, e.g., here, regions highlighted in red 612 (as shown here with different cross-hatching) represent "high risk", orange 616 represents "intermediate risk", yellow 614 represents "low risk", etc., based on the assessment of lymph nodes belonging to the region. Thus, as visualized in FIG. 6(b), the heat map informs the user of critical image areas. In Figure 6(c) particular lymph nodes within a region are highlighted that are lymph nodes that have a particular impact on the risk level of a region, or any potentially malignant lymph nodes within a region (again in this example lymph nodes are highlighted by their corresponding risk, where lymph nodes highlighted in red 622 represent "high risk", orange 626 represent "intermediate risk", and yellow 624 represent "low risk", in this example these colors are indicated by different cross-hatching). This allows easy inspection by the user of particularly at-risk lymph nodes.

[0055] Thus, the user may be informed which areas of the image volume are of particular interest and / or the order in which areas of the image volume should be considered (e.g., which areas of the image volume should be considered first.) Furthermore, these methods allow for robust comparison of region scores across multiple exams without the need for tracking of individual lymph nodes.

[0056] Alternatively, a predictive model can be trained to combine segmentation, contouring, classification, and scoring to predict one combined risk score (e.g., number of potentially malignant lymph nodes) per region.

[0057] The advantages associated with region-based assessment are that the need for more complex node-by-node tracking is negated by region-based evaluation, and furthermore, it allows for more robust progression monitoring with respect to imprecision in lymph node segmentation. Furthermore, lymph nodes with the greatest impact on the regional risk judgment can be displayed to the user (see Fig. 6(c)). The color-coded rendering (e.g., shown (as cross-hatching) in Fig. 6) is concatenated with the original image volume, so that a user interaction, such as a mouse click on the color-coded volume rendering, navigates to the corresponding location in the standard slice-by-slice viewport for further inspection, and vice versa. In the case of (semi-automatic) adaptation of lymph node masks, the region scores are updated as the user interacts with the image (or result, assessment). For example, if a user manually adds a lymph node or adjusts the shape of the current contouring result, the steps of classifying the lymph node as belonging to a predefined region, evaluating the region of the image based on an assessment of the risk that the region contains a malignant lymph node, and indicating the evaluation of the region are repeated.

[0058] In particular, in the case of enlarged lymph nodes and connected lymph node clusters, lymph nodes contact multiple nodal regions. Advantageously, the methods described herein allow the contribution of a lymph node to multiple regions to be calculated using a probability mask of the regions, which can be an alternative to assigning it to a specific regional class. Furthermore, given the location of the primary tumor, regional risk assessment allows for N and M stage classification, which requires examination of the spread of cancer to the lymphatic system on a regional basis.

[0059] In another embodiment, a computer program product is provided comprising a computer readable medium having computer readable code embodied therein, the computer readable code being configured, upon execution by a suitable computer or processor, to cause the computer or processor to perform one or more of the methods described herein.

[0060] It will therefore be understood that the present disclosure also applies to a computer program, in particular a computer program on or in a carrier adapted for implementing the embodiments. The program may be in the form of object code, such as source code, object code, code intermediate source and partially compiled form, or in any other form suitable for use in the implementation of the methods according to the embodiments described herein.

[0061] Moreover, it will be understood that such programs have many different architectural designs. For example, the program code implementing the functions of the method or system is subdivided into one or more subroutines. Many different ways of distributing functionality among these subroutines will be apparent to those skilled in the art. The subroutines are stored together in one executable file to form a self-contained program. Such an executable file includes computer executable instructions, such as processor instructions and / or interpreter instructions (e.g. Java interpreter instructions). Alternatively, one or more or all of the subroutines are stored in at least one external library file and linked, either statically or dynamically, e.g., at run-time, with the main program. The main program includes at least one call to at least one of the subroutines. The subroutines further include function calls to each other.

[0062] A carrier for a computer program is any entity or device capable of carrying the program. For example, the carrier includes a data storage such as a ROM, e.g. a CD-ROM or a semiconductor ROM, or a magnetic recording medium, e.g. a hard disk. Furthermore, the carrier is a transmissible carrier such as an electric or optical signal conveyed via an electric or optical cable or by radio or other means. When the program is embodied in such a signal, the carrier is constituted by such a cable or other device or means. Alternatively, the carrier is an integrated circuit in which the program is embodied, the integrated circuit being adapted to perform or used in the performance of the relevant method.

[0063] Variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the principles and techniques described herein, from a study of the drawings, the disclosure, and the appended claims. In the claims, the term "comprises" does not exclude other elements or steps, and the singular elements do not exclude a plurality. A single processor or other unit fulfills the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage. A computer program can be stored or distributed on a suitable medium, such as an optical storage medium or a solid-state medium provided together with or as part of other hardware, but also distributed in other forms, such as via the Internet or other wired or wireless communication systems. Any reference signs in the claims should not be interpreted as limiting the scope.

Claims

1. 1. A computer-implemented method for processing an image of a subject including lymph nodes, the method comprising: segmenting lymph nodes in image data corresponding to said image; delineating lymph nodes from the segmented image data; classifying lymph nodes as belonging to an anatomical nodal region; evaluating regions of the image based on an assessment of the risk that the anatomic nodal region contains a malignant lymph node; and indicating the assessment of the region.

2. The computer-implemented method of claim 1 , wherein the region is evaluated with respect to at least one risk factor.

3. 3. The computer-implemented method of claim 2, wherein the risk factors include one of lymph node volume, lymph node short axis length, lymph node long axis length, quantitative tissue size, positron emission tomography intensity, increased lymph node size, high metabolic activity, formation of connected lymph node clusters, regional lymphatic tissue volume, and gray value statistics.

4. 4. The computer-implemented method of claim 1, wherein the evaluation comprises evaluating a plurality of lymph nodes for at least one risk factor and evaluating the region based on an evaluation of at least one of the plurality of lymph nodes corresponding to the region.

5. The computer-implemented method of claim 2 , wherein the risk factors are region-dependent.

6. 2. The computer-implemented method of claim 1, wherein the step of presenting the assessment comprises outputting at least one of risk statistics for the region, a color overlay corresponding to the image showing the risk associated with the region, a heat map corresponding to the image showing the risk associated with the region, and an overlay on potentially malignant lymph nodes.

7. 10. The computer-implemented method of claim 1, wherein the evaluating step further comprises comparing an assessment of risk in an area with a previously obtained assessment of risk in the area.

8. 8. The computer-implemented method of claim 7, wherein the evaluating step further comprises determining whether the region has at least one of an increased risk compared to a previously determined risk and a high variability between previous risk assessments and current risk assessments compared to the previously determined risk.

9. The computer-implemented method of claim 1 , wherein the evaluation is performed for multiple regions of the image.

10. 10. The computer-implemented method of claim 9, wherein the indication of the assessment includes at least one of area risk statistics, a list of areas with an indication of the assessment of each area, a list of areas ordered based on the assessment of each area, and a heat map corresponding to the assessment of each area.

11. The computer-implemented method of claim 1 , wherein the evaluation is performed by a trained model.

12. 12. The computer-implemented method of claim 11, wherein the trained model is trained to make the assessment based on training data comprising images of a subject with at least one annotation of a lymph node, a set of rules for determining the risk of the lymph node being malignant or whether the lymph node is malignant, and an annotation indicating at least one region.

13. 1. A method of training a machine learning model for use in processing images of a subject including lymph nodes, the method comprising: acquiring training data, the training data comprising images of a subject with at least one annotation of a lymph node, a set of rules for determining the risk of the lymph node being malignant or whether the lymph node is malignant, and an annotation indicating at least one anatomical nodal region; The method further comprises training the machine learning model based on the training data to evaluate regions of the image based on an assessment of risk of the anatomical node region containing a malignant lymph node.

14. A computer-readable medium having computer-readable code embodied therein, the computer-readable code being configured, when executed by a suitable computer or processor, to cause the computer or processor to perform the method of claim 1.

15. 1. A system for processing an image of a subject including a lymph node, the system comprising: a memory containing instruction data representing a set of instructions; a processor in communication with the memory and configured to execute the set of instructions, the set of instructions, when executed by the processor, causing the processor to: segmenting lymph nodes in image data corresponding to said image; delineating lymph nodes from the segmented image data; Classifying lymph nodes as belonging to an anatomical nodal region; evaluating regions of the image based on an assessment of the risk that the anatomic nodal region contains a malignant lymph node; and indicating the assessment of the anatomical nodal region.

Images