Machine learning-based longitudinal analysis of positron emission tomography and computed tomography scans to evaluate disease progression and treatment response.

Abstract

The method may include determining a first tumor mask corresponding to a first lesion present in a first PET scan and a first computed tomography (CT) scan, based on a first positron emission tomography (PET) scan and a first computed tomography (CT) scan from a first time point. A second tumor mask corresponding to a second lesion present in a second PET scan and a second CT scan may be determined based on a second PET scan and a second CT scan from a second time point. A longitudinal segmentation model may be applied to update the first and second tumor masks, respectively, based on the first PET scan, the first CT scan, the second PET scan, and the second CT scan. Based on at least one of the first updated tumor mask and the second updated tumor mask, the response to disease treatment may be determined. Related systems and computer program products are also provided.

Description

[Technical Field]

[0001] Cross-reference of related applications This application claims priority to U.S. Provisional Application No. 63 / 497,660, filed on 21 April 2023, entitled "MACHINE LEARNING ENABLED LONGITUDINAL ANALYSIS OF POSITRON EMISSION TOMOGRAPHY AND COMPUTED TOMOGRAPHY SCANS FOR ASSESSMENT OF DISEASE PROGRESSION AND TREATMENT RESPONSE," the disclosure thereof incorporated herein by reference in its entirety.

[0002] The subject matter described herein generally relates to machine learning, and more specifically, to machine learning-based techniques for evaluating disease progression and treatment response based on positron emission tomography (PET) and computed tomography (CT) scans. [Background technology]

[0003] Medical imaging refers to the techniques and processes for obtaining data that characterizes the internal anatomical form and pathophysiology of a subject, including images created by detecting radiation passing through the body (e.g., X-rays) or radiation emitted by administered radiopharmaceuticals (e.g., gamma rays from intravenously administered radiotracers). By revealing internal anatomical structures obscured by other tissues such as skin, subcutaneous fat, and bone, medical imaging is essential for numerous medical diagnoses and / or treatments. Examples of medical imaging modalities include two-dimensional imaging such as X-ray plain film, bone scintigraphy, and thermography. Examples of three-dimensional imaging modalities include magnetic resonance imaging (MRI), computed tomography (CT), cardiac sestamivi scanning, and positron emission tomography (PET). [Overview of the Initiative]

[0004] Systems, methods, and products including computer program products are provided for machine learning-based longitudinal analysis of positron emission tomography (PET) and computed tomography (CT) scans for evaluating disease progression and treatment response. Embodiments of this subject matter include, but are not limited to, methods according to the descriptions provided herein, and articles comprising a tangibly embodied machine-readable medium capable of operating one or more machines (e.g., computers) to cause operations that implement one or more of the features described herein. Similarly, computer systems are also described, which may include one or more processors and one or more memories coupled to the one or more processors. Memories, which may include non-temporary computer-readable or machine-readable storage media, may encode, store, etc., one or more programs that cause one or more processors to perform one or more of the operations described herein. Computer implementations of one or more embodiments of this subject matter may be implemented by one or more data processors in a single computing system or in multiple computing systems. Such multiple computing systems may be connected, for example, via one or more connections including connections through a network (e.g., the Internet, a wireless wide area network, a local area network, a wide area network, a wired network, etc.), or via direct connections between one or more of the multiple computing systems, and may exchange data and / or instructions or other commands.

[0005] In one embodiment, a system is provided for machine learning-based longitudinal analysis of positron emission tomography (PET) and computed tomography (CT) scans to evaluate disease progression and treatment response. The system may include at least one processor and at least one memory. The at least one memory may include program code that, when executed by at least one processor, brings about an operation. This operation may include at least determining a first tumor mask corresponding to a first lesion present in a first PET scan and a first CT scan, based on a first positron emission tomography (PET) scan and a first computed tomography (CT) scan from a first time point; at least determining a second tumor mask corresponding to a second lesion present in a second PET scan and a second CT scan, based on a second PET scan and a second CT scan from a second time point; at least applying a longitudinal segmentation model to update the first tumor mask and the second tumor mask, respectively, based on the first PET scan, the first CT scan, the second PET scan, and the second CT scan; and determining the response of the disease to treatment based on at least one of the first updated tumor mask and the second updated tumor mask.

[0006] In another embodiment, a method is provided for machine learning-based longitudinal analysis of positron emission tomography (PET) and computed tomography (CT) scans to evaluate disease progression and treatment response. The method may include at least determining a first tumor mask corresponding to a first lesion present in a first PET scan and a first CT scan based on a first positron emission tomography (PET) scan and a first computed tomography (CT) scan from a first time point; determining a second tumor mask corresponding to a second lesion present in a second PET scan and a second CT scan based on a second PET scan and a second CT scan from a second time point; applying a longitudinal segmentation model to update each of the first and second tumor masks based on the first PET scan, a first CT scan, a second PET scan, and a second CT scan; and determining the response of the disease to treatment based on at least one of the first updated tumor mask and the second updated tumor mask.

[0007] In another embodiment, a computer program product is provided for machine learning-based longitudinal analysis of positron emission tomography (PET) and computed tomography (CT) scans to evaluate disease progression and response to treatment. The computer program product may include a non-temporary computer-readable medium that stores instructions that cause an action when executed by at least one data processor. This action may include at least determining a first tumor mask corresponding to a first lesion present in a first PET scan and a first CT scan, based on a first positron emission tomography (PET) scan and a first computed tomography (CT) scan from a first time point; at least determining a second tumor mask corresponding to a second lesion present in a second PET scan and a second CT scan, based on a second PET scan and a second CT scan from a second time point; at least applying a longitudinal segmentation model to update the first tumor mask and the second tumor mask, respectively, based on the first PET scan, the first CT scan, the second PET scan, and the second CT scan; and determining the response to treatment of the disease based on at least one of the first updated tumor mask and the second updated tumor mask.

[0008] In some variations of the method, system, and non-temporary computer-readable medium, one or more of the following features may be included, at will, in any feasible combination:

[0009] In some variations, the method may determine a first tumor mask corresponding to a first lesion present in a first PET scan and a first CT scan, based at least on a first positron emission tomography (PET) scan and a first computed tomography (CT) scan from a first time point; determine a second tumor mask corresponding to a second lesion present in a second PET scan and a second CT scan, based at least on a second PET scan and a second CT scan from a second time point; apply a longitudinal segmentation model to update each of the first and second tumor masks based at least on the first PET scan, the first CT scan, the second PET scan, and the second CT scan; and determine the response to disease treatment based on at least one of the first updated tumor mask and the second updated tumor mask.

[0010] In some modifications, a first tumor mask may identify a first set of pixels in each of a first PET scan and a first CT scan depicting a first lesion, and a second tumor mask may identify a set of pixels from a second PET scan and a second CT scan depicting a second lesion.

[0011] In some variations, the method may register a first CT scan, a first PET scan, a second CT scan, and a second PET scan in order to align the first CT scan and the first PET scan with the second CT scan and the second PET scan.

[0012] In some modifications, the method can identify a second lesion as a new lesion based on at least a first updated tumor mask and a second updated tumor mask, and in response to the identification of the second lesion as a new lesion, the response to treatment can be determined as progressive disease (PMD).

[0013] In some variations, the method may, at least, determine the distance between a first lesion and a second lesion based on a first updated tumor mask and a second updated tumor mask; identify the second lesion as a new lesion based at least on the distance between the first lesion and the second lesion meeting one or more thresholds; and identify the second lesion as the same lesion as the first lesion based at least on the distance between the first lesion and the second lesion not meeting one or more thresholds.

[0014] In some variations, the method may determine the response to treatment of the disease, at least based on the change in metabolic activity indicated by the lesion between the first and second time points, in response to determining that the first and second lesions are the same lesion.

[0015] In some variations, the change in metabolic activity between a first and second time point can be determined by determining a first level of metabolic activity indicated by the lesion at the first time point based on at least a first updated tumor mask and a first PET scan, determining a second level of metabolic activity indicated by the lesion at the second time point based on at least a second updated tumor mask and a second PET scan, and determining the change in metabolic activity between the first and second time points based on at least the first and second levels of metabolic activity.

[0016] In some variations, the method determines whether a patient has a progressive metabolic disorder (PMD) based at least on changes in metabolic activity between a first time point and a second time point in which the response to treatment meets a first threshold.

[0017] In some variations, the response to treatment may be determined as no metabolic response (NMR) based on changes in metabolic activity between at least the first and second time points.

[0018] In some variations, the response to treatment can be determined as a partial metabolic response (PMR) based at least on the change in metabolic activity between the first time point and the second time point not meeting the first threshold and the second threshold.

[0019] In some variations, the first level of metabolic activity can correspond to a first standardized uptake value (SUV), and the second level of metabolic activity can correspond to a second standardized uptake (SUV) value.

[0020] In some variations, each of the first metabolic activity level and the second metabolic activity level can correspond to the maximum value, minimum value, median value, average value, or most frequent value of the metabolic activity indicated by the lesion at the corresponding time point.

[0021] In some variations, a first CT scan and a first PET scan can be performed before treatment of the disease, and a second CT scan and a second PET scan can be performed after treatment of the disease.

[0022] In some variations, the method can determine the change in tumor volume based at least on the first updated tumor mask and the second updated tumor mask, and can determine the response to treatment of the disease based at least on the change in tumor volume.

[0023] In some variations, the method can determine the variance of the first change in metabolic activity and / or the second change in tumor volume between the first time point and the second time point across various lesions based at least on the first updated tumor mask and the second updated tumor mask, and can determine the response to treatment based at least on the variance of the change in metabolic activity and / or tumor volume between the first time point and the second time point indicated by various lesions.

[0024] In some variations, the method can determine the progression of the disease based at least on the first updated tumor mask and the second updated tumor mask.

[0025] In some variations, the first tumor mask may be determined by applying the segmentation model to the first PET scan and the first CT scan, and the second tumor mask may be determined by applying the segmentation model to the second PET scan and the second CT scan.

[0026] In some variations, the longitudinal segmentation model may be an artificial neural network or a visual transducer.

[0027] In some variations, each of the first CT scan, first PET scan, second CT scan, and second PET scan may be a three-dimensional volume containing multiple two-dimensional patches.

[0028] In some variations, each pixel in the first and second PET scans may be associated with an intensity value corresponding to the level of metabolic activity.

[0029] In some variations, each pixel in the first and second CT scans may be associated with an intensity value corresponding to tissue density or X-ray attenuation.

[0030] In some variations, the method can train a longitudinal segmentation model to update two or more tumor masks, each of which is generated from positron emission tomography (PET) and computed tomography (CT) scans from a single time point.

[0031] In some variations, the response to treatment for a disease may be a complete metabolic response (CMR) or an incomplete metabolic response (non-CMR).

[0032] In some variations, the response to treatment for the disease may be either a responder or a non-responder.

[0033] In some variations, the response to disease treatment may be a total metabolic response (CMR), a partial metabolic response (PMR), no metabolic response (NMR), or a progressive metabolic disease (PMD).

[0034] In some variations, the method may extract a first patch containing a first lesion associated with a first tumor mask from a first PET scan and a first CT scan, extract a second patch containing a second lesion associated with a second tumor mask from a second PET scan and a first CT scan, and apply a longitudinal segmentation model to the first and second patches to update the first and second tumor masks, respectively.

[0035] Details of one or more variations of the subject matter described herein are described in the accompanying drawings and the following description. Other features and advantages of the subject matter described herein will become apparent from the description and drawings, as well as from the claims. Certain features of the subject matter of this disclosure are described for illustrative purposes in relation to fluorodeoxyglucose-dependent (FDG-dependent) cancers, such as certain types of non-Hodgkin lymphoma (NHL), but it should be readily understood that such features are not limiting. The claims following this disclosure define the scope of the subject matter protected. [Brief explanation of the drawing]

[0036] The accompanying drawings incorporated herein and constituting part of this specification illustrate specific aspects of the subject matter disclosed herein and, together with the description, help to illustrate some of the principles relating to the disclosed embodiments. In the drawings,

[0037] [Figure 1] The diagram shows an example of a machine learning-based medical imaging analysis system, illustrating several exemplary embodiments.

[0038] [Figure 2]A schematic diagram is shown illustrating an example of a process for machine learning-based longitudinal analysis of positron emission tomography (PET) and computed tomography (CT) scans, based on several exemplary embodiments.

[0039] [Figure 3] A flowchart illustrating an example process for machine learning-based longitudinal analysis of positron emission tomography (PET) and computed tomography (CT) scans, based on several exemplary embodiments, is provided.

[0040] [Figure 4] A flowchart illustrating another example of a process for machine learning-based longitudinal analysis of positron emission tomography (PET) and computed tomography (CT) scans is shown, based on several exemplary embodiments.

[0041] [Figure 5] A flowchart illustrating another example of a process for machine learning-based longitudinal analysis of positron emission tomography (PET) and computed tomography (CT) scans is shown, based on several exemplary embodiments.

[0042] [Figure 6] This paper presents a comparison of the accuracy of total metabolic response (CMR) and objective response (OR) assessments performed by machine learning-based longitudinal analysis and inter-radiologist agreement across different clinical datasets, using several exemplary embodiments.

[0043] [Figure 7] Several exemplary embodiments demonstrate a comparison of longitudinal analysis using machine learning and the capabilities of professional radiologists.

[0044] [Figure 8A] This paper presents a comparison of the accuracy of total metabolic response (CMR) and objective response (OR) assessments performed by machine learning-based longitudinal analysis and interradiologist agreement in several exemplary embodiments.

[0045] [Figure 8B] This paper presents a comparison of the accuracy of response assessments performed by machine learning-based longitudinal analysis and interradiologist agreement in several exemplary embodiments.

[0046] [Figure 8C] This paper presents a comparison of F1 scores from machine learning-based longitudinal analysis and radiologist-based analysis using several exemplary embodiments.

[0047] [Figure 8D] This paper presents several exemplary embodiments of progression-free survival (PFS) to the end of treatment evaluation performed by machine learning-based longitudinal analysis across different clinical datasets, as well as PFS evaluation to the end of treatment performed by a specialist radiology committee.

[0048] [Figure 8E] This paper presents several exemplary embodiments comparing the accuracy of longitudinal analysis using machine learning and assessments of total metabolic response (CMR), objective response (OR), and four categories of assessments performed by expert radiologists.

[0049] [Figure 8F] This paper presents a comparison of the accuracy of longitudinal analysis using machine learning versus analysis by a professional radiologist, using several exemplary embodiments.

[0050] [Figure 8G] This document presents several exemplary embodiments of a comparison of overall (OS) up to the end of treatment evaluation, performed by machine learning-based longitudinal analysis and determined responses across different clinical datasets.

[0051] [Figure 9] This shows overall survival (OS) to treatment completion and early discontinuation as determined by machine learning longitudinal analysis across various clinical datasets in several exemplary embodiments.

[0052] [Figure 10] This paper presents a comparison of Deauville score (DS) assessments performed by machine learning-based longitudinal analysis and determined responses across different clinical datasets, using several exemplary embodiments.

[0053] [Figure 11] Examples of true positive, true negative, false positive, and false negative cases are shown.

[0054] [Figure 12] This diagram depicts a block diagram illustrating an example of a computing system in several exemplary embodiments.

[0055] The disclosure of "Materials," and its contents, are incorporated herein by reference in their entirety.

[0056] In practical terms, similar reference numbers indicate similar structures, features, or elements. [Modes for carrying out the invention]

[0057] Various medical imaging modalities can be applied to obtain data characterizing the anatomical structures and pathophysiology of a patient's body. Computed tomography (CT) is an example of a three-dimensional imaging modality that captures a series of X-rays to create cross-sectional images (e.g., patches, slices, etc.) of bone, blood vessels, and soft tissues within the body. A computed tomography scan can be a three-dimensional volume formed by a series of two-dimensional images, where each pixel is associated with an intensity value indicating tissue density or X-ray attenuation at a corresponding location in the body of the subject. Another example of a three-dimensional imaging modality is positron emission tomography (PET), which captures radioactive signals indicating cellular metabolic activity within the body of the subject. A positron emission tomography scan can be a three-dimensional volume formed by a series of two-dimensional images, where each pixel is associated with an intensity value indicating the level of cellular metabolic activity (e.g., glucose uptake) at a corresponding location in the body of the subject. In some cases, a single gantry incorporating both a positron emission tomography (PET) scanner and a computed tomography (CT) scanner may be able to acquire both PET and CT scans during the same session. The acquired PET and CT scans can be combined into a single superimposed (e.g., jointly aligned) image (e.g., a PET-CT scan) in which the spatial distribution of metabolic activity shown in the PET scan is aligned with the anatomical structures shown in the CT scan.

[0058] In some exemplary embodiments, the analysis controller may perform longitudinal analysis of positron emission tomography (PET) and computed tomography (CT) scans to assess disease progression and treatment response. In some cases, the analysis controller may apply a longitudinal segmentation model trained to update two or more individual tumor masks generated from positron emission tomography (PET) and computed tomography (CT) scans from individual time points. For example, the longitudinal segmentation model may be determined based on a first positron emission tomography (PET) scan and a first computed tomography (CT) scan from a first time point and incorporate a first tumor mask corresponding to a first lesion present in the first positron emission tomography (PET) scan and the first computed tomography (CT) scan. Furthermore, the longitudinal segmentation model may be determined based on a second positron emission tomography (PET) scan and a second computed tomography (CT) scan from a second time point and incorporate a second tumor mask corresponding to a second lesion present in the second positron emission tomography (PET) scan and the second computed tomography (CT) scan. The longitudinal segmentation model can update the first tumor mask and the second tumor mask based on the first positron emission tomography (PET) scan, the first computed tomography (CT) scan, the second positron emission tomography (PET) scan, and the second computed tomography (CT) scan, respectively. In doing so, the longitudinal segmentation model can improve the first and second tumor masks to reduce false positives, in which one or more pixels that are not part of the lesion are thus incorrectly identified.

[0059] In some exemplary embodiments, the analysis controller may determine the response to disease treatment (e.g., total metabolic response (CMR), objective response (OR), four-category assessment, etc.) based on updated tumor masks generated by at least a longitudinal segmentation model. Alternatively and / or additionally, the analysis controller may determine disease progression based on updated tumor masks generated by at least a longitudinal segmentation model. For example, in some cases, the analysis controller may determine the response to disease treatment associated with a first lesion and a second lesion based on at least a first and second updated tumor mask. In some cases, the analysis controller may determine the response to disease treatment as progressive metabolic disease (PMD), no metabolic response (NMR), partial metabolic response (PMR), or total metabolic response (CMR) based on at least a first and second updated tumor mask. Alternatively, in some cases, the analysis controller may determine the response to disease treatment as total metabolic response (CMR) or incomplete metabolic response (non-CMR) based on at least a first and second updated tumor mask.

[0060] In some exemplary embodiments, the analysis controller may, based on at least a first updated tumor mask and a second updated tumor mask, identify a second lesion present in a second positron emission tomography (PET) scan and a second computed tomography (CT) scan as a new lesion not present in the first PET scan and the first CT scan. For example, in some cases, the analysis controller may determine that the second lesion is a new lesion if the distance between the first and second lesions (e.g., minimum distance, average distance, etc.) meets one or more thresholds (e.g., minimum distance greater than 10 millimeters). Otherwise, if the distance between the first and second lesions does not meet one or more thresholds, the analysis controller may determine that the first and second lesions are the same lesion. Therefore, if the second lesion is identified as a new lesion, the analysis controller may determine the response to disease treatment as a progressive metabolic response (PMR). If the second lesion is identified as the same lesion as the first, the analysis controller may further determine the response to treatment of the disease based on the change in the level of metabolic activity indicated by the lesion between the first and second time points. For example, the analysis controller may determine the response to treatment of the disease as progressive metabolic disease (PMD), where the change in metabolic activity between the first and second time points meets the first threshold; no metabolic response (NMR), where the change in metabolic activity meets the second threshold but not the first threshold; and partial metabolic response (PMR), where the change in metabolic activity does not meet either the first or second threshold.

[0061] In some exemplary embodiments, the analysis controller may determine, based on at least a first updated tumor mask and a second updated tumor mask, the change in the level of metabolic activity indicated by the lesion between a first and second time point. The change in the level of metabolic activity may correspond to changes in various metrics derived based on the updated tumor masks. Examples of such metrics include standard uptake values ​​(e.g., maximum standard uptake, minimum standard uptake, median standard uptake, mean standard uptake, mode standard uptake, etc.) and lesion size. For example, in some cases, the change in the level of metabolic activity indicated by the lesion between a first and second time point may correspond to the difference between a first maximum level of metabolic activity at the first time point and a second maximum level of metabolic activity at the second time point. Thus, in some cases, the analysis controller may determine, based on at least the first updated tumor mask, the first maximum level of metabolic activity at the first time point (e.g., the first maximum standard uptake (SUV)). max Furthermore, the analysis controller can determine the second maximum level of metabolic activity at the second time point (e.g., the second maximum standard uptake value (SUV)) based on at least a second updated tumor mask. max )) can be determined. As described above, in some cases in which the analysis controller could not identify a new lesion, the analysis controller may determine the response to treatment based on whether the change in the level of metabolic activity exhibited by the lesion between the first and second time points meets one or more thresholds.

[0062] In some exemplary embodiments, the analysis controller may also determine the change in tumor volume (e.g., total metabolic tumor volume (TMTV)) between a first and second time point, based on at least a first updated tumor mask and a second updated tumor mask. In some cases, the response to disease treatment may be determined based on the change in tumor volume between the first and second time point. Alternatively and / or additionally, the analysis controller may determine the variance of the first change in metabolic activity and / or the second change in tumor volume between the first and second time point across various lesions, based on at least a first updated tumor mask and a second updated tumor mask. In some cases, the analysis controller may further determine the response to disease treatment based on the variance of the change in metabolic activity and / or tumor volume between the first and second time point across various lesions.

[0063] Figure 1 shows a diagram of a system illustrating an example of a machine learning-based medical imaging analysis system 100 according to several exemplary embodiments. Referring to Figure 1, the machine learning-based medical imaging analysis system 100 may include an analysis controller 110, one or more imaging devices 120, and a client device 130. As shown in Figure 1, the analysis controller 110, one or more imaging devices 120, and the client device 130 may be communicably coupled via a network 140. One or more imaging devices 120 may include, for example, a computed tomography (CT) scanner 121 and a positron emission tomography (PET) scanner 123. The client device 130 may be a processor-based device, including, for example, a smartphone, a tablet computer, a wearable device, a virtual assistant, or an Internet of Things (IoT) device. The network 140 may be a wired network and / or a wireless network, including, for example, a wide area network (WAN), a local area network (LAN), a virtual local area network (VLAN), a public land mobile network (PLMN), or the internet.

[0064] In some exemplary embodiments, the analysis controller 110 may perform longitudinal analysis of positron emission tomography (PET) and computed tomography (CT) scans generated by one or more imaging devices 120 (e.g., computed tomography scanner 121, positron emission tomography (PET) scanner 123, etc.). In some cases, the analysis controller 110 may apply a longitudinal segmentation model 113 that can be trained to update two or more individual tumor masks generated from positron emission tomography (PET) and computed tomography (CT) scans from individual time points in time. For further explanation, Figure 2 shows a schematic diagram illustrating an example of a process 200 for machine learning-based longitudinal analysis of positron emission tomography (PET) and computed tomography (CT) scans according to some exemplary embodiments.

[0065] Referring to Figure 2, the analysis controller 110 may receive a first positron emission tomography (PET) scan 210a and a first computed tomography (CT) scan 220a from a first time point from one or more imaging devices 120. Furthermore, the analysis controller 110 may also receive a second positron emission tomography (PET) scan 210b and a second computed tomography (CT) scan 220b from a second time point from one or more imaging devices 120. In some cases, the analysis controller 110 may include a preprocessing controller 110. In the example shown in Figure 2, the preprocessing controller 110 may preprocess the first positron emission tomography (PET) scan 210a and the first computed tomography (CT) scan 220a to generate a first tumor mask 230a. Furthermore, the preprocessing controller 110 may also preprocess the second positron emission tomography (PET) scan 210b and the second computed tomography (CT) scan 220b to generate a second tumor mask 230b.

[0066] In some exemplary embodiments, preprocessing may include alignment to align the first positron emission tomography (PET) scan 210a and the first computed tomography (CT) scan 220a with the second positron emission tomography (PET) scan 210b and the second computed tomography (CT) scan 220b. For example, in some cases, the preprocessing engine 111 may perform affine and block matching alignment based on the first computed tomography (CT) scan 220a from a first time point and the second computed tomography (CT) scan 220b from a second time point. Thus, the first positron emission tomography (PET) scan 210a may be superimposed (or aligned together) with the first computed tomography (CT) scan 220a, and the second positron emission tomography (PET) scan 210b may be superimposed (or aligned together) with the second computed tomography (CT) scan 220b. Furthermore, the first positron emission tomography (PET) scan 210a and the first computed tomography (CT) scan 220a from the first time point can be superimposed (or aligned together) with the second positron emission tomography (PET) scan 210b and the second computed tomography (CT) scan 220b. In this case, the preprocessing engine 111 can map the first pixel of the first positron emission tomography (PET) scan 210a to the second pixel of the first computed tomography (CT) scan 220a, the third pixel of the second positron emission tomography (PET) scan 210b, and the fourth pixel of the second computed tomography (CT) scan 220b.

[0067] In some exemplary embodiments, preprocessing may also include dividing the first positron emission tomography (PET) scan 210a, the first computed tomography (CT) scan 220a, the second positron emission tomography (PET) scan 210b, and the second computed tomography (CT) scan 220b into two or more regions. For example, in some cases, the preprocessing engine 111 may divide the first positron emission tomography (PET) scan 210a, the first computed tomography (CT) scan 220a, the second positron emission tomography (PET) scan 210b, and the second computed tomography (CT) scan 220b into two or more anatomical regions based on one or more anatomical landmarks. Examples of anatomical regions include the head and neck region, the chest region, and the abdominal and pelvic regions.

[0068] In some exemplary embodiments, preprocessing may further include segmenting the first positron emission tomography (PET) scan 210a and the first computed tomography (CT) scan 220a to generate a first tumor mask 230a, and segmenting the second positron emission tomography (PET) scan 210b and the second computed tomography (CT) scan 220b to generate a second tumor mask 230b. For example, in some cases, the preprocessing engine 111 may apply a segmentation model 112 to segment the first positron emission tomography (PET) scan 210a and the first computed tomography (CT) scan 220a by identifying at least a first plurality of pixels corresponding to one or more lesions present in the first positron emission tomography (PET) scan 210a and the first computed tomography (CT) scan 220a. Similarly, the preprocessing engine 111 may segment the second positron emission tomography (PET) scan 210b and the second computed tomography (CT) scan 220b by applying a segmentation model 112 to identify at least a second set of pixels corresponding to one or more lesions present in the second positron emission tomography (PET) scan 210b and the second computed tomography (CT) scan 220b. As described above, each pixel in the positron emission tomography (PET) scan may be associated with an intensity value corresponding to a level of metabolic activity (e.g., standardized UV) and each corresponding pixel in the together aligned computed tomography (CT) scan may be associated with an intensity value corresponding to tissue density or X-ray attenuation. Thus, in some cases, the aforementioned segmentation model may determine whether a pixel is part of a lesion based on the level of metabolic activity and tissue density (or X-ray attenuation) indicated by the pixel and one or more adjacent pixels.

[0069] In some exemplary embodiments, the segmentation model 112 may include one or more machine learning models trained to segment two-dimensional images and / or three-dimensional volumes. For example, in some cases, the segmentation model 112 may include one or more artificial neural networks, such as convolutional neural networks, visual transducers, etc. If the segmentation model 112 includes one or more visual transducers, the segmentation model 112 may be applied to identify individual patches containing lesions, and then thresholds may be applied to those patches to select pixels where the level of metabolic activity (e.g., standardized uptake (SUV)), tissue density, and / or X-ray attenuation satisfy one or more thresholds. For example, the standardized uptake (SUV) of a pixel exceeds a certain minimum value (e.g., 2.5 or 4), a certain minimum value relative to the standardized uptake (SUV) of the liver (e.g., 1.5 times the standardized uptake (SUV) of the liver + 2 standard deviations from the standardized uptake (SUV) of the liver), the maximum standardized uptake (SUV) of the identified tumor region. max If the percentage exceeds, for example, a pixel can be identified as depicting a lesion.

[0070] In some cases, the segmentation model 112 may perform segmentation via object classification. In those cases, the segmentation model 112 may include one or more machine learning models (e.g., logistic regression models, tree-based classifiers, fully connected neural networks, etc.) trained to perform object classification. For example, to segment the first positron emission tomography (PET) scan 210a and the first computed tomography (CT) scan 220a, the preprocessing engine 111 may first apply thresholds to identify one or more objects present in the first positron emission tomography (PET) scan 210a and the first computed tomography (CT) scan 220a. For example, in some cases, the preprocessing engine 111 may identify pixels that depict objects in the first positron emission tomography (PET) scan 210a and the first computed tomography (CT) scan 220a based on at least the intensity value of each pixel. As described above, the intensity value of each pixel in a computed tomography (CT) scan corresponds to tissue density or X-ray attenuation, while the intensity value of a pixel in a positron emission tomography (PET) scan corresponds to the level of metabolic activity. Therefore, in some cases, the preprocessing controller 110 can identify objects present in the first positron emission tomography (PET) scan 210a and the first computed tomography (CT) scan 220a by applying a first threshold to the intensity value of each pixel in the first positron emission tomography (PET) scan 210a and / or by applying a second threshold to the intensity value of each pixel in the first computed tomography (CT) scan 210b. In this case, the preprocessing engine 111 can identify objects that exhibit threshold levels of metabolic activity, threshold levels of tissue density, and / or threshold levels of X-ray attenuation. Once objects present in the first positron emission tomography (PET) scan 210a and the first computed tomography (CT) scan 220a are identified, the preprocessing engine 111 may apply a segmentation model 112 to classify each object as either a lesion or a non-lesion.

[0071] Referring again to Figure 2, in some exemplary embodiments, the analysis controller 110 may apply a longitudinal segmentation model 113 to update the first tumor mask 230a and the second tumor mask 230b. For example, in some cases, the first patch may be extracted from a first positron emission tomography (PET) scan 210a and a first computed tomography (CT) scan 220a and include a first lesion associated with the first tumor mask 230a, while the second patch may be extracted from a second positron emission tomography (PET) scan 210b and a second computed tomography (CT) scan 220b and include a second lesion associated with the second tumor mask 230b. In some cases, the longitudinal segmentation model 113 may incorporate the first and second patches together in the first positron emission tomography (PET) scan 210a, the first computed tomography (CT) scan 220a, the second positron emission tomography (PET) scan 210b, and the second computed tomography (CT) scan 220b, in order to increase the signal-to-noise ratio (SNR) associated with the pixels depicting the first and second lesions.

[0072] In some exemplary embodiments, the longitudinal segmentation model 113 may update the first tumor mask 230a and the second tumor mask 230b based on at least a first and a second patch to generate the first updated tumor mask 240a and the second updated tumor mask 240b. For example, the longitudinal segmentation model 113 may, in some cases, determine whether a pixel is part of a lesion based on the level of metabolic activity and tissue density (or X-ray attenuation) indicated by the pixel and one or more adjacent pixels across a first and a second time point. In doing so, the longitudinal segmentation model 113 may update the first tumor mask 230a and / or the second tumor mask 230b, for example, by updating the labels assigned to one or more pixels within the first tumor mask 230a and / or the second tumor mask 230b. For example, pixels previously classified as part of a lesion (e.g., by a segmentation model) may be reclassified as not part of a lesion by the longitudinal segmentation model 113, and pixels previously classified as not part of a lesion (e.g., by a segmentation model) may be reclassified as part of a lesion by the longitudinal segmentation model 113.

[0073] Referring again to Figure 2, the analysis controller 110 may include an evaluation engine 115 that determines the treatment response 180 based on at least a first updated tumor mask 240a and a second updated tumor mask 240b. For example, the treatment response 180 may be a complete metabolic response (CMR) if lesions present in the first updated tumor mask 240a are not present in the second updated tumor mask 240b, and there are no new lesions in the second updated tumor mask 240b. The treatment response 180 may be a partial metabolic response (PMR) if some, but not all, of the lesions present in the first updated tumor mask 240a are present in the second updated tumor mask 240b, and there are no new lesions in the second updated tumor mask 240b. The treatment response 180 may be a progressive metabolic disease (PMD) if the second updated tumor mask 240b includes one or more new lesions. Even if no new lesions are present in the second updated tumor mask 240b, if the changes in metabolic activity levels indicated by the lesions in the first updated tumor mask 240a and the same lesions in the second updated tumor mask 240b meet one or more thresholds, the treatment response 180 may still be a progressive metabolic disease (PMD).

[0074] Therefore, in some cases, the evaluation engine 115 may determine the treatment response 180 as a responder based on at least a first updated tumor mask 240a and a second updated tumor mask 240b, for example, the evaluation engine 115 may detect a complete metabolic response (CMR) or a partial metabolic response. Alternatively, the evaluation engine 115 may determine the treatment response 180 as a non-responder based on at least a first updated tumor mask 240a and a second updated tumor mask 240b, for example, if the evaluation engine 115 detects no metabolic response (NMR) or progressive metabolic disease (PMD). In some cases, the evaluation engine 115 may determine the treatment response 180 as a complete metabolic disease (CMR), a partial metabolic response (NMR), no metabolic response (PMR), or progressive metabolic disease (PMD) based on at least a first updated tumor mask 240a and a second updated tumor mask 240b. Alternatively, the evaluation engine 115 may determine the treatment response 180 as a complete metabolic response (CMR) or an incomplete metabolic response (non-CMR) based on at least a first updated tumor mask 240a and a second updated tumor mask 240b. Furthermore, in some cases, in addition to or instead of the treatment response 180, the evaluation engine 115 may determine disease progression, overall survival (OS), and / or progression-free survival (PFS) based on at least a first updated tumor mask 240a and a second updated tumor mask 240b.

[0075] In some exemplary embodiments, the evaluation engine 115 may determine the treatment response 180 by at least determining, based on at least a first updated tumor mask 240a and a second updated tumor mask 240b, whether the second lesion associated with the second updated tumor mask 240b is a new lesion or the same lesion as the first lesion associated with the first updated tumor mask 240a. For example, the evaluation engine 115 may determine the distance between the first and second lesions based on at least a first updated tumor mask 240a and a second updated tumor mask 240b. In some cases, the distance between the first and second lesions may be quantified by one or more of the maximum, minimum, mean, median, and / or mode of the distance between a first set of pixels in the first updated tumor mask 240a and a second set of pixels in the second updated tumor mask 240b. If the distance between the first and second lesions is determined to meet one or more thresholds (e.g., a minimum distance greater than 10 millimeters), the evaluation engine 115 may determine that the second lesion is a new lesion. Alternatively, if the distance between the first lesion and the second lesion does not meet one or more thresholds (for example, a minimum distance not exceeding 10 millimeters), the evaluation engine 115 may determine that the second lesion is the same lesion as the first lesion.

[0076] In some exemplary embodiments, if the second lesion is identified as a new lesion, the evaluation engine 115 may determine that the treatment response 180 is a progressive metabolic disease (PMD). Alternatively, if the second lesion is identified as the same lesion as the first lesion, the evaluation engine 115 may further determine the disease response 180 based on the change in the level of metabolic activity exhibited by the lesion between the first and second time points. For example, in some cases, the evaluation engine 115 may determine a first level of metabolic activity exhibited by the lesion at the first time point based on at least a first updated tumor mask 240a and a first positron emission tomography (PET) scan 210a. Furthermore, the evaluation engine 115 may determine a second level of metabolic activity exhibited by the lesion at the second time point based on at least a second updated tumor mask 240b and a second positron emission tomography (PET) scan 210b. In some cases, the level of metabolic activity exhibited by the lesion at each time point may correspond to the maximum, minimum, mean, median, and / or mode of the level of metabolic activity. For example, the level of metabolic activity exhibited by a lesion at each point in time may correspond to the maximum, minimum, median, mean, and / or mode of standardized uptake (SUV) values ​​associated with those pixels in a positron emission tomography (PET) scan from the point in time when they were identified as part of the lesion by the corresponding updated tumor mask.

[0077] In some cases, the change in metabolic activity levels indicated by the lesion between the first and second time points may correspond to the difference between the first maximum level of metabolic activity indicated by the lesion at the first time point and the second maximum level of metabolic activity at the second time point. For example, the change in metabolic activity indicated by the lesion between the first and second time points may correspond to the maximum standard uptake (SUV) across the two time points. max This can correspond to the difference between the two time points. Alternatively and / or additionally, the level of metabolic activity exhibited by the lesion at each time point can correspond to the size of the lesion observed at each time point. Thus, in some cases, the change in metabolic activity exhibited by the lesion can also be determined based on the difference between at least the first size of the lesion at the first time point and the second size of the lesion at the second time point.

[0078] As described above, in some exemplary embodiments, if the second lesion is identified as the same lesion as the first lesion, the evaluation engine 115 may further determine the treatment response 180 based on the change in metabolic activity shown by the lesion between the first and second time points. For example, in some cases, if the change in metabolic activity shown by the lesion between the first and second time points meets a first threshold, the evaluation engine 115 may determine that the treatment response 180 is a progressive metabolic disease (PMD). Alternatively, if the change in metabolic activity shown by the lesion between the first and second time points meets a second threshold but not a first threshold, the evaluation engine 115 may determine that the treatment response 180 is no metabolic response (NMR). Furthermore, if the change in metabolic activity shown by the lesion between the first and second time points does not meet either the first or second threshold, the evaluation engine 115 may determine that the treatment response 180 is a partial metabolic response (PMR).

[0079] Figure 3 shows a flowchart illustrating an example of process 300 for machine learning-based longitudinal analysis of positron emission tomography (PET) and computed tomography (CT) scans, according to several exemplary embodiments. Referring to Figures 1–3, process 300 can be performed by an analysis controller 110. The analysis controller 110 may perform process 300 to determine, for example, a therapeutic response 180. As shown by the capability metrics in Figures 6–7, 8A–8G, 9, and 10, the analysis controller 110 performing process 300 can achieve highly accurate results consistent with those determined by a specialist radiologist. Process 300 can be performed independently (e.g., without requiring the intervention of a specialist radiologist) to achieve highly accurate results, which means that process 300 provides a more efficient diagnostic solution that requires fewer resources than conventional techniques for analyzing positron emission tomography (PET) and computed tomography (CT) scans.

[0080] In 302, the analysis controller 110 may train a longitudinal segmentation model 113 to update tumor masks generated based on positron emission tomography (PET) and computed tomography (CT) scans from a single time point. In some exemplary embodiments, the analysis controller 110 may train the longitudinal segmentation model 113 based on at least a training set to update two or more tumor masks generated from positron emission tomography (PET) and computed tomography (CT) scans, each from a single time point. In some cases, the training set may include one or more annotated training samples, each containing positron emission tomography (PET) and computed tomography (CT) scans from two or more different time points, along with a corresponding ground truth tumor mask. Furthermore, in some cases, each annotated training sample may include regions extracted from positron emission tomography (PET) and computed tomography (CT) scans from two or more different time points (e.g., head and neck region, chest region, abdominal and pelvic region) and a corresponding ground truth tumor mask. In this context, each pixel in the ground truth tumor mask may be associated with a ground truth label having a first value (e.g., "1") indicating that the pixel is part of the lesion, or a second value (e.g., "0") indicating that the pixel is not part of the lesion.

[0081] In 304, the analysis controller 110 may apply a trained longitudinal segmentation model 113 to update the first tumor mask from a first time point and the second tumor mask from a second time point. In some exemplary embodiments, as shown in Figure 2, the analysis controller 110 may apply a trained longitudinal segmentation model 113 to update the first tumor mask 230a from a first time point and the second tumor mask 230b from a second time point, based on at least a first positron emission tomography (PET) scan 210a and a first computed tomography (CT) scan 220a from a first time point and a second positron emission tomography (PET) scan 210b and a second computed tomography (CT) scan 220b from a second time point. As will be described in more detail below, the first tumor mask 230a may be generated based on the first positron emission tomography (PET) scan 210a and the first computed tomography (CT) scan 220a, while the second tumor mask 230b may be generated based on the second positron emission tomography (PET) scan 210b and the second computed tomography (CT) scan 220b. The longitudinal segmentation model 113 can utilize information across multiple time points (e.g., levels of metabolic activity, tissue density (or X-ray attenuation), etc.) to refine each of the first tumor mask 230a and the second tumor mask 230b, thereby reducing false positives in which one or more pixels that are not part of the lesion are thus misidentified in the first tumor mask 230a and / or the second tumor mask 230b.

[0082] In 306, the analysis controller 110 may determine the response to disease treatment based on at least a first updated tumor mask and a second updated tumor mask. In some exemplary embodiments, Figure 2 shows that the evaluation engine 115 may determine the treatment response 180 based on at least a first updated tumor mask 240a and a second updated tumor mask 240b. As will be described in more detail below, the evaluation engine 115 may determine the treatment response 180 at least based on whether the second lesion associated with the second updated tumor mask 240b is a new lesion or the same lesion as the first lesion associated with the first updated tumor mask 240a. If the evaluation engine 115 determines that the second lesion associated with the second updated tumor mask 240b is not a new lesion but the same lesion as the first lesion associated with the first updated tumor mask 240a, the evaluation engine 115 may determine the treatment response 180 based on the first updated tumor mask 240a, the first positron emission tomography (PET) scan 210a, the second updated tumor mask 240b, and the changes in metabolic activity determined based on the second positron emission tomography (PET) scan 210b.

[0083] In some cases, in addition to or instead of the treatment response 180, the evaluation engine 115 may determine disease progression, such as non-Hodgkin lymphoma (NHL) or another fluorodeoxyglucose-affinity (FDG-affinity) cancer, observable on positron emission tomography (PET) scans, based on at least the first updated tumor mask 240a and the second updated tumor mask 240b, and in some cases, the evaluation engine 115 may also determine changes in tumor volume (e.g., total metabolic tumor volume (TMTV), etc.) that sequentially indicate the treatment response 180 and / or disease progression, based on at least the first updated tumor mask 240a and the second updated tumor mask 240b. Alternatively and / or additionally, in some cases, the evaluation engine 115 may determine the dispersion of the first change in metabolic activity and / or the second change in tumor volume between a first and second time point across various lesions, based on at least the first updated tumor mask 240a and the second updated tumor mask 240b.

[0084] Figure 4 shows a flowchart illustrating another example of process 400 for machine learning-based longitudinal analysis of positron emission tomography (PET) and computed tomography (CT) scans, according to several exemplary embodiments. Referring to Figures 1 to 4, process 400 may be performed by the analysis controller 110, for example, operation 304 of process 300.

[0085] In 402, the analysis controller 110 may determine a first tumor mask corresponding to a first lesion present in a first positron emission tomography (PET) scan and a first computed tomography (CT) scan, based on a first positron emission tomography (PET) scan and a first computed tomography (CT) scan from at least a first time point. For example, in the example shown in Figure 2, the preprocessing engine 111 may determine a first tumor mask 230a corresponding to a first lesion shown in each of the first positron emission tomography (PET) scan 210a and the first computed tomography (CT) scan 220a, based on at least a first positron emission tomography (PET) scan 210a and a first computed tomography (CT) scan 220a. In some cases, the preprocessing engine 111 may determine the first tumor mask 230a by applying a segmentation model to at least the first positron emission tomography (PET) scan 210a and the first computed tomography (CT) scan 220a. Furthermore, in some cases, the preprocessing controller 110 may align the first positron emission tomography (PET) scan 210a and the first computed tomography (CT) scan 220a to generate a first superimposed (or jointly aligned) image (e.g., the first PET-CT scan). Alternatively and / or additionally, the preprocessing controller 110 may extract a first patch containing a first lesion for incorporation by the longitudinal segmentation model 113 from the first positron emission tomography (PET) scan 210a aligned with the first computed tomography (CT) scan 220a.

[0086] In step 404, the analysis controller 110 may determine a second tumor mask corresponding to a second lesion present in the second positron emission tomography (PET) scan and the second computed tomography (CT) scan, based on at least the second positron emission tomography (PET) scan and the second computed tomography (CT) scan from the second time point. For example, in the example shown in Figure 2, the preprocessing engine 111 may determine a second tumor mask 230b corresponding to a second lesion shown in the second positron emission tomography (PET) scan 210b and the second computed tomography (CT) scan 220b, based on at least the second positron emission tomography (PET) scan 210b and the second computed tomography (CT) scan 220b, respectively. In some cases, such as the first tumor mask 230a, the preprocessing engine 111 may determine the second tumor mask 230b by applying a segmentation model to at least the second positron emission tomography (PET) scan 210b and the second computed tomography (CT) scan 220b. In addition, in some cases, the preprocessing controller 110 may align the second positron emission tomography (PET) scan 210b and the second computed tomography (CT) scan 220b to generate a second superimposed (or jointly aligned) image (e.g., a second PET-CT scan). Alternatively and / or additionally, the preprocessing controller 110 may extract a second patch containing the second lesion for incorporation by the longitudinal segmentation model 113 from the second positron emission tomography (PET) scan 210b aligned with the second computed tomography (CT) scan 220b.

[0087] In 406, the analysis controller 110 may apply the longitudinal segmentation model 113 to update the first tumor mask and the second tumor mask, respectively, based on at least the first positron emission tomography (PET) scan, the first computed tomography (CT) scan, the second positron emission tomography (PET) scan, and the second computed tomography (CT) scan. For example, in the example shown in Figure 2, the analysis controller 110 may apply the longitudinal segmentation model 113 to update the first tumor mask 230a and the second tumor mask 230b, respectively, based on at least the first patch extracted from the first positron emission tomography (PET) scan 210a and the first computed tomography (CT) scan 220a, and the second patch extracted from the second positron emission tomography (PET) scan 210b and the second computed tomography (CT) scan 220b. As described above, in some cases, the longitudinal segmentation model 113 may determine whether a pixel is part of a lesion based on the level of metabolic activity and tissue density (or X-ray attenuation) indicated by the pixel and one or more adjacent pixels across a first and second time point. Accordingly, the longitudinal segmentation model 113 may update the first tumor mask 230a and / or the second tumor mask 230b, for example, by updating the labels assigned to one or more pixels in the first tumor mask 230a and / or the second tumor mask 230b. For example, in some cases, this update may include the longitudinal segmentation model 113 reclassifying pixels that were previously classified (e.g., by the segmentation model) as not being part of a lesion. Alternatively and / or additionally, this update may include the longitudinal segmentation model 113 reclassifying pixels that were previously classified (e.g., by the segmentation model) as not being part of a lesion.

[0088] Figure 5 shows a flowchart illustrating another example of process 500 for machine learning-based longitudinal analysis of positron emission tomography (PET) and computed tomography (CT) scans, according to several exemplary embodiments. Referring to Figures 1-3 and 5, process 500 may be performed by the selection engine 110, for example, operation 306 of process 300.

[0089] In some cases, process 500 may perform the Lugano classification process. Thus, in some cases, the therapeutic response 180 produced by the analysis controller 110 may follow the Lugano classification (or tumor staging) paradigm, which includes identifying the patient's positron emission tomography (PET) and computed tomography (CT) scans as indicating progressive metabolic disease (PMD) (or incomplete metabolic response (non-CMR)), no metabolic response (NMR), or partial metabolic response (PMR). The capability metrics shown in Figures 6–7, 8A–8G, 9, and 10 demonstrate that the analysis controller 110 performing process 500 can achieve results with high accuracy. That is, the Lugano classification obtained by the analysis controller 110 performing process 500 is consistent with the determination made by a specialist radiologist. Furthermore, Process 500 can be run independently (e.g., without the intervention of a specialist radiologist) to achieve highly accurate results, and is therefore faster, more efficient, and requires fewer resources than conventional techniques for analyzing positron emission tomography (PET) and computed tomography (CT) scans. The speed and efficiency of Process 500 can facilitate many downstream clinical tasks, including treatment decisions. In some cases, the treatment response 180 generated by Process 500 can be applied to generate immediate treatment decisions, thereby eliminating significant bottlenecks in conventional clinical workflows.

[0090] As will be explained in more detail below, Process 500 can be run to leverage insights derived from machine learning-based analysis of longitudinal positron emission tomography (PET) and computed tomography (CT) scans to generate a more granular Lugano classification than conventional techniques, particularly those that simply examine data from a single point in time. For example, Process 500 can be run to provide a precise and granular distinction between progressive metabolic disease (PMD) (or incomplete metabolic response (non-CMR)), no metabolic response (NMR), and partial metabolic response (PMR), which can be more insightful than a binary classification (e.g., responders and non-responders). The accuracy and granularity of the therapeutic responses 180 generated by Process 500 means that Process 500 also improves the accuracy and precision of downstream clinical tasks, such as identifying relapsed and refractory patients, as well as treatment decisions, which depend on the output of Process 500.

[0091] In 502, the analysis controller 110 may determine, based on the first updated tumor mask from at least a first time point and the second updated tumor mask from a second time point, whether the second lesion associated with the second updated tumor mask is a new lesion or the same lesion as the first lesion associated with the first updated tumor mask. In some exemplary embodiments, the evaluation engine 115 may determine the distance between the first lesion associated with the first updated tumor mask 240a and the second lesion associated with the second updated tumor mask 240b, based on the first updated tumor mask 240a from at least a first time point and the second updated tumor mask 240b from a second time point. In some cases, the distance between the first and second lesions may be quantified by one or more of the maximum, minimum, mean, median, and / or mode of the distance between a first set of pixels in the first updated tumor mask 240a and a second set of pixels in the second updated tumor mask 240b. Whether a second lesion associated with a second updated tumor mask 240b is a new lesion or the same lesion as the first lesion associated with a first updated tumor mask 240a can be determined based on whether the distance between the first lesion and the second lesion meets one or more thresholds. For example, in some cases, the analysis engine 115 may determine that the second lesion is a new lesion if the distance between the first lesion and the second lesion (e.g., maximum distance) exceeds a threshold (e.g., more than 10 millimeters). Alternatively, the evaluation engine 115 may determine that the second lesion is not a new lesion and is the same lesion as the first lesion associated with the first updated tumor mask 240a if the distance between the first lesion and the second lesion (e.g., maximum distance) does not exceed a threshold (e.g., 10 millimeters).

[0092] In 503-Y, the analysis controller 110 may identify the second lesion associated with the second updated tumor mask as a new lesion. As described above, in some cases, the evaluation engine 115 may determine that the second lesion associated with the second updated tumor mask 240b is a new lesion and not the same lesion as the first lesion associated with the first updated tumor mask 240a if the distance between the first and second lesions meets one or more thresholds (e.g., the maximum distance between the first and second lesions exceeds 10 millimeters). Thus, in 504, the analysis controller 110 may determine the response to treatment of the disease as a progressive metabolic disease (PMD). For example, if the second lesion associated with the second updated tumor mask 240b is identified as a new lesion, the evaluation engine 115 may determine that the treatment response 180 is a progressive metabolic disease (PMD), or in some cases an incomplete metabolic response (non-CMR).

[0093] Alternatively, in 503-N, the analysis controller 110 may identify the second lesion associated with the second updated tumor mask not as a new lesion, but as the same lesion as the first lesion associated with the first updated tumor mask. For example, in some cases, the evaluation engine 115 may determine that the second lesion associated with the second updated tumor mask 240b is not a new lesion, but the same lesion as the first lesion associated with the first updated tumor mask 240a, if the distance between the first and second lesions does not meet one or more thresholds (e.g., the maximum distance between the first and second lesions does not exceed 10 millimeters). Thus, in 506, the analysis controller 110 may determine the change in the level of metabolic activity indicated by the lesion between the first and second time points, based at least on the first updated tumor mask, the first positron emission tomography (PET) scan from the first time point, the second updated tumor mask, and the second positron emission tomography (PET) scan from the second time point. For example, in some cases, the evaluation engine 115 may determine the change in the level of metabolic activity exhibited by the lesion between a first and a second time point by at least determining a first level of metabolic activity exhibited by the lesion at a first time point based on at least a first updated tumor mask 240a and a first positron emission tomography (PET) scan 210a. Furthermore, in some cases, the evaluation engine 115 may determine the change in the level of metabolic activity exhibited by the lesion between a first and a second time point by at least determining a second level of metabolic activity exhibited by the lesion at a second time point based on at least a second updated tumor mask 240b and a second positron emission tomography (PET) scan 210b. The change in the level of metabolic activity exhibited by the lesion between a first and a second time point may correspond to the difference between the first metabolic activity level and the second metabolic activity level.In some cases, it should be understood that the level of metabolic activity indicated by a lesion at any given time point can be quantified by the maximum, minimum, average, median, and / or most frequent value of the level of metabolic activity (e.g., standardized uptake value (SUV)) indicated by each pixel in a positron emission tomography (PET) scan identified as part of the lesion by the corresponding updated tumor mask. In some cases, the change in the level of metabolic activity ΔSUV indicated by a lesion between a first time point and a second time point can be determined based on the following formula (1). [Number] In the formula, SUVmax t1 represents the first level of metabolic activity at the first time point SUVmax t1 (e.g., at screening, before treatment, etc.), and SUVmax t1 represents the second level of metabolic activity at the second time point SUVmax t1 (e.g., at follow-up, after treatment, etc.).

[0094] In 508, the analysis controller 110 can determine whether the change in the level of metabolic activity indicated by a lesion between the first time point and the second time point meets a first threshold. For example, in some cases, the evaluation engine 115 can determine whether the change in the level of metabolic activity indicated by a lesion between the first time point and the second time point, such as the difference between the first maximum standardized uptake value (SUVmax t1 ) from the first time point and the second maximum standardized uptake value (SUVmax t1 ) from the second time point, meets a first threshold (e.g., SUVmax t1 ).

[0095] In 509-Y, the analysis controller 110 can determine that the change in metabolic activity indicated by a lesion between the first time point and the second time point meets the first threshold. For example, in some cases, the evaluation engine 115 can determine the first maximum standardized uptake value (SUVmax t1 ) from the first time point and the second maximum standardized uptake value (SUVmax t2The change in the level of metabolic activity exhibited by the lesion between the first and second time points, such as the difference between ), may be determined to satisfy the first threshold (e.g., ΔSUV > 0.5). Thus, process 500 may be restarted in operation 504 in which the analysis controller 110 determines the response to treatment of the disease as a progressive metabolic disease (PMD). For example, if the change in the level of metabolic activity between the first and second time points is determined to satisfy the first threshold (e.g., ΔSUV > 0.5), the evaluation engine 115 may determine that the treatment response 180 is a progressive metabolic disease (PMD), or possibly an incomplete metabolic response (non-CMR).

[0096] Alternatively, in 509-N, the analysis controller 110 may determine that the change in metabolic activity indicated by the lesion between the first and second time points does not meet the first threshold. For example, in some cases, the evaluation engine 115 may determine that the change in the level of metabolic activity between the first and second time points does not meet the first threshold (e.g., ΔSUV>0.5). Therefore, in 510, the analysis controller 110 may determine whether the change in metabolic activity indicated by the lesion between the first and second time points meets the second threshold. For example, if the change in the level of metabolic activity between the first and second time points does not meet the first threshold (e.g., ΔSUV>0.5), the evaluation engine 115 may further determine whether the change in the level of metabolic activity between the first and second time points meets the second threshold (e.g., ΔSUV>-0.25).

[0097] In 511-Y, the analysis controller 110 may determine that the change in metabolic activity indicated by the lesion between the first and second time points meets the second threshold. For example, in some cases, the evaluation engine 115 may determine the first maximum standard uptake value (SUVmax) from the first time point. t1 ) and the second maximum standard intake value from the second point in time (SUVmax t2The difference between the first and second time points, for example, can determine that the change in the level of metabolic activity indicated by the lesion between the first and second time points satisfies the second threshold but not the first threshold (e.g., 0.5 > ΔSUV > -0.25). Therefore, in 512, the analysis controller 110 may determine that the response to treatment of the disease is no metabolic response (NMR). For example, if the change in the level of metabolic activity indicated by the lesion between the first and second time points satisfies the second threshold but not the first threshold (0.5 > ΔSUV > -0.25), the evaluation engine 115 may determine that the treatment response 180 is no metabolic response (NMR).

[0098] Alternatively, in 511-N, the analysis controller 110 may determine that the change in metabolic activity indicated by the lesion between the first and second time points does not meet the second threshold. For example, in some cases, the evaluation engine 115 may determine the first maximum standard uptake value (SUVmax) from the first time point. t1 ) and the second maximum standard intake value from the second point in time (SUVmax t2 The difference between the first and second time points, for example, can determine that the change in the level of metabolic activity exhibited by the lesion between the first and second time points does not meet both the first threshold (e.g., ΔSUV < -0.25) and the second threshold. Therefore, in 514, the analysis controller 110 can determine the response to treatment of the disease as a partial metabolic response (PMR). For example, if the evaluation engine 115 determines that the change in metabolic activity exhibited by the lesion between the first and second time points does not meet either the first or second threshold (ΔSUV < -0.25), the evaluation engine 115 can determine that the treatment response 180 is a partial metabolic response (PMR).

[0099] As described above, the analysis controller 110 can determine the treatment response 180 with better efficiency and a higher level of accuracy, as measured by consistency with results generated by specialist radiologists. The capability of the analysis controller 110 can be evaluated based on datasets from different populations and treatment protocols. The accuracy of the analysis controller 110 was evaluated in 2,266 evaluable follow-up visits from 678 specific patients. The treatment response 180 determined by the analysis controller 110 showed strong agreement with the assessments of specialist radiologists on the same dataset. In six of the nine experiments (Table 1, Figure 7(B)), there was no statistically significant difference between the capability of the analysis controller 110 compared to the final response from the specialist radiologist committee and between radiologist agreement. In the other two experiments, the difference was less than 5%. The same comparison as the F1 score showed no significant difference in seven of the nine comparisons, and a difference of less than 5% in the other two comparisons (Figure 8C). Table 1 below shows the accuracy of the analysis controller 110 based on the final responses from specialist radiologists to the test set, compared to inter-radiologist agreement.

[0100] Table 1 includes three datasets: (1) GOYA (NCT01287741), (2) GO29365 (NCT02257567), and (3) GO29781 (NCT02500407). For GOYA, the first PET scan (baseline scan) was performed 1 to 35 days before treatment, and the second PET scan (post-treatment scan) was performed 6 to 8 weeks after the last dose. For early discontinuation, the second PET scan was performed 4 to 8 weeks after the last dose. For GO29365, the first PET scan (baseline scan) was performed before treatment, and the second PET scan (intermediate scan) was performed during 6 weeks of treatment, 3 months of treatment, and every 3 months while treatment was progressing. In addition, a second PET scan (follow-up scan) was performed every 3 months for 18 months, and then every 12 months. For GO29781, the first scan (baseline scan) was performed before treatment. A second PET scan was performed at 6 weeks during treatment (intermediate scan), upon completion of treatment (post-treatment scan), and every 6 months for two years (follow-up scans).

[0101] [Table 1]

[0102] Figure 6 shows the total metabolic response (CMR) assessment performed by the analysis controller 110 in (A) clinical trial dataset GO29781 / NCT02500407 and (C) clinical trial dataset GO29365 / NCT02257567, as well as the objective response (OR) assessment performed by the analysis controller 110 in (E) clinical trial dataset GO29781 / NCT02500407 and (G) clinical trial dataset GO29365 / NCT02257567. These depict the accuracy compared to the inter-leader agreement for total metabolic response (CMR) assessment in (B) clinical trial dataset GO29781 / NCT02500407 and (D) clinical trial dataset GO29365 / NCT02257567, and to the inter-leader agreement for objective response (OR) assessment in (F) clinical trial dataset GO29781 / NCT02500407 and (H) clinical trial dataset GO29365 / NCT02257567.

[0103] Figure 7(A) shows a comparison of the accuracy of the analysis controller 110 and the specialist radiologist, and Figure 7(B) shows error bars representing the difference between inter-leader agreement and the accuracy of the analysis controller 110 compared to the final response. Figure 7(C) shows the overall survival (OS) and progression-free survival (PFS) hazard ratios for patients in each dataset identified by the analysis controller 110 and the specialist radiologist as exhibiting a complete metabolic response (CMR) at the end of treatment (e.g., an incomplete metabolic response).

[0104] For the prediction of total metabolic response (CMR) for the clinical trial dataset GO29781 / NCT02500407, the clinical trial dataset GO29365 / NCT02257567, and the GOYA holdout set (NCT01287741) (Table 1, Figure 6, Figures 7(A) to 7(B) and Figure 8A), the agreement between the treatment response 180 determined by the analysis controller 110 and that of specialist radiologists was 0.87 (95% confidence interval (CI): 0.84, 0.89), 0.83 (95% CI: 0.79 to 0.86), and 0.87 (95% CI: 0.82, 0.92), respectively. These comparisons did not show statistically significant differences compared to inter-leader agreement, and were estimated to be -0.02 (95% CI: -0.06, 0.00); 0.01 (95% CI: -0.05 to 0.05); and -0.03 (95% CI: -0.09, 0.02), respectively. Objective response (OR), accuracy of the four response categories, and inter-radiologist agreement are shown in Table 1. The capabilities of the analysis controller 110 did not show significant difference between leader agreement and objective response (OR) prediction (Figures 6-7) in the GOYA holdout set (NCT01287741) -0.02 (95% CI: -0.06, 0.02), and differences of less than 5% were observed in other test sets (-0.04, 95% CI: -0.09, -0.01, GO29365 / NCT02257567, and -0.05, 95% CI: -0.07, -0.02, GO29781 / NCT02500407). No statistically significant differences were observed in the accuracy of the four response categories of the analysis controller 110 for the GOYA holdout set (NCT01287741) (-0.04, 95% CI: -0.11, 0.01) and GO29365 / NCT02257567 (-0.03, 95% CI: -0.08, 0.03), while -0.07 (95% CI: -0.11, -0.03) was observed for GO29781 / NCT02500407 (Figures 6-7 and 8B).

[0105] Specialist radiologists reviewed the intermediate and final outputs of the analysis controller 110 (including tumor masks at screening and follow-up, levels of metabolic activity (e.g., maximum uptake value (SUVmax) of the hottest lesions), indicators of new lesions, and predicted metabolic responses) to evaluate responses to 114 intermediate or final treatment scans. No correction of predicted responses was required for 81% (95% CI: 74–88) of visits. The mean time spent by radiologists reviewing the models was 2.02 minutes per visit (ranging from 1–15 minutes). The degree of agreement between the model and the specialist radiologists was similar to the radiologist agreement with the final specialist radiologist committee responses. The agreement between the analysis controller 110's prediction of total metabolic response (CMR) and the radiologists (Figures 8D–8E) was 88% (95% CI: 82–93). The agreement with objective response predictions was 91% (95% CI: 86–96), and the agreement for predictions of the four response classes was 81% (95% CI: 74–88). There were no statistically significant differences between the agreement between radiologists and the proposed method, and between radiologists and the final expert radiology committee response (Figure 8E), for objective response assessment (-0.01, 95% CI: -0.07, 0.05) and assessment of the four response classes (-0.05, 95% CI: -0.14, 0.4). However, a difference of -0.09 (95% CI: -0.16, -0.03) was observed for the accuracy of total metabolic response (CMR) assessment.

[0106] Survival analyses were performed on three study datasets. Predicted total metabolic response (CMR) strongly predicted overall survival (OS), and researchers assessed progression-free survival (PFS) (Table 2, Figure 7(C) and Figure 8D). The overall survival (OS) hazard ratios (HRs) for patients identified by the analysis controller 110 as exhibiting a complete metabolic response (CMR) at the end of treatment were 0.123 (95% CI: 0.055, 0.276) for the GOYA holdout set (NCT01287741), 0.205 (95% CI: 0.098, 0.426) for the R / R DLBCL bendamustine + rituximab (BR) and polatuzumab (pola) + BR cohorts (GO29365 / NCT02257567), and 0.054 (95% CI: 0.01, 0.442) for the Phase 2 R / R FL expanded cohort (GO29781 / NCT02500407). Table 2 below shows the hazard ratios (HR) for progression-free survival (PFR) and for responses from specialist radiology committees.

[0107] [Table 2]

[0108] Patients predicted to exhibit a complete metabolic response by the analysis controller 110 had a higher or equal mortality risk at landmark survival compared to patients identified by the specialist radiology committee as not exhibiting a complete metabolic response (CMR) (Figure 9). The 2-year overall survival rate for patients with a predicted incomplete metabolic response (non-CMR) in the GOYA holdout set (NCT01287741) (Figure 9(A)) was 57% (95% CI: 40-81), compared to 69% (95% CI: 55-86) for patients with an incomplete metabolic response (CMR) identified by the specialist radiology committee (Figure 9(D)). In the GO29781 / NCT02500407 dataset (Figure 9(B)) and the GO29365 / NCT02257567 dataset (Figure 9(C)), the 18-month overall survival (OS) for patients with total metabolic response identified by the analysis controller 110 was 60% (95% CI: 41-88) and 34% (95% CI: 19-60), respectively, compared to 78% (95% CI: 63-97, Figure 9(E)) and 34% (95% CI: 19-61, Figure 9(F)) for those identified by the specialist radiologist committee. Patients exhibiting total metabolic response (CMR) and identified by the analysis controller 110 had higher or equal survival rates compared to patients identified by the specialist radiologist committee at the same landmark survival time across the three study sets (Figure 7). Kaplan-Meier curves and PFS estimates for progression-free survival (PFS) are shown in Figure 9. Specialist radiologists and analysis controllers 110 assessed the Deauville score (DS) based on FDG incorporation for all three datasets, which are shown in Figure 10.

[0109] Log-rank tests of overall survival based on CMR vs. non-CMR and final IRC responses for each test set in the analysis controller 110. This analysis demonstrates that there was no significant difference between the results from the analysis controller 110 and the final IRC responses of all groups except one (GO29781, final IRC response), as the p-value was less than 0.05 (Table 3).

[0110] [Table 3] The segmentation and lesion detection capabilities of an exemplary longitudinal segmentation model 113 in follow-up FDG-PET / CT scans in the GOYA (NCT01287741) set were performed. The longitudinal segmentation model 113 is deployed to improve tumor segmentation in registered follow-up FDG-PET / CT scans. First, region-specific VNet models are trained to improve tumor segmentation in different regions (e.g., abdominal / pelvic region, thoracic region, and head / neck region). The input to the VNet consists of four modality patches—screening PET, screening CT, registered follow-up PET, and CT—centered on the tumor centroid predicted at follow-up by the tumor segmentation at a given point in time. The VNet input size for the thoracic and head / neck regions is 96*96*96*4, and the VNet input size for the abdominal / pelvic region is 128*128*128*4. Next, the predicted patch tumor mask is inserted into the whole-body tumor mask. The pre-trained models Unet and Swin UNETR were also tested for longitudinal lesion segmentation in registered FDG-PET / CT follow-up scans. Ablation studies were also conducted to evaluate the capabilities obtained for lesion segmentation in follow-up scans with the addition of longitudinal segmentation models, and for lesion segmentation in follow-up scans with the addition of screening FDG-PET / CT scan information as input to the longitudinal segmentation models.

[0111] A longitudinal tumor segmentation model was trained using the GOYA training set with 1919 samples from 740 follow-up scans enrolled in baseline scans (including 384 scans with no tumors). There were 1196 patches in the abdominal / pelvic region (397 negative), 379 in the thoracic region (54 negative), and 344 in the head / neck region (85 negative). The GOYA trial set included 179 enrolled follow-up scans. In the trial set, the abdominal / pelvic VNet achieved a DICE score (DSC) of 0.802, while the thoracic and head / neck VNets achieved DICE scores of 0.813 and 0.778, respectively. The overall DICE score at the overall scan level was 0.796. The final tumor mask had a mean of 0.11 false-positive (FP) lesions, 0.39 false-negative (FN) lesions, and 1.47 true-positive (TP) lesions per scan (Table 4). The addition of the longitudinal segmentation VNet step reduces false positives (average 1.21 false positive lesions per scan when using a tumor mask from a single-time-point tumor segmentation model). Using both screening and follow-up FDG-PET / CT information in the longitudinal segmentation model increases sensitivity to lesions in follow-up scans compared to models that use only follow-up scans as input (average 0.65 false negative lesions per scan when using only follow-up scans as input). VNet demonstrated superior capabilities for longitudinal segmentation compared to UNet and Swin UNET®. A detailed evaluation of the capabilities of the different models is shown in Table 4.

[0112] [Table 4]

[0113] Figure 11 shows examples of true positive (TP), true negative (TN), false positive (FP), and false negative (FN). Figures 11A and 11B show examples of correctly classified true negative CMRs, with A) no tumors present in ground truth TMTV annotation and B) no tumors present in tumor segmentation by this method. Figures 11C and 11D demonstrate examples of false positive CMRs, with C) no metabolically active tumors according to IRC annotation and D) false positive lesions from this method. Figures 11E and 11F show examples of correctly classified true positives of non-CMRs for the same lesions detected by E) IRC and F) the model. Figures 11G and 11H show examples of false negatives of PMD G) by IRC for small lesions misclassified as CMR H) by the model.

[0114] Taking into consideration the above-described embodiments of the subject matter, the present application discloses the following list of examples, which are further examples included in the disclosure of the present application, by combining one feature of a single example or two or more features of the aforementioned examples, or optionally, one or more features of one or more further examples.

[0115] Item 1: A computer implementation method comprising: determining a first tumor mask corresponding to a first lesion present in a first PET scan and a first CT scan based on a first positron emission tomography (PET) scan and a first computed tomography (CT) scan from a first time point; determining a second tumor mask corresponding to a second lesion present in a second PET scan and a second CT scan based on a second PET scan and a second CT scan from a second time point; applying a longitudinal segmentation model to update each of the first and second tumor masks based on a first PET scan, a first CT scan, a second PET scan, and a second CT scan; and determining a response to disease treatment based on at least one of the first updated tumor mask and the second updated tumor mask.

[0116] Item 2: The method according to Item 1, wherein a first tumor mask identifies a first set of pixels in each of a first PET scan and a first CT scan depicting a first lesion, and a second tumor mask identifies a set of pixels from a second PET scan and a second CT scan depicting a second lesion.

[0117] Item 3: The method of Item 1 or 2, further comprising registering the first CT scan, the first PET scan, the second CT scan, and the second PET scan in order to align the first CT scan and the first PET scan with the second CT scan and the second PET scan.

[0118] Item 4: The method according to any one of items 1 to 3, further comprising at least identifying a second lesion as a new lesion based on a first updated tumor mask and a second updated tumor mask, and determining the response to treatment as a progressive disease (PMD) in response to the identification of the second lesion as a new lesion.

[0119] Item 5: Determine the distance between the first lesion and the second lesion based on at least the first updated tumor mask and the second updated tumor mask. The method according to item 4, further comprising identifying the second lesion as a new lesion based at least on the distance between the first lesion and the second lesion satisfying one or more thresholds, and identifying the second lesion as the same lesion as the first lesion based at least on the distance between the first lesion and the second lesion not satisfying one or more thresholds.

[0120] Item 6: The method of Item 4 or 5, further comprising determining the response to treatment of the disease, at least on the basis of the change in metabolic activity indicated by the lesion between the first and second time points, in response to the determination that the first and second lesions are the same lesion.

[0121] Item 7: The method according to Item 6, wherein the change in metabolic activity between a first and second time point is determined by determining a first level of metabolic activity indicated by the lesion at the first time point based on at least a first updated tumor mask and a first PET scan, determining a second level of metabolic activity indicated by the lesion at the second time point based on at least a second updated tumor mask and a second PET scan, and determining the change in metabolic activity between the first and second time point points based on at least the first and second levels of metabolic activity.

[0122] Item 8: The method according to Item 7, wherein the response to treatment is determined to be a progressive metabolic disease (PMD) based at least on a change in metabolic activity between a first time point in time when a first threshold is met and a second time point in time.

[0123] Item 9: The method according to Item 8, wherein the response to treatment is determined to be no metabolic response (NMR) based at least on the change in metabolic activity between the first and second time points meeting the second threshold but not the first threshold.

[0124] Item 10: The method according to Item 9, wherein the response to treatment is determined to be a partial metabolic response (PMR) based at least on the fact that the change in metabolic activity between the first and second time points does not meet the first and second thresholds.

[0125] Item 11: The method according to any one of items 7 to 10, wherein the first level of metabolic activity corresponds to the first standardized uptake (SUV) value, and the second level of metabolic activity corresponds to the second standardized uptake (SUV) value.

[0126] Item 12: The method according to any one of items 7 to 10, wherein each of the first level of metabolic activity and the second level of metabolic activity corresponds to the maximum, minimum, median, mean, or mode of metabolic activity indicated by the lesion at the corresponding time point.

[0127] Item 13: The method according to any one of items 1 through 12, wherein a first CT scan and a first PET scan are performed before treatment of the disease, and a second CT scan and a second PET scan are performed after treatment of the disease.

[0128] Item 14: The method according to any one of items 1 to 13, further comprising determining a change in tumor volume based on a first updated tumor mask and a second updated tumor mask, and determining a response to treatment of the disease based on the change in tumor volume.

[0129] Item 15: The method according to any one of items 1 to 13, further comprising determining, based on a first updated tumor mask and a second updated tumor mask, the variance of a first change in metabolic activity and / or a second change in tumor volume between a first and second time point across a variety of lesions, and determining the response to treatment, at least based on the variance of changes in metabolic activity and / or tumor volume between a first and second time point as indicated by the variety of lesions.

[0130] Item 16: The method according to any one of items 1 to 15, further comprising determining disease progression based on at least a first updated tumor mask and a second updated tumor mask.

[0131] Item 17: The method according to any one of items 1 to 16, wherein the first tumor mask is determined by applying the segmentation model to a first PET scan and a first CT scan, and the second tumor mask is determined by applying the segmentation model to a second PET scan and a second CT scan.

[0132] Item 18: The method according to any one of items 1 through 17, wherein the longitudinal segmentation model is an artificial neural network or a visual transducer.

[0133] Item 19: The method according to any one of items 1 through 18, wherein each of the first CT scan, first PET scan, second CT scan, and second PET scan is a three-dimensional volume containing multiple two-dimensional patches.

[0134] Item 20: The method according to any one of items 1 to 19, wherein each pixel in the first PET scan and the second PET scan is associated with an intensity value corresponding to a level of metabolic activity.

[0135] Item 21: The method according to any one of items 1 to 20, wherein each pixel in the first CT scan and the second CT scan is associated with an intensity value corresponding to tissue density or X-ray attenuation.

[0136] Item 22: The method according to any one of items 1 to 21, further comprising training a longitudinal segmentation model to update two or more tumor masks, wherein each of the two or more tumor masks is generated from a positron emission tomography (PET) scan and computed tomography (CT) scan from a single time point.

[0137] Item 23: The method described in any one of items 1 through 22, wherein the response to treatment of a disease is a complete metabolic response (CMR) or an incomplete metabolic response (non-CMR).

[0138] Item 24: The method described in any one of items 1 through 22, wherein the response to treatment for a disease is either a responder or a non-responder.

[0139] Item 25: The method according to any one of items 1 through 22, wherein the response to treatment of the disease is a total metabolic response (CMR), a partial metabolic response (PMR), no metabolic response (NMR), or a progressive metabolic disease (PMD).

[0140] Item 26: The method according to any one of items 1 to 25, further comprising extracting a first patch containing a first lesion associated with a first tumor mask from a first PET scan and a first CT scan, extracting a second patch containing a second lesion associated with a second tumor mask from a second PET scan and a first CT scan, and applying a longitudinal segmentation model to the first and second patches to update the first and second tumor masks, respectively.

[0141] Item 27: A system comprising at least one data processor and at least one memory for storing instructions that, when executed by the at least one data processor, result in an operation including the method described in any of Items 1 to 26.

[0142] Item 28: A non-temporary computer-readable medium that stores instructions that, when executed by at least one data processor, result in an operation including the method described in any one of Items 1 through 26.

[0143] Figure 12 shows a block diagram illustrating an example of a computing system according to an embodiment of the subject. Referring to Figures 1 to 10, the computing system 1200 may be used to implement an analysis controller 110, one or more imaging devices 120, a client device 130, and / or any component thereof.

[0144] As shown in Figure 12, the computing system 1200 may include a processor 1210, memory 1220, storage device 1230, and input / output device 1240. The processor 1210, memory 1220, storage device 1230, and input / output device 1240 may be interconnected via a system bus 1250. The processor 1210 is capable of processing instructions for execution within the computing system 1200. Such instructions for execution may implement, for example, one or more components of an analysis controller 110, one or more imaging devices 120, and a client device 130. In some exemplary embodiments, the processor 1210 may be a single-threaded processor. Alternatively, the processor 1210 may be a multi-threaded processor. The processor 1210 is capable of processing instructions stored in memory 1220 and / or storage device 1230 to display graphical information for a user interface provided via the input / output device 1240.

[0145] Memory 1220 is a computer-readable medium, such as volatile or non-volatile, that stores information within the computing system 1200. Memory 1220 can store, for example, data structures representing a configuration object database. Storage device 1230 can provide persistent storage for the computing system 1200. Storage device 1230 may be a solid-state drive, floppy disk device, hard disk device, optical disk device, or tape device, or other suitable persistent storage means. Input / output device 1240 provides input / output operations for the computing system 1200. In some exemplary embodiments, input / output device 1240 includes a keyboard and / or pointing device. In various embodiments, input / output device 1240 includes a display device for displaying a graphical user interface.

[0146] According to some exemplary embodiments, the input / output device 1240 may provide input / output operations for network devices. For example, the input / output device 1240 may include an Ethernet port or other networking port to communicate with one or more wired and / or wireless networks (e.g., a local area network (LAN), a wide area network (WAN), the Internet).

[0147] In some exemplary embodiments, the computing system 1200 can be used to run various interactive computer software applications that can be used for organizing, analyzing, and / or storing various forms of data. Alternatively, the computing system 1200 may be used to run any type of software application. These applications may be used to perform various functions, such as planning functions (e.g., generating, managing, and editing spreadsheet documents, word processing documents, and / or any other objects), computing functions, communication functions, etc. An application may include various add-in functions or may be a standalone computing product and / or function. When activated within an application, functionality may be used to generate a user interface provided via the input / output device 1240. The user interface may be generated by the computing system 1200 and presented to the user (e.g., on a computer screen monitor).

[0148] One or more aspects or features of the subject matter described herein may be realized in digital electronic circuits, integrated circuits, specially designed ASICs, field-programmable gate array (FPGA) computer hardware, firmware, software, and / or combinations thereof. These various aspects or features may include implementations in one or more computer programs executable and / or interpretable on a programmable system which includes at least one programmable processor, which may be for special or general purposes, coupled to receive data and instructions from a storage system, at least one input device, and at least one output device, and to transmit data and instructions to the storage system, at least one input device, and at least one output device. A programmable system or computing system may include a client and a server. The client and server are generally located geographically separated from each other and typically interact through a communication network. The client-server relationship arises from computer programs running on each computer and having a client-server relationship with each other.

[0149] These computer programs, sometimes called programs, software, software applications, applications, components, or code, contain machine instructions for a programmable processor and may be implemented in high-level procedural and / or object-oriented programming languages, as well as / or assembly / machine languages. As used herein, the term “machine-readable medium” means any computer program product, apparatus, and / or device used to provide machine instructions and / or data to a programmable processor, including, for example, magnetic disks, optical disks, memory, and programmable logic devices (PLDs), and includes machine-readable medium that receives machine instructions as machine-readable signals. The term “machine-readable signals” means any signals used to provide machine instructions and / or data to a programmable processor. Machine-readable medium can store such machine instructions non-temporarily, for example, non-temporarily, solid-state memory, magnetic hard drives, or any equivalent storage medium. Machine-readable medium can, alternatively or additionally, store such machine instructions temporarily, for example, a processor cache or other random-access memory associated with one or more physical processor cores.

[0150] To provide user interaction, one or more aspects or features of the subject matter described herein may be implemented on a computer having, for example, a display device such as a cathode ray tube (CRT), liquid crystal display (LCD), or light-emitting diode (LED) monitor for displaying information to the user, and a keyboard, and a pointing device such as a mouse or trackball by which the user can provide input to the computer. User interaction may also be provided using other types of devices. For example, repetition provided to the user may be any form of sensory repetition, such as visual repetition, auditory repetition, or tactile repetition, and input from the user may be received in any form, including acoustic input, voice input, and tactile input. Other possible input devices include touchscreens, or other touch-sensing devices such as single-point or multi-point resistive or capacitive trackpads, speech recognition hardware and software, optical scanners, optical pointers, digital image acquisition devices, and associated interpretation software.

[0151] In the above specification and claims, phrases such as “at least one of ~” or “one or more of ~” may precede a list of consecutive elements or features. The term “and / or” may also be used in an enumeration of two or more elements or features. Unless implicitly or explicitly contradicted by the context in which it is used, such phrases are intended to mean any of the enumerated elements or features individually, or any of the enumerated elements or features in combination with any of the other enumerated elements or features. For example, the phrases “at least one of A and B,” “one or more of A and B,” and “A and / or B” are intended to mean “A only,” “B only,” or “A and B together,” respectively. A similar interpretation is intended for enumerations containing three or more items. For example, the phrases “at least one of A, B, and C,” “one or more of A, B, and C,” and “A, B, and / or C” are intended to mean, respectively, “A only, B only, C only, A and B together, A and C together, B and C together, or A, B, and C together.” The use of the term “based on” in the above and in the claims is intended to mean “at least partially based on,” thereby allowing features or elements that are not enumerated.

[0152] The subject matter described herein may be embodied in systems, apparatus, methods, and / or articles, depending on the desired configuration. The embodiments described above do not necessarily represent all embodiments of the subject matter described herein. Rather, they are merely some examples that correspond to aspects relating to the described subject matter. While some variations have been detailed above, other modifications and additions are possible. In particular, further features and / or variations may be provided in addition to those described herein. For example, the embodiments described above may be directed to various combinations and subcombinations of the disclosed features, and / or combinations and subcombinations of some of the further features disclosed above. In addition, the logical flows depicted in the accompanying drawings and / or described herein do not necessarily require a specific order or sequence shown to achieve the desired result. Other embodiments may fall within the scope of the following claims.

Claims

1. A computer implementation method, At a minimum, based on a first positron emission tomography (PET) scan and a first computed tomography (CT) scan from a first time point, a first tumor mask corresponding to a first lesion present in the first PET scan and the first CT scan is determined. At a minimum, based on a second PET scan and a second CT scan from a second time point, a second tumor mask corresponding to a second lesion present in the second PET scan and the second CT scan is determined. At a minimum, apply a longitudinal segmentation model to update the first tumor mask and the second tumor mask based on the first PET scan, the first CT scan, the second PET scan, and the second CT scan, and Determining the response to disease treatment based on at least one of the first updated tumor mask and the second updated tumor mask. Methods that include...

2. The method according to claim 1, wherein the first tumor mask identifies a first plurality of pixels in each of the first PET scan and the first CT scan depicting the first lesion, and the second tumor mask identifies a plurality of pixels from the second PET scan and the second CT scan depicting the second lesion.

3. The method according to claim 1 or 2, further comprising registering the first CT scan, the first PET scan, the second CT scan, and the second PET scan in order to align the first CT scan and the first PET scan with the second CT scan and the second PET scan.

4. At a minimum, the second lesion is identified as a new lesion based on the first updated tumor mask and the second updated tumor mask, and In response to the identification of the second lesion as the new lesion, the response to the treatment is determined to be a progressive disease (PMD). The method according to any one of claims 1 to 3, further comprising:

5. At a minimum, the distance between the first lesion and the second lesion is determined based on the first updated tumor mask and the second updated tumor mask. Identifying the second lesion as the new lesion based at least on the distance between the first lesion and the second lesion satisfying one or more thresholds, and Identifying the second lesion as the same lesion as the first lesion, at least based on the fact that the distance between the first lesion and the second lesion does not satisfy one or more thresholds. The method according to claim 4, further comprising:

6. The method according to claim 4 or 5, further comprising determining the response of the disease to the treatment based at least on a change in metabolic activity indicated by the lesion between the first time point and the second time point, in response to the determination that the first lesion and the second lesion are the same lesion.

7. The change in metabolic activity between the first time point and the second time point is at least Based at least the first updated tumor mask and the first PET scan, determine the first level of metabolic activity indicated by the lesion at the first time point, Based at least the second updated tumor mask and the second PET scan, determine the second level of metabolic activity indicated by the lesion at the second time point, and Determining the change in metabolic activity between the first time point and the second time point based on at least the first level of metabolic activity and the second level of metabolic activity. The method according to claim 6, as determined by...

8. The method according to claim 7, wherein the response to the treatment is determined to be a progressive metabolic disease (PMD) based at least on the change in metabolic activity between the first time point in time when a first threshold is met and the second time point in time.

9. The method according to claim 8, wherein the response to the treatment is determined to be no metabolic response (NMR) at least on the basis that the change in metabolic activity between the first time point and the second time point satisfies a second threshold but does not satisfy the first threshold.

10. The method according to claim 9, wherein the response to the treatment is determined to be a partial metabolic response (PMR) based at least on the fact that the change in metabolic activity between the first time point and the second time point does not satisfy the first threshold and the second threshold.

11. The method according to any one of claims 7 to 10, wherein the first level of metabolic activity corresponds to a first standardized uptake value (SUV), and the second level of metabolic activity corresponds to a second standardized uptake (SUV) value.

12. The method according to any one of claims 7 to 10, wherein each of the first level of metabolic activity and the second level of metabolic activity corresponds to the maximum, minimum, median, mean, or mode of metabolic activity shown by the lesion at the corresponding time point.

13. The method according to any one of claims 1 to 12, wherein the first CT scan and the first PET scan are performed before treatment of the disease, and the second CT scan and the second PET scan are performed after treatment of the disease.

14. At a minimum, the change in tumor volume is determined based on the first updated tumor mask and the second updated tumor mask, and The method according to any one of claims 1 to 13, further comprising determining the response of the disease to the treatment based on at least a change in the volume of the tumor.

15. At a minimum, based on the first updated tumor mask and the second updated tumor mask, determine the dispersion of the first change in metabolic activity and / or the second change in tumor volume between the first and second time points across various lesions, and The method according to any one of claims 1 to 13, further comprising determining the response to the treatment based at least on the dispersion of the changes in metabolic activity and / or tumor volume between the first time point and the second time point as indicated by the various lesions.

16. The method according to any one of claims 1 to 15, further comprising determining the progression of the disease based on at least the first updated tumor mask and the second updated tumor mask.

17. The method according to any one of claims 1 to 16, wherein the first tumor mask is determined by applying a segmentation model to the first PET scan and the first CT scan, and the second tumor mask is determined by applying the segmentation model to the second PET scan and the second CT scan.

18. The method according to any one of claims 1 to 17, wherein the longitudinal segmentation model is an artificial neural network or a visual transducer.

19. The method according to any one of claims 1 to 18, wherein each of the first CT scan, the first PET scan, the second CT scan, and the second PET scan is a three-dimensional volume comprising a plurality of two-dimensional patches.

20. The method according to any one of claims 1 to 19, wherein each pixel in the first PET scan and the second PET scan is associated with an intensity value corresponding to a level of metabolic activity.

21. The method according to any one of claims 1 to 20, wherein each pixel in the first CT scan and the second CT scan is associated with an intensity value corresponding to tissue density or X-ray attenuation.

22. The method according to any one of claims 1 to 21, further comprising training the longitudinal segmentation model to update two or more tumor masks, wherein each of the two or more tumor masks is generated from positron emission tomography (PET) scans and computed tomography (CT) scans from a single point in time.

23. The method according to any one of claims 1 to 22, wherein the response of the disease to the treatment is a complete metabolic response (CMR) or an incomplete metabolic response (non-CMR).

24. The method according to any one of claims 1 to 22, wherein the response of the disease to the treatment is either a responder or a non-responder.

25. The method according to any one of claims 1 to 22, wherein the response of the disease to the treatment is a total metabolic response (CMR), a partial metabolic response (PMR), no metabolic response (NMR), or a progressive metabolic disease (PMD).

26. Extracting a first patch containing the first lesion associated with the first tumor mask from the first PET scan and the first CT scan, Extracting a second patch containing the second lesion associated with the second tumor mask from the second PET scan and the first CT scan, and Applying the longitudinal segmentation model to the first patch and the second patch in order to update the first tumor mask and the second tumor mask, respectively. The method according to any one of claims 1 to 25, further comprising:

27. It is a system, At least one data processor, A memory for storing instructions that, when executed by the at least one data processor, result in an operation including the method according to any one of claims 1 to 26, and A system equipped with these features.

28. A non-temporary computer-readable medium storing instructions that, when executed by at least one data processor, result in an operation including the method according to any one of claims 1 to 26.

Images