Measuring changes in tumor volume in medical images

Abstract

The techniques disclosed herein facilitate tracking the extent to which the size of a biological structure changes over time. In some cases, an initial biological image (collected at a first time) can be segmented to characterize the boundary and size. Subsequent biological images can be processed to identify deformation and / or transformation variables (e.g., one or more Jacobian matrices and / or one or more Jacobian determinants). The deformation and / or transformation variables and the initial segmentation can be used to predict the size of the biological structure at a subsequent time. The predicted size can inform treatment recommendations.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the benefit of and priority to U.S. Provisional Patent Application No. 62 / 958,926, filed January 9, 2020, and U.S. Provisional Patent Application No. 63 / 017,946, filed April 30, 2020, each of which is incorporated herein by reference in its entirety for all purposes. [Background technology]

[0002] background Cancer is a leading cause of death in many countries. Identifying effective treatments requires both effectively diagnosing early symptoms and effectively characterizing the degree of effectiveness of each treatment administered to a subject to provide an opportunity to modify and / or adjust treatment strategies.

[0003] For example, statistics from the American Cancer Society indicate that lung and bronchial cancer is the leading cause of death in the United States, with an estimated 228,150 new cases and 147,670 deaths in 2019. The cost of developing an approved cancer drug is estimated to be between $200 million and $2.9 billion. However, evaluating the effectiveness of newly developed cancer drugs can involve labor-intensive manual review of collected data, which increases the cost and time consumption of the evaluation.

[0004] For example, images (e.g., CT images) can be analyzed to monitor target tumors using the RECIST criteria. The RECIST criteria stipulate that the longest axis diameter of the tumor is used as a parameter for monitoring the progression of solid tumors. The efficacy of the experimental drug is then calculated based on the change in tumor diameter.

[0005] Considering diameter change works well for oval tumors. However, for non-oval tumors, the change in tumor volume after drug delivery is a better indicator of drug efficacy. For non-oval tumors, measuring the maximum diameter can correspond to a completely different change in tumor volume. Tumor volume measurement is also being investigated as a potentially more sensitive outcome metric for clinical trials in oncology. Volume change can be measured by delineating specific lesions at baseline and follow-up. This manual segmentation task is time-consuming, user-dependent, and therefore error-prone. Furthermore, manual segmentation often involves employing input from trained experts, which can add cost.

[0006] Therefore, it would be advantageous to identify an automated approach for reliably and accurately tracking tumor volume. Summary of the Invention

[0007] overview In some embodiments, techniques are disclosed for tracking the volume of a biological structure using images collected at different times and using a contour of the biological structure generated from one of the images.

[0008] For example, a baseline image (e.g., a three-dimensional image) depicting the tumor can be collected at a baseline time point. The tumor can be delineated (e.g., segmented) by a human annotator to define a mask for the baseline time point. The delineation can be performed using one or more semi-automated segmentation tools. Another "subsequent" image (e.g., a three-dimensional image) showing the tumor at a subsequent time point can be processed to estimate the volume (or volume change) of the tumor at the subsequent time point. The processing can include performing a nonlinear registration technique so that individual points, boundaries, or other geometric features on the subsequent image are associated with corresponding features in the initial image. The relationship between the original and subsequent features (e.g., distance between points, distortion between lines, etc.) and the size of the tumor at the baseline time point can be used to estimate the size of the tumor at the subsequent time point. The relationship can also, or alternatively, be used to estimate a segmentation and / or mask for the subsequent image. Thus, the size of the tumor at the subsequent time point can be estimated without delineating, segmenting, or annotating the tumor in the subsequent image. Such an approach can improve efficiency, reduce or eliminate reliance on the use of anatomical landmarks, and / or reduce the extent to which sequential assessments are erroneous due to different types of subjective features. Volumetric tracking can be used (for example) to estimate current disease progression or predict future disease progression, evaluate the effectiveness of particular treatments, and / or inform the selection of new treatments.

[0009] In some embodiments, a computer-implemented method is provided. A first image is accessed. The first image may depict a portion of a subject and may have been captured first. A mask is generated for the first image. The mask outlines a particular biological structure depicted in the first image. A second image is accessed. The second image may depict a similar portion of the subject and may have been captured at a second time after the first time. The second image is registered to the first image. For each voxel of at least some voxels in the mask, a transformation variable is calculated using the registration. The transformation variable characterizes the displacement (e.g., spatial difference) between a first position of the voxel in the first image and a second position of the corresponding voxel in the second image. The transformation variable is used to estimate a size of the biological structure at the second time. The estimated size of the biological structure at the second time is output.

[0010] In some cases, calculating the transformation variables includes calculating a spatial Jacobian matrix for the voxels using the registration, and calculating a Jacobian determinant for the voxels using the spatial Jacobian matrix for the voxels, wherein an estimated size of the biological structure at a second time is generated using the Jacobian determinant for the voxels.

[0011] In some cases, generating the estimated size of the biological structure may include summing a Jacobian determinant over multiple voxels in the mask.

[0012] In some cases, generating the estimated size of the biological structure can include summing or averaging Jacobian determinants across multiple voxels in the mask, and estimating the size the biological structure was at the second time can include determining the product of the sum or average of the Jacobian determinants across multiple voxels in the mask and the estimated volume of the biological structure at the first time.

[0013] In some cases, generating the estimated size of the biological structure may include summing Jacobian determinants across at least a plurality of voxels in the mask, and the estimated size of the biological structure at the second time may be determined based on the Jacobian determinants.

[0014] In some cases, aligning the second image to the first image may use a non-linear B-spline transformation.

[0015] In some cases, identifying the mask for the first image may include processing detected user input that defines an outline of a particular biological structure.

[0016] In some cases, each of the first and second images may include a CT scan, an MRI image, or an X-ray.

[0017] In some embodiments, a system is provided that includes one or more data processors and a non-transitory computer-readable storage medium containing instructions that, when executed on the one or more data processors, cause the one or more data processors to perform some or all of one or more of the methods disclosed herein.

[0018] In some embodiments, a computer program product is provided that is tangibly embodied in a non-transitory machine-readable storage medium and includes instructions configured to cause one or more data processors to perform some or all of one or more of the methods disclosed herein.

[0019] Some embodiments of the present disclosure include a system including one or more data processors. In some embodiments, the system includes a non-transitory computer-readable storage medium containing instructions that, when executed on the one or more data processors, cause the one or more data processors to perform some or all of one or more methods and / or some or all of one or more processes disclosed herein. Some embodiments of the present disclosure include a computer program product tangibly embodied in a non-transitory machine-readable storage medium containing instructions configured to cause one or more data processors to perform some or all of one or more methods and / or some or all of one or more processes disclosed herein.

[0020] The terms and expressions which have been employed are used as terms of description rather than of limitation, and there is no intention in the use of such terms and expressions to exclude equivalents of the features shown and described or portions thereof, but it is recognized that various modifications are possible within the scope of the invention as claimed. Thus, although the claimed invention has been specifically disclosed by embodiments and optional features, it will be understood that modifications and variations of the concepts disclosed herein may be resorted to by those skilled in the art, and that such modifications and variations are deemed to be within the scope of the invention as defined by the appended claims. [Brief explanation of the drawings]

[0021] The present disclosure is described in conjunction with the accompanying drawings, in which:

[0022] [Figure 1] 1 illustrates an exemplary tumor tracking network according to some embodiments.

[0023] [Figure 2] 1 shows a flowchart of a process for estimating tumor size according to some embodiments.

[0024] [Figure 3] 10 shows a demonstration of calculating the volume of a follow-up volume using a Jacobian determinant matrix, according to some embodiments.

[0025] [Figure 4] 10 shows exemplary data illustrating a comparison of volume changes calculated using the Jacobian method with actual volume changes.

[0026] [Figure 5] 1 shows an exemplary pipeline for registering data from exemplary subject A with CT lung data.

[0027] [Figure 6] Shown are overlays of baseline axial slices and follow-up slices corresponding to CT images of the lungs of three subjects.

[0028] [Figure 7] For example, the alignment result for subject B is shown.

[0029] [Figure 8] For example, the alignment result for subject C is shown.

[0030] [Figure 9] 10 shows the mean and difference of paired volume estimates generated using manual annotation or by using the Jacobian method, according to one embodiment.

[0031] [Figure 10A] 1 shows a comparison of volume changes calculated using an embodiment of the Jacobian method with actual volume changes in lung lesions. [Figure 10B] 1 shows a comparison of volume changes calculated using an embodiment of the Jacobian method with actual volume changes in lung lesions. [Figure 10C]1 shows a comparison of volume changes calculated using an embodiment of the Jacobian method with actual volume changes in lung lesions. [Figure 10D] 1 shows a comparison of volume changes calculated using an embodiment of the Jacobian method with actual volume changes in lung lesions.

[0032] [Figure 11] 1 shows an example of poor annotation from a radiologist annotator.

[0033] [Figure 12] 1 shows a comparison of volume changes calculated using an embodiment of the Jacobian method with actual volume changes for lung lesions with volume changes <30%.

[0034] In the accompanying drawings, similar components and / or features may have the same reference label. Furthermore, various components of the same type may be distinguished by following the reference label with a dash and a second label that distinguishes between the similar components. When only a first reference label is used in this specification, the description is applicable to any of the similar components having the same first reference label, regardless of the second reference label. DETAILED DESCRIPTION OF THE INVENTION

[0035] Detailed Description I. Overview The systems, methods, and software disclosed herein can facilitate reducing the time commitment, costs, and errors involved in monitoring and characterizing tumors. More specifically, annotations of the tumor in a first baseline image and subsequent images can be used to predict the size (e.g., volume) of the tumor at the time the subsequent image is collected. The size can be estimated (for example) by registering the subsequent image to the first baseline image, automatically determining one or more deformation variables (e.g., one or more Jacobian matrices and / or one or more Jacobian determinants), and collecting and processing the deformation variables and annotations performed using the first baseline image. For example, the Jacobian matrices can be calculated at the image level.

[0036] More specifically, a baseline image depicting a body portion of a subject can be collected using an imaging device at a first time point. The subject can include a subject diagnosed with cancer (e.g., lung cancer, bronchial cancer, breast cancer, prostate cancer, colorectal cancer, or any other type of cancer), and the body portion can depict some or all of one or more tumors.

[0037] The baseline image can be transmitted to and presented on an annotator's device (e.g., a radiologist's device). Input received at the annotator's device can be used to identify which portions of the baseline image correspond to particular biological structures. For example, the input can correspond to the boundaries of particular biological structures. In some cases, the volume of the biological structure can be estimated based on the baseline image and the boundaries. The boundaries can be used to generate a mask (e.g., each voxel within the boundaries is assigned a value of "1" and each voxel outside the boundaries is assigned a value of "0").

[0038] Another image can depict a similar or identical portion of the subject's body, but may be collected at a subsequent time point (e.g., a defined number of days, weeks, months, or years from the first time point). The other image can be registered to the baseline image. The registration can be performed by performing a spline registration (e.g., B-spline registration), an affine transformation, or a joint entropy or mutual information based transformation. In some cases, before registering the other image, the baseline image can be cropped around the depicted biological structure (e.g., using a box shape extending to a specific margin, such as a 30-voxel margin around the maximum length and width of the depicted biological structure). This cropping can reduce the time and processing commitment for registering the other image.

[0039] The registration can be used to identify a deformation field comprising a vector image in which each voxel contains a displacement vector. A spatial Jacobian matrix can be defined as the first derivative of the deformation field. The determinant of the Jacobian matrix can indicate the degree of local compression or expansion (a value less than 1 indicates local compression, and a value greater than 1 indicates expansion). The voxel-specific determinant can be multiplied by a mask, and the sum or average over pixels can predict the change in volume of the biological structure. This change can be multiplied by the estimated volume of the biological structure at a first time point to estimate the volume change of the biological structure at a subsequent time point. The volume change and volume at the baseline time point can be used to estimate the volume of the biological structure at a subsequent time point.

[0040] The volume of a biological structure can be estimated across multiple time points while using only boundary detection at a single time point. Thus, if the boundary is defined based on input provided by an annotator, such input need only be received for a single image and / or images associated with a single time point. This can reduce the cost and time spent tracking the size of biological structures. Furthermore, the automated approach can reduce discrepancies in object detection.

[0041] The baseline and / or subsequent images may include CT images, MRI images, or X-rays. The biological structures may include tumors, lesions, cell types, vasculature, etc. The baseline and subsequent images may, but need not, be acquired by the same imaging device and / or the same type of imaging device.

[0042] In some cases, a condition is evaluated to determine whether to estimate tumor volume at subsequent time points without relying on annotations at the subsequent time points (e.g., instead, using a Jacobian-based approach performed using data from registering images from the subsequent time points to images from the baseline time point). For example, the condition may indicate that the Jacobian-based approach should be used when a first time point corresponding to the baseline image and a subsequent time point corresponding to another image are within a predetermined time period. As another example, the condition may indicate that the Jacobian-based approach should be used if a metric indicating the quality or reliability of the registration of the other image to the baseline image exceeds a predetermined threshold.

[0043] II. Exemplary Tumor Tracking Network FIG. 1 illustrates an exemplary tumor tracking network 100 according to some embodiments. The tumor tracking network 100 includes an imaging system 105 configured to collect one or more images of a portion of a subject's body. Each image can depict at least a portion of one or more biological structures (e.g., at least a portion of one or more tumors and / or at least a portion of one or more organs). The subject can include a person who has been diagnosed with a particular disease or has a probable diagnosis of a particular disease. The particular disease can include cancer or a particular type of cancer (e.g., lung cancer or non-small cell lung cancer).

[0044] The images include one or more two-dimensional images and / or one or more three-dimensional images. The image generation system 105 may include (for example) a computed tomography (CT) scanner, an X-ray machine, or a magnetic resonance imaging (MRI) machine. The images may include radiological images, CT images, X-ray images, or MRI images. The images may have been collected without a contrast agent being administered to the subject, or after a contrast agent has been administered to the subject. In some cases, the image generation system 105 may first collect a set of two-dimensional images and use the two-dimensional images to generate the three-dimensional image.

[0045] The images acquired by the imaging system 105 may be acquired without a contrast agent being administered to the subject or after a contrast agent has been administered to the subject. The subject being imaged may include a subject who has been diagnosed with cancer, has a possible or preliminary diagnosis of cancer, and / or has symptoms consistent with cancer or a tumor.

[0046] The image generation system 105 can store the collected images in an image data store 110, which can include (for example) a cloud data store. Each image can be stored in association with one or more identifiers, such as an identifier for the subject and / or an identifier for a caregiver associated with the subject. Each image may also be stored in association with the date the image was collected.

[0047] In some cases, the one or more images are further utilized by an annotation system 115, which can facilitate identification of annotations of one or more tumors depicted in the images. The annotations can include contours or boundaries that define a mask for a particular biological structure (e.g., in the case of a tumor). The mask may be defined (for example) to include a value of 0 over an area outside the annotated biological structure region or volume. The mask can be defined to include a value of 1 over an area inside the inner annotated biological structure region or volume. In some cases, the mask can be defined to be a value of 1 over the perimeter and / or boundary of the annotated biological structure region or volume.

[0048] The annotation system 115 controls and / or utilizes an annotation interface that presents part or all of one or more images and includes an input component for identifying one or more boundaries, perimeters, and / or regions. For example, the annotation system 115 may include a “pencil” or “pen” tool that can be positioned based on the input and generate markings along the identified boundaries. In some cases, the annotation system 115 facilitates the identification of closed shapes, such that small gaps in line segments are connected. In some cases, the annotation system 115 facilitates the identification of potential boundaries by performing (for example) intensity and / or contrast analysis. Thus, the annotation system 115 may support tools that facilitate performing semi-automated segmentation. The annotation system 115 may be a web server that provides an interface via a website.

[0049] The annotation interface is utilized for an annotation device 120 that can be associated with, owned by, used by, and / or controlled by a human annotator. The annotator can be (for example) a radiologist, pathologist, or oncologist. The annotation device 120 receives input from an annotator user and sends a representation of the input (e.g., an identification of a set of pixels) to the annotation system 115. The annotation system 115 can store the representation of the annotation (e.g., in the image data store 110). The representation can include (for example) a set of pixels and / or a set of voxels.

[0050] The annotation system 115 uses the annotations to generate a mask for the image. The mask may include binary values ​​that are set to 0 outside the annotated boundary and set to 1 inside the annotated boundary. The mask image may be generated by multiplying the mask with the original image.

[0051] Images annotated using annotation system 115 and annotator 120 include images acquired at one or more baseline time points (also referred to herein as one or more baseline times). Images collected at baseline time points may be referred to herein as baseline images. A baseline time point may be the time the first image depicting a particular biological structure was collected, the time the first image depicting a particular biological structure recognized by an annotator was collected, and / or the time considered to be a baseline for future comparison (e.g., by a human technician or caregiver). In some cases, a baseline time point is defined for each subject as the date one or more medical images (e.g., radiology images) were collected, the one or more medical images depicting one or more tumors. In some cases, a baseline time point is defined for each tumor and is defined as the first date a medical image depicted the tumor and / or the first date a tumor was identified and annotated on the image.

[0052] For each subject and / or each tumor, imaging system 105 (or another imaging system) collects one or more other images of the subject and / or tumor at a subsequent time after the baseline time point associated with the subject and / or tumor. The one or more other images may be of the same type as the image collected at the baseline time point (e.g., a CT image, an MRI image, or an X-ray). The other images collected at the subsequent time points may be referred to herein as subsequent images.

[0053] The subsequent time can be (for example) at least 1 week, at least 1 month, at least 2 months, at least 3 months, at least 6 months, at least 1 year, or at least 2 years after the baseline time point. The subsequent time can also be less than 15 years, 10 years or less, 8 years or less, 5 years or less, 3 years or less, 2 years or less, 1 year or less, 9 months or less, 6 months or less, 3 months or less, 2 months or less, or 1 month or less after the baseline time point. For example, another image can be collected between 1 month and 2 months after the baseline time point.

[0054] The one or more other images can depict the same or similar portion of the subject as depicted in the image collected at the baseline time point. For example, the one or more first images collected at the baseline time point and the one or more second images collected at the subsequent time points can each depict the same biological structure (e.g., in its entirety or in cross-section). If the one or more images collected at the baseline time point and the one or more images collected at the subsequent time points are three-dimensional images, the image viewpoints of the subsequent images can be the same or similar (e.g., within 30°, within 20°, or within 10° along each of one, two, or three viewing angles) to the viewpoint of the baseline image.

[0055] In some cases, the other images may be defined to be and / or include the area indicated by the human user. The area may correspond to (for example) an oval, rectangular, and / or cuboid area. In some cases, each of the other images (collected at subsequent times) and each of the images (collected at the baseline time) are the same size (e.g., a predetermined size), so that a buffer can be introduced around the area to generate a second image of a target size.

[0056] In some cases, each subsequent image is automatically or semi-automatically processed to identify the annotation of each of the one or more tumors.

[0057] Automatic identification can be performed without input from a human annotator. Semi-automatic identification can be performed based on input from a human annotator that is less than complete annotation. For example, the annotation system 115 can be configured to present an image and an input tool configured to be positioned and sized to identify a region that generally corresponds to a representation of a tumor. Thus, the region can include a representation of the tumor (e.g., a cross-section of the tumor or a three-dimensional volume of the tumor) and can further include a representation of one or more non-tumor biological structures. The input tool can include a box tool configured to be positioned and sized to define a rectangular region, a square region, a cubic volume, or a rectangular parallelepiped volume.

[0058] Image processing system 125 (which may include, for example, a remote and / or cloud-based computing system) is configured to predict, for each of one or more tumors, tumor annotations identifying tumor boundaries. Image processing system 125 includes a preprocessing controller 130 that initiates and / or controls image preprocessing. Preprocessing may include (for example) converting images to a predetermined format, resampling images to a predetermined sampling size, normalizing images, cropping images to a predetermined size, modifying images to have a predetermined resolution, registering multiple images, generating a three-dimensional image based on multiple two-dimensional images, generating one or more images with different (e.g., target) viewpoints, adjusting (e.g., standardizing or normalizing) intensity values, and / or adjusting color values. In some cases, for each tumor, preprocessing controller 130 generates a cropped image in which the image is cropped to a region or volume (e.g., a rectangular region, a square region, a rectangular volume, or a cubic volume) identified via input indicating a region containing a depiction of the tumor. In some cases, for each tumor, the preprocessing controller 130 generates a cropped image in which the image is cropped to a region or volume set equal to the identified region of the input (e.g., received at the annotation device 120) plus a buffer (e.g., corresponding to a predetermined number of pixels or voxels). For example, the baseline image can be cropped to a boundary defined to be a predetermined distance (e.g., 10 mm, 20 mm, 30 mm, 50 mm, or 1 cm) beyond the boundary identified in the annotation. As another example, the baseline image may be cropped to have a predetermined shape (e.g., rectangular, rectangular prism, elliptical, etc.), with each shape dimension defined to extend from a minimum position to a maximum position along an axis, and each of the minimum and maximum positions defined to be the boundary plus the buffer.

[0059] The inclusion of a buffer can facilitate subsequent alignment analysis. If the buffer extension extends beyond the image edge of the original image, cropping can be performed such that the crop region or volume extends to the edge of the original image.

[0060] The alignment controller 135 aligns the image or a pre-processed version thereof to the corresponding baseline image to generate an alignment. M (T(x)) F In terms of finding a coordinate transformation T(x) that spatially aligns with (x), I M (x) is a moving image, and I F (x) is a fixed image. The registration can be generated using (for example) a deformable registration technique. The registration can be generated by using a spline function (e.g., a B-spline function), an affine transformation, or a transformation based on joint entropy or mutual information. For example, the registration may be performed using a nonlinear B-spline transformation, such as the transformation Tμ(x) shown in equation (1): TIFF0007802671000001.tif11170where x k are the control points, and β 3 (x) is a cubic multidimensional B-spline polynomial, and p k is the B-spline coefficient vector, σ is the B-spline control point spacing, and N x is the set of all control points in the compact support of the B-spline at x.

[0061] The registration may include performing a function that compares intensities and / or features between the baseline image and the subsequent image associated with a structure using (for example) a correlation function or a feature-based function. The registration may include determining a transformation function that relates the baseline image and the subsequent image. The registration may include performing a technique that compares spatial domain characteristics between the baseline image and the subsequent image, or that compares spatial frequency information from the baseline image with spatial frequency information from the subsequent image. The registration may include using a spline registration function (e.g., B-spline registration), an affine transformation, or a joint entropy or mutual information based transformation. The registration may be configured to determine, for each voxel of one or more voxels (or pixels) in the subsequent image, which voxel corresponds to which voxel or pixel in the baseline image. The registration (produced as a result of the registration) may, for each subsequent image voxel of a set of voxels in the subsequent image, associate the subsequent image voxel with a voxel from the baseline image. The registration may additionally or alternatively indicate, for each subsequent image voxel of a set of voxels in the subsequent image, a displacement vector indicating the positional separation between the subsequent image voxel and the corresponding voxel in the baseline image.

[0062] The deformation detector 140 uses the registration to characterize the deformation between the depiction of the tumor between the baseline and subsequent images. The deformation detector 140 can use the registration to generate a deformation field (e.g., a continuous deformation field). The deformation field can include, for each voxel in the set of voxels, or for each pixel in the set of pixels, a displacement vector in physical coordinates that indicates the positional difference between the location of the corresponding pixel or voxel in the baseline image (or a preprocessed version thereof) and the subsequent image (or a preprocessed version thereof). The set of pixels or voxels can include all pixels or voxels within a mask defined for the baseline image (e.g., associated with non-zero values ​​in the mask), potentially within an annotated boundary defined for the baseline image, or within the baseline image.

[0063] The deformation detector 140 uses (1) the annotations and / or masks associated with the baseline image and (2) the registration (e.g., the deformation field) to calculate one or more Jacobian matrices (e.g., one or more spatial Jacobian matrices) and / or one or more Jacobian determinants. A Jacobian matrix can be defined as the first derivative of the deformation field. In some cases, the Jacobian matrix is ​​used to determine a Jacobian (or other deformation variable) for each voxel in the subsequent image. The Jacobian can represent the degree of relative movement of a voxel from a first time to a second time. A value greater than 1 can represent local expansion, and a value less than 1 can represent local compression. The Jacobian is advantageously invariant to linear registration, so that the subsequent image does not need to be registered with the baseline image.

[0064] The volume detector 145 uses the deformation variables (e.g., Jacobian determinants) to predict the volume of the tumor at subsequent times. For example, the volume detector 145 can multiply, for each voxel in the subsequent image, the voxel-related Jacobian determinant by the corresponding value of a mask generated using the annotations of the baseline image. Thus, a product can be generated for each voxel based on the corresponding deformation variables and the corresponding value in the mask.

[0065] The volume detector 145 predicts the volume change of the tumor at a subsequent time by generating statistics based on voxel-related products. For example, a sum (or other statistic) of voxel-specific products (associated with multiple voxels) can be generated, which can indicate the predicted change in volume between the baseline and the subsequent time. The absolute volume at the subsequent time can be estimated to be the volume at the baseline time and the predicted change. In some cases, the change in volume is calculated using equation (2): TIFF0007802671000002.tif11170 Where, M B is the radiologist-annotated baseline mask (M B ) and J is the Jacobian matrix.

[0066] Another computational approach for predicting size change involves calculating the sum or average of a Jacobian determinant over a set of voxels, the set of voxels including those in a tumor mask (e.g., whose corresponding mask values ​​are non-zero). The estimated difference between the tumor at the baseline time point and the subsequent time point can be defined as or based on the sum (e.g., or average) of the Jacobian determinant over the set of voxels. The estimated size difference can be added to the size of the tumor at the baseline time to generate an estimated size of the tumor at the subsequent time point. In some cases, the values ​​in the mask are used as weights applied to corresponding voxels to generate a weighted sum or weighted average. In some cases, normalization is applied (e.g.,) to normalize the interim results to generate the estimated size. The normalization can be performed using the size of the biological structure at the baseline time when the baseline images were collected.

[0067] In some cases, for each tumor, the tumor volume change is defined as the tumor size at the subsequent time minus the tumor size at the baseline time. For a given subject, a statistical value (e.g., a mean value) can be defined as the tumor volume change calculated using the baseline annotations and the statistic (e.g., average) of the tumor volume change using the transformation variable.

[0068] The output is returned to the user device 150 (e.g., presented to or transmitted to the user device). The output can include a predicted volume of the tumor at a subsequent time. In some cases, rather than or in addition to outputting the predicted volume of the tumor, another result is determined and output based on the predicted volume. For example, a predicted cumulative volume of multiple tumors can be determined and output (e.g., by summing the estimated sizes of each of the multiple tumors). As another example, the pattern and extent to which one or more tumors have grown and / or shrunk can be used to identify potential treatment approaches. As yet another example, a predicted lesion change can be defined for each tumor (e.g., as the predicted tumor size at the subsequent time minus the tumor size at the baseline time), and the returned results can include statistics calculated based on the lesion change (e.g., cumulative additive predicted change, mean predicted percentile change, median percentile change, ratio of cumulative predicted subsequent-time tumor volume to cumulative baseline-time tumor volume, etc.).

[0069] In some cases, rather than or in addition to outputting a predicted tumor volume, a result is generated and / or output that predicts the extent to which a current treatment approach is effectively treating the subject's cancer. For example, the rules may indicate that an effective treatment is associated with tumor shrinkage (or cumulative shrinkage of all tumors) of at least a predetermined threshold amount.

[0070] In some cases, post-processing techniques can be implemented to receive the predicted change in tumor volume and determine whether or how reliable one or more automatically generated tumor outputs (e.g., characterizing volume, volume change, size, size change, etc.) are. For example, the post-processing technique can use a monotonic or step-wise decreasing function that correlates the confidence of the predicted volume or size change and / or inversely relates the confidence to the duration between the subsequent and baseline image captures. The predicted output can be more reliable when the predicted change in tumor volume or size is small relative to a larger change. Additionally or alternatively, the predicted output can be more reliable when one or more subsequent images are collected within a shorter time period compared to one or more baseline images. For example, the post-processing technique can use a monotonic or step-wise decreasing function that correlates the confidence of the predicted volume to the duration between the subsequent and baseline image captures. For example, the step function may indicate that a predicted volume (or volume change) is output when a follow-up image is collected within a predetermined time period (e.g., 3 months, 1 month, 2 weeks, or 1 week) relative to the time the baseline image was captured.

[0071] User device 150 includes a device that requests estimated tumor metrics corresponding to one or more tumors, such as tumor volume or change in tumor volume. User device 150 may be associated with a medical professional and / or care provider who is treating and / or evaluating the subject being imaged. In some cases, image processing system 125 can return an estimate of the volume of one or more tumors to image generation system 105 (e.g., which can then transmit the estimated volume to the user device).

[0072] In some cases, deformation detector 140 uses the deformation variables and the mask and / or annotations from the baseline image to predict the exact location of the tumor boundary in the subsequent image. Annotated versions of the subsequent image can then be generated, which may include the boundary overlaid on the image. The annotated versions of the subsequent image, along with the volume estimate, can be made available to user device 150 and / or image generation system 105.

[0073] It will be appreciated that in some cases, the tumor tracking network 100 can be used to estimate the volume of each of multiple tumors at a subsequent time (e.g., using corresponding annotations and / or masks generated for the tumors at a baseline time). In some cases, each of multiple tumors is depicted in both the baseline image and the subsequent image. Alternatively, additional tumors may appear between the baseline time at which one or more initial images are collected and the subsequent time at which one or more subsequent images are collected. If the annotator provides input indicating the detection of a new tumor, the annotator may be prompted to perform a full annotation, and a mask defined based on the annotation can be associated with a new baseline time point for the new tumor. In such cases, identifying the total number and / or cumulative size of a particular type of tumor in a subject can use both one or more transformation variables and the size estimated using one or more manual annotations.

[0074] It will also be appreciated that the techniques described herein can be used to estimate the size of one or more different types of biological objects other than tumors. For example, the techniques can be used to estimate the volume (and / or volume change) of a lesion (e.g., a brain lesion) and / or the area of ​​a mole.

[0075] III. Exemplary Tumor Tracking Process 2 shows a flowchart of a process 200 for estimating tumor size, according to some embodiments. Part or all of the process 200 may be performed by an image processing system, such as image processing system 125 from network 100.

[0076] Process 200 begins at block 210 by image processing system 125 accessing a first image. The first image may have been acquired and / or utilized by image generation system 105. The first image may depict a portion of a subject at a first time. The first image may depict at least a portion of a biological structure. The first image may be a two-dimensional image (e.g., showing at least a portion or all of a cross-section of a biological structure) or a three-dimensional image (e.g., showing at least a portion or all of a volume of a biological structure). The subject may include a person who has been diagnosed with or has a probable diagnosis of a particular disease. The particular disease may be cancer and / or a particular type of cancer.

[0077] The first image may be an image collected at a baseline time, which may be the time when the first image depicting a particular biological structure was collected, the time when the first image depicting a particular biological structure recognized by an annotator was collected, and / or the time considered to be a baseline for future comparison (e.g., by a human technician or care provider).

[0078] The first image may include a CT image, an MRI image, or an X-ray image. The first image may include a radiological image. The first image may be acquired without administering a contrast agent to the subject or after administering a contrast agent to the subject.

[0079] In block 215, image processing system 125 identifies a mask that outlines a particular biological structure. In some cases, annotation system 115 identifies the mask or receives input used to identify the mask, and data indicative of the mask is utilized by image processing system 125. The mask may be generated using annotation data that identifies the boundaries of the biological structure. The annotation data may include and / or represent annotations identified via input from an annotator that indicate the boundaries of the biological structure. For example, the annotator may be interacting with an interface that shows one or more images that correspond to and / or include the first image to indicate the boundaries.

[0080] In block 220, the image processing system 125 accesses a second image. The second image may have been acquired and / or utilized by the image generation system 105. The second image may depict a portion of the same subject depicted (partially) in the first image accessed in block 210. The portion of the same subject in the second image may be similar to the portion of the subject depicted in the first image. For example, each of the first and second images may depict the same biological structure (e.g., in its entirety or in cross section). In some cases, the second image may be defined to be and / or may include a region indicated by a human user.

[0081] In block 225, the image processing system 125 registers the second image to the first image to generate a registration. The first image and / or the second image may be pre-processed (e.g., by the pre-processing controller 130) before the registration.

[0082] In block 230, the deformation detector 140 calculates one or more transformation variables. In some cases, the transformation variables are calculated within a mask defined for the first image (e.g., associated with non-zero values ​​in the mask), within an annotated boundary defined for the first image, and / or for each voxel (or pixel) in the first image. Computing the transformation variables may include calculating a deformation field (e.g., using registration). In some cases, a Jacobian matrix (e.g., a spatial Jacobian matrix) may be calculated for each voxel using the deformation field. A Jacobian determinant may be determined for each voxel using the Jacobian matrix.

[0083] In block 235, the volume detector 145 estimates the size of the biological structure at a second time when the second image was acquired. Estimating the size difference of the biological structure between the first time and the second time may include calculating the sum (e.g., or average) of a Jacobian determinant over a set of voxels. The set of voxels may include voxels within the mask identified in block 215 (e.g., the mask value may be set to a non-zero value). The estimated size difference may be added to the size of the biological structure at the first time to generate an estimate of the size of the biological structure at the second time.

[0084] In some cases, the values ​​in the mask are used as weights applied to corresponding voxels to generate a weighted sum or weighted average. In some cases, normalization is applied, for example, to normalize the interim results to generate an estimated size. The normalization can be performed using the size of the biological structure at the first time when the first image was collected.

[0085] In block 240, the image processing system 125 outputs the estimated size of the biological structure at the second time (e.g., to the user device 150). The estimated size can be transmitted and / or displayed.

[0086] In some cases, rather than or in addition to outputting the estimated size of the biological structure, another result is determined and output based on the estimated size. For example, the estimated cumulative size of multiple biological structures can be determined and output (e.g., by summing the estimated sizes of each of multiple tumors). As another example, the pattern and extent to which one or more biological structures have grown and / or shrunk can be used to identify potential treatment approaches.

[0087] In some cases, rather than or in addition to outputting an estimated size of a biological structure, results and / or outputs are generated that predict the extent to which a current treatment approach is effectively treating a subject's cancer. For example, the rules may indicate that an effective treatment is associated with tumor shrinkage (or cumulative shrinkage of all tumors) of at least a predetermined threshold amount.

[0088] It will be appreciated that process 200 provides a technique that supports estimating the size of biological objects in recent images without requiring detailed input from an annotator to identify the exact boundaries of the objects in the recent images. Rather, identification of a general region is sufficient input (e.g., when combined with annotation of structures from previous time points and automated processing). It will be appreciated that additional biological structures (e.g., additional tumors) may have appeared between the first and second time points. If the annotator detects new structures, the annotator may be prompted to perform a full annotation, and a mask defined based on the annotations can be associated with a new baseline time point for the particular structure. In such cases, identifying the total number and / or cumulative size of a particular type of biological structure (e.g., a subject's tumors) can use both one or more transformation variables and sizes estimated using one or more manual annotations.

[0089] IV. Working Examples Registration techniques were used to measure lung tumor changes in subjects during treatment. The dataset corresponded to 329 lung tumor subjects. Initial tumor volume was estimated, and registration techniques were used to estimate volume change.

[0090] IV.A. Dataset This retrospective study used CT scans from 329 subjects with stage IV non-small cell lung cancer (NSCLC) enrolled in the Impower 150 trial (NCT02366143). Scans were collected between March 2015 and June 2019. The Impower 150 trial included a total of 1201 subjects. Of these, 1068 subjects had lung lesions, of which 948 subjects had measurable lung lesions (a measurable lesion was defined as a lesion with a length along at least one dimension of at least 10 mm). Of these 948 subjects, tumor volume data (based on central radiological assessment) were available for 353 subjects. From this cohort, lung volumetric measurements were obtained for 329 subjects for both baseline and follow-up scans. Therefore, the study subject set was defined as related to these 329 subjects. For each of the other 24 subjects, either baseline or follow-up scans were unavailable.

[0091] For each subject in the study set, the subject was scanned on two days, six weeks apart. Scans were acquired at a total of 260 locations. The first scan is hereafter referred to as the baseline scan, and the second scan is hereafter referred to as the follow-up scan.

[0092] IV.B. Method IV.B.1 Pretreatment Computed tomography (CT) DICOM (Digital Imaging and Communications in Medicine) volumetric images were converted to the nifti (Neuroimaging Information Technology Initiative) format. During conversion, images were resampled to 1 mm isotropic resolution. The full dynamic range of the CT scans in Hounsfield units was used for the experiments, and no normalization was performed on the CT scan values. A radiologist used a semi-automated segmentation tool to delineate lung lesion boundaries in baseline and follow-up CT scans for up to three lung lesions (according to RECIST 1.1 criteria). These manually annotated boundaries provided a mask for the lesion that was used to calculate ground truth tumor progression. To align the local images, a region with a 30 mm border was cropped around the lesion.

[0093] IV.B.2 Alignment Image alignment is M (T(x)) F In terms of finding a coordinate transformation T(x) that spatially aligns with (x), I M (x) is a moving image, and I F (x) is the fixed image. The follow-up images were spatially registered to the baseline image to obtain the deformation field. Each deformation field was represented as a vector image in which each voxel contains a displacement vector in physical coordinates. The spatial Jacobian matrix is ​​the first derivative of the deformation field. The determinant of this Jacobian matrix (J) indicates the amount of local compression or expansion. A value less than 1 indicates local compression, a value greater than 1 indicates local expansion, and a value of 1 indicates volume preservation.

[0094] In this embodiment, the nonlinear B-spline transformation T μ (x) was used.

[0095] SimpleElastix was used to perform a nonlinear B-spline registration technique. The transformation variables (e.g., Jacobian determinants) calculated from the nonlinear registration technique were used to calculate tumor volume change.

[0096] The Jacobian determinant is invariant to linear registration, and the estimated change can characterize the volumetric spatial distribution for each voxel. Using this determinant, the total tumor volume identified only at baseline (and not in follow-up scans) can be sufficient to estimate the total tumor volume at subsequent time points after the baseline scan. This can eliminate the need to delineate the tumor boundary in follow-up scans.

[0097] To demonstrate the utility of the Jacobian determinant in calculating volume change, a synthetic data set was created. The data set, containing baseline and follow-up ellipsoids with known volumes, was processed so that the follow-up ellipsoid was aligned with the baseline ellipsoid. Figure 3 shows representative synthetic baseline and follow-up images 305 and 310.

[0098] The volumetric change between the baseline and follow-up was then calculated using the Jacobian determinant generated based on the registration. More specifically, each follow-up image was registered to the corresponding baseline image using a B-spline registration. In particular, in the illustrative example shown in FIG. 3, the size of the ellipsoid of the composite follow-up image 310 (6735 mm 3 ) is the size of the ellipsoid of the synthetic baseline image 305 (4999 mm 3 ) is larger than the actual volume change. Therefore, the actual volume change is 1736 mm 3 is.

[0099] However, the ellipsoid in the registered version 315 of the synthetic follow-up image 310 appears (as a result of the registration) to be approximately the same shape and size as the ellipsoid in the synthetic baseline image 305. For each voxel, a Jacobian was calculated using a deformation vector relating the voxel in the registered version 315 to the corresponding voxel in the synthetic follow-up image 310. The deformation vector was then used to determine the Jacobian matrix and Jacobian for the voxel.

[0100] The Jacobian representation 320 indicates the Jacobian associated with each voxel. A mask was defined to include values ​​of 1 across voxels within the ellipsoid of the synthetic baseline image 305 and values ​​of 0 across other voxels (thereby assuming perfect annotation). The mask was multiplied (e.g., using a dot product) by the Jacobian representation 320 to generate a masked Jacobian representation 325.

[0101] The values ​​in the mask are then summed to generate an estimated volume change using the Jacobian, which is 1746 mm 3 A representative case is shown in Figure 3 (which is the "true" volume difference in this illustrative case).

[0102] The actual volume change was compared to the Jacobian-calculated volume change for five different composite volumes. Figure 4 shows a comparison of the volume change calculated using the Jacobian determinant technique specified in this example with the actual volume change. The correlation between the two methods was 0.99.

[0103] For CT lung lesions, a 30 mm bounding box was cropped around the lesion in the baseline and follow-up scans. The follow-up scans were then registered to the baseline scan. The Jacobian determinant of this transformation provides information about the expansion or contraction of all voxels in the baseline scan. Figure 5 shows the baseline and follow-up performance of the image cropping and registration pipeline for the first example subject ("Subject A").

[0104] In particular, baseline raw image 505a shows a CT image depicting a lesion, which is prominent in masked lesion baseline representation 510a. Cropped baseline raw image 515a and cropped masked lesion baseline representation 520a include only a portion of the voxels corresponding to the full image. The mask was identified based on radiologist annotations.

[0105] Similarly, subsequent raw image 505b shows a CT image depicting the lesion, where subsequent raw image 505b was acquired after the time that baseline raw image 505a was acquired. The lesion from subsequent raw image 505b is highlighted in masked subsequent representation of lesion 510b. Cropped subsequent raw image 515b and cropped masked subsequent representation of lesion 520b include only a portion of the voxels corresponding to the full image.

[0106] The aligned image 525 shows a version of the cropped subsequent raw image 505b that has been transformed to be aligned to the baseline raw image 515a. Using the alignment data, the Jacobian can be determined for each voxel (the Jacobians are collectively represented in the Jacobian image 530).

[0107] Radiologist-annotated baseline mask (M B ) and the Jacobian determinant (J), the change in volume of the lesion in the follow-up scan was calculated using equation (2).

[0108] IV.C. Results IV.C.1 Alignment Assessment The alignment results were qualitatively judged by visualizing the aligned images. Distorted images appeared similar to their corresponding baseline images. The aligned images were subtracted from the baseline image, and the difference was plotted as an image. Perfect alignment results in a difference image with no edges and only noise. The baseline and aligned scans were overlaid on top of each other to assess the differences between the images. Figure 6 (left two columns of images) shows an overlay of the baseline and follow-up images before alignment of the follow-up images to the baseline images. Gray areas in the composite image indicate locations where the two images have the same intensity. Magenta and green areas indicate locations where the intensity differs. Figure 6 (right two columns of images) shows the baseline and aligned images after alignment of the lung lesions. As shown by the results summarized in Table 1, alignment of the follow-up images to the baseline images was successful for subjects A and B, but alignment of the follow-up images to the baseline images failed for subject C. [Table 1]

[0109] IV.C.2 Subject B Figure 7 shows another example of the lesion, mask, cropping results, and registration corresponding to the type of image depicted in Figure 5. However, the image in Figure 7 pertains to a different subject (Subject B). In this case, the volume of the lesion decreased in the follow-up scan compared to the baseline scan. The lesion shrinkage is visible in the follow-up image.

[0110] The volume change calculated by the Jacobian method is -2060mm 3 This result was compared with the ground truth volume change (measured by manually delineating the lesions in baseline and follow-up images and then subtracting their volumes). The ground truth volume change was -2066 mm 3 and the Jacobian method predicted -2060mm 3The negative signs of the ground truth volume change and Jacobian change indicate lesion shrinkage.

[0111] IV.C.3 Subject C Figure 8 shows yet another example of lesions, masks, cropping results and registrations corresponding to the type of images shown in Figure 5. However, the images in Figure 8 relate to yet another subject (Subject C).

[0112] In this particular example, the nonlinear B-spline registration failed. The ground truth lesion volume change from baseline to follow-up was 91% (a negative sign indicates lesion shrinkage). The registered axial slices do not look similar to the baseline because the registered volumes do not match. In this case, the Jacobian-based approach predicted that the lesion had grown (see Table 1), which did not match the ground truth volume change (which indicated that the lesion had actually shrunk).

[0113] IV.C.4 High-Level Analysis The volumetric changes of 280 lesions were estimated using the Jacobian determinant. Figure 7 shows a Bland-Altman plot. In this plot, the mean lesion change value was calculated for each lesion and plotted along the x-axis. Specifically, for each ground truth instance and Jacobian calculation instance, the lesion change was defined as the lesion size at the subsequent time minus the lesion size at the baseline time. For each subject, the mean value ("Mean") was defined as the average of the lesion change calculated using the ground truth annotations and the lesion change calculated using the Jacobian technique. For each subject, the difference value ("Diff") was defined as the lesion change calculated using manual annotations minus the lesion change calculated using the Jacobian method. Figure 9 plots the difference values ​​against the mean values. Notably, many of the points fall near the y = 0 line, indicating high agreement between the Jacobian and ground truth metrics.

[0114] Brand and Altman recommended that 95% of the data points should fall within two standard deviations of the mean difference. For the analyzed data set, 11 subjects fall outside the two standard deviation limit. Thus, 96.07% of the measurements fall within the standard deviation interval.

[0115] Figure 10A shows a plot correlating statistics derived from manual annotations with those derived from the Jacobian method across all 329 lesions. The correlation between the two methods of measuring volumetric change is 0.24. The registration appeared suboptimal, especially for lesions with volumetric changes greater than 90% from baseline to follow-up, and the two registered volumes were highly variable anatomically. Therefore, 28 lesions were excluded due to volumetric changes greater than 90%. Manual inspection of the registered volumes resulted in the exclusion of 33 lesions because the follow-up images could not be registered to the corresponding baseline images. Subject C in Figure 6 is one of the examples of registration failures. Figure 10B shows a plot correlating statistics derived from manual annotations with those derived from the Jacobian method for 268 lesions, excluding failures and large volumetric changes.

[0116] Volume calculations using Jacobian determinants were more accurate when the actual change in volume was less than 40%. Registration of baseline and follow-up volumes had high image similarity when compared with lesions with volume changes of more than 40%. Figure 10C shows a plot correlating the statistics derived from manual annotation with those derived from the Jacobian determinant method for lesions with volume changes of less than 40% (N = 107). The correlation coefficient is 0.8861. Figure 10D shows a plot correlating the statistics derived from manual annotation with those derived from the Jacobian determinant method for lesions with volume changes of more than 40% (N = 161) with a correlation coefficient of 0.8187.

[0117] IV.C.5 Selective Use of Transformation-Based Approaches The deformation-based approach described herein can more accurately predict the size and / or size changes of biological structures in some situations compared to other situations.

[0118] Figure 11 shows one of the cases where the radiologist's annotation assessment on the follow-up scan did not agree with the lesion boundary. The Jacobian volume calculation was 1172 mm 3 According to manual annotation, the tumor volume was 14 mm 3 has increased by just

[0119] The Jacobian method of measuring change is more efficient at measuring small changes in lesions compared to techniques that rely on iterative manual annotation. Therefore, when the anatomical structures are similar between baseline and follow-up scans, we were able to successfully align the follow-up lesion volume to the baseline lesion volume. The Jacobian-based approach can be particularly advantageous for estimating lesion volume when scans are acquired at short intervals (less than 2 weeks). Short-term CT follow-up scans are often used for enrollment criteria. Figure 12 shows a plot comparing the two methods when the volume change (absolute change from baseline to follow-up) is less than 30%. Notably, the correlation between predicted and observed lesion volume change is well correlated for these data points (R 2 =0.9286).

[0120] Therefore, it may be advantageous to use a deformation-based approach (e.g., relying on deformation fields, Jacobian matrices, etc.) as long as the predicted change in volume is below a predetermined threshold. In other situations, other assessments of the biological structure (e.g., may rely on semi-automated or manual annotation of the biological structure at subsequent time points) may be used.

[0121] IV.C. Discussion The calculation of volumetric changes at follow-up was semi-automated by processing the baseline scan, follow-up scans, and baseline annotations. The accurate Jacobian matrix depends on the registration quality. Advanced Mattes mutual information was used as a similarity measure (characterizing the similarity between the registered and baseline images) to characterize the registration quality of the two registered volumes. Confounding factors for the registration method include gross morphological changes and large volumetric changes (>60%). The calculated Jacobian matrix can include anatomical changes along with tumor changes (e.g., volumetric changes in the tumor and normal anatomy). The data for subject C demonstrate that registration can fail due to large volumetric changes.

[0122] Registration assessment is often traditionally performed by implementing registration-based techniques, such as those that rely on calculating the target registration error or DICE coefficient and / or those that rely on segmenting anatomical structures within images associated with each of multiple time periods. These two matrices require the segmentation of anatomical landmarks or known anatomical structures on fixed and dynamic images. Manual annotation at baseline and follow-up to identify boundaries for volume estimation is time-consuming, potentially expensive (due to payment for skilled expert time), and can introduce errors. To avoid these potential effects, the technique in this example uses automated techniques to track local volumes (with 30 mm bounding boxes around specific lesions) to determine registration and size characterization techniques. These approaches reduce the required computational power and also reduce failure rates compared to approaches that require iterative segmentation with human annotator input (e.g., identifying specific portions of both baseline and follow-up images that show at least part of the boundary of a biological structure).

[0123] V. Further Considerations Some embodiments of the present disclosure include a system including one or more data processors. In some embodiments, the system includes a non-transitory computer-readable storage medium containing instructions that, when executed on the one or more data processors, cause the one or more data processors to perform some or all of one or more methods and / or some or all of one or more processes disclosed herein. Some embodiments of the present disclosure include a computer program product tangibly embodied in a non-transitory machine-readable storage medium containing instructions configured to cause one or more data processors to perform some or all of one or more methods and / or some or all of one or more processes disclosed herein.

[0124] The terms and expressions which have been employed are used as terms of description rather than of limitation, and there is no intention in the use of such terms and expressions to exclude equivalents of the features shown and described or portions thereof, but it is recognized that various modifications are possible within the scope of the invention as claimed. Thus, although the claimed invention has been specifically disclosed by embodiments and optional features, it will be understood that modifications and variations of the concepts disclosed herein may be resorted to by those skilled in the art, and that such modifications and variations are deemed to be within the scope of the invention as defined by the appended claims.

[0125] The description provides only preferred exemplary embodiments and is not intended to limit the scope, applicability, or configuration of the present disclosure. Rather, the description of preferred exemplary embodiments provides those skilled in the art with an enabling description for implementing various embodiments. It will be understood that various changes can be made in the function and arrangement of elements without departing from the spirit and scope of the appended claims.

[0126] In the description, specific details are set forth to provide a thorough understanding of the embodiments. However, it will be understood that the embodiments can be practiced without these specific details. For example, circuits, systems, networks, processes, and other components may be shown as components in block diagram form in order to avoid obscuring the embodiments in unnecessary detail. In other instances, well-known circuits, processes, algorithms, structures, and techniques may be shown without unnecessary detail in order to avoid obscuring the embodiments.

Claims

1. 1. A computer-implemented method comprising: accessing a first image captured at a first time depicting a portion of the subject; identifying a mask for the first image, the mask outlining a particular biological structure depicted in the first image; accessing a second image depicting a similar portion of the subject captured at a second time after the first time; and registering the second image to the first image; calculating, for each voxel of a plurality of voxels in the mask, a transformation variable using the registration, the transformation variable characterizing a spatial difference between a first position of the voxel in the first image and a second position of a corresponding voxel in the second image; using the transformed variables to estimate the size the biological structure was at the second time; and outputting the estimated size of the biological structure at the second time; and 11. A computer-implemented method comprising:

2. Calculating the transformation variables calculating a spatial Jacobian matrix for the voxels using the registration; calculating a Jacobian determinant for the voxel using the spatial Jacobian matrix for the voxel; 2. The computer-implemented method of claim 1, wherein estimating the size of the biological structure at the second time further comprises estimating using the Jacobian determinant for the voxel.

3. The computer-implemented method of claim 2 , wherein generating the estimated size of the biological structure comprises summing the Jacobian determinant over the plurality of voxels in the mask.

4. generating the estimated size of the biological structure includes summing or averaging the Jacobian determinants across the plurality of voxels in the mask to generate the sum or average Jacobian determinant; and estimating the size the biological structure was at the second time includes: the sum or the average of the Jacobian determinants, and 4. The computer-implemented method of claim 2 or 3, further comprising determining the product of: a) the estimated volume of the biological structure at the first time;

5. The computer-implemented method of any one of claims 1 to 4, wherein the alignment of the second image to the first image uses a non-linear B-spline transformation.

6. 6. The computer-implemented method of claim 1, wherein identifying the mask for the first image comprises processing detected user input that defines an outline of the biological structure.

7. The computer-implemented method of any one of claims 1 to 6, wherein each of the first image and the second image comprises a CT scan, an MRI image, or an X-ray.

8. inputting, by a user, identification information corresponding to said subject and / or corresponding to image data related to said subject into an interface, said input of said identification information triggering execution of a computer-implemented method according to any one of claims 1 to 7; receiving the outputted estimated size of the biological structure by the user; determining a treatment strategy based on said estimated size; facilitating implementation of said treatment strategy for said subject; and A method comprising:

9. determining a treatment strategy based on said estimated size; facilitating implementation of said treatment strategy for said subject; and The computer-implemented method of any one of claims 1 to 7, further comprising:

10. 1. A system comprising: one or more data processors; a non-transitory computer-readable storage medium containing instructions that, when executed on the one or more data processors, cause the one or more data processors to perform a set of operations, the set of operations including: accessing a first image captured at a first time depicting a portion of the subject; identifying a mask for the first image, the mask outlining a particular biological structure depicted in the first image; accessing a second image depicting a similar portion of the subject captured at a second time after the first time; and registering the second image to the first image; calculating, for each voxel of a plurality of voxels in the mask, a transformation variable using the registration, the transformation variable characterizing a spatial difference between a first position of the voxel in the first image and a second position of a corresponding voxel in the second image; using the transformed variables to estimate the size the biological structure was at the second time; and outputting the estimated size of the biological structure at the second time; and Including, the system.

11. Calculating the transformation variables calculating a spatial Jacobian matrix for the voxels using the registration; 11. The system of claim 10, further comprising: calculating a Jacobian determinant for the voxel using the spatial Jacobian matrix for the voxel, wherein the estimated size of the biological structure at the second time is generated using the Jacobian determinant for the voxel.

12. The system of claim 11 , wherein generating the estimated size of the biological structure comprises summing the Jacobian determinant over the plurality of voxels in the mask.

13. generating the estimated size of the biological structure includes summing or averaging the Jacobian determinants across the plurality of voxels in the mask to generate the sum or average Jacobian determinant; and estimating the size the biological structure was at the second time includes: the sum or the average of the Jacobian determinants, and 13. The system of claim 11 or 12, further comprising determining a product of: an estimated volume of the biological structure at the first time;

14. The system of any one of claims 10 to 13, wherein the registration of the second image to the first image uses a non-linear B-spline transformation.

15. 15. The system of claim 10, wherein identifying the mask for the first image comprises processing detected user input that defines an outline of the biological structure.

16. The system of any one of claims 10 to 15, wherein each of the first image and the second image comprises a CT scan, an MRI image, or an X-ray.

17. A computer program tangibly embodied in a non-transitory machine-readable storage medium comprising instructions configured to cause one or more data processors to perform a set of operations, said set of operations comprising: accessing a first image captured at a first time depicting a portion of the subject; identifying a mask for the first image, the mask outlining a particular biological structure depicted in the first image; accessing a second image depicting a similar portion of the subject captured at a second time after the first time; and registering the second image to the first image; calculating, for each voxel of a plurality of voxels in the mask, a transformation variable using the registration, the transformation variable characterizing a spatial difference between a first position of the voxel in the first image and a second position of a corresponding voxel in the second image; using the transformed variables to estimate the size the biological structure was at the second time; and outputting the estimated size of the biological structure at the second time; and a computer program comprising:

18. Calculating the transformation variables calculating a spatial Jacobian matrix for the voxels using the registration; 20. The computer program product of claim 17, further comprising: calculating a Jacobian determinant for the voxel using the spatial Jacobian matrix for the voxel, wherein the estimated size of the biological structure at the second time is generated using the Jacobian determinant for the voxel.

19. 20. The computer program of claim 18, wherein generating the estimated size of the biological structure comprises summing the Jacobian determinant over the plurality of voxels in the mask.

20. generating the estimated size of the biological structure includes summing or averaging the Jacobian determinants across the plurality of voxels in the mask to generate the sum or average Jacobian determinant; and estimating the size the biological structure was at the second time includes: the sum or the average of the Jacobian determinants, and 20. The computer program of claim 18 or 19, comprising determining the product of: a) the estimated volume of the biological structure at the first time; and b) the estimated volume of the biological structure at the first time.

Images