Systems and methods for detecting and analyzing bone lesions

Abstract

Disclosed herein are systems and methods for diagnosing one or more bone disorders in a patient. The method can include receiving two or more images of the patient, segmenting one or more bones in each of the two or more images, overlaying the one or more bones in each of the two or more images to form a progression overlaid image, and determining a progression of the one or more bone disorders based on the progression overlaid image.

Description

PATENT Attorney Docket No. 052592-858121 SYSTEMS AND METHODS FOR DETECTING AND ANALYZING BONE LESIONSCross-Reference to Related Applications

[0001] This application claims the benefit of U.S. Application No. 63 / 740,071 , filed December 30, 2024, the entire contents of which are incorporated herein by reference in their entirety.Field of Disclosure

[0002] The present disclosure relates to systems and methods for detecting and analyzing bone lesions.Background

[0003] Current radiology methods are qualitative, not quantitative, which makes it very difficult to directly compare two sets of imaging data, as patients are never in exactly the same position in two scans.

[0004] Therefore, there is a need for quantitative radiology methods.Summary

[0005] Provided herein is a method for diagnosing one or more bone disorders in a patient. The method can include receiving two or more images of the patient; segmenting, via a segmentation model, one or more bones in each of the two or more images; overlaying the one or more bones in each of the two or more images to form a progression overlaid image; and determining a progression of the one or more bone disorders based on the progression overlaid image.

[0006] In some aspects, the one or more bone disorders can include one or more focal lesions and / or osteolytic lesions. In some aspects, the two or more images can include CT scans, PET scans, MRI images, and / or x-ray images. In some aspects, the two or more images can include a first image taken at a first timepoint and a second image taken at a second timepoint. In some aspects, the first timepoint and the second timepoint can be about one day to about ten years apart. In some aspects, the method can further include converting the progression overlaid image to a bone mineral density image.107112419.1PATENT Attorney Docket No. 052592-858121

[0007] In some aspects, the method can further include forming a standard image of a healthy patient; segmenting one or more healthy bones of the standard image of the healthy patient; and overlaying the one or more bones of at least one of the two or more images with the one or more healthy bones of the standard image of the healthy patient to form a diagnostic image. In some aspects, the method can further include converting the diagnostic image to a bone mineral density image.

[0008] Further provided herein is a method for detecting and diagnosing one or more bone disorders in a patient. The method can include forming a standard image from a plurality of healthy patient images; obtaining one or more images of the patient; segmenting, via a segmentation model, one or more bones of the one or more images; overlaying the one or more bones of the one or more images with one or more average healthy bones of the standard image, thereby producing a diagnostic overlaid image; and detecting the one or more bone disorders on the diagnostic overlaid image.

[0009] In some aspects, forming the standard image can include compiling images from two or more healthy patients using non-rigid registration. In some aspects, the one or more images of the patient can include two or more images. In some aspects, the method can further include segmenting, via the segmentation model, the two or more images into one or more bones of each of the two or more images; and overlaying the one or more bones of each of the two or more images to form a progression overlaid image.

[0010] In some aspects, the method can further include determining a progression of the one or more bone disorders based on the progression overlaid image. In some aspects, the method can further include converting the diagnostic overlaid image to a bone mineral density image. In some aspects, the one or more bone disorders can include one or more focal lesions and / or osteolytic lesions.

[0011] Further provided herein is a system for detecting and diagnosing one or more bone disorders. The system can include an imaging system operable to obtain one or more images of the patient and at least one processor. The at least one processor can be configured to form a standard image from a plurality of healthy patient images; receive the one or more images of the patient; segment, via a segmentation model, one or more bones of the one or more images; and overlay the one or more bones of the one or more 2107112419.1PATENT Attorney Docket No. 052592-858121 images with one or more average healthy bones of the standard image, thereby forming a diagnostic overlaid image.

[0012] In some aspects, the one or more images of the patient can include two or more images. In some aspects, the at least one processor can be further configured to segment, via the segmentation model, the two or more images into one or more bones of the two or more images; overlay the one or more bones of each of the two or more images to form a progression overlaid image; and determine a progression of the one or more bone disorders based on the progression overlaid image. In some aspects, the at least one processor can be further configured to convert the diagnostic overlaid image to a bone mineral density image. In some aspects, the standard image can include images from two or more healthy patients.Brief Description of Figures

[0013] The description will be more fully understood with reference to the following figures and graphs, which are presented as various embodiments of the disclosure and should not be construed as a complete recitation of the scope of the disclosure. It is noted that, for purposes of illustrative clarity, certain elements in various drawings may not be drawn to scale. Understanding that these drawings depict only exemplary embodiments of the disclosure and are not therefore to be considered limiting of its scope, the principles herein are described and explained with additional specificity and detail through the use of the accompanying drawings in which:

[0014] FIG. 1 illustrates a flowchart of a method for diagnosing one or more lesions.

[0015] FIG. 2 illustrates a flowchart of a method for detecting and diagnosing one or more lesions.

[0016] FIG. 3 illustrates a flowchart for overlaying two or more segmented bones.

[0017] FIG. 4 illustrates a flowchart for overlaying two or more segmented bones.

[0018] FIG. 5A illustrates progression overlaid images of a skull.

[0019] FIG. 5B illustrates progression overlaid images of a vertebrae.

[0020] FIG. 5C illustrates progression overlaid images of a pelvis.

[0021] FIG. 5D illustrates progression overlaid images of a femur.3107112419.1PATENT Attorney Docket No. 052592-858121

[0022] FIG. 6 illustrates a process for forming a standard patient image and overlaying the standard patient image with a current patient image to form a diagnostic image.

[0023] FIG. 7 illustrates an example of a deep learning neural network that can be used to implement segmentation and overlaying of bones.

[0024] FIG. 8 is a diagram illustrating an example of a computing system.

[0025] Reference characters indicate corresponding elements among the views of the drawings. The headings used in the figures do not limit the scope of the claims.Detailed Description

[0026] Various embodiments of the disclosure are discussed in detail below. While specific implementations are discussed, it should be understood that this is done for illustration purposes only. A person skilled in the relevant art will recognize that other components and configurations may be used without parting from the spirit and scope of the disclosure. Thus, the following description and drawings are illustrative and are not to be construed as limiting. Numerous specific details are described to provide a thorough understanding of the disclosure. However, in certain instances, well-known or conventional details are not described in order to avoid obscuring the description. References to one or an embodiment in the present disclosure can be references to the same embodiment or any embodiment; and such references mean at least one of the embodiments.

[0027] Reference to “one embodiment”, “an embodiment”, or “an aspect” means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the disclosure. The appearances of the phrase “in one embodiment” or “in one aspect” in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Moreover, various features are described which may be exhibited by some embodiments and not by others.

[0028] The terms used in this specification generally have their ordinary meanings in the art, within the context of the disclosure, and in the specific context where each term is used. Alternative language and synonyms may be used for any one or more of the 4107112419.1PATENT Attorney Docket No. 052592-858121 terms discussed herein, and no special significance should be placed upon whether or not a term is elaborated or discussed herein. In some cases, synonyms for certain terms are provided. A recital of one or more synonyms does not exclude the use of other synonyms. The use of examples anywhere in this specification including examples of any terms discussed herein is illustrative only and is not intended to further limit the scope and meaning of the disclosure or of any example term. Likewise, the disclosure is not limited to various embodiments given in this specification.

[0029] As used herein, “about” refers to numeric values, including whole numbers, fractions, percentages, etc., whether or not explicitly indicated. The term “about” generally refers to a range of numerical values, for instance, ± 0.5-1%, ± 1-5% or ± 5-10% of the recited value, that one would consider equivalent to the recited value, for example, having the same function or result.

[0030] Additional features and advantages of the disclosure will be set forth in the description which follows, and in part will be obvious from the description, or can be learned by practice of the herein disclosed principles. The features and advantages of the disclosure can be realized and obtained by means of the instruments and combinations particularly pointed out in the appended claims. These and other features of the disclosure will become more fully apparent from the following description and appended claims or can be learned by the practice of the principles set forth herein.

[0031] Provided herein are systems and methods for detecting and analyzing bone disorders in images of patients. The systems and methods described herein provide quantitative analysis of bone disorders, allowing for the accurate tracking of specific bone disorders, as well as diagnosis of bone disorders generally. For example, the systems and methods described herein allow physicians to directly compare two sets of imaging data from different timepoints. Further, the systems and methods described herein provide for diagnosis of a single patient image by overlaying the single patient image with a standard healthy patient image (i.e. , atlas).

[0032] Typical diagnostic methods for determining the progression of bone disorders require radiologists to look at the images (e.g., computed tomography (CT) images, positron emission tomography (PET) images, magnetic resonance imaging (MRI) images, x-ray images, or other images of bones) to assess all bone disorders (e.g.,5107112419.1PATENT Attorney Docket No. 052592-858121 lesions). However, these diagnostic methods result in significant problems. The intensity of potential bone disorders (e.g., lesions shown as bright spots on the PET or CT scan) are usually compared to a background signal, which is derived from the intensity of the lumbar spine (if it is free from lesions). As described herein, this process can be automated. Further, there may not be any defined bone disorders (e.g., focal lesions) visible in the images (e.g., CT images, PET images, MRI images, x-ray images, or other images of bones), but there may be increased intensity throughout the bone marrow, or the marrow of particular bones. The increased intensity may be spotty rather than a distinct bright spot, which is difficult for radiologists to quantify. In multiple myeloma patients, lesions do not tend to have hard edges because these lesions are clumps of cancer in bone marrow, as opposed to a well-defined tumor with defined boundaries, thereby presenting difficult diagnostic progression issues. In some examples, a patient has so many lesions that it is impractical for a radiologist to track all of the lesions, especially when comparing multiple timepoints. Therefore, there is a need for assessing treatment response and / or disease progression at multiple timepoints.

[0033] FIG. 1 illustrates a method 100 for comparing two or more images of a patient. In some examples, comparing the two or more images can provide diagnostic information related to the progression of bone lesions. In some examples, the one or more lesions can be bone lesions. In some examples, the patient can be a multiple myeloma patient. In some examples, the bone lesions can be focal lesions. In some examples, the bone lesions can be osteolytic lesions. In some examples, comparing the two or more images can provide diagnostic information related to changes in bone mineral density (BMD). In some examples, comparing the two or more images can provide diagnostic information related to fractures. In some examples, comparing the two or more images can provide diagnostic information related to changes in the pattern and / or texture of the images (e.g., diagnosis of diffuse disease where increased intensity is found on the comparison of images).

[0034] The method 100 can begin at block 102. At block 102, the method 100 can include receiving two or more images of a patient. In some examples, receiving the two or more images can include obtaining the two or more images using an imaging system. In some examples, the two or more images are from different timepoints. For example,6107112419.1PATENT Attorney Docket No. 052592-858121 the time between the images may be a progression duration (e.g., duration between a first timepoint and a second timepoint). In some examples, the progression duration can be about 1 day to about 10 days, about 10 days to about 20 days, about 20 days to about 1 month, about 1 month to about 2 months, about 2 months to about 3 months, about 3 months to about 4 months, about 4 months to about 5 months, about 5 months to about 6 months, about 6 months to about 7 months, about 7 months to about 8 months, about 8 months to about 9 months, about 9 months to about 10 months, about 10 months to about 11 months, about 11 months to about 1 year, about 1 year to about 5 years, about 5 years to about 10 years, or more. Certain bone disorders can require comparison in a very short progression duration, such as 1 to 7 days. For example, with rapidly progressing bone disorders, such as osteomyelitis, the progression duration can be about 1 day to about 7 days. In other examples, a comparison of a healthy patient’s skeleton over decades can be beneficial. In this example, the progression duration can be longer, such as 10 years or more.

[0035] In some examples, the two or more images can be computed tomography (CT) images, positron emission tomography (PET) images, magnetic resonance imaging (MRI) images, x-ray images, or other images of bones. In some examples, the two or more images can be different types of images. For example, a first image can be a CT image, and a second image can be an MRI image. In some examples, the two or more images can include a first image and a second image. In some examples, the first image can include two or more different images (e.g., CT and PET images) taken at the same timepoint. In some examples, the second image can include two or more different images (e.g., CT and PET images) taken at the same timepoint. The two or more images can be obtained utilizing a CT scanner, PET scanner, MRI machine, X-ray machine, or other imaging systems used to acquire images of bones.

[0036] At block 104, the method 100 can include segmenting, via a segmentation model, one or more bones in each of the two or more images. In some examples, the segmentation model can be a neural network model, as described further herein. The segmentation model can be operable to label all voxels in an image containing bones. The segmentation model can then segment the bones into individual bones for further analysis. In some examples, the segmentation model creates masks for each bone in an 7107112419.1PATENT Attorney Docket No. 052592-858121 image. In other examples, the segmentation model segments the entire image by labeling all voxels containing bones. In other examples, the one or more bones can be segmented from the two or more images via manual segmentation.

[0037] In some examples, the segmented images of the one or more bones can be converted to bone mineral density (BMD) images. For example, when the two or more images are CT images obtained from a CT scanner, the segmented images of the one or more bones can be converted to BMD images. The CT images are typically in Hounsfield units. The CT scanner can be equipped with quantitative CT (QCT) phantom. The QCT phantom contains several rods with known radiodensities. A linear model can then be used to map Hounsfield units to BMD units. The BMD images can provide significant advantages in diagnosing and determining the progression of bone disorders. For example, comparing two BMD images can provide information related to the increase or decrease in bone mineral density between two timepoints.

[0038] At block 106, the method 100 can include overlaying the one or more bones in each of the two or more images to form a progression overlaid image. The progression overlaid image can allow for a radiologist to directly compare the progression of a bone disorder in a specific bone. In some examples, the progression overlaid in image can include three, four, five, six, seven, eight, nine, ten or more images taken of a bone at different timepoints. The progression overlaid image can be produced by registering the segmented bone from the two or more images. In some examples, the registration method can be a rigid registration method. The rigid registration method can require aligning two or more images such that the images are overlaid, without requiring any deformation of the images. The rigid registration method can be a linear registration method. In some examples, the progression overlaid image can allow for the transformation of multiple images into an image indicative of the progression of a bone disease, injury, or deficiency over time. In this manner, the original images of patient bones can be transformed into a diagnostic tool (e.g., progression overlaid image) operable to provide insight into the progression of bone injuries, diseases, and other bone abnormalities.

[0039] FIGS. 3-4 illustrate rigid registration methods. As illustrated in FIG. 3, rigid registration requires aligning two images of a single patient from different timepoints.8107112419.1PATENT Attorney Docket No. 052592-858121 Once the images are aligned, a progression overlaid image 300 is produced. FIG. 4 illustrates rigid registration with segmented bones using the segmentation model described herein. As illustrated the rigid registration method can be conducted using a first image 400 and a second image 402. The first image 400 can be segmented to include a single bone utilizing the segmentation model described herein. The second image 402 can be segmented to include a single bone utilizing the segmentation model as described herein. As illustrated, the first image 400 and the second image 402 can then be aligned and registered, thereby producing progression overlaid image 404.

[0040] In some examples, the rigid registration method can be a direct registration method or an indirect registration method. For example, direct registration can register the segmentations of the bone itself to form the progression overlaid image. An indirect registration method can include masking the bones and then registering the original image data within the masks.

[0041] FIGS. 5A-5D illustrate progression overlaid images produced using the method 100. The progression overlaid images were produced by obtaining three images taken at different timepoints, segmenting the images to form an image of a single bone at the three timepoints, and then overlaying the single bone from the three timepoints using a rigid registration method. FIG. 5A illustrates a progression overlaid image 500 of a skull produced using the method 100 described herein. As illustrated in FIG. 5A, a first image 502, a second image 504, and a third image 506 can be overlaid to form a progression overlaid image 500. As illustrated in FIG. 5B, a first image 510, a second image 512, and a third image 514 can be overlaid to form a progression overlaid image 508. As illustrated in FIG. 5C, a first image 518, a second image 520, and a third image 522 can be overlaid to form a progression overlaid image 516. As illustrated in FIG. 5D, a first image 526, a second image 528, and a third image 530 can be overlaid to form a progression overlaid image 524. As illustrated, the progression overlaid images 500, 508, 516, and 524 can visually indicate an increase or decrease in Hounsfield units, where the darker portions of the progression overlaid images 500, 508, 516, and 524 illustrate areas of the bone where the difference in Hounsfield units has increased. In some examples, the progression overlaid image can be converted from the units of the images (e.g., Hounsfield units for a CT image) to BMD.9107112419.1PATENT Attorney Docket No. 052592-858121

[0042] Referring back to FIG. 1, at step 108, the method 100 can further include determining a progression based on the progression overlaid image. As described herein, the method 100 can output progression overlaid images (e.g., progression overlaid images 300, 404, 500, 508, 516, 524) of a bone for diagnosis of a bone disorder and / or progression of a bone disorder. The method 100 can produce progression overlaid images for any and all bones of a patient. The progression overlaid image can provide a quantitative image illustrating the difference in a particular bone over time. The progression overlaid image can provide a physician with a quantitative image for determining treatment options. In some examples, the method 100 can further include displaying the progression overlaid image on a display screen.

[0043] In some examples, the progression overlaid image can be displayed on a graphical user interface. The graphical user interface can allow a physician to interact with the progression overlaid image. For example, the graphical user interface can allow a physician to zoom in on a region of a bone. In some examples, a region of interest in the bone can be cortical and / or trabecular bone. In some examples, a difference in values (e.g., Hounsfield units and / or BMD) can be displayed for the region of the bone. For example, the difference in values can be displayed between the first image, the second image, and / or any additional images used to form the progression overlaid image. In this manner, physicians can analyze different regions of a single bone to determine treatment options and / or progression of a bone disorder in a particular region of a singular bone.

[0044] In some examples, the method 100 can further include generating a report based on the progression overlaid image. For example, the report can provide mean values (e.g., Hounsfield units and / or BMD) in a bone for both the first image and the second image, as well as a difference in values in the bone between the first image and the second image. In some examples, the report can include areas of interest (e.g., particular lesions in a bone, trabecular bone only statistics, cortical bone only statistics, etc.) where the bone value has increased or decreased in a significant manner (e.g., providing confirmation of effective treatment or indication of disease progression). In some examples, the report can include summary statistics of the bone for both the first image and the second image, as well as a difference in the summary statistics between the first image and the second image. In some examples, the summary statistics can 10107112419.1PATENT Attorney Docket No. 052592-858121 include a mean, median, and / or standard deviation of a value of interest (e.g., Hounsfield units and / or BMD).

[0045] Various treatment decisions can be made based on the progression overlaid images. For example, the progression overlaid image can be used to detect decreases in BMD or osteolytic lesions, thereby directing the physician to prescribe one or more pharmaceutical drugs such as bisphosphonates and / or anti-sclerostin antibodies. The decreased BMD or osteolytic lesions can also lead to the recommendation for further testing and / or recommendations to increase monitoring of the patient and limiting physical activity (e.g., to reduce the risk of falling). In some examples, the progression overlaid image can provide information on whether myeloma patients should be referred for spinal surgery or other osteopathic surgical procedures (e.g., based on a level of loss of BMD), as hardware (e.g., strengthening screws, plates, etc.) may be required to reinforce and / or stabilize the bone. In some examples, the progression overlaid images can be used to determine fractures in a bone. For example, minor fractures can occur that are difficult to detect from a single image. By overlaying an image of a bone before a patient sustained an injury with an image of a bone after a patient sustained an injury, the presence, location, and severity of a fracture can be determined. In some examples, the progression overlaid image can provide information related to new and / or progressing lesions. For example, the progression overlaid image can indicate a loss of BMD in a certain area, indicating that a new lesion has formed. In some examples, the progression overlaid image can indicate an increase in BMD, thereby confirming that a current treatment regimen (e.g., chemotherapy, radiotherapy, or surgical intervention) has been successful.

[0046] FIG. 2 illustrates a method 200 for detecting and diagnosing a bone disorder in a patient. In some examples, the method 200 can be used to diagnose a patient with a bone disorder using a single image of the patient. At block 202, the method 200 can begin by obtaining one or more healthy patient images of a plurality of healthy patients (e.g., a plurality of healthy patient images). Obtaining one or more images of healthy patients can include utilizing an imaging system. In some examples, the imaging system can be one or more of a CT scanner, a PET scanner, an MRI machine, an X-ray machine, or other common bone imaging systems.11107112419.1PATENT Attorney Docket No. 052592-858121

[0047] At block 204, the method 200 can include segmenting, via a segmentation model, one or more healthy bones of the one or more healthy patient images. For example, the segmentation model can segment one or more healthy bones from each of a plurality of healthy patients. In some examples, the segmentation model can be a neural network model, as described further herein. The segmentation model can be operable to label all voxels in an image containing bones. The segmentation model can then segment the healthy bones into individual bones for further analysis. In some examples, the segmentation model can apply masks to the image to label the healthy bones, as described herein.

[0048] In some examples, the segmented images of the one or more healthy bones can be converted to bone mineral density (BMD) images. For example, when the one or more images are CT images obtained from a CT scanner, the segmented images of the one or more healthy bones can be converted to BMD images. The CT images are typically in Hounsfield units. The CT scanner can be equipped with quantitative CT (QCT) phantom. The QCT phantom contains several rods with known radiodensities. A linear model can then be used to map Hounsfield units to BMD units. The BMD images can provide significant advantages in diagnosing and determining the progression of bone disorders. For example, comparing two BMD images can provide information related to the increase or decrease in bone mineral density between the two images.

[0049] At block 206, the method 200 can include forming a standard image. The standard image can include one or more average healthy bones. The standard image can be referred to as an atlas. The atlas can be a standard image formed from a plurality of healthy patient images (e.g., the segmented one or more healthy bones from the one or more healthy patient images). The atlas can include an entire skeleton (e.g., all bones of a standard healthy patient). In some examples, the atlas can be limited to one or more specific bones. In some examples, multiple atlases can be formed. For example, an atlas for a standard healthy male patient can be formed and an atlas for a standard healthy female patient can be formed. In some examples, multiple atlases can be formed based on gender and / or height. It will be appreciated that atlases can be formed based on any criteria necessary for proper diagnosis of a patient.12107112419.1PATENT Attorney Docket No. 052592-858121

[0050] FIG. 6 illustrates a process for forming an atlas in one example. As illustrated, a plurality of healthy patient images are acquired. In some examples, the plurality of healthy patient images can include images from 2 patients to 10 patients, 10 patients to 50 patients, 50 patients to 100 patients, 100 patients to 500 patients, 500 patients to 1000 patients, or more.

[0051] Forming the atlas can include registering the plurality of healthy patient images. Since not all healthy patients have the same size bones, the registration requires rigid, non-rigid, and non-linear registration to form the atlas. The registration to form the atlas requires creating standard space for each bone by compiling the healthy control data (i.e., plurality of images) into an average male and an average female. Non-rigid, rigid, and non-linear registration are used to form a single shared-space that encodes the average intensity and morphology of the input images (e.g., one or more healthy bones from the plurality of healthy patient images). In some examples, the atlas is formed by calculating T-scores and Z-scores for every voxel in each bone, which are representative of the patient population used to create the atlas. The atlas provides a healthy standard image (e.g., one or more average healthy bones) for comparison to any patient images obtained for diagnosis.

[0052] In some examples, the atlas can be in Hounsfield units. In some examples, the atlas can be in BMD units. In other examples, the atlas can be in appropriate units for the modality of the image(s). In some examples, the atlas can include an average morphology of the one or more average healthy bones. In some examples, the atlas can provide the average intensity and morphology of healthy patient bones.

[0053] The method 200 can further include segmenting one or more average healthy bones of the standard image of the healthy patient. For example, when the atlas is a full skeleton atlas, the specific bone to be analyzed can be segmented from the atlas. The average healthy bone can be segmented using the segmentation model described herein. In other examples, when an atlas for every bone is formed, the correct bone atlas can be selected, and no segmentation is necessary.

[0054] At block 208, the method 200 can include obtaining one or more images of a patient to be diagnosed. As described herein, obtaining one or more images of the patient can include utilizing an imaging system. In some examples, the imaging system 13107112419.1PATENT Attorney Docket No. 052592-858121 can be one or more of a CT scanner, a PET scanner, an MRI machine, an X-ray machine, or other common bone imaging systems.

[0055] At block 210, the method 200 can include segmenting, via a segmentation model, one or more bones of the one or more images of the patient to be diagnosed. In some examples, the segmentation model can be a neural network model, as described further herein. The segmentation model can be operable to label all voxels in an image containing bones. The segmentation model can then segment the bones into individual bones for further analysis. In some examples, the segmentation model can apply masks to the image to label the bones, as described herein.

[0056] In some examples, the segmented images of the one or more bones can be converted to bone mineral density (BMD) images. For example, when the one or more images are CT images obtained from a CT scanner, the segmented images of the one or more bones can be converted to BMD images. The CT images are typically in Hounsfield units. The CT scanner can be equipped with quantitative CT (QCT) phantom. The QCT phantom contains several rods with known radiodensities. A linear model can then be used to map Hounsfield units to BMD units. The BMD images can provide significant advantages in diagnosing and determining the progression of bone disorders. For example, comparing two BMD images can provide information related to the increase or decrease in bone mineral density between two images.

[0057] At block 212, the method 200 can include overlaying the one or more bones of the one or more images of the patient (e.g., patient to be diagnosed) with the one or more average healthy bones (e.g., from the atlas), thereby forming a diagnostic overlaid image. In some examples, overlaying the one or more bones of the patient with the one or more average healthy bones (e.g., from the atlas) includes utilizing a diffeomorphic transform to overlay the one or more bones onto the one or more average healthy bones. In some examples, similar to the progression overlaid image described herein, the diagnostic overlaid image can indicate areas of the bone where the patient has increased or decreased BMD values and / or Hounsfield values in comparison to the standard healthy patient image. In some examples, the diagnostic overlaid image can be converted from the units of the images (e.g., Hounsfield units for a CT image) to BMD.14107112419.1PATENT Attorney Docket No. 052592-858121

[0058] In some examples, the one or more bones of the patient to be diagnosed can be overlaid on the one or more average healthy bones without using a diffeomorphic transform. For example, when morphological differences are to be measured, rigid registration can be used to overlay the one or more bones of the patient to be diagnosed with the one or more average healthy bones.

[0059] At block 214, the method 200 can include detecting one or more bone disorders on the diagnostic overlaid image. The diagnostic overlaid image can provide indications of various bone disorders in the patient. For example, the diagnostic image can indicate that a patient has decreased BMD in comparison to a healthy patient in portions of the bone or throughout the entire bone. In some examples, the diagnostic overlaid image provides quick and efficient identification of lesions, by illustrating the areas of the bone where the patient has decreased BMD.

[0060] In some examples, the one or more bone disorders can be based on a difference in morphometry between the one or more average healthy bones and the one or more bones of the patient. For example, the diagnostic overlaid image can indicate that a patient has longer or shorter bones than the one or more average healthy bones. Further, the diagnostic overlaid image can also provide indications of other morphological differences, such as exostoses (e.g., bone spurs).

[0061] In some examples, the diagnostic overlaid image can be displayed on a graphical user interface. The graphical user interface can allow a physician to interact with the diagnostic overlaid image. For example, the graphical user interface can allow a physician to zoom in on a region of a bone (e.g., cortical and / or trabecular bone). In some examples, a difference in values (e.g., Hounsfield units and / or BMD) can be displayed for the region of the bone. In this manner, physicians can analyze different regions of a single bone to determine diagnosis of bone disorders (e.g., lesions, diffuse disease, fractures, etc.). In some examples, the graphical user interface can also indicate morphological differences between the patient’s bone and the one or more healthy bones (e.g., length, exostoses, or other morphological differences).

[0062] In some examples, the method 200 can further include generating a report based on the diagnostic overlaid image. For example, the report can provide mean and / or median values (e.g., Hounsfield units and / or BMD) in the diagnostic overlaid image for 15107112419.1PATENT Attorney Docket No. 052592-858121 both the average healthy bone and the patient’s bone, as well as a difference in values in the diagnostic overlaid image between the healthy bone and the patient’s bone. In some examples, the report can include areas of interest (e.g., particular lesions in a bone, fractures, diffuse disease, etc.) where the patient’s bone value (e.g., Hounsfield units and / or BMD) has increased or decreased in a significant manner in comparison to the healthy bone from the atlas. In some examples, the report can include summary statistics of the patient’s bones to the one or more average healthy bones, as a well as a difference in the summary statistics between the patient’s bones and the one or more average healthy bones. In some examples, the summary statistics can include a mean, median, and / or standard deviation of a value of interest (e.g., Hounsfield units and / or BMD). In some examples, the report can also indicate morphological differences between the patient’s bone and the one or more healthy bones (e.g., length, exostoses, or other morphological differences).

[0063] In some examples, method 200 can be performed first and then method 100 can be performed. For example, method 200 can be performed to diagnose a patient with one or more bone disorders. Method 100 can then be performed to monitor the progression of the one or more bone disorders. In this manner, the one or more bone disorders can be monitored to ensure that a treatment regimen is working and / or to determine whether more aggressive treatments are necessary. In other examples, method 100 can be performed first and then method 200 can be performed. For example, method 100 can indicate a change in a patient’s bones and then method 200 can be used to diagnose the patient based on the comparison to healthy bones.

[0064] Further provided herein is a system for detecting and diagnosing one or more bone disorders. The system can include one or more imaging systems operable to obtain images of a patient. In some examples, the imaging system can include one or more of a CT imaging system, an MRI imaging system, an X-ray machine, a PET imaging system, and / or any other imaging system operable to obtain images of a patient’s bones. In some examples, the CT imaging system can include a QCT phantom for obtaining BMD data. The system can further include at least one processor. The at least one processor can be configured to perform one or more steps of methods 100, 200.16107112419.1PATENT Attorney Docket No. 052592-858121

[0065] Various aspects of the present disclosure can use machine learning models or systems. FIG. 7 is an illustrative example of a deep learning neural network 700 that can be used to implement the machine learning-based alignment and / or segmentation described herein. An input layer 720 includes input data. In one illustrative example, the input layer 320 can include one or more images of a patient. The neural network 700 includes multiple hidden layers 722a, 722b, through 722n. The hidden layers 722a, 722b, through 722n include “n” number of hidden layers, where “n” is an integer greater than or equal to one. The number of hidden layers can be made to include as many layers as needed for the given application. The neural network 700 further includes an output layer 721 that provides an output resulting from the processing performed by the hidden layers 722a, 722b, through 722n. In one illustrative example, the output layer 321 can provide a classification for one or more bones in the input data. The classification can include a class identifying the type of activity or object (e.g., bone, etc.).

[0066] The neural network 700 is a multi-layer neural network of interconnected nodes. Each node can represent a piece of information. Information associated with the nodes is shared among the different layers and each layer retains information as information is processed. In some cases, the neural network 700 can include a feedforward network, in which case there are no feedback connections where outputs of the network are fed back into itself. In some cases, the neural network 700 can include a recurrent neural network, which can have loops that allow information to be carried across nodes while reading in input.

[0067] Information can be exchanged between nodes through node-to-node interconnections between the various layers. Nodes of the input layer 720 can activate a set of nodes in the first hidden layer 722a. For example, as shown, each of the input nodes of the input layer 720 is connected to each of the nodes of the first hidden layer 722a. The nodes of the first hidden layer 722a can transform the information of each input node by applying activation functions to the input node information. The information derived from the transformation can then be passed to and can activate the nodes of the next hidden layer 722b, which can perform their own designated functions. Example functions include convolutional, up-sampling, data transformation, and / or any other suitable functions. The output of the hidden layer 722b can then activate nodes of the 17107112419.1PATENT Attorney Docket No. 052592-858121 next hidden layer, and so on. The output of the last hidden layer 722n can activate one or more nodes of the output layer 721, at which an output is provided. In some cases, while nodes (e.g., node 726) in the neural network 700 are shown as having multiple output lines, a node has a single output and all lines shown as being output from a node represent the same output value.

[0068] In some cases, each node or interconnection between nodes can have a weight that is a set of parameters derived from the training of the neural network 700. Once the neural network 700 is trained, it can be referred to as a trained neural network, which can be used to classify one or more objects. For example, an interconnection between nodes can represent a piece of information learned about the interconnected nodes. The interconnection can have a tunable numeric weight that can be tuned (e.g., based on a training dataset), allowing the neural network 700 to be adaptive to inputs and able to learn as more and more data is processed.

[0069] The neural network 700 is pre-trained to process the features from the data in the input layer 720 using the different hidden layers 722a, 722b, through 722n in order to provide the output through the output layer 721. In an example in which the neural network 700 is used to segment bones from an image, the neural network 700 can be trained using training data that includes images with one or more bones, as described herein. For instance, training data including images and segmented bones can be input into the network, with each training frame having a label indicating the features of the bones (for a feature extraction machine learning model).

[0070] The neural network 700 can include any suitable deep network. One example includes a convolutional neural network (CNN), which includes an input layer and an output layer, with multiple hidden layers between the input and output layers. The hidden layers of a CNN include a series of convolutional, nonlinear, pooling (for downsampling), and fully connected layers. The neural network 700 can include any other deep network other than a CNN, such as an autoencoder, a deep belief nets (DBNs), a Recurrent Neural Networks (RNNs), BNNs, among others.

[0071] FIG. 8 shows an example of computing system 800, which can be for example any computing device making up various parts of the system described herein, or any component thereof in which the components of the system are in communication 18107112419.1PATENT Attorney Docket No. 052592-858121 with each other using connection 805. Connection 805 can be a physical connection via a bus, or a direct connection into processor 810, such as in a chipset architecture. Connection 805 can also be a virtual connection, networked connection, edge network connection, or logical connection.

[0072] In some embodiments, computing system 800 is a distributed system in which the functions described in this disclosure can be distributed within a datacenter, multiple data centers, a peer network, etc. In some embodiments, one or more of the described system components represents many such components each performing some or all of the function for which the component is described. In some embodiments, the components can be physical or virtual devices.

[0073] Example system 800 includes at least one processing unit (CPU or processor) 810 and connection 805 that couples various system components including system memory 815, such as read-only memory (ROM) 820 and random-access memory (RAM) 825 to processor 810. Computing system 800 can include a cache of high-speed memory 812 connected directly with, in close proximity to, or integrated as part of processor 810.

[0074] Processor 810 can include any general purpose processor and a hardware service or software service, such as services 832, 832, and 836 stored in storage device 830, configured to control processor 810 as well as a special-purpose processor where software instructions are incorporated into the actual processor design. Processor 810 may essentially be a completely self-contained computing system, containing multiple cores or processors, a bus, memory controller, cache, etc. A multi-core processor may be symmetric or asymmetric.

[0075] To enable user interaction, computing system 800 includes an input device 825, which can represent any number of input mechanisms, such as a microphone for speech, a touch-sensitive screen for gesture or graphical input, keyboard, mouse, motion input, speech, etc. Computing system 800 can also include output device 835, which can be one or more of a number of output mechanisms known to those of skill in the art. In some instances, multimodal systems can enable a user to provide multiple types of input / output to communicate with computing system 800. Computing system 800 can include communications interface 820, which can generally govern and manage the user 19107112419.1PATENT Attorney Docket No. 052592-858121 input and system output. There is no restriction on operating on any particular hardware arrangement, and therefore the basic features here may easily be substituted for improved hardware or firmware arrangements as they are developed.

[0076] Storage device 830 can be a non-volatile memory device and can be a hard disk or other types of computer readable media which can store data that are accessible by a computer, such as magnetic cassettes, flash memory cards, solid state memory devices, digital versatile disks, cartridges, random access memories (RAMs), read-only memory (ROM), and / or some combination of these devices.

[0077] The storage device 830 can include software services, servers, services, etc., that when the code that defines such software is executed by the processor 810, it causes the system to perform a function. In some embodiments, a hardware service that performs a particular function can include the software component stored in a computer-readable medium in connection with the necessary hardware components, such as processor 810, connection 805, output device 835, etc., to carry out the function.

[0078] For clarity of explanation, in some instances, the present technology may be presented as including individual functional blocks including functional blocks comprising devices, device components, steps or routines in a method embodied in software, or combinations of hardware and software.

[0079] Any of the steps, operations, functions, or processes described herein may be performed or implemented by a combination of hardware and software services or services, alone or in combination with other devices. In some embodiments, a service can be software that resides in memory of a client device and / or one or more servers of a content management system and perform one or more functions when a processor executes the software associated with the service. In some embodiments, a service is a program or a collection of programs that carry out a specific function. In some embodiments, a service can be considered a server. The memory can be a non-transitory computer-readable medium.

[0080] In some embodiments, the computer-readable storage devices, mediums, and memories can include a cable or wireless signal containing a bit stream and the like. However, when mentioned, non-transitory computer-readable storage media expressly20107112419.1PATENT Attorney Docket No. 052592-858121 exclude media such as energy, carrier signals, electromagnetic waves, and signals per se.

[0081] Methods according to the above-described examples can be implemented using computer-executable instructions that are stored or otherwise available from computer-readable media. Such instructions can comprise, for example, instructions and data which cause or otherwise configure a general purpose computer, special purpose computer, or special purpose processing device to perform a certain function or group of functions. Portions of computer resources used can be accessible over a network. The executable computer instructions may be, for example, binaries, intermediate format instructions such as assembly language, firmware, or source code. Examples of computer-readable media that may be used to store instructions, information used, and / or information created during methods according to described examples include magnetic or optical disks, solid-state memory devices, flash memory, USB devices provided with non-volatile memory, networked storage devices, and so on.

[0082] Devices implementing methods according to these disclosures can comprise hardware, firmware and / or software, and can take any of a variety of form factors. Typical examples of such form factors include servers, laptops, smartphones, small form factor personal computers, personal digital assistants, and so on. The functionality described herein also can be embodied in peripherals or add-in cards. Such functionality can also be implemented on a circuit board among different chips or different processes executing in a single device, by way of further example.

[0083] The instructions, media for conveying such instructions, computing resources for executing them, and other structures for supporting such computing resources are means for providing the functions described in these disclosures.

[0084] The disclosures shown and described above are only examples. Even though numerous characteristics and advantages of the present technology have been set forth in the foregoing description, together with details of the structure and function of the present disclosure, the disclosure is illustrative only, and changes may be made in the detail, especially in matters of shape, size and arrangement of the parts within the principles of the present disclosure to the full extent indicated by the broad general21107112419.1PATENT Attorney Docket No. 052592-858121 meaning of the terms used in the attached claims. It will therefore be appreciated that the examples described above may be modified within the scope of the appended claims.22107112419.1

Claims

PATENT Attorney Docket No. 052592-858121 ClaimsWhat is claimed is:

1. A method for diagnosing one or more bone disorders in a patient, the method comprising:receiving two or more images of the patient;segmenting, via a segmentation model, one or more bones in each of the two or more images;overlaying the one or more bones in each of the two or more images to form a progression overlaid image; anddetermining a progression of the one or more bone disorders based on the progression overlaid image.

2. The method of claim 1, wherein the one or more bone disorders includes one or more focal lesions and / or osteolytic lesions.

3. The method of claim 1 , wherein the two or more images comprise CT scans, PET scans, MRI images, or x-ray images.

4. The method of claim 1, wherein the two or more images comprise a first image taken at a first timepoint and a second image taken at a second timepoint.

5. The method of claim 4, wherein the first timepoint and the second timepoint are about one day to about ten years apart.

6. The method of claim 1 , the method further comprising converting the progression overlaid image to a bone mineral density image.

7. The method of claim 1 , the method further comprising:forming a standard image of a healthy patient;segmenting one or more healthy bones of the standard image of the healthy patient; and23107112419.1PATENT Attorney Docket No. 052592-858121 overlaying the one or more bones of at least one of the two or more images with the one or more healthy bones of the standard image of the healthy patient to form a diagnostic image.

8. The method of claim 7, the method further comprising converting the diagnostic image to a bone mineral density image.

9. A method for detecting and diagnosing one or more bone disorders in a patient, the method comprising:forming a standard image from a plurality of healthy patient images, the standard image comprising one or more average healthy bones;obtaining one or more images of the patient;segmenting, via a segmentation model, one or more bones of the one or more images;overlaying the one or more bones of the one or more images with the one or more average healthy bones, thereby producing a diagnostic overlaid image; and detecting the one or more bone disorders on the diagnostic overlaid image.

10. The method of claim 9, wherein forming the standard image comprises compiling images from two or more healthy patients using non-rigid registration.

11. The method of claim 9, wherein the one or more images of the patient include two or more images.

12. The method of claim 11 , the method further comprising:segmenting, via the segmentation model, the two or more images into one or more bones of each of the two or more images; andoverlaying the one or more bones of each of the two or more images to form a progression overlaid image.

13. The method of claim 12, the method further comprising determining a progression of the one or more bone disorders based on the progression overlaid image.24107112419.1PATENT Attorney Docket No. 052592-858121 14. The method of claim 9, the method further comprising converting the diagnostic overlaid image to a bone mineral density image.

15. The method of claim 9, wherein the one or more bone disorders include one or more focal lesions and / or osteolytic lesions.

16. A system for detecting and diagnosing one or more bone disorders, the system comprising:an imaging system operable to obtain one or more images of a patient; and at least one processor configured to:form a standard image from a plurality of healthy patient images, the standard image comprising one or more average healthy bones;receive the one or more images of the patient;segment, via a segmentation model, one or more bones of the one or more images; andoverlay the one or more bones of the one or more images with the one or more average healthy bones, thereby forming a diagnostic overlaid image.

17. The system of claim 16, wherein the one or more images of the patient include two or more images.

18. The system of claim 17, wherein the at least one processor is further configured to:segment, via the segmentation model, the two or more images into one or more bones of the two or more images;overlay the one or more bones of each of the two or more images to form a progression overlaid image; anddetermine a progression of the one or more bone disorders based on the progression overlaid image.

19. The system of claim 18, wherein the at least one processor is further configured to convert the diagnostic overlaid image to a bone mineral density image.25107112419.1PATENT Attorney Docket No. 052592-858121 20. The system of claim 16, wherein the standard image comprises images from two or more healthy patients.26107112419.1

Images