Methods and apparatus for deep learning based motion detection and correction for image reconstruction

Abstract

Systems and methods for motion estimation and correction in nuclear imaging systems are disclosed. In some examples, a processing device receives measurement data, such as positron emission tomography (PET) measurement data, from an image scanning system. The processing device applies a histo-imaging process to the measurement data and, based on applying the histo-imaging process to the measurement data, generates a histo-image. Further, the processing device applies a trained machine learning process to the histo-image and, based on applying the trained machine learning process to the histo-image, the processing device generates a short-frame image. The processing device also determines a position of a mass in the short-frame image, and generates motion data characterizing a motion of the mass based on the position. In some examples, a reconstructed image is generated based on the motion data.

Description

Docket No.: 2024P18223WOMETHODS AND APPARATUS FOR DEEP LEARNING BASED MOTION DETECTION AND CORRECTION FOR IMAGE RECONSTRUCTION FIELD

[0001] Aspects of the present disclosure relate in general to medical diagnostic systems and, more particularly, to reconstructing images from nuclear imaging systems for diagnostic and reporting purposes.BACKGROUND

[0002] Nuclear imaging systems can employ various technologies to capture images. For example, some nuclear imaging systems employ positron emission tomography (PET) to capture images. PET is a nuclear medicine imaging technique that produces tomographic images representing the distribution of positron emitting isotopes within a body. Some nuclear imaging systems employ computed tomography (CT), for example, as a co-modality. CT is an imaging technique that uses x-rays to produce anatomical images. Magnetic Resonance Imaging (MRI) is an imaging technique that uses magnetic fields and radio waves to generate anatomical and functional images, and can also be used as a co-modality. Typically, these nuclear imaging systems capture measurement data, and process the captured measurement data using mathematical algorithms to reconstruct medical images. Some nuclear imaging systems combine images from PET and CT scanners during an image fusion process to produce images that show information from both a PET scan and a CT scan (e.g., PET / CT systems). For these PET / CT systems, the CT measurement information can be used to correct the PET measurement data for attenuation (z.e., attenuation correction of the PET image). Similarly, some nuclear imaging systems combine images from PET and MRI scanners to produce images that show information from both a PET scan and an MRI scan.

[0003] These conventional systems, however, can suffer from drawbacks. For instance, subjects may move during imaging scans, thereby causing motion artifacts in reconstructed images. For instance, the movement may cause blurring, smearing, or ghosting in reconstructed images. In addition, motion may cause difficulties to a medical practitioner when diagnosing images. As such, there are opportunities to address deficiencies in nuclear imaging systems.Docket No.: 2024P18223WOSUMMARY

[0004] Systems and methods for detecting and correcting for motion based on deep learning-based processes are disclosed.

[0005] In some embodiments, a computer-implemented method includes receiving measurement data (e.g., time-of-flight sinogram data, list mode data) from an image scanning system. The method also includes applying a histo-imaging process to the measurement data and, based on applying the histo-imaging process to the measurement data, generating a histo-image. Further, the method includes applying a trained machine learning process to the histo-image and, based on applying the trained machine learning process to the histo-image, generating a short-frame image. The method also includes determining a position of a mass in the short-frame image. The method further includes generating motion data characterizing a motion of the mass based on the position.

[0006] In some embodiments, a non-transitory computer readable medium stores instructions that, when executed by at least one processor, cause the at least one processor to perform operations. The operations include receiving measurement data (e.g., TOF sinogram data, list mode data) from an image scanning system. The operations also include applying a histo-imaging process to the measurement data and, based on applying the histo-imaging process to the measurement data, generating a histo-image. Further, the operations include applying a trained machine learning process to the histo-image and, based on applying the trained machine learning process to the histo-image, generating a short-frame image. The operations also include determining a position of a mass in the short-frame image. The operations further include generating motion data characterizing a motion of the mass based on the position.

[0007] In some embodiments, an apparatus includes a memory storing instructions, and at least one processor communicatively coupled the memory. The at least one processor is configured to execute the instructions to perform operations. The operations include receiving measurement data (e.g., TOF sinogram data, list modeDocket No.: 2024P18223WOdata) from an image scanning system. The operations also include applying a histo-imaging process to the measurement data and, based on applying the histo-imaging process to the measurement data, generating a histo-image. Further, the operations include applying a trained machine learning process to the histo-image and, based on applying the trained machine learning process to the histo-image, generating a shortframe image. The operations also include determining a position of a mass in the shortframe image. The operations further include generating motion data characterizing a motion of the mass based on the position.BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The following will be apparent from elements of the figures, which are provided for illustrative purposes and are not necessarily drawn to scale.

[0009] FIG. 1 illustrates a nuclear image reconstruction system, in accordance with some embodiments.

[0010] FIG. 2 illustrates a block diagram of an example computing device that can perform one or more of the functions described herein, in accordance with some embodiments.

[0011] FIG. 3 illustrates a motion detection engine of a nuclear image reconstruction system, in accordance with some embodiments.

[0012] FIG. 4 illustrates motion detection signals, in accordance with some embodiments.

[0013] FIG. 5 is a flowchart of an example method to detect motion from measurement data, in accordance with some embodiments.

[0014] FIG. 6 is a flowchart of another example method to detect motion from measurement data, in accordance with some embodiments.Docket No.: 2024P18223WODETAILED DESCRIPTION

[0015] This description of the exemplary embodiments is intended to be read in connection with the accompanying drawings, which are to be considered part of the entire written description. Independent of the grammatical term usage, individuals with male, female, or other gender identities are included within the term.

[0016] The exemplary embodiments are described with respect to the claimed systems as well as with respect to the claimed methods. Furthermore, the exemplary embodiments are described with respect to methods and systems for image reconstruction, as well as with respect to methods and systems for training functions used for image reconstruction. Features, advantages, or alternative embodiments herein can be assigned to the other claimed objects and vice versa. For example, claims for the providing systems can be improved with features described or claimed in the context of the methods, and vice versa. In addition, the functional features of described or claimed methods are embodied by objective units of a providing system. Similarly, claims for methods and systems for training image reconstruction functions can be improved with features described or claimed in context of the methods and systems for image reconstruction, and vice versa.

[0017] Various embodiments of the present disclosure can employ machine learning methods or processes to provide clinical information from nuclear imaging systems. For example, the embodiments can employ machine learning methods or processes to reconstruct images based on captured measurement data, and provide the reconstructed images for clinical diagnosis. In some embodiments, machine learning methods or processes are trained, to improve the reconstruction of images.

[0018] With quantitative positron emission tomography (PET) or single-photon emission computed tomography (SPECT), motion estimation and correction is often performed retrospectively, and typically when conventionally reconstructed PET images are suggestive of motion artefacts. These data-driven motion detection methods that calculate motion information from raw PET data are typically noisy. In contrast to these conventional techniques, the embodiments described herein may provide low-noise, on-Docket No.: 2024P18223WOthe-fly (e.g., real-time or near real-time) motion information during scan acquisitions. The embodiments can indicate when motion occurs in at least near real-time throughout acquisitions, and can also facilitate a motion corrected reconstruction immediately upon completion of a scan. Moreover, the embodiments may reduce the use of costly postprocessing requirements.

[0019] For instance, in some examples, measurement data, such as PET measurement data, is received from an image scanning system (e.g., a PET scanner). Histo-images are generated based on the measurement data. For instance, a most likely annihilation position histogrammer may place photon coincidences as indicated by the measurement data into a histo-image representation. A trained machine learning model, such as a trained FastPET neural network, is applied to the histo-images to generate short-frame images (e.g., short-frame FastPET images) in at least near realtime. A short-frame may be a frame lasting between, for instance, .1 seconds and 1 second, such as .2 seconds. The short-frame images may be at least partially denoised. Motion between the short-frame images is then detected, and a motion signal is generated identifying the detected motion. For instance, motion may be detected by determining a location of a mass in each of the short frame images, and further determining that the locations differ (e.g., are offset in one or more directions by at least a minimum amount). The motion signal may be a one-dimensional signal characterizing the detected motion between the short frame images in one or more directions. For instance, in some examples, a first signal may characterize motion in a first direction (e.g., X direction, along transverse plane), a second signal may characterize motion in a second direction (e.g., Y direction, along coronal plane), and a third signal may characterize motion in a third direction (e.g., Z direction, along sagittal plane). In some instances, at least one of the short-frame images are corrected for motion based on the motion signal. In some instances, an indication of the motion is displayed (e.g., to a medical practitioner) based on the motion signal.

[0020] FIG. 1 illustrates an embodiment of a nuclear imaging system 100. As illustrated, nuclear imaging system 100 includes image scanning system 102 and image reconstruction system 104. Image scanning system 102 can be, for instance, a PETDocket No.: 2024P18223WOscanner that can capture PET scans, a SPECT scanner that can capture SPECT scans, a PET / CT scanner that can capture PET and CT scans, or a PET / MR scanner that can capture PET and MRI scans, for instance.

[0021] In this example, image scanning system 102 can scan a subject (e.g., with a PET scanner), and generate PET measurement data 101 (e.g., PET raw data, such as sinogram data or list mode data) based on the captured scans. The PET measurement data 101 can represent anything imaged in the scanner's field-of-view (FOV) containing positron emitting isotopes. For example, the PET measurement data 101 can represent whole-body image scans, such as image scans from a patient’s head to thigh, or a more directed scan, such as of a particular organ or body area. In some examples, the image scanning system 102 also captures anatomical scans (e.g., CT scans), and can provide an attenuation map 119 that characterizes densities of the scanned area. The attenuation map 119 can include, or be used to generate, a spatial distribution of linear attenuation coefficients to correct for attenuation in the PET measurement data 101.

[0022] Image reconstruction system 104 includes histo-image generation engine 112, short frame reconstruction engine 116, motion detection engine 122, and, in at least some examples, motion correction based image reconstruction engine 124. In some examples, all or parts of image reconstruction system 104, including histo-image generation engine 112, short frame reconstruction engine 116, motion detection engine 122, and motion correction based image reconstruction engine 124, are implemented in hardware, such as in one or more field-programmable gate arrays (FPGAs), one or more application-specific integrated circuits (ASICs), one or more state machines, one or more computing devices, digital circuitry, or any other suitable circuitry. In some examples, all or parts of image reconstruction system 104 can be implemented in software as executable instructions such that, when executed by one or more processors, cause the one or more processors to perform respective functions as described herein. The instructions can be stored in a non-transitory, computer-readable storage medium, for instance.Docket No.: 2024P18223WO

[0023] For example, FIG. 2 illustrates a computing device 200 that can be employed by the image reconstruction system 104. Computing device 200 can implement, for example, one or more of the functions of image reconstruction system 104 described herein.

[0024] Computing device 200 can include one or more processors 201 , working memory 202, one or more input / output devices 203, instruction memory 207, a transceiver 204, one or more communication ports 209, and a display 206, all operatively coupled to one or more data buses 208. Data buses 208 allow for communication among the various devices. Data buses 208 can include wired, or wireless, communication channels.

[0025] Processors 201 can include one or more distinct processors, each having one or more cores. Each of the distinct processors can have the same or different structure. Processors 201 can include one or more central processing units (CPUs), one or more graphics processing units (GPUs), application specific integrated circuits (ASICs), digital signal processors (DSPs), and the like. Processors 201 can be configured to perform a certain function or operation by executing code, stored on instruction memory 207, embodying the function or operation. For example, processors 201 can be configured to perform one or more of any function, method, or operation disclosed herein by executing instructions stored in instruction memory 207.

[0026] Instruction memory 207 can store instructions that can be accessed (e.g., read) and executed by processors 201. For example, instruction memory 207 can be a non-transitory, computer-readable storage medium such as a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), flash memory, a removable disk, CD-ROM, any non-volatile memory, or any other suitable memory. For example, instruction memory 207 can store instructions that, when executed by the one or more processors 201 , cause one or more processors 201 to perform one or more of the functions of image reconstruction system 104, such as one or more of the functions of any of histo-image generation engine 112, short frame reconstruction engine 116,Docket No.: 2024P18223WOmotion detection engine 122, and motion correction based image reconstruction engine 124, described herein.

[0027] Processors 201 can store data to, and read data from, working memory 202. For example, processors 201 can store a working set of instructions to working memory 202, such as instructions loaded from instruction memory 207. Processors 201 can also use working memory 202 to store dynamic data created during the operation of computing device 200. Working memory 202 can be a random access memory (RAM) such as a static random access memory (SRAM) or dynamic random access memory (DRAM), or any other suitable memory.

[0028] Input / output devices 203 can include any suitable device that allows for data input or output. For example, input-output devices 203 can include one or more of a keyboard, a touchpad, a mouse, a stylus, a touchscreen, a physical button, a speaker, a microphone, or any other suitable input or output device.

[0029] Communication port(s) 209 can include, for example, a serial port such as a universal asynchronous receiver / transmitter (UART) connection, a Universal Serial Bus (USB) connection, or any other suitable communication port or connection. In some examples, communication port(s) 207 allows for the programming of executable instructions in instruction memory 207. In some examples, communication port(s) 207 allow for the transfer (e.g., uploading or downloading) of data, such as histo-images 113 and short-frame image data 121 described further herein.

[0030] Display 206 can display user interface 205. User interfaces 205 can enable user interaction with computing device 200. For example, user interface 205 can be a user interface for an application that allows for the viewing of the motion corrected final image 191 described further herein. In some examples, a user can interact with user interface 205 by engaging (e.g., touching with a finger or stylus) an input / output device 203. In some examples, display 206 can be a touchscreen, where user interface 205 is displayed on the touchscreen. In some examples, display 206 can be used to display images, such as any of the reconstructed images (e.g., motion corrected final image 191) described herein.Docket No.: 2024P18223WO

[0031] Transceiver 204 allows for communication with a network, such as a WiFi® network, an Ethernet network, a cellular network, or any other suitable communication network. For example, if operating in a Wi-Fi® network, transceiver 204 is configured to allow communications with the Wi-Fi® network (e.g., and, in some examples, other devices on the Internet to which the Wi-Fi® network is connected). Processor(s) 201 is operable to receive data from, or send data to, a network via transceiver 204.

[0032] Referring back to FIG. 1 , histo-image generation engine 112 receives PET measurement data 101. Based on applying a histo-imaging process to the PET measurement data 101 , the histo-image generation engine 112 generates histo-images 113. The histo-image generation engine 112 can generate the histo-images based on any suitable method known in the art. For instance, the histo-image generation engine 112 can generate a histo-image 113 based on applying a time-of-flight (TOF) back-projection process to TOF sinograms (as characterized by the PET measurement data 101), or applying a back-projection process to each event of list mode data (as characterized by the PET measurement data 101). In some examples, the histo-image generation engine 112 can be a most likely annihilation position histogrammer that places photon coincidences into a histo-image representation, e.g., histo-image 113.

[0033] Further, the short frame reconstruction engine 116 receives histo-images 113 from the histo-image generation engine 112, and applies a trained machine learning process to the histo-images 113 to generate short-frame image data 121. The short-frame image data 121 characterizes short frame images, such as denoised histo-images. For example, the short frame reconstruction engine 116 may input a histo-image 113 into a trained neural network 117 and, based on inputting the histo-image 113 to the trained neural network 117, may generate the short-frame image data 121. In some examples, the trained neural network is a trained FastPET neural network, and can generate the short-frame image data 121 that is at least partially (denoised. For example, the trained neural network 117 may be based on the ll-NET neural network described in “FastPET: Near Real-Time PET Reconstruction from Histo-images Using aDocket No.: 2024P18223WONeural Network,” by Whiteley et al., 15 June 2020.1As recognized, a FastPET-based reconstruction process is fast and inherently capable of providing low-noise and robust images.2

[0034] In some examples, the short frame reconstruction engine 116 may receive a corresponding attenuation map 119 from the image scanning system 102. The attenuation map 119 may be generated based on corresponding CT scans, for instance. The short frame reconstruction engine 116 may apply the trained machine learning process to the histo-image 113 and the attenuation map 119 to generate the shortframe image data 121, which may be at least partially denoised. For example, the short frame reconstruction engine 116 may input the histo-image 113 and the attenuation map 119 into a trained neural network 117 and, based on inputting the histo-image 113 and the attenuation map 119 to the trained neural network 117, may generate the shortframe image data 121.

[0035] Motion detection engine 122 can receive the short-frame image data 121 from the short frame reconstruction engine 116, and can apply a motion detection process to the short-frame image data 121 to detect motion. The motion may be, for instance, a rigid motion (e.g., brain movements, patient movements), as well as non-rigid motions (e.g., respiratory and cardiac movements). The motion detection process can include, for example, detecting a change in a position of a mass (e.g., organ, tissue) between a first short-frame image data 121 A and a second short-frame image data 121B. Each of the first short-frame image data 121 A and the second short-frame image data 121 B characterize corresponding short-frame images. The second short-frame image data 121 B may be generated and received subsequent to the first short-frame image data 121 A. The change in the position of the mass may be, for instance, a change in a computed center-of-mass of a detected mass between the second shortframe image data 121 B and the first short-frame image data 121 A.1Available at h ttps: / / a xiv. orq / a bs / 2002.04865 (last accessed on 11 / 21 / 2024).2Id.Docket No.: 2024P18223WO

[0036] Further, and based on the motion detection process, the motion detection engine 122 can generate motion data 123 identifying and / or characterizing the detected motion. For instance, the motion data 123 can be a motion correction vector characterizing motion in one or more directions over a period of time. The motion data 123 can include low-noise, on-the-fly motion information that is generated during PET measurement data 101 acquisition. Further, the motion data 123 can indicate when motion occurs and, as described further below, can facilitate a motion-corrected reconstruction upon completion of the scan, in near real-time. In some examples, the motion data 123 is a signal (e.g., a digital or analog signal) characterizing the detected motion. For instance, the motion data 123 may be a one-dimensional signal characterizing the detected motion between the second short-frame image data 121 B and the first short-frame image data 121A.

[0037] For example, FIG. 4 illustrates a graph 400 that includes a data signal 402 that characterizes a detected position of a center of mass of a mass 401 detected in various reconstructed short-frame images 403 (e.g., FastPET generated short-frame images 403). In this example, the Y-axis represents a relative position of the center of mass (e.g., axial position), and the X-axis identifies time. As illustrated, the data signal 402 indicates that, as time progresses, the center of mass is moving. For instance, the center of mass is detected at a first position 404 at a first time 406, and at a second position 414 at a second time 416. The data signal 402 can indicate a distance from a baseline position. The baseline position may be determined, for instance, based on an initially generated short-frame image data 121. In some examples, the data signal 402 may be a digital signal, where the data signal 402 is at a first level (e.g., 5 Volts) when the detected position is above a first threshold 425 or below a second threshold 427, and at a second level (e.g., 0 Volts) when the detected position is between the first threshold 425 and second threshold 427, or at any of the first threshold 425 and second threshold 427. The data signal 402 is less noisy than, for instance, the data signal 452 illustrated in the graph 450, which is generated from raw histo-images (histo-images before short frame reconstruction). For instance, the data signal 452 requires preprocessing (for example, by Fourier domain filtering) before being used, which is impractical for real-time use.Docket No.: 2024P18223WO

[0038] Referring back to FIG. 1 , in some examples, the motion data 123 is stored in data repository 150, and can be provided for image reconstruction. For example, the motion correction based image reconstruction engine 124 receives the motion data 123 from the motion detection engine 122, as well as the short-frame image data 121 from the short frame reconstruction engine 116. The motion correction based image reconstruction engine 124 may apply a motion correction based reconstruction process to the motion data 123 and the short-frame image data 121 to generate motion corrected final image 191. For instance, the motion correction based image reconstruction engine 124 may generate longer gated images (e.g., longer gated than the short frame image data 121 ) based on the short frame image data 121 , and may motion correct the longer gated images based on the motion data 123. For example, the motion correction based image reconstruction engine 124 may determine a pixel offset (e.g., a 3-dimensional pixel offset) based on the motion data 123, and may adjust pixel locations of the longer gated images based on the determined pixel offset. In some examples, the motion correction based image reconstruction engine 124 adjusts the pixel locations of the longer gated images when the pixel offset is above a corresponding threshold. The motion correction based image reconstruction engine 124 stores the motion corrected final image 191 in data repository 150. In some examples, the motion correction based image reconstruction engine 124 provides the motion corrected final image 191 for display, such as to a display 140.

[0039] Additionally or alternatively, in some examples, the motion correction based image reconstruction engine 124 generates motion display data 171 based on the motion data 123, where the motion display data 171 characterizes a visual representation (e.g., a graph, chart, icon, etc.) of the motion data 123. For instance, the motion display data 171 may include vertical bars identifying corresponding motion levels for various scans. In some examples, the motion display data 171 may indicate motion levels with varying graphical features, such as icon size, icon color, icon placement, etc. For instance, the motion display data 171 may indicate motion at or above a corresponding threshold level in one color (e.g., red), and indicate motion above the threshold level in another color (e.g., green).Docket No.: 2024P18223WO

[0040] FIG. 3 illustrates an example of the motion detection engine 122 of FIG. 1. As illustrated, the motion detection engine 122 may include a mass location engine 302, a mass movement determination engine 304, and a motion threshold engine 306. In this example, the mass location engine 302 receives a first short-frame image data 121 A. The mass location engine 302 may detect a mass in the first short-frame image data 121 A, and may further determine a center of the detected mass. The center of the detected mass may be a 3-dimensional pixel position, where each pixel position is an average pixel position for the detected mass in the corresponding direction, for instance. The mass location engine 302 may generate first mass location data 303A characterizing the position of the mass (e.g., the center of the mass).

[0041] In addition, and subsequent to receiving the first short-frame image data 121 A, the mass location engine 302 receives a second short-frame image data 121 B. The mass location engine 302 may detect the mass in the second short-frame image data 121 B, and may further determine the center of the detected mass in the second short-frame image data 121 B. The mass location engine 302 may generate second mass location data 303B characterizing the position of the mass in the second shortframe image data 121B.

[0042] The mass movement determination engine 304 receives the first mass location data 303A and the second mass location data 303B, and determines whether the center of the mass has moved. For instance, the mass movement determination engine 304 may compare the first mass location data 303A with the second mass location data 303B to determine a difference in positions of the center of the mass. The mass movement determination engine 304 generates mass movement data 305 characterizing the determined mass movement. In some instances, the mass movement determination engine 304 subtracts the pixel positions in each of the three dimensions to generate the mass movement data 305. As such, the mass movement data 305 can characterize a movement in an X direction (e.g., along the transverse plane), a movement in a Y direction (e.g., along the coronal plane), and a movement in a Z direction (e.g., along the sagittal plane).Docket No.: 2024P18223WO

[0043] Further, the motion threshold engine 306 receives the mass movement data 305 from the mass movement determination engine 304, and generates the motion data 123 based on the movement indicated by the mass movement data 305. For instance, the motion threshold engine 306 may generate motion data 123 for one direction at a base level (e.g., at a baseline value such as zero) if the mass movement data 305 indicates movement in that direction that is under a corresponding threshold, and may generate the motion data 123 at another level (e.g., above or below zero) when the movement is at or above the corresponding threshold. Similarly, the motion threshold engine 306 may generate motion data 123 for each other direction at the base level if the mass movement data 305 indicates movement in each corresponding direction that is under the corresponding threshold, and at another level when the movement is at or above each corresponding threshold. In other words, the motion data 123 can identify a movement in each direction separately.

[0044] In other examples, the motion data 123 can identify an overall movement, where the motion threshold engine 306 determines the overall movement based on a combination of the movements in each direction as identified by the mass movement data 305. For instance, the motion threshold engine 306 may compute a sum of the squares of the mass movement data 305 ( / .e., the difference in the X direction squared, plus the difference in the Y direction squared, plus the difference in the Z direction squared), and compare the sum of the squares to a threshold to generate the motion data 123.

[0045] FIG. 5 is a flowchart of an example method 500 to detect motion from measurement data. The method can be performed by, for example, the image reconstruction system 104.

[0046] Beginning at block 502, measurement data, such as PET measurement data 101, is received. At block 504, a histo-imaging process is applied to the measurement data and, based on the histo-imaging process, a histo-image (e.g., histo-image 113) is generated.Docket No.: 2024P18223WO

[0047] Further, at block 506, a trained machine learning process is applied to the histo-image and, based on the trained machine learning process, short-frame image data (e.g. , short-frame image data 121) is generated. For instance, the short-frame image data may be generated based on applying a FastPET neural network to the histo-image. The short-frame image data characterizes a short-framed image.Moreover, at block 508, a motion detection process is applied to the short-frame image data and, based on the motion detection process, motion data (e.g., motion data 123) is generated. The motion data may be a signal that characterizes mass movement between the short-frame image and a previously generated short-frame image, for example.

[0048] Further, at block 510, graphical elements (e.g., motion display data 171) are generated based on the motion data. At block 512, the graphical elements are transmitted for display. For example, the graphical elements may be displayed on display 140 or 206.

[0049] FIG. 6 is a flowchart of an example method 600 to detect motion from measurement data. The method can be performed by, for example, the image reconstruction system 104.

[0050] Beginning at block 602, a first short-frame image is received. At block 604, a first position of a mass is determined based on the first short-frame image.Further, at block 606, a second short-frame image is received. At block 608, a second position of the mass is determined based on the second short-frame image.

[0051] Proceeding to block 610, a mass movement value is generated based on the first position of the mass and the second position of the mass. The mass movement value characterizes a movement of the mass. For example, a difference in each of three dimensions may be determined based on corresponding direction values (e.g., X, Y, and Z positions) of the first position of the mass and the second position of the mass.

[0052] At block 612, the mass movement value (e.g., in each direction) is compared to a corresponding threshold. If the mass movement value is at or above theDocket No.: 2024P18223WOthreshold, the process proceeds to block 614, where motion data (e.g., motion data 123) is generated to include a first value (e.g., above or below zero). If, however, at block 612 the mass movement value is below the threshold, the process proceeds to block 616, where the motion data is generated to include a second value (e.g., zero).

[0053] At block 618, the motion data is stored in a data repository. In some examples, the motion data is used to correct for motion in a reconstructed image.

[0054] The following is a list of non-limiting illustrative embodiments disclosed herein:

[0055] Illustrative Embodiment 1 : A computer-implemented method comprising:receiving measurement data from an image scanning system;applying a histo-imaging process to the measurement data and, based on applying the histo-imaging process to the measurement data, generating a histo-image;applying a trained machine learning process to the histo-image and, based on applying the trained machine learning process to the histo-image, generating a shortframe image;determining a position of a mass in the short-frame image; andgenerating motion data characterizing a motion of the mass based on the position.

[0056] Illustrative Embodiment 2: The computer-implemented method of illustrative embodiment 1 wherein the position of the mass is a first position and the short-frame image is a first short-frame image, the computer-implemented method further comprising:determining a second position of the mass in a second short-frame image; andDocket No.: 2024P18223WO generating the motion data characterizing the motion of the mass based on the first position and the second position.

[0057] Illustrative Embodiment 3: The computer-implemented method of illustrative embodiment 2, further comprising:determining a difference between the first position of the mass and the second position of the mass; andgenerating the motion data characterizing the motion of the mass based on the difference.

[0058] Illustrative Embodiment 4: The computer-implemented method of illustrative embodiment 3, further comprising:comparing the difference to a threshold; andgenerating the motion data characterizing the motion of the mass based on the comparison.

[0059] Illustrative Embodiment 5: The computer-implemented method of any of illustrative embodiments 2-4, further comprising:receiving additional measurement data from the image scanning system; applying the histo-imaging process to the additional measurement data and, based on applying the histo-imaging process to the additional measurement data, generating a second histo-image; andDocket No.: 2024P18223WO applying the trained machine learning process to the second histo-image and, based on applying the trained machine learning process to the second histo-image, generating the second short-frame image.

[0060] Illustrative Embodiment 6: The computer-implemented method of any of illustrative embodiments 1-5, further comprising correcting the short-frame image for motion based on the motion data.

[0061] Illustrative Embodiment 7: The computer-implemented method of any of illustrative embodiments 1-6, further comprising generating graphical elements based on the motion data, and providing the graphical elements for display.

[0062] Illustrative Embodiment 8: The computer-implemented method of any of illustrative embodiments 1-7, wherein the measurement data is positron emission tomography (PET) measurement data.

[0063] Illustrative Embodiment 9: The computer-implemented method of any of illustrative embodiments 1-8, wherein the trained machine learning process is based on a FastPET network.

[0064] Illustrative Embodiment 10: The computer-implemented method of any of illustrative embodiments 1-9, further comprising:receiving an attenuation map; andDocket No.: 2024P18223WO applying the trained machine learning process to the histo-image and the attenuation map and, based on applying the trained machine learning process to the histo-image and the attenuation map, generating the short-frame image.

[0065] Illustrative Embodiment 11: A non-transitory, computer readable medium storing instructions that, when executed by at least one processor, cause the at least one processor to perform operations comprising:receiving measurement data from an image scanning system;applying a histo-imaging process to the measurement data and, based on applying the histo-imaging process to the measurement data, generating a histo-image;applying a trained machine learning process to the histo-image and, based on applying the trained machine learning process to the histo-image, generating a shortframe image;determining a position of a mass in the short-frame image; andgenerating motion data characterizing a motion of the mass based on the position.

[0066] Illustrative Embodiment 12: The non-transitory, computer readable medium of illustrative embodiment 11 wherein the position of the mass is a first position and the short-frame image is a first short-frame image, and storing instructions that, when executed by the at least one processor, further cause the at least one processor to perform operations comprising:determining a second position of the mass in a second short-frame image; andDocket No.: 2024P18223WO generating the motion data characterizing the motion of the mass based on the first position and the second position.

[0067] Illustrative Embodiment 13: The non-transitory, computer readable medium of illustrative embodiment 12 storing instructions that, when executed by the at least one processor, further cause the at least one processor to perform operations comprising:determining a difference between the first position of the mass and the second position of the mass; andgenerating the motion data characterizing the motion of the mass based on the difference.

[0068] Illustrative Embodiment 14: The non-transitory, computer readable medium of illustrative embodiment 13 storing instructions that, when executed by the at least one processor, further cause the at least one processor to perform operations comprising:comparing the difference to a threshold; andgenerating the motion data characterizing the motion of the mass based on the comparison.

[0069] Illustrative Embodiment 15: The computer-implemented method of any of illustrative embodiments 12-14, further comprising:receiving additional measurement data from the image scanning system;Docket No.: 2024P18223WO applying the histo-imaging process to the additional measurement data and, based on applying the histo-imaging process to the additional measurement data, generating a second histo-image; andapplying the trained machine learning process to the second histo-image and, based on applying the trained machine learning process to the second histo-image, generating the second short-frame image.

[0070] Illustrative Embodiment 16: The computer-implemented method of any of illustrative embodiments 11-15, further comprising correcting the short-frame image for motion based on the motion data.

[0071] Illustrative Embodiment 17: The non-transitory, computer readable medium of any of illustrative embodiments 11-16 storing instructions that, when executed by the at least one processor, further cause the at least one processor to perform operations comprising generating graphical elements based on the motion data, and providing the graphical elements for display.

[0072] Illustrative Embodiment 18: The non-transitory, computer readable medium of any of illustrative embodiments 11-17, wherein the measurement data is positron emission tomography (PET) measurement data.Docket No.: 2024P18223WO

[0073] Illustrative Embodiment 19: non-transitory, computer readable medium of any of illustrative embodiments 11-18, wherein the trained machine learning process is based on a FastPET network.

[0074] Illustrative Embodiment 20: The non-transitory, computer readable medium of any of illustrative embodiments 11-19 storing instructions that, when executed by the at least one processor, further cause the at least one processor to perform operations comprising:receiving an attenuation map; andapplying the trained machine learning process to the histo-image and the attenuation map and, based on applying the trained machine learning process to the histo-image and the attenuation map, generating the short-frame image.

[0075] Illustrative Embodiment 21: A system comprising:a memory storing instructions; andat least one processor communicatively coupled to the memory and configured to execute the instructions to:receive measurement data from an image scanning system;apply a histo-imaging process to the measurement data and, based on applying the histo-imaging process to the measurement data, generating a histo-image;apply a trained machine learning process to the histo-image and, based on applying the trained machine learning process to the histo-image, generating a shortframe image;Docket No.: 2024P18223WO determine a position of a mass in the short-frame image; and generate motion data characterizing a motion of the mass based on the position.

[0076] Illustrative Embodiment 22: The system of illustrative embodiment 21 , wherein the position of the mass is a first position and the short-frame image is a first short-frame image, and wherein the at least one processor is configured to execute the instructions to:determine a second position of the mass in a second short-frame image; and generate the motion data characterizing the motion of the mass based on the first position and the second position.

[0077] Illustrative Embodiment 23: The system of illustrative embodiment 22, wherein the at least one processor is configured to execute the instructions to:determine a difference between the first position of the mass and the second position of the mass; andgenerate the motion data characterizing the motion of the mass based on the difference.

[0078] Illustrative Embodiment 24: The system of illustrative embodiment 23, wherein the at least one processor is configured to execute the instructions to:compare the difference to a threshold; andDocket No.: 2024P18223WO generate the motion data characterizing the motion of the mass based on the comparison.

[0079] Illustrative Embodiment 25: The system of any of illustrative embodiments 22-24, wherein the at least one processor is configured to execute the instructions to:receive additional measurement data from the image scanning system; apply the histo-imaging process to the additional measurement data and, based on the application of the histo-imaging process to the additional measurement data, generate a second histo-image; andapply the trained machine learning process to the second histo-image and, based on the application of the trained machine learning process to the second histo-image, generate the second short-frame image.

[0080] Illustrative Embodiment 26: The system of any of illustrative embodiments 21-25, wherein the at least one processor is configured to execute the instructions to correct the short-frame image for motion based on the motion data.

[0081] Illustrative Embodiment 27: The system of any of illustrative embodiments 21-26, wherein the at least one processor is configured to execute the instructions to generate graphical elements based on the motion data, and providing the graphical elements for display.Docket No.: 2024P18223WO

[0082] Illustrative Embodiment 28: The system of any of illustrative embodiments 21-27, wherein the measurement data is positron emission tomography (PET) measurement data.

[0083] Illustrative Embodiment 29: The system of any of illustrative embodiments 21-28, wherein the trained machine learning process is based on a FastPET network.

[0084] Illustrative Embodiment 30: The system of any of illustrative embodiments 21-29, wherein the at least one processor is configured to execute the instructions to:receive an attenuation map; andapply the trained machine learning process to the histo-image and the attenuation map and, based on the application of the trained machine learning process to the histo-image and the attenuation map, generate the short-frame image.

[0085] The apparatuses and processes are not limited to the specific embodiments described herein. In addition, components of each apparatus and each process can be practiced independent and separate from other components and processes described herein.

[0086] The previous description of embodiments is provided to enable any person skilled in the art to practice the disclosure. The various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein can be applied to other embodiments without the use of inventive faculty. The present disclosure is not intended to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

Docket No.: 2024P18223WO What is claimed is:

1. A computer-implemented method comprising:receiving measurement data from an image scanning system;applying a histo-imaging process to the measurement data and, based on applying the histo-imaging process to the measurement data, generating a histo-image;applying a trained machine learning process to the histo-image and, based on applying the trained machine learning process to the histo-image, generating a shortframe image;determining a position of a mass in the short-frame image; andgenerating motion data characterizing a motion of the mass based on the position.

2. The computer-implemented method of claim 1 wherein the position of the mass is a first position and the short-frame image is a first short-frame image, the computer-implemented method further comprising:determining a second position of the mass in a second short-frame image; and generating the motion data characterizing the motion of the mass based on the first position and the second position.

3. The computer-implemented method of claim 2, further comprising:determining a difference between the first position of the mass and the second position of the mass; andgenerating the motion data characterizing the motion of the mass based on the difference.Docket No.: 2024P18223WO4. The computer-implemented method of claim 3, further comprising:comparing the difference to a threshold; andgenerating the motion data characterizing the motion of the mass based on the comparison.

5. The computer-implemented method of claim 2, further comprising:receiving additional measurement data from the image scanning system; applying the histo-imaging process to the additional measurement data and, based on applying the histo-imaging process to the additional measurement data, generating a second histo-image; andapplying the trained machine learning process to the second histo-image and, based on applying the trained machine learning process to the second histo-image, generating the second short-frame image.

6. The computer-implemented method of claim 1 , further comprising correcting the short-frame image for motion based on the motion data.

7. The computer-implemented method of claim 1 , further comprising generating graphical elements based on the motion data, and providing the graphical elements for display.Docket No.: 2024P18223WO 8. The computer-implemented method of claim 1 , wherein the measurement data is positron emission tomography (PET) measurement data.

9. The computer-implemented method of claim 1 , wherein the trained machine learning process is based on a FastPET network.

10. The computer-implemented method of claim 1 , further comprising:receiving an attenuation map; andapplying the trained machine learning process to the histo-image and the attenuation map and, based on applying the trained machine learning process to the histo-image and the attenuation map, generating the short-frame image.

11. A non-transitory, computer readable medium storing instructions that, when executed by at least one processor, cause the at least one processor to perform operations comprising:receiving measurement data from an image scanning system;applying a histo-imaging process to the measurement data and, based on applying the histo-imaging process to the measurement data, generating a histo-image;applying a trained machine learning process to the histo-image and, based on applying the trained machine learning process to the histo-image, generating a shortframe image;determining a position of a mass in the short-frame image; andDocket No.: 2024P18223WO generating motion data characterizing a motion of the mass based on the position.

12. The non-transitory, computer readable medium of claim 11 wherein the position of the mass is a first position and the short-frame image is a first short-frame image, and storing instructions that, when executed by the at least one processor, further cause the at least one processor to perform operations comprising:determining a second position of the mass in a second short-frame image; and generating the motion data characterizing the motion of the mass based on the first position and the second position.

13. The non-transitory, computer readable medium of claim 12 storing instructions that, when executed by the at least one processor, further cause the at least one processor to perform operations comprising:determining a difference between the first position of the mass and the second position of the mass; andgenerating the motion data characterizing the motion of the mass based on the difference.

14. The non-transitory, computer readable medium of claim 13 storing instructions that, when executed by the at least one processor, further cause the at least one processor to perform operations comprising:comparing the difference to a threshold; andDocket No.: 2024P18223WO generating the motion data characterizing the motion of the mass based on the comparison.

15. The non-transitory, computer readable medium of claim 11 storing instructions that, when executed by the at least one processor, further cause the at least one processor to perform operations comprising generating graphical elements based on the motion data, and providing the graphical elements for display.

16. A system comprising:a memory storing instructions; andat least one processor communicatively coupled to the memory and configured to execute the instructions to:receive measurement data from an image scanning system;apply a histo-imaging process to the measurement data and, based on applying the histo-imaging process to the measurement data, generating a histo-image;apply a trained machine learning process to the histo-image and, based on applying the trained machine learning process to the histo-image, generating a shortframe image;determine a position of a mass in the short-frame image; and generate motion data characterizing a motion of the mass based on the position.Docket No.: 2024P18223WO 17. The system of claim 16, wherein the position of the mass is a first position and the short-frame image is a first short-frame image, and wherein the at least one processor is configured to execute the instructions to:determine a second position of the mass in a second short-frame image; and generate the motion data characterizing the motion of the mass based on the first position and the second position.

18. The system of claim 17, wherein the at least one processor is configured to execute the instructions to:determine a difference between the first position of the mass and the second position of the mass; andgenerate the motion data characterizing the motion of the mass based on the difference.

19. The system of claim 18, wherein the at least one processor is configured to execute the instructions to:compare the difference to a threshold; andgenerate the motion data characterizing the motion of the mass based on the comparison.

20. The system of claim 16, wherein the at least one processor is configured to execute the instructions to generate graphical elements based on the motion data, and providing the graphical elements for display.

Images