System and method for hyper-time-of-flight resolution

Abstract

Systems and methods of generating histogram images with improved resolution are disclosed. A system includes: a positron emission tomography (PET) imaging modality configured to perform a first scan to obtain a first PET data set; and a processor, and a processor configured to back-project the first PET data set to generate a first histogram image having a first resolution, input the first histogram image to the trained neural network, receive a second histogram image from the trained neural network, and input the second histogram image to a reconstruction process configured to generate a reconstructed PET image. The second histogram image has a second resolution higher than the first resolution. The second histogram image represents the first PET data set.

Description

Background Art

[0001] According to conventional positron emission tomography (PET) imaging, a tracer compound including a radionuclide is introduced into the patient's body by injection, inhalation or ingestion. The radioactive decay of the radionuclide generates a positron, which eventually encounters an electron and is thereby annihilated. The annihilation event produces two gamma photons traveling in generally opposite directions. Thus, an annihilation event is identified when two detectors disposed on opposite sides of the patient's body detect the arrival of two oppositely traveling gamma photons within a specific coincidence time window.

[0002] Because the two gamma photons travel in roughly opposite directions, the positions of the two detectors determine the line of response (LOR) along which the annihilation event occurs. Time-of-flight (TOF) PET measures the difference between the detection times of the two gamma photons produced by the annihilation event. This difference can be used to estimate the specific location along the LOR where the annihilation event occurred. The resolution of a PET scanner can be defined based on the temporal resolution of the difference between the detection times, and is typically measured in the range of picoseconds (ps).

[0003] Although the resolution of PET scanners has been improved over time, current low-cost systems can provide a response time resolution of only about 600 ps, ​​while more expensive systems can provide higher resolutions, such as 200 ps. Although PET scanners with 10 ps response times have been discussed, such scanners are currently still theoretical. Improvements in response time resolution directly lead to improvements in images and data, thereby providing better clinical outcomes. Summary of the invention

[0004] In various embodiments, a system is disclosed. The system includes: a PET imaging modality configured to perform a first scan to acquire a first PET data set; and a processor configured to: back-project the first PET data set to generate a first histogram image having a first resolution, input the first histogram image to a trained neural network, receive a second histogram image from the trained neural network, and input the second histogram image to a reconstruction process configured to generate a reconstructed PET image. The second histogram image is an improved version of the first histogram image, having a second resolution that is better than the first resolution.

[0005] In various embodiments, a method for generating a reconstructed positron emission tomography (PET) image is disclosed. The method includes the steps of performing a first scan to acquire a first PET data set, back-projecting the first PET data set to generate a first histogram image having a first resolution, inputting the first histogram image to a trained neural network, receiving a second histogram image from the trained neural network, and inputting the second histogram image to a reconstruction process configured to generate a reconstructed PET image. The second histogram image is an improved version of the first histogram image having a second resolution that is better than the first resolution.

[0006] In various embodiments, a method for training a neural network to generate a histogram image with improved resolution is disclosed. The method includes the following steps: receiving a training data set, the training data set including a first plurality of histogram images having a first resolution and a second plurality of histogram images having a second resolution. Each of the first plurality of histogram images has at least one corresponding histogram image of the second plurality of histogram images. The second resolution of each of the second plurality of histogram images is better than the first resolution of the corresponding histogram image of the first plurality of histogram images. The method also includes the following steps: inputting each of the first plurality of histogram images to a neural network configured to generate a histogram image with improved resolution having an estimated second resolution; and modifying the neural network based on a difference between each of the histogram images with improved resolution and a histogram image of the second plurality of histogram images corresponding to the histogram image of the first plurality of histogram images provided to the neural network. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] The features and advantages of the present invention will be more fully disclosed in or become apparent from the following detailed description of the preferred embodiments, which will be described in conjunction with the accompanying drawings. Figure 1 are contemplated herein, wherein like numerals refer to like parts, and further wherein:

[0008] Figure 1 A nuclear imaging system is shown in accordance with some embodiments.

[0009] Figure 2 An embodiment of an artificial neural network in accordance with some embodiments is shown.

[0010] Figure 3 is a flow chart illustrating a method of generating a reconstructed PET image using a resolution-improved histogram image, according to some embodiments.

[0011] Figure 4 is configured to implement Figure 3 A block diagram of a system showing various parts of the method.

[0012] Figure 5 is configured to implement Figure 3 Block diagram of the system showing the various parts of the method, including the histogram image generated at 550ps and the histogram image with 200ps resolution improvement.

[0013] Figure 6 is configured to implement Figure 3 Block diagram of the system showing the various parts of the method, including the histogram image generated at 200 ps and the histogram image with 10 ps resolution improvement.

[0014] Figure 7 is a flow chart illustrating a method of training a neural network, according to various embodiments.

[0015] Figure 8 According to some embodiments, Figure 7 00143] The process flow of the various steps of the method for training a neural network is shown in FIG.

[0016] Fig. 9 A neural network architecture according to some embodiments is shown.

[0017] Fig.10 A computer system configured to implement one or more processes according to some embodiments is shown. DETAILED DESCRIPTION

[0018] This description of the exemplary embodiments is intended to be read in conjunction with the accompanying drawings, which are to be considered a part of the entire written description.

[0019] In the following, various embodiments are described for the claimed systems and for the claimed methods. Features, advantages, or alternative embodiments herein may be assigned to other claimed objects, and vice versa. In other words, claims for systems may be improved with features described or claimed in the context of methods. In this case, the functional features of the method are embodied by the target unit of the system.

[0020] In addition, various embodiments are described below for methods and systems for generating PET images with improved resolution using data obtained by a PET scanner with poor response time resolution and a trained neural network, and for methods and systems for training a neural network to generate histogram images with improved resolution from lower resolution histogram images. Features, advantages, or alternative embodiments herein may be assigned to other claimed objects, and vice versa. In other words, claims for methods and systems for training a neural network to generate high resolution histogram images from lower resolution PET data may be improved using features described or claimed in the context of methods and systems for generating PET images with improved resolution using data obtained by a lower resolution PET scanner and a trained neural network, and vice versa.

[0021] As used herein, the term "histogram image" is used to refer to a data representation of PET data in image space, and a histogram image representing PET data is a data representation of specific PET data in image space. In addition, as used herein, an "improved histogram image" refers to an improved version of a histogram image in image space that represents an improved version of specific PET data (i.e., mimicking the PET data as if it had been acquired at a better resolution). Although an embodiment is illustrated as including a visual representation of a histogram image, it will be appreciated that such a visual representation represents a data structure generated and used by the various components of the systems and methods disclosed herein. Operations performed on or including the histogram images disclosed herein can be performed on the underlying data of the data representation without converting the data into a visual format. References to improvements in the resolution or quality of an image should be interpreted as including improvements in the resolution of the underlying data of the data representation.

[0022] In general, the trained function mimics the cognitive functions that humans associate with other human minds. In particular, through training based on training data, the trained function is able to adapt to new environments and is able to detect and infer patterns.

[0023] In general, the parameters of the trained function can be adapted by means of training. In particular, a combination of supervised training, semi-supervised training, unsupervised training, reinforcement learning and / or active learning can be used. In addition, representation learning (an alternative term is "feature learning") can be used. In particular, the parameters of the trained function can be adapted iteratively through several training steps.

[0024] In particular, the trained function may include a neural network, a support vector machine, a decision tree and / or a Bayesian network, and / or the trained function may be based on k-means clustering, Q-learning, a genetic algorithm and / or association rules. In particular, the neural network may be a deep neural network, a convolutional neural network, a convolutional deep neural network, an adversarial neural network, a generative adversarial network (GAN), a deep adversarial neural network, and / or any other suitable neural network known or later developed.

[0025] In various embodiments, a neural network is disclosed that improves the resolution of an input histogram image to simulate a scan with increased resolution. The trained neural network is configured to receive a histogram image having a first resolution and generate an output histogram image having a simulated improved resolution. The resolution-improved histogram image can be used for clinical analysis and / or as input to additional methods for reconstructing and / or analyzing PET image data. In various embodiments, the trained neural network can be configured to increase the resolution of a low-resolution scan, for example from 600 ps to 200 ps, ​​and / or to increase the resolution of a high-resolution scan to an even higher resolution, for example from 200 ps to 10 ps. In some embodiments, the trained neural network is configured to generate a histogram image having a better effective resolution than is currently achievable using clinically deployed devices.

[0026] Figure 1 An embodiment of a nuclear imaging system 2 according to some embodiments is shown. The nuclear imaging system 2 includes a scanner for at least a first modality 12 provided in a first gantry 16a. The first modality 12 may include any suitable imaging modality, such as a positron emission tomography (PET) modality. A patient 17 lies on a movable bed 18 that can be movable within a gantry 19. In some embodiments, the nuclear imaging system 2 includes a scanner for a second imaging modality 14 provided in a second gantry 16b. The second imaging modality 14 may be any suitable imaging modality, such as, for example, a PET modality, a SPECT modality, a CT modality, a magnetic resonance (MR) modality, and / or any other suitable imaging modality. Each of the first modality 12 and / or the second modality 14 may include one or more detectors 50, the one or more detectors 50 being configured to detect annihilation photons, gamma rays, and / or other nuclear imaging events. In some embodiments, the one or more detectors 50 generate background radiation data during scanning.

[0027] Scan data from the first modality 12 and / or the second modality 14 is stored at one or more computer databases 40 and processed by one or more computer processors 60 of the computer system 30 . Figure 1The graphical depiction of the computer system 30 in is provided by way of illustration only, and the computer system 30 may include one or more separate computing devices. The nuclear imaging data set may be provided by the first modality 12, the second modality 14, and / or may be provided as a separate data set, such as, for example, from a memory coupled to the computer system 30. The computer system 30 may include one or more processing electronics for processing signals received from one of the plurality of detectors 50. In some embodiments, the scan data includes attenuation based on background radiation. The computer system 30 may use one or more attenuation maps based on background radiation to correct for background radiation attenuation during image reconstruction.

[0028] In some embodiments, the computer system 30 is configured to generate a reconstructed PET image using a resolution-improved histogram image having an effective resolution higher than the resolution of the first modality 12 (e.g., a PET scanning modality). The resolution-improved histogram image is generated by a neural network implemented by a processor 60, which is configured to generate a histogram image having an effective resolution higher than the first modality 12 could otherwise produce. In some embodiments, the computer system 30 is configured to generate the resolution-improved histogram image and store the resolution-improved histogram image in a computer database 40 for later use in PET image reconstruction and / or training additional or modified neural networks.

[0029] Figure 2 An embodiment of an artificial neural network 100 is shown. Alternative terms for "artificial neural network" are "neural network", "artificial neural net", "neural net" or "trained function". Figure 2 1 is provided as an example of an artificial neural network 100, and the structures disclosed therein are not limited with respect to the present disclosure. The artificial neural network 100 includes nodes 120-132 and edges 140-142, wherein each edge 140-142 is a directed connection from a first node 120-132 to a second node 120-132. Generally, the first node 120-132 and the second node 120-132 are different nodes 120-132, although the first node 120-132 and the second node 120-132 may also be the same. For example, in Figure 2 , edge 140 is a directed connection from node 120 to node 123, and edge 142 is a directed connection from node 130 to node 132. Edges 140-142 from a first node 120-132 to a second node 120-132 are also labeled as "incoming edges" of the second node 120-132 and "outgoing edges" of the first node 120-132.

[0030] In this embodiment, the nodes 120-132 of the artificial neural network 100 can be arranged in layers 110-113, wherein the layers can include an inherent order introduced by the edges 140-142 between the nodes 120-132. In particular, the edges 140-142 can exist only between adjacent node layers. In the embodiment shown, there is an input layer 110 that includes only nodes 120-122 without incoming edges, an output layer 113 that includes only nodes 131, 132 without outgoing edges, and hidden layers 111, 112 between the input layer 110 and the output layer 113. In general, the number of hidden layers 111, 112 can be arbitrarily selected. The number of nodes 120-122 within the input layer 110 is generally related to the number of input values ​​of the neural network, and the number of nodes 131, 132 within the output layer 113 is generally related to the number of output values ​​of the neural network.

[0031] In particular, a (real) number may be assigned as a value to each node 120-132 of the neural network 100. Here, x (n) i Indicates the value of the i-th node 120-132 of the n-th layer 110-113. The values ​​of the nodes 120-122 of the input layer 110 are equivalent to the input values ​​of the neural network 100, and the values ​​of the nodes 131, 132 of the output layer 113 are equivalent to the output values ​​of the neural network 100. In addition, each edge 140-142 may include a weight as a real number, in particular, the weight is a real number in the interval [-1,1] or in the interval [0,1]. Here, w (m,n) i,j The weight of the edge between the i-th node 120-132 of the m-th layer 110-113 and the j-th node 120-132 of the n-th layer 110-113 is indicated. (n,n+1) i,j Definition of the abbreviation w (n) i,j .

[0032] Specifically, to calculate the output value of the neural network 100, the input value is propagated through the neural network. Specifically, the values ​​of the nodes 120-132 of the (n+1)th layer 110-113 can be calculated based on the values ​​of the nodes 120-132 of the nth layer 110-113 by the following formula:

[0033]

[0034] In this article, function f is a transfer function (another term is "activation function"). Known transfer functions are step functions, sigmoid functions (e.g., logistic function, generalized logistic function, hyperbolic tangent, inverse tangent function, error function, smoothed step function) or rectifier functions. Transfer functions are mainly used for normalization purposes.

[0035] In particular, values ​​are propagated layer by layer through the neural network, where the value of the input layer 110 is given by the input of the neural network 100, where the value of the first hidden layer 111 can be calculated based on the value of the input layer 110 of the neural network, where the value of the second hidden layer 112 can be calculated based on the value of the first hidden layer 111, and so on.

[0036] To set the value of edge w (m,n) i,j , the neural network 100 must be trained using training data. In particular, the training data includes training input data and training output data (denoted as t i ). For the training step, the neural network 100 is applied to the training input data to generate the calculated output data. In particular, the training data and the calculated output data include a certain number of values, which is equal to the number of nodes of the output layer.

[0037] In particular, the comparison between the calculated output data and the training data is used to recursively adapt the weights within the neural network 100 (back propagation algorithm). In particular, the weights are changed according to the following formula

[0038]

[0039] where γ is the learning rate, and if the (n+1)th layer is not the output layer, then the number δ (n) j Can be based on δ (n+1) j Recursively computed as

[0040]

[0041] And if the (n+1)th layer is the output layer 113, then the number δ (n) j It can be recursively calculated as

[0042]

[0043] where f' is the first derivative of the activation function, and y (n+1) j is the comparative training value of the j-th node of the output layer 113.

[0044] In some embodiments, the neural network 100 is configured or trained to generate a histogram image with improved resolution. For example, in some embodiments, the neural network 100 is configured to receive a histogram image generated from TOF PET data acquired at a first response time resolution. The neural network 100 may receive the histogram image in any suitable form, such as, for example, list mode or sinogram PET data (in which case the neural network 100 includes one or more layers configured to generate a histogram image), a pre-generated data representation in image space (e.g., a pre-generated histogram image), etc. The neural network 100 is trained to generate a histogram image with improved resolution having an effective second response time resolution that is higher (e.g., faster) than the first response time resolution.

[0045] Figure 3 2 is a flow chart 200 illustrating a method of generating a reconstructed PET image using a resolution-increased histogram image according to some embodiments. Figure 4 is a block diagram illustrating a system 250 configured to implement Figure 3 The various parts of the method shown in . Figure 3 and Figure 4 Methods for generating reconstructed PET images using histogram images with increased resolution are discussed.

[0046] In step 202, nuclear imaging data is obtained from a scanning modality (such as the first modality 12), and specifically TOF PET data 252. The TOF PET data 252 is obtained within a predetermined time period and at a first response time resolution. The TOF PET data 252 is acquired as known in the art. For example, a TOF sinogram and / or TOF list mode data can be acquired and / or generated by a PET scanner (e.g., an imaging modality 12 and a computer 30) after a radioactive tracer is injected into a test volume (e.g., a patient, an animal, or an affected limb). The TOF PET data 252, such as a specific TOF sinogram and / or a specific portion of the TOF list mode data, corresponds to a corresponding axial position of the test volume at which a coincident annihilation event within the volume is detected. The TOF PET data 252 includes the position of a line of response (LOR) for each coincident event.

[0047] The TOF PET data 252 may be obtained in a list mode format and / or may be converted into a plurality of sinograms. The list mode data includes the coordinates of each coincident event detected during the image acquisition period. The sinogram includes an array of data for the angle versus displacement of each LOR at a specific axial position in the subject volume over a specific time period. Each TOF sinogram includes a row containing a specific azimuth angle, a specific position, and a specific position. Each of these rows corresponds to a one-dimensional parallel projection of the tracer compound distribution at a different coordinate.

[0048] At step 204, the TOF PET data 252 is provided to an event backprojection module 254, and at step 206, the event backprojection module 254 applies a backprojection algorithm to the TOF PET data 252 to generate a histogram image 256. The generated histogram image 256 includes a blur image (e.g., data representation) of an expected distribution along a particular TOF direction of the subject volume. The event backprojection module 254 can be configured to apply any suitable backprojection algorithm, for example, as determined by the format of the received TOF PET data 252 (e.g., sinogram or list mode data).

[0049] For example, when the TOF PET data 252 includes a sinogram, the event backprojection module 254 can apply the method of Joseph (PM Joseph, "An improved algorithm for reprojecting rays through pixel images," IEEE Transactions on Medical Imaging, vol. 1 no. 3, pp. 192-196, November 1982, which is incorporated herein by reference in its entirety) to generate a histogram image 256. As another example, when the TOF PET data 252 includes list mode data, the event backprojection module 254 can assign events within a predetermined time frame and / or range to image voxels along the LOR based on their timing information within a predetermined range. The histogram image generated by backprojecting the list mode data can be more accurate than the histogram image generated by backprojecting a sinogram generated from the list mode data because the list mode data can exhibit higher resolution timing data than the TOF sinogram data generated therefrom.

[0050] In some embodiments, the TOF PET data 252, whether in sinogram form or list mode form, may be corrected using attenuation correction and / or normalization factors prior to backprojection to obtain a modified histogram image 256. In this case, the modified histogram image 256 is provided as input to a trained neural network 258, as discussed below.

[0051] The histogram image 256 generated by the back-projection module is generated with a first response time resolution. The first response time resolution is determined by the parameters of the first modality 12 and the associated scanning parameters used to obtain the TOF PET data 252. In some embodiments, the first response time resolution is approximately 600ps, such as, for example, 550ps. A histogram image 256 having a resolution higher than about 350ps (e.g., 550ps, 600ps, etc.) is referred to herein as a "low resolution" image. In some embodiments, the first resolution is approximately 200ps, such as, for example, 215ps. A histogram image 256 having a resolution of about 100ps to about 350ps (e.g., 200ps, 215ps, etc.) is referred to herein as a "high resolution" image. Although specific ranges are given herein, it will be appreciated that the disclosed systems and methods can be used to improve the resolution of any histogram image from a first resolution to a higher second resolution, and are within the scope of the present disclosure.

[0052] At step 208, the histogram image 256 having the first response time resolution is provided to a trained neural network 258. As discussed in more detail below, the trained neural network 258 is generated by an iterative training process using a training data set that includes a first set of histogram images having the first response time resolution and a second set of associated histogram images having a higher (e.g., faster) second response time resolution. The trained neural network 258 is configured to receive the histogram image 256 having the first resolution and generate a histogram image 260 with improved resolution.

[0053] At step 210, the trained neural network 258 generates a resolution-improved histogram image 260 having an effective second response time resolution that is higher than otherwise possible based on the scan parameters and imaging modality used to obtain the TOF PET data 252. The effective second response time resolution of the resolution-improved histogram image 260 is higher than (i.e., faster than or better than) the first response time resolution of the histogram image 256 generated by the event backprojection module 254. For example, in embodiments that include a low-resolution generated histogram image 256 (e.g., having a first resolution of about 550 ps, ​​about 600 ps, ​​etc.), the resolution-improved histogram image 260 may have an effective high resolution (e.g., a resolution of about 200 ps, ​​215 ps, etc.). Similarly, in embodiments that include a high-resolution generated histogram image 256 (e.g., having a first resolution of about 200 ps, ​​about 215 ps, etc.), the resolution-improved histogram image 260 may have an "ultra-high" resolution (e.g., having a resolution of about 100 ps or better, such as, for example, about 10 ps, ​​about 1 ps, etc.). In some embodiments, the resolution-improved histogram image 260 has a higher resolution than would otherwise be possible based on the scan parameters and imaging modality used to obtain the TOF PET data 252 (e.g., providing an effective temporal response resolution of 10 ps, ​​which is not possible on current systems). Although specific embodiments and scopes are discussed herein, it will be appreciated that, in general, the trained neural network 258 may be any neural network configured to generate a resolution-improved histogram image having a higher effective response time resolution, such as compared to the response time resolution of an input histogram image.

[0054] At step 212, the resolution-improved histogram image 260 is provided to a PET image reconstruction module 262, which is configured to generate a reconstructed PET image 264 from the resolution-improved histogram image 260. The image reconstruction module 262 may be configured to implement any suitable image reconstruction process, such as conventional methods and / or methods utilizing a trained neural network. For example, in some embodiments, the PET image reconstruction module 262 may be configured to implement a conventional reconstruction process (e.g., algorithm), such as a filtered backprojection algorithm (e.g., 2D or 3D), a maximum likelihood expectation maximization algorithm (MLEM), an ordered subset expectation maximization algorithm (OSEM), a maximum likelihood estimation of activity and attenuation (MLAA) algorithm, any other iterative and / or analytical method, and / or any other suitable conventional reconstruction method known or later developed. As another example, in some embodiments, instead of and / or in addition to a "conventional" reconstruction method, the PET image reconstruction module 262 may be configured to implement a trained neural network for generating a reconstructed PET image, such as, for example, a novel deep learning or neural network based method. For example, as disclosed in: U.S. Patent No. 11,164,344, entitled “PET image reconstruction using TOF data and neural network,” issued November 2, 2021, and “FastPET: Near Real-Time Reconstruction of PET Histo-Image Data Using a Neural Network” by William Whitley et al. (IEEE Transactions on Radiation and Plasma Medical Sciences, vol. 5, no. 1, January 2021), each of which is incorporated herein by reference in its entirety, each discloses a reconstruction method based on deep learning or a neural network. As is known in the art, the reconstructed PET image can be used for clinical and / or other purposes. Due to the use of the resolution-improved histogram image 260, the resolution of the reconstructed PET image 264 will be higher than the resolution of the reconstructed PET image that can be obtained directly from the TOF PET data 252.

[0055] Figure 5 2 shows an embodiment of a system 250a configured to generate a histogram image 256a having a first effective response time resolution of 550 ps and a histogram image 260a having an improved resolution of a second effective response time resolution of 200 ps, ​​according to some embodiments. Figure 5As shown in , the second response time resolution of 200 ps of the resolution-improved histogram image 260a provides a significant increase in detail relative to the 550 ps response time resolution of the generated histogram image 256a. Figure 5 As shown in , system 250a allows a low-resolution imaging system, such as a conventional or low-cost PET imaging system, to provide scan details at resolutions previously available only on higher cost and / or new machines.

[0056] Figure 6 2 shows an embodiment of a system 250b configured to generate a histogram image 256b having a first effective response time resolution of 200 ps and a histogram image 260b with improved resolution having a second effective response time resolution of 10 ps, ​​according to some embodiments. Figure 6 As shown in , the 10 ps second response time resolution of the improved histogram image 260b provides a significant increase in detail relative to the original 200 ps response time resolution of the generated histogram image 256b. In some embodiments, the system 250b allows existing PET systems to provide effective response time resolutions that are not possible on current hardware, such as 10 ps, ​​1 ps second resolution, and / or any other ultra-high resolution.

[0057] Figure 7 is a flowchart 300 illustrating a method of training a neural network according to various embodiments. Figure 8 is a diagram showing a method according to some embodiments Figure 7 350 is a process flow of the various steps of a method of training a neural network shown in FIG. Figure 7 The method shown in may be used to train any neural network discussed herein, such as, for example, neural networks 258-258b, although it will be appreciated that the method of training a neural network may be configured to train any suitable neural network.

[0058] At step 302, a training data set 352 is acquired. The training data set 352 may include any suitable type of training data, such as, for example, a TOF PET data set 354 and / or a pre-generated histogram image 356. The TOF PET data set 354 may include a PET acquisition of list mode data and / or may include a PET sinogram generated from the TOF PET list mode data. The TOF PET data set 354 and / or the pre-generated histogram image 356 have a first response time resolution. The training data set 352 may include data acquired from a repository of PET data sets, such as a training data database or other non-transitory storage mechanism. The training data set 352 and / or each of the plurality of TOF PET data sets may depict any type of volume and may have been acquired using any number of PET scanners and PET acquisition settings. In some embodiments, the acquired PET data set is associated with the same volume (e.g., anatomical or metabolic region), PET scanner, and PET acquisition parameters that will be imaged / utilized using the trained neural network.

[0059] In optional step 304, each TOF PET data set 354 can be histogrammed into a plurality of sinograms, and in optional step 306, each TOF PET data set 354 and / or the plurality of sinograms can be back-projected into image space by an event back-projection module 358 to generate a histogram image 356 corresponding to the TOF PET data set 354. The TOF back-projection can be performed in step 306 using any suitable method, such as, for example, the Joseph method. In some embodiments, optional step 306 is omitted, and the training data set 352 includes pre-generated histogram images 356. If the training data set 352 includes pre-generated histogram images 356, the training data set 352 can also omit raw PET data, such as list mode PET data and / or TOF sinograms. Each generated histogram image 356 has a first response time resolution. In some embodiments, the response time resolution of each generated histogram image 356 is similar (e.g., each histogram image has a response time of about 200 ps, ​​about 215 ps, about 550 ps, ​​about 600 ps, ​​etc.). In some embodiments, the resolution of the generated histogram images 356 varies within a predetermined range (e.g., each histogram image has a response time resolution greater than about 350 ps, ​​in a range of about 100 ps to about 350 ps, ​​etc.).

[0060] At step 308, each histogram image 356 is input into an untrained and / or partially trained neural network 360, which is configured to generate a resolution-improved histogram image 362 having an effective second response time resolution. That is, the neural network 360 may generate a resolution-improved histogram image 362 based on each input histogram image 356. The resolution-improved histogram image 362 has an effective second response time resolution that is higher (e.g., faster) than the first response time resolution of the histogram image 356. In some embodiments, the target effective response time resolution of each resolution-improved histogram image 362 generated by the neural network 360 is substantially the same (e.g., having an effective resolution response time of approximately 200 ps, ​​approximately 10 ps, ​​etc.). In some embodiments, the resolution of the resolution-improved histogram image 362 is related to the resolution of the input histogram image 356 (e.g., an input histogram image 356 having a first response time resolution of approximately 600 ps will result in a resolution-improved histogram image having an effective second response time resolution of approximately 200 ps, ​​an input histogram image 356 having a first response time resolution of approximately 200 ps will result in a resolution-improved histogram image having an effective second response time resolution of approximately 10 ps, ​​etc.). The effective second response time resolution of the resolution-improved histogram image 362 is based on the response time resolution of the target image used to train the neural network 360, as discussed below.

[0061] In some embodiments, the training data set 352 includes a set of target images 364. The set of target images 364 includes at least one target histogram image associated with each of the TOF PET data set 354 and / or the generated histogram images 356. In some embodiments, the set of target images 364 includes target histogram images generated from the same subject volume as used to generate the corresponding TOF PET data set 354, but generated using a higher performance scanner (e.g., a scanner capable of producing a higher (e.g., faster) response time resolution). For example, in some embodiments, a first scan using a first PET scanner can be performed on the subject volume to generate a first TOF PET data set with a first response time resolution. Before, simultaneously with, and / or after the first scan, a second scan is performed on the subject volume using a second PET scanner to generate a second TOF PET data set with a higher second response time resolution. The generated histogram image 356 is generated from the first TOF PET data set, and at least one target histogram image 364 is generated from the second TOF PET data set. In some embodiments, the resolution of the target histogram image 364 generated from the second TOF PET dataset is equal to the desired resolution of the resolution-improved histogram image 362 to be generated by the neural network 360 .

[0062] In some embodiments, the TOF PET data set 354 and / or the set of target images 364 include simulated data generated by one or more simulation processes. For example, a subject volume histogram image (e.g., subject volume representative data in image space) can be simulated based on a Monte Carlo simulation performed at two or more TOF resolutions, such as a first response time resolution equal to the expected resolution of the TOF PET data set 252 to be received by the trained neural network 258, and a higher second response time resolution equal to the expected resolution of the resolution-improved histogram image 260 to be generated by the trained neural network 258, as discussed above. The Monte Carlo simulation can use any suitable simulation mechanism, such as, for example, Geant4, GATE (Geant4 application for tomographic emission), a graphics processing unit (GPU)-based Monte Carlo simulation, or any available Monte Carlo-based method. The TOF PET data simulated at the first resolution is used to generate the generated histogram image 356, and the TOF PET data simulated at the second resolution is used to generate the corresponding target image 364.

[0063] At step 310, the neural network 360 is trained (e.g., iteratively modified) based on the differences between the generated resolution-improved histogram images 362 and the target image 364. For example, in some embodiments, the comparison module 370 is configured to compare the associated target histogram image 364 with the output resolution-improved histogram image 362 to determine the difference between each resolution-improved histogram image 362 and the target image 364, and provide a modification value to the neural network 360 based on the difference. Although the embodiment is shown with a separate comparison module 370, it will be appreciated that many neural network models are configured to perform ground truth comparisons internally during the training process. The method iteratively repeats steps 308 and 310 until the neural network 360 produces an output within an acceptable range (e.g., generates a resolution-improved histogram image 362 within a certain percentage error relative to the target histogram image 364).

[0064] Figure 7-Figure 8 The iterative training process shown in can utilize any suitable neural network training that is or becomes known. For example, a method configured to implement Figure 7-Figure 8 The system of the iterative training process shown in can determine a loss based on a comparison between the k output resolution-improved histogram images 362 generated by the neural network 360 and the corresponding target histogram images in the target histogram images 364. The loss can include an L1 loss and an L2 loss, or any other suitable measure of the total loss. The L1 loss is the sum of the absolute differences between each output image and its corresponding ground truth PET image, and the L2 loss is the sum of the squared differences between each output image and its corresponding ground truth PET image. In some embodiments, a method similar to the above combined Figure 2 Describe the training process.

[0065] At step 310, the determined loss is back-propagated to the neural network 360, which changes its internal weights or kernel parameter values ​​based on the back-propagated loss. Steps 308-310 may be repeated until it is determined that the loss has reached an acceptable level or training is otherwise terminated. Upon termination, the neural network 360 may be considered trained, and at step 312, the trained neural network 372 is output for use in one or more methods, such as in conjunction with Figure 3-Figure 4 The methods described.

[0066] The neural networks described herein may be implemented using any neural network architecture that is or becomes known. For example, as discussed above, Figure 2 A neural network architecture 100 is shown that may be implemented in some embodiments. Fig. 9A neural network architecture 400 is shown in accordance with some embodiments. Architecture 400 is similar to the U-NET architecture, but is fully convolutional. Specifically, the pooling of the U-Net architecture has been replaced by strided convolutions, and the "up-convolution" has been replaced by the PixelShuffle upsampling method. Embodiments are not limited to the specific elements of architecture 400.

[0067] According to some embodiments, the training is performed at steps 308 and 310 using an Adam optimizer to minimize the target image x i * and output image x i =F r (y i ) and a balanced combination of the mean absolute error (MAE) and multi-scale structural similarity difference (SSIMΔ) between . The corresponding loss function is shown below.

[0068]

[0069] Loss = (1-α)MAE + αSSIMΔ

[0070] The above loss function dynamically balances the minimization of the absolute error between each corresponding pixel (i.e., MAE) and a perceptual loss function (SSIM) that evaluates the mean μ, variance σ2, and covariance σ between images. A moving average of n samples of each loss type is used to calculate a balanced loss scalar α at each training step.

[0071] Fig.10 A computer system 30 configured to implement one or more processes according to some embodiments is shown. System 30 is a representative device and may include a processor subsystem 72, an input / output subsystem 74, a memory subsystem 76, a communication interface 78, and a system bus 80. In some embodiments, one or more of the system 30 components may be combined or omitted, such as, for example, not including the input / output subsystem 74. In some embodiments, system 30 may include other components not shown in FIG. 12. For example, system 30 may also include, for example, a power subsystem. In other embodiments, system 30 may include several instances of the components shown in FIG. 12. For example, system 30 may include multiple memory subsystems 76. For simplicity and clarity, and not limitation, one of each component is shown in FIG. 12.

[0072] The processor subsystem 72 may include any processing circuitry operable to control the operation and performance of the system 30. In various aspects, the processor subsystem 72 may be implemented as a general purpose processor, a chip multiprocessor (CMP), a special purpose processor, an embedded processor, a digital signal processor (DSP), a network processor, an input / output (I / O) processor, a media access control (MAC) processor, a radio baseband processor, a coprocessor, a microprocessor (such as a complex instruction set computer (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, and / or a very long instruction word (VLIW) microprocessor), or other processing device. The processor subsystem 72 may also be implemented by a controller, a microcontroller, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a programmable logic device (PLD), etc.

[0073] In various aspects, the processor subsystem 72 may be arranged to run an operating system (OS) and various applications. Examples of OS include, for example, operating systems commonly known by the trade names of Apple OS, Microsoft Windows OS, Android OS, Linux OS, and any other proprietary or open source OS. Examples of applications include, for example, network applications, local applications, data input / output applications, user interaction applications, etc.

[0074] In some embodiments, the system 30 may include a system bus 80 that couples various system components, including the processing subsystem 72, the input / output subsystem 74, and the memory subsystem 76. The system bus 80 may be any of several types of bus structures, including a memory bus or memory controller, a peripheral bus or external bus, and / or a local bus using any of a variety of available bus architectures, including but not limited to a 9-bit bus, Industry Standard Architecture (ISA), Micro Channel Architecture (MSA), Extended ISA (EISA), Intelligent Drive Electronics (IDE), VESA Local Bus (VLB), Peripheral Component Interconnect Card International Association bus (PCMCIA), Small Computer Interface (SCSI), or other proprietary bus, or any custom bus suitable for computing device applications.

[0075] In some embodiments, the input / output subsystem 74 may include any suitable mechanisms or components to enable a user to provide input to the system 30, and to enable the system 30 to provide output to the user. For example, the input / output subsystem 74 may include any suitable input mechanism, including but not limited to buttons, keypads, keyboards, click wheels, touch screens, motion sensors, microphones, cameras, etc.

[0076] In some embodiments, the input / output subsystem 74 may include a visual peripheral output device for providing a display visible to a user. For example, the visual peripheral output device may include a screen, such as, for example, a liquid crystal display (LCD) screen. As another example, the visual peripheral output device may include a removable display or projection system for providing content display on a surface away from the system 30. In some embodiments, the visual peripheral output device may include an encoder / decoder (also referred to as a codec) to convert digital media data into an analog signal. For example, the visual peripheral output device may include a video codec, an audio codec, or a codec of any other suitable type.

[0077] The visual peripheral output device may include a display driver, circuitry for driving the display driver, or both. The visual peripheral output device may be operable to display content under the direction of the processor subsystem 72. For example, the visual peripheral output device may be capable of playing media playback information, application screens of applications implemented on the system 30, information about ongoing communication operations, information about incoming communication requests, or device operation screens, to name just a few examples.

[0078] In some embodiments, the communication interface 78 may include any suitable hardware, software, or combination of hardware and software that can couple the system 30 to one or more networks and / or additional devices. The communication interface 78 may be arranged to operate using any suitable technology to control information signals using a set of desired communication protocols, services, or operating procedures. The communication interface 78 may include an appropriate physical connector to connect to a corresponding communication medium, whether wired or wireless.

[0079] The communication means include a network. In various aspects, the network may include a local area network (LAN) and a wide area network (WAN), including but not limited to the Internet, a wired channel, a wireless channel, a communication device including a telephone, a computer, a wire, a radio, an optical channel or other electromagnetic channel, and combinations thereof (including other devices and / or components capable of transmitting / associated with transmitting data). For example, the communication environment includes in-vivo communication, various devices, and various communication modes (such as wireless communication, wired communication, and combinations thereof).

[0080] Wireless communication modes include any communication mode between points (e.g., nodes) that utilize, at least in part, wireless technology, including various protocols and protocol combinations associated with wireless transmission, data, and devices. Points include, for example, wireless devices (such as wireless headsets), audio and multimedia devices and equipment (such as audio players and multimedia players), phones (including mobile phones and cordless phones), and computers and computer-related devices and components (such as printers, network-connected machines, and / or any other suitable devices or third-party devices).

[0081] The wired communication mode includes any communication mode between points that utilize wired technology, including various protocols and protocol combinations associated with wired transmission, data, and devices. Points include, for example: devices, such as audio and multimedia devices and apparatuses, such as audio players and multimedia players; phones, including mobile phones and cordless phones; and computers and computer-related devices and components, such as printers, network-connected machines, and / or any other suitable devices or third-party devices. In various implementations, the wired communication module can communicate according to a variety of wired protocols. Examples of wired protocols may include universal serial bus (USB) communications, RS-232, RS-422, RS-423, RS-485 serial protocols, FireWire, Ethernet, Fiber Channel, MIDI, ATA, Serial ATA, PCI Express, T-1 (and variants), Industry Standard Architecture (ISA) parallel communications, Small Computer System Interface (SCSI) communications, or Peripheral Component Interconnect (PCI) communications, just to name a few examples.

[0082] Thus, in various aspects, the communication interface 78 may include one or more interfaces, such as, for example, a wireless communication interface, a wired communication interface, a network interface, a transmission interface, a reception interface, a media interface, a system interface, a component interface, a switch interface, a chip interface, a controller, etc. For example, when implemented by a wireless device or within a wireless system, the communication interface 78 may include a wireless interface including one or more antennas, transmitters, receivers, transceivers, amplifiers, filters, control logic, etc.

[0083] In various aspects, the communication interface 78 can provide data communication functions according to a variety of protocols. Examples of protocols may include various wireless local area network (WLAN) protocols, including the Institute of Electrical and Electronics Engineers (IEEE) 802.xx series of protocols, such as IEEE 802.11a / b / g / n / ac, IEEE 802.16, IEEE 802.20, etc. Other examples of wireless protocols may include various wireless wide area network (WWAN) protocols, such as GSM cellular radiotelephone system protocols with GPRS, CDMA cellular radiotelephone communication systems with 1xRTT, EDGE systems, EV-DO systems, EV-DV systems, HSDPA systems, etc. Additional examples of wireless protocols may include wireless personal area network (PAN) protocols, such as infrared protocols, protocols from the Bluetooth Special Interest Group (SIG) series of protocols (e.g., Bluetooth specification versions 5.0, 6, 7, traditional Bluetooth protocols, etc.), and one or more Bluetooth profiles, etc. Another example of a wireless protocol may include near field communication technology and protocols, such as electromagnetic induction (EMI) technology. An example of EMI technology may include passive or active radio frequency identification (RFID) protocols and devices. Other suitable protocols may include ultra-wideband (UWB), digital office (DO), digital home, trusted platform module (TPM), ZigBee, etc.

[0084] In some embodiments, at least one non-transitory computer-readable storage medium is provided, on which computer-executable instructions are embodied, wherein the computer-executable instructions, when executed by at least one processor, cause the at least one processor to perform an embodiment of the method described herein. The computer-readable storage medium may be embodied in the memory subsystem 76.

[0085] In some embodiments, the memory subsystem 76 may include any machine-readable or computer-readable medium capable of storing data, including both volatile / non-volatile memory and removable / non-removable memory. The memory subsystem 76 may include at least one non-volatile memory unit. The non-volatile memory unit is capable of storing one or more software programs. The software program may include, for example, an application, user data, device data, and / or configuration data, or a combination thereof, to name a few examples. The software program may include instructions that can be executed by various components of the system 30.

[0086] In various aspects, the memory subsystem 76 may include any machine-readable or computer-readable medium capable of storing data, including both volatile / non-volatile memory and removable / non-removable memory. For example, the memory may include read-only memory (ROM), random access memory (RAM), dynamic RAM (DRAM), double data rate DRAM (DDR-RAM), synchronous DRAM (SDRAM), static RAM (SRAM), programmable ROM (PROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory (e.g., NOR or NAND flash memory), content addressable memory (CAM), polymer memory (e.g., ferroelectric polymer memory), phase change memory (e.g., bidirectional memory), ferroelectric memory, silicon-oxide-nitride-oxide-silicon (SONOS) memory, disk storage (e.g., floppy disk, hard drive, optical disk, magnetic disk) or card (e.g., magnetic card, optical card), or any other type of medium suitable for storing information.

[0087] In one embodiment, the memory subsystem 76 may include an instruction set in the form of a file for performing various methods, such as a method including generating a histogram image with improved resolution using a trained neural network, as described herein. The instruction set may be stored in any acceptable machine-readable instruction form, including source code or various appropriate programming languages. Some examples of programming languages ​​that may be used to store the instruction set include, but are not limited to, Java, C, C++, C#, Python, Objective-C, Visual Basic, or .NET programming. In some embodiments, a compiler or interpreter is included to convert the instruction set into machine executable code for execution by the processing subsystem 72.

[0088] Each functional component described herein can be implemented in computer hardware, in program code, and / or in one or more computing systems that execute such program code, as is known in the art. Figure 1 As discussed in connection with FIG. 12 , such a computing system may include one or more processing units that execute processor-executable program code stored in a memory system. Similarly, each disclosed method and other processes described herein may be performed using any suitable combination of hardware and software. The software program code embodying these processes may be stored by any non-transitory tangible medium, such as the above referenced Figure 1 and as discussed in Figure 12.

[0089] While the subject matter has been described in terms of exemplary embodiments, it is not limited thereto. Rather, the appended claims should be interpreted broadly to encompass other variants and embodiments that may be made by those skilled in the art.

Claims

1. A system comprising a positron emission tomography (PET) imaging modality configured to perform a first scan to acquire a first PET dataset; and A processor configured to: back-projecting the first PET dataset to generate a first histogram image representing the first PET dataset and having a first resolution; Inputting the first histogram image into the trained neural network; receiving a second histogram image from the trained neural network, wherein the second histogram image has a second resolution, wherein the second resolution is better than the first resolution, and wherein the second histogram image represents an improved version of the first PET dataset; and The second histogram image is input to a reconstruction process configured to generate a reconstructed PET image. 2 . The system of claim 1 , wherein the first resolution is approximately 600 ps and the second resolution is approximately 200 ps.

3. The system of claim 1, wherein the first resolution is approximately 200 ps and the second resolution is approximately 10 ps.

4. The system of claim 1, wherein the trained neural network is a trained convolutional neural network. 5 . The system of claim 1 , wherein the first PET dataset is selected from the group consisting of a list mode dataset, a sinogram dataset, and a histogram image dataset.

6. The system of claim 1 , wherein the trained neural network is generated by a training data set, the training data set comprising a plurality of first histogram images having a first resolution and a plurality of second histogram images having a second resolution, wherein each histogram image in the plurality of first histogram images has at least one corresponding histogram image in the second plurality of histogram images, and wherein the second resolution of each histogram image in the second plurality of histogram images is higher than the first resolution of the corresponding histogram image in the first plurality of histogram images.

7. The system of claim 1, wherein the first plurality of histogram images correspond to a first set of PET data acquired at a first imaging resolution, and the second histogram image corresponds to a second set of PET data acquired at a second imaging resolution.

8. The system of claim 1, wherein the first plurality of histogram images comprises a first plurality of simulated histogram images generated at a first simulated resolution, and the second plurality of histogram images comprises a second plurality of simulated histogram images generated at a second simulated resolution.

9. The system of claim 8, wherein the first plurality of simulated histogram images and the second plurality of histogram images are generated by Monte Carlo based simulations.

10. A method of generating a reconstructed positron emission tomography (PET) image, comprising: performing a first scan to acquire a first PET dataset; back-projecting the first PET data set to generate a first histogram image representing the first PET data and having a first resolution; Inputting the first histogram image into the trained neural network; receiving a second histogram image from the trained neural network, wherein the second histogram image has a second resolution, wherein the second resolution is higher than the first resolution, and wherein the second histogram image represents an improved version of the first PET dataset; and The second histogram image is input to a reconstruction process configured to generate a reconstructed PET image. The method of claim 10 , wherein the first resolution is approximately 600 ps and the second resolution is approximately 200 ps.

12. The method of claim 10, wherein the first resolution is approximately 200 ps and the second resolution is approximately 10 ps.

13. The method of claim 10, wherein the trained neural network is a trained convolutional neural network. The method of claim 10 , wherein the first PET dataset is a list mode dataset.

15. The method of claim 10, wherein the trained neural network is generated by a training data set comprising a first plurality of histogram images having a first resolution and a second plurality of histogram images having a second resolution, wherein each histogram image in the first plurality of histogram images has at least one corresponding histogram image in the second plurality of histogram images, and wherein the second resolution of each histogram image in the second plurality of histogram images is higher than the first resolution of the corresponding histogram image in the first plurality of histogram images.

16. The method of claim 10, wherein the first plurality of histogram images correspond to a first set of PET data acquired at a first imaging resolution, and the second histogram image corresponds to a second set of PET data acquired at a second imaging resolution.

17. The method of claim 10, wherein the first plurality of histogram images comprises a first plurality of simulated histogram images generated at a first simulated resolution, and the second plurality of histogram images comprises a second plurality of simulated histogram images generated at a second simulated resolution.

18. The method of claim 17, wherein the first plurality of simulated histogram images and the second plurality of histogram images are generated by Monte Carlo based simulation.

19. A method of training a neural network to generate a histogram image with improved resolution, comprising: receiving a training data set including a first plurality of histogram images having a first resolution and a second plurality of histogram images having a second resolution, wherein each histogram image in the first plurality of histogram images has at least one corresponding histogram image in the second plurality of histogram images, and wherein the second resolution of each histogram image in the second plurality of histogram images is higher than the first resolution of the corresponding histogram image in the first plurality of histogram images; inputting each histogram image of the first plurality of histogram images to a neural network configured to generate a corresponding resolution-improved histogram image having an estimated second resolution; comparing each resolution-improved histogram image to a corresponding histogram image in the second plurality of histogram images to determine any differences between each resolution-improved histogram image and a histogram image in the second plurality of histogram images that corresponds to a histogram image in the first plurality of histogram images; as well as The neural network is modified based on the determined difference between each resolution improved histogram image and a histogram image in the second plurality of histogram images corresponding to a histogram image in the first plurality of histogram images provided to the neural network.

20. The method of claim 19, wherein the first plurality of histogram images and the second plurality of histogram images are simulated histogram images.

Images