Method for estimating missing dynamic pet image based on physical information neural network

Abstract

This invention discloses a method for estimating missing dynamic PET images based on a physical information neural network. It proposes a dynamic PET data estimation framework combining a spatiotemporal coding-decoding network, a kinetic parameter prediction network, and a hidden neural representation network. By introducing a dual-compartment model as a physical constraint, unsupervised learning is achieved without the need for complete labeled data or supervision by real kinetic parameters. This allows the neural network to simultaneously satisfy data consistency and kinetic consistency in estimating missing PET images. This invention solves the problem of conventional deep learning-based methods failing to guarantee physiological interpretability when reconstructing missing frames in dynamic PET images, achieving unsupervised, high-precision, and interpretable estimation of missing frames. This method is applicable to dynamic PET imaging scenarios with low doses, shortened scan times, or data loss due to motion.

Description

Technical Field

[0001] This invention belongs to the field of medical imaging and deep learning technology, and in particular relates to a method for estimating missing dynamic PET images based on physical information neural networks. Background Technology

[0002] Positron emission tomography (PET) is a molecular imaging technique that reflects changes in metabolism, receptor binding, and blood flow within the body. In dynamic PET imaging, the scan typically lasts for tens of minutes, acquiring multiple time frames to obtain a dynamic PET image sequence, thereby constructing a time-activity curve (TAC) of total radioactivity in the target tissue. These continuous time-frame dynamic PET images provide crucial information for clinically interpreting the anatomical location of lesions and the characteristics of radioactive distribution, while the TAC curve quantitatively characterizes the dynamic metabolic process of tracers within tissues, thus enabling the determination of the perfusion rate constant. intra-tissue exchange rate constant Pharmacokinetic parameters, etc. These two elements complement each other, jointly providing crucial imaging information and quantitative evidence for tumor classification, treatment response assessment, and the study of the pathological mechanisms of neurological diseases. However, the clinical application of dynamic PET technology is still limited by the following: (1) Long scanning time and unavoidable patient movement: Dynamic PET usually requires continuous acquisition for tens of minutes. During this period, the movement of the patient's whole body or local organs will cause the loss, blurring or misalignment of time frame information, thus affecting the subsequent dynamic estimation.

[0003] (2) Low dose and short acquisition time lead to insufficient signal-to-noise ratio: In order to reduce radiation exposure or shorten the scanning time, clinicians often reduce the dose of radioactive tracer or reduce the temporal resolution of each frame, which makes some time frames severely noisy or even impossible to reconstruct.

[0004] (3) Deep learning reconstruction methods lack physiological interpretability: Existing dynamic PET reconstruction methods based on deep neural networks are mostly data-driven and do not explicitly model the pharmacokinetic laws of PET. For example, U-Net or Transformer is used to fill in missing frames, but the reconstruction results may be close to the real image in appearance, but the TAC curve does not meet the physiological kinetic equations such as the Two-Tissue Compartment Model (2TCM), thus leading to distortion of medical analysis.

[0005] (4) Estimation of dynamic parameters depends on complete time frames: Traditional dynamic modeling methods (such as Logan Plot, Patlak analysis, 2TCM fitting, etc.) usually require complete dynamic PET data. Missing frames will lead to the indistinguishability of dynamic parameters or unstable fitting.

[0006] Given the above problems, the existing technology lacks a reconstruction method that can both fill in missing dynamic PET images and ensure dynamic consistency and improve physiological interpretability. Summary of the Invention

[0007] To address the problems existing in the prior art, the present invention aims to provide a missing dynamic PET image estimation method based on physical information neural networks. Based on physical information neural networks (PINNs), the method achieves high-precision completion and consistent reconstruction of missing frames by explicitly integrating the PET dynamic equations into a deep learning model.

[0008] According to a first aspect of the embodiments of this application, a method for estimating missing dynamic PET images based on a physical information neural network is provided, comprising: Incomplete dynamic PET image sequences and corresponding arterial input functions are obtained, wherein the arterial input functions are used to characterize the change in tracer concentration in plasma over time; Based on the dynamic PET image sequence, a trained physical information neural network is used to generate a total tissue radioactivity concentration curve over a continuous time range and to estimate missing dynamic PET images. In the physical information neural network, a spatiotemporal coding and decoding network extracts latent feature representations from the dynamic PET image sequence, and a kinetic parameter prediction network performs voxel-by-voxel kinetic estimation on the latent feature representations to obtain a kinetic parameter map corresponding to each voxel. The physical constraint module estimates the total tissue radioactivity concentration over a continuous time range based on the kinetic parameter map, thereby generating a total tissue radioactivity time concentration curve and missing dynamic PET images.

[0009] Furthermore, the spatiotemporal coding-decoding network adopts a multi-scale coding-decoding structure; Each layer of the encoder path performs spatiotemporal convolution decoupling operation through serial spatial convolution kernels and temporal convolution kernels to extract feature maps. The feature maps are then input into a three-dimensional max pooling layer via a nonlinear activation function for spatiotemporal joint downsampling. Each layer of the decoder path is upsampled through a series of transposed spatial convolution kernels and transposed temporal convolution kernels. The upsampled feature map is then connected to the feature map output by the encoder at the corresponding scale level via a skip connection. The fused feature map is then input into a series of spatial and temporal convolution kernels via a nonlinear activation function for spatiotemporal convolution decoupling.

[0010] Furthermore, the upsampled feature map is skip-connected to the feature map output by the encoder at the corresponding scale level, specifically as follows: For the i-th layer decoder, the input feature map of the i-th layer encoder and the feature map obtained by spatiotemporal convolution are directly concatenated, and then concatenated again with the upsampled feature map of the corresponding scale in the decoder in the channel dimension. The fused feature map is then processed by spatiotemporal decoupling convolution and used as the input of the nonlinear activation function.

[0011] Furthermore, the kinetic parameter prediction network employs a multi-head parameter prediction structure, where each decoupled voxel-by-voxel parameter prediction head is used to predict the perfusion rate constant. intra-organism exchange rate constant Dephosphorylation rate constant and blood volume fraction At each voxel position Obtain the corresponding dynamic parameter vector at the location. The dynamic parameter vectors at all voxel locations are combined in space to form a voxel-by-voxel dynamic parameter map. .

[0012] Furthermore, in each single-parameter prediction head: latent feature representation The data passes through two convolutional layers. The output of the second convolutional layer is then fed into a random deactivation layer with a predetermined dropout rate. Finally, a third convolutional layer is used to obtain the location of each voxel. The corresponding dynamic parameters at that location; The dynamic parameters output by each single-parameter prediction head are tensor-stitched along the channel dimension to obtain the voxel-by-voxel dynamic parameter map.

[0013] Furthermore, the physical constraint module includes an implicit neural representation network and a dual-compartment model; The implicit neural representation network employs a multilayer perceptron structure with continuous time variables. With the aforementioned dynamic parameter diagram As input, tissue compartment concentration is generated. and ; The dual-compartment model is based on the free compartment concentration at time t. Combination chamber concentration and blood volume fraction Calculate the radioactivity concentration value of the tissue : , ; in, This indicates the total duration of the dynamic PET scan. For arterial input function, blood volume fraction The range of values ​​is ; The radioactive concentration value that changes over time This constructs a total radioactivity time-concentration curve, thereby generating missing dynamic PET images.

[0014] Furthermore, the physical information neural network also includes a data consistency module, which establishes a mapping relationship between dynamic parameters and observable PET signals, and works in conjunction with the physical constraint module to improve the accuracy of completing missing dynamic PET images; Based on this, the loss function during the training of the physical information neural network is: , , , , , in These are, respectively, data consistency loss, physical residual loss, total variation regularization loss of the kinetic parameter plot, and self-consistent consistency loss. , and These are the weighting coefficients for the corresponding loss terms; This represents the main loss term based on the total radioactivity time-concentration curve. This represents the secondary loss term based on time-frame PET images. These are the weighting coefficients used to balance the effects of the two losses; Represents physical residuals. This represents the set of time sampling points used to calculate the physical residual; A diagram showing the kinetic parameters of each voxel; This is a PET image output by the data consistency module. It is the total radioactivity concentration. This represents a mapping operation that reshapes the concentration curves of each voxel into the spatial dimensions of the image. This represents the set of actual available dynamic PET observation time frames.

[0015] According to a second aspect of the embodiments of this application, a computer program product is provided, including a computer program / instructions that, when executed by a processor, implement the method described in the first aspect.

[0016] According to a third aspect of the embodiments of this application, an electronic device is provided, comprising: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors perform the method as described in the first aspect.

[0017] According to a fourth aspect of the embodiments of this application, a computer-readable storage medium is provided that stores computer instructions thereon, which, when executed by a processor, implement the steps of the method as described in the first aspect.

[0018] The technical solutions provided by the embodiments of this application may include the following beneficial effects: As can be seen from the above embodiments, this application constructs a dual-path integrated network framework that combines spatiotemporal decoupling feature extraction, independent estimation of voxel-by-voxel dynamic parameters, continuous-time implicit neural representation, and strong constraints of physical equations of dual-tissue compartment models. This framework can effectively solve the problems of missing dynamic PET data, severe noise, and lack of physiological consistency in traditional reconstruction methods, and achieve high-quality, physically consistent, and clinically interpretable unsupervised reconstruction of dynamic PET.

[0019] Unlike existing supervised learning methods that rely on complete dynamic PET sequences or annotations of real kinetic parameters, this invention combines a kinetic parameter prediction network with an implicit neural representation network. By introducing a dual-tissue compartment model as a physical prior constraint, and using only a subset of dynamic PET observation frames, it achieves unsupervised estimation of missing time frames. This allows the neural network to simultaneously satisfy data consistency and kinetic consistency in estimating PET images with unobserved frames. This method avoids dependence on high-quality, fully annotated data and improves the algorithm's applicability and generalization ability under clinical conditions of low-dose, short-duration scans or motion-induced data gaps.

[0020] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0021] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0022] Figure 1 This is a schematic diagram illustrating a process for estimating missing dynamic PET images based on a physical information neural network according to an exemplary embodiment.

[0023] Figure 2 This is a schematic diagram of the framework of a physical information neural network according to an exemplary embodiment.

[0024] Figure 3 This is a comparison diagram showing a real reference image and the prediction result of the method of the present invention, according to an exemplary embodiment.

[0025] Figure 4 This is a block diagram illustrating a missing dynamic PET image estimation device based on a physical information neural network according to an exemplary embodiment.

[0026] Figure 5 This is a schematic diagram of an electronic device according to an exemplary embodiment. Detailed Implementation

[0027] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application.

[0028] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0029] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0030] like Figure 1 As shown, this invention is a method for estimating missing dynamic PET images based on physical information neural networks, comprising the following steps: S1: Obtain incomplete dynamic PET image sequences and corresponding arterial input functions. The incomplete dynamic PET image sequence has some missing time frames in the time dimension, and the arterial input function is used to characterize the change of tracer concentration in plasma over time; In specific implementation, the incomplete dynamic PET image sequence may be in the following two situations: (1) In actual clinical imaging, dynamic PET sequences often cannot be correctly reconstructed due to patient movement, or, under rapid or low-dose scanning conditions, time frames are missing or quality is degraded due to insufficient signal-to-noise ratio of some frames; (2) The complete dynamic PET scan time is too long, which increases the patient's radiation exposure risk and is also very likely to cause patient discomfort. Therefore, there is an urgent need to shorten the scanning protocol in clinical practice. For example, the total time of a complete dynamic PET scan is 60 minutes. This method can select the first 25 minutes of dynamic PET images obtained in the early stage of the scan as the model input, or select the first 15 minutes of dynamic PET images obtained in the early stage of the scan and the last 10 minutes of dynamic PET images obtained in the late stage of the scan as the model input for subsequent spatial-temporal feature encoding and dynamic parameter estimation. The dynamic PET images of the remaining time periods are used as the missing time frames to be estimated. The model input is expressed as: in, This represents the input sequence of dynamic PET images. Indicates the first A single-frame PET image actually measured within a scanning time interval. Corresponding to the The frame acquisition time interval, , This refers to the actual time points in dynamic PET scans where valid data is obtained. This represents the total number of time frames observed.

[0031] S2: Based on the dynamic PET image sequence, a total tissue radioactivity concentration curve within a continuous time range is generated using a trained physical information neural network, and missing dynamic PET images are estimated. In the physical information neural network, a spatiotemporal coding and decoding network extracts the latent feature representation of the dynamic PET image sequence, and a kinetic parameter prediction network performs voxel-by-voxel kinetic estimation on the latent feature representation to obtain a kinetic parameter map corresponding to each voxel. The physical constraint module estimates the total tissue radioactivity concentration within a continuous time range based on the kinetic parameter map, thereby generating a total tissue radioactivity time concentration curve and missing dynamic PET images. (1) Spatiotemporal coding and decoding network (ST-UNet) The dynamic PET image sequence first enters the ST-UNet module. This module adopts a multi-scale encoder-decoder structure (i.e., a U-shaped network architecture), extracting high-level semantic features through layer-by-layer dimensionality reduction and then layer-by-layer dimensionality upscaling to restore spatial resolution, thereby achieving joint encoding of spatial structural information and temporal dynamic information of the dynamic PET image. Each layer of the encoder path performs spatiotemporal convolution decoupling operations through serial spatial and temporal convolution kernels to extract feature maps. These feature maps are then input into a three-dimensional max-pooling layer via a nonlinear activation function for spatiotemporal joint downsampling. Each layer of the decoder path performs upsampling through serial transposed spatial and transposed temporal convolution kernels. The upsampled feature map is then connected to the feature map output by the corresponding scale level encoder via a skip connection. The fused feature map is then input into serial spatial and temporal convolution kernels via a nonlinear activation function for spatiotemporal convolution decoupling operations.

[0032] In one embodiment, the specific network structure and inter-layer connection details of the ST-UNet module are as follows: (i) Spatiotemporally Decoupled Convolution Strategy: In each feature extraction stage of the encoder and decoder, the network abandons the traditional three-dimensional convolution and instead adopts a serial spatiotemporally decoupled convolution kernel. The input features first pass through a 1×3×3 spatial convolution layer with a stride of 1 and spatial dimension padding of (0, 1, 1) to extract local spatial anatomical information; its output features are then immediately input into a 3×1×1 temporal convolution layer with a stride of 1 and temporal dimension padding of (1, 0, 0) to capture the temporal dynamics features between adjacent frames. After each spatiotemporal convolution operation, the feature map is given a non-linear expression through the ELU (Exponential Linear Unit) activation function. This decoupling design explicitly separates the learning process of spatial and temporal features, reducing the complexity of network parameters while enhancing the ability to extract time-series features.

[0033] (ii) Multi-scale downsampling and feature compression: In the encoder path, after each layer undergoes the above spatiotemporal decoupling convolution operation, the network uses a 3D max-pooling layer (MaxPool3d) with a pooling kernel size of 2×2×2 to perform spatiotemporal joint downsampling of the feature map. The downsampling operation halves the spatial and temporal resolution of the feature map while multiplying the number of channels, thereby extracting deeper global spatiotemporal semantic information.

[0034] (iii) Multi-scale upsampling and feature reconstruction: In the decoder path, the network uses a three-dimensional transposed convolutional layer (ConvTranspose3d) for upsampling. Corresponding to the encoder, the upsampling operation also adopts a decoupling strategy, successively using transposed convolutions in the spatial dimension and transposed convolutions in the temporal dimension to gradually restore the spatiotemporal resolution of the feature map.

[0035] (iv) Skip Connections: To prevent the loss of fine-grained spatial and temporal details during multi-layer downsampling, ST-UNet introduces hierarchical skip connections. For the i-th layer decoder, the input feature map of the i-th layer encoder and the feature map obtained by spatiotemporal decoupling convolution are directly concatenated to form a short skip connection feature containing the original feature and spatiotemporal enhancement feature. This feature serves as a long skip connection feature, which is concatenated with the upsampled feature map of the corresponding scale in the decoder along the channel dimension. The concatenated fused feature is then input into the spatiotemporal decoupling convolution module of the current decoding layer for processing.

[0036] The ST-UNet module, based on the aforementioned multi-scale encoder-decoder structure, performs forward propagation on the input dynamic PET image sequence, ultimately yielding a latent feature representation: in, This represents the space-time coding mapping function of the ST-UNet network. This represents the latent feature representation obtained after the spatial-temporal encoding, used to characterize the spatial structure and temporal dynamics information of dynamic PET images.

[0037] (2) Dynamic parameter prediction network Based on the latent feature representation The system employs a multi-head parameter prediction structure for independent estimation of voxel-by-voxel dynamic parameters. To prevent gradient interference between different dynamic parameters and to avoid introducing additional spatial neighborhood aliasing (maintaining pure voxel-by-voxel mapping), this module consists of five parallel single-parameter prediction heads, each used for independent parameter prediction. and blood volume fraction In each single-parameter prediction head: input features The system sequentially passes through two convolutional layers (each configured with the ELU activation function). After the output of the second convolutional layer, a random deactivation layer with a dropout rate of 10% is introduced to suppress overfitting. Finally, a third convolutional layer is used to compress the number of feature channels to 1, resulting in a prediction result for a single physical parameter.

[0038] At each voxel location, the five prediction heads output corresponding parameter scalars. These five scalar results are then tensor concatenated along the channel dimension to directly construct a five-channel, voxel-by-voxel dynamics parameter map in spatial dimension. : in, This represents the forward mapping function of the single-parameter prediction head. Due to the use of voxel-level decoupled convolution processing, this module performs independent mapping processing on only a single voxel feature, effectively avoiding the technical flaw of constructing false features using spatial neighborhood information to cater to model training, thereby significantly enhancing the physical interpretability of the estimated dynamic parameters.

[0039] (3) Physical constraint module The physical constraint module includes an implicit neural representation network (INR) and a two-tissue compartment model (2TCM).

[0040] The INR network employs a multilayer perceptron (MLP) architecture, containing a 4-layer MLP with 128 neurons per layer, and uses the hyperbolic tangent (Tanh) activation function. This network uses continuous-time variables... With the voxel-by-voxel kinetic parameter diagram As input, the dynamic evolution of the tracer in different compartments within the tissue can be characterized as a continuous function, generating the tissue compartment concentration in a two-compartment model. and : The 2TCM (Two-Channel Transmission Model) is a linear pharmacokinetic model describing the interaction of radiotracers between two kinetic compartments (i.e., the free / unmetabolized compartment and the binding / metabolized compartment) in arterial plasma and outside the blood vessel. This model takes the tracer concentration in arterial plasma as input and uses mathematical equations to characterize the dynamic trans-compartment transport process of the tracer. Its rate parameters reflect both the tissue penetration characteristics of the tracer across the vessel wall and can quantitatively assess the binding or metabolic rate of the tracer within the tissue, thereby enabling precise analysis of the tracer's microscopic dynamic distribution. Its calculation logic is: the radioactive concentration value in the tissue at time t... The concentration of the free compartment (i.e., the unmetabolized compartment) Concentration of binding compartment (i.e., metabolic compartment) and blood volume fraction Joint decision: in, The mapping function representing the implicit neural representation network. This indicates the total duration of the dynamic PET scan. For arterial input function, blood volume fraction The range of values ​​is .

[0041] The radioactivity concentration value The total radioactivity time concentration curve (TAC) varies over time and is particularly useful for completing missing time frames in dynamic PET imaging.

[0042] (4) Data Consistency Module The data consistency module, namely the PET synthesizer, receives the kinetic parameter vector. PET frames are synthesized on a voxel-by-voxel basis. To ensure the physical interpretability of the parameter-to-image mapping and to prevent the network from using spatial neighborhood information to forge data or excessively smooth noise, the PET synthesizer abandons conventional convolutional kernels with spatial receptive fields (such as 3×3 convolutions) in its structural design. Instead, it uses pure 1×1 two-dimensional convolutions (1×1Conv2d) to achieve independent voxel-by-voxel mapping, establishing a mapping relationship between dynamic parameters and observable PET signals. This mapping relationship provides a gradient shortcut for training the physical information neural network while improving the model's robustness to noise interference.

[0043] After the above modules are completed, the model can output simultaneously: (a) Total radioactivity time concentration curve (TAC) Based on the tissue compartment concentration function output by the implicit neural representation network, and combined with the mapping relationship of kinetic parameters, a total tissue radioactivity concentration curve over a continuous time range can be generated, which is expressed as follows: in, Indicates the organization in time The radioactivity concentration value at that location, This indicates the total duration of a dynamic PET scan.

[0044] (b) PET image frames at missing time points It should be noted that in actual dynamic PET imaging, TAC curves are extracted from discrete-time frame images. This invention, however, employs a reverse reconstruction method. First, a continuous TAC curve is modeled using a hidden neural representation network, and then PET images are generated by inverting the continuous TAC curve. This achieves continuous reconstruction of missing data while ensuring physical constraints. For time intervals during dynamic PET scanning where no valid image data is obtained... The trained model is used to reconstruct the TAC curves of each voxel. Within this time interval, continuous TAC curves are integrated and divided by the interval time interval to simulate the time-cumulative averaging imaging physical process of actual PET scans, yielding the average radioactivity concentration value for the corresponding scan frame. Finally, the calculation results of all voxels are arranged according to their original spatial positions to generate the PET image prediction results corresponding to the missing time frames, which are expressed as follows: in, Indicates the missing time interval The generated PET image prediction frame.

[0045] The above model not only completes the frames but also the entire TAC curve, and the results have clear kinetic interpretability and stronger robustness to noise and missing frames. Therefore, this invention can significantly improve the clinical usability of low-dose or missing frame dynamic PET.

[0046] The model training process will be explained in detail below.

[0047] (i) Data consistency loss The data consistency loss includes a main loss term based on the total radioactivity time-concentration curve (TAC) and a secondary loss term based on time-frame PET images. The main loss term measures the overall difference between the predicted and actual time-concentration curves, while the secondary loss term constrains the local residuals between the time-frame synthesized PET images and the corresponding measurement frames. Its overall form can be expressed as: in, This represents the master data consistency loss constructed based on the total radioactivity time-concentration curve, used to measure the difference between the true TAC and the network-predicted TAC; This represents the secondary data consistency loss constructed based on time-frame PET images, used to measure the residual between the synthesized PET image and the corresponding measurement image. These are the weighting coefficients used to balance the effects of the two losses.

[0048] In practical implementation, the main loss term It can be represented as: Secondary loss items Based on the construction of time-frame-by-time PET image residuals, its form is as follows: in, This represents the actual set of available dynamic PET observation time frames. This represents the total radioactivity concentration over time curve obtained from actual dynamic PET data. This represents the total radioactivity time-concentration curve obtained from network prediction. Indicates the first Actual measured PET images obtained within a scanning time interval, Indicates the first Predicted PET images generated by the PET synthesizer within a scanning time interval. This represents the L2 norm.

[0049] The PET synthesizer provides a secondary loss term in the data consistency constraint for network training and updates the dynamic parameter prediction network and implicit neural representation network based on the loss through the backpropagation algorithm to progressively approximate and correct the generated dynamic parameter map.

[0050] During network training, an iterative error correction mechanism based on residuals is introduced. The voxel-by-voxel residuals are applied to the dynamic parameter prediction network and the implicit neural representation network through backpropagation to progressively correct the dynamic parameter map. The parameter update form is expressed as follows: in, Indicates the first Voxel-by-voxel dynamics parameter diagram at the next iteration. Indicates the first In the nth iteration, the first The voxel-by-voxel residuals calculated between the predicted PET image and the actual measured PET image within a scanning time interval. This represents the learning rate.

[0051] Through multiple iterative optimizations, the predicted PET image generated by the PET synthesizer gradually approximates the actual measured PET image.

[0052] (ii) Self-Consistency Loss The self-consistency loss is used to constrain the consistency between the PET image prediction results generated by the PET synthesizer and the total radioactivity temporal concentration curve generated by the implicit neural representation network at corresponding spatial locations and time points. Specifically, this invention comprises two parallel prediction branches: one is a physical prediction path driven by dynamic parameters and computed via a hidden neural representation network (INR) and physical equations; the other is an image synthesis path driven by latent features and generated via a PET synthesizer. A self-consistency loss forces the outputs of these two branches to remain consistent, requiring that the synthesized image must be interpretable by the current dynamic parameters. Its expression is: in, This is a PET image output from a PET synthesizer. The total radioactivity concentration is calculated using a combination of implicit neural network representations and kinetic equations. This represents a mapping operation that reshapes the concentration curves of each voxel into the image spatial dimension. By minimizing this loss, artifacts caused by the PET synthesizer escaping physical constraints can be prevented, thereby significantly improving the accuracy of kinetic parameter estimation.

[0053] (iii) Total variation regularization loss based on kinetic parameter plot To improve the stability and continuity of the kinetic parameter map in the spatial dimension, this embodiment introduces a regularization term based on Total Variation (TV) into the loss function. This guides the parameter estimation results towards a spatially smooth solution from the optimization objective level, thereby improving the stability and physiological rationality of the kinetic parameter map. For the predicted kinetic parameter map... Its total variation regularization term is defined as the weighted sum of parameter differences between adjacent voxels, used to constrain drastic changes in parameters in spatial directions, and its expression can be expressed as: in, Represents the regularization term based on the total variation, kinetic parameter plot. The dynamic parameter vectors at each voxel position Combining and forming in the spatial dimension, This indicates the position of a voxel in the parametric graph.

[0054] (iv) Physical residual loss The physical residual term is constructed based on the dynamic equation of the dual-compartment model and combined with the fifth-order Runge-Kutta (RK5) numerical integration method. The physical residual loss is calculated by comparing the prediction results of the implicit neural representation network with the RK5 integration results.

[0055] Specifically, the two-tissue compartment model (2TCM) is introduced as a physical constraint into the network training process, and its dynamic equation is expressed as: Based on the above kinetic equations, this invention uses the RK5 method to integrate the equations to measure whether the change in tissue compartment concentration output by the implicit neural representation network over time satisfies the aforementioned kinetic relationship. The physical residual is defined as: in, and By RK5 method Moment and As initial values, substitute them into the 2TCM dynamic equation, and then use step sizes... from Points to , get in Numerical solution at time; and This indicates that the implicit neural representation network model is in The predicted value given at any given time.

[0056] During network training, by analyzing the physical residuals The physical residual loss term is constructed by measuring statistics at time sampling points. Its general form is defined as: in, This represents the set of time sampling points used to calculate the physical residuals. By minimizing the loss of these physical residuals, the total radioactivity time concentration curve output by the network is made to satisfy the physiological dynamics evolution described by the two-compartment model.

[0057] Considering the above constraints, the total loss function for network training in this embodiment is composed of data consistency loss, self-consistency consistency loss, physical residual loss, and parameter smoothing regularization loss, and its overall form can be expressed as: in, , and These are the weighting coefficients for the corresponding loss terms, used to balance the impact of different constraints during the training process.

[0058] Without relying on real dynamic parameters and complete total radioactivity time-concentration curves, the network is trained end-to-end unsupervised using the gradient descent method.

[0059] The trained model is used to generate dynamic PET observable signals at any given time point, enabling the estimation and completion of missing time frames.

[0060] In one embodiment, this method is implemented on a machine equipped with an Intel Ultra 9 285K and an NVIDIA RTX 5090 GPU (32 GB VRAM). To verify the effectiveness of the invention, this embodiment constructs a model incorporating anatomical complexity and realistic physiological movements. 18 F-FDG dynamic PET simulation dataset. Data source defined in 128 128 A static Zubal brain phantom with 40 grids was used, interspersed with two simulated tumors of different sizes. Image reconstruction was performed using the Maximum Likelihood Expectation Maximization (MLEM) algorithm (4 iterations, 8 subsets) combined with attenuation correction. The dynamic parameters of all tissues and tumors were set according to clinical standard ranges. The total simulation scan duration was 60 minutes, generating approximately 60 million coincidence events. To simulate a realistic dynamic acquisition process, 30% uniform random noise was introduced in addition to simulated scattering. The dynamic sequence was divided into 27 time frames in the time dimension, specifically using the following partitioning scheme: 2 6s, 4 12s, 4 30s, 4 60s, 4 120s, 9 300s.

[0061] The original reconstructed image is large, 128 pixels. 128. To focus on the brain and tumor regions and reduce computational redundancy, this embodiment crops the image center to 80%. 80 voxels in size. According to the missing frame completion settings of the present invention, the first 25 minutes (i.e., the first 20 time frames) of the dynamic sequence are used as the input to the model. This model is designed to simulate short-duration scans in early clinical settings or scenarios where subsequent data becomes unavailable due to motion. The remaining seven timeframes (corresponding to 25 to 60 minutes of data acquisition) are treated as unknown missing frames, and their ground truth values ​​are used only during validation and testing to evaluate completion accuracy. Model training employs an end-to-end unsupervised learning strategy. The first 20 input frames are used to extract spatiotemporal features through ST-UNet, and a kinetic parameter prediction network generates a kinetic parameter map. The method utilizes implicit neural representation combined with RK5 numerical integration to generate continuous-time TAC curves. Regarding the physical dynamics evolution, the RK5 solver step size used in this embodiment is set to [value missing]. Minutes, random sampling in each iteration The gradients at each time point are approximated as continuous over time. During training, the computational power of the NVIDIA RTX 5090 is utilized to simultaneously minimize data consistency loss, self-consistency consistency loss, physical residual loss, and TV regularization loss until the model converges. Regarding hyperparameter settings, to compensate for the relatively small numerical magnitude of the single-step RK5 integral residuals, the weight of the physical residual loss is set to [value missing]. The weights for other losses are configured as follows: Sub-data consistency loss weight. Parameter smoothing regularization weights and loss of self-consistency .like Figure 1 The diagram shown is a schematic representation of the missing dynamic PET image estimation method based on physical information neural networks according to the present invention. Figure 2 This is a block diagram of a network framework model device for missing dynamic PET image estimation based on a physical information neural network, according to an exemplary embodiment.

[0062] Figure 3 The comparison results of missing frames (frames 23-27) at the end of a dynamic PET sequence are shown. (a) is the ideal noise-free ground truth PET image, i.e., the original phantom data without superimposed simulated noise, serving as the gold standard for evaluating the algorithm's recovery capability. (b) is the prediction result generated by the method of this invention under the condition of inputting a noisy reconstructed image. The comparison shows that although the model input data is affected by low-dose noise, the predicted image generated by this invention not only completes the missing time frames but also effectively suppresses noise, maintaining visual quality highly consistent with the noise-free ground truth while restoring the anatomical structure, thus verifying the robustness of the algorithm under low signal-to-noise ratio conditions. This method is applicable to dynamic PET imaging scenarios with low doses, shortened scan times, or data loss due to motion, and can achieve unsupervised estimation of missing time frames.

[0063] Corresponding to the aforementioned embodiments of the missing dynamic PET image estimation method based on physical information neural network, this application also provides embodiments of the missing dynamic PET image estimation device based on physical information neural network.

[0064] Figure 4 This is a block diagram illustrating a missing dynamic PET image estimation device based on a physical information neural network according to an exemplary embodiment. (Refer to...) Figure 4 The device may include: The acquisition module 21 is used to acquire incomplete dynamic PET image sequences and corresponding arterial input functions, wherein the arterial input functions are used to characterize the change of tracer concentration in plasma over time. The estimation module 22 is used to generate a total tissue radioactivity concentration curve over a continuous time range based on the dynamic PET image sequence using a trained physical information neural network, and to estimate missing dynamic PET images. In the physical information neural network, a spatiotemporal coding and decoding network extracts the latent feature representation of the dynamic PET image sequence, and a kinetic parameter prediction network performs voxel-by-voxel kinetic estimation on the latent feature representation to obtain a kinetic parameter map corresponding to each voxel. The physical constraint module estimates the total tissue radioactivity concentration over a continuous time range based on the kinetic parameter map, thereby generating a total tissue radioactivity time concentration curve and missing dynamic PET images.

[0065] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0066] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this application according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0067] Accordingly, this application also provides a computer program product, including a computer program / instruction that, when executed by a processor, implements the missing dynamic PET image estimation method based on physical information neural network as described above.

[0068] Accordingly, this application also provides an electronic device, comprising: one or more processors; a memory for storing one or more programs; and, when the one or more programs are executed by the one or more processors, causing the one or more processors to implement the above-described method for estimating missing dynamic PET images based on a physical information neural network. Figure 5 The diagram shown is a hardware structure diagram of any device with data processing capabilities, including a missing dynamic PET image estimation device based on a physical information neural network provided in an embodiment of the present invention. (Except for...) Figure 5 In addition to the processor, memory, and network interface shown, any data processing device in the embodiment may also include other hardware depending on the actual function of the data processing device, which will not be described in detail here.

[0069] Accordingly, this application also provides a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the aforementioned method for estimating missing dynamic PET images based on a physical information neural network. The computer-readable storage medium can be an internal storage unit of any data-processing device as described in any of the foregoing embodiments, such as a hard disk or memory. The computer-readable storage medium can also be an external storage device, such as a plug-in hard disk, smart media card (SMC), SD card, flash card, etc., equipped on the device. Furthermore, the computer-readable storage medium can include both internal storage units of any data-processing device and external storage devices. The computer-readable storage medium is used to store the computer program and other programs and data required by the data-processing device, and can also be used to temporarily store data that has been output or will be output.

[0070] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.

Claims

1. A method for estimating missing dynamic PET images based on physical information neural networks, characterized in that, include: Incomplete dynamic PET image sequences and corresponding arterial input functions are obtained, wherein the arterial input functions are used to characterize the change in tracer concentration in plasma over time; Based on the dynamic PET image sequence, a trained physical information neural network is used to generate a total tissue radioactivity concentration curve over a continuous time range and to estimate missing dynamic PET images. In the physical information neural network, a spatiotemporal coding and decoding network extracts latent feature representations from the dynamic PET image sequence, and a kinetic parameter prediction network performs voxel-by-voxel kinetic estimation on the latent feature representations to obtain a kinetic parameter map corresponding to each voxel. The physical constraint module estimates the total tissue radioactivity concentration over a continuous time range based on the kinetic parameter map, thereby generating a total tissue radioactivity time concentration curve and missing dynamic PET images.

2. The method according to claim 1, characterized in that, The spatiotemporal coding-decoding network adopts a multi-scale coding-decoding structure; Each layer of the encoder path performs spatiotemporal convolution decoupling operation through serial spatial convolution kernels and temporal convolution kernels to extract feature maps. The feature maps are then input into a three-dimensional max pooling layer via a nonlinear activation function for spatiotemporal joint downsampling. Each layer of the decoder path is upsampled through a series of transposed spatial convolution kernels and transposed temporal convolution kernels. The upsampled feature map is then connected to the feature map output by the encoder at the corresponding scale level via a skip connection. The fused feature map is then input into a series of spatial and temporal convolution kernels via a nonlinear activation function for spatiotemporal convolution decoupling.

3. The method according to claim 2, characterized in that, The upsampled feature map is then skipped and connected to the feature map output by the encoder at the corresponding scale level, specifically as follows: For the i-th layer decoder, the input feature map of the i-th layer encoder and the feature map obtained by spatiotemporal convolution are directly concatenated, and then concatenated again with the upsampled feature map of the corresponding scale in the decoder in the channel dimension. The fused feature map is then processed by spatiotemporal decoupling convolution and used as the input of the nonlinear activation function.

4. The method according to claim 1, characterized in that, The kinetic parameter prediction network employs a multi-head parameter prediction structure, where each decoupled voxel-by-voxel parameter prediction head is used to predict the perfusion rate constant. intra-organism exchange rate constant Dephosphorylation rate constant and blood volume fraction At each voxel position Obtain the corresponding dynamic parameter vector at the location. The dynamic parameter vectors at all voxel locations are combined in space to form a voxel-by-voxel dynamic parameter map. .

5. The method according to claim 4, characterized in that, In each single-parameter prediction head: latent feature representation The data passes through two convolutional layers. The output of the second convolutional layer is then fed into a random deactivation layer with a predetermined dropout rate. Finally, a third convolutional layer is used to obtain the location of each voxel. The corresponding dynamic parameters at that location; The dynamic parameters output by each single-parameter prediction head are tensor-stitched along the channel dimension to obtain the voxel-by-voxel dynamic parameter map.

6. The method according to claim 1, characterized in that, The physical constraint module includes an implicit neural representation network and a dual-compartment model. The implicit neural representation network employs a multilayer perceptron structure with continuous time variables. With the aforementioned dynamic parameter diagram As input, tissue compartment concentration is generated. and ; The dual-compartment model is based on the free compartment concentration at time t. Combination chamber concentration and blood volume fraction Calculate the radioactivity concentration value of the tissue : ， ； in, This indicates the total duration of the dynamic PET scan. For arterial input function, blood volume fraction The range of values ​​is ; The radioactive concentration value that changes over time This constructs a total radioactivity time-concentration curve, thereby generating missing dynamic PET images.

7. The method according to claim 1, characterized in that, The physical information neural network also includes a data consistency module, which establishes a mapping relationship between dynamic parameters and observable PET signals, and works in conjunction with the physical constraint module to improve the accuracy of filling missing dynamic PET images; Based on this, the loss function during the training of the physical information neural network is: ， ， ， ， ， in These are, respectively, data consistency loss, physical residual loss, total variation regularization loss of the kinetic parameter plot, and self-consistent consistency loss. , and These are the weighting coefficients for the corresponding loss terms; This represents the main loss term based on the total radioactivity time-concentration curve. This represents the secondary loss term based on time-frame PET images. These are the weighting coefficients used to balance the effects of the two losses; Represents physical residuals. This represents the set of time sampling points used to calculate the physical residual; A diagram showing the kinetic parameters of each voxel; This is a PET image output by the data consistency module. It is the total radioactivity concentration. This represents a mapping operation that reshapes the concentration curves of each voxel into the spatial dimensions of the image. This represents the set of actual available dynamic PET observation time frames.

8. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instruction is executed by the processor, it implements the method as described in any one of claims 1-7.

9. An electronic device, characterized in that, include: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-7.

10. A computer-readable storage medium storing computer instructions thereon, characterized in that, When executed by the processor, this instruction implements the steps of the method as described in any one of claims 1-7.

Images