Low dose sparse view SPECT reconstruction and image denoising method based on implicit neural representation

Abstract

The application discloses a low-dose sparse view SPECT reconstruction and image denoising method based on implicit neural representation. The method includes two stages: low-dose sparse view SPECT reconstruction and image denoising. Firstly, by mapping the input points to wavelet features and performing frequency regularization, the multilayer perceptron (MLP) can learn high-frequency functions in a low-dimensional problem domain. In prediction, dense projection views not containing training views are generated, and then the trained sparse projection views and the predicted generated dense projection views are combined to realize the SPECT reconstruction of the low-dose sparse view through the PAPA reconstruction algorithm. Secondly, in the image denoising stage, the architecture induction bias of the implicit neural representation (INR) is utilized: the INR tends to fit the low-frequency clean image signal faster than the high-frequency noise, and the INR layer closer to the output plays a more critical role in fitting the high-frequency part. By penalizing the growth of weights in deeper layers, the inherent denoising capability of the INR is maximized, so that the SPECT image denoising is realized.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of biomedical image reconstruction, and particularly relates to a low-dose sparse view SPECT reconstruction and image denoising method based on implicit neural representation. BACKGROUND

[0002] Single-Photon Emission Computed Tomography (SPECT) is a commonly used nuclear medicine imaging technique for detecting and diagnosing various diseases. SPECT obtains information about tissue and organ function by measuring gamma rays emitted from the patient's body. However, due to the noise (image may have Poisson noise), scattering and limited angle sampling problems existing in the SPECT imaging process, it has always been a challenging task to reconstruct an accurate three-dimensional image.

[0003] Traditional SPECT reconstruction methods can be divided into three categories: analytical methods, iterative methods and learning-based methods. Analytical methods estimate the distribution of radionuclides by solving the Radon transform and its inverse problem. Among them, the classic FDK algorithm performs well in ideal cases, but the reconstruction effect is poor in the case of incomplete data such as sparse views. Iterative methods are another commonly used reconstruction method, which model the reconstruction problem as a minimization process. These methods use an optimization framework combined with a regularization module for reconstruction. Although iterative methods perform well in handling incomplete data problems, they require long computation time and large memory. With the rise of artificial intelligence, learning-based methods have become increasingly popular in SPECT reconstruction. These methods use deep neural networks to implement projection data prediction and extrapolation, and estimate the distribution of radionuclides by data regression similar to making the reconstruction process differentiable. Most of these methods require large-scale data sets for network training and rely on neural networks to remember the features of SPECT images. SUMMARY

[0004] To solve the problems in the background art, the application provides a low-dose sparse view SPECT reconstruction and image denoising method based on implicit neural representation. The method first generates dense views from low-dose and high-noise sparse views through implicit neural representation, combines the trained sparse views and generated dense views, and obtains the corresponding SPECT reconstruction image through the Preconditioned Alternating Projection Algorithms reconstruction algorithm. Then, by penalizing the growth of deeper weights and maximizing the inherent denoising ability of implicit neural representation, the noisy SPECT image is denoised to achieve high-quality SPECT image reconstruction.

[0005] The low-dose sparse view SPECT reconstruction and image denoising method based on implicit neural representation of the application comprises the following steps:

[0006] S1: mapping the three-dimensional coordinates of the low-dose, high-noise sparse view SPECT to high-frequency features through wavelet series, and performing frequency regularization;

[0007] S2: inputting the sparse view after wavelet series mapping and frequency regularization into a fully connected multilayer perceptron (MLP) to learn the high-frequency function in a low-dimensional problem domain;

[0008] S3: training the network by minimizing the error between the real projection and the synthesized projection, and generating dense views that do not contain training views during prediction;

[0009] S4: combining the sparse view during training and the generated dense view to realize SPECT reconstruction of the low-dose high-noise sparse view through the Preconditioned Alternating Projection Algorithms reconstruction algorithm;

[0010] S5: using a denoising algorithm to maximize the inherent denoising capability of the implicit neural representation (INR) by penalizing the growth of deeper weights, and realizing the denoising of high-quality SPECT images.

[0011] Further, the three-dimensional coordinates of the sparse view SPECT in step S1 refer to that assuming K sparse view SPECT projections, then the two-dimensional coordinates x'=(u,v) in each projection are converted to three-dimensional coordinates x=(x,y,z) in the world coordinate system through a rotation matrix R∈R 4×4 , so as to convert to three-dimensional coordinates x=(x,y,z) in the world coordinate system. The three-dimensional space coordinates x=(x,y,z)∈R 3 are normalized, and then a haar wavelet series with different frequencies is used to map the normalized three-dimensional coordinates to a higher-dimensional space. In some examples, the wavelet series mapping can be expressed as follows:

[0012]

[0013] where L represents a hyperparameter that controls the maximum encoding frequency. When L is equal to 6, k has 32 cases, which are 0, 1...31. The input point is input into the wavelet function,

[0014] where the haar wavelet function is expressed as:

[0015]

[0016] x'=[x,γ L(x)] concatenation, the 3D input points are mapped to 192D high-frequency features, which are applied to the input of the MLP;

[0017] At a given position encoding x' = [x, γ L (x)] of the position encoding, the visible spectrum is adjusted using a linearly increasing frequency mask α based on the training time step, as follows:

[0018] γ L (t, T; x) = γ L (x) ⊙ α(t, T, L, i)

[0019] where

[0020] where α i (t, T, L, i) is the i-th value of α(t, T, L, i); t and T are the current training iteration and the final iteration of frequency regularization, respectively. Specifically, starting from the original input without position encoding, the visible frequency is linearly increased by 3 bits each time as the training proceeds. Frequency regularization bypasses the unstable and vulnerable high-frequency signals at the beginning of training, gradually provides high-frequency information of SPECT projection views, and avoids over-smoothing.

[0021] Further, the fully connected multi-layer perceptron network in the step S2 comprises 8 fully connected layers, wherein the first layer and the fourth layer of the fully connected multi-layer perceptron network have a number of neurons different from those of other layers of the fully connected multi-layer perceptron network, and each of the other layers of the fully connected multi-layer perceptron network has 256 neurons, and a periodic sin activation function is arranged after each fully connected layer except the last layer, and the periodic sin activation function is very suitable for representing complex natural signals and their derivatives.

[0022] Further, the loss function between the real projection I r and the synthetic projection I s in the step S3 is:

[0023]

[0024] where the projection I can be represented as I = I0+ δ i , and δ 2 is the distance between adjacent points, where B is a batch of rays, and the fully connected multi-layer perceptron network parameter θ is updated during the training process.

[0025] Further, the SPECT image corresponding thereto is obtained by the Preconditioned Alternating Projection Algorithms reconstruction algorithm in the step S4.

[0026] Further, the denoising algorithm in step S5, wherein the first stage of fully connected multilayer perceptron weight parameters are used as pre-training weights for the current network, training is stopped when the mean squared error (MSE) between the image generated by the fully connected multilayer perceptron and the target noise image is less than the estimated noise level of the target image. Wherein the mean squared error is represented as:

[0027]

[0028] Where C is the set of all pixel coordinate pairs c = (x, y) ∈ R 2 Assuming 128x128 acquisition image pixels, for example, the coordinate pair settings can be a grid

[0029]

[0030] Where each pair represents the center coordinates of a pixel.

[0031] When The noise power estimate of the image function g satisfies

[0032]

[0033] Training is stopped.

[0034] The stop criterion is designed to consider the ideal case where the implicit neural representation bias provides perfect temporal separation of image and noise fitting (i.e., fitting the underlying clean signal before fitting the noise). In practice, if the fully connected multilayer perceptron generated image is exactly the same as the clean image, the stop criterion will satisfy equality. In addition, to reinforce the implicit neural representation bias, weight decay is selectively applied on the fully connected perceptron of the last two layers. Observations found that the later layers of implicit neural representation contribute more to fitting the noise signal than the previous layer. More specifically, the front layer neurons mainly represent low frequency patterns, while the back layer neurons produce fine-grained details. One of the significant differences between the fully trained and early stopped models is the behavior of the latter layer neurons. In particular, the implicit neural representation (INR) trained to convergence differs mainly in its latter layer neuron outputs compared to the implicit neural representation (INR) at the optimal early stopping period; many of the outputs of the latter layer lose semantic information and converge to noise. On the other hand, the outputs of the early layers hardly change

[0035] Advantages of the present invention:

[0036] The application provides a low-dose sparse view SPECT reconstruction and image denoising method based on implicit neural representation, which has the following beneficial effects: first, by using implicit neural representation and wavelet series mapping, three-dimensional reconstruction of sparse views can be realized under low-dose and high-noise conditions. The method can effectively extract and learn high-frequency features, and realize optimization of the reconstruction process through frequency regularization. Second, by using the pre-trained multilayer perceptron weight parameters, combined with the Preconditioned Alternating Projection Algorithms reconstruction algorithm, the SPECT reconstruction of low-dose high-noise sparse views is realized, thereby providing high-quality reconstruction results. Finally, in terms of image denoising, the inherent denoising capability of implicit neural representation is utilized, and the previously trained weight parameters are used to realize effective denoising of noisy SPECT images, further improving the quality of the reconstructed images. Therefore, the beneficial effects of the application lie in realizing high-quality results of low-dose sparse view SPECT reconstruction and image denoising, which can further improve the quality of SPECT reconstruction images compared with traditional deep learning methods. BRIEF DESCRIPTION OF DRAWINGS

[0037] Figure 1 is a schematic diagram of the overall model framework

[0038] Figure 2 is a flowchart of the low-dose sparse view SPECT reconstruction and image denoising method based on implicit neural representation

[0039] Figure 3 is the architecture of the fully connected multilayer perceptron in the first stage

[0040] Figure 4 is the architecture of the fully connected multilayer perceptron in the second stage denoising DETAILED DESCRIPTION

[0041] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict, and the present application will be further described in detail below in combination with the drawings and specific embodiments.

[0042] The low-dose sparse view SPECT reconstruction and image denoising method based on implicit neural representation of the present application is implemented as follows:

[0043] The built implicit neural representation structure is shown in Figure 1 , and the schematic diagram of the overall model framework is shown in Figure 2 .

[0044] The experimental data used herein was obtained by Monte Carlo simulation software Simind, and the size of each low-dose view was 128x128. There were a total of 120 projection views in a 360-degree scan, of which the training set was 1 view every 6 degrees, that is, 20 views were selected as the training set, and then all the remaining 100 projection views were rendered. Then the two-dimensional coordinates x'=(u,v) in each training set projection view were converted to three-dimensional coordinates x=(x,y,z) in the world coordinate system through a rotation matrix R∈R 4×4 , so as to convert to three-dimensional coordinates x=(x,y,z) in the world coordinate system. The three-dimensional space coordinates x=(x,y,z)∈R 3 need to be normalized. Then a haar wavelet series with different frequencies was used to map the normalized three-dimensional coordinates to a higher-dimensional space. In some examples, the wavelet series mapping can be expressed as follows:

[0045]

[0046] where L represents a hyperparameter that controls the maximum encoding frequency. When L is equal to 6, k has 32 cases, which are 0, 1...31. The input point is input into the wavelet function,

[0047] where the haar wavelet function is expressed as:

[0048]

[0049] x'=[x,γ L (x)] is spliced, the 3-dimensional input point is mapped to a 192-dimensional high-frequency feature, which is applied to the input of the MLP.

[0050] x'=[x,γ L (x)] is spliced, the 3-dimensional input point is mapped to a 192-dimensional high-frequency feature, which is applied to the input of the MLP.

[0051] Given the position encoding of x'=[x,γ L (x)], based on the training time step, a linearly increasing frequency mask a is used to adjust the visible spectrum as follows:

[0052] γ' L (t,T;x) = γ L (x)⊙α(t,T,L,)

[0053] where

[0054] where α i(t,T,L,) represents the i-th bit value of α(t,T,L,); t and T are the current and final training iterations of frequency regularization, respectively. Specifically, starting from the raw input without positional encoding, the visible frequency is linearly increased by 3 bits each time training progresses. Frequency regularization bypasses the unstable and easily affected high-frequency signals at the beginning of training, gradually providing high-frequency information from the SPECT projection view and avoiding over-smoothing.

[0055] The architecture of the first-stage fully connected multilayer perceptron is shown in the attached figure. Figure 3 As shown, the fully connected multilayer perceptron network comprises eight fully connected layers. Layers 1 and 4 of this fully connected multilayer perceptron have a different number of neurons than the other layers, and each of the other layers has 256 neurons. Except for the last layer, each fully connected layer is followed by a periodic sin activation function. Due to spectral bias, standard ReLU MLPs cannot adequately represent the details of these complex low-dimensional signals. By replacing ReLU activation with a sinusoidal sin function, coordinate-based MLPs can learn from fitting many signals within a category, thus enabling rapid optimization to fit any new signal with fewer steps and fewer observations. Therefore, this periodic sin activation function is well-suited for representing complex natural signals and their derivatives.

[0056] Appendix Figure 1 True projection I in r and composite projection I s The loss function between them is:

[0057]

[0058] Among them, projection I can be used Let I0 represent the initial intensity, and δ represent the initial intensity. i Let B be the distance between adjacent points. Here, B represents a set of rays, and the parameters θ of the fully connected multilayer perceptron network are updated during training. Then, the corresponding SPECT image is obtained using the Preconditioned Alternating Projection Algorithms reconstruction algorithm.

[0059] Appendix Figure 4 This illustrates the architecture of the fully connected multilayer perceptron during the second-stage denoising process. The weight parameters from the first-stage fully connected multilayer perceptron are used as pre-training weights for the current network. Training stops when the mean squared error (MSE) between the image generated by the fully connected multilayer perceptron and the target noisy image is less than the estimated noise level of the target image. The mean squared error is expressed as:

[0060]

[0061] where C is the set of all pixel coordinate pairs c = (x, y) e R 2 Assume that the 128 x 128 acquisition image pixels are used, for example, the coordinate pairs setting can be a grid

[0062]

[0063] where each pair represents the center coordinate of a pixel.

[0064] When the noise power estimate of the image function g satisfies

[0065]

[0066] the training is stopped.

[0067] The design of this stopping criterion takes into account the ideal case where the implicit neural representation bias provides a perfect temporal separation of the image and noise fitting (i.e., fitting the underlying clean signal before fitting the noise). In practice, if the image generated by the fully connected multilayer perceptron is exactly the same as the clean image, the stopping criterion will satisfy equality. In addition, to reinforce the implicit neural representation (INR) bias, weight decay is selectively applied on the fully connected perceptrons of the last two layers. Observations find that the later layers of the implicit neural representation contribute more to fitting the noise signal than the previous layer. More specifically, the front layer neurons mainly represent low-frequency patterns, while the back layer neurons produce fine-grained details. One significant difference between the fully trained and early-stopped models is the behavior of the latter layer neurons. In particular, the implicit neural representation (INR) trained to convergence differs mainly in its latter layer neuron outputs compared to the implicit neural representation (INR) at the best early-stopping epoch; many of the outputs of the latter layer lose semantic information and converge to noise. On the other hand, the outputs of the early layers change little.

[0068] The entire algorithm of the embodiment is tested in an Ubuntu 16.04.1 (64-bit) system, where the cpu is Intel Xeon Silver 4210R (2.4GHz), and the graphics card model is GeForce RTX 3090 (8GB video memory). In programming, the Pytorch 1.8.0 platform is used to build the network model.

[0069] The foregoing description of the embodiments has been presented for the purpose of illustration and description. It is not intended to be exhaustive or to limit the application to the precise form disclosed. Modifications and variations are possible in light of the above teachings or can be acquired from practice of the application. As well, the application has been described above with the assistance of illustrative figures and detailed descriptions. It is clear that various modifications can be made by one skilled in the art without departing from the spirit of the application. Accordingly, such modifications should be considered as within the scope of the application.

Claims

1. A method for low-dose sparse view SPECT reconstruction and image denoising based on implicit neural representation, characterized in that, Includes the following steps: S1: The three-dimensional coordinates of the low-dose, high-noise, sparse view in the world coordinate system are mapped to high-frequency features through wavelet series and then frequency regularized. The specific process is as follows: S21: Transform three-dimensional spatial coordinates = Normalization is performed, and then Haar wavelet series with different frequencies are used to map the normalized 3D coordinates to a higher-dimensional space. In some instances, the wavelet series mapping can be expressed as follows: Where L represents the hyperparameter controlling the maximum coding frequency, and when L equals 6, There are 32 possible cases, 0, 1...31, which are input points into the wavelet function. The Haar wavelet function is expressed as follows: Will This maps 3D input points to 192-dimensional high-frequency features, which are then applied to the input of the MLP. S22: In the given The positional encoding is based on the training time step and uses a linearly increasing frequency mask. To adjust the visible spectrum, as shown below: in in for The Bit value; t and T are the current training iteration and the final iteration of frequency regularization, respectively. Specifically, starting from the original input without position encoding, as training progresses, the visible frequency is increased by 3 bits each time. Frequency regularization bypasses the unstable and easily affected high-frequency signals at the beginning of training, and gradually provides high-frequency information of the SPECT projection view, avoiding over-smoothing. S2: Input the sparse view after wavelet series mapping and frequency regularization into a fully connected multilayer perceptron to learn high-frequency functions in a low-dimensional problem domain; S3: The network is trained by minimizing the error between the real projection and the synthetic projection, and a dense view without the training view is generated during prediction. S4: Combine the sparse view during training with the generated dense view, and use the Preconditioned Alternating Projection Algorithms reconstruction algorithm to achieve SPECT reconstruction of low-dose, high-noise sparse views; S5: In the image denoising part, the weight parameters of the fully connected multilayer perceptron in the first part are used as the pre-training weights of the current network. The inherent denoising ability of the implicit neural representation is maximized by penalizing the growth of deeper weights, so as to achieve high-quality SPECT image denoising.

2. The method for low-dose sparse view SPECT reconstruction and image denoising based on implicit neural representation according to claim 1, characterized in that, The fully connected multilayer perceptron network in step S2 includes 8 fully connected layers. The first and fourth layers of the fully connected multilayer perceptron network have a different number of neurons than the other layers of the fully connected multilayer perceptron network. The other layers of the fully connected multilayer perceptron network each have 256 neurons. Except for the last layer, each fully connected layer is followed by a periodic sin activation function.

3. The method for low-dose sparse view SPECT reconstruction and image denoising based on implicit neural representation according to claim 1, characterized in that, The real projection in step S3 and composite projection The loss function between them is: Among them, projection I can be used To indicate, For initial strength, Let B be the distance between adjacent points, where B is a set of gamma rays. The parameters of the fully connected multilayer perceptron network are updated during training. .

4. The method for low-dose sparse view SPECT reconstruction and image denoising based on implicit neural representation according to claim 1, characterized in that, In step S4, the corresponding SPECT image is obtained by the Preconditioned Alternating Projection Algorithms reconstruction algorithm.

5. The method for low-dose sparse view SPECT reconstruction and image denoising based on implicit neural representation (INR) according to claim 1, characterized in that, The denoising algorithm in step S5 uses the weight parameters of the fully connected multilayer perceptron from the first stage as the pre-training weights of the current network. Training stops when the mean square error (MSE) between the image generated by the fully connected multilayer perceptron and the target noise image is less than the noise level estimated for the target image. The mean square error is expressed as follows: Where C is the set of all pixel coordinate pairs in the noisy image; image pixels are obtained using a 128×128 grid, and the coordinate pairs can be set as a grid. Each pair represents the center coordinates of a pixel. Training stops when the mean squared error (MSE) between the INR-generated image and the target noisy image is less than the noise level estimated from the target image, i.e., it satisfies... in Image function The noise power is considered in the ideal case where the implicit neural representation bias provides perfect temporal separation between the image and the noise fit. In fact, the stopping criterion will satisfy equality if the image generated by the fully connected multilayer perceptron is exactly the same as the clean image. In addition, to mitigate implicit neural representation bias, weight decay is selectively applied to the last two fully connected perceptron layers.

Images