A general low-dose PET image synthesis method based on pattern alignment

Abstract

The application discloses a general low-dose PET image synthesis method based on style alignment, and relates to the technical field of image synthesis. The general low-dose PET image synthesis method based on style alignment comprises the following steps: estimating the residual of an input image through a residual learning structure to generate a synthesized PET image; aligning the styles of low-dose PET images with different dose reduction coefficients; realizing the recovery of a network learning style area; training a convolution network model to minimize the difference between the synthesized PET image and a full-dose PET image; and verifying the trained convolution network model through data with different dose reduction coefficients. The application proposes a style alignment network (SAN), so that the model can adaptively align and recover the styles of low-dose PET images with different dose reduction factors (DRFs), thereby enabling the model to have strong generalization on low-dose PET images with different DRFs.

Description

Technical Field

[0001] This invention relates to the field of image synthesis technology, and more specifically, to a general low-dose PET image synthesis method based on style alignment. Background Technology

[0002] Positron emission tomography (PET) is a functional imaging device capable of detecting metabolic activity in body tissues and cells, and it has numerous applications in clinical oncology imaging and cancer metastasis research. Generally, acquiring high-quality full-dose PET images in clinical applications requires injecting patients with a sufficient amount of radioactive tracer, which poses potential radiation hazards. However, using less tracer results in poor-quality low-dose PET images with severe noise and artifacts, thus affecting clinical diagnosis. How to obtain high-quality images while reducing the radioactive tracer dose is a focus for researchers.

[0003] Subsequently, the task of PET image synthesis was proposed, aiming to use image processing techniques to restore low-dose PET images to full-dose PET images. With the development of deep learning, significant progress has been made in the PET image synthesis task. Due to the powerful mapping capabilities of convolutional neural networks and Transformer networks, researchers have been able to train mapping network models using pairs of low-dose and full-dose PET images, achieving the mapping from low-dose PET images to full-dose PET images, thereby improving the quality of low-dose PET images.

[0004] However, most deep learning methods are trained on PET images with a specific dose reduction factor (DRF, defined as the degree of reduction of the radioactive tracer; for example, if the full dose is three times that of the low dose, then DRF=3), assuming that the DRF of the low-dose PET images is fixed and known. In practical applications, due to variations in dosing regimens and scan times, the true DRF of PET images is often unknown and varies within a certain range. When the DRF of the input PET image differs significantly from the DRF used in model training, the performance of these DRF-specific models will drop sharply.

[0005] In this situation, there are two solutions. One is to train a model for all possible DRFs, but this is clearly impractical. On the one hand, training a specific model for each DRF requires a large amount of computational resources; on the other hand, the number of models would be enormous and difficult to maintain. Furthermore, due to the uncertainty of DRFs in practical applications, selecting an appropriate model from so many models would also be very inconvenient.

[0006] Another solution is to train a general model on PET image data with different DRFs to achieve mapping from low-dose PET images to full-dose PET images with different DRFs. However, our experiments found that the general model suffers from severe style (image texture and detail) loss compared to models specific to a single DRF. The general model tends to remove styles from images with different DRFs, resulting in overly smoothed image outputs. This is because the styles of images with different DRFs are misaligned, and the general model lacks a mechanism to handle style misalignment. Therefore, during training, the model tends to ignore and remove these misaligned styles to achieve stronger generalization ability. However, the loss of style information leads to the loss of detail in the final recovered image, resulting in oversmoothing. Therefore, how to solve the style loss problem in the general model is crucial.

[0007] No effective solutions have yet been proposed to address the problems in the relevant technologies. Summary of the Invention

[0008] In response to the problems in related technologies, this invention proposes a general low-dose PET image synthesis method based on pattern alignment to overcome the aforementioned technical problems existing in the prior art.

[0009] Therefore, the specific technical solution adopted by the present invention is as follows:

[0010] A general low-dose PET image synthesis method based on pattern alignment, comprising the following steps:

[0011] S1. Based on the convolutional network model, and by estimating the residual of the input image through the residual learning structure, a synthetic PET image is generated;

[0012] S2. Align the low-dose PET image styles with different dose reduction coefficients using a style alignment network;

[0013] S3. Utilize a region-aware learning strategy to guide the network to focus on pattern regions during training, and enable the network to learn the recovery of pattern regions;

[0014] S4. Define and optimize the loss function, combining PET image content restoration, style alignment loss and generative adversarial loss, to train the convolutional network model to minimize the difference between synthetic PET images and full-dose PET images;

[0015] S5. Use a deep learning framework to train the convolutional network model, and validate the trained convolutional network model using data with different dose reduction coefficients.

[0016] Furthermore, based on a convolutional network model and by estimating the residuals of the input image through a residual learning structure, the generation of synthetic PET images includes the following steps:

[0017] S11. Input low-dose PET images with dose reduction coefficient;

[0018] S12. Use a convolutional layer to perform a convolution operation on the low-dose PET image with the input dose reduction coefficient to obtain shallow features;

[0019] S13. Extract features from the shallow features obtained by using several residual blocks to obtain deep features;

[0020] S14. The obtained deep features are convolved again using the convolutional layer to generate the estimated residual features.

[0021] S15. Combine the low-dose PET image with the input dose reduction coefficient and the residual features, and generate the final synthetic PET image by summing element by element.

[0022] Furthermore, the formula for generating the final synthetic PET image by summing element by element is as follows:

[0023] ;

[0024] In the formula, F represents the characteristic;

[0025] d represents the dose reduction factor;

[0026] Represents low-dose PET images;

[0027] L indicates a low dose;

[0028] R represents the residual;

[0029] This represents a composite PET image;

[0030] Low-dose PET images representing the input dose reduction factor;

[0031] This represents the residual characteristics.

[0032] Furthermore, aligning the low-dose PET image patterns with different dose reduction coefficients using a pattern alignment network includes the following steps:

[0033] S21. Extract shallow features from the input low-dose PET images using a style alignment network;

[0034] S22. The extracted shallow features are compressed by global average pooling, and the domain information encoding is obtained.

[0035] S23. Perform an affine transformation on the acquired domain information encoding to obtain the pattern encoding;

[0036] S24. Combine the obtained style encoding with the convolution parameters in the style alignment network to calculate the dynamic convolution weights;

[0037] S25. Divide the residual blocks in the style alignment network into several groups, with each group consisting of two groups, corresponding to several different levels of style codes.

[0038] S26. Integrate the convolution weights of each group with the corresponding style codes to achieve fine-tuning of feature styles at different levels;

[0039] S27. For each dose reduction coefficient, calculate the pattern feature matrix of the residual block output features for each low-dose PET image pattern.

[0040] S28. Define the style alignment loss as the norm of the difference between the style feature matrix of a low-dose PET image with different dose reduction coefficients and the style feature matrix of a full-dose PET image.

[0041] S29. By minimizing the style alignment loss, the model is guided to learn to align the styles of low-dose PET images with the styles of full-dose PET images at different dose reduction coefficients, thus obtaining the final style alignment loss.

[0042] Furthermore, by performing an affine transformation on the acquired domain information encoding, the formula for the pattern encoding is obtained as follows:

[0043] S = A(GAP(P));

[0044] In the formula, A represents affine transformation;

[0045] S represents the style code;

[0046] P represents the domain information encoding;

[0047] GAP stands for Global Average Pooling.

[0048] Furthermore, the formula for calculating the dynamic convolution weights is:

[0049] ;

[0050] In the formula, These are the weight parameters of the dynamic convolution after integrating the style encoding S;

[0051] These are the original convolution weight parameters in the convolutional network model;

[0052] ;

[0053] .

[0054] Furthermore, by minimizing the style alignment loss, the model is guided to learn to align the styles of low-dose PET images with different dose reduction coefficients to those of full-dose PET images, resulting in the final formula for the style alignment loss:

[0055] ;

[0056] In the formula, Indicates style alignment loss;

[0057] This represents a low-dose PET image style with a dose reduction factor of d.

[0058] This indicates the style of a full-dose PET image.

[0059] Furthermore, the use of region-aware learning strategies to guide the network to focus on pattern regions during training, and the realization of the network learning the recovery of pattern regions, includes the following steps:

[0060] S31. For full-dose PET images, use a window to calculate the variance at the spatial location of each voxel and identify regions with rich pattern information in the full-dose PET images;

[0061] S32. Variance is controlled by setting a threshold. Figure 2 Value-encode and generate a style area mask;

[0062] S33. Combine the generated style region mask with the adversarial network, and constrain the learning region of the adversarial network to the region with rich style regions to enhance its style recovery ability.

[0063] S34. Generative adversarial learning is performed using a discriminant network with the pattern regions of full-dose PET images and synthetic PET images as inputs to the discriminant network.

[0064] Furthermore, by setting a threshold, the variance can be... Figure 2 The formula for value-enhancing and generating a style region mask is:

[0065] ;

[0066] In the formula, Indicates a mask;

[0067] Indicates the current voxel position;

[0068] Represents a variance plot;

[0069] This represents the threshold.

[0070] Furthermore, the formula for generative adversarial learning is:

[0071] ;

[0072] In the formula, This represents the adversarial learning loss function of the discriminant network;

[0073] Represents the sampled image;

[0074] This indicates the distribution of the pattern region image in the composite PET image;

[0075] This represents the distribution of the pattern region image in a true full-dose PET image;

[0076] express and The sampling distribution between;

[0077] Represents the weight coefficients of the gradient penalty;

[0078] This indicates the discriminator output with respect to the input. The second derivative of the gradient is used to calculate the norm of the gradient;

[0079] The second norm of the gradient;

[0080] This represents the operator for calculating the expectation.

[0081] The beneficial effects of this invention are as follows:

[0082] (1) This invention proposes a pattern alignment network (SAN), which enables the model to adaptively align and restore the patterns of low-dose PET images with different DRFs, thereby enabling the model to have strong generalization ability in low-dose PET images with different DRFs.

[0083] (2) This invention proposes the Region Aware Learning Strategy (RALS) to guide the network to focus on regions rich in style information, which is beneficial for the model to learn the alignment and recovery of style information. Attached Figure Description

[0084] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0085] Figure 1 This is a flowchart of a general low-dose PET image synthesis method based on style alignment according to an embodiment of the present invention;

[0086] Figure 2 This is a general low-dose PET image synthesis network framework diagram according to an embodiment of the present invention;

[0087] Figure 3 This is a comparison chart of the present invention and the conventional method 3D-cGAN when faced with different DRF low-dose PET image inputs. Detailed Implementation

[0088] To further illustrate the various embodiments, the present invention provides accompanying drawings, which are part of the disclosure of the present invention. These drawings are mainly used to illustrate the embodiments and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. With reference to these drawings, those skilled in the art should be able to understand other possible implementation methods and the advantages of the present invention.

[0089] According to embodiments of the present invention, a general low-dose PET image synthesis method based on style alignment is provided.

[0090] The present invention will now be further described in conjunction with the accompanying drawings and specific embodiments, such as... Figure 1 As shown, the general low-dose PET image synthesis method based on pattern alignment according to an embodiment of the present invention includes the following steps:

[0091] S1. Based on the convolutional network model (i.e., the basic synthetic network BSN), and by estimating the residual of the input image through the residual learning structure, a synthetic PET image is generated.

[0092] S2. Align the low-dose PET image styles with different dose reduction coefficients using a style alignment network;

[0093] S3. Utilize a region-aware learning strategy to guide the network to focus on pattern regions during training, and enable the network to learn the recovery of pattern regions;

[0094] S4. Define and optimize the loss function, combining PET image content restoration, style alignment loss and generative adversarial loss, to train the convolutional network model to minimize the difference between synthetic PET images and full-dose PET images;

[0095] It needs to be explained that we use Loss function to measure synthetic PET images Compared with full-dose PET images Minimize the differences between them Loss represents minimizing the synthesized PET image Compared with full-dose PET images The differences between them. In addition, consider the style alignment loss. and generating adversarial losses The final loss function of the model is:

[0096] .

[0097] In the formula, This represents the final loss function of the model; Represents the loss function; Indicates style alignment loss; This indicates the generation of adversarial loss.

[0098] S5. Use a deep learning framework to train the convolutional network model, and validate the trained convolutional network model using data with different dose reduction coefficients.

[0099] It should be noted that, for the model settings, we set the number of residual blocks in the BSN to... The network was divided into four groups according to its depth for four different levels of pattern modulation, with each group containing a certain number of residual blocks. The number of intermediate features in the network is set to .

[0100] It should be noted that for model training, we implemented our proposed method using PyTorch on a workstation equipped with an NVIDIA A100 GPU. We employed the Adam optimizer with a fixed learning rate of 0.0001. The training data came from different DRFs, and we randomly cropped 64×64×64 blocks from the full dataset for model training. The batch size was set to 4, and the training was performed for a total of 600 epochs.

[0101] Preferably, generating a synthetic PET image based on a convolutional network model and estimating the residuals of the input image through a residual learning structure includes the following steps:

[0102] S11. Input low-dose PET images with dose reduction coefficient;

[0103] S12. Use a convolutional layer to perform a convolution operation on the low-dose PET image with the input dose reduction coefficient to obtain shallow features;

[0104] S13. Extract features from the shallow features obtained by using several residual blocks to obtain deep features;

[0105] S14. The obtained deep features are convolved again using the convolutional layer to generate the estimated residual features.

[0106] S15. Combine the low-dose PET image with the input dose reduction coefficient and the residual features, and generate the final synthetic PET image by summing element by element.

[0107] It should be explained that the Basic Synthesis Network (BSN) is the backbone of general PET image synthesis, and it can be any existing convolutional-based network model. In this invention, we select a network with a residual structure as the BSN for our method, such as... Figure 2 As shown. BSN employs a residual learning structure to estimate the input residuals. Given a low-dose PET image with a specific DRF=d. BSN first applies 3×3×3 convolutional layers to obtain shallow features. These shallow features Deep features are obtained by feature extraction from N=8 residual blocks. Next, another 3×3×3 convolutional layer processes the depth features. Convolution operations are performed to generate estimated residual features. The final output is a composite image. By inputting an image With residual characteristics Element-wise summation produces: .in The symbol represents the feature, and d represents the dose reduction factor. Represents an image. This represents the residual.

[0108] Preferably, the formula for generating the final synthetic PET image by element-wise summation is:

[0109] ;

[0110] In the formula, F represents the characteristic;

[0111] d represents the dose reduction factor;

[0112] Represents low-dose PET images;

[0113] L indicates a low dose;

[0114] R represents the residual;

[0115] This represents a composite PET image;

[0116] Low-dose PET images representing the input dose reduction factor;

[0117] This represents the residual characteristics.

[0118] Preferably, aligning low-dose PET image patterns with different dose reduction coefficients using a pattern alignment network includes the following steps:

[0119] S21. Extract shallow features from the input low-dose PET images using a style alignment network;

[0120] S22. The extracted shallow features are compressed by global average pooling, and the domain information encoding is obtained.

[0121] S23. Perform an affine transformation on the acquired domain information encoding to obtain the pattern encoding;

[0122] S24. Combine the obtained style encoding with the convolution parameters in the style alignment network to calculate the dynamic convolution weights;

[0123] S25. Divide the residual blocks in the style alignment network into several groups, with each group consisting of two groups, corresponding to several different levels of style codes.

[0124] S26. Integrate the convolution weights of each group with the corresponding style codes to achieve fine-tuning of feature styles at different levels;

[0125] S27. For each dose reduction coefficient, calculate the pattern feature matrix of the residual block output features for each low-dose PET image pattern.

[0126] S28. Define the style alignment loss as the norm of the difference between the style feature matrix of a low-dose PET image with different dose reduction coefficients and the style feature matrix of a full-dose PET image.

[0127] S29. By minimizing the style alignment loss, the model is guided to learn to align the styles of low-dose PET images with the styles of full-dose PET images at different dose reduction coefficients, thus obtaining the final style alignment loss.

[0128] It should be noted that PET images from different DRFs are not aligned in style, and the BSN lacks a mechanism to handle the styles of different DRF data, which ultimately leads to the problem of missing styles. In order to effectively align and recover the styles of PET images from different DRFs, we propose a Style Alignment Network (SAN), which mainly consists of three parts: 1) domain information encoding, 2) style modulation, and 3) style alignment loss.

[0129] 1) Domain information encoding:

[0130] Domain information needs to reflect the characteristics of different DRF data and provide effective guidance for the image restoration process of the base synthesis network (BSN), enabling the BSN to adaptively process the input low-dose PET images. Previous studies typically used the low-dose image itself or its high-frequency components as domain information. In this invention, we reveal the shallow features extracted by the base synthesis network (BSN). It is a more effective type of domain information, on the one hand Because it is close to the input end, the information it contains can reflect the characteristics of the input image. On the other hand... It is the starting point for subsequent feature extraction in the basic synthetic network BSN, and is aligned with the following feature extraction process, so This can provide more refined guidance for feature extraction in BSN.

[0131] After obtaining the domain information, we use a 4-layer encoding module to encode the domain information. Each layer of the encoding module is composed of a combination of convolution, PReLU activation function, channel attention and convolution. After four consecutive layers of encoding, each layer has a domain information encoding result, resulting in four different domain information encodings P with different depths and levels.

[0132] 2) Style Modulation:

[0133] After obtaining the domain information encodings at four different levels, the key is how to use these encodings to adjust the style information in the basic synthesis network BSN. We borrow the style modulation mechanism from StyleGAN to manipulate and control the feature styles in the basic synthesis network BSN.

[0134] Specifically, we first compress the domain information encoding P using Global Average Pooling (GAP), then obtain the pattern encoding S through affine transformation A. After obtaining the pattern encoding S, we combine the pattern encoding S with the convolution parameters in the basic synthesis network BSN. The process involves integrating ordinary convolutions into dynamic convolutions with style modulation capabilities.

[0135] 3) Style alignment loss:

[0136] The purpose of pattern modulation is to align the patterns of different DRF image features. In convolutional network models, the image's pattern features are contained in the feature mean and standard deviation at the channel level. Therefore, given... =1, the i-th residual block in BSN Output features The style can be represented as Mean of channel level and standard deviation The concatenation matrix is:

[0137] ;

[0138] DRF style This can be represented as the pattern concatenation matrix of all N residual blocks in the BSN as follows:

[0139] ;

[0140] The difference between the styles of any two DRFs can then be expressed as ,in , and Let represent any two different dose reduction coefficients d. However, directly minimizing this loss function requires cumbersome loading of different DRF data for the same patient, which is very time-consuming and computationally expensive, and the final convergence pattern space is uncertain. To solve this problem, we use full-dose data, that is... The style of the PET image is used as a reference, and then the styles of other DRFs are aligned with it to obtain the final style alignment loss. This style alignment loss is then used... This prompts the pattern modulation mechanism to align the patterns of low-dose PET images with the patterns of full-dose PET images from different DRFs, projecting different DRF data patterns into the same space, thus enabling the model to generalize on different DRF data.

[0141] Preferably, the acquired domain information encoding is subjected to an affine transformation to obtain the formula for the pattern encoding:

[0142] S = A(GAP(P));

[0143] In the formula, A represents affine transformation;

[0144] Specifically, A represents an affine transformation, which is a linear layer with a bias.

[0145] S represents the style code;

[0146] P represents the domain information encoding;

[0147] GAP stands for Global Average Pooling.

[0148] Preferably, the formula for calculating the dynamic convolution weights is:

[0149] ;

[0150] In the formula, These are the weight parameters of the dynamic convolution after integrating the style encoding S;

[0151] These are the original convolution weight parameters in the convolutional network model (i.e., the basic synthetic network BSN);

[0152] ;

[0153] .

[0154] It should be explained that the weight parameters of ordinary convolution are independent of the image content, while dynamic convolution, because it integrates the image's style encoding, can dynamically process and modulate image features according to the content of the input image.

[0155] It should be explained that, in order to perform more precise pattern modulation, since there are four different levels of pattern encoding, we divided the eight residual blocks in the BSN into four groups according to their arrangement, with each group consisting of two adjacent residual blocks. The four different levels of pattern encoding are then integrated with the convolutions in the four groups of residual blocks, as shown in the formula. As shown, this allows for more precise and flexible style modulation.

[0156] Preferably, by minimizing the style alignment loss, the model is guided to learn to align the styles of low-dose PET images with different dose reduction coefficients to the styles of full-dose PET images, resulting in the formula for the final style alignment loss:

[0157] ;

[0158] In the formula, Indicates style alignment loss;

[0159] This represents a low-dose PET image style with a dose reduction factor of d.

[0160] This indicates the style of a full-dose PET image.

[0161] Specifically, .

[0162] Preferably, using a region-aware learning strategy to guide the network to focus on pattern regions during training and to achieve the network's learning of pattern region recovery includes the following steps:

[0163] S31. For full-dose PET images, use a window to calculate the variance at the spatial location of each voxel and identify regions with rich pattern information in the full-dose PET images;

[0164] S32. Variance is controlled by setting a threshold. Figure 2 Value-encode and generate a style area mask;

[0165] S33. Combine the generated style region mask with the adversarial network, and constrain the learning region of the adversarial network to the region with rich style regions to enhance its style recovery ability.

[0166] S34. Generative adversarial learning is performed using a discriminant network with the pattern regions of full-dose PET images and synthetic PET images as inputs to the discriminant network.

[0167] It's important to explain that because most PET images consist of simple, flat regions with only a small subset possessing rich pattern information, this data imbalance makes it difficult for networks to learn how to process pattern information. While some studies have successfully used Generative Adversarial Networks (GANs) to recover image pattern information, GAN-based methods are prone to overfitting when faced with data imbalance. To mitigate this issue, we propose a Region-Aware Learning Strategy (RALS). RALS guides the network to focus on pattern regions during training, which is beneficial for the network to learn pattern region recovery and align different DRF patterns. We will introduce RALS from two perspectives: 1) Pattern Region Identification and 2) Region-Aware Adversarial Learning.

[0168] 1) Style area identification:

[0169] We divided PET images into two regions: flat regions and patterned regions. Flat regions refer to areas with simple or average textures, while patterned regions are areas with rich texture information and can directly reflect the characteristics of the image. To distinguish between flat and patterned regions, we measured full-dose PET images. Each spatial location The variance of voxels in a window region of size 3 is used to obtain the variance plot. .

[0170] It needs to be explained that, for example Figure 2 As shown in Chinese diagram 6, the variance plot The formula is:

[0171] ;

[0172] In the formula, Both represent serial numbers. Indicates the current voxel position. This refers to the current location. Based on the distance moved, This refers to the operator used to calculate variance.

[0173] 2) Region-aware adversarial learning:

[0174] Obtain the style area mask Next, we combined it with a Generative Adversarial Network (GAN) to constrain the GAN's learning region to the pattern region, enhancing its pattern recovery capabilities. Specifically, we used full-dose images... And a synthetic image of a certain dose With style area mask Perform a dot product to obtain their respective style regions. and :

[0175] ;

[0176] ;

[0177] Then, a discriminator network (D) from PatchGAN (Markov Discriminator) is introduced, using the pattern regions of the full-dose image and the generated image as inputs to the discriminator network D for generative adversarial learning. When training the discriminator network D, its loss function... .

[0178] Preferably, such as Figure 2 As shown in Equation 7, the variance is controlled by setting a threshold. Figure 2 The formula for value-enhancing and generating a style region mask is:

[0179] ;

[0180] In the formula, Indicates a mask;

[0181] Indicates the current voxel position;

[0182] Represents a variance plot;

[0183] This represents the threshold.

[0184] Preferably, the formula for generative adversarial learning is:

[0185] ;

[0186] In the formula, This represents the adversarial learning loss function of the discriminant network;

[0187] Represents the sampled image;

[0188] This indicates the distribution of the pattern region image in the composite PET image;

[0189] This represents the distribution of the pattern region image in a true full-dose PET image;

[0190] express and The sampling distribution between;

[0191] Represents the weight coefficients of the gradient penalty;

[0192] This indicates the discriminator output with respect to the input. The second derivative of the gradient is used to calculate the norm of the gradient;

[0193] The second norm of the gradient;

[0194] It expresses expectation.

[0195] Specifically, the first two terms of the above generative adversarial learning formula mean that the synthetic PET image pattern region is scored low and the real full-dose image pattern region is scored high; the third term of the formula is the gradient penalty term, which makes the training of the generative adversarial network (GAN) smoother and more stable.

[0196] When training the generative network (whose role is to generate images; in this invention, image generation is achieved jointly by BSN and SAN, such as...) Figure 2 As shown, the generator network of this invention refers to the overall BSN and SAN), and the loss function of the generator network. for:

[0197] ;

[0198] In the formula, This indicates the generation of adversarial loss;

[0199] D represents the discriminant network;

[0200] Represents the sampled image;

[0201] This indicates the distribution of the pattern region image in the composite PET image;

[0202] E z~O Represents sampled image From the synthetic PET image style region image The expected value obtained from sampling.

[0203] Its purpose is to ensure that the pattern regions of the images generated by the generative network score as high as possible for the discriminative network D, thereby deceiving D. This adversarial learning ultimately directly narrows the gap between the pattern regions of full-dose PET images and synthetic PET images. In this process, we constrain generative adversarial training to the pattern regions, making the network model focus more on recovering pattern information. This training strategy aligns with the fundamental principles of GANs because, compared to traditional GANs that use the entire image as input to the discriminative network D, our operation of only inputting pattern regions merely changes the distribution of the input image and does not violate the principles of GANs.

[0204] In addition, it needs to be explained that, such as Figure 2 The diagram shows the framework of the general low-dose PET image synthesis network proposed in this invention. It mainly consists of a Base Synthesis Network (BSN), a Style Alignment Network (SAN), and a Region-aware Learning Strategy (RALS). The BSN is the backbone network for image restoration, and its structure can be any existing convolutional network model. Our innovation lies in proposing the Style Alignment Network (SAN) and the Region-aware Learning Strategy (RALS) to enhance the style alignment and restoration performance of the Base Synthesis Network (BSN). The role of the SAN is to align and restore the styles of different DRFs in the BSN, while the role of the RALS is to allow the entire model to focus on regions rich in style information during training, enabling the model to better learn the restoration of style information.

[0205] In addition, such as Figure 3 As shown, the traditional 3D-cGAN method performs poorly on some dose images (e.g., DRF=6 and 12) when faced with low-dose PET image inputs with different DRFs, exhibiting excessive smoothing. In contrast, our proposed method demonstrates consistently better performance across different DRF datasets.

[0206] Specifically, Table 1 shows the comparison between our method and 3D-cGAN in terms of image similarity metrics PSNR and SSIM under four different dose reduction factors (DRF=2, 3, 6, 12):

[0207] Table 1:

[0208]

[0209] In summary, by utilizing the above-mentioned technical solutions of this invention, the present invention proposes a Pattern Alignment Network (SAN), enabling the model to adaptively align and restore the patterns of low-dose PET images with different DRFs, thereby achieving strong generalization ability for low-dose PET images with different DRFs. Furthermore, this invention proposes a Region Aware Learning Strategy (RALS) to guide the network to focus on regions rich in pattern information, which is beneficial for the model to learn the alignment and restoration of pattern information.

[0210] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A general low-dose PET image synthesis method based on pattern alignment, characterized in that, This style-alignment-based general low-dose PET image synthesis method includes the following steps: S1. Based on the convolutional network model, and by estimating the residual of the input image through the residual learning structure, a synthetic PET image is generated; S2. Align the low-dose PET image styles with different dose reduction coefficients using a style alignment network; S3. Utilize a region-aware learning strategy to guide the network to focus on pattern regions during training, and enable the network to learn the recovery of pattern regions; S4. Define and optimize the loss function, combining PET image content restoration, style alignment loss and generative adversarial loss, to train the convolutional network model to minimize the difference between synthetic PET images and full-dose PET images; S5. Use a deep learning framework to train the convolutional network model, and validate the trained convolutional network model using data with different dose reduction coefficients. The process of aligning low-dose PET image patterns with different dose reduction coefficients using a pattern alignment network includes the following steps: S21. Extract shallow features from the input low-dose PET images using a style alignment network; S22. The extracted shallow features are compressed by global average pooling, and the domain information encoding is obtained. S23. Perform an affine transformation on the acquired domain information encoding to obtain the pattern encoding; S24. Combine the obtained style encoding with the convolution parameters in the style alignment network to calculate the dynamic convolution weights; S25. Divide the residual blocks in the style alignment network into several groups, with each group consisting of two groups, corresponding to several different levels of style codes. S26. Integrate the convolution weights of each group with the corresponding style codes to achieve fine-tuning of feature styles at different levels; S27. For each dose reduction coefficient, calculate the pattern feature matrix of the residual block output features for each low-dose PET image pattern. S28. Define the style alignment loss as the norm of the difference between the style feature matrix of a low-dose PET image with different dose reduction coefficients and the style feature matrix of a full-dose PET image. S29. By minimizing the style alignment loss, the model is guided to learn to align the styles of low-dose PET images with the styles of full-dose PET images at different dose reduction coefficients, thus obtaining the final style alignment loss. The method of using a region-aware learning strategy to guide the network to focus on pattern regions during training and to enable the network to learn the recovery of pattern regions includes the following steps: S31. For full-dose PET images, use a window to calculate the variance at the spatial location of each voxel and identify regions with rich pattern information in the full-dose PET images; S32. Binarize the variance plot by setting a threshold and generate a pattern region mask; S33. Combine the generated style region mask with the adversarial network, and constrain the learning region of the adversarial network to the region with rich style regions to enhance its style recovery ability. S34. Generative adversarial learning is performed using a discriminant network with the pattern regions of full-dose PET images and synthetic PET images as inputs to the discriminant network.

2. The general low-dose PET image synthesis method based on pattern alignment according to claim 1, characterized in that, The process of generating synthetic PET images based on a convolutional network model and estimating the residuals of the input image through a residual learning structure includes the following steps: S11. Input low-dose PET images with dose reduction coefficient; S12. Use a convolutional layer to perform a convolution operation on the low-dose PET image with the input dose reduction coefficient to obtain shallow features; S13. Extract features from the shallow features obtained by using several residual blocks to obtain deep features; S14. The obtained deep features are convolved again using the convolutional layer to generate the estimated residual features. S15. Combine the low-dose PET image with the input dose reduction coefficient and the residual features, and generate the final synthetic PET image by summing element by element.

3. The general low-dose PET image synthesis method based on pattern alignment according to claim 2, characterized in that, The formula for generating the final synthetic PET image by element-wise summation is as follows: ； In the formula, F represents the characteristic; d represents the dose reduction coefficient; Represents low-dose PET images; L indicates a low dose; R represents the residual; This represents a composite PET image; Low-dose PET images representing the input dose reduction factor; This represents the residual characteristics.

4. The general low-dose PET image synthesis method based on pattern alignment according to claim 1, characterized in that, The formula for obtaining the pattern encoding by performing an affine transformation on the acquired domain information encoding is as follows: S = A(GAP(P)); In the formula, A represents affine transformation; S represents the style code; P represents the domain information encoding; GAP stands for Global Average Pooling.

5. A general low-dose PET image synthesis method based on pattern alignment according to claim 1, characterized in that, The formula for calculating the dynamic convolution weights is as follows: ； In the formula, These are the weight parameters of the dynamic convolution after integrating the style encoding S; These are the original convolution weight parameters in the convolutional network model; ； 。 6. The general low-dose PET image synthesis method based on pattern alignment according to claim 1, characterized in that, The process involves minimizing the style alignment loss to guide the model in learning to align the styles of low-dose PET images with those of full-dose PET images at different dose reduction coefficients. The final formula for the style alignment loss is as follows: ； In the formula, Indicates style alignment loss; This represents a low-dose PET image style with a dose reduction factor of d. This indicates the style of a full-dose PET image.

7. A general low-dose PET image synthesis method based on pattern alignment according to claim 1, characterized in that, The formula for binarizing the variance map and generating a pattern region mask by setting a threshold is as follows: ； In the formula, Indicates a mask; Indicates the current voxel position; Represents a variance plot; This represents the threshold.

8. A general low-dose PET image synthesis method based on pattern alignment according to claim 1, characterized in that, The formula for generative adversarial learning is: ； In the formula, This represents the adversarial learning loss function of the discriminant network; Represents the sampled image; This indicates the distribution of the pattern region image in the composite PET image; This represents the distribution of the pattern region image in a true full-dose PET image; express and The sampling distribution between; Represents the weight coefficients of the gradient penalty; This indicates the discriminator output with respect to the input. The second derivative of the gradient is used to calculate the norm of the gradient; The second norm of the gradient; This represents the operator for calculating the expectation.

Images