A 3D Gaussian model generation and rendering method for low-dose PET image processing

Abstract

The present invention relates to the technical field of medical image processing, and in particular, to a 3D Gaussian model generation and rendering method for low-dose PET image processing. It comprises the following steps: inputting a low-dose brain PET image, wherein the brain PET image is composed of a set of voxel points; clustering the voxel points to generate a simplified Gaussian model; based on the simplified Gaussian model, resampling the Gaussian ellipsoid model, and finally constructing a Gaussian distribution model of the brain PET image; a UNet model receives the brain PET image as input to extract and segment the outer contour of the brain PET image; reconstructing a 3D Gaussian distribution model based on a VQ‑VAE autoencoder and a Stable Diffusion model; and rendering the 3D Gaussian distribution model into a 3D image by calculating the attenuation characteristics when light interacts with voxel point data. In the present invention, under the condition of low-dose brain PET imaging, the noise and artifacts in the image are significantly reduced, and the image resolution is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to a 3D Gaussian model generation and rendering method for low-dose PET image processing, which is used to optimize the processing and reconstruction of low-dose brain positron emission tomography (PET) images. Background Art

[0002] Positron emission tomography (PET) is an important medical imaging technology that generates three-dimensional images to provide dynamic observation of biochemical processes by detecting the distribution of radioactive markers in the body. PET is widely used in early screening and diagnosis of cancer, neurodegenerative diseases, etc. Especially in brain imaging, PET provides spatial distribution images of brain metabolic activities by detecting the emission and attenuation characteristics of gamma photons, providing key support for the diagnosis and research of neurological diseases (such as Alzheimer's disease, Parkinson's disease, etc.).

[0003] However, existing brain PET technology still faces many challenges:

[0004] 1. Image resolution and noise issues: Low-dose brain PET images are often accompanied by high noise and artifacts due to insufficient number of collected photons under patient safety requirements, resulting in reduced image resolution and blurred details. This problem particularly affects the accurate imaging of complex brain structures.

[0005] 2. Patient radiation exposure risk: The dose of radioactive tracers directly determines the image quality, and higher doses increase the patient's radiation exposure risk. For patients with long-term monitoring of neurological diseases (such as the elderly with multiple follow-ups) and vulnerable groups (such as children), the contradiction between reducing dose and improving image quality is particularly prominent.

[0006] 3. Insufficient accuracy of three-dimensional modeling: Existing brain three-dimensional image modeling technologies mostly use simple volume rendering or linear interpolation methods, which cannot accurately express the complex anatomical structure details of the brain and are difficult to meet the needs of high-precision diagnosis.

[0007] In recent years, deep learning technology has made breakthrough progress in the field of medical image processing, especially in the field of 3D modeling and reconstruction of the brain. The technology combined with convolutional neural network (CNN) can generate higher quality 2D slices from low-dose data and try to build 3D models. However, the application of these methods in brain PET imaging faces the following problems:

[0008] 1. Lack of effective integration of prior information: Existing deep learning methods find it difficult to fully utilize the correlation between low-dose and high-dose brain PET images to achieve higher quality image reconstruction.

[0009] 2. Limitations of 3D modeling: Traditional methods have limited processing capabilities for sparse data when generating 3D models of the brain, resulting in the inability to accurately express the details of the reconstructed model, especially in the edges and small structure areas of the brain.

[0010] To address the above problems, the 3D Gaussian Model proposed in recent years has shown superior performance in processing sparse data and noise. Through the clustering and optimization of discrete voxel data, the 3D Gaussian model can generate a simplified voxel grid, significantly improving the quality of brain 3D modeling. In addition, the Diffusion Model, as an emerging generation technology, can generate a 3D model that is highly consistent with the target data by gradually adding noise and then reverse denoising. This technology effectively solves the shortcomings of traditional deep learning in generating 3D brain images with rich details and complex structures.

[0011] However, brain PET imaging research based on 3D Gaussian models and diffusion technology is still in the exploratory stage:

[0012] 1. The clustering accuracy of the Gaussian model is limited: Under high noise conditions in the brain, existing methods are difficult to effectively distinguish the characteristics of neighborhood data, which affects the accuracy of clustering results.

[0013] 2. Generation efficiency of diffusion model: Diffusion technology has high computational complexity in the multi-step denoising process, which makes it difficult to balance efficiency and accuracy in practical applications.

[0014] 3. Lack of joint optimization of brain 3D reconstruction and image edge information: Existing solutions often handle modeling and segmentation tasks independently, and are unable to simultaneously take into account the spatial consistency and boundary accuracy of brain images in the generative model.

[0015] Based on the above problems, this paper combines 3D Gaussian model and diffusion technology to propose an innovative method for low-dose brain PET images. By introducing a multi-loss function optimization mechanism, integrating voxel, edge and rendering characteristics, the quality and detail expression ability of three-dimensional brain images under low-dose conditions are significantly improved. Summary of the invention

[0016] The purpose of the present invention is to provide a 3D Gaussian model generation and rendering method for low-dose PET image processing to solve several challenges encountered by existing low-dose PET imaging technology in clinical applications, including decreased image resolution, high noise and artifacts, insufficient three-dimensional modeling accuracy, and insufficient utilization of low-dose and high-dose data correlation.

[0017] In the 3D Gaussian model generation and rendering method for low-dose PET image processing, the multi-loss function optimization method that combines the 3D Gaussian model and diffusion technology can effectively improve the image's detail expression ability and spatial consistency, and greatly reduce the impact of noise and artifacts. Through innovative voxel clustering, edge optimization and rendering mechanisms, high-precision three-dimensional modeling of low-dose data is achieved, significantly improving the accuracy and efficiency of the model under sparse data conditions. At the same time, it also integrates the prior information correlation of low-dose and high-dose images, optimizes the image reconstruction quality, and meets the clinical needs for high-quality PET imaging under low-dose conditions.

[0018] The above-mentioned 3D Gaussian model generation and rendering method for low-dose PET image processing comprises the following steps:

[0019] S1. inputting a low-dose brain PET image, wherein the brain PET image is composed of a set of voxel points;

[0020] S2. Voxel point clustering to generate simplified Gaussian model ,in, is the index value, which refers to the A simplified Gaussian model;

[0021] Based on the simplified Gaussian model, the Gaussian ellipsoid model is resampled to finally construct the Gaussian distribution model of the brain PET image. ;

[0022] S3, the UNet model receives the brain PET image as input to extract and segment the outer contour of the brain PET image;

[0023] S4. Reconstruct the 3D Gaussian distribution model based on VQ-VAE autoencoder and Stable Diffusion model ;

[0024] S5. By calculating the attenuation characteristics of the interaction between light and voxel point data, the 3D Gaussian distribution model Render as a 3D image.

[0025] As a further improvement of the technical solution, the voxel point includes a spatial position coordinate , normal vector and signal strength ;

[0026] At initialization, the normal vector It is derived from the signal gradient direction of the voxel point and is used to describe the directional characteristics of the local structure of the voxel point in the brain PET image. It is defined as follows:

[0027] ;

[0028] In the formula, is the signal intensity of the voxel point The gradient of is the modulus of the gradient.

[0029] As a further improvement of the technical solution, the voxel point clustering in S2 generates a simplified Gaussian model The steps are as follows:

[0030] S2.1. Set the voxel points All voxel points in are initialized to an unassigned index state;

[0031] S2.2. Randomly select a point from the voxel points that are not assigned an index , assigning it an index value , and push it onto the stack;

[0032] Initialize the stack to ;

[0033] S2.3. Pop the stack vertices from the stack one by one As the current processing point, and its neighboring point set Traverse; for each neighborhood point , if the following conditions are met at the same time, the point is assigned the current index value And push onto the stack to continue spreading:

[0034] Neighborhood points are not indexed;

[0035] Normal vector of the neighboring point Normal vector to the current point The projection of lies in the same triangle on the Gaussian sphere;

[0036] The Euclidean distance of the normal vectors of the neighborhood points satisfies:

[0037] ;

[0038] in, The subdivision level for the Gaussian sphere is The maximum side length when ;

[0039] ;

[0040] S2.4, when the stack is empty, the index diffusion process of the current seed point is completed;

[0041] If there are still voxel points that have not been assigned an index, a new seed point is selected and assigned a new index value. , repeat S2.1-S2.4.

[0042] As a further improvement of the technical solution, constraints are introduced in S2.3, and the constraints include:

[0043] For the two vertices of the triangle whose normal vector projections of two adjacent voxel points happen to be located on the Gaussian sphere, the distance constraint of the normal vector will be added so that the two adjacent voxel points belong to different clusters.

[0044] As a further improvement of the present technical solution, the UNet model in S3 is a pre-trained model, which has established a deep characterization capability for multi-scale features by learning on a large number of annotated data sets, and is used to identify key spatial patterns and boundary features in brain PET images;

[0045] The UNet model extracts features layer by layer and fuses multi-scale information through skip connections;

[0046] The binary image output by the UNet model clearly marks the boundaries of the target brain contour. , used for subsequent three-dimensional reconstruction and morphological analysis.

[0047] As a further improvement of the technical solution, the 3D Gaussian distribution model in S4 The reconstruction steps are as follows:

[0048] S4.1. Use VQ-VAE autoencoder to efficiently encode and quantize sparse point cloud data of low-dose brain PET images, where:

[0049] In the encoding stage, the preprocessed Gaussian distribution model Use an encoder consisting of sparse transposed convolutions , extract high-dimensional feature representation ;

[0050] In the quantification phase, Input vector quantization layer to get the quantized representation of the embedding space quantize , the quantized representation It has a high compression rate and retains the important features of low-dose images;

[0051] After passing the Stable Diffusion model, we enter the decoding stage, and the denoised potential samples Input to decoder D to reconstruct a new 3D Gaussian distribution model :

[0052] ;

[0053] S4.2. Use the Stable Diffusion model to represent the quantized embedding Perform diffusion and denoising;

[0054] In the diffusion stage, Add Gaussian noise , so that the data distribution gradually approaches the standard Gaussian distribution and generates noise samples:

[0055] ;

[0056] in, and is the noise control coefficient; express It obeys the standard normal distribution, that is, a normal distribution with a mean of 0 and a variance of 1. In other words, is from The values ​​randomly sampled from this distribution; t represents the time step, which is used to represent different stages in the denoising process. In the diffusion model, the noise evolves over time, and t represents the time index of the current processing.

[0057] In the denoising process, the denoising network , for the noise sample Perform reverse diffusion denoising to generate new latent space samples :

[0058] ;

[0059] S4.3, the Stable Diffusion model is trained by minimizing the mean square error between the predicted noise and the real noise:

[0060] ;in, represents the loss function, which is the mean square error (MSE) loss during diffusion model training, and its purpose is to minimize the denoising network Predicted noise The error between Represents a random variable Here, it is used to represent the mathematical expectation of all possible The average is taken to measure the overall loss.

[0061] As a further improvement of the technical solution, the calculation steps of the attenuation characteristics in S5 are as follows:

[0062] S5.1. Light intensity According to Lambert-Beer's law, it is defined as:

[0063] ;

[0064] Among them, R represents the path of the light, which is used to calculate the attenuation of the light in the medium; is the initial intensity of the light; is the total number of voxel points that the light passes through; For the The attenuation coefficient of an individual pixel is defined as: ; For light to pass through The distance between individual pixels depends on the spatial coordinates , in fact, R refers to the entire light, and It represents a specific point on the light. Here, i is also used as a subscript index to represent a position in the light, which is consistent with the indexing method of the Gaussian model or normal vector.

[0065] S5.2. Calculate the ray attenuation effect through multiple perspective projections to generate high-precision 3D images that meet boundary conditions ;

[0066] The loss of each voxel point is defined as:

[0067] ;

[0068] in, is the reference signal intensity of the corresponding voxel point in the high-dose brain PET image; is the direction vector of the corresponding voxel point in the high-dose brain PET image; is the weight of the directional error, which is used to control the influence of directional information in the total loss; Represents the loss function for each voxel point. The loss function consists of two parts: the first Measuring signal strength With reference signal strength The mean square error between Measure the normal vector Direction with reference The error between Weighting is performed to control the impact of directional information on the total loss.

[0069] I represents the set of voxel points, that is, the set of all voxel point indices considered, in other words, the entire image, and these points are accumulated and summed in the loss calculation.

[0070] As a further improvement of this technical solution, the boundary contour Enter as boundary conditions;

[0071] For each voxel point, check whether it is within the boundary conditions;

[0072] Among them, the definition , find the nearest boundary point of the voxel through the nearest neighbor search , calculate the distance between the two:

[0073] ;

[0074] When the distance exceeds the serious deviation boundary, it is removed directly.

[0075] As a further improvement of this technical solution, the voxel points that have not been clustered are marked as isolated points and assigned special index values. The final output includes multiple cluster sets:

[0076] ;

[0077] And the set of isolated points:

[0078] ;

[0079] For the isolated point set , that is, the set of all voxel points considered to be isolated, each isolated point is represented by Indicates that we first search for its adjacent cluster set , which contains different clusters, each with Indicates that k takes values ​​from 1 to above, where k represents the total number of adjacent clusters, that is, the number of adjacent clusters found, and the normal vectors of adjacent clusters are represented by the weighted average of the normal vectors of the voxel points within the cluster:

[0080] ;

[0081] in, For a specific adjacent cluster; is the weight coefficient, used for weighted average calculation; is the normal vector of voxel point p; For clustering The normal vector of represents the weighted average of the normal vectors of the voxel points in the cluster.

[0082] Next, according to the normal vector of the isolated point The Euclidean distance between the normal vectors of adjacent clusters defines the matching score:

[0083] ;

[0084] And merge the isolated points into the cluster with the smallest score ,in:

[0085] ;

[0086] Target Clustering The normal vector is then updated as:

[0087] ;

[0088] in, is the weight of the isolated point. For isolated points that have not found adjacent clusters, they are classified into a special processing set .

[0089] As a further improvement of this technical solution, for the non-regular clusters in the clustering, first calculate the clustering The center point And the covariance matrix:

[0090] ;

[0091] The regularity of clustering is calculated by using the two largest eigenvalues ​​through the eigenvalue decomposition of the covariance matrix , expressed as a matrix The ratio of the two largest eigenvalues ​​here (regularity measure) reflects whether the shape of the cluster is regular; if the ratio is high, it indicates that the cluster is significantly elongated in a certain direction, showing non-regular characteristics;

[0092] Cluster Split into two sub-clusters along the principal axis corresponding to the maximum eigenvalue, with the splitting plane passing through the cluster center and perpendicular to the principal axis;

[0093] After cluster optimization, the error metric function is used to determine the representative voxel point of each cluster. The overall error metric function is defined as:

[0094] ;

[0095] in, Indicates the point The error metric function calculated is used to optimize the features of the clustered representative voxel points. is based on the class C to which p belongs, is a point in class C, and the overall error is based on Point loss Multiply by weight Got it.

[0096] Among them, r is the position, n is the normal vector, and e is the intensity; Transpose the matrix.

[0097] Further expanded to:

[0098] ;

[0099] Among them, the weight Defined as:

[0100] ;

[0101] Among them, h is an important parameter that controls the distance attenuation speed; r is the position of point p; is a point in cluster C where p is located location.

[0102] The gradient of the error metric function is:

[0103] ;

[0104] in, Represents the gradient of the error metric function with respect to the normal vector n, which is used for iterative update during the optimization process to obtain a more reasonable normal vector n.

[0105] By minimizing the error metric function, the normal vector of the cluster representative voxel point is obtained , and intensity value e, where:

[0106] ;

[0107] Clusters represent the locations of voxel points is the weighted center of all voxel points in the cluster:

[0108] .

[0109] Compared with the prior art, the present invention has the following beneficial effects:

[0110] 1. By combining the multi-loss function optimization mechanism of 3D Gaussian model and diffusion technology, the noise and artifacts in the image are significantly reduced and the image resolution is improved under low-dose brain PET imaging conditions. In particular, the details of the complex structure of the brain are more clearly expressed, providing reliable imaging support for the clinical diagnosis of neurological diseases (such as Alzheimer's disease, Parkinson's disease, etc.).

[0111] In addition, the voxel clustering optimization and three-dimensional model generation method can accurately express the details of brain anatomical structures under sparse data conditions, especially in the modeling of brain edges and small anatomical structures, solving the problems of insufficient spatial consistency and boundary accuracy in three-dimensional reconstruction of existing methods.

[0112] 2. By effectively utilizing the prior information correlation between low-dose and high-dose brain PET images, high-quality brain PET images can be generated while reducing the dose of radioactive tracers, providing safer diagnostic conditions for children, the elderly, and patients with neurological diseases who are undergoing long-term follow-up.

[0113] 3. By introducing an efficient 3D Gaussian model clustering method and optimizing the design of the diffusion model denoising process, the speed and accuracy of brain 3D modeling are significantly improved, solving the bottleneck of existing technology in processing efficiency.

[0114] 4. By integrating brain voxels, edges, and rendering characteristics and constructing a multi-dimensional loss function, the coordinated optimization of brain 3D modeling and image edge information is achieved, making the generated brain PET images more reliable in terms of spatial consistency, detail expression, and boundary accuracy, providing all-round support for clinical diagnosis and treatment decisions. BRIEF DESCRIPTION OF THE DRAWINGS

[0115] Figure 1 A schematic diagram of a process for constructing a 3D Gaussian ellipsoid model based on low-dose brain PET images provided by the present invention.

[0116] Figure 2 A schematic diagram of a process for constructing brain contour information based on low-dose brain PET images provided by the present invention.

[0117] Figure 3 A schematic diagram of the process of generating a 3D brain Gaussian model and voxel rendering from a 3D Gaussian ellipsoid model and a brain outer contour provided by the present invention. DETAILED DESCRIPTION

[0118] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0119] This example provides a 3D Gaussian model generation and rendering method for low-dose brain PET image processing, which is mainly used to improve the quality of low-dose brain PET images and the accuracy of three-dimensional modeling, while reducing the risk of radiation exposure to patients. The method includes four processes: 3D Gaussian ellipsoid model construction, contour information extraction and segmentation, generation of 3D brain Gaussian model and voxel rendering:

[0120] Among them, see Figure 1 As shown, the contents of the 3D Gaussian ellipsoid model are as follows:

[0121] Discrete voxel point set: Input low-dose brain PET image ,in Represents a set of voxel points, each voxel point includes : spatial position coordinates; : normal vector; : Signal strength. At initialization, the normal vector From the voxel point The signal gradient direction is derived to describe the directional characteristics of the local structure of the voxel point in the brain PET image. Its definition is as follows:

[0122] ;

[0123] in: Voxel point Signal strength The gradient of is the modulus of the gradient, which is used to normalize the normal vector so that it represents direction rather than intensity. It is usually approximated by the difference in signal intensity in the local neighborhood of the voxel point.

[0124] It should be noted that the index value Acts as an index to identify a point, normal vector or Gaussian model. Therefore, it can be understood that all They are all subscript indexes, and their functions are the same, but they refer to specific objects in different scenarios, such as Gaussian models, points, or normal vectors.

[0125] Clustering to generate Gaussian model: This method realizes the clustering of voxel points based on index diffusion technology, which mainly includes seed point selection, index diffusion, constraint application and isolated point processing. First, all voxel points Initialized to an unassigned index state, where is the position of the voxel point, is the normal vector, is the signal intensity. Then, a point is randomly selected from the voxel points that have not been assigned an index. , assigning it an index value , and push it onto the stack, initializing the stack to During the diffusion process, the stack vertices are popped out from the stack one by one As the current processing point, and its neighboring point set For each neighborhood point , if the following conditions are met at the same time, the point is assigned the current index value And push onto the stack to continue spreading:

[0126] Neighborhood points are not indexed;

[0127] Normal vector of the neighboring point Normal vector to the current point The projection of lies in the same triangle on the Gaussian sphere;

[0128] The Euclidean distance of the normal vectors of the neighborhood points satisfies:

[0129] ;

[0130] in:

[0131] , the subdivision level of the Gaussian sphere is When the stack is empty, the index diffusion process of the current seed point is completed. If there are still voxel points that have not been assigned an index, a new seed point is selected and assigned a new index value , repeat the above process until all points are processed.

[0132] In order to avoid incorrect clustering, this method introduces constraints in the diffusion process. If the normal vector projections of two adjacent voxel points are exactly located at the two vertices of the triangle on the Gaussian sphere, in order to avoid misclassification, the distance constraint of the normal vector will be added to ensure that these points belong to different clusters.

[0133] In addition, for points that fail to meet the clustering conditions, that is, points whose neighboring points do not meet the above projection and distance constraints, they are marked as isolated points and assigned special index values ​​(such as -1). The final output includes multiple cluster sets:

[0134] ;

[0135] And the set of isolated points:

[0136] ;

[0137] For voxel points marked as isolated points during the clustering process, a merging strategy based on the weighted normal vectors of adjacent clusters is adopted. , first search its adjacent cluster set , and the normal vectors of adjacent clusters are represented by the weighted average of the normal vectors of the voxel points within the cluster:

[0138] ;

[0139] Next, according to the normal vector of the isolated point The Euclidean distance between the normal vectors of adjacent clusters defines the matching score:

[0140] ;

[0141] And merge the isolated points into the cluster with the smallest score ,in:

[0142] ;

[0143] Target Clustering The normal vector of is then updated as:

[0144] ;

[0145] in, is the weight of the isolated point. For isolated points that have not found adjacent clusters, they are classified into a special processing set ,This method effectively improves the classification accuracy of isolated points and enhances the ,consistency and robustness of the overall clustering results by combining the ,characteristics of adjacent clusters.

[0146] For the non-regular clusters existing in the initial clustering, first calculate the clustering The center point And the covariance matrix:

[0147] ;

[0148] The regularity of clustering is calculated by using the two largest eigenvalues ​​through the eigenvalue decomposition of the covariance matrix If the ratio is high, it indicates that the cluster is significantly elongated in a certain direction, showing irregular characteristics. Irregular clusters usually appear elongated or unevenly distributed, which is difficult to meet the ideal spherical or compact distribution requirements. The cluster is split into two sub-clusters along the principal axis corresponding to the maximum eigenvalue, and the splitting plane passes through the cluster center and is perpendicular to the principal axis. After cluster optimization, the error metric function is used to determine the representative voxel of each cluster. The overall error metric function is defined as:

[0149] ;

[0150] Further expanded to:

[0151] ;

[0152] Among them, the weight Defined as:

[0153] ;

[0154] Among them, h is an important parameter that controls the distance attenuation speed, which determines the point The influence range of r is the key adjustment factor in the clustering or weight distribution process. Here, h can be taken as a fixed value, or the bandwidth parameter h can be dynamically adjusted according to the distribution density of local points.

[0155] The gradient of the error metric function is:

[0156] ;

[0157] By minimizing the error metric function, the normal vector of the cluster representative voxel is obtained , and intensity value e, where:

[0158] ;

[0159] Clusters represent the locations of voxels is the weighted center of all voxel points in the cluster:

[0160] ;

[0161] After the above optimization and simplification process, a simplified Gaussian model is generated Based on the simplified Gaussian model, the Gaussian ellipsoid model is resampled G(), and finally the Gaussian distribution model of the image is constructed .

[0162] Among them, see Figure 2 As shown, the contents of contour information extraction and segmentation are as follows:

[0163] Using the pre-trained UNet model Extract and segment the outer contour of a 3D brain image (i.e., brain PET image). The pre-trained model has established a deep representation capability for multi-scale features by learning on a large number of annotated data sets, and can efficiently identify key spatial patterns and boundary features in images. In the specific implementation process, the UNet network receives 3D brain PET images as input, extracts features layer by layer, and fuses multi-scale information through jump connections. The binary image output by the model clearly marks the boundaries of the target brain outer contour. , which helps in subsequent accurate three-dimensional reconstruction and morphological analysis.

[0164] = .

[0165] The contents of generating the 3D brain Gaussian model are as follows:

[0166] This is a 3D Gaussian distribution brain model generation method based on VQ-VAE autoencoder and Stable Diffusion model, which is suitable for high-precision reconstruction of three-dimensional brain PET medical images.

[0167] Encoding and quantization: Use VQ-VAE (Vector Quantized-Variational AutoEncoder) to efficiently encode and quantize the sparse point cloud data of low-dose brain PET images. VQ-VAE is divided into three steps: In the encoding stage, the preprocessed Gaussian distribution model Use an encoder consisting of sparse transposed convolutions , extract high-dimensional feature representation The next step is the quantization phase. Input vector quantization layer to get the quantized representation of the embedding space quantize , the quantized representation It has a high compression rate and retains the important features of low-dose images. After passing the Stable Diffusion model, it enters the decoding stage. The denoised latent samples Input to decoder D to reconstruct a new 3D Gaussian distribution model :

[0168] ;

[0169] Diffusion denoising: Using the Stable Diffusion model to represent the quantized embedding Diffusion and denoising are performed. In the diffusion stage, Add Gaussian noise , so that the data distribution gradually approaches the standard Gaussian distribution and generates noise samples:

[0170] ;

[0171] in, and is the noise control factor.

[0172] In the denoising process, the denoising network , for the noise sample Perform reverse diffusion denoising to generate new latent space samples :

[0173] ;

[0174] The Stable Diffusion model is trained by minimizing the mean square error between the predicted noise and the true noise:

[0175] .

[0176] See also Figure 3 As shown, the contents of voxel rendering are as follows:

[0177] The goal of voxel rendering is to transform a 3D Gaussian distribution model into a Rendered into 3D images, the generated images are guaranteed to match the high-dose images with high accuracy in signal intensity, boundary characteristics and optical consistency.

[0178] During rendering, based on the position of each voxel point , signal strength , and the normal vector Calculates the attenuation of the light. The intensity of the light According to Lambert-Beer's law, it is defined as:

[0179] ;

[0180] in: : The initial intensity of the light, determined by the intensity value at the decay point : The total number of voxels that the ray passes through, : No. The attenuation coefficient of a voxel is defined as: : The light passes through the The distance between voxels depends on . Calculate the ray attenuation effect through multiple perspective projections to generate high-precision 3D images that meet boundary conditions .

[0181] In order to optimize the rendering results and make the generated images consistent with high-dose PET images in multiple dimensions, a voxel-based rendering loss function is defined. The loss of each voxel point is defined as:

[0182] ;

[0183] in: : Reference signal intensity of the corresponding voxel in high-dose PET images. : Direction vector of the corresponding voxel in the high-dose image. : The weight of the direction error, used to control the influence of the direction information in the total loss.

[0184] Boundary condition control: During the optimization process, the input boundary contour As constraint input, the matching effect between the generated 3D Gaussian ellipsoid and the real image is accurately controlled. If points outside the boundary appear, a pruning strategy is adopted to improve the accuracy and efficiency of the model.

[0185] For each voxel point , check whether it is within the boundary. For points outside the boundary, it is defined as:

[0186] if , then find the voxel point through the nearest neighbor search The nearest boundary point , calculate the distance between the two:

[0187] ;

[0188] Pruning strategy: When the distance exceeds a certain value (that is, the point deviates seriously from the boundary), remove it directly:

[0189] Remove

[0190] in, is the pruning distance threshold.

[0191] Used to penalize voxel points outside the boundary, defined as:

[0192] ;

[0193] The spatial position of the nearest boundary point. The penalty term forces points outside the boundary to move inside the boundary by constraining the distance between the voxel point and the boundary.

[0194] The total loss is defined as:

[0195] ;

[0196] Weight Usually it needs to be adjusted according to the boundary matching accuracy requirements.

[0197] Through these calculations, the present invention can transform the generated 3D Gaussian distribution model into a 3D image with high optical and morphological consistency, which is suitable for reconstruction of brain PET images.

[0198] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments. The above embodiments and descriptions are only preferred examples of the present invention and are not intended to limit the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A 3D Gaussian model generation and rendering method for low-dose PET image processing, characterized in that: The method comprises the following steps: S1. inputting a low-dose brain PET image, wherein the brain PET image is composed of a set of voxel points; S2. Voxel point clustering to generate simplified Gaussian model ; Where i is the index value; Based on the simplified Gaussian model, the Gaussian ellipsoid model is resampled to finally construct the Gaussian distribution model of the brain PET image. ; S3, the UNet model receives the brain PET image as input to extract and segment the outer contour of the brain PET image; S4. Reconstruct the 3D Gaussian distribution model based on VQ-VAE autoencoder and Stable Diffusion model ; S4.

1. Use VQ-VAE autoencoder to efficiently encode and quantize sparse point cloud data of low-dose brain PET images, where: In the encoding stage, the preprocessed Gaussian distribution model Use encoder E composed of sparse transposed convolution to extract high-dimensional feature representation ; In the quantification phase, Input vector quantization layer to get the quantized representation of the embedding space quantize , the quantized representation It has a high compression rate and retains the important features of low-dose images; After passing the Stable Diffusion model, we enter the decoding stage, and the denoised potential samples Input to decoder D to reconstruct a new 3D Gaussian distribution model : ； S5. By calculating the attenuation characteristics of the interaction between light and voxel point data, the 3D Gaussian distribution model Render as a 3D image.

2. The 3D Gaussian model generation and rendering method for low-dose PET image processing according to claim 1, characterized in that: The voxel point includes a spatial position coordinate , normal vector and signal strength ; At initialization, the normal vector It is derived from the signal gradient direction of the voxel point and is used to describe the directional characteristics of the local structure of the voxel point in the brain PET image. It is defined as follows: ； In the formula, is the signal intensity of the voxel point The gradient of is the modulus of the gradient.

3. The 3D Gaussian model generation and rendering method for low-dose PET image processing according to claim 2, characterized in that: The voxel point clustering in S2 generates a simplified Gaussian model The steps are as follows: S2.

1. Set the voxel points All voxel points in are initialized to an unassigned index state; S2.

2. Randomly select a point from the voxel points that are not assigned an index , assign it index value i and push it onto the stack; Initialize the stack to ; S2.

3. Pop the stack vertices from the stack one by one As the current processing point, and its neighboring point set Traverse; for each neighborhood point , if the following conditions are met at the same time, the point is assigned the current index value i and pushed into the stack to continue spreading: Neighborhood points are not indexed; Normal vector of the neighboring point Normal vector to the current point The projection of lies in the same triangle on the Gaussian sphere; The Euclidean distance of the normal vectors of the neighborhood points satisfies: ； in, The subdivision level for the Gaussian sphere is The maximum side length when ; ； S2.4, when the stack is empty, the index diffusion process of the current seed point is completed; If there are still voxel points that have not been assigned an index, a new seed point is selected and assigned a new index value. , repeat S2.1-S2.

4.

4. The 3D Gaussian model generation and rendering method for low-dose PET image processing according to claim 3, characterized in that: S2.3 introduces constraints, which include: For the two vertices of the triangle whose normal vector projections of two adjacent voxel points happen to be located on the Gaussian sphere, the distance constraint of the normal vector will be added so that the two adjacent voxel points belong to different clusters.

5. The 3D Gaussian model generation and rendering method for low-dose PET image processing according to claim 2, characterized in that: The UNet model in S3 is a pre-trained model, which has established a deep representation capability for multi-scale features by learning on a large number of annotated data sets, and is used to identify key spatial patterns and boundary features in brain PET images; The UNet model extracts features layer by layer and fuses multi-scale information through skip connections; The binary image output by the UNet model clearly marks the boundaries of the target brain contour. , used for subsequent three-dimensional reconstruction and morphological analysis.

6. The 3D Gaussian model generation and rendering method for low-dose PET image processing according to claim 1, characterized in that: The S4 further comprises the following steps: S4.

2. Use the Stable Diffusion model to represent the quantized embedding Perform diffusion and denoising; In the diffusion stage, Add Gaussian noise , so that the data distribution gradually approaches the standard Gaussian distribution and generates noise samples: ； in, and is the noise control coefficient; express It obeys the standard normal distribution; t represents the time step; In the denoising process, the denoising network , for the noise sample Perform reverse diffusion denoising to generate new latent space samples : ； S4.3, the Stable Diffusion model is trained by minimizing the mean square error between the predicted noise and the real noise: ； in, represents the loss function; Represents a random variable The mathematical expectation of .

7. The 3D Gaussian model generation and rendering method for low-dose PET image processing according to claim 5, characterized in that: The calculation steps of the attenuation characteristics in S5 are as follows: S5.

1. Light intensity According to Lambert-Beer's law, it is defined as: ； Among them, R represents the path of the light, which is used to calculate the attenuation of the light in the medium; is the initial intensity of the light; is the total number of voxel points that the light passes through; is the attenuation coefficient of the i-th voxel point, defined as: ; is the distance that the light passes through the i-th voxel point, which depends on the spatial position coordinates ; S5.

2. Calculate the ray attenuation effect through multiple perspective projections to generate high-precision 3D images that meet boundary conditions ; The loss of each voxel point is defined as: ； in, is the reference signal intensity of the corresponding voxel point in the high-dose brain PET image; is the direction vector of the corresponding voxel point in the high-dose brain PET image; is the weight of the directional error, which is used to control the influence of directional information in the total loss; represents the loss function of each voxel point; I represents the set of voxel points.

8. The 3D Gaussian model generation and rendering method for low-dose PET image processing according to claim 7, characterized in that: The boundary outline Enter as boundary conditions; For each voxel point, check whether it is within the boundary conditions; Among them, the definition , find the nearest boundary point of the voxel through the nearest neighbor search , calculate the distance between the two: ； When the distance exceeds the serious deviation boundary, it is removed directly.

9. The 3D Gaussian model generation and rendering method for low-dose PET image processing according to claim 4, characterized in that: The voxel points that have not been clustered are marked as isolated points and assigned special index values. The final output includes multiple cluster sets: ； And the set of isolated points: ； For the isolated point set , first search its adjacent cluster set , which contains different clusters, each with Represented by, and the normal vectors of adjacent clusters are represented by the weighted average of the normal vectors of the voxel points within the cluster: ； in, For a specific adjacent cluster; is the weight coefficient; is the normal vector of voxel point p; For clustering The normal vector of Next, according to the normal vector of the isolated point The Euclidean distance between the normal vectors of adjacent clusters defines the matching score: ； And merge the isolated points into the cluster with the smallest score ,in: ； Target Clustering The normal vector is then updated as: ； in, is the weight of the isolated point. For isolated points that have not found adjacent clusters, they are classified into a special processing set .

10. The 3D Gaussian model generation and rendering method for low-dose PET image processing according to claim 4, characterized in that: For non-regular clusters in clustering, first calculate the clustering The center point And the covariance matrix: ； The regularity of clustering is calculated by using the two largest eigenvalues ​​through the eigenvalue decomposition of the covariance matrix ; Cluster Split into two sub-clusters along the principal axis corresponding to the maximum eigenvalue, with the splitting plane passing through the cluster center and perpendicular to the principal axis; After cluster optimization, the error metric function is used to determine the representative voxel point of each cluster. The overall error metric function is defined as: ； in, Indicates the point The error metric function calculated; r is the position; n is the normal vector; e is the intensity; Transpose the matrix; Further expanded to: ； Among them, the weight Defined as: ； Among them, h is an important parameter that controls the distance attenuation speed; The gradient of the error metric function is: ； in, represents the gradient of the error metric function with respect to the normal vector n; By minimizing the error metric function, the normal vector and intensity value e of the cluster representative voxel point are obtained, where: ； Clusters represent the locations of voxel points is the weighted center of all voxel points in the cluster: 。

Images