Heart three-dimensional reconstruction method, system and program product based on improved GRAF algorithm

Abstract

The invention discloses a heart three-dimensional reconstruction method, system and program product based on an improved GRAF algorithm. The three-dimensional reconstruction method comprises the following steps: acquiring and preprocessing to generate digital reconstruction radiation images of different patients, different contrast types and normal and abnormal heart anatomical structures; constructing and training an improved GRAF algorithm model; and inputting a small amount of CT image data to the trained GRAF algorithm model to render continuous image frames for three-dimensional reconstruction, and the like. According to the method, under limited CT image input, accurate reconstruction of a heart three-dimensional structure can be realized through the improved GRAF algorithm model obtained by training based on a method of combining data enhancement transformation and multiple training heads.

Description

Technical Field

[0001] The present invention relates to the technical field of robot-assisted surgery systems, and more specifically, to a method, system and program product for three-dimensional reconstruction of the heart based on an improved GRAF algorithm. Background Art

[0002] In the application field of cardiac surgical robots, accurate three-dimensional reconstruction of the heart before surgery is a crucial step. Traditional technologies usually rely on computed tomography (CT) to obtain detailed structural data of the heart. Through CT scanning, a series of two-dimensional images can be obtained, and then a three-dimensional model can be reconstructed through image processing algorithms. However, traditional three-dimensional reconstruction methods rely on continuous CT image input, and continuous CT scanning is not only time-consuming, but also accompanied by a higher risk of radiation exposure to patients, which poses a potential threat to human health. Therefore, it is crucial to seek a high-precision method to reconstruct a three-dimensional model of the heart using low-radiation non-continuous CT images.

[0003] GRAF (Generative Radiance Field) uses an adversarial framework to train a generative model of a radiation field, which enables the synthesis of high-resolution 3D-aware images by introducing a multi-scale patch-based discriminator (see K Schwarz, Y Liao, M Niemeyer and A Geiger, "GRAF: Generative Radiance Fields for 3D-Aware Image Synthesis", NeurIPS 2020). However, medical imaging systems are essentially significantly different from conventional photographic equipment, and the medical field has more stringent requirements on the accuracy of generative models. These factors lead to challenges in the application of traditional neural radiation field models and their variants when applied to three-dimensional reconstruction of CT images. In order to solve these problems, it is necessary to develop more advanced cardiac three-dimensional reconstruction methods based on the improved GRAF algorithm to overcome the above limitations and further improve the safety and effectiveness of preoperative three-dimensional reconstruction of the heart during cardiac surgery. Summary of the invention

[0004] In order to solve the problem of how to reduce CT data input during CT reconstruction of the heart, the present invention proposes a heart three-dimensional reconstruction method, system and program product based on an improved GRAF algorithm, which can realize accurate reconstruction of the three-dimensional structure of the heart through advanced deep learning technology under limited CT data input.

[0005] To achieve the above objectives, in a first aspect, the present invention provides a method for three-dimensional reconstruction of a heart based on an improved GRAF algorithm, characterized by comprising:

[0006] Step S1, collecting information on cardiac CT scan data: collecting images containing different patients, different contrast types, and normal and abnormal cardiac anatomical structures;

[0007] Step S2, performing comprehensive and digitized processing including denoising, standardization and digitally reconstructed radiographic image generation on the original CT image data, and performing a number of data enhancement transformations on the processed digitally reconstructed radiographic images to construct a training set and a test set;

[0008] Step S3, constructing and training an improved GRAF algorithm model: the improved GRAF algorithm model includes a generator network and a discriminator network based on the generative radiation field principle; the generator network samples rays from the digitally reconstructed radiographic image obtained in step S2 by ray sampling, and uses positioning coding to map the 3D position and observation direction to a high-dimensional feature space, and further uses shape coding and appearance coding to determine the shape and appearance of the object, and then predicts the density and color of any point in the scene through a fully connected layer, and finally synthesizes continuous image frames through volume rendering technology; the discriminator network applies multiple discriminant heads corresponding to different data enhancement transformations, each discriminant head focuses on extracting features under a specific data enhancement transformation and making predictions, and optimizes model parameters by combining the output of the discriminant head and the result of data enhancement through a comprehensive loss function; the training is iterated until the loss function value of the discriminator network tends to be stable or reaches a predetermined number of iterations;

[0009] Step S4: Use the trained GRAF algorithm model and non-continuous cardiac CT images to perform three-dimensional reconstruction of the cardiac structure.

[0010] The present invention trains an improved GRAF algorithm model based on feature discrimination of data enhancement transformation by using non-continuous CT image data of a specific type of patient, thereby obtaining a method and system that can well generate continuous CT images from limited CT images to complete three-dimensional reconstruction of the heart. Therefore, it not only significantly reduces the radiation dose and potential harm to the human body, but also improves the efficiency and accuracy of image reconstruction, making the entire reconstruction process more efficient and economical.

[0011] Furthermore, in step S1, patient data of different ages, genders, weights and races are collected, including X-ray images of normal hearts and abnormal hearts, and different concentrations of iodinated contrast agents are used during sampling to obtain images with different contrasts, so as to increase the generalization ability of the data set, and good results can be achieved by using a small amount of data for training.

[0012] Furthermore, in step S2, for each original CT image, a denoising algorithm is first used to reduce random noise in the image, and then the image contrast is enhanced. The image is again standardized and segmented, and the heart area is extracted using threshold segmentation, region growing or deep learning methods. The image segmentation results are further used to generate digitally reconstructed radiographic images (DRRs), and finally the generated DRRs are resampled to have a uniform resolution and size; data enhancement technology is applied to the sampled DRRs to obtain diversified DRRs; and the enhanced DRRs are organized into training sets and test sets.

[0013] Furthermore, the data enhancement technology includes at least two of flipping, rotation, scaling, and translation, so as to increase the diversity of DRRs.

[0014] Furthermore, the total loss function calculation formula of the generator and discriminator network is:

[0015]

[0016] In the above formula, L d It is used to measure the discriminant head for the real data P p And its enhanced version P p,k The loss of the difference in predicted values, L r For the generator G θ and the discriminator The adversarial loss between them, k is the number of the discriminant head, n is the number of discriminant heads, λ is used to measure the weights between different losses, and θ is the random weight of the generator or the pre-trained weight parameter; is the kth discriminator parameter.

[0017] Furthermore, in step S4, firstly, a small number or a single cardiac CT scan image is selected as the input of the improved GRAF algorithm model, and the input image covers the required anatomical structure and contrast type; secondly, the input CT image is preprocessed to match the data distribution during training; the trained generator network is loaded again, and the generator is used to generate continuous image frames from the latent space vector; finally, a three-dimensional reconstruction algorithm is used to reconstruct a cardiac 3D image from the continuous image frames.

[0018] In a second aspect, the present invention provides a heart three-dimensional reconstruction system based on an improved GRAF algorithm, characterized in that it is used to implement the heart three-dimensional reconstruction method as described above, comprising:

[0019] An image acquisition module that acquires images of different patients, different contrast types, and normal and abnormal cardiac anatomy;

[0020] X-ray image synthesis and data processing module, which performs synthesis and data processing on the original CT data, including denoising, standardization and digital reconstruction of radiological images;

[0021] The improved GRAF algorithm model includes a generator module and a discriminator module based on the generative radiation field principle; the generator module includes ray sampling, scene representation, density prediction network, color prediction network, and volume rendering submodules; the discriminator module includes image block extraction, a discriminator backbone network, and multiple data enhancement transformation discriminant heads, and optimizes model parameters by combining the output of the discriminant head and the result of data enhancement through a comprehensive loss function; the GRAF algorithm model with optimized model parameters is used to generate continuous image frames from a small number of images or a single view;

[0022] The three-dimensional reconstruction module uses a three-dimensional reconstruction algorithm to reconstruct a 3D image of the heart from continuous image frames.

[0023] Furthermore, the data enhancement transformation discriminant head includes a flip discriminant head, a rotation discriminant head and a scaling discriminant head.

[0024] Furthermore, the total loss function calculation formula of the generator and discriminator network is:

[0025]

[0026] In the above formula, L d It is used to measure the discriminant head for the real data P p And its enhanced version P p,k The loss of the difference in predicted values, L r For the generator G θ and the discriminator The adversarial loss between them, k is the number of the discriminant head, n is the number of discriminant heads, λ is used to measure the weights between different losses, and θ is the random weight of the generator or the pre-trained weight parameter; is the kth discriminator parameter.

[0027] In the last aspect, the present invention provides a computer program product, characterized in that when the computer program product is run on a computer, the computer is enabled to execute the three-dimensional heart reconstruction method based on the improved GRAF algorithm as described above.

[0028] Compared with the prior art, the present invention has the following technical effects:

[0029] (1) The present invention introduces the generative radiation field principle into the problem of cardiac CT reconstruction. The improved GRAF algorithm model based on feature discrimination of data enhancement transformation is trained by using non-continuous CT image data of a specific type of patient. During the model application stage, continuous frame images can be obtained through a small number or a single CT image, which effectively solves the disadvantage of long-term radiation exposure during CT image reconstruction.

[0030] (2) The improved GRAF algorithm model of the present invention provides a new discriminator structure, which improves the discriminator training performance in the case of insufficient data by combining data enhancement method and multiple training heads.

[0031] (3) The image reconstruction method adopted by the present invention is based on the principle of generative radiation field. Compared with the traditional reconstruction method, it has the advantages of multi-view consistency, high efficiency and quality of image reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 FIG. 4 is a flowchart of an execution of a method for three-dimensional reconstruction of a heart in one embodiment of the present invention.

[0033] Figure 2 This is a framework diagram of an improved GRAF algorithm model in one embodiment of the present invention.

[0034] Figure 3 FIG. 4 is a framework diagram of a heart three-dimensional reconstruction system in one embodiment of the present invention. DETAILED DESCRIPTION

[0035] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, but they are not intended to limit the present invention.

[0036] In the following detailed description, many specific details are set forth to provide a more thorough understanding of the present invention. However, it is obvious to those skilled in the art that well-known algorithms are not shown in detail to avoid obscuring the subject matter of the present invention.

[0037] In addition, the order of execution of actions, steps, etc. in the devices and methods shown in the claims, specifications and drawings can be implemented in any order as long as there is no special explicit limitation on the order and the output of the previous processing is not used in the subsequent processing.

[0038] Example 1

[0039] See also Figure 1 This embodiment provides a method for three-dimensional reconstruction of a heart based on an improved GRAF algorithm, comprising the following steps:

[0040] Step S1, collect information on cardiac CT scan data to construct a training data set for model training, the data set contains images of different patients, different contrast types, and normal and abnormal anatomical structures. More specifically, it includes the following sub-steps:

[0041] a1: Choose a multi-detector CT scanner suitable for cardiac imaging that can provide high resolution and high quality images, such as a high-end CT scanner such as the GE Healthcare Discovery CT750 HD, and ensure that the equipment is properly calibrated and maintained to ensure image quality.

[0042] a2: Select the appropriate tube voltage to obtain sufficient X-ray penetration. For adult cardiac CT, 100-140kVp is usually selected; adjust the tube current to ensure the brightness and clarity of the image. Depending on the patient's body size and the required image quality, select 200-400mA; set the exposure time according to the required image quality and patient condition, usually a single rotation is completed within 1 second; select the appropriate layer thickness to obtain detailed anatomical structure information, which can be set to a layer thickness of 0.5-1.0mm.

[0043] a3: The patient is positioned on the scanning table, and the position of the scanner is adjusted to ensure that the heart area is in the center of the scanning field of view. Start the CT scan, and the device will automatically rotate and collect X-ray images.

[0044] a4: Collect patient data of different ages, genders, weights and races to increase the generalization ability of the data set, including X-ray images of normal and abnormal hearts, and consider using different concentrations of iodinated contrast agents to obtain images with different contrasts. The model in the present invention can be trained with a small amount of data to achieve good results, and the collected patient data can be set to 10 groups.

[0045] Step S2, the collected original X-ray images are integrated and digitized, including denoising, standardization and generation of digitally reconstructed radiographic images (DRRs), and several data enhancement transformations are performed on the processed digitally reconstructed radiographic images to construct training sets and test sets. More specifically, the steps include:

[0046] b1: Use a denoising algorithm on each original X-ray image to reduce random noise in the image, where I d is the denoised image, I r is the original image, (x, y) is the pixel coordinate, (i x ,i y ) is the offset of the Gaussian kernel, and N is the size of the kernel:

[0047]

[0048] b2: Enhance the contrast of the denoised image to make the anatomical structure clearer, using contrast-limited adaptive histogram equalization (CLAHE) or other contrast enhancement techniques. e is the enhanced image, α and β are parameters used to adjust the contrast:

[0049] I e (x,y)=α·I d (x,y)+β

[0050] b3: Standardize the contrast-enhanced image to ensure that the image intensity value distribution is consistent, where I n is the normalized image, I min and I max are the minimum and maximum intensities of the enhanced image respectively:

[0051]

[0052] b4: Segment the standardized image and extract the heart area using threshold segmentation, region growing or deep learning methods, where S is the segmented binary image and T is the threshold:

[0053]

[0054] b5: Generate digitally reconstructed radiographic images (DRRs) using image segmentation results. 3D (x, y, z) is the 3D image data, which represents the volume data of the segmented heart area. δ is used to represent the path of the X-ray, θ is the direction of the ray, t is the position along the ray path, and z is the coordinate along the ray direction:

[0055]

[0056] b6: Resample the generated DRRs to ensure they have uniform resolution and size.

[0057] b7: Apply data augmentation techniques such as flipping, rotation, scaling, and translation to increase the diversity of DRRs.

[0058] b8: Organize the enhanced DRRs into training sets and test sets, and perform validation to ensure the quality of the dataset.

[0059] Step S3: This embodiment constructs an improved GRAF algorithm model. Figure 2 , the model can be effectively trained on small data sets without relying on complex pose information or 3D supervision. This model will be able to reconstruct continuous image frames from a small number of or single view images, providing a new solution for the field of medical imaging. More specifically, it includes:

[0060] c1: A generator module based on the principle of neural radiation field, which includes sub-modules such as ray sampling, scene representation, density prediction network, color prediction network, and volume rendering.

[0061] c1.1: The first step of the generator is ray sampling, which involves sampling rays from the image. It involves sampling rays from the image, and the sampled X-ray beam is defined as follows, where u and s represent the image coordinates and the sampling step size, respectively:

[0062] v=(u,s)

[0063] To numerically integrate the radiation field, we sample N points along each ray r

[0064] c1.2: The second step is scene representation, using localization encoding to map the 3D position x and viewing direction d to a higher-dimensional feature representation. This encoding is applied element-by-element to all three components of x and dd, as follows

[0065] φ(p)=(sin(20πp),cos(20πp),sin(21πp),cos(21πp),sin(22πp),cos(22πp),…)

[0066] 3D position encoding and d r The formula is as follows:

[0067]

[0068]

[0069] Apart from and d r There are two potential additional latent coding conditions, shape code z s and appearance code Z a , which is a set of latent variables sampled from a Gaussian distribution, which are used to determine the shape of the object in the model. These four encodings form a radiation field, and the relationship between the 3D position and viewing direction d mapped to c,σ is:

[0070]

[0071] c1.3: Shape embedding: This network uses a fully connected layer as the network representation, which represents the encoding information of the object shape. The formula is as follows:

[0072]

[0073] c1.4: Density prediction network (density logit): This network uses a fully connected layer as the network representation and is responsible for predicting the density of any point in the scene. The formula is as follows

[0074]

[0075] c1.5: Color prediction network (color logit): This network uses a fully connected layer as the network representation, and predicts the color value based on the density prediction result and the viewing direction. The formula is as follows:

[0076]

[0077] C1.6: Volume rendering: Use alpha compositing to composite color and density along ray r to generate the final pixel color C r :

[0078]

[0079] Where Δ j r is the adjacent sampling point x j and x j+1 The distance between them, exp(-σ j Δ j r) Calculate the light from x j to x j+1 The degree of attenuation along the path.

[0080] c2: Build the discriminator module, which includes image block extraction, discriminator backbone network, loss function, etc.

[0081] c2.1: Image patch extraction

[0082] P=sample(I,θ)

[0083] Where θ = (u, s) is the parameter extracted from the distribution pθ, u is the image coordinate, s is the sampling step, and sample represents bilinear interpolation sampling on image I.

[0084] c2.2: Discriminator backbone network, which uses a pre-trained convolutional neural network to extract the basic features of the image. The formula is as follows:

[0085]

[0086] G b It is a basic feature extraction network, usually a pre-trained convolutional neural network, such as VGG, ResNet, etc. I is the input image, and f is the feature representation of the network output. These features capture the key visual information of the image and can be used for more detailed analysis in the subsequent discriminant head.

[0087] c2.3: Data enhancement transformation, the network applies different data enhancement transformations to increase data diversity. The formula is as follows:

[0088]

[0089] Where k represents different types of enhancement transformations, and the following takes flip, rotation, and scaling enhancement transformations as examples.

[0090] c2.4 Flip Discriminator Head: This discriminator head focuses on extracting features that have been transformed through flip enhancement and making predictions. The formula is as follows: p,1 is the prediction of the flipped discriminator:

[0091]

[0092] c2.5: Rotation Discriminator Head, which focuses on extracting features after rotation enhancement transformation and making predictions. The formula is as follows: p,2 is the prediction of the rotation discriminator:

[0093]

[0094] c2.6: Scale Discriminator Head, which focuses on extracting features after scaling enhancement transformation and making predictions. The formula is as follows: p,3 is the prediction of the scaled discriminative head:

[0095]

[0096] c2.7: Loss function, which combines the output of each discriminant head and the result of data enhancement to calculate the total loss. The formula is as follows:

[0097]

[0098] In the above formula, L d It is used to measure the discriminant head for the real data P p And its enhanced version P p,k The loss of the difference in predicted values, L r For the generator G θ and the discriminator The adversarial loss between them, k is the number of the discriminant head, n is the number of discriminant heads, λ is used to measure the weights between different losses, and θ is the random weight of the generator or the pre-trained weight parameter; is the kth discriminator parameter.

[0099] c3: Train the GRAF model using the preprocessed dataset. During training, the model learns to decouple 3D shape and view direction from 2D images to generate high-fidelity images:

[0100] c3.1: Initialize the generator network G θ , typically using random weights or pre-trained weights. Initialize the discriminator network Usually use random weights or pre-trained weights. Make sure the network parameters are initialized properly to avoid weight values ​​that are too large or too small.

[0101] c3.2: Select the Adam optimizer, which is an adaptive learning rate optimization algorithm suitable for most deep learning tasks. Set the initial learning rate to 0.0002, which is a commonly used starting learning rate that can converge quickly in the early stages of training. During training, adjust the learning rate based on model performance, such as using a learning rate decay or warm-up strategy.

[0102] c3.3: In each iteration, the generator G θ Generate images from random noise or latent space vectors, discriminator Evaluate generated images and real images. The discriminator tries to distinguish between generated images and real images, and the goal of the generator is to maximize the probability that its generated images are mistakenly classified as real by the discriminator. Calculate the loss function of the generator and discriminator, and update the network parameters through backpropagation. Training iterates until the loss function values ​​of the generator and discriminator tend to stabilize or reach a predetermined number of iterations.

[0103] Step S4: After the model training is completed, the trained model is used for reasoning. By inputting a small amount of CT data, the model can render continuous image frames for 3D reconstruction.

[0104] d1: Select a small number or a single CT scan image as the input for model inference. These images should cover the required anatomical structures and contrast types. Perform necessary preprocessing on the input CT images, such as denoising, contrast enhancement, and normalization, to match the data distribution during training, and ensure that the format and size of the input data are consistent with those during training, such as resolution, image depth, etc.

[0105] d2: Load the trained generator network G θ , the network responsible for generating images from the latent space, making sure to load the network weights from the checkpoint saved during training.

[0106] d3: Generates random or predefined latent space vectors for the generator, which will guide the generation of images. θGenerate continuous image frames from the latent space vector. These image frames should cover a 360-degree viewing angle to facilitate subsequent 3D reconstruction. Synthesizing images according to the required viewing angle can be achieved by adjusting the input parameters of the generator.

[0107] d4: Select an appropriate 3D reconstruction algorithm, such as voxel reconstruction, surface reconstruction, or depth estimation, to reconstruct the 3D shape from the continuous image frames, and use the generated image frames as input to integrate into the 3D reconstruction algorithm. Adjust the parameters of the generator and reconstruction algorithm based on the reconstruction results to improve the accuracy and quality of the reconstruction.

[0108] From the above method steps, it can be seen that by training the improved GRAF algorithm model based on feature discrimination of data enhancement transformation with non-continuous CT image data of a specific patient and a specific type, continuous CT images can be well generated from limited CT images to complete the three-dimensional reconstruction of the heart. This method can not only significantly reduce the radiation dose suffered by the patient and reduce the potential harm to the human body, but also improve the efficiency and accuracy of image reconstruction, making the entire reconstruction process more efficient and economical.

[0109] Example 2

[0110] See also Figure 3 This embodiment provides a heart 3D reconstruction system based on an improved GRAF algorithm, which is used to implement the heart 3D reconstruction method as described in Example 1, including:

[0111] An image acquisition module 100 that acquires images of different patients, different contrast types, and normal and abnormal cardiac anatomical structures;

[0112] X-ray image synthesis and digitization processing module 200 performs synthesis and digitization processing on the original CT data, including denoising, standardization and digital reconstruction of radiological images;

[0113] An improved GRAF algorithm model 300 includes a generator module and a discriminator module based on the generative radiation field principle; the generator module includes ray sampling, scene representation, density prediction network, color prediction network, and volume rendering submodules; the discriminator module includes image block extraction, a discriminator backbone network, and multiple data enhancement transformation discriminant heads, and optimizes model parameters by combining the output of the discriminant head and the result of data enhancement through a comprehensive loss function; the GRAF algorithm model after optimizing the model parameters is used to generate continuous image frames from a small number of images or a single view;

[0114] The three-dimensional reconstruction module 400 reconstructs a 3D image of the heart from continuous image frames using a three-dimensional reconstruction algorithm.

[0115] This cardiac three-dimensional reconstruction system based on the improved GRAF algorithm model can achieve accurate reconstruction of the three-dimensional structure of the heart through advanced deep learning technology with limited CT data input.

[0116] The above-mentioned heart three-dimensional reconstruction method can be embodied in the form of a computer program product or a software functional unit. If the above-mentioned heart three-dimensional reconstruction method is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Therefore, the essence of the technical solution or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for an electronic system (which can be a personal computer, a server, or a network system, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), disk or optical disk and other media that can store program codes.

[0117] Those skilled in the art will appreciate that the units, i.e., algorithm steps, of the various examples described in conjunction with this embodiment can be implemented in electronic hardware or in a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0118] In summary, the present invention provides a method, system and program product for three-dimensional reconstruction of the heart based on an improved GRAF algorithm. The three-dimensional reconstruction method includes the steps of collecting and preprocessing digitally reconstructed radiographic images of different patients, different contrast types, and normal and abnormal cardiac anatomical structures; constructing and training an improved GRAF algorithm model; and rendering continuous image frames by inputting a small amount of CT image data into the trained GRAF algorithm model, and using them for three-dimensional reconstruction. The present invention can achieve accurate reconstruction of the three-dimensional structure of the heart under limited CT image input through an improved GRAF algorithm model trained based on data enhancement transformation and a method combining multiple training heads.

[0119] Those skilled in the art should understand that those skilled in the art can implement variations by combining the prior art and the above embodiments, which will not be described in detail here. Such variations do not affect the essential content of the present invention, and will not be described in detail here.

[0120] The above describes the preferred embodiments of the present invention. It should be understood that the present invention is not limited to the above-mentioned specific embodiments, and the systems and structures that are not described in detail should be understood to be implemented in a common manner in the art; any technician familiar with the art can use the above-disclosed methods and technical contents to make many possible changes and modifications to the technical solutions of the present invention without departing from the scope of the technical solutions of the present invention, or modify them into equivalent embodiments of equivalent changes, which does not affect the essential content of the present invention. Therefore, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solutions of the present invention are still within the scope of protection of the technical solutions of the present invention.

Claims

1. A three-dimensional cardiac reconstruction method based on an improved GRAF algorithm, characterized in that: include: Step S1, collecting information on cardiac CT scan data: collecting images containing different patients, different contrast types, and normal and abnormal cardiac anatomical structures; Step S2, performing comprehensive and digitized processing including denoising, standardization and digitally reconstructed radiographic image generation on the original CT image data, and performing a number of data enhancement transformations on the processed digitally reconstructed radiographic images to construct a training set and a test set; Step S3, constructing and training an improved GRAF algorithm model: the improved GRAF algorithm model includes a generator network and a discriminator network based on the generative radiation field principle; the generator network samples rays from the digitally reconstructed radiographic image obtained in step S2 by ray sampling, and uses positioning coding to map the 3D position and observation direction to a high-dimensional feature space, further using shape coding and appearance coding to determine the shape and appearance of the object, and then predicting the density and color of any point in the scene through a fully connected layer, and finally synthesizing continuous image frames through volume rendering technology; the discriminator network applies multiple discriminant heads corresponding to different data enhancement transformations, each discriminant head focuses on extracting features under a specific data enhancement transformation and making predictions, and optimizes model parameters by combining the output of the discriminant head and the result of data enhancement through a comprehensive loss function; The training is iterated until the loss function values ​​of the generator network and the discriminator network tend to be stable or the predetermined number of iterations is reached; Step S4: Use the trained GRAF algorithm model and non-continuous cardiac CT images to perform three-dimensional reconstruction of the cardiac structure.

2. The method for three-dimensional reconstruction of heart based on improved GRAF algorithm according to claim 1, characterized in that: In step S1, patient data of different ages, genders, weights and races are collected, including X-ray images of normal hearts and abnormal hearts, and different concentrations of iodinated contrast agents are used during sampling to obtain images with different contrasts.

3. The method for three-dimensional reconstruction of heart based on improved GRAF algorithm according to claim 1, characterized in that: In step S2, for each original CT image, a denoising algorithm is first used to reduce random noise in the image, and then the image contrast is enhanced. The image is then standardized and segmented, and the heart region is extracted using threshold segmentation, region growing or deep learning methods. The image segmentation result is further used to generate digitally reconstructed radiographic images (DRRs), and finally the generated DRRs are resampled to have a uniform resolution and size; data enhancement technology is applied to the sampled DRRs to obtain diversified DRRs; The enhanced DRRs are organized into training sets and test sets.

4. The method for three-dimensional reconstruction of heart based on improved GRAF algorithm according to claim 3, characterized in that: The data enhancement technology includes at least two of flipping, rotation, scaling, and translation.

5. The method for three-dimensional reconstruction of heart based on improved GRAF algorithm according to claim 1, characterized in that: The total loss function calculation formula of the generator and discriminator network is: In the above formula, L d It is used to measure the discriminant head for the real data P p And its enhanced version P p,k The loss of the difference in predicted values, L r For the generator G θ and the discriminator The adversarial loss between them, k is the number of the discriminant head, n is the number of discriminant heads, λ is used to measure the weights between different losses, and θ is the random weight of the generator or the pre-trained weight parameter; is the kth discriminator weight parameter.

6. The method for three-dimensional reconstruction of heart based on improved GRAF algorithm according to claim 1, characterized in that: In step S4, firstly, a small number or a single cardiac CT scan image is selected as the input of the improved GRAF algorithm model, and the input image covers the required anatomical structure and contrast type; secondly, the input CT image is preprocessed to match the data distribution during training; the trained generator network is loaded again, and the generator is used to generate continuous image frames from the latent space vector; finally, a 3D reconstruction algorithm is used to reconstruct a cardiac 3D image from the continuous image frames.

7. A heart three-dimensional reconstruction system based on an improved GRAF algorithm, characterized in that: A method for implementing the three-dimensional reconstruction of a heart as claimed in any one of claims 1 to 6, comprising: An image acquisition module that acquires images of different patients, different contrast types, and normal and abnormal cardiac anatomy; X-ray image synthesis and data processing module, which performs synthesis and data processing on the original CT data, including denoising, standardization and digital reconstruction of radiological images; The improved GRAF algorithm model includes a generator module and a discriminator module based on the generative radiation field principle; the generator module includes ray sampling, scene representation, density prediction network, color prediction network, and volume rendering submodules; the discriminator module includes image block extraction, a discriminator backbone network, and multiple data enhancement transformation discriminant heads, and optimizes model parameters by combining the output of the discriminant head and the result of data enhancement through a comprehensive loss function; the GRAF algorithm model with optimized model parameters is used to generate continuous image frames from a small number of images or a single view; The three-dimensional reconstruction module uses a three-dimensional reconstruction algorithm to reconstruct a 3D image of the heart from continuous image frames.

8. The heart three-dimensional reconstruction system based on the improved GRAF algorithm according to claim 7, characterized in that: The data enhancement transformation discriminant head includes a flip discriminant head, a rotation discriminant head and a scaling discriminant head.

9. The heart three-dimensional reconstruction system based on the improved GRAF algorithm according to claim 7 or 8, characterized in that: The total loss function calculation formula of the generator and discriminator network is: In the above formula, L d It is used to measure the discriminant head for the real data P p And its enhanced version P p,k The loss of the difference in predicted values, L r For the generator G θ and the discriminator The adversarial loss between them, k is the number of the discriminant head, n is the number of discriminant heads, λ is used to measure the weights between different losses, and θ is the random weight of the generator or the pre-trained weight parameter; is the kth discriminator parameter.

10. A computer program product, characterized in that When the computer program product is run on a computer, the computer is enabled to execute the heart three-dimensional reconstruction method based on the improved GRAF algorithm as claimed in any one of claims 1 to 6.

Images