Improved 3D Gaussian splash CT image reconstruction methods, equipment, and CT systems

Abstract

This invention belongs to the field of CT image processing, specifically relating to an improved CT image reconstruction method, device, and CT system using 3D Gaussian splashing. The method includes: acquiring multiple frames of two-dimensional X-ray images; performing three-dimensional reconstruction on each frame of the two-dimensional X-ray image to generate three-dimensional point cloud data and camera position information; generating a two-dimensional plane for each frame of the two-dimensional X-ray image using 3D Gaussian splashing based on the three-dimensional point cloud data and camera position information; performing adaptive rasterization processing on the data projected onto the two-dimensional plane to generate two-dimensional image data; and performing adaptive rendering processing on the rasterized two-dimensional image data to generate a CT reconstructed image. This invention eliminates the need for depth sorting and does not rely on large-scale labeled datasets. It effectively suppresses noise, reduces artifacts, and maintains the edge and detail clarity of the image, while possessing high-efficiency reconstruction performance, making it suitable for rapid, high-quality reconstruction of low-dose CT imaging.

Description

Technical Field

[0001] This invention belongs to the field of CT reconstruction technology, specifically relating to an improved 3D Gaussian splash CT image reconstruction method, equipment, and CT system. Background Technology

[0002] In the fields of medical imaging and industrial inspection, computed tomography (CT) reconstruction technology is a crucial tool that can reconstruct a three-dimensional image of a scanned object by processing a series of two-dimensional X-ray images. However, traditional CT reconstruction techniques, such as filtered back projection (FBP), often produce image artifacts and reconstruction errors when processing noisy, low-dose scan data.

[0003] In recent years, deep learning technology has been introduced into CT image reconstruction, especially methods based on Convolutional Neural Networks (CNNs), which have shown certain advantages in improving image quality and reducing noise. However, the dependence on large amounts of labeled data, the complexity of the training process, and the insufficient generalization ability of these methods limit their application in specific medical scenarios. Furthermore, methods based on neural implicit representations, such as Neural Radiance Fields (NeRF), while showing potential in simulating complex optical properties and geometric structures, suffer from high computational costs and long training times, which severely hinder their application in real-time or near-real-time medical imaging.

[0004] In light of this, 3D Gaussian Splatting (3DGS) technology offers an innovative solution for CT reconstruction. By spatially mapping and smoothing voxel data through a Gaussian kernel, 3DGS effectively suppresses noise and reduces artifacts while maintaining edge and detail sharpness—a balance difficult to achieve in traditional and deep learning methods. This technology does not rely on large-scale training data, making the computation process simpler and more efficient, suitable for rapid reconstruction needs, and providing higher stability and interpretability, which is particularly important for clinical diagnosis. However, traditional 3DGS technology relies primarily on simple blending strategies for transparency blending during rendering, failing to adequately consider the overlapping relationships of different tissue structures in CT slices. This results in insufficient rendering quality when processing medical images with complex tissue overlaps, especially in terms of image detail and structural visualization. Summary of the Invention

[0005] In summary, addressing the shortcomings of existing CT reconstruction techniques, this invention proposes an improved 3D Gaussian splash CT image reconstruction method, device, and CT system. This invention combines the advantages of 3DGS and order-independent transparent rendering (OIT), providing fast, efficient, and stable image reconstruction capabilities without relying on large amounts of training data, while ensuring high quality and accuracy of the reconstructed images. This method effectively handles transparency mixing issues and correctly renders the overlap of various tissue structures in CT images, thereby significantly improving the visualization quality of reconstructed images and the accuracy of clinical diagnosis. It not only significantly enhances the usability of low-dose CT imaging but also brings revolutionary improvements to the field of medical imaging.

[0006] In a first aspect, the present invention provides an improved 3D Gaussian splash CT image reconstruction method, comprising the following steps:

[0007] S1. Acquire multiple frames of two-dimensional X-ray images, wherein the multiple frames of two-dimensional X-ray images are obtained by X-ray imaging equipment emitting X-rays at multiple arbitrary imaging angles to image the medical object;

[0008] S2. Perform three-dimensional reconstruction on each frame of two-dimensional X-ray image to generate three-dimensional point cloud data and camera position information; the three-dimensional point cloud data and camera position information are used to indicate the three-dimensional geometric information of the medical object.

[0009] S3: Based on the three-dimensional point cloud data and camera position information of each frame of two-dimensional X-ray image, use 3D Gaussian splashing to generate a two-dimensional plane for each frame of two-dimensional X-ray image;

[0010] S4. Perform adaptive rasterization on the data projected onto the two-dimensional plane to generate two-dimensional image data;

[0011] S5. Adaptive rendering processing is performed on the rasterized two-dimensional image data to generate CT reconstructed images.

[0012] In a second aspect, the present invention provides an improved 3D Gaussian splash CT image reconstruction apparatus, comprising:

[0013] The acquisition unit is used to acquire multiple frames of two-dimensional X-ray images, which are obtained by an X-ray imaging device emitting X-rays at multiple arbitrary imaging angles to image a medical object.

[0014] A segmentation unit is used to perform three-dimensional reconstruction on each frame of two-dimensional X-ray images to generate three-dimensional point cloud data and camera position information; the three-dimensional point cloud data and camera position information are used to indicate the three-dimensional geometric information of the medical object; based on the three-dimensional point cloud data and camera position information of each frame of two-dimensional X-ray images, a two-dimensional plane of each frame of two-dimensional X-ray images is generated using 3D Gaussian sputtering; the data projected onto the two-dimensional plane is subjected to adaptive rasterization processing to generate two-dimensional image data;

[0015] The reconstruction unit is used to perform adaptive rendering processing on rasterized two-dimensional image data to generate CT reconstructed images.

[0016] In a third aspect, the present invention provides a CT system that scans with X-rays and outputs two-dimensional X-ray images, comprising:

[0017] X-ray imaging equipment uses X-rays to scan and obtain two-dimensional X-ray images;

[0018] CT image reconstruction equipment, such as the CT image reconstruction equipment described in the second aspect of the present invention;

[0019] A CT image output device that outputs CT images reconstructed by the CT image reconstruction device.

[0020] Compared with the prior art, the advantages of the present invention are as follows:

[0021] This invention overcomes many limitations of existing CT reconstruction techniques. First, it introduces a 3D Gaussian kernel to smooth spatial data and combines it with a weighted blending order-independent transparent rendering technique for adaptive transparency blending, effectively suppressing noise and reducing artifacts. Simultaneously, it achieves higher-quality transparency processing for visualizing image details and tissue structures, ensuring the clarity of edges and details. Second, compared to traditional reconstruction methods based on filtered back projection (FBP) and deep learning, this invention does not rely on large-scale labeled datasets, significantly reducing computational complexity and training costs. The weighted blending order-independent transparent rendering technique also eliminates the need for depth sorting of transparency data, further simplifying the processing flow and improving rendering efficiency. Furthermore, the improved 3D Gaussian splashing technique after using weighted blending order-independent transparent rendering exhibits higher efficiency and real-time performance in image reconstruction, meeting the needs of rapid reconstruction, especially suitable for high-quality reconstruction in low-dose scanning scenarios. Finally, this invention improves the stability and interpretability of reconstructed images, enabling images to accurately reflect the relationships between different tissues, providing reliable image support for clinical diagnosis, and greatly enhancing the overall performance and application value of medical imaging systems. Attached Figure Description

[0022] Figure 1This is a flowchart of the CT image reconstruction method in an embodiment of the present invention;

[0023] Figure 2 This is a schematic diagram of the CT image reconstruction device according to an embodiment of the present invention;

[0024] Figure 3 This is a schematic diagram of the CT image reconstruction device according to a preferred embodiment of the present invention;

[0025] Figure 4 This is a schematic diagram of a CT system according to an embodiment of the present invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0027] Currently, traditional 3DGS technology relies primarily on simple blending strategies for transparency blending during rendering, failing to adequately consider the overlapping relationships of different tissue structures in CT slices. This results in insufficient rendering quality when processing medical images with complex tissue overlaps, particularly in terms of image detail and structural visualization. To address the shortcomings of existing CT reconstruction techniques, there is an urgent need to develop a novel CT reconstruction technology. This technology should combine the advantages of 3DGS to provide fast, efficient, and stable image reconstruction capabilities without relying on large amounts of training data, while ensuring high-quality and accurate reconstructed images. This approach would not only significantly improve the usability of low-dose CT imaging but also bring revolutionary improvements to the field of medical imaging.

[0028] This invention proposes an improved 3D Gaussian splash CT image reconstruction method, such as... Figure 1 As shown, the method includes the following:

[0029] S1. Acquire multiple frames of two-dimensional X-ray images, wherein the multiple frames of two-dimensional X-ray images are obtained by X-ray imaging equipment emitting X-rays at multiple arbitrary imaging angles to image the medical object;

[0030] In some embodiments, two-dimensional X-ray images from multiple angles are acquired using an X-ray imaging device, and then denoised, contrast-adjusted, and geometrically corrected to improve image quality.

[0031] In some embodiments, in a clinical setting, X-ray imaging equipment is used to image a medical object from multiple angles. The medical object is typically a part of the human body, such as the head or chest, and is fixed on a scanning platform. The X-ray imaging equipment may rotate around the medical object, typically acquiring two-dimensional X-ray images from 20 to 30 different angles, generating one image for each angle. All images record the X-ray diffraction information of the object at different angles. These images will be used for subsequent three-dimensional reconstruction processing.

[0032] In practice, the system is set to perform an image every 12 degrees, for a total of 30 scans. Images acquired at each angle are saved in DICOM format, typically with a resolution of 1024×1024 pixels. Each image contains complete grayscale information to reflect the different densities of the tissue structure.

[0033] In some embodiments, to ensure the image quality of the reconstructed two-dimensional X-ray image and remove noise that may be caused by imaging equipment noise or external interference, a median filtering algorithm is applied to each frame of the two-dimensional X-ray image for denoising. Median filtering is commonly used in medical image processing to reduce noise while preserving the edge structural details of the image.

[0034] In some embodiments, the denoised two-dimensional X-ray images may exhibit significant differences in grayscale values, thus requiring normalization. Normalization aims to standardize the pixel values ​​of the two-dimensional X-ray image to the range [0,1], ensuring consistency in grayscale values ​​between different two-dimensional X-ray images and facilitating subsequent three-dimensional reconstruction operations. In practice, the minimum and maximum grayscale values ​​in each two-dimensional X-ray image are first calculated, and then a normalization formula is applied to each pixel, mapping its grayscale value to the range of 0 to 1.

[0035] For a two-dimensional X-ray image, the pixel values ​​of the denoised image range from 50 to 2000. After normalization, all pixel values ​​will be mapped to the range of 0 to 1, so that the data standard in subsequent processing steps will remain consistent.

[0036] S2. Perform three-dimensional reconstruction on each frame of two-dimensional X-ray image to generate three-dimensional point cloud data and camera position information; the three-dimensional point cloud data and camera position information are used to indicate the three-dimensional geometric information of the medical object.

[0037] In some embodiments, this embodiment can utilize the Structure from Motion (SfM) algorithm to perform 3D reconstruction on the preprocessed 2D image, generating sparse point cloud data and camera position information to provide 3D geometric information of the medical object. The camera pose matrix [R|t] is further calculated using the essential matrix E, where R is the camera's rotation matrix and t is the translation vector. This information is used to describe the relative positions between different 2D X-ray images.

[0038] After obtaining the camera pose, a 3D sparse point cloud is generated using triangulation. The specific steps are as follows:

[0039] For feature point u in image 1 i and the corresponding point u in Figure 2 i′ The triangulation calculation is performed using the camera's projection matrices P1 and P2. The mathematical formula is as follows:

[0040]

[0041] Among them, X i λ1 and λ2 are the corresponding depth parameters, representing the depth of the point in space.

[0042] By triangulation, the position of each pair of matched feature points in 3D space is calculated, forming a sparse 3D point cloud. These point clouds are automatically optimized and error points are removed, making the point clouds more accurate.

[0043] In practice, X-ray images of medical objects, such as the head or chest, are processed to generate data containing thousands to tens of thousands of sparse 3D points. These point clouds accurately reflect the object's geometry, especially the changes in surface contour and density at different imaging angles. The generated point cloud data and camera pose information can be used for subsequent 3D model reconstruction and medical analysis.

[0044] By preprocessing the image in S1, feature points can be detected more accurately, and accurate sparse point clouds can be generated through camera pose estimation, ensuring the reliability and accuracy of the reconstruction of the three-dimensional geometric information of medical objects.

[0045] In some preferred embodiments, after performing three-dimensional reconstruction on each frame of two-dimensional X-ray image to generate three-dimensional point cloud data and camera position information, the method further includes:

[0046] The three-dimensional point cloud data are stacked into a three-dimensional volume data;

[0047] The three-dimensional volume data is resampled to generate smoothed three-dimensional point cloud data.

[0048] Specifically, a three-dimensional interpolation algorithm is used to resample the three-dimensional volume data to generate resampled three-dimensional volume data; wavelet transform is performed on the resampled three-dimensional volume data to remove high-frequency noise information and retain low-frequency structural information, thereby generating smoothed three-dimensional point cloud data.

[0049] It should be noted that in this embodiment, the three-dimensional volume data is stacked from two-dimensional slice data and resampled. During the resampling process, a target resolution is selected, and a three-dimensional interpolation algorithm (such as B-spline interpolation) is used to smooth the calculation of the three-dimensional volume data. This algorithm can avoid significant visual artifacts in CT images when the resolution is reduced. After obtaining the resampled three-dimensional voxel data, wavelet transform is performed on the entire voxel data to identify and remove high-frequency noise while retaining the structural information of the low-frequency components. Then, inverse wavelet transform is performed to recover the processed three-dimensional voxel data. This ensures that the final three-dimensional voxel data obtained before 3D Gaussian splatting will have high quality and noise removal.

[0050] S3: Based on the three-dimensional point cloud data and camera position information of each frame of two-dimensional X-ray image, use 3D Gaussian splashing to generate a two-dimensional plane for each frame of two-dimensional X-ray image;

[0051] In practical implementation, the system first uses the 3D sparse point cloud generated by S2 and camera position information as the input dataset. Next, the system progressively converts this point cloud data into a 3D Gaussian kernel representation, with the specific steps as follows:

[0052] In the preferred example, each 3D point X is first selected. i The covariance matrix is ​​calculated based on its neighboring points. The criterion for selecting neighboring points is point X. i Other points within a 5 mm radius. In the rib region, points in the vicinity are usually more densely distributed, while points in the soft tissue region are relatively more dispersed.

[0053] Calculate the covariance matrix Σ using the following formula:

[0054]

[0055] in, The centroid of this neighborhood is denoted by Σ, and the distribution of points within the neighborhood is reflected by the location of the centroid. The covariance matrix Σ describes the geometric structure of the point cloud in a local region.

[0056] Next, we perform eigenvalue decomposition on the covariance matrix Σ:

[0057] Σ=QΛQ T (3)

[0058] Where Q is the eigenvector matrix of the covariance matrix, representing the main directions of the 3D point cloud, and Λ is a diagonal matrix containing eigenvalues. By analyzing the eigenvalues ​​λ1, λ2, and λ3, the system can determine the axis length of each Gaussian kernel; the larger the eigenvalue, the wider the distribution in that direction.

[0059] After obtaining the eigenvalues ​​and eigenvectors, the system adjusts the shape and orientation of the Gaussian kernel based on the local geometry of each point. In practice, the system uses the eigenvalues ​​to determine the axis length of the Gaussian kernel and the eigenvector Q to determine the rotation direction of the Gaussian kernel.

[0060] Shape Adjustment: The magnitudes of the eigenvalues ​​λ1, λ2, and λ3 determine the proportional relationship of the three axes of the Gaussian kernel. For the rib region, the eigenvalue λ1 is usually larger, indicating that the geometry of this region is elongated; while for the soft tissue region, the eigenvalues ​​may be relatively close, indicating that the geometry is more symmetrical and approximately spherical.

[0061] Orientation adjustment: The Gaussian kernel is rotated using the eigenvector Q to align it with the principal orientation of the point cloud. For the rib region, the Gaussian kernel is extended along the curvature of the rib to accurately represent its structure.

[0062] To visualize a 3D scene, the system needs to assign transparency and color information to each Gaussian kernel.

[0063] Transparency Assignment: The system assigns transparency to the Gaussian kernel based on the local density of the point cloud. For dense skeletal regions, the Gaussian kernel has lower transparency, typically between 0.2 and 0.4, making these regions stand out more in the 3D image. For sparse soft tissue regions, the transparency is higher, typically between 0.6 and 0.8, making these regions appear softer in the image.

[0064] Color information calculation: The system uses spherical harmonic functions to calculate the color information of a Gaussian kernel. In specific implementation, the spherical harmonic functions are used to estimate the color information of different parts based on the illumination angle.

[0065] In skeletal areas, color information is typically presented as grayish-white; while in soft tissue areas, the color is presented as light red or pink, simulating the real colors of different tissues in the human body.

[0066] In practice, the system projects the 3D Gaussian ellipsoid generated in S3 onto a 2D plane using 3DGS technology to convert 3D medical data into a 2D image. This step mainly involves calculating the projection area of ​​the 3D Gaussian kernel, mapping it to the 2D image coordinate system, and generating a 2D projection result for subsequent medical analysis and visualization.

[0067] In practical implementation, the system first utilizes a three-dimensional Gaussian kernel generated by S3, the position and shape of which are determined by the three-dimensional covariance matrix Σ of the Gaussian kernel.3D and spatial location X i Confirmed. To achieve the mapping from three dimensions to two dimensions, the system needs to project these three-dimensional coordinates into a two-dimensional image space.

[0068] First, define the perspective projection matrix P, which is mathematically represented as follows:

[0069]

[0070] Among them, f x and f y These are the focal length parameters of a two-dimensional X-ray image, c x and c y This represents the coordinates of the image center point. Using this matrix, the 3D point X... i It will be converted into two-dimensional coordinates.

[0071] Next, the system uses the perspective projection matrix P to project the three-dimensional point X. i Projecting onto a two-dimensional image plane, the formula is:

[0072] x i =PX i (5)

[0073] Among them, X i Represents the coordinates of a three-dimensional point, x i Represents the coordinates projected onto a two-dimensional plane.

[0074] To correctly map the shape of the 3D Gaussian kernel onto the 2D image, the system utilizes 3DGS technology based on the 3D covariance matrix Σ. 3D Calculate the two-dimensional covariance matrix Σ 2D This allows for an accurate description of the distribution of the Gaussian kernel in a two-dimensional image.

[0075] Jacobian matrix calculation: The core of 3DGS technology lies in mapping a three-dimensional Gaussian kernel to a two-dimensional plane through affine transformation. Specifically, the system uses the Jacobian matrix J to linearize the mapping transformation from three-dimensional space to the two-dimensional plane. The Jacobian matrix J is derived from the local linear transformation of the projection matrix P and is used to capture the orientation and shape changes of the three-dimensional ellipsoid.

[0076] Calculation of the two-dimensional covariance matrix: Based on the Jacobian matrix J and the three-dimensional covariance matrix Σ 3D The system calculates the two-dimensional covariance matrix Σ. 2D The formula is:

[0077] Σ 2D =JΣ 3D J T (6)

[0078] Where, Σ 2DΣ represents the two-dimensional covariance matrix of the Gaussian kernel distribution shape on the two-dimensional image plane, describing the direction and magnitude of the Gaussian kernel's expansion in the two-dimensional plane. J represents the Jacobian matrix, indicating the affine transformation from three-dimensional to two-dimensional space. The Jacobian matrix plays a linear approximation role in the projection process, mapping changes in three dimensions to the two-dimensional plane through local linearization. 3D The three-dimensional covariance matrix represents the shape of the Gaussian kernel distribution in three-dimensional space, reflecting the extension direction and shape of the Gaussian kernel in three-dimensional space.

[0079] The two-dimensional covariance matrix represents the shape and orientation of the Gaussian kernel in the two-dimensional plane, reflecting the extension direction and size of the ribs in the image.

[0080] After calculating the two-dimensional covariance matrix, the system then maps the three-dimensional Gaussian kernel onto the two-dimensional image plane, generating a two-dimensional Gaussian kernel projection region. This process is accomplished using the Gaussian projection formula:

[0081]

[0082] Wherein G(x) i ) represents the distribution value of the Gaussian kernel on the two-dimensional plane, and represents the value of point x in the image plane. i The shape and intensity of the surrounding Gaussian distribution. x represents the coordinates of any point in the two-dimensional plane, used to indicate its position on the plane. i This represents the coordinates of the Gaussian kernel center point projected onto a two-dimensional plane. The projection matrix P is used to project the three-dimensional point X... i It was converted from. This represents the inverse of the two-dimensional covariance matrix, used to calculate the relationship between point x and the center point x of the Gaussian kernel. i Changes in distance and direction between them.

[0083] This formula describes the distribution shape of the Gaussian kernel in a two-dimensional image. The two-dimensional Gaussian projection of the rib region will appear as an elongated ellipse, which corresponds to the actual shape of the rib.

[0084] Through step S3, the system successfully projects a three-dimensional Gaussian kernel onto a two-dimensional image plane, generating a two-dimensional image that reflects the three-dimensional structure of a medical object. This image can be used by doctors for clinical diagnosis and anatomical analysis, helping them to understand the patient's internal structure more intuitively.

[0085] S4. Perform adaptive rasterization on the data projected onto the two-dimensional plane to generate two-dimensional image data;

[0086] In some embodiments, the data mapped to the two-dimensional plane is rasterized, the centroid coordinates are interpolated, and the depth is subjected to perspective correction interpolation.

[0087] In S3, the Gaussian kernel has been mapped onto a 2D plane using a splashing technique. The main task of S4 is to rasterize this mapped 2D data to generate the final 2D image data in pixel form. The key to rasterization is calculating the centroid coordinates and then performing perspective-corrected interpolation on the depth to obtain the correct depth value, which facilitates the execution of the Weighted Blended OIT algorithm in S5 to achieve the correct rendering effect.

[0088] In practice, to accelerate rasterization processing, the system first divides the image space into multiple 16×16 grid blocks. Each grid block contains 256 pixels, a partitioning method that facilitates parallel computation. The system determines the grid block position for each Gaussian point. The grid block division is based on the (x,y) coordinates of the Gaussian point in the two-dimensional image space, and each Gaussian point is associated with its corresponding grid block.

[0089] In the GPU, a thread block is allocated for each grid tile. Each thread block is responsible for processing all pixels within a grid tile and loading a list of all Gaussian points in that grid tile into the thread block's shared memory. In this way, all Gaussian point data intersecting with the grid tile can be accessed and processed efficiently.

[0090] For each grid block in the reconstruction of a chest CT image, Gaussian points in the reconstruction space are grouped according to their adjacent positions. A triangle is formed by assembling three adjacent Gaussian points. For each triangle, the centroid coordinates of any pixel in screen space relative to the three vertices of that triangle are calculated.

[0091] In the specific implementation of chest CT image reconstruction, the calculated centroid coordinates are used to calculate the precise depth value of the target pixel using a perspective correction interpolation method, ensuring that the depth information of the reconstructed image accurately reflects the three-dimensional anatomical structure of the thoracic cavity. The calculated centroid coordinates are used to calculate the correct depth value of the pixel using an adaptive perspective correction interpolation formula. Based on the correct depth value of each pixel, two-dimensional image data is generated.

[0092] The specific depth interpolation formula is as follows:

[0093]

[0094] Where D is the perspective-corrected depth value of the target pixel, D A D B D C These are the depth values ​​of the three vertices of the triangle formed by the Gaussian points, b1, b2, and b3 are the weighting coefficients calculated using the centroid coordinates, and a1(ξ), a2(ξ), and a3(ξ) represent the X-ray attenuation coefficients at different angles at position ξ.

[0095] It should be noted that by combining the X-ray attenuation coefficients at different angles, the impact of tissue density on CT image reconstruction when viewed from different angles can be more accurately reflected. For example, in 3D reconstruction of tumors or the chest, the X-ray attenuation characteristics of different tissues will directly affect the accuracy of reconstruction and image quality.

[0096] Through step S4, the system successfully rasterizes the data projected onto the two-dimensional plane, calculates the correct depth value of each pixel in the chest CT image through perspective correction interpolation, and generates two-dimensional image data representing the chest CT image.

[0097] S5. Adaptive rendering processing is performed on the rasterized two-dimensional image data to generate CT reconstructed images.

[0098] In some embodiments, this embodiment combines rasterized two-dimensional data with Weighted Blended OIT for transparency blending to generate medical CT images for medical diagnosis and analysis.

[0099] In S4, the system rasterizes the 3D data into 2D pixel data. The goal of S5 is to use Weighted Blended OIT (Optical Integer Interaction) technology to weightedly accumulate the color and transparency of each pixel, ultimately generating a rendered image for medical diagnosis and analysis. This step performs color and transparency blending calculations pixel-by-pixel, optimizing the rendering process and generating the final medical image.

[0100] In practice, the system processes each pixel of the chest CT image one by one, and calculates the weight for each Gaussian point that intersects with the pixel. The weight function form that yielded the best results in the implementation is as follows:

[0101]

[0102] in, Let ξ be the normalization factor, α be the transparency, d(ξ) be the voxel density at pixel depth ξ, and m(ξ) be the mask value for a specific structure (e.g., the lung), weighted based on density. These weights are used to calculate the color and transparency values ​​for each pixel in the chest CT image, resulting in the final color C. f The calculation formula is as follows:

[0103] C f (x,y)=C·(1-exp(-∑(K*(log(1-α·w′)))(x,y)))+C0·exp(-∑(K*(log(1-α·w′))))(x,y)) (10)

[0104] Among them, C f(x,y) is the final color of the CT reconstructed image, (x,y) is the image pixel position, C is the color value of each pixel in the original CT image, α is the pixel transparency, C0 is the background color, w′ is the adaptive weight function, which is related to depth, transparency and density; K represents the convolution kernel, which is used to weight the pixel contribution in the neighborhood.

[0105] When accumulating transparency, the system continuously checks whether the transparency of each pixel has reached the set threshold of 0.95. When the accumulated transparency of a pixel reaches 0.95, the system assumes that the pixel is completely opaque, and subsequent Gaussian points will no longer affect that pixel. At this point, the system skips further processing of that pixel, thereby reducing computation and improving rendering efficiency.

[0106] Once the color and transparency of all pixels have been calculated, the system stores the results in the frame buffer and generates the final chest CT image. The frame buffer size is consistent with the image resolution (1024×1024), and each pixel stores its final color value. The system converts the data in the frame buffer into a standard image format for display, storage, or further medical analysis.

[0107] Understandably, traditional 3DGS algorithms perform depth sorting when blending transparency, and this doesn't fully account for the overlapping relationships of different complex tissue structures in CT slices, leading to rendering errors. An improved rendering method based on Order-Independent Transparency (OIT) technology is proposed to correctly render the overlap of various tissue structures in CT images. This algorithm not only renders CT images correctly but also eliminates the need for depth sorting, further improving efficiency. In this way, the final rendered image clearly displays the anatomical structures in a chest CT scan. In the rendered image, rib areas are displayed as white areas, and soft tissues as light red areas, allowing doctors to visually identify rib fracture locations or abnormal lung features.

[0108] It should be noted that in this embodiment, the input data for processing medical CT images is three-dimensional volumetric data, typically a set of voxels composed of multiple slices, and the output is a reconstructed three-dimensional volumetric image or a corresponding two-dimensional slice. This process not only focuses on the detailed representation of the image but also needs to accurately restore physical properties and internal structures. In medical CT image reconstruction, the reconstruction goal is to provide accurate representation of anatomical structures and physical properties. The output image is used for clinical diagnosis and medical analysis. This process requires special attention to the accuracy of details, contrast, and noise reduction. The output is not only a two-dimensional slice image but also requires the ability to provide multi-dimensional visualization effects.

[0109] Figure 2 This is a schematic diagram of the CT image reconstruction device according to an embodiment of the present invention, as shown below. Figure 2 As shown, it includes:

[0110] The acquisition unit 201 is used to acquire multiple frames of two-dimensional X-ray images, which are obtained by an X-ray imaging device emitting X-rays at multiple arbitrary imaging angles to image a medical object.

[0111] The segmentation unit 202 is used to perform three-dimensional reconstruction on each frame of two-dimensional X-ray image to generate three-dimensional point cloud data and camera position information; the three-dimensional point cloud data and camera position information are used to indicate the three-dimensional geometric information of the medical object; based on the three-dimensional point cloud data and camera position information of each frame of two-dimensional X-ray image, a two-dimensional plane of each frame of two-dimensional X-ray image is generated using 3D Gaussian splashing; the data projected onto the two-dimensional plane is subjected to adaptive rasterization processing to generate two-dimensional image data;

[0112] The reconstruction unit 203 is used to perform adaptive rendering processing on rasterized two-dimensional image data to generate CT reconstructed images.

[0113] Figure 3 This is a schematic diagram of the CT image reconstruction device according to a preferred embodiment of the present invention, as shown below. Figure 3 As shown, it includes:

[0114] The acquisition unit 301 is used to acquire multiple frames of two-dimensional X-ray images, which are obtained by an X-ray imaging device emitting X-rays at multiple arbitrary imaging angles to image a medical object.

[0115] The calibration unit 302 is used to stack the three-dimensional point cloud data into a three-dimensional volume data; and to resample the three-dimensional volume data to generate smoothed three-dimensional point cloud data.

[0116] The segmentation unit 303 is used to perform three-dimensional reconstruction on each frame of two-dimensional X-ray image to generate three-dimensional point cloud data and camera position information; the three-dimensional point cloud data and camera position information are used to indicate the three-dimensional geometric information of the medical object; based on the three-dimensional point cloud data and camera position information of each frame of two-dimensional X-ray image, a two-dimensional plane of each frame of two-dimensional X-ray image is generated using 3D Gaussian splashing; the data projected onto the two-dimensional plane is subjected to adaptive rasterization processing to generate two-dimensional image data;

[0117] The reconstruction unit 304 is used to perform adaptive rendering processing on rasterized two-dimensional image data to generate CT reconstructed images.

[0118] Figure 4 This is a schematic diagram of the CT imaging system structure according to an embodiment of the present invention, as shown below. Figure 4 As shown, it includes:

[0119] X-ray imaging device 401 scans with X-rays to obtain two-dimensional X-ray images;

[0120] The X-ray imaging device 401 may include an X-ray scanner that scans the object in the scanning area using X-rays. Here, the object being scanned may be a living organism such as a human body, and may include moving objects such as a heart.

[0121] CT image reconstruction device 402, wherein the CT image reconstruction device may be the CT image reconstruction device described above;

[0122] The CT image reconstruction device 402 is implemented by a general-purpose computer or a dedicated integrated circuit. The CT image reconstruction device 402 generates CT images based on the two-dimensional X-ray images output by the CT scanner 401, using, for example, the CT image reconstruction methods in steps S1-S5 of the embodiments of the present invention.

[0123] The CT image output device 403 outputs CT images reconstructed by the CT image reconstruction device.

[0124] The CT image output device 403 is typically a CT image display device, which displays the CT image output by the CT image reconstruction device 402 on a screen. Of course, the CT image output device 403 is not limited to a CT image display device; it can also be a data transmission interface that sends the CT image output by the CT image reconstruction device 402 via a network, a printer that prints the CT image output by the CT image reconstruction device 402, etc.

[0125] Understandably, CT images output by a CT system can be used in various fields, such as scientific research and biological data acquisition. Furthermore, these CT images can also serve as intermediate data for applications in disease diagnosis and health management.

[0126] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, which may include ROM, RAM, disk, or optical disk, etc.

[0127] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An improved 3D Gaussian splash CT image reconstruction method, characterized in that, Includes the following steps: Acquire multiple frames of two-dimensional X-ray images, which are obtained by an X-ray imaging device emitting X-rays at multiple arbitrary imaging angles onto a medical object; Perform 3D reconstruction on each frame of 2D X-ray image to generate 3D point cloud data and camera position information; The three-dimensional point cloud data and camera position information are used to indicate the three-dimensional geometric information of the medical object; Based on the three-dimensional point cloud data and camera position information of each frame of two-dimensional X-ray image, a two-dimensional plane of each frame of two-dimensional X-ray image is generated using 3D Gaussian splashing. Using the generated 3D sparse point cloud and camera position information as input datasets, the system progressively converts these point cloud data into a 3D Gaussian kernel representation. The specific steps are as follows: First, select each 3D point X. i The covariance matrix is ​​calculated based on its neighboring points; the criterion for selecting neighboring points is point X. i Other points within a 5 mm radius; Perform eigenvalue decomposition on the covariance matrix Σ; After obtaining the eigenvalues ​​and eigenvectors, the system adjusts the shape and orientation of the Gaussian kernel according to the local geometry of each point; The process of adaptive rasterization to generate two-dimensional image data from data projected onto a two-dimensional plane includes the following steps: Based on the three-dimensional position of each Gaussian point, determine the grid block it belongs to and its depth relative to the view plane; Assign a thread block to each grid block and load a list of all Gaussian points that intersect with that grid block into the shared memory of the thread block; In each grid block, three adjacent Gaussian points are combined into a triangle by primitive assembly. For each triangle, the centroid coordinates of any pixel in screen space are calculated. Using the calculated centroid coordinates, the correct depth value of the pixel is calculated through adaptive perspective correction interpolation. Generate two-dimensional image data based on the correct depth value of each pixel; The interpolation formula for the adaptive perspective correction interpolation technique is as follows: Where D is the perspective-corrected depth value of the target pixel, D A D B D C These are the depth values ​​of the three vertices of the triangle formed by the Gaussian points, b1, b2, and b3 are the weighting coefficients calculated using the centroid coordinates, and a1(ξ), a2(ξ), and a3(ξ) represent the X-ray attenuation coefficients at different angles at position ξ. The process of adaptively rendering rasterized 2D image data to generate CT reconstructed images includes the following steps: Transparency blending is performed using weighted blending order-independent transparent rendering. For each Gaussian point intersecting with a pixel, adaptive weights based on depth and transparency are calculated, and these adaptive weights are used to calculate the color value and transparency value of that pixel. If the transparency value of a pixel reaches the set threshold, further processing of that pixel will stop. After calculating the color and transparency of all pixels, the results are stored in the frame buffer to generate the final CT reconstructed image.

2. The CT image reconstruction method according to claim 1, characterized in that, Before generating the two-dimensional plane of each two-dimensional X-ray image using 3D Gaussian splashing based on the three-dimensional point cloud data and camera position information of each frame of the two-dimensional X-ray image, the following steps are also included: The three-dimensional point cloud data are stacked into a three-dimensional volume data; The three-dimensional volume data is resampled to generate smoothed three-dimensional point cloud data.

3. The CT image reconstruction method according to claim 2, characterized in that, Resampling the three-dimensional volume data to generate smoothed three-dimensional point cloud data includes: The three-dimensional volume data is resampled using a three-dimensional interpolation algorithm to generate resampled three-dimensional volume data. Wavelet transform is performed on the resampled 3D volume data to remove high-frequency noise information and retain low-frequency structural information, generating smoothed 3D point cloud data.

4. The CT image reconstruction method according to claim 1, characterized in that, The formula for calculating the final color of the CT reconstructed image is expressed as follows: C f (x,y)=C·(1-exp(-∑(K*(log(1-α·w′)))(x,y)))+C0·exp(-∑(K*(log(1-α·w′)))(x,y)) Among them, C f (x,y) is the final color of the CT reconstructed image, (x,y) is the image pixel position, C is the color value of each pixel in the original CT image, α is the pixel transparency, C0 is the background color, w′ is the adaptive weight function, which is related to depth, transparency and density; K represents the convolution kernel, which is used to weight the pixel contribution in the neighborhood.

5. An improved CT image reconstruction device using 3D Gaussian splashing, characterized in that, include: The acquisition unit is used to acquire multiple frames of two-dimensional X-ray images, which are obtained by an X-ray imaging device emitting X-rays at multiple arbitrary imaging angles to image a medical object. The segmentation unit is used to perform three-dimensional reconstruction on each frame of two-dimensional X-ray image, generating three-dimensional point cloud data and camera position information; The three-dimensional point cloud data and camera position information are used to indicate the three-dimensional geometric information of the medical object; Based on the 3D point cloud data and camera position information of each frame of 2D X-ray image, a 2D plane of each frame of 2D X-ray image is generated using 3D Gaussian splashing; the data projected onto the 2D plane is subjected to adaptive rasterization processing to generate 2D image data. Using 3D sparse point cloud and camera position information as input datasets, the system progressively converts these point cloud data into a 3D Gaussian kernel representation. The specific steps are as follows: First, select each 3D point X. i The covariance matrix is ​​calculated based on its neighboring points; the criterion for selecting neighboring points is point X. i Other points within a 5 mm radius; Perform eigenvalue decomposition on the covariance matrix Σ; After obtaining the eigenvalues ​​and eigenvectors, the system adjusts the shape and orientation of the Gaussian kernel according to the local geometry of each point; The process of adaptive rasterization to generate two-dimensional image data from data projected onto a two-dimensional plane includes the following steps: Based on the three-dimensional position of each Gaussian point, determine the grid block it belongs to and its depth relative to the view plane; Assign a thread block to each grid block and load a list of all Gaussian points that intersect with that grid block into the shared memory of the thread block; In each grid block, three adjacent Gaussian points are combined into a triangle by primitive assembly. For each triangle, the centroid coordinates of any pixel in screen space are calculated. Using the calculated centroid coordinates, the correct depth value of the pixel is calculated through adaptive perspective correction interpolation. Generate two-dimensional image data based on the correct depth value of each pixel; The interpolation formula for the adaptive perspective correction interpolation technique is as follows: Where D is the perspective-corrected depth value of the target pixel, D A D B D C These are the depth values ​​of the three vertices of the triangle formed by the Gaussian points, b1, b2, and b3 are the weighting coefficients calculated using the centroid coordinates, and a1(ξ), a2(ξ), and a3(ξ) represent the X-ray attenuation coefficients at different angles at position ξ. The reconstruction unit, used for adaptive rendering processing of rasterized two-dimensional image data to generate CT reconstructed images, includes the following steps: Transparency blending is performed using weighted blending order-independent transparent rendering. For each Gaussian point intersecting with a pixel, adaptive weights based on depth and transparency are calculated, and these adaptive weights are used to calculate the color value and transparency value of that pixel. If the transparency value of a pixel reaches the set threshold, further processing of that pixel will stop. After calculating the color and transparency of all pixels, the results are stored in the frame buffer to generate the final CT reconstructed image.

6. The CT image reconstruction apparatus according to claim 5, characterized in that, Also includes: A calibration unit is used to stack the three-dimensional point cloud data into a three-dimensional volume data. The three-dimensional volume data is resampled to generate smoothed three-dimensional point cloud data.

7. A CT system that scans with X-rays and outputs two-dimensional X-ray images, characterized in that, include: X-ray imaging equipment uses X-rays to scan and obtain two-dimensional X-ray images; A CT image reconstruction device, as described in claim 5 or 6; A CT image output device that outputs CT images reconstructed by the CT image reconstruction device.

Images