CT (Computed Tomography) image reconstruction method and equipment for improving 3D (Three-Dimensional) Gaussian splashing and CT system

Abstract

The invention belongs to the field of CT image processing, and particularly relates to a CT image reconstruction method and device for improving 3D Gaussian splashing and a CT system. The method comprises the following steps: acquiring multiple frames of two-dimensional X-ray images; performing three-dimensional reconstruction on each frame of two-dimensional X-ray image to generate three-dimensional point cloud data and camera position information; according to the three-dimensional point cloud data of each frame of two-dimensional X-ray image and the camera position information, generating a two-dimensional plane of each frame of two-dimensional X-ray image by using 3D Gaussian splashing; performing adaptive rasterization processing on the data projected to the two-dimensional plane to generate two-dimensional image data; and performing adaptive rendering processing on the rasterized two-dimensional image data to generate a CT reconstructed image. The method does not need to carry out depth sorting, does not need to depend on a large-scale annotation data set, can effectively suppress noise, reduce artifacts and maintain the edge and detail definition of the image, has efficient reconstruction performance, and is suitable for rapid and high-quality reconstruction of low-dose CT imaging.

Description

Technical Field

[0001] The present invention belongs to the technical field of CT reconstruction, and in particular relates to a CT image reconstruction method, a device and a CT system for improving 3D Gaussian splashing. Background Art

[0002] In the field of medical imaging and industrial inspection, computed tomography (CT) reconstruction technology is a vital tool that can reconstruct a three-dimensional image of the scanned object by processing a series of two-dimensional X-ray images. However, traditional CT reconstruction techniques, such as the Filtered Back Projection (FBP) algorithm, often produce image artifacts and reconstruction errors when processing noisy low-dose scan data.

[0003] In recent years, deep learning techniques have been introduced into CT image reconstruction, especially methods based on convolutional neural networks (CNNs), which have shown certain advantages in improving image quality and reducing noise. However, the reliance of these methods on a large amount of labeled data, the complexity of the training process, and the lack of model generalization ability limit their application in specific medical scenarios. In addition, although neural implicit representation-based methods such as Neural Radiance Fields (NeRF) have potential in simulating complex optical properties and geometric structures, the high computational complexity of the reconstruction process and the long training time have also seriously hindered their application in real-time or near-real-time medical imaging.

[0004] In view of this, 3D Gaussian Splatting (3DGS) technology provides an innovative solution for CT reconstruction. By spatially mapping and smoothing voxel data through a Gaussian kernel, 3DGS technology can effectively suppress noise and reduce artifacts while maintaining the clarity of edges and details, which is difficult to balance in traditional and deep learning methods. This technology does not rely on large-scale training data, and the calculation process is simpler and more efficient. It is suitable for the needs of rapid reconstruction, while providing higher stability and interpretability, which is particularly important for clinical diagnosis. However, the transparency blending method used in the rendering process of traditional 3DGS technology mainly relies on a simple hybrid overlay strategy, which fails to fully consider the overlapping relationship between different tissue structures in CT slices, resulting in insufficient rendering effects when processing medical images with complex tissue overlap, especially in terms of image details and structural visualization. Summary of the invention

[0005] In summary, in the face of the shortcomings of existing CT reconstruction technology, the present invention proposes a CT image reconstruction method, device and CT system with improved 3D Gaussian splattering. The present invention combines the advantages of 3DGS and OIT based on order-independent transparent rendering, and can provide fast, efficient and stable image reconstruction capabilities without relying on a large amount of training data, while ensuring the high quality and accuracy of the reconstructed image. This method can effectively handle the transparency mixing problem and correctly render the overlap of various tissue structures in CT images, thereby significantly improving the visualization quality of the reconstructed image and the accuracy of clinical diagnosis, which can not only significantly improve the usability of low-dose CT imaging, but also bring revolutionary improvements to the field of medical imaging.

[0006] In a first aspect of the present invention, the present invention provides a CT image reconstruction method with improved 3D Gaussian splashing, comprising the following steps:

[0007] S1. Acquire multiple frames of two-dimensional X-ray images, where the multiple frames of two-dimensional X-ray images are obtained by an X-ray imaging device emitting X-rays to a medical object at multiple arbitrary imaging angles;

[0008] S2, performing three-dimensional reconstruction on each frame of two-dimensional X-ray image to generate three-dimensional point cloud data and camera position information; the three-dimensional point cloud data and camera position information are used to indicate the three-dimensional geometric information of the medical object;

[0009] S3: Generate a two-dimensional plane of each frame of two-dimensional X-ray image using 3D Gaussian splashing according to the three-dimensional point cloud data of each frame of two-dimensional X-ray image and the camera position information;

[0010] S4, performing adaptive rasterization processing on the data projected onto the two-dimensional plane to generate two-dimensional image data;

[0011] S5. Perform adaptive rendering processing on the rasterized two-dimensional image data to generate a CT reconstructed image.

[0012] In a second aspect of the present invention, the present invention provides a CT image reconstruction device with improved 3D Gaussian splashing, comprising:

[0013] An acquisition unit, used for acquiring a plurality of two-dimensional X-ray images, wherein the plurality of two-dimensional X-ray images are obtained by an X-ray imaging device emitting X-rays to a medical object at a plurality of arbitrary imaging angles;

[0014] a segmentation unit, configured to perform three-dimensional reconstruction on each frame of two-dimensional X-ray image to generate three-dimensional point cloud data and camera position information; the three-dimensional point cloud data and camera position information are used to indicate three-dimensional geometric information of the medical object; generate a two-dimensional plane of each frame of two-dimensional X-ray image using 3D Gaussian splashing according to the three-dimensional point cloud data and camera position information of each frame of two-dimensional X-ray image; and perform adaptive rasterization processing on the data projected onto the two-dimensional plane to generate two-dimensional image data;

[0015] The reconstruction unit is used to perform adaptive rendering processing on the rasterized two-dimensional image data to generate a CT reconstructed image.

[0016] In a third aspect of the present invention, the present invention provides a CT system for scanning by X-rays and outputting a two-dimensional X-ray image, comprising:

[0017] X-ray imaging equipment, which scans with X-rays to obtain a two-dimensional X-ray image;

[0018] A CT image reconstruction device, such as the CT image reconstruction device according to the second aspect of the present invention;

[0019] The CT image output device outputs the CT image reconstructed by the CT image reconstruction device.

[0020] Compared with the prior art, the advantages of the present invention are:

[0021] Many limitations of existing CT reconstruction technology are overcome. First, the present invention introduces a 3D Gaussian kernel to smooth the spatial data, and combines the weighted mixed order-independent transparent rendering technology to perform adaptive transparency mixing, effectively suppressing noise and reducing artifacts, while achieving higher-quality transparency processing in the visualization of image details and tissue structures, ensuring the clarity of edges and details. Secondly, compared with the traditional reconstruction method based on filtered back projection (FBP) and deep learning, the present invention does not need to rely on large-scale annotated data sets, significantly reducing the computational complexity and training costs. The weighted mixed order-independent transparent rendering technology also does not require deep sorting of transparency data, thereby further simplifying the processing flow and improving rendering efficiency. In addition, the improved 3D Gaussian splashing technology after weighted mixed order-independent transparent rendering shows higher efficiency and real-time performance in the image reconstruction process, meets the needs of rapid reconstruction, and is particularly suitable for high-quality reconstruction in low-dose scanning scenarios. Finally, the present invention improves the stability and interpretability of the reconstructed image, so that the image can accurately reflect the relationship between different tissues, provides reliable image support for clinical diagnosis, and greatly improves the overall performance and application value of the medical imaging system. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1This is a flow chart of a CT image reconstruction method in an embodiment of the present invention;

[0023] Figure 2 It is a schematic diagram of the structure of a CT image reconstruction device according to an embodiment of the present invention;

[0024] Figure 3 A schematic diagram of the structure of a CT image reconstruction device according to a preferred embodiment of the present invention;

[0025] Figure 4 FIG. 4 is a schematic diagram of a CT system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0027] At present, the transparency blending method used in the rendering process of traditional 3DGS technology mainly relies on a simple blending overlay strategy, which fails to fully consider the overlapping relationship between different tissue structures in CT slices, resulting in insufficient rendering effects when processing medical images with complex tissue overlap, especially in terms of image details and structural visualization. In the face of the shortcomings of existing CT reconstruction technology, it is urgent to develop a new type of CT reconstruction technology that combines the advantages of 3DGS technology and can provide fast, efficient and stable image reconstruction capabilities without relying on a large amount of training data, while ensuring the high quality and accuracy of the reconstructed image. This solution can not only significantly improve the availability of low-dose CT imaging, but also bring revolutionary improvements to the field of medical imaging.

[0028] The present invention proposes a CT image reconstruction method with improved 3D Gaussian splashing, such as Figure 1 As shown, the method includes the following contents:

[0029] S1. Acquire multiple frames of two-dimensional X-ray images, where the multiple frames of two-dimensional X-ray images are obtained by an X-ray imaging device emitting X-rays to a medical object at multiple arbitrary imaging angles;

[0030] In some embodiments, two-dimensional X-ray images at multiple angles are acquired by an X-ray imaging device, and denoising, contrast adjustment and geometric correction are performed on the images to improve image quality.

[0031] In some embodiments, in a clinical environment, an X-ray imaging device is used to perform multi-angle imaging of a medical object. The medical object is usually a part of the human body, such as the head or chest, which is fixed on a scanning platform. The X-ray imaging device can perform rotational imaging around the medical object, usually selecting to obtain two-dimensional X-ray images from 20 to 30 different angles, generating an image at each angle, and all images record the X-ray diffraction information of the object at different angles. These images will be used for subsequent three-dimensional reconstruction processing.

[0032] In specific implementation, the system is set to perform imaging every 12 degrees, for a total of 30 scans. The images acquired at each angle are saved in DICOM format, and the image resolution is usually 1024×1024 pixels. Each image contains complete grayscale information to reflect the different densities of tissue structures.

[0033] In some embodiments, in order to ensure the image quality of the two-dimensional X-ray image to be reconstructed and remove noise points that may be caused by imaging device noise or external interference, a median filtering algorithm is applied to each frame of the two-dimensional X-ray image for denoising. The median filtering method is commonly used in medical image processing to reduce noise while protecting the edge structure details of the image.

[0034] In some embodiments, the two-dimensional X-ray image after denoising may have large differences in grayscale values, so it needs to be normalized. The purpose of normalization is to standardize the pixel values ​​of the two-dimensional X-ray image to the range of [0,1] to ensure the consistency of grayscale values ​​between different two-dimensional X-ray images, which is convenient for subsequent three-dimensional reconstruction operations. In implementation, the minimum and maximum grayscale values ​​in each two-dimensional X-ray image are first calculated, and then the normalization formula is applied to each pixel to map its grayscale value to the range of 0 to 1.

[0035] For a two-dimensional X-ray image, the pixel values ​​of the denoised image range from 50 to 2000. After normalization, all pixel values ​​will be mapped to the range of 0 to 1, so that the data standards in subsequent processing steps will remain consistent.

[0036] S2, performing three-dimensional reconstruction on each frame of two-dimensional X-ray image to generate three-dimensional point cloud data and camera position information; the three-dimensional point cloud data and camera position information are used to indicate the three-dimensional geometric information of the medical object;

[0037] In some embodiments, the present embodiment can use the structure from motion (SfM) algorithm to perform three-dimensional reconstruction on the preprocessed two-dimensional image, generate sparse point cloud data and camera position information, and provide three-dimensional geometric information of the medical object. Through the essential matrix E, the camera pose matrix [R|t] is further calculated, where R is the rotation matrix of the camera and t is the translation vector. This information is used to describe the relative positions between different two-dimensional X-ray images.

[0038] After obtaining the camera's position and posture, a 3D sparse point cloud is generated through triangulation. The specific operations are as follows:

[0039] For the feature point u in image 1 i and the corresponding point u in image 2 i′ , using the camera's projection matrices P1 and P2 for triangulation, the mathematical formula is:

[0040]

[0041] Among them, X i is a point in three-dimensional space, λ1 and λ2 are the corresponding depth parameters, indicating the depth of the point in space.

[0042] Through triangulation, the position of each pair of matching feature points in three-dimensional space is calculated to form a sparse three-dimensional point cloud. These point clouds are automatically optimized and error points are removed to make the point cloud more accurate.

[0043] In practice, after processing, X-ray images of medical objects such as the head or chest generate data containing thousands to tens of thousands of sparse three-dimensional points. These point clouds accurately reflect the geometric structure of the object, especially the surface contour and density changes of the object at different imaging angles. The generated point cloud data and camera pose information can be used for subsequent three-dimensional model reconstruction and medical analysis.

[0044] Through the preprocessing of the image in S1, feature points can be detected more accurately, and accurate sparse point clouds can be generated through camera pose estimation, ensuring the reliability and accuracy of the reconstruction of the three-dimensional geometric information of the medical object.

[0045] In some preferred embodiments, after performing three-dimensional reconstruction on each frame of two-dimensional X-ray image to generate three-dimensional point cloud data and camera position information, the method further includes:

[0046] Stacking the three-dimensional point cloud data into a three-dimensional volume data;

[0047] The three-dimensional volume data is resampled to generate smoothed three-dimensional point cloud data.

[0048] The three-dimensional volume data is resampled using a three-dimensional interpolation algorithm to generate resampled three-dimensional volume data; the resampled three-dimensional volume data is subjected to wavelet transformation to remove high-frequency noise information, retain low-frequency structural information, and generate smoothed three-dimensional point cloud data.

[0049] It should be noted that in this embodiment, the two-dimensional slice data is stacked into a three-dimensional volume data through resampling. During the resampling process, the target resolution is selected, and a three-dimensional interpolation algorithm (such as a B-spline interpolation algorithm) is used to smoothly calculate the three-dimensional volume data. This algorithm can avoid significant visual artifacts in CT images when the resolution is reduced. After obtaining the resampled three-dimensional voxel data, the entire voxel data is subjected to wavelet transformation to identify and remove high-frequency noise while retaining the structural information of the low-frequency part. After that, an inverse wavelet transformation is performed to restore the processed three-dimensional voxel data. This ensures that the three-dimensional voxel data finally obtained before 3D Gaussian splatting will have high quality and remove noise.

[0050] S3: Generate a two-dimensional plane of each frame of two-dimensional X-ray image using 3D Gaussian splashing according to the three-dimensional point cloud data of each frame of two-dimensional X-ray image and the camera position information;

[0051] In the actual implementation, the 3D sparse point cloud and camera position information generated by S2 are first used as the input data set. Next, the system gradually converts these point cloud data into 3D Gaussian kernel representation. The specific steps are as follows:

[0052] In the preferred embodiment, first select each three-dimensional point X i , and calculate the covariance matrix based on its neighborhood points. The standard for selecting neighborhood points is point X i Other points within a 5 mm radius. In the rib area, the points in the neighborhood are usually densely distributed, while the points in the soft tissue area are relatively scattered.

[0053] The covariance matrix Σ is calculated using the following formula:

[0054]

[0055] in, is the centroid position of the neighborhood, and the points in the neighborhood reflect their overall distribution through the position of the centroid. The covariance matrix Σ describes the geometric structure of the point cloud in the local area.

[0056] Next, we perform eigenvalue decomposition on the covariance matrix Σ:

[0057] Σ=QΛQ T (3)

[0058] Among them, Q is the eigenvector matrix of the covariance matrix, which represents the main directions of the three-dimensional point cloud, and Λ is a diagonal matrix containing eigenvalues. By analyzing the eigenvalues ​​λ1, λ2, and λ3, the system can determine the axis length of each Gaussian kernel. The larger the eigenvalue, the wider the distribution in that direction.

[0059] After obtaining the eigenvalues ​​and eigenvectors, the system adjusts the shape and direction of the Gaussian kernel according to the local geometric structure of each point. In specific implementation, the system uses the eigenvalues ​​to determine the axis length of the Gaussian kernel and determines the rotation direction of the Gaussian kernel based on the eigenvector Q.

[0060] Shape adjustment: The size of the eigenvalues ​​λ1, λ2, and λ3 determines the proportional relationship of the three axes of the Gaussian kernel. For the rib region, the eigenvalue λ1 is usually large, indicating that the geometric structure of the region is long and narrow; while for the soft tissue region, the eigenvalues ​​may be relatively close, indicating that the geometric shape is more symmetrical and approximately spherical.

[0061] Orientation: The Gaussian kernel is rotated using the eigenvector Q to align with the main direction of the point cloud. The Gaussian kernel for the rib region is stretched along the curvature of the rib to accurately represent its structure.

[0062] To realize the visualization of three-dimensional scenes, the system needs to assign transparency and color information to each Gaussian kernel.

[0063] Transparency assignment: Based on the local density of the point cloud, the system assigns transparency to the Gaussian kernel. For areas with dense bones, the transparency of the Gaussian kernel is lower, usually between 0.2-0.4, making these areas more prominent in the 3D image. For areas with sparse soft tissue, the transparency is higher, usually between 0.6-0.8, making these areas appear softer in the image.

[0064] Color information calculation: The system uses spherical harmonics to calculate the color information of the Gaussian kernel. In specific implementation, spherical harmonics are used to estimate the color information of different parts according to the illumination angle.

[0065] In the bone area, the color information is usually presented as grayish white; while in the soft tissue area, the color appears as light red or pink, simulating the real colors of different tissues in the human body.

[0066] In actual operation, the system projects the three-dimensional Gaussian ellipsoid generated in S3 onto a two-dimensional plane through the 3DGS technology to achieve the conversion of three-dimensional medical data into two-dimensional images. This step mainly calculates the projection area of ​​the three-dimensional Gaussian kernel, maps it to the two-dimensional image coordinate system, and generates a two-dimensional projection result for subsequent medical analysis and visualization.

[0067] In the specific implementation, the system first uses the three-dimensional Gaussian kernel generated by S3, whose position and shape are determined by the three-dimensional covariance matrix Σ3D and spatial position X i In order to achieve 3D to 2D mapping, the system needs to project these 3D coordinates into a 2D image space.

[0068] First, set the perspective projection matrix P, which is expressed in mathematical form as follows:

[0069]

[0070] Among them, f x and f y are the focal length parameters of the two-dimensional X-ray image, c x and c y is the coordinate of the center point of the image. Through this matrix, the three-dimensional point X i will be converted to 2D coordinates.

[0071] The system then uses the perspective projection matrix P to transform the three-dimensional point X i Projected onto the two-dimensional image plane, the formula is:

[0072] x i =PX i (5)

[0073] Among them, X i Represents the coordinates of a three-dimensional point, x i Represents the coordinates projected onto a two-dimensional plane.

[0074] In order to correctly map the shape of the three-dimensional Gaussian kernel to the two-dimensional image, the system uses 3DGS technology according to the three-dimensional covariance matrix Σ 3D Compute the two-dimensional covariance matrix Σ 2D , thereby accurately describing the distribution of the Gaussian kernel in the two-dimensional image.

[0075] Calculation of Jacobian Matrix: The core of 3DGS technology is to map the three-dimensional Gaussian kernel to the two-dimensional plane through affine transformation. Specifically, the system uses the Jacobian matrix J to linearize the mapping transformation from three-dimensional space to two-dimensional plane. The Jacobian matrix J is derived from the local linear transformation of the projection matrix P and is used to capture the direction and shape changes of the three-dimensional ellipsoid.

[0076] Calculation of the two-dimensional covariance matrix: Based on the Jacobian matrix J and the three-dimensional covariance matrix Σ 3D , the system calculates the two-dimensional covariance matrix Σ 2D , the formula is:

[0077] Σ 2D =JΣ 3D J T (6)

[0078] Among them, Σ 2DThe two-dimensional covariance matrix represents the shape of the Gaussian kernel distribution on the two-dimensional image plane, and describes the expansion direction and size of the Gaussian kernel in the two-dimensional plane. J represents the Jacobian matrix, which represents the affine transformation from three-dimensional to two-dimensional space. The Jacobian matrix plays a role of linear approximation in the projection process, and maps the changes in three dimensions to the two-dimensional plane through local linearization. Σ 3D The three-dimensional covariance matrix representing the shape of the Gaussian kernel distribution in three-dimensional space reflects the extension direction and shape of the Gaussian kernel in three-dimensional space.

[0079] The two-dimensional covariance matrix represents the shape and direction of the Gaussian kernel in the two-dimensional plane, and reflects the extension direction and size of the ribs in the image.

[0080] After calculating the 2D covariance matrix, the system then maps the 3D Gaussian kernel to the 2D image plane to generate a 2D Gaussian kernel projection area. This process is done using the Gaussian projection formula:

[0081]

[0082] Among them, G(x i ) represents the distribution value of the Gaussian kernel on the two-dimensional plane, which represents the point x in the image plane. i The shape and intensity of the Gaussian distribution around it. x represents the coordinates of any point in the two-dimensional plane, which is used to represent the position on the plane. i Represents the coordinates of the center point of the Gaussian kernel after projection on the two-dimensional plane. The three-dimensional point X is transformed by the projection matrix P. i Converted from. Represents the inverse matrix of the two-dimensional covariance matrix, which is used to calculate the difference between point x and the center point x of the Gaussian kernel i The distance and direction between them change.

[0083] This formula describes the distribution shape of the Gaussian kernel in a two-dimensional image. The two-dimensional Gaussian projection of the rib area will present an elongated elliptical shape, which is consistent with the actual shape of the rib.

[0084] Through step S3, the system successfully projects the 3D Gaussian kernel onto the 2D image plane, generating a 2D image that reflects the 3D structure of the medical object. This image can be used by doctors for clinical diagnosis and anatomical analysis, helping them to understand the patient's internal structure more intuitively.

[0085] S4, performing adaptive rasterization processing on the data projected onto the two-dimensional plane to generate two-dimensional image data;

[0086] In some embodiments, data mapped to a two-dimensional plane is rasterized, barycentric coordinates are interpolated, and depth is perspective-corrected interpolated.

[0087] In S3, the Gaussian kernel has been mapped to a two-dimensional plane through the splattering technique. The main task of S4 is to rasterize these mapped two-dimensional data and generate the final two-dimensional image data in the form of pixels. The key to rasterization is to calculate the barycentric coordinates and then perform perspective correction interpolation on the depth to obtain the correct depth value, which is convenient for executing the Weighted Blended OIT algorithm in S5 to obtain the correct rendering effect.

[0088] In actual operation, in order to speed up the rasterization process, the system first divides the image space into multiple 16×16 grid blocks. Each grid block contains 256 pixels, which is convenient for parallel computing. The system determines the grid block location for each Gaussian point. The grid blocks are divided according to the (x, y) coordinates of the Gaussian points in the two-dimensional image space, and each Gaussian point is associated with its corresponding grid block.

[0089] A thread block is allocated to each grid block in the GPU. Each thread block will be responsible for processing all pixels within a grid block and loading the list of all Gaussian points in the grid block into the shared memory of the thread block. In this way, all Gaussian point data intersecting the grid block can be efficiently accessed and processed.

[0090] For each grid block in chest CT image reconstruction, the Gaussian points in the reconstruction space are combined according to adjacent positions, and every three adjacent Gaussian points form a triangle through primitive assembly. For each triangle, the centroid coordinates of any pixel point in the screen space relative to the three vertices of the triangle are calculated.

[0091] In the specific implementation of chest CT image reconstruction, the calculated barycentric coordinates are used to calculate the precise depth value of the target pixel using the perspective correction interpolation method to ensure that the depth information of the reconstructed image can accurately reflect the three-dimensional anatomical structure of the chest cavity. The calculated barycentric coordinates are used to calculate the correct depth value of the pixel through the adaptive perspective correction interpolation formula. Based on the correct depth value of each pixel, two-dimensional image data is generated.

[0092] The specific depth interpolation formula is:

[0093]

[0094] Where D is the depth value of the target pixel after perspective correction, D A , D B , D C are the depth values ​​of the three vertices of the triangle formed by the Gaussian points, b1, b2, and b3 are the weight coefficients calculated by the barycentric coordinates, and a1(ξ), a2(ξ), and a3(ξ) represent the X-ray attenuation coefficients at different angles at position ξ.

[0095] It should be noted that by combining the X-ray attenuation coefficients at different angles, the effect of tissue density on CT image reconstruction when observed from different angles can be more accurately reflected. For example, when performing 3D reconstruction of a tumor or chest, the X-ray attenuation characteristics of different tissues will directly affect the accuracy of the reconstruction and the image quality.

[0096] Through step S4, the system successfully rasterizes the data projected onto the two-dimensional plane, calculates the correct depth value of each pixel in the chest CT image through perspective correction interpolation, and generates two-dimensional image data representing the chest CT image.

[0097] S5. Perform adaptive rendering processing on the rasterized two-dimensional image data to generate a CT reconstructed image.

[0098] In some embodiments, this embodiment combines rasterized two-dimensional data with Weighted Blended OIT for transparency blending to generate a medical CT image for medical diagnosis and analysis.

[0099] In S4, the system has rasterized the 3D data into 2D pixel data. The goal of S5 is to use the Weighted Blended OIT blending technology to perform weighted accumulation of the color and transparency of each pixel, and finally generate a rendered image for medical diagnosis and analysis. This step will perform a blending calculation of color and transparency pixel by pixel, optimize the rendering process, and generate the final medical image.

[0100] In specific implementation, the system processes each pixel of the chest CT image one by one, and calculates the weight for each Gaussian point that intersects the pixel. The weight function that achieves the best effect in implementation is as follows:

[0101]

[0102] in, The normalization factor ξ represents the corrected depth value, α represents the transparency, d(ξ) represents the voxel density at the pixel depth position ξ, and m(ξ) represents the mask value of a specific structure (such as the lungs), based on the density weight. These weights are used to calculate the color value and transparency value of each pixel in the chest CT image, and the final color C f The calculation formula is as follows:

[0103] C f (x,y)=C·(1-exp(-∑(K*(log(1-α·w′)))(x,y)))+C0·exp(-∑(K*(log(1-α·w′))))(x,y)) (10)

[0104] Among them, C f(x, y) is the final color of the CT reconstructed image, (x, y) is the image pixel position, C is the color value of each pixel in the original CT image, α is the transparency of the pixel, C0 is the background color, w′ is an adaptive weight function related to depth, transparency and density; K represents the convolution kernel, which is used to weight the pixel contribution in the neighborhood.

[0105] When accumulating transparency, the system will continuously check whether the transparency of each pixel reaches the set threshold of 0.95. When the accumulated transparency of a pixel reaches 0.95, the system assumes that the pixel is completely opaque and subsequent Gaussian points no longer affect the pixel. At this point, the system will skip further processing of the pixel, thereby reducing the amount of calculation and improving rendering efficiency.

[0106] When the color and transparency of all pixels are calculated, the system stores the results in the frame buffer and generates the final chest CT image. The size of the frame buffer is consistent with the resolution of the image (1024×1024), and each pixel stores its final color value. The system converts the data in the frame buffer into a standard image format for display, storage, or further medical analysis.

[0107] It is understandable that the traditional 3DGS algorithm needs to perform depth sorting when blending transparency. Transparency blending is performed based on depth sorting. This situation does not fully consider the overlapping relationship of different complex tissue structures in CT slices, which will lead to rendering errors. An improved rendering method is proposed based on the Order-Independent Transparency (OIT) technology to correctly render the overlap of various tissue structures in CT images. This algorithm can not only correctly render CT images, but also further improve efficiency without depth sorting. In this way, the final rendered image clearly shows the anatomical structure in the chest CT scan. In the rendered image, the rib area is displayed as a white area and the soft tissue is displayed as a light red area. Doctors can intuitively see the fracture location of the ribs or the abnormal characteristics of the lungs through the image.

[0108] It should be noted that in the processing of medical CT images in this embodiment, the input data is three-dimensional volume data, usually a set of voxels consisting of multiple slices, and the output is a reconstructed three-dimensional volume image or a corresponding two-dimensional slice. This process not only focuses on the detailed expression of the image, but also requires accurate restoration of physical properties and internal structures. In medical CT image reconstruction, the reconstruction goal is to provide accurate anatomical structure representation and physical property display. The output image is used for clinical diagnosis and medical analysis. This process requires special attention to the accuracy of details, contrast, and noise reduction. The output is not only a two-dimensional slice image, but also requires the ability to provide multi-dimensional visualization effects.

[0109] Figure 2 FIG. 4 is a schematic diagram of the structure of a CT image reconstruction device according to an embodiment of the present invention. Figure 2 As shown, including:

[0110] An acquisition unit 201 is used to acquire multiple frames of two-dimensional X-ray images, where the multiple frames of two-dimensional X-ray images are obtained by an X-ray imaging device emitting X-rays to a medical object at multiple arbitrary imaging angles;

[0111] The segmentation unit 202 is used to perform three-dimensional reconstruction on each frame of two-dimensional X-ray image to generate three-dimensional point cloud data and camera position information; the three-dimensional point cloud data and camera position information are used to indicate the three-dimensional geometric information of the medical object; according to the three-dimensional point cloud data and camera position information of each frame of two-dimensional X-ray image, use 3D Gaussian splashing to generate a two-dimensional plane of each frame of two-dimensional X-ray image; perform adaptive rasterization processing on the data projected onto the two-dimensional plane to generate two-dimensional image data;

[0112] The reconstruction unit 203 is used to perform adaptive rendering processing on the rasterized two-dimensional image data to generate a CT reconstructed image.

[0113] Figure 3 FIG. 1 is a schematic diagram of the structure of a CT image reconstruction device according to a preferred embodiment of the present invention. Figure 3 As shown, including:

[0114] An acquisition unit 301 is used to acquire multiple frames of two-dimensional X-ray images, where the multiple frames of two-dimensional X-ray images are obtained by an X-ray imaging device emitting X-rays to a medical object at multiple arbitrary imaging angles;

[0115] The calibration unit 302 is used to stack the three-dimensional point cloud data into three-dimensional volume data; and resample the three-dimensional volume data to generate smoothed three-dimensional point cloud data.

[0116] The segmentation unit 303 is used to perform three-dimensional reconstruction on each frame of two-dimensional X-ray image to generate three-dimensional point cloud data and camera position information; the three-dimensional point cloud data and camera position information are used to indicate the three-dimensional geometric information of the medical object; according to the three-dimensional point cloud data and camera position information of each frame of two-dimensional X-ray image, use 3D Gaussian splashing to generate a two-dimensional plane of each frame of two-dimensional X-ray image; perform adaptive rasterization processing on the data projected onto the two-dimensional plane to generate two-dimensional image data;

[0117] The reconstruction unit 304 is used to perform adaptive rendering processing on the rasterized two-dimensional image data to generate a CT reconstructed image.

[0118] Figure 4 FIG. 1 is a schematic diagram of the structure of a CT image system according to an embodiment of the present invention. Figure 4 As shown, including:

[0119] X-ray imaging device 401, which scans with X-rays to obtain a two-dimensional X-ray image;

[0120] The X-ray imaging device 401 may include an X-ray scanner to scan a scan object in a scan area by using X-rays. Here, the scan object is, for example, a biological body such as a human body, and may include a moving object such as a heart.

[0121] A CT image reconstruction device 402, which may be the CT image reconstruction device described above;

[0122] The CT image reconstruction device 402 is implemented by a general-purpose computer or a dedicated integrated circuit. The CT image reconstruction device 402 generates a CT image based on the two-dimensional X-ray image output by the CT scanner 401, using, for example, the CT image reconstruction method of steps S1-S5 of the embodiment of the present invention.

[0123] The CT image output device 403 outputs the CT image reconstructed by the CT image reconstruction device.

[0124] The CT image output device 403 is typically a CT image display device, which displays on a screen the CT image output by the CT image reconstruction device 402. Of course, the CT image output device 403 is not limited to a CT image display device, and may also be a data transmission interface for sending the CT image output by the CT image reconstruction device 402 through a network, a printer for printing the CT image output by the CT image reconstruction device 402, and the like.

[0125] It is understandable that the CT images output by the CT system can be used in many fields, such as scientific research, biological data acquisition, etc. In addition, the CT images can also be used as intermediate data, such as in the fields of disease diagnosis and health management.

[0126] A person skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the relevant hardware through a program, and the program can be stored in a computer-readable storage medium, which can include: ROM, RAM, disk or CD, etc.

[0127] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An improved 3D Gaussian splashing CT image reconstruction method, characterized in that: The following steps are involved: Acquire multiple frames of two-dimensional X-ray images, where the multiple frames of two-dimensional X-ray images are obtained by an X-ray imaging device emitting X-rays to a medical object at multiple arbitrary imaging angles; Perform 3D reconstruction on each frame of 2D X-ray image to generate 3D point cloud data and camera position information; The three-dimensional point cloud data and camera position information are used to indicate the three-dimensional geometric information of the medical object; Generate a two-dimensional plane of each frame of two-dimensional X-ray image using 3D Gaussian splashing according to the three-dimensional point cloud data of each frame of two-dimensional X-ray image and the camera position information; Adaptively rasterizing the data projected onto the two-dimensional plane to generate two-dimensional image data; Adaptive rendering is performed on the rasterized two-dimensional image data to generate a CT reconstructed image.

2. The CT image reconstruction method according to claim 1, characterized in that: Before generating a two-dimensional plane of each frame of the two-dimensional X-ray image using 3D Gaussian splashing according to the three-dimensional point cloud data of each frame of the two-dimensional X-ray image and the camera position information, the following steps are also included: Stacking the three-dimensional point cloud data into a three-dimensional volume data; The three-dimensional volume data is resampled to generate smoothed three-dimensional point cloud data.

3. The CT image reconstruction method according to claim 2, characterized in that: Resampling the three-dimensional volume data to generate smoothed three-dimensional point cloud data includes: resampling the three-dimensional volume data using a three-dimensional interpolation algorithm to generate resampled three-dimensional volume data; The resampled three-dimensional volume data is subjected to wavelet transformation to remove high-frequency noise information, retain low-frequency structural information, and generate smoothed three-dimensional point cloud data.

4. The CT image reconstruction method according to claim 1, characterized in that: The adaptive rasterization processing of the data projected onto the two-dimensional plane to generate two-dimensional image data comprises the following steps: According to the three-dimensional position of each Gaussian point, determine the grid block where it is located and its depth relative to the viewing plane; Assign a thread block to each grid block and load a list of all Gaussian points that intersect this grid block into the shared memory of the thread block; In each grid block, three adjacent Gaussian points are combined into triangles through primitive assembly. For each triangle, the centroid coordinates of any pixel point in the screen space are calculated; Using the calculated barycentric coordinates, the correct depth value of the pixel is calculated through adaptive perspective correction interpolation technology; Generate 2D image data based on the correct depth value for each pixel.

5. The CT image reconstruction method according to claim 4, characterized in that: The interpolation formula of the adaptive perspective correction interpolation technology is: Where D is the depth value of the target pixel after perspective correction, D A , D B , D C are the depth values ​​of the three vertices of the triangle formed by the Gaussian points, b1, b2, and b3 are the weight coefficients calculated by the barycentric coordinates, and a1(ξ), a2(ξ), and a3(ξ) represent the X-ray attenuation coefficients at different angles at position ξ.

6. The CT image reconstruction method according to claim 1, characterized in that: The adaptive rendering process of the rasterized two-dimensional image data to generate a CT reconstructed image specifically comprises the following steps: Weighted blending order-independent transparent rendering is used for transparency blending. Adaptive weights based on depth and transparency are calculated for each Gaussian point that intersects a pixel, and the color value and transparency value of the pixel are calculated using these adaptive weights. If the transparency value of a pixel reaches the set threshold, further processing of the pixel is stopped; After completing the color and transparency calculations for all pixels, the results are stored in the frame buffer to generate the final CT reconstructed image.

7. The CT image reconstruction method according to claim 6, characterized in that: The calculation formula of the final color of the CT reconstructed image is expressed as: C f (x,y)=C·(1-exp(-∑(K*(log(1-α·w′)))(x,y)))+C0·exp(-∑(K*(log(1-α·w′)))(x,y)) Among them, C f (x, y) is the final color of the CT reconstructed image, (x, y) is the image pixel position, C is the color value of each pixel in the original CT image, α is the transparency of the pixel, C0 is the background color, w′ is an adaptive weight function related to depth, transparency and density; K represents the convolution kernel, which is used to weight the pixel contribution in the neighborhood.

8. An improved 3D Gaussian splash CT image reconstruction device, characterized in that: include: An acquisition unit, used for acquiring a plurality of two-dimensional X-ray images, wherein the plurality of two-dimensional X-ray images are obtained by an X-ray imaging device emitting X-rays to a medical object at a plurality of arbitrary imaging angles; A segmentation unit, used to perform three-dimensional reconstruction on each frame of two-dimensional X-ray image to generate three-dimensional point cloud data and camera position information; The three-dimensional point cloud data and camera position information are used to indicate the three-dimensional geometric information of the medical object; According to the three-dimensional point cloud data and camera position information of each frame of the two-dimensional X-ray image, a two-dimensional plane of each frame of the two-dimensional X-ray image is generated by using 3D Gaussian splashing; the data projected onto the two-dimensional plane is adaptively rasterized to generate two-dimensional image data; The reconstruction unit is used to perform adaptive rendering processing on the rasterized two-dimensional image data to generate a CT reconstructed image.

9. The CT image reconstruction device according to claim 8, characterized in that: Also includes: A calibration unit, used for stacking the three-dimensional point cloud data into a three-dimensional volume data; The three-dimensional volume data is resampled to generate smoothed three-dimensional point cloud data.

10. A CT system that scans with X-rays and outputs two-dimensional X-ray images, characterized in that: include: X-ray imaging equipment, which scans with X-rays to obtain a two-dimensional X-ray image; A CT image reconstruction device, a CT image reconstruction device as claimed in claim 8 or 9; The CT image output device outputs the CT image reconstructed by the CT image reconstruction device.

Images