X-ray three-dimensional reconstruction method and system based on geometric prior and perspective alignment

Abstract

The invention belongs to the field of computer vision and artificial intelligence, and provides an X-ray three-dimensional reconstruction method and system based on geometric prior and perspective alignment, and the method comprises the steps: carrying out the projection of CT (Computed Tomography) volume data, generating a multi-angle original X-ray projection image, and carrying out the preprocessing of the multi-angle original X-ray projection image, and obtaining a multi-angle X-ray projection image; performing first initialization on the multi-angle X-ray projection image to obtain an initialized Gaussian point cloud; based on the initialized Gaussian point cloud, performing iterative training of a first set number of times by using a radiation Gaussian splashing model of perspective alignment, and optimizing Gaussian attributes in combination with a loss function after each time of training to obtain a three-dimensional Gaussian point cloud after iteration of the first set number of times; performing second initialization on the three-dimensional Gaussian point cloud iterated for the first set number of times to obtain a re-initialized Gaussian point cloud; and inputting the re-initialized Gaussian point cloud into a perspective alignment radiation Gaussian splash model for iterative training, and iterating for a second set number of times to obtain a three-dimensional reconstruction result.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision and artificial intelligence technology, and specifically relates to an X-ray three-dimensional reconstruction method and system based on geometric prior and perspective alignment. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] With the rapid development of medical imaging technology, three-dimensional reconstruction of X-ray scenes has become a key research direction for improving the accuracy of preoperative diagnosis. It can not only accurately restore the geometric form of complex human structures and ensure the physical authenticity of the modeling, but also significantly enhance the ability to display details (such as bone microstructure and soft tissue boundaries). X-ray new perspective synthesis technology can solve the challenges in multi-perspective projection, such as perspective projection mismatch, medium-dependent attenuation effects, and insufficient efficiency of Gaussian distribution initialization, and promote the innovation of medical imaging modeling technology. More importantly, this technology provides new tools for preoperative guidance, radiotherapy planning and quantitative pathology analysis. Its explicit anatomical representation supports accurate geometric measurement and radiation dose calculation, thereby enabling the construction of intelligent diagnosis and treatment systems. By optimizing the X-ray new perspective synthesis framework, we can achieve high-fidelity and high-efficiency three-dimensional reconstruction, laying a technical foundation for the research and development of precision medicine and intelligent medical equipment.

[0004] In recent years, with the introduction of Neural Radiation Field (NeRF) technology, X-ray 3D reconstruction methods based on implicit neural representation have made significant progress. However, these methods suffer from two inherent drawbacks: First, training a single scene requires several hours of GPU computing, making it difficult to meet real-time clinical needs; second, implicit representation lacks explicit anatomical topological information, resulting in a lack of interpretability for quantitative analyses such as radiation dose calculation and lesion volume measurement. To address this, researchers have turned their attention to explicit 3D Gaussian sputtering, which provides explicit geometric representation while ensuring real-time performance through differentiable point cloud rendering, becoming a new paradigm for medical image reconstruction. However, existing Gaussian sputtering-based methods often suffer from geometric mismatch in complex anatomical scenes, such as overlapping areas of dense tissue or soft tissue boundaries. For example, the limited representation capability of traditional 3D Gaussian models leads to blurred bone edges or severe artifacts at organ boundaries. Existing methods typically rely on uniform sampling strategies to initialize the Gaussian distribution, which struggles to capture the curvature characteristics and topological relationships of anatomical structures. Reconstruction accuracy is significantly reduced, especially under sparse viewing angles or low-dose imaging conditions. The simplification of the attenuation mechanism also leads to radiation intensity distortion in multi-layer tissue penetration scenarios. These problems seriously limit the reliability of the reconstruction results in clinical quantitative analysis (such as radiation dose assessment or lesion volume measurement).

[0005] In summary, existing Gaussian primitives have limited representation capabilities in complex X-ray scenes, making it difficult for primitive generation to closely fit perspective structures, resulting in insufficient geometric modeling accuracy. Traditional uniform sampling initialization strategies ignore the density distribution patterns of biological tissues, resulting in redundant Gaussian distributions and a lack of guidance for primitive generation. Existing radiation attenuation models oversimplify physical mechanisms, using homogeneous media or a single attenuation coefficient, making it difficult to accurately model the nonlinear attenuation effects of multilayered heterogeneous tissue penetration. The dynamic coupling between medium type and attenuation coefficient is not explicitly modeled, resulting in distorted surface modeling at structural boundaries, severely impacting reconstruction quality. Summary of the Invention

[0006] In order to solve the above problems, the present invention proposes an X-ray three-dimensional reconstruction method and system based on geometric prior and perspective alignment. The present invention uses a curvature-aware re-initialization module to adaptively adjust the Gaussian primitive density in combination with the curvature characteristics of the anatomical surface, densely sample high-curvature areas such as bone edges and organ boundaries, and sparsely distribute non-critical areas to avoid redundant calculations caused by traditional uniform sampling. At the same time, a parameterized two-dimensional elliptical disk projection model is further designed to explicitly align perspective geometric constraints and model medium-dependent radiation attenuation effects to ensure geometric consistency under multiple perspectives. This breakthrough provides high-precision and high-efficiency imaging support for preoperative guidance and radiotherapy dose planning, and promotes the evolution of intelligent diagnosis and treatment systems towards clinical practicality.

[0007] According to some embodiments, a first solution of the present invention provides an X-ray 3D reconstruction method based on geometric prior and perspective alignment, which adopts the following technical solutions:

[0008] X-ray 3D reconstruction method based on geometric prior and perspective alignment, including:

[0009] Projecting the CT volume data to generate multi-angle original X-ray projection images and preprocessing them to obtain multi-angle X-ray projection images;

[0010] The angle-attitude cube uniform initialization strategy is used to perform the first initialization on the multi-angle X-ray projection image to obtain the initialized Gaussian point cloud.

[0011] Based on the initialized Gaussian point cloud, a perspective-aligned radiometric Gaussian splatter model is used to perform a first set number of iterative trainings. After each training, the Gaussian properties are optimized using a loss function to obtain a three-dimensional Gaussian point cloud after the first set number of iterations.

[0012] A reinitialization strategy based on surface curvature characteristics is used to perform a second initialization on the three-dimensional Gaussian point cloud after the first set number of iterations to obtain a reinitialized Gaussian point cloud;

[0013] The reinitialized Gaussian point cloud is input into the perspective-aligned radiation Gaussian splash model for iterative training. After the second set number of iterations, the three-dimensional reconstruction result is obtained.

[0014] Furthermore, the angle attitude cube uniform initialization strategy is used to perform the first initialization on the multi-angle X-ray projection image to obtain the initialized Gaussian point cloud, specifically:

[0015] Calculate the camera's intrinsic and extrinsic matrix using the X-ray scanner's parameters;

[0016] Based on the intrinsic parameter matrix and the extrinsic parameter matrix and the multi-angle X-ray projection image, the center of the cube is used as the center of the object and the origin of the world coordinate system to determine the cube that can enclose the object;

[0017] The cube is divided into n grids, and points are uniformly sampled at intervals a within the grid to obtain an initialized Gaussian point cloud.

[0018] Furthermore, based on the initialized Gaussian point cloud, the perspective-aligned radiometric Gaussian splatter model is used to perform a first set number of iterative trainings, and after each training, the Gaussian properties are optimized in combination with the loss function to obtain a three-dimensional Gaussian point cloud after the first set number of iterations, specifically:

[0019] Convert the initialized 3D Gaussian point cloud into a 2D Gaussian point cloud, and determine the pixels occupied by each 2D Gaussian through the covariance matrix and position of the 2D Gaussian point cloud;

[0020] Calculate the color of each pixel based on the two-dimensional elliptical disk primitive representation model, and calculate the Gaussian contribution of each pixel in parallel to obtain the final Gaussian training image;

[0021] The loss function is calculated by comparing the Gaussian training image with the original X-ray projection image at the same angle. The Gaussian properties are optimized by backpropagation of the loss function. The optimized Gaussian properties are used to guide the 3D reconstruction of the object, and a 3D Gaussian point cloud is obtained after one iteration.

[0022] The three-dimensional Gaussian point cloud after one iteration is re-input into the perspective-aligned radiation Gaussian splash model, and the iterative training process is repeated. After iterating to a first set number of times, a three-dimensional Gaussian point cloud after the first set number of iterations is obtained.

[0023] Furthermore, the color of each pixel is calculated based on the two-dimensional elliptical disk primitive representation model, and the Gaussian contribution of each pixel is calculated in parallel to obtain the final Gaussian training image, specifically:

[0024] Based on the two-dimensional elliptical disk primitive representation model, a Gaussian elliptical disk is used to represent the two-dimensional Gaussian basis element in the two-dimensional space;

[0025] Based on the Beer-Lambert law, the intensity attenuation of X-rays when penetrating the medium is modeled, and the attenuation factor of each two-dimensional Gaussian basis element is obtained;

[0026] Based on the order of the two-dimensional Gaussian primitives in each pixel, the color, attenuation factor, opacity of each two-dimensional Gaussian primitive and the opacity of all previous two-dimensional Gaussian primitives are alpha blended to obtain the color of each pixel, and the Gaussian contribution of each pixel is calculated in parallel to obtain the final Gaussian training image.

[0027] Furthermore, the re-initialization strategy based on the surface curvature feature is adopted to perform a second initialization on the three-dimensional Gaussian point cloud after the first set number of iterations to obtain a re-initialized Gaussian point cloud, specifically:

[0028] After the first set number of iterations k, the three-dimensional Gaussian point cloud P k , calculate each point p i Normalized local curvature characteristics;

[0029] Based on the normalized local curvature features and random threshold screening, the three-dimensional Gaussian point cloud after the first set number of iterations is downsampled to obtain a reinitialized point cloud.

[0030] Furthermore, the loss function is specifically:

[0031]

[0032] Among them, the L1 loss directly minimizes the pixel absolute error between the Gaussian training image and the original X-ray projection image; the SSIM loss measures the similarity of the overall structure and texture between the Gaussian training image and the original X-ray projection image; γ is a weight coefficient used to balance the contribution of the two loss terms.

[0033] According to some embodiments, a second solution of the present invention provides an X-ray 3D reconstruction system based on geometric prior and perspective alignment, which adopts the following technical solutions:

[0034] X-ray 3D reconstruction system based on geometric prior and perspective alignment, including:

[0035] a data projection processing module configured to project the CT volume data to generate multi-angle original X-ray projection images and pre-process the images to obtain multi-angle X-ray projection images;

[0036] An initialization module is configured to use an angle-pose cube uniform initialization strategy to perform a first initialization on the multi-angle X-ray projection image to obtain an initialized Gaussian point cloud;

[0037] a three-dimensional model iterative training module configured to perform a first set number of iterative trainings based on the initialized Gaussian point cloud using a perspective-aligned radiometric Gaussian splatter model, and optimize Gaussian properties in combination with a loss function after each training, thereby obtaining a three-dimensional Gaussian point cloud after the first set number of iterations;

[0038] a reinitialization module configured to employ a reinitialization strategy based on surface curvature features to perform a second initialization on the three-dimensional Gaussian point cloud after iterating a first set number of times, thereby obtaining a reinitialized Gaussian point cloud;

[0039] The three-dimensional reconstruction module is configured to input the reinitialized Gaussian point cloud into the perspective-aligned radiation Gaussian splash model for iterative training, and obtain a three-dimensional reconstruction result after iterating a second set number of times.

[0040] According to some embodiments, a third aspect of the present invention provides a computer-readable storage medium.

[0041] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the X-ray three-dimensional reconstruction method based on geometric prior and perspective alignment as described in the first aspect above.

[0042] According to some embodiments, a fourth aspect of the present invention provides a computer device.

[0043] A computer device comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the X-ray three-dimensional reconstruction method based on geometric prior and perspective alignment as described in the first aspect above are implemented.

[0044] According to a fifth aspect of the present invention, there is provided a computer program product or computer program according to some embodiments.

[0045] The present invention provides a computer program product or computer program, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps of the method for X-ray 3D reconstruction based on geometric priors and perspective alignment as described in the first aspect above.

[0046] Compared with the prior art, the present invention has the following beneficial effects:

[0047] The present invention relates to the fields of medical imaging and computer vision, and in particular, in the context of new X-ray view synthesis and three-dimensional reconstruction, a method for X-ray scene reconstruction based on perspective-aligned radiation Gaussian splattering is proposed. By introducing perspective structure-aware Gaussian representation optimization and curvature-driven reinitialization strategy, the geometric consistency of X-ray projection and the accuracy of radiation attenuation modeling are significantly improved. This method can not only achieve high-fidelity three-dimensional reconstruction in complex anatomical areas (such as bone edges and soft tissue boundaries), but also avoid the computational overhead of traditional structural motion recovery pipelines through efficient initialization strategies, and maintain excellent detail capture capabilities at real-time training speeds. Its explicit Gaussian representation has both interpretability and quantitative analysis capabilities, providing efficient and reliable technical support for clinical diagnosis, radiotherapy planning, and real-time pathology analysis, promoting the leap from traditional reconstruction to intelligent three-dimensional visualization in medical image processing, and helping to improve precision medical decision-making and diagnosis and treatment efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The accompanying drawings, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0049] Figure 1 is a flow chart of an X-ray 3D reconstruction method based on geometric prior and perspective alignment in an embodiment of the present invention;

[0050] Figure 2 This is a diagram of the overall architecture of the X-ray 3D reconstruction method based on geometric prior and perspective alignment in an embodiment of the present invention. DETAILED DESCRIPTION

[0051] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0052] It should be noted that the following detailed descriptions are illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present invention belongs.

[0053] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.

[0054] In the absence of conflict, the embodiments of the present invention and the features thereof may be combined with each other.

[0055] Explanation of terms:

[0056] Gaussian point cloud: refers to a collection of Gaussians.

[0057] Example 1

[0058] like Figure 1 As shown, this embodiment provides an X-ray three-dimensional reconstruction method based on geometric prior and perspective alignment. This embodiment uses the method applied to a server as an example for illustration. It is understandable that the method can also be applied to a terminal, and can also be applied to a system including a terminal, a server, and a server, and is implemented through the interaction between the terminal and the server. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network servers, cloud communications, middleware services, domain name services, security services CDN, and big data and artificial intelligence platforms. The terminal can be a smart phone, tablet computer, laptop computer, desktop computer, smart speaker, smart watch, etc., but is not limited to this. The terminal and the server can be directly or indirectly connected by wired or wireless communication, which is not limited in this application. In this embodiment, the method includes the following steps:

[0059] Projecting the CT volume data to generate multi-angle original X-ray projection images and preprocessing them to obtain multi-angle X-ray projection images;

[0060] The angle-attitude cube uniform initialization strategy is used to perform the first initialization on the multi-angle X-ray projection image to obtain the initialized Gaussian point cloud.

[0061] Based on the initialized Gaussian point cloud, a perspective-aligned radiometric Gaussian splatter model is used to perform a first set number of iterative trainings. After each training, the Gaussian properties are optimized using a loss function to obtain a three-dimensional Gaussian point cloud after the first set number of iterations.

[0062] A reinitialization strategy based on surface curvature characteristics is used to perform a second initialization on the three-dimensional Gaussian point cloud after the first set number of iterations to obtain a reinitialized Gaussian point cloud;

[0063] The reinitialized Gaussian point cloud is input into the perspective-aligned radiation Gaussian splash model for iterative training. After the second set number of iterations, the three-dimensional reconstruction result is obtained.

[0064] This embodiment proposes an X-ray 3D reconstruction method based on geometric prior and perspective alignment. The overall method architecture is as follows: Figure 2 As shown, the following steps are included:

[0065] Step S1: Projecting the CT volume data to generate a multi-angle original X-ray projection image and preprocessing it to obtain a multi-angle X-ray projection image.

[0066] Furthermore, the operation in step S1 is specifically implemented as follows:

[0067] S1-1: To comprehensively evaluate the model's performance in various complex X-ray imaging scenarios, this example collects data from public medical imaging datasets. The data includes CT volume data of human organs (such as the chest, foot, head, abdomen, and pancreas). The dataset simulates common degradation scenarios found in real X-ray scans, including low-dose noise, sparse projections, motion artifacts, and other degradation types, to ensure data diversity and clinical representativeness.

[0068] S1-2: Performing a projection operation on the CT volume data to generate simulated X-ray projection original image data, and preprocessing the multi-angle original X-ray projection image to obtain a multi-angle X-ray projection image.

[0069] During the initial construction of this method, the multi-angle X-ray projection image data can be divided into a data set with a ratio of 1:1 between the training set and the test set. The training set is used to train the implementation method of steps 2-5, and then the implementation method of steps 2-5 is tested based on the test set.

[0070] The process of step S1 is to project the CT volume data (three-dimensional model) to obtain images from different camera perspectives.

[0071] Step S2: Use the angle attitude cube uniform initialization strategy to perform the first initialization on the multi-angle X-ray projection image to obtain the initialized Gaussian point cloud, specifically:

[0072] Calculate the camera's intrinsic and extrinsic matrix using the X-ray scanner's parameters;

[0073] Based on the intrinsic parameter matrix and the extrinsic parameter matrix and the multi-angle X-ray projection image, the center of the cube is used as the center of the object and the origin of the world coordinate system to determine the cube that can enclose the scanned object;

[0074] The cube is divided into n grids, and points are uniformly sampled at intervals a within the grid to obtain an initialized Gaussian point cloud.

[0075] Furthermore, the operation in step S2 is specifically implemented as follows:

[0076] S2-1: Use the parameters of the X-ray scanner to calculate the camera's intrinsic and extrinsic matrix. The calculation formulas for the intrinsic and extrinsic matrix are:

[0077]

[0078]

[0079] Among them, M ext is the external parameter matrix, M int is the internal parameter matrix, φ is the azimuth angle of the X-ray source, L SO L is the distance from the X-ray source to the object being scanned. SD is the distance from the X-ray source to the detector, and W and H are the width and height of the detector, respectively.

[0080] S2-2: Based on the intrinsic parameter matrix and the extrinsic parameter matrix and the multi-angle X-ray projection image, the center of the cube is used as the center of the object and the origin of the world coordinate system to determine the cube that can enclose the scanned object;

[0081] A cube of size S1×S2×S3 (mm) is set up to completely surround the scanned object. The center of the cube is also the center of the object and the origin of the world coordinate system.

[0082] Step S2-3: Divide the cube into n grids, and uniformly sample points at intervals a within the grids to obtain an initialized Gaussian point cloud.

[0083] The initialized Gaussian point cloud contains many parameters describing the Gaussian shape position, including the center position, the length of the major and minor axes of the ellipse, etc. The Gaussian point cloud here is an ellipsoid in three-dimensional space, which is equivalent to using many ellipsoids to represent the scene.

[0084] This step uses the X-ray scanner's parameters to calculate the camera's intrinsic and extrinsic parameter matrices. Points are then uniformly sampled within a cube that completely encloses the scanned object to initialize the center position of the 3D Gaussian point cloud. This approach avoids the Structure from Motion (SfM) pipeline, significantly reducing training time. By skipping the time-consuming SfM algorithm, the camera parameters and the Gaussian point cloud center position can be quickly and accurately initialized, providing a good starting point for subsequent training and optimization.

[0085] Step S3: Based on the initialized Gaussian point cloud, a perspective-aligned radiometric Gaussian splatter model is used to perform iterative training for a first set number of times, and after each training, the Gaussian properties are optimized in combination with a loss function to obtain a three-dimensional Gaussian point cloud after the first set number of iterations;

[0086] Furthermore, the operation in step S3 is specifically implemented as follows:

[0087] Although traditional 3D Gaussian rendering (3DGS) performs well in natural scenes, it has inherent defects in X-ray imaging. X-ray imaging is based on the physical properties of perspective projection and radiation attenuation, requiring the rendering model to simultaneously meet the requirements of geometric alignment and accurate modeling of energy attenuation. However, due to multi-perspective geometric inconsistencies during the projection process, traditional 3D Gaussian primitives have difficulty accurately representing the layered structure of biological tissues (such as bone edges and organ boundaries), resulting in surface fractures, volumetric artifacts and other problems in the rendering results. In addition, the simplified assumptions of radiation attenuation (such as homogeneous media or fixed attenuation coefficients) used by existing methods cannot adapt to the complex distribution of density differences in biological tissues, further limiting the reconstruction accuracy.

[0088] S3-1: Convert the initialized 3D Gaussian point cloud into a 2D Gaussian point cloud, and determine the pixels occupied by each 2D Gaussian through the covariance matrix and position of the 2D Gaussian point cloud;

[0089] It can be understood that the initial three-dimensional Gaussian point cloud is transformed in the first iteration, and then after repeated iterations, the three-dimensional Gaussian point cloud obtained from each iteration training is used as the input of the perspective-aligned radiation Gaussian splash model for retraining.

[0090] S3-2: Calculate the color of each pixel based on the two-dimensional elliptical disk primitive representation model, and calculate the Gaussian contribution of each pixel in parallel to obtain the final Gaussian training image.

[0091] S3-2-1: To accurately characterize the anatomical structure in X-ray scenes, a two-dimensional elliptical disk primitive model based on physical constraints is proposed. Based on the two-dimensional elliptical disk primitive characterization model, a Gaussian elliptical disk is used to represent the two-dimensional Gaussian basis element in the two-dimensional space. Each two-dimensional Gaussian basis element is defined by a six-tuple parameter set:

[0092] Θ u,v,i ={φ i ,ω u,i ,ω v,i ,θ u,i ,θ v,i ,σ i} (3);

[0093] Among them, φ i ∈R 3 represents the spatial coordinates of the center of the i-th two-dimensional Gaussian basis element; ω u,i ,ω v,i ∈R 3 represents the orthogonal tangent vector of the local tangent plane, defining the direction of the i-th two-dimensional Gaussian basis element; θ u,i ,θ v,i ∈R + represents the learnable scaling factor of the i-th two-dimensional Gaussian basis element, which is used to control the length of the major axis of the Gaussian elliptical disk; σ iIt represents the attenuation factor of the i-th two-dimensional Gaussian basis element, that is, the attenuation factor based on the physical interaction between X-rays and biological tissues, which describes the nonlinear attenuation of the X-ray scene; wherein, the two-dimensional Gaussian point cloud is a set of Gaussian elliptical disks, and the number of Gaussians is usually millions.

[0094] S3-2-2: Based on the Beer-Lambert law, the intensity attenuation of X-rays when penetrating a medium is modeled to obtain the attenuation factor for each Gaussian basis element, including:

[0095] The Beer-Lambert law formula is as follows:

[0096]

[0097] Among them, μ E is the attenuation coefficient of the medium atomic species E, and d is the penetration depth. Therefore, for each Gaussian basis element, its attenuation factor σ i Determined by the medium type and spatial position:

[0098] σ i =1-exp(-Φ(α i ,V i -1 )|d i |) (5);

[0099] Among them, d i is the depth of the center of the i-th 2D Gaussian primitive in camera space; Φ(·) is the mixed density function, which is determined by the opacity α of the i-th 2D Gaussian primitive. i and spatial density V i -1 (inverse of volume) is calculated based on the parameter calculation method proposed by the physical formula of Beer-Lambert law.

[0100] S3-2-3: Based on the order of the two-dimensional Gaussian primitives in each pixel, alpha blend the color, attenuation factor, opacity of each two-dimensional Gaussian primitive and the opacity of all previous two-dimensional Gaussian primitives to obtain the color of each pixel, and calculate the Gaussian contribution of each pixel in parallel to obtain the final Gaussian training image.

[0101] S3-3: Compare the Gaussian training image with the original X-ray projection image at the same angle to calculate the loss function. Backpropagation of the loss function is used to optimize the Gaussian properties. The optimized Gaussian properties guide the 3D reconstruction of the object, resulting in a 3D Gaussian point cloud after one iteration. The Gaussian properties here refer to position, covariance matrix, color, attenuation factor, and opacity.

[0102] This embodiment uses L1 loss and SSIM loss to jointly optimize the geometric reconstruction error, ensuring that Gaussian generation strictly fits the complex surface structure in the X-ray scene, achieving a balance between visual effects and detail capture. The specific implementation process is as follows:

[0103] The loss function is as follows:

[0104]

[0105] Here, γ is a weight coefficient used to balance the contributions of the two loss terms. The L1 loss directly minimizes the absolute pixel error between the Gaussian training image and the original X-ray projection image, ensuring accurate reconstruction of local details (such as bone edges and soft tissue boundaries). The SSIM loss measures the similarity of the overall structure and texture between the Gaussian training image and the original X-ray projection image, avoiding the oversmoothing problem caused by relying solely on pixel error, especially preserving the coherence of anatomical structures in low-contrast areas (such as lung texture). By adjusting the γ value, a flexible trade-off between pixel accuracy and structural fidelity can be achieved.

[0106] Specifically, each Gaussian attribute (including position, covariance matrix, color, attenuation factor, and opacity) undergoes gradient calculation and iterative updates through a differentiable rendering process. The 3D Gaussian is mapped to 2D screen space, and its contribution is calculated via alpha blending to compute pixel color. Gradients are backpropagated from the pixel error of the rendered output to the Gaussian attribute parameters via the chain rule, achieving physically consistent modeling of the geometric alignment of anatomical structures and the radiation attenuation properties.

[0107] S3-4: Re-input the three-dimensional Gaussian point cloud after one iteration into the perspective-aligned radiation Gaussian splash model, repeat the iterative training process steps S3-1-S3-3, and after iterating to the first set number of times k, obtain the three-dimensional Gaussian point cloud after the first set number of iterations.

[0108] The iterative training process using the perspective-aligned radiation Gaussian splatter model starts with initializing the Gaussian point cloud, then iteratively splits, moves, rotates, and scales it up to gradually form a three-dimensional Gaussian point cloud model. Each iteration, the resulting Gaussian training image is compared with the original X-ray projection image obtained from the same angle as the object, and the loss function is calculated. If any part of the comparison does not match, optimization will continue in the next round.

[0109] During rounds of iterative training, the cube of the initialized Gaussian point cloud gradually approaches the real three-dimensional model of the object. However, if the initial initialized Gaussian point cloud is not a cube, but a prototype of the object to be reconstructed, the speed of iterative training will be greatly accelerated. However, cube initialization has to be adopted, which can skip the time-consuming SfM and quickly and accurately initialize the camera parameters and the center position of the Gaussian point cloud. This method avoids the pipeline of the motion recovery structure (SfM) algorithm, thereby significantly reducing the training time. Therefore, starting with the cube, after the first set number of training times, the prototype of the object to be reconstructed is obtained. At this time, the three-dimensional Gaussian point cloud file after the first set number of iterations is initialized for the second time, and the re-initialized Gaussian point cloud is used as the input of the perspective-aligned radiation Gaussian splatter model, so step S4 is introduced.

[0110] The initial Gaussian point cloud is represented by a perspective-aligned 2D elliptical disk primitive model, combined with medium-type-dependent attenuation mechanisms to achieve dual-domain consistency optimization of perspective structure and radiation energy distribution. The 2D elliptical disk primitive is used to explicitly align the projection plane, eliminating the geometric distortion of traditional 3D Gaussian methods. The medium-dependent attenuation factor is used to accurately simulate X-ray energy attenuation, enhancing the realism of radiation distribution.

[0111] Step S4: using a re-initialization strategy based on surface curvature features, the three-dimensional Gaussian point cloud after the first set number of iterations is initialized for the second time to obtain a re-initialized Gaussian point cloud.

[0112] After the first set number of training iterations, a second initialization occurs. A reinitialization strategy based on surface curvature features provides geometric prior guidance for Gaussian generation. The resulting reinitialized Gaussian point cloud is then re-input into the perspective-aligned radiometric Gaussian splatter model. To overcome the Gaussian explosion caused by the lack of geometric prior guidance in steps S2 and S3, a curvature-aware reinitialization strategy is proposed to optimize the generation of Gaussian distributions in X-ray scenes, improving training efficiency and the geometric accuracy of anatomical structure representation.

[0113] Furthermore, the operation in step S4 is specifically implemented as follows:

[0114] S4-1: After the first set number of iterations, the three-dimensional Gaussian point cloud P k , calculate each point p i The normalized local curvature feature ω of i This feature is obtained by analyzing the point p i The surface geometric characteristics of the n nearest neighbor point cloud are obtained, reflecting the local curvature intensity (high curvature areas such as bone edges retain more Gaussian elements).

[0115] S4-2: Based on curvature features and random threshold screening, the 3D Gaussian point cloud after the first set number of iterations is downsampled to generate a reinitialized Gaussian point cloud:

[0116]

[0117] Among them, U(0,1) is a uniform random number, and random threshold screening is introduced to avoid patterned noise and enhance the robustness of the algorithm. τ is the preset curvature threshold, This formula ensures that high curvature areas retain dense sampling and low curvature areas are sparse, avoiding redundant calculations.

[0118] Step S5: inputting the reinitialized Gaussian point cloud into the perspective-aligned radiation Gaussian splash model for iterative training, and obtaining a three-dimensional reconstruction result after a second set number of iterations.

[0119] Furthermore, the operation in step S5 is specifically implemented as follows:

[0120] Reinitialize the Gaussian point cloud As input, the perspective-aligned radiometric Gaussian splatter model from step 3 is used for iterative training, and the loss function is calculated after each training.

[0121] After iterating a second set number of times, a three-dimensional reconstruction result is obtained.

[0122] In step 5, As the initial Gaussian point cloud elements are re-injected into the iterative training process, the Gaussian distribution and training quality are optimized in combination with the anatomical geometric topological constraints. In other words, after obtaining the trained Gaussian point cloud file of the scene, it can be directly visualized without the need to input CT volume data. However, if the scene is changed, the overall steps of the method need to be repeated.

[0123] Example 2

[0124] This embodiment provides an X-ray 3D reconstruction system based on geometric prior and perspective alignment, including:

[0125] a data projection processing module configured to project the CT volume data to generate multi-angle original X-ray projection images and pre-process the images to obtain multi-angle X-ray projection images;

[0126] An initialization module is configured to use an angle-pose cube uniform initialization strategy to perform a first initialization on the multi-angle X-ray projection image to obtain an initialized Gaussian point cloud;

[0127] a 3D model iterative training module configured to perform a set number of iterative trainings based on an initialized Gaussian point cloud using a perspective-aligned radiometric Gaussian splatter model, and calculate a geometric reconstruction error after each training to obtain a 3D Gaussian point cloud after the set number of iterations;

[0128] The reinitialization module is configured to adopt a reinitialization strategy based on surface curvature features to perform a second initialization on the three-dimensional Gaussian point cloud after a set number of training iterations to obtain a reinitialized Gaussian point cloud;

[0129] The 3D reconstruction module is configured to input the reinitialized Gaussian point cloud into the perspective-aligned radiation Gaussian splash model for iterative training until the geometric reconstruction error meets the set threshold, stop the iteration, and obtain the 3D reconstruction result.

[0130] The examples and application scenarios implemented by the above modules and corresponding steps are the same, but are not limited to the contents disclosed in the above embodiment 1. It should be noted that the above modules as part of the system can be executed in a computer system such as a set of computer executable instructions.

[0131] The descriptions of the various embodiments in the above embodiments have different focuses. For parts not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0132] The proposed system can be implemented in other ways. For example, the system embodiment described above is merely illustrative. For example, the above module division is only a logical function division. In actual implementation, other division methods may be used. For example, multiple modules can be combined or integrated into another system, or some features can be ignored or not implemented.

[0133] Example 3

[0134] This embodiment provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the steps of the X-ray three-dimensional reconstruction method based on geometric prior and perspective alignment as described in the first embodiment above are implemented.

[0135] Example 4

[0136] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps of the X-ray three-dimensional reconstruction method based on geometric prior and perspective alignment as described in the first embodiment above are implemented.

[0137] Example 5

[0138] This embodiment provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps of the X-ray 3D reconstruction method based on geometric priors and perspective alignment described in the first embodiment.

[0139] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage and optical storage, etc.) containing computer-usable program code.

[0140] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0141] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0142] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0143] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware through a computer program. The program can be stored in a computer-readable storage medium, and when executed, the program can include the processes in the above-described method embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0144] Although the above describes the specific embodiments of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without any creative work are still within the scope of protection of the present invention.

Claims

1. An X-ray 3D reconstruction method based on geometric prior and perspective alignment, characterized in that: include: Projecting the CT volume data to generate multi-angle original X-ray projection images and preprocessing them to obtain multi-angle X-ray projection images; The angle-attitude cube uniform initialization strategy is used to perform the first initialization on the multi-angle X-ray projection image to obtain the initialized Gaussian point cloud. Based on the initialized Gaussian point cloud, a perspective-aligned radiometric Gaussian splatter model is used to perform a first set number of iterative trainings. After each training, the Gaussian properties are optimized using a loss function to obtain a three-dimensional Gaussian point cloud after the first set number of iterations. A reinitialization strategy based on surface curvature characteristics is used to perform a second initialization on the three-dimensional Gaussian point cloud after the first set number of iterations to obtain a reinitialized Gaussian point cloud; The reinitialized Gaussian point cloud is input into the perspective-aligned radiation Gaussian splash model for iterative training. After the second set number of iterations, the three-dimensional reconstruction result is obtained.

2. The X-ray 3D reconstruction method based on geometric prior and perspective alignment according to claim 1, characterized in that: The angle attitude cube uniform initialization strategy is used to perform the first initialization on the multi-angle X-ray projection image to obtain the initialized Gaussian point cloud, specifically: Calculate the camera's intrinsic and extrinsic matrix using the X-ray scanner's parameters; Based on the intrinsic parameter matrix and the extrinsic parameter matrix and the multi-angle X-ray projection image, the center of the cube is used as the center of the object and the origin of the world coordinate system to determine the cube that can enclose the object; The cube is divided into n grids, and points are uniformly sampled at intervals a within the grid to obtain an initialized Gaussian point cloud.

3. The X-ray 3D reconstruction method based on geometric prior and perspective alignment according to claim 1, characterized in that: Based on the initialized Gaussian point cloud, the perspective-aligned radiometric Gaussian splatter model is used to perform the first set number of iterative training. After each training, the Gaussian properties are optimized in combination with the loss function to obtain the three-dimensional Gaussian point cloud after the first set number of iterations. Specifically, Convert the initialized 3D Gaussian point cloud into a 2D Gaussian point cloud, and determine the pixels occupied by each 2D Gaussian through the covariance matrix and position of the 2D Gaussian point cloud; Calculate the color of each pixel based on the two-dimensional elliptical disk primitive representation model, and calculate the Gaussian contribution of each pixel in parallel to obtain the final Gaussian training image; The loss function is calculated by comparing the Gaussian training image with the original X-ray projection image at the same angle. The Gaussian properties are optimized by backpropagation of the loss function. The optimized Gaussian properties are used to guide the 3D reconstruction of the object, and a 3D Gaussian point cloud is obtained after one iteration. The three-dimensional Gaussian point cloud after one iteration is re-input into the perspective-aligned radiation Gaussian splash model, and the iterative training process is repeated. After iterating to a first set number of times, a three-dimensional Gaussian point cloud after the first set number of iterations is obtained.

4. The X-ray 3D reconstruction method based on geometric prior and perspective alignment according to claim 3, characterized in that: The color of each pixel is calculated based on the two-dimensional elliptical disk primitive representation model, and the Gaussian contribution of each pixel is calculated in parallel to obtain the final Gaussian training image. Specifically: Based on the two-dimensional elliptical disk primitive representation model, a Gaussian elliptical disk is used to represent the two-dimensional Gaussian basis element in the two-dimensional space; Based on the Beer-Lambert law, the intensity attenuation of X-rays when penetrating the medium is modeled, and the attenuation factor of each two-dimensional Gaussian basis element is obtained; Based on the order of the two-dimensional Gaussian primitives in each pixel, the color, attenuation factor, opacity of each two-dimensional Gaussian primitive and the opacity of all previous two-dimensional Gaussian primitives are alpha blended to obtain the color of each pixel, and the Gaussian contribution of each pixel is calculated in parallel to obtain the final Gaussian training image.

5. The X-ray 3D reconstruction method based on geometric prior and perspective alignment according to claim 1, characterized in that: The re-initialization strategy based on surface curvature features is adopted to perform a second initialization on the three-dimensional Gaussian point cloud after the first set number of iterations to obtain a re-initialized Gaussian point cloud, specifically: After the first set number of iterations k, the three-dimensional Gaussian point cloud P k , calculate each point p i Normalized local curvature characteristics; Based on the normalized local curvature features and random threshold screening, the three-dimensional Gaussian point cloud after the first set number of iterations is downsampled to obtain a reinitialized point cloud.

6. The X-ray 3D reconstruction method based on geometric prior and perspective alignment according to claim 1, characterized in that: The loss function is specifically: Among them, the L1 loss directly minimizes the pixel absolute error between the Gaussian training image and the original X-ray projection image; the SSIM loss measures the similarity of the overall structure and texture between the Gaussian training image and the original X-ray projection image; γ is a weight coefficient used to balance the contribution of the two loss terms.

7. X-ray 3D reconstruction system based on geometric prior and perspective alignment, characterized by: include: a data projection processing module configured to project the CT volume data to generate multi-angle original X-ray projection images and pre-process the images to obtain multi-angle X-ray projection images; An initialization module is configured to use an angle-pose cube uniform initialization strategy to perform a first initialization on the multi-angle X-ray projection image to obtain an initialized Gaussian point cloud; a three-dimensional model iterative training module configured to perform a first set number of iterative trainings based on the initialized Gaussian point cloud using a perspective-aligned radiometric Gaussian splatter model, and optimize Gaussian properties in combination with a loss function after each training, thereby obtaining a three-dimensional Gaussian point cloud after the first set number of iterations; a reinitialization module configured to employ a reinitialization strategy based on surface curvature features to perform a second initialization on the three-dimensional Gaussian point cloud after iterating a first set number of times, thereby obtaining a reinitialized Gaussian point cloud; The three-dimensional reconstruction module is configured to input the reinitialized Gaussian point cloud into the perspective-aligned radiation Gaussian splash model for iterative training, and obtain a three-dimensional reconstruction result after iterating a second set number of times.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the X-ray three-dimensional reconstruction method based on geometric prior and perspective alignment as described in any one of claims 1 to 6 are implemented.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the X-ray three-dimensional reconstruction method based on geometric prior and perspective alignment are implemented as described in any one of claims 1 to 6.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the computer program implements the steps of the X-ray three-dimensional reconstruction method based on geometric prior and perspective alignment according to any one of claims 1 to 6.

Images