Method and system for three-dimensional ct bone diffusion reconstruction based on two-dimensional x-rays

Abstract

The application belongs to the technical field of medical images, and discloses a three-dimensional CT bone diffusion reconstruction method and system based on two-dimensional X-rays, which comprises the following steps: acquiring an X-ray image; using a CNN encoder to extract features from the X-ray image to obtain a first feature map, using a Mamba encoder to extract features from the intermediate layer feature map of the CNN encoder to obtain a second feature map, and fusing the first feature map and the second feature map to obtain a conditional feature; reconstructing a noise image with the same size as the X-ray image, performing multi-scale cross-attention calculation on the conditional feature and the noise image to obtain a fusion feature; gradually denoising the noise image based on the fusion feature through an inverse diffusion process, reconstructing a three-dimensional feature latent map of the bone, and performing a decoding operation on the three-dimensional feature latent map to obtain a three-dimensional CT image. The application realizes one-step and end-to-end three-dimensional bone structure reconstruction from a two-dimensional X-ray image, and solves the deficiencies of the existing reconstruction method in complex bone detail preservation and multi-scale dependency processing.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, and in particular to a method and system for three-dimensional CT bone diffusion reconstruction based on two-dimensional X-rays. Background Technology

[0002] In orthopedic diagnosis, two-dimensional X-ray and three-dimensional CT imaging each have their advantages and disadvantages. While two-dimensional X-rays are simple to operate and have low radiation exposure, their two-dimensional nature limits the comprehensive visualization of complex skeletal structures. Three-dimensional CT, on the other hand, provides detailed skeletal information, but has higher radiation exposure and is unsuitable for certain surgical procedures. To overcome these limitations, techniques for reconstructing three-dimensional CT images from two-dimensional X-rays have emerged in recent years. Although existing methods such as Reconstruction Adversarial Networks (GANs) and diffusion models have made some progress in image reconstruction, they still have shortcomings in preserving skeletal details, handling multi-scale dependencies, and integrating with clinical diagnostic tasks. Summary of the Invention

[0003] To address the aforementioned issues, this invention proposes a three-dimensional CT bone diffusion reconstruction method and system based on two-dimensional X-rays, which enables one-step, end-to-end three-dimensional bone structure reconstruction from two-dimensional X-ray images, overcoming the shortcomings of existing reconstruction methods in preserving complex bone details and handling multi-scale dependencies.

[0004] To achieve the above objectives, the present invention adopts the following technical solution:

[0005] In a first aspect, the present invention provides a three-dimensional CT bone diffusion reconstruction method based on two-dimensional X-rays, comprising the following steps:

[0006] Acquire two-dimensional X-ray images;

[0007] The X-ray image is used to extract features using a CNN encoder to obtain a first feature map. The intermediate layer feature map of the CNN encoder is used to extract features using a Mamba encoder to obtain a second feature map. The first and second feature maps are then fused to obtain conditional features.

[0008] A Gaussian noise image of the same size as the X-ray image is reconstructed, and multi-scale cross-attention calculation is performed on the conditional features and the Gaussian noise image to obtain fused features.

[0009] Based on the fusion features, the noisy image is gradually denoised through the inverse diffusion process, the three-dimensional feature latent map of the bone is reconstructed, and the three-dimensional feature latent map is decoded to obtain a three-dimensional CT image.

[0010] Define a loss function and train the conditional encoder and the 3D denoising model to obtain a trained 3D CT skeleton diffusion reconstruction model.

[0011] As an alternative implementation, it also includes classifying scoliosis using features extracted during the diffusion process, outputting scoliosis classification results, and using the loss function for training the scoliosis classifier as follows:

[0012] ;

[0013] In the formula, These are the true category labels for scoliosis samples. It is the probability that the scoliosis classifier predicts it as scoliosis.

[0014] As an alternative implementation, the conditional features are orthogonally positionally encoded, and the orthogonally positionally encoded conditional features are used to perform multi-scale cross-attention calculation with the Gaussian noise image.

[0015] As an alternative implementation, a three-dimensional UKAN diffusion backbone network is used to extract multi-scale features of the X-ray image, and a multi-scale convolutional pyramid is used to integrate features of different scales to estimate noise. Based on the estimated noise, a Gaussian noise image with the same size as the X-ray image is reconstructed.

[0016] As an alternative implementation method, the loss function of the 3D denoising model is:

[0017] ;

[0018] In the formula, It is a Gaussian noise image of the same size as the X-ray image. It is a conditional feature. It's noise, t is the time step, and T is the total number of time steps. These are potential variables at different levels.

[0019] As an alternative implementation, a Gaussian noise image of the same size as the X-ray image is initialized into a latent space, and noise is gradually added in the latent space to obtain a pure noise image.

[0020] Secondly, the present invention provides a three-dimensional CT bone diffusion reconstruction system based on two-dimensional X-rays, comprising:

[0021] The data acquisition module is configured to acquire two-dimensional X-ray images;

[0022] The conditional coding module is configured to: extract features from the X-ray image using a CNN encoder to obtain a first feature map; extract features from the intermediate layer feature map of the CNN encoder using a Mamba encoder to obtain a second feature map; and fuse the first and second feature maps to obtain conditional features.

[0023] The feature fusion module is configured to: reconstruct a Gaussian noise image of the same size as the X-ray image, perform multi-scale cross-attention calculation on the conditional features and the Gaussian noise image, and obtain fused features;

[0024] The diffusion reconstruction module is configured to: based on fusion features, gradually denoise the noisy image through an inverse diffusion process, reconstruct the three-dimensional feature latent map of the bone, and perform decoding operations on the three-dimensional feature latent map to obtain a three-dimensional CT image;

[0025] The model training module is configured to: define a loss function, train the conditional encoder and the 3D denoising model, and obtain a trained 3D CT skeleton diffusion reconstruction model.

[0026] Thirdly, the present invention provides an electronic device including a memory and a processor, and computer instructions stored in the memory and running on the processor, wherein the computer instructions, when executed by the processor, perform the method described in the first aspect.

[0027] Fourthly, the present invention provides a computer-readable storage medium for storing computer instructions, which, when executed by a processor, perform the method described in the first aspect.

[0028] Fifthly, the present invention provides a computer program product, including a computer program that, when executed by a processor, implements the method described in the first aspect.

[0029] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0030] The present invention proposes a three-dimensional CT bone diffusion reconstruction method based on two-dimensional X-rays, which proposes a three-dimensional CT bone reconstruction framework based on a diffusion model. By integrating the cross-dimensional conditional encoder (ECond), the three-dimensional UKAN diffusion backbone network (DM3DL), and the diffusion-based scoliosis classifier (DSC), it effectively solves the shortcomings of existing two-dimensional X-ray to three-dimensional CT reconstruction methods in terms of preserving complex bone details and handling multi-scale dependencies.

[0031] The present invention proposes a three-dimensional CT bone diffusion reconstruction method based on two-dimensional X-rays. It extracts anatomical prior information from two-dimensional X-rays through a CNN-Mamba joint architecture and orthogonal position coding method, enhances feature representation using Kolmogorov-Arnold network (KAN), and integrates a scoliosis classifier during the reconstruction process. This effectively solves the problem of existing methods being out of touch with clinical diagnostic tasks, and improves reconstruction quality and clinical application value.

[0032] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0033] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0034] Figure 1 The flowchart of the three-dimensional CT bone diffusion reconstruction method based on two-dimensional X-rays of the present invention is as follows: Figure 1 ;

[0035] Figure 2 This is a comparison chart of qualitative results from different reconstruction methods on a spine dataset.

[0036] Figure 3 This is a comparison chart of qualitative results from different reconstruction methods on a hip joint dataset.

[0037] Figure 4 This is a comparison chart of qualitative results from different reconstruction methods on a knee joint dataset.

[0038] Figure 5 The flowchart of the three-dimensional CT bone diffusion reconstruction method based on two-dimensional X-rays of the present invention is as follows: Figure 2 ;

[0039] Figure 6 This is a framework diagram of the three-dimensional CT bone diffusion reconstruction method based on two-dimensional X-rays according to the present invention. Detailed Implementation

[0040] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0041] It should be noted that the following detailed description is exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0042] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments of the invention. As used herein, unless the context clearly indicates otherwise, the singular form is intended to include the plural form as well. Furthermore, it should be understood that the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but includes other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0043] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0044] Terminology Explanation:

[0045] 3D Reconstruction: 3D reconstruction refers to the technology of reconstructing the three-dimensional structure of an object from a 2D image. In the medical field, its core value lies in overcoming the limitations of traditional X-rays—reconstructing an observable three-dimensional anatomical model (such as a bone CT scan) from single or multiple two-dimensional projections (such as frontal / lateral views), enabling precise quantitative analysis (such as joint spaces, tumor volume).

[0046] 3D diffusion: 3D diffusion is a 3D data reconstruction technique based on reconstructive artificial intelligence. Its core is to achieve high-fidelity structural reconstruction by simulating the physical diffusion process. This method includes two key stages: first, forward noise reduction, which gradually adds Gaussian noise to the original 3D voxel data, eventually degenerating it into a completely random distribution; second, condition-guided backward reconstruction, which trains a neural network to gradually remove noise based on conditional inputs and restore the anatomical structure.

[0047] Example 1

[0048] like Figure 1 , Figure 5 and Figure 6 As shown, this embodiment provides a three-dimensional CT bone diffusion reconstruction method based on two-dimensional X-rays, including the following steps:

[0049] Acquire two-dimensional X-ray images;

[0050] The X-ray image is used to extract features using a CNN encoder to obtain a first feature map. The intermediate layer feature map of the CNN encoder is used to extract features using a Mamba encoder to obtain a second feature map. The first and second feature maps are then fused to obtain conditional features.

[0051] A Gaussian noise image of the same size as the X-ray image is reconstructed, and multi-scale cross-attention calculation is performed on the conditional features and the Gaussian noise image to obtain fused features.

[0052] Based on the fusion features, the noisy image is gradually denoised through the inverse diffusion process, the three-dimensional feature latent map of the bone is reconstructed, and the three-dimensional feature latent map is decoded to obtain a three-dimensional CT image.

[0053] Define a loss function and train the conditional encoder and the 3D denoising model to obtain a trained 3D CT skeleton diffusion reconstruction model.

[0054] The specific solution of the present invention is as follows:

[0055] This invention discloses a three-dimensional CT bone diffusion reconstruction method based on two-dimensional X-rays, particularly suitable for reconstructing three-dimensional skeletal structures from two-dimensional X-ray images. The method consists of several key components, including a cross-dimensional conditional encoder (ECond), a three-dimensional latent diffusion model (DM3DL), and a diffusion-based scoliosis classifier (DSC). First, ECond extracts skeletal structure information from two-dimensional X-ray images using a CNN-Mamba hybrid architecture and converts it into an embedding conditional vector. Then, DM3DL uses this conditional vector to progressively denoise in the latent space, reconstructing a three-dimensional feature map of the bone. Finally, DSC classifies scoliosis during the reconstruction process to aid in clinical diagnosis. Through this series of innovative steps, this method can not only accurately reconstruct three-dimensional skeletal structures from two-dimensional X-ray images but also simultaneously diagnose scoliosis during reconstruction, providing an efficient and accurate solution for orthopedic imaging and clinical decision-making.

[0056] First, two-dimensional X-ray images of front-to-back (AP) and left-to-right (RL) positions are received as input data. The two-dimensional X-ray images are then processed using a cross-dimensional conditional encoder, which is a CNN-Mamba hybrid architecture.

[0057] CNN-Mamba Block: Local and Global Feature Merging. The CNN-Mamba block (BCM) merges local and global receptive fields from CNN and Mamba to effectively guide the diffusion model. This allows conditional features to capture both the overall structure and intricate details of the skeleton. Integrated Compact Convolutional Kernels and Normalization Processing. To efficiently perform Mamba, a series of layers are utilized, combining compact kernels with integrated normalization and activation functions. The design of these layers improves computational efficiency and the accuracy of feature extraction.

[0058] A cross-dimensional conditional encoder is set up. The conditional encoder extracts skeletal structure information from two-dimensional X-ray images through a CNN-Mamba hybrid architecture and converts it into embedded conditional vectors. The specific implementation process is as follows:

[0059] Feature extraction and fusion: CNN feature extraction, using a CNN encoder. For the input X-ray image Feature extraction is performed. The CNN captures detailed skeletal information through local convolution operations and reconstructs feature map C1 (i.e., the first feature map).

[0060] ;

[0061] Mamba feature extraction from the intermediate layers of the CNN encoder Extract feature maps as Mamba encoders The input is Mamba. Mamba uses a selective state-space mechanism to grasp the overall structural features and improves computational efficiency through parallel training to reconstruct the feature map C2 (i.e., the second feature map).

[0062] ;

[0063] Feature fusion combines the feature maps C1 and C2 from CNN and Mamba to obtain the final conditional features. .

[0064] ;

[0065] Here, ⊕ represents the fusion method, which can be addition, splicing, or other fusion operations.

[0066] Orthogonal Position Encoding (PoE) of Conditional Features: The positional information of the skeleton plays a crucial role in orthogonal inputs; this hidden information is indispensable for ensuring the model accurately reconstructs the 3D skeletal structure. To enhance the model's understanding of the 3D positional relationships of the input skeletal features, orthogonal position encoding... It is integrated into the BCM as a residual embedding. Position encoding is defined as:

[0067] ;

[0068] The encoding is calculated in mutually orthogonal directions, specifically as follows:

[0069] ;

[0070] This encoding helps guide the reconstruction direction, ensuring that the diffusion output not only reflects realistic skeletal features but also matches the distribution of the target X-ray image.

[0071] A Gaussian noise image of the same size as the X-ray image is reconstructed. Multi-scale cross-attention calculation is then performed on the conditional features encoded at orthogonal positions and the Gaussian noise image to obtain fused features. Specifically:

[0072] Multiscale cross-attention (ψ): The stepwise introduction of conditional information is used to ensure that the reconstructed CT results correspond to the target X-rays and to enhance detail reconstruction during diffusion. Multiscale cross-attention information ψ( It was gradually introduced into the intermediate and decoding layers of the diffusion model.

[0073] Integration of Conditional Information: Through this mechanism ψ, the image reconstruction network effectively integrates conditional information into the process, enabling the model to selectively emphasize relevant aspects when reconstructing CT images. Specifically, each diffusion step t estimates the noise ϵ from the noisy CT. and conditional X-ray embedding ψ( The interaction between the two modalities occurs during noise prediction, and their feature embeddings are fused via ψ.

[0074] Potential Bone Features in Noisy CT Projected onto the query matrix Conditional X-ray embeddings are projected onto the bond matrix. Sum matrix .in, , and It is a linear projection transformation obtained through learning. The output of multi-scale cross-attention. Defined as:

[0075] ;

[0076] Among them, elements Defined as the attention weights of pixel feature vector i and conditional embedding j, where d is the projection dimension of the key and query. It is an activation function.

[0077] Based on fusion features, diffusion reconstruction is performed on 3D CT images. The specific implementation process is as follows:

[0078] Design of diffusion model in latent space: Decoupling 3D feature information to achieve diffusion from 2D input To map to 3D skeletal reconstruction, a diffusion model, DM3DL, was designed in the latent space of skeletal features. This model decouples 3D multi-slice feature information from the 2D input, progressively refining an abstract skeletal target for reconstruction. Compared to traditional diffusion models operating in pixel space, the use of the latent space offers significant advantages. This approach can more effectively capture and utilize the inherent structure of the data, thus achieving superior performance in 3D reconstruction tasks. DM3DL embeds diffusion in the latent space, progressively refining the reconstruction target from the latent representation. This not only improves computational efficiency but also better preserves 2D conditional information.

[0079] Feature Extraction: Due to the challenges of mapping 2D input to 3D space, quantization can degrade reconstruction quality. To mitigate this impact, we employ an improved feature extraction method that enhances the skeletal feature representation by embedding 2D planar information into a pre-trained codebook Z. Feature extraction is achieved through multi-scale convolution:

[0080] ;

[0081] Where k represents the kernel size, BN is batch normalization, and σ is the activation function. The fused features are weighted as follows: It is connected to the 3D codebook Z in the channel dimension, and then spatial consistency is maintained through Conv1×1.

[0082] Computation of Latent Representation: To compute the latent representation, we directly process the input data using an encoder and decoder. The encoder compresses the input data into a low-dimensional latent space, while the decoder is responsible for restoring the representation from the latent space back to the original data dimension. Input The potential form is obtained through nearest neighbor codebook vector selection:

[0083] ;

[0084] Given input Its potential form is , where s is the compression factor.

[0085] Forward Diffusion Process: To train the reconstruction algorithm, DM3DL operates in the latent space. The input is initialized to the latent space before incrementally adding noise ε to aid convergence. The forward diffusion process follows:

[0086] ;

[0087] in It is a predefined variance table that ensures that for large T values, the latent variables... It is a nearly isotropic Gaussian distribution.

[0088] use and get:

[0089] ;

[0090] Raw data Noise version It follows a Gaussian distribution, and its center is located at A scaled version, where variance increases over time. As t increases, the mean... Gradually approaching zero, while the variance... Increase, gradually It becomes pure noise.

[0091] To enhance the noise estimation capability of the diffusion network, the operation process is as follows:

[0092] Construction of the 3D UNet backbone network: To better extract and reconstruct the 3D features of the skeleton, the 2D blocks in the mainstream diffusion model are replaced with 3D blocks, and 3D UNet is used as the backbone network. This replacement allows the network to directly process 3D data, thereby capturing 3D structural information more effectively. 3D UNet contains multiple 3D convolutional layers and downsampling / upsampling modules, enabling it to progressively extract and reconstruct the 3D features of the skeleton.

[0093] Integrated KAN Module: In 3D UNet, selected ResNet blocks are replaced by KAN blocks. KAN blocks combine the Kolmogorov-Arnold theorem and B-spline basis functions to achieve efficient and powerful nonlinear modeling. The formula for each KAN block is:

[0094] ;

[0095] in, Φi is the SiLU activation function, and Φi represents the i-th learnable B-spline basis function. This design enhances the expressiveness and efficiency of feature representation through a learnable nonlinear activation function.

[0096] Multi-scale feature extraction and processing: 3D UKAN uses convolutional layers and KAN layers to extract multi-scale features from the input noisy skeleton image. These features are then labeled and processed through a series of KAN blocks, refining the latent space through nonlinearly modulated activation. Through this processing, the network is able to capture the complex structural dependencies in the 3D skeleton features, thereby reconstructing the skeleton structure more accurately.

[0097] Definition of the objective function: The objective function of 3D UKAN is defined as follows:

[0098] ;

[0099] in, The data is noise data at time step t. Here, υ is the conditional feature, and υ is the noise estimation function. This objective function aims to minimize the difference between the predicted noise and the actual noise, thereby improving the accuracy of noise estimation.

[0100] To further integrate feature representations at different scales, we innovatively introduce the Multi-Scale Convolutional Pyramid (MCP). The introduction of the Multi-Scale Convolutional Pyramid (MCP) integrates feature representations at multiple scales. The MCP extracts features from different scales through a series of convolutional and pooling layers and fuses these features together to enhance the network's ability to capture multi-scale features.

[0101] A conditional mechanism was introduced during training, in conjunction with the conditional encoder. The X-ray image embedding is introduced into the 3D UKAN via multi-scale cross-attention ψ to influence its output. Attention layers are applied within the encoder and decoder of 3D ResNet blocks with downsampling factors of 2, 4, and 8. This design allows the network to better utilize the conditional information in the X-ray image, thereby improving the accuracy and detail of the reconstruction.

[0102] The inverse process of DM3DL (i.e., the denoising process) is implemented as follows:

[0103] Stepwise denoising process: Noise samples ϵ are denoised through the inverse process of DM3DL. Guided by the algorithm, noise is gradually removed. At each time step t, the model estimates and removes noise, progressively recovering the latent 3D feature map of the skeleton. Condition vector It provides continuous guidance throughout the reverse diffusion process, ensuring that the reconstructed 3D features are anatomically consistent with the input X-ray image.

[0104] Reconstruction and Decoding: After multiple reverse diffusion steps, the final three-dimensional feature latent map of the skeleton is obtained. This latent map contains detailed feature information for reconstructing the skeleton. A pre-trained decoder is then used. Potential graph The decoding process converts the feature map in the latent space into an actual 3D CT image, completing the reconstruction from 2D X-ray to 3D CT.

[0105] The implementation of the diffusion-based scoliosis classifier DSC is as follows:

[0106] Shared latent feature representation: Latent feature representation extracted by DSC using the 3D diffusion model DM3DL These features are represented by the diffusion process DM3DL from the input noisy image. What I learned:

[0107] ;

[0108] Where S represents the shared latent features learned from the two tasks (spinal reconstruction and scoliosis classification).

[0109] Cross-task parameter sharing: To improve model efficiency, DSC enhances target feature extraction capabilities by sharing parameters across tasks. This parameter sharing ensures that the model can effectively utilize the same feature representations in both spinal reconstruction and scoliosis classification tasks.

[0110] Channel-level fusion and cross-attention features: In addition to sharing parameters, DSC also incorporates cross-attention features through channel-level fusion. With shared potential features This combination supplements two-dimensional feature information and improves the ability to extract target features.

[0111] ;

[0112] in, and These are convolution weights and biases obtained through learning.

[0113] Hierarchical Network Structure: To maintain accuracy in scoliosis classification tasks, DSC fully utilizes a hierarchical network structure and designs an advanced feature extraction subnetwork. This subnetwork can extract features from the fused data. It learns feature information that helps with classification.

[0114] Joint optimization of multi-task loss: SC jointly optimizes the loss across multiple tasks, including the reconstruction task. This means that during training, the model needs to optimize not only the performance of spinal reconstruction but also the performance of scoliosis classification. This joint optimization ensures that the model achieves good results on both tasks.

[0115] Classifier Design: DSC consists of three classifiers, each responsible for a different classification task:

[0116] Diagnostic classifier Used to determine if scoliosis exists.

[0117] Location classifier Used to determine the location of scoliosis.

[0118] Curvature classifier Used to assess the degree of curvature of scoliosis.

[0119] Specifically, it is expressed as follows:

[0120] : (diagnosis);

[0121] : (Location);

[0122] : (Curvature);

[0123] Classification Decision: As the diffusion process iterates, DSC determines whether scoliosis exists based on rich diffusion feature information. This decision can be expressed by the following formula:

[0124] ;

[0125] Where Pscoliosis is the probability of scoliosis given the extracted features, W and b are the learned weights and biases, and σ is the activation function (such as the Sigmoid function).

[0126] The implementation of the loss function is as follows:

[0127] Loss Function: To improve reconstruction accuracy, the mean squared error (MSE) of the autoencoder is replaced with a perceptual loss. Perceptual loss better captures the semantic information of the image, thus improving the visual quality of the reconstruction. The pre-trained loss function includes the reconstruction loss function. and GAN loss function :

[0128] ;

[0129] Where λ is the weighting coefficient that balances the two.

[0130] The specific forms of reconstruction error:

[0131] ;

[0132] Where x is the input data, It is the output of the reconstruction. is the latent representation of the embedding, sg represents the stopping gradient, and β is another balancing coefficient.

[0133] The specific form of the GAN loss function:

[0134] ;

[0135] Where D is the discriminator, used to distinguish between real data and reconstructed data.

[0136] Conditional encoder Loss function: To prevent gradient vanishing, the conditional encoder Using the mean squared error (MSE) loss function:

[0137] ;

[0138] in, It is a target feature. is the output of the encoder, and n is the number of samples.

[0139] Loss function of 3D denoising model: In latent space, 3D denoising model The training loss function is:

[0140] ;

[0141] in, The original input is a Gaussian noise image of the same size as the X-ray image. It is a conditional feature. It's noise, t is the time step, and T is the total number of time steps. Through an attention mechanism, conditional features... Latent variables of different layers Having the same dimensions allows for better guidance of the diffusion denoising process.

[0142] The loss function of the scoliosis classifier DSC: The DSC consists of three classifiers, trained using the cross-entropy loss function. For each classifier, the cross-entropy loss function is defined as:

[0143] ;

[0144] in, These are the true category labels for scoliosis samples. It is the probability that the scoliosis classifier predicts it as scoliosis.

[0145] verify:

[0146] Figure 2 This paper presents a qualitative comparison of different reconstruction methods on a spine dataset. By visually comparing the reconstructed 3D images with reference images, it can be seen that the method of this invention performs best in preserving spinal details and vertebral structure. This figure visually verifies the superiority of the method of this invention in quantitative results, especially in capturing the complex 3D structure of the spine.

[0147] Figure 3 The results demonstrate the superior 3D reconstruction quality of the method described in this invention compared to other methods on a hip joint dataset. By comparing the reconstruction results with reference images, the excellent performance of the method described in this invention in accurately reconstructing the pelvic structure and femoral head shape is clearly evident; these details are crucial for clinical diagnosis and treatment planning.

[0148] Figure 4 The qualitative results of the method of the present invention in 3D reconstruction on a knee joint dataset are presented. Compared with other methods, the method of the present invention can more accurately reconstruct the skeletal structure of the knee joint, including clear separation of the tibia and fibula, as well as details of the soft tissues surrounding the knee joint.

[0149] Example 2

[0150] This embodiment provides a three-dimensional CT bone diffusion reconstruction system based on two-dimensional X-rays, including:

[0151] The data acquisition module is configured to acquire two-dimensional X-ray images;

[0152] The conditional coding module is configured to: extract features from the X-ray image using a CNN encoder to obtain a first feature map; extract features from the intermediate layer feature map of the CNN encoder using a Mamba encoder to obtain a second feature map; and fuse the first and second feature maps to obtain conditional features.

[0153] The feature fusion module is configured to: reconstruct a Gaussian noise image of the same size as the X-ray image, perform multi-scale cross-attention calculation on the conditional features and the Gaussian noise image, and obtain fused features;

[0154] The diffusion reconstruction module is configured to: based on fusion features, gradually denoise the noisy image through an inverse diffusion process, reconstruct the three-dimensional feature latent map of the bone, and perform decoding operations on the three-dimensional feature latent map to obtain a three-dimensional CT image;

[0155] The model training module is configured to: define a loss function, train the conditional encoder and the 3D denoising model, and obtain a trained 3D CT skeleton diffusion reconstruction model.

[0156] It should be noted that the above modules correspond to the steps in Embodiment 1, and the examples and application scenarios implemented by the above modules and their corresponding steps are the same, but are not limited to the content disclosed in Embodiment 1. It should also be noted that the above modules can be executed in a computer system as part of the system.

[0157] In further embodiments, the following is also provided:

[0158] An electronic device includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor, which, when executed by the processor, perform the method described in Embodiment 1. For brevity, further details are omitted here.

[0159] It should be understood that in this embodiment, the processor can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0160] A computer-readable storage medium for storing computer instructions that, when executed by a processor, perform the method of Embodiment 1.

[0161] The method in Example 1 can be directly executed by a hardware processor, or it can be executed by a combination of hardware and software modules within the processor. The software modules can reside in readily available storage media in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory; the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, a detailed description is not provided here.

[0162] A computer program product includes a computer program that, when executed by a processor, implements the method in Embodiment 1.

[0163] The present invention also provides at least one computer program product tangibly stored on a non-transitory computer-readable storage medium. The computer program product includes computer-executable instructions, such as instructions included in program modules, which execute in a device on a target real or virtual processor to perform the processes / methods described above. Typically, program modules include routines, programs, libraries, objects, classes, components, data structures, etc., that perform specific tasks or implement specific abstract data types. In various embodiments, the functionality of program modules can be combined or divided among program modules as needed. The machine-executable instructions for the program modules can execute within a local or distributed device. In a distributed device, the program modules can reside in both local and remote storage media.

[0164] The computer program code used to implement the methods of the present invention may be written in one or more programming languages. This computer program code may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the computer or other programmable data processing device, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a computer, partially on a computer, as a stand-alone software package, partially on a computer and partially on a remote computer, or entirely on a remote computer or server.

[0165] In the context of this invention, computer program code or related data may be carried by any suitable carrier to enable a device, apparatus, or processor to perform the various processes and operations described above. Examples of carriers include signals, computer-readable media, and the like. Examples of signals may include electrical, optical, radio, sound, or other forms of propagation signals, such as carrier waves, infrared signals, etc.

[0166] Those skilled in the art will recognize that the units and algorithm steps described in conjunction with the embodiments herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0167] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A three-dimensional CT bone diffusion reconstruction method based on two-dimensional X-rays, characterized in that, Includes the following steps: Acquire two-dimensional X-ray images; The X-ray image is used to extract features using a CNN encoder to obtain a first feature map. The intermediate layer feature map of the CNN encoder is used to extract features using a Mamba encoder to obtain a second feature map. The first and second feature maps are then fused to obtain conditional features. A Gaussian noise image of the same size as the X-ray image is reconstructed, and multi-scale cross-attention calculation is performed on the conditional features and the Gaussian noise image to obtain fused features. Based on the fusion features, the noisy image is gradually denoised through the inverse diffusion process, the three-dimensional feature latent map of the bone is reconstructed, and the three-dimensional feature latent map is decoded to obtain a three-dimensional CT image. Define a loss function, train the conditional encoder and the 3D denoising model to obtain a trained 3D CT bone diffusion reconstruction model; It also includes classifying scoliosis using features extracted during the diffusion process and outputting scoliosis classification results; The conditional features are orthogonally positionally encoded, and then multi-scale cross-attention calculation is performed using the orthogonally positionally encoded conditional features and the Gaussian noise image.

2. The three-dimensional CT bone diffusion reconstruction method based on two-dimensional X-rays as described in claim 1, characterized in that, The loss function for training the scoliosis classifier is: ； In the formula, These are the true category labels for scoliosis samples. It is the probability that the scoliosis classifier predicts it as scoliosis.

3. The three-dimensional CT bone diffusion reconstruction method based on two-dimensional X-rays as described in claim 1, characterized in that, A three-dimensional UKAN diffusion backbone network is used to extract multi-scale features from X-ray images. Multi-scale convolutional pyramids are used to integrate features of different scales to estimate noise. Based on the estimated noise, a Gaussian noise image with the same size as the X-ray image is reconstructed.

4. The three-dimensional CT bone diffusion reconstruction method based on two-dimensional X-rays as described in claim 1, characterized in that, The loss function of the 3D denoising model is: ； In the formula, It is a Gaussian noise image of the same size as the X-ray image. It is a conditional feature. It's noise, t is the time step, and T is the total number of time steps. These are potential variables at different levels.

5. The three-dimensional CT bone diffusion reconstruction method based on two-dimensional X-rays as described in claim 1, characterized in that, A Gaussian noise image of the same size as the X-ray image is initialized into the latent space. Noise is then added incrementally within the latent space to obtain a pure noise image.

6. A three-dimensional CT bone diffusion reconstruction system based on two-dimensional X-rays, characterized in that, The method for three-dimensional CT bone diffusion reconstruction based on two-dimensional X-rays as described in any one of claims 1-5 includes: The data acquisition module is configured to acquire two-dimensional X-ray images; The conditional encoding module is configured to: extract features from the X-ray image using a CNN encoder to obtain a first feature map; extract features from the intermediate layer feature map of the CNN encoder using a Mamba encoder to obtain a second feature map; and fuse the first and second feature maps to obtain conditional features. The feature fusion module is configured to: reconstruct a Gaussian noise image of the same size as the X-ray image, perform multi-scale cross-attention calculation on the conditional features and the Gaussian noise image, and obtain fused features; The diffusion reconstruction module is configured to: based on fusion features, gradually denoise the noisy image through an inverse diffusion process, reconstruct the three-dimensional feature latent map of the bone, and perform decoding operations on the three-dimensional feature latent map to obtain a three-dimensional CT image; The model training module is configured to: define a loss function, train the conditional encoder and the 3D denoising model, and obtain a trained 3D CT skeleton diffusion reconstruction model.

7. An electronic device, characterized in that, It includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor, which, when executed by the processor, perform the method according to any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, Used to store computer instructions, which, when executed by a processor, perform the method described in any one of claims 1-5.

9. A computer program product, characterized in that, Includes a computer program, which, when executed by a processor, implements the method described in any one of claims 1-5.

Images