Method for generating EOS-like three-dimensional data from CT scanning data, computer program product and terminal equipment

Abstract

The invention relates to an EOS-like three-dimensional data generation method, in particular to a method for generating EOS-like three-dimensional data through CT scanning data, a computer program product and terminal equipment, and solves the technical problem that an existing EOS-like three-dimensional data generation method is low in efficiency and precision. According to the method, a physical projection model based on EOS system geometric parameters is constructed, the geometric projection process of EOS system imaging is precisely restored, and a style migration network and a three-dimensional reconstruction network are established by means of deep learning; the style migration network realizes conversion from CT scanning data to biplanar EOS data by utilizing the view angle complementary characteristic of front and side projection; according to the three-dimensional reconstruction network, an adversarial learning architecture of double generators and double discriminators is utilized, and high fidelity of generated EOS-like three-dimensional data in geometric structure and texture details is ensured through collaborative optimization of adversarial training and cyclic consistency constraint. The matching of the style migration network and the three-dimensional reconstruction network can greatly improve the generation efficiency and precision of the EOS-like three-dimensional data.

Description

Technical Field

[0001] The present invention relates to a method for generating EOS-like three-dimensional data, and particularly to a method, a computer program product, and a terminal device for generating EOS-like three-dimensional data from CT scan data. Background Art

[0002] Traditional CT scan imaging technology generates three-dimensional voxel data through multi-angle X-ray projections. Although it can provide high-precision anatomical information, its single scan radiation dose is relatively high, making it unsuitable for sensitive populations requiring repeated examinations and functional evaluations in the natural weight-bearing state. The EOS biplanar X-ray imaging system (a three-dimensional whole-body bone modeling imaging system) adopts a standing anteroposterior and lateral synchronous imaging method, achieving dynamic evaluation in the weight-bearing state while reducing the radiation dose by 95%, and can avoid the drawbacks of CT scan imaging technology. However, the EOS biplanar X-ray imaging system faces the problem of scarce publicly available datasets due to insufficient global installations during clinical promotion, which severely restricts the development and application of deep learning algorithms.

[0003] Existing methods for generating EOS-like three-dimensional data mainly rely on manually annotating anatomical landmark points for geometric modeling, suffering from technical bottlenecks such as serious time consumption, cumulative manual errors, and significant attenuation of reconstruction accuracy with changes in body position. The idealized geometric parameter DRR generation method adopted in Chinese Patent CN118411443A fails to fully adapt to the physical characteristic parameter system of real imaging devices, resulting in systematic deviations between the projection model and clinically acquired data. The single-view X-ray deep learning reconstruction scheme proposed in Chinese Patent CN118608693A, although generating approximate EOS-like three-dimensional data through adversarial training, has limited depth information recovery ability due to the lack of projection dimensions and is prone to topological distortion in the reconstruction of complex anatomical structures. Summary of the Invention

[0004] The object of the present invention is to solve the technical problems of low efficiency and accuracy in existing methods for generating EOS-like three-dimensional data, and to provide a method, a computer program product, and a terminal device for generating EOS-like three-dimensional data from CT scan data.

[0005] To achieve the above object, the technical solution adopted by the present invention is as follows:

[0006] A method for generating EOS-like three-dimensional data from CT scan data, characterized by comprising the following steps:

[0007] Step 1, obtaining a CT scan dataset through a public medical image database and preprocessing the CT scan dataset to obtain a CT standardized dataset;

[0008] Step 2: Construct a projection model based on parameter constraints according to the key projection parameters of the EOS system. Use the projection model to project the CT standardized data in the CT standardized dataset respectively to generate bi-plane DRR data, and obtain a bi-plane DRR dataset. The key projection parameters of the EOS system are obtained by an EOS system parameter experimental calibration method based on geometric constraints.

[0009] Step 3: Use the EOS system to obtain a bi-plane EOS image set, and preprocess the bi-plane EOS image set and the bi-plane DRR dataset to obtain a bi-plane EOS standardized dataset and a bi-plane DRR standardized dataset respectively.

[0010] Step 4: Build an initial style transfer network. Use the bi-plane DRR standardized dataset as the input and the bi-plane EOS standardized dataset as the label to train the initial style transfer network, and obtain a bi-plane EOS-like dataset and a style transfer network.

[0011] Step 5: Construct an initial 3D reconstruction network. Unify the sizes of the bi-plane EOS-like dataset and the CT standardized dataset, and perform data augmentation to obtain a bi-plane EOS-like input dataset and a 3D CT label dataset. Then use the bi-plane EOS-like input dataset as the input and the 3D CT label dataset as the label to train the initial 3D reconstruction network, and obtain a 3D reconstruction network.

[0012] Step 6: Obtain the CT scan data to be processed, and preprocess it to obtain the CT standardized data to be processed. Then, based on the projection model constructed in Step 2, project the CT standardized data to be processed to obtain the bi-plane DRR data to be processed.

[0013] Step 7: Input the bi-plane DRR data to be processed into the style transfer network to generate the bi-plane EOS-like data to be processed. Then input the bi-plane EOS-like data to be processed into the 3D reconstruction network to generate the EOS-like 3D data.

[0014] Further, in Step 4, the initial style transfer network is a DC-STN network. The DC-STN network adopts a symmetric generative adversarial framework, including a two-branch generator and a discriminator.

[0015] The two-branch generator includes two generator branches respectively used to independently process the frontal projection data and the lateral projection data. Both generator branches adopt an encoder-transformer-decoder architecture.

[0016] The encoder backends of the two generator branches jointly set up a shared feature extraction module. The output ends of the shared feature extraction module are respectively connected to the input ends of the transformers of the two generator branches, and are used to fuse the frontal projection features and lateral projection features of the bi-plane DRR data through a cross-view attention mechanism;

[0017] The discriminator adopts a multi-scale PatchGAN structure.

[0018] Further, in step 5, the initial three-dimensional reconstruction network is the X2CT-CycleGAN network, and the X2CT-CycleGAN network includes a 3D reconstruction branch and a 2D reconstruction branch;

[0019] The 3D reconstruction branch includes a 2Dto3D generator and a 3D discriminator. The 2Dto3D generator is used to generate class EOS three-dimensional data, including two 2D encoders for respectively extracting the frontal view features and lateral view features of the bi-plane class EOS input data, a DV-CFM module for aligning the frontal view features and lateral view features, and a 3D decoder for fusing the frontal view features and lateral view features;

[0020] The input ends of the two 2D encoders are respectively used to receive the frontal view and lateral view of the bi-plane class input EOS data, the output ends are respectively connected to the input end of the DV-CFM module, the output end of the DV-CFM module is connected to the input end of the 3D decoder, and the output end of the 3D decoder is connected to the 3D discriminator;

[0021] The 2D reconstruction branch includes a 3Dto2D generator and a 2D discriminator. The 3Dto2D generator is used to generate bi-plane class EOS data, including a 3D encoder and a 2D decoder connected in sequence; the input end of the 3D encoder is used to receive three-dimensional CT label data, and the output end of the 2D decoder is connected to the input end of the 2D discriminator.

[0022] Further, the DV-CFM module includes a feature alignment module, a cross-view attention module, and a multi-scale feature fusion module;

[0023] The feature alignment module includes a lateral encoding unit and an affine transformation unit. The cross-view attention module includes a deformable convolution unit, a frontal encoding unit, a first mapping unit, a second mapping unit, and an attention weight calculation unit;

[0024] The input ends of the front - view encoding unit and the side - view encoding unit are respectively used to connect to the output ends of two 2D encoders. The output end of the front - view encoding unit is respectively connected to the input end of the second mapping unit and the input end of the 3D decoder; the output end of the side - view encoding unit is connected to the input end of the affine transformation unit, the output end of the affine transformation unit is connected to the input end of the deformable convolution unit, and the output end of the deformable convolution unit is connected to the input end of the first mapping unit;

[0025] The input end of the attention weight calculation unit is respectively connected to the output end of the first mapping unit and the output end of the second mapping unit, and the output end is connected to the input end of the multi - scale feature fusion module. The output end of the multi - scale feature fusion module is connected to the input end of the 3D decoder;

[0026] The feature alignment module is used to achieve the spatial alignment of the side - view features and the front - view features, and the cross - view attention module is used to establish the global dependence relationship between the front - view features and the side - view features.

[0027] Further, in step 4, the loss function of the initial style transfer network is:

[0028] L1 = L DA + L BCC + L CMFC + L MSD + L CAR

[0029] L DA = E A～p(A) [(D B (G(A)) - 1) 2 + E B～p(B) [(D A (F(B)) - 1) 2

[0030] L BCC = λ cyc (E A～p(A) [||F(G(A)) - A||1]+ E B～p(B) [||G(F(B)) - B||1])

[0031]

[0032] Among them, L1 is the loss function of the initial style transfer network;

[0033] L DA is the two - channel adversarial loss, A is the front - view, G(A) represents generating a side - view according to the front - view; B is the side - view, F(B) represents generating a front - view according to the side - view; D B (G(A)), D A ​Let \(E[D(G(A))]\) and \(E[D(F(B))]\) denote the discriminative losses corresponding to \(G(A)\) and \(F(B)\) respectively; \(E A～p(A) denotes the expected value of the frontal view under its probability distribution, \(E B～p(B) denotes the expected value of the side view under its probability distribution;

[0034] Let \(L BCC be the bidirectional cycle consistency loss, \(\lambda cyc be the cycle consistency weight coefficient, \(F(G(A))\) represents restoring the side view generated from the frontal view to the frontal view, \(G(F(B))\) represents restoring the frontal view generated from the side view to the side view, and \(\|\cdot\|_1\) represents taking the L1 norm;

[0035] Let \(L CMFC be the cross-modal feature consistency loss, \(\lambda feat be the feature alignment strength, \(E A,B denotes taking the expectation of the joint distribution of the frontal view and the side view; Let \(C_l l be the number of channels of the shared feature map at the \(l\)-th layer, \(H l be the height dimension of the shared feature map at the \(l\)-th layer, \(W l be the width dimension of the shared feature map at the \(l\)-th layer, \(l\) is an integer, and \(1\leq l\leq L\), where \(L\) is the total number of layers of the shared feature map;

[0036] Let \(L MSD be the multi-scale discriminative loss, and be the \(s\)-th level discriminative losses corresponding to \(G(A)\) and \(F(B)\) respectively, \(s\) is an integer, and \(1\leq s\leq S\), where \(S\) is the total number of levels of the discriminator;

[0037] Let \(L CAR be the channel attention regularization loss, \(\lambda attn be the channel attention adjustment parameter, \(W c be the learnable attention weight for the \(c\)-th channel, \(f c be the feature of the \(c\)-th channel, \(c\) is an integer, and \(1\leq c\leq C\), where \(C\) is the number of channels, \(\text{softmax}()\) represents the normalization function, and \(\text{ReLU}()\) represents the rectified linear unit function.

[0038] Furthermore, in step 5, the loss function of the initial three-dimensional reconstruction network is:

[0039] \(L_2 = L GAN (G A2B , D_3, X, Y)+L GAN (G B2A , D_2, Y, X)+L cyc (G A2B , G B2A )

[0040] +L identity (G A2B ,G B2A ,X,Y)+L D (D3,X,Y,G)+L D (D2,X,Y,G)

[0041]

[0042] Among them, L2 is the loss function of the initial 3D reconstruction network, G A2B represents the 2Dto3D generator, D3 represents the 3D discriminator, G A2B represents the 3Dto2D generator, D2 represents the 2D discriminator; X represents the dual-plane class EOS input data set, Y represents the 3D CT label data set;

[0043] L GAN (G A2B ,D3,X,Y) is the generative adversarial loss of the 3D reconstruction branch, x is the dual-plane class EOS input data, y is the 3D CT label data, represents the expected value of the dual-plane class EOS input data under its probability distribution, represents the expected value of the 3D CT label data under its probability distribution; D3(y) represents the probability of classifying the 3D CT label data as a real image, G A2B (x) represents the class EOS 3D data generated according to the dual-plane class EOS input data, D2(G A2B (x)) represents the probability of classifying the class EOS 3D data generated according to the dual-plane class EOS input data as a real image;

[0044] L GAN (G B2A ,D2,Y,X) represents the generative adversarial loss of the 2D reconstruction branch, D2(x) represents the probability of classifying the dual-plane class EOS input data as a real image, G B2A (y) represents the generation of dual-plane class EOS data according to the 3D CT label data, D2(G B2A (y)) represents the probability of classifying the dual-plane class EOS data generated according to the 3D CT label data as a real image;

[0045] L cyc (G A2B ,G B2A ) is the cycle consistency loss, G B2A (G A2B (x)) represents restoring the class EOS 3D data generated according to the dual-plane class EOS input data to the dual-plane class EOS data, G A2B (G B2A(y) represents restoring the dual - plane - like EOS data generated from 3D CT label data to 3D CT data;

[0046] L identity (G A2B ,G B2A ,X,Y) is the identity loss, G A2B (y) represents generating EOS - like 3D data from 3D CT label data, G B2A (x) is generating dual - plane - like EOS data from dual - plane - like EOS input data;

[0047] L D (D3,X,Y,G) represents the discrimination loss of the 3D discriminator; D2(G B2A (G A2B (x))) represents the probability of discriminating the dual - plane - like EOS data restored from the EOS - like 3D data generated from dual - plane - like EOS input data as a real image;

[0048] L D (d2,X,Y,G) represents the discrimination loss of the 2D discriminator, D3(G A2B (G B2A (y))) represents the probability of discriminating the 3D CT data restored from the dual - plane - like EOS data generated from 3D CT label data as a real image.

[0049] Furthermore, in step 4, the specific method for training the initial style transfer network is as follows: Using the Adam optimizer, alternately optimizing the parameters of the dual - branch generator and the discriminator by the gradient descent method;

[0050] Among them, the specific method of alternately optimizing the parameters of the dual - branch generator and the discriminator is as follows:

[0051] First, fix the parameters of the dual - branch generator, and adjust the parameters of the discriminator according to the difference between the dual - plane - like EOS data generated by the dual - branch generator and the corresponding EOS dual - plane standardized data;

[0052] Second, fix the parameters of the discriminator, and adjust the parameters of the dual - branch generator according to the similarity between the dual - plane - like EOS data generated by the dual - branch generator and the corresponding EOS dual - plane standardized data.

[0053] Furthermore, step 2 is specifically as follows:

[0054] Step 2.1, obtaining the key projection parameters of the EOS system by the experimental calibration method of the EOS system parameters based on geometric constraints; the key projection parameters of the EOS system include the distance from the projection source to the detector of the EOS system, the projection position of the projection source on the detector of the EOS system, and the distance from the projection source to the rotation center of the EOS system;

[0055] Step 2.2. Iteratively optimize the key projection parameters of the EOS system through the dynamic coordinate transformation algorithm, so that the optimized key projection parameters of the EOS system satisfy the physical alignment constraint between the projection ray bundle and the EOS system detector as shown in the following formula:

[0056] ||R·v - v'||2 < 0.1 mm

[0057] where R is the rotation matrix representing the rotational transformation of the EOS system detector coordinate system or the projection ray bundle direction; v is the direction vector of the projection ray bundle, v' is the target direction vector received by the EOS system detector, and || ||2 represents taking the L2 norm;

[0058] Step 2.3. Construct a projection model based on parameter constraints according to the optimized key projection parameters of the EOS system; the projection model includes an anteroposterior projection coordinate system and a lateral projection coordinate system;

[0059] Step 2.4. In the anteroposterior projection coordinate system of the projection model, use the multi-angle ray integration method to make the X-ray source emit discrete ray bundles at multiple preset angles respectively, calculate the attenuation amount of each voxel in the CT standardized data along the path of the ray bundle passing through the CT standardized data, and obtain the multi-angle anteroposterior projection data through the following formula:

[0060]

[0061] where I is the anteroposterior projection data, μ i represents the linear attenuation coefficient of the i-th layer voxel in the CT standardized data, which is converted from the HU value through the proportional coefficient k = 0.85, l i is the path length of the ray bundle passing through the i-th layer voxel, i is an integer, and 0 ≤ i ≤ N, where N is the number of voxel layers of the CT standardized data;

[0062] Step 2.5. Establish a two-parameter correction matrix, and correct the image distortion of the multi-angle anteroposterior projection data through the following formula to obtain the multi-angle anteroposterior projection correction data:

[0063]

[0064] where (x', y') are the coordinates of each pixel point in the anteroposterior projection correction data, (x, y) are the coordinates of each pixel point in the anteroposterior projection data, θ is the rotation offset angle of the EOS system detector plane, Δd x is the position deviation compensation amount of the x-axis, Δd y is the position deviation compensation amount of the y-axis;

[0065] Step 2.6: Using the weighted back-projection algorithm, map the multi-angle frontal projection correction data to the detector of the EOS system through the following formula to obtain the frontal DRR data:

[0066]

[0067] where P DRR is the frontal DRR data, ω i is the angular weight factor of the i-th voxel in the CT normalization data, P j is the integration result of the frontal projection correction data at the j-th angle, and m is the number of preset angles;

[0068] Step 2.7: According to the method of Step 2.4 - Step 2.6, in the lateral projection coordinate system of the projection model, obtain the lateral DRR data, thereby obtaining the bi-plane DRR data;

[0069] Step 2.8: Repeat Step 2.4 - Step 2.7 until all CT normalization data in the CT normalization dataset are traversed to obtain the bi-plane DRR dataset.

[0070] Further, in Step 1, the specific method for preprocessing the CT scan dataset is as follows:

[0071] Convert the pixel values of the CT scan data in the CT scan dataset into HU values respectively, and then perform dynamic optimization adjustment of the window width and window level on the HU values to obtain the CT scan HU dataset; then establish a two-level regional annotation system including the overall anatomical region mask and the bone region mask, and use the two-level regional annotation system to generate the overall anatomical region mask and the bone region mask by the label image method and annotate them in the CT scan HU dataset;

[0072] In Step 3, the specific method for preprocessing the bi-plane EOS image set and the bi-plane DRR dataset is as follows:

[0073] Convert the bi-plane EOS images in the bi-plane EOS image dataset and the bi-plane DRR data in the bi-plane DRR dataset into 512×512 pixel resolution images respectively through the bicubic interpolation algorithm, then adopt the random geometric enhancement strategy, and then linearly map the pixel values to the interval [0,1] through maximum-minimum normalization;

[0074] In Step 5, the data enhancement includes random rotation of ±5°, translation of ±10%, and gray-scale jitter of ±15%.

[0075] The present invention also provides a computer program product, including a computer program, which when executed by a processor implements the steps of the above method for generating EOS-like three-dimensional data from CT scan data.

[0076] The present invention also provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the method for generating EOS-like three-dimensional data from CT scan data described above are implemented.

[0077] Compared with the prior art, the present invention has the following beneficial technical effects:

[0078] 1. The method for generating EOS-like three-dimensional data from CT scan data provided by the present invention constructs a physical projection model based on the geometric parameters of the EOS system, accurately restores the geometric projection process of EOS system imaging, and establishes a style transfer network and a three-dimensional reconstruction network with the aid of deep learning to achieve imaging style adaptation, greatly improving the generation efficiency of EOS-like three-dimensional data. At the same time, it effectively solves the problems of parameter generalization and feature mismatch, and improves the accuracy of generating EOS-like three-dimensional data.

[0079] 2. The method for generating EOS-like three-dimensional data from CT scan data provided by the present invention has a dual-channel anatomical constraint in the initial style transfer network. By using the perspective complementary characteristics of the anteroposterior and lateral projections, the conversion of CT scan data into dual-plane EOS-like data is realized. In cooperation with the physical projection model, noise pattern adaptation, gray-scale space alignment, and edge sharpness adjustment are achieved through style conversion, generating high-quality dual-plane EOS-like data, and further improving the accuracy of subsequent generated EOS-like three-dimensional data.

[0080] 3. The method for generating EOS-like three-dimensional data from CT scan data provided by the present invention realizes high-precision cross-modal reconstruction of dual-plane EOS-like data into EOS-like three-dimensional data through the adversarial learning architecture of a dual-generator and a dual-discriminator in the initial three-dimensional reconstruction network. The anatomical structure rationality of the generated image is ensured through bidirectional mapping and cycle consistency constraints, and the high fidelity of the generated result in geometric structure and texture details is ensured through the collaborative optimization of adversarial training and cycle consistency constraints.

[0081] 4. The method for generating EOS-like three-dimensional data from CT scan data provided by the present invention embeds a DV-CFM module in the 2Dto3D generator. The lateral view features are aligned to the coordinate system of the anteroposterior view through affine transformation, and the spatial position of the features is adjusted by combining deformable convolutional units through adaptive learning of the offset amount, so as to more accurately achieve feature alignment. At the same time, the cross-view attention unit can capture the global dependence relationship between the dual-view features while adaptively enhancing the expression ability of important features. At the same time, the multi-scale feature fusion unit is used to capture context information at different scales to improve the model's perception ability of targets at different scales and enhance the robustness of the three-dimensional reconstruction network.

[0082] 5. A method for generating EOS-like three-dimensional data from CT scan data provided by the present invention uses the Adam optimizer to alternately optimize the parameters of the dual-branch generator and the dual-branch discriminator through the gradient descent method, trains the initial style transfer network, can effectively balance the adjustment of the gradient direction and amplitude, effectively suppress the mode collapse and gradient disappearance problems in training, and achieve high-fidelity cross-view conversion of multi-modal medical images;

[0083] 6. A method for generating EOS-like three-dimensional data from CT scan data provided by the present invention uses the multi-angle ray integration method, can dynamically adjust the integration step size to balance the calculation efficiency and accuracy, and uses the two-parameter correction matrix to correct the image distortion caused by the non-ideal layout of the detector, which can improve the accuracy of generating dual-plane DRR data from CT standardized data projections;

[0084] 7. A method for generating EOS-like three-dimensional data from CT scan data provided by the present invention preprocesses the CT scan data set, and through HU conversion and dynamic optimization adjustment of window width and window level, not only improves the recognition of soft tissue structures, but also can customize the display parameters according to specific application scenarios, avoiding detail loss or noise interference caused by improper dynamic range;

[0085] 8. A method for generating EOS-like three-dimensional data from CT scan data provided by the present invention preprocesses the dual-plane EOS image set and the dual-plane DRR data set. Through the random geometric enhancement strategy, combined with the linear mapping of pixel values, it can improve the robustness of the network to projection angle deviation and eliminate the difference in gray distribution between different devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0086] Figure 1 is the flowchart of the method of the embodiment of the present invention;

[0087] Figure 2 is the schematic diagram of the projection plane in step 2.1 of the embodiment of the present invention;

[0088] Figure 3 is the schematic diagram of the projection plane coordinate system in step 2.1 of the embodiment of the present invention, and the rotation states of the steel rulers in (a) and (b) are different;

[0089] Figure 4 is the architecture diagram of the dual-branch generator of the DC-STN network in step 4 of the embodiment of the present invention;

[0090] Figure 5 is the discriminator architecture diagram of the DC-STN network in step 4 of the embodiment of the present invention;

[0091] Figure 6 is the architecture diagram of the X2CT-CycleGAN network in step 5 of the embodiment of the present invention;

[0092] Figure 7 This is the architecture diagram of the DV-CFM module of the X2CT-CycleGAN network in step 5 of the embodiment of the present invention. Detailed implementation manners

[0093] The following further elaborates in detail a method, a computer program product, and a terminal device for generating EOS-like three-dimensional data from CT scan data proposed by the present invention in conjunction with the accompanying drawings and specific implementation manners. Those skilled in the art should understand that these implementation manners are only used to explain the technical principle of the present invention, and the purpose is not to limit the protection scope of the present invention.

[0094] A method for generating EOS-like three-dimensional data from CT scan data, as Figure 1 shown, includes the following steps:

[0095] Step 1: Obtain a CT scan data set through a public medical image database, and preprocess the CT scan data set to obtain a CT standardized data set.

[0096] The specific method for preprocessing the CT scan data set is as follows: Convert the pixel values of the CT scan data in the CT scan data set into HU values respectively, and then perform dynamic optimization adjustment of the window width and window level on the HU values to obtain a CT scan HU data set; then establish a two-level regional annotation system including an overall anatomical region mask and a bone region mask, and use the two-level regional annotation system to generate the overall anatomical region mask and the bone region mask by the label image method and label them in the CT scan HU data set.

[0097] Converting the image pixel values of the CT scan data set into HU values can provide a standardized benchmark for subsequent tissue recognition and quantitative analysis by quantifying the X-ray absorption characteristics of different tissues, and effectively eliminate the systematic errors caused by equipment parameter differences. Then perform dynamic optimization adjustment of the window width and window level on the HU values. The window width controls the displayed grayscale range, and the window level sets the central reference value. The coordinated adjustment of the two can significantly enhance the contrast of the target tissue. The above steps not only improve the recognition rate of soft tissue structures by simulating the real X-ray attenuation characteristics, but also can customize the display parameters according to specific application scenarios to avoid the loss of details or noise interference caused by improper dynamic range.

[0098] Meanwhile, for the subsequent projection calculation requirements, a two-level regional annotation system including an overall anatomical region mask and a bone region mask needs to be established in the preprocessing steps of the CT scan dataset. The overall anatomical region mask is used to eliminate the interference of non-biological structures such as the scanning bed, and the bone region mask is used to accurately extract the skeletal structures of interest to enhance their feature expression in subsequent experiments. The binary mask is generated by the label image method, which can accurately extract anatomical structures while maintaining the original spatial resolution, effectively improving the signal-to-noise ratio of the projection data, ensuring that only the attenuation information related to the target anatomy is retained during DRR generation, and providing a reliable input data basis for subsequent steps.

[0099] Step 2: According to the key projection parameters of the EOS system, construct a projection model based on parameter constraints, and use the projection model to project the CT standardized data in the CT standardized dataset to generate bi-plane DRR data respectively, obtaining a bi-plane DRR dataset. Specifically:

[0100] Step 2.1: Obtain the key projection parameters of the EOS system based on the geometric constraint-based experimental calibration method for EOS system parameters. Among them, the key projection parameters of the EOS system include the distance from the projection source to the EOS system detector, the projection position of the projection source on the EOS system detector, and the distance from the projection source to the rotation center of the EOS system. As Figure 2 shown, the distance from the projection source to the EOS system detector is denoted by l SC According to the geometric relationship, we can get:

[0101]

[0102] where l ruler is the length of the steel ruler, l AE is the projection length of the steel ruler on the EOS system detector, and l GC is the distance between the two phantoms, all of which are known quantities.

[0103] The projection point of the projection source S on the EOS system detector is point C. Using l AC and l EC to represent the projection position of the projection source on the EOS system detector, we have:

[0104]

[0105] where l AB and l ED are the differences between the upper and lower boundaries of the steel ruler projection respectively, and l FB = l HD = l GC is the distance between the two phantoms, all of which are known quantities.

[0106] Figure 3In (a), the coordinates of the two endpoints P1 and P2 of the steel ruler are respectively:

[0107]

[0108]

[0109] Figure 3 In (b), the coordinates of the two endpoints P1 and P2 of the steel ruler are respectively:

[0110]

[0111] Among them, x1, x2, y1, and y2 can all be measured in the Figure 2 shown projection plane. At this time, the unknowns are x and y, and the known length of the steel ruler is l ruler , according to ||P1P2|| = l ruler , substituting it can obtain the relationship between x and y. Let x = y, then x = y = 40.404 can be obtained, and the distance from the projection source to the rotation center of the EOS system can be obtained as l SC -x.

[0112] Step 2.2: Iteratively optimize the key projection parameters of the EOS system through the dynamic coordinate transformation algorithm, so that the optimized key projection parameters of the EOS system satisfy the physical alignment constraint of the projection ray bundle and the EOS system detector shown in the following formula:

[0113] ||R·v - v'||2 < 0.1mm

[0114] Among them, R is the rotation matrix, which is used to describe the rotation transformation of the EOS system detector coordinate system or the projection ray bundle direction; v is the direction vector of the projection ray bundle, and v' is the target direction vector received by the EOS system detector.

[0115] Step 2.3: Construct a projection model based on parameter constraints according to the optimized key projection parameters of the EOS system. Among them, the projection model includes an anterior-posterior projection coordinate system and a lateral projection coordinate system.

[0116] The key projection parameters of the EOS system are used as the physical constraint conditions for ray tracing to ensure that the simulated projection path strictly matches the real projection path.

[0117] Step 2.4: In the anterior-posterior projection coordinate system of the projection model, use the multi-angle ray integration method to make the X-ray source emit discrete ray bundles at multiple preset angles respectively, calculate the attenuation amount of each voxel in the CT standardized data along the path of the ray bundle passing through the CT standardized data, and obtain the multi-angle anterior-posterior projection data through the following formula:

[0118]

[0119] Among them, I is the frontal projection data, and μ i represents the linear attenuation coefficient of the i-th voxel in the CT standardized data, which is converted from the HU value through the proportionality coefficient k = 0.85. l i is the path length of the ray beam passing through the i-th voxel. i is an integer, and 0 ≤ i ≤ N, where N is the number of voxel layers in the CT standardized data.

[0120] In this step, it is necessary to traverse the paths of all X-ray beams in the entire three-dimensional space and dynamically adjust the integration step size to balance the calculation efficiency and accuracy.

[0121] Step 2.5: Establish a two-parameter correction matrix, and correct the image distortion of the multi-angle frontal projection data through the following formula to obtain the multi-angle frontal projection correction data:

[0122]

[0123] Among them, (x′, y′) are the coordinates of each pixel point in the frontal projection correction data, (x, y) are the coordinates of each pixel point in the frontal projection data, θ is the rotation offset angle of the EOS system detector plane, and Δd x is the position deviation compensation amount of the x-axis, and Δd y is the position deviation compensation amount of the y-axis.

[0124] Step 2.6: Adopt the weighted back-projection algorithm, and map the multi-angle frontal projection correction data to the detector of the EOS system through the following formula to obtain the frontal DRR data:

[0125]

[0126] Among them, P DRR is the frontal DRR data, ω i is the angular weight factor of the i-th voxel in the CT standardized data, and P j is the integration result of the frontal projection correction data at the j-th angle, and m is the number of preset angles.

[0127] In other embodiments, the projection data can also be processed by frequency domain filtering by embedding the spatial response function of the EOS detector to suppress high-frequency noise and enhance the sharpness of the bone edges.

[0128] Step 2.7: According to the methods in Step 2.4 - Step 2.6, obtain the lateral DRR data in the lateral projection coordinate system of the projection model, so as to obtain the bi-plane DRR data.

[0129] Step 2.8: Repeat Step 2.4 - Step 2.7 until all CT standardized data in the CT standardized dataset are traversed to obtain the bi-plane DRR dataset.

[0130] Step 2 realizes the cross - dimensional mapping from three - dimensional CT standardized data to two - dimensional projection image double - plane DRR data by simulating the physical characteristics of EOS system X - ray imaging.

[0131] Step 3: Use the EOS system to obtain a double - plane EOS image set, and pre - process the double - plane EOS image set and the double - plane DRR data set to obtain a double - plane EOS standardized data set and a double - plane DRR standardized data set respectively.

[0132] Among them, the specific method for pre - processing the double - plane EOS image set and the double - plane DRR data set is as follows: The specific method for pre - processing the double - plane EOS image data set is: convert the double - plane EOS image data set into an image with a resolution of 512×512 pixels through the bicubic interpolation algorithm, then adopt a random geometric enhancement strategy, and then linearly map the pixel values to the interval [0,1] through maximum - minimum normalization, which can eliminate the differences in gray - level distributions between different devices. Among them, the random geometric enhancement strategy includes rotation of ±15°, translation of ±10%, and scaling of 0.9 - 1.1 times, which can improve the robustness of the model to projection - angle deviations.

[0133] Step 4: Build an initial style - transfer network, use the double - plane DRR standardized data set as the input, and the double - plane EOS standardized data set as the label, and train the initial style - transfer network to obtain a double - plane EOS - like data set and a style - transfer network.

[0134] Although the double - plane DRR data generated based on the physical parameters of the EOS system can reproduce the geometric characteristics of the EOS system, due to the unique linear scanning technology and low - radiation - dose mode of the real EOS system, its noise distribution, gray - level hierarchy, and soft - tissue contrast are all different from those of the double - plane DRR data. Moreover, the EOS system uses standing - position imaging, while the CT scanning system uses supine - position imaging, resulting in differences in texture details and brightness distributions between the real double - plane EOS data and the double - plane DRR data derived from CT scanning data. To solve the above problems, a style - transfer network is introduced. On the basis of retaining the anatomical accuracy of the double - plane DRR data, noise - pattern adaptation, gray - space alignment, and edge - sharpness adjustment are realized through style conversion to generate high - quality double - plane EOS - like data.

[0135] In this embodiment, the initial style - transfer network is a DC - STN network. The DC - STN network adopts a symmetric generative adversarial framework, including a two - branch generator and a discriminator. As Figure 4As shown, the dual-branch generator includes two generator branches that independently process the frontal projection data and the lateral projection data respectively. Both generator branches adopt an encoder-transformer-decoder architecture. Among them, the first layer of the encoder uses a large kernel convolution of 7×7 with a stride of 2 to capture the global anatomical structure, and subsequent cascaded 3×3 grouped convolution layers refine the local features. A shared feature extraction module is jointly set at the back end of the encoders of the two generator branches. The output end of the shared feature extraction module is respectively connected to the input end of the transformer of the two generator branches, which is used to fuse the frontal projection features and the lateral projection features of the dual-plane DRR data through the cross-view attention mechanism. The transformer introduces deformable convolution to establish multi-scale spatial associations, combines channel attention weighting to dynamically strengthen the anatomical features, automatically identifies the common structure between the frontal projection features and the lateral projection features, and improves the conversion accuracy. The dual-branch generator finds the association between the frontal projection features and the lateral projection features through a bidirectional cyclic reconstruction path.

[0136] The discriminator adopts a multi-scale PatchGAN structure, as Figure 5 shown, synchronously discriminates local textures and global anatomical features through a hierarchical feature pyramid. The local discrimination branch with a receptive field of 16×16 focuses on fine structures, and the global discrimination branch with a receptive field of 64×64 pays attention to the overall structure.

[0137] For the architecture of the DC-STN network in the multi-view conversion task of medical images, its loss function needs to expand the multi-dimensional supervision mechanism on the basis of the traditional CycleGAN, as shown in the following formula:

[0138] L1 = L DA + L BCC + L CMFC + L MSD + L CAR

[0139] L DA = E A～p(A) [(D B (G(A)) - 1) 2 + E B～p(B) [(D A (F(B)) - 1) 2

[0140] L BCC = λ cyc (E A～p(A) [||F(G(A)) - A||1] + E B～[(B) [||G(F(B)) - B||1])

[0141]

[0142] Among them, L1 is the loss function of the initial style transfer network;​

[0143] L DA is the dual-channel adversarial loss, A is the frontal view, and G(A) represents generating a side view according to the frontal view; B is the side view, and F(B) represents generating a frontal view according to the side view; D B (G(A)), D A (F(B)) represent the discriminative losses corresponding to G(A) and F(B) respectively; E A～p(A) represents the expected value of the frontal view under its probability distribution, and E B～p(B) represents the expected value of the side view under its probability distribution;

[0144] L BCC is the bidirectional cycle consistency loss, and λ cyc is the cycle consistency weight coefficient. F(G(A)) represents restoring the side view generated from the frontal view to the frontal view, and G(F(B)) represents restoring the frontal view generated from the side view to the side view, and ||||1 represents taking the L1 norm;

[0145] L CMFC is the cross-modal feature consistency loss, and λ feat is the feature alignment strength, and E A,B represents taking the expectation of the joint distribution of the frontal view and the side view; represents the shared feature map of the l-th layer of the encoder, C l is the number of channels of the shared feature map of the l-th layer, H l is the dimensionality of the height of the shared feature map of the l-th layer, W l is the dimensionality of the width of the shared feature map of the l-th layer. l is an integer, and 1 ≤ l ≤ L, where L is the total number of layers of the shared feature map;

[0146] L MSD is the multi-scale discriminative loss, and are the s-th level discriminative losses corresponding to G(A) and F(B) respectively. s is an integer, and 1 ≤ s ≤ S, where S is the total number of levels of the discriminator;

[0147] L CAR is the channel attention regularization loss, and λ attn is the channel attention adjustment parameter, W c is the learnable attention weight of the c-th channel, f c is the feature of the c-th channel. c is an integer, and 1 ≤ c ≤ C, where C is the number of channels. softmax() represents the normalization function, and ReLU() represents the rectified linear unit function.

[0148] Among them, the dual-channel adversarial loss is used to drive the generator to generate images with a distribution consistent with the target domain. At the same time, the discriminator is used to capture the differences in global anatomical features between the frontal and lateral projections, replacing the traditional cross-entropy loss and alleviating the problem of gradient disappearance. The bidirectional cycle consistency loss is used to enforce the topological constraints in the generation process, ensuring the spatial continuity of anatomical structures during the projection conversion process. The cross-modal feature consistency loss is used to constrain the feature fusion process of the shared layers of the dual-branch encoder, strengthening the three-dimensional anatomical correlation between the frontal and lateral projections. The multi-scale discriminative loss is used to synchronously optimize the local texture fidelity and the global anatomical structure rationality through the hierarchical feature pyramid discriminator, enhancing the discriminative ability for cross-resolution features such as organ boundaries and vascular branches. The channel attention regularization loss is used to enhance the feature response of key anatomical regions through sparsity constraints, guiding the cross-view attention module to dynamically weight the shared anatomical features and suppressing redundant noise interference.

[0149] During the training process of the DC-STN network, the Adam optimizer is adopted, and the parameters of the dual-branch generator and discriminator are alternately optimized by the gradient descent method. That is, in the first stage, the parameters of the dual-branch generator are fixed, and the parameters of the discriminator are adjusted according to the difference between the dual-plane-like EOS data generated by the dual-branch generator and the corresponding EOS dual-plane standardized data; in the second stage, the parameters of the discriminator are fixed, and the parameters of the dual-branch generator are adjusted according to the similarity between the dual-plane-like EOS data generated by the dual-branch generator and the corresponding EOS dual-plane standardized data. The alternating optimization process realizes the dynamic balance of the generative adversarial through iterative optimization, and finally prompts the DC-STN network to converge to a stable state. Combining with the gradient self-adaptive characteristics of Adam, it can effectively suppress the mode collapse and gradient disappearance problems that are prone to occur during training.

[0150] The Adam optimizer is adopted to achieve efficient update of parameters. It can combine the advantages of the momentum method and the adaptive learning rate adjustment, dynamically calculate the mean and uncentered variance of the gradients, so as to allocate independent learning rates for different parameters, effectively balancing the adjustment of the gradient direction and amplitude. It is especially suitable for the gradient sparse and non-stationary target scenarios common in medical image multi-modal transformation tasks. The built-in bias correction mechanism of the Adam optimizer can correct the gradient estimation in the initial stage of training, avoiding the parameter update from biasing towards zero. The parameters of the Adam optimizer are set as follows: the learning rate is fixed at 2×10 -4 , which can ensure the convergence speed while avoiding the training oscillation caused by too high a learning rate; the momentum parameter β1 is set to 0.5, and β2 is set to 0.999. Among them, β1 can reduce the influence of historical gradients on the current update, while β2 enhances the stability of the uncentered variance. The combined effect of the two can alleviate the bias problem caused by gradient sparsity.

[0151] Step 5: Construct an initial 3D reconstruction network. Unify the sizes of the biplane-like EOS dataset and the CT standardized dataset, and perform data augmentation using random rotation of ±5°, translation of ±10%, and gray-scale jitter of ±15% to obtain the biplane-like EOS input dataset and the 3D CT label dataset. Then, use the biplane-like EOS input dataset as the input and the 3D CT label dataset as the label to train the initial 3D reconstruction network to obtain the 3D reconstruction network.

[0152] The initial 3D reconstruction network is the X2CT-CycleGAN network, which is an extension based on CycleGAN. As Figure 6 shown, the X2CT-CycleGAN network includes a 3D reconstruction branch and a 2D reconstruction branch. Through the adversarial learning architecture of dual generators and dual discriminators, it realizes high-precision cross-modal reconstruction from biplane-like EOS data to EOS-like 3D data, and ensures the anatomical structure rationality of the generated images through bidirectional mapping and cycle consistency constraints.

[0153] The 3D reconstruction branch includes a 2Dto3D generator and a 3D discriminator. The 2Dto3D generator is used to generate EOS-like 3D data, including two 2D encoders for respectively extracting the frontal view features and the lateral view features of the biplane-like EOS input data, a DV-CFM module for aligning the frontal view features and the lateral view features, and a 3D decoder for fusing the frontal view features and the lateral view features. The input ends of the two 2D encoders are respectively used to receive the frontal view and the lateral view of the biplane-like input EOS data, the output ends are respectively connected to the input end of the DV-CFM module, the output end of the DV-CFM module is connected to the input end of the 3D decoder, and the output end of the 3D decoder is connected to the 3D discriminator.

[0154] The 2D reconstruction branch includes a 3Dto2D generator and a 2D discriminator. The 3Dto2D generator is used to generate biplane-like EOS data, including a 3D encoder and a 2D decoder connected in sequence. The input end of the 3D encoder is used to receive the 3D CT label data, and the output end of the 2D decoder is connected to the input end of the 2D discriminator.

[0155] The 2D discriminator and the 3D discriminator adopt a multi-scale convolution module, and through collaborative optimization of adversarial training and cycle consistency constraints, ensure the high fidelity of the generated images in geometric structure and texture details.

[0156] The 2D-to-3D generator combines the multi-scale feature retention ability of U-Net and the dynamic feature fusion characteristics of the attention mechanism. The inputs are the frontal view and the lateral view of bi-plane EOS-like data with a size of 256×256 respectively, and feature extraction is performed through independent 2D encoders. Each 2D encoder contains five levels of downsampling modules, each level consisting of two 3×3 convolutional layers with a stride of 1, instance normalization, and ReLU activation functions, followed by a 2×2 max pooling operation. The number of convolutional channels increases gradually from 16 initially to 256, and the size of the feature map is compressed from 256×256 to 8×8. Deformable convolution is introduced after the third feature layer, and the sampling position of the convolutional kernel is dynamically adjusted through learnable offsets to correct the spatial misalignment caused by the difference in projection angles between the frontal view and the lateral view.

[0157] The DV-CFM module, as the core component of the 2D-to-3D generator, is responsible for the dynamic alignment and fusion of bi-plane EOS-like data features to support the generation of high-quality EOS-like 3D data. After receiving the multi-scale features output by the encoder, this module maps the lateral view features to the coordinate system of the frontal view through affine transformation, calculates the feature correlation using the cross-view attention mechanism, and then generates context-enhanced features through weighted summation. The fused features are compressed to the original number of channels through channel concatenation and 1×1 convolution, and are injected into the corresponding levels of the 3D decoder in combination with U-Net-style skip connections. In this embodiment, the lateral view features are mapped to the coordinate system of the frontal view to align the lateral view features with the frontal view features. In other embodiments, the frontal view features can also be mapped to the coordinate system of the lateral view to align the frontal view features with the lateral view features.

[0158] As Figure 7 shown, the DV-CFM module includes a feature alignment module, a cross-view attention module, and a multi-scale feature fusion module. The feature alignment module includes a lateral encoding unit and an affine transformation unit, and the cross-view attention module includes a deformable convolution unit, a frontal encoding unit, a first mapping unit, a second mapping unit, and an attention weight calculation unit.

[0159] The input ends of the frontal encoding unit and the lateral encoding unit are respectively used to connect to the output ends of the two 2D encoders. The output end of the frontal encoding unit is respectively connected to the input end of the second mapping unit and the input end of the 3D decoder. The output end of the lateral encoding unit is connected to the input end of the affine transformation unit, the output end of the affine transformation unit is connected to the input end of the deformable convolution unit, and the output end of the deformable convolution unit is connected to the input end of the first mapping unit. The input end of the attention weight calculation unit is respectively connected to the output ends of the first mapping unit and the second mapping unit, and the output end is connected to the input end of the multi-scale feature fusion module. The output end of the multi-scale feature fusion module is connected to the input end of the 3D decoder.

[0160] Among them, the feature alignment module is used to achieve the spatial alignment of the lateral view features and the frontal view features, and the cross-view attention module is used to establish the global dependence relationship between the frontal view features and the lateral view features. The first mapping unit is used to generate the key vector and the value vector according to the aligned lateral view features, the second mapping unit is used to generate the query vector according to the frontal view features, and the attention weight calculation unit is used to calculate the cross-view attention weight matrix according to the query vector, the key vector and the value vector, and generate the enhanced frontal view features and lateral view features according to the cross-view attention weight matrix.

[0161] The purpose of the DV-CFM module is to solve the problems of spatial and semantic misalignment of bi-plane EOS-like data, and at the same time make full use of the complementarity of the two-view features to enhance the expression ability of the context information, so as to improve the accuracy of 3D reconstruction and the ability to restore details. In the task of generating EOS-like 3D data from bi-plane EOS-like data, the frontal view and the lateral view obtain the projection information of the target through different imaging angles. Due to the difference in the imaging perspectives of the two views, there are obvious asymmetry problems in the spatial distribution and semantic expression of their image features. If a direct fusion strategy is adopted for processing, information mismatch and reconstruction distortion will be caused. The DV-CFM module can achieve the functions of dynamic feature alignment and fusion. Based on the natural information complementarity characteristics of the frontal and lateral views, the 3D structure of the target can be restored more completely by integrating the projection data information from different perspectives. Aiming at the problem of misalignment of the spatial features of the two views, an affine transformation and an attention mechanism are introduced to achieve dynamic feature alignment. Finally, combined with the high sensitivity requirement of 3D reconstruction for global context information, a cross-view attention network is used to establish the global dependence relationship between features and enhance the expression ability of context information.

[0162] The core task of the DV-CFM module is to extract hierarchical feature expressions from bi-plane EOS-like data and achieve dynamic feature alignment and semantic fusion of features through cross-view interaction. Two 2D encoders receive the frontal view and the lateral view of the bi-plane EOS-like data as inputs, and extract multi-scale features through CNN. Each layer of features is respectively represented as and where k represents the feature level.

[0163] Aiming at the problem of misalignment of features in space caused by different imaging perspectives of the frontal and lateral views, the DV-CFM module maps the lateral view features to the coordinate system of the frontal view through an affine transformation to obtain the aligned features The affine transformation is a linear transformation method that can align the lateral view features to the coordinate system of the frontal view through operations such as translation, rotation and scaling. Its mathematical expression is as follows:

[0164]

[0165] Among them, is the lateral view feature of the k-th layer, is the mapped lateral view feature of the k-th layer. T(·; θ) represents the affine transformation function, and θ is the transformation parameter, including translation, rotation, and scaling. By optimizing θ, the spatial alignment between the lateral view feature and the frontal view feature can be achieved.

[0166] In order to further enhance the alignment effect of the features, in this embodiment, a deformable convolution unit is also introduced in the DV-CFM module. By adaptively learning the offset, the spatial position of the feature is adjusted, so as to more precisely achieve feature alignment. The mathematical expression of deformable convolution is as follows:

[0167]

[0168] Among them, represents the spatial position after the alignment of the lateral view feature, x represents the spatial position of the feature, R is the convolution kernel sampling area, ω(p) is the convolution weight, and Δp is the learnable offset.

[0169] By introducing the deformable convolution unit, the DV-CFM module can more flexibly adjust the spatial position of the feature and further improve the alignment accuracy.

[0170] Substituting the mathematical expression of the affine transformation into the deformable convolution, the mathematical expression for the DV-CFM module in this embodiment to achieve feature alignment is obtained:

[0171]

[0172] Among them, represents the aligned lateral view feature of the k-th layer.

[0173] The cross-view attention mechanism is the core component of the DV-CFM module and is used to establish the global dependency relationship between the dual-view features. For the frontal view feature of the k-th layer and the aligned lateral view feature Query (Query), key (Key), and value (Value) vectors are generated through linear projection. The query vector is generated from the frontal view feature, while the key vector and value vector are generated from the aligned lateral view feature, as shown in the following formula:

[0174]

[0175] Among them, is the query vector of the k-th layer feature, is the key vector of the k-th layer feature, is the value vector of the k-th layer feature; W Q 、WK , W V is a learnable parameter matrix.

[0176] Subsequently, the cross-view attention weight matrix is calculated by dot product and normalized using the Softmax function:

[0177]

[0178] where A k is the cross-view attention weight matrix of the k-th layer features, d k is the dimension of the key vector, is the transpose of.

[0179] Finally, the enhanced features are generated by weighted summation:

[0180]

[0181] where is the enhanced k-th layer frontal view feature.

[0182] Similarly, the lateral view features are updated through interaction with the frontal view to

[0183] The cross-view attention mechanism can not only capture the global dependencies between the dual-view features, but also adaptively enhance the expressive power of important features.

[0184] The multi-scale fusion module improves the model's perception ability of targets at different scales by capturing context information at different scales, enhancing the robustness of 3D reconstruction. The enhanced features are compressed to the original number of channels through channel concatenation and 1×1 convolution to retain the rich information of the dual-view features and reduce the computational complexity through dimensionality reduction, expressed by the following formula:

[0185]

[0186] The DV-CFM module injects the fused features into the corresponding levels of the 3D decoder through a U-Net style skip connection structure to retain multi-scale detail information. The skip connection can combine the low-level features of the 2D encoder with the high-level features of the 3D decoder, thus better recovering the detailed structure of the target during generation.

[0187] The 3D decoder consists of five levels of 3D transposed convolutional modules. Each level contains a 3×3×3 transposed convolution with a stride of 2, IN, and ReLU activation. The number of channels decreases gradually from 256 to 16. The 3D transposed convolution gradually restores the spatial dimension of the feature map through upsampling, and finally outputs class EOS three-dimensional data with 256×256×256 voxels. The output layer compresses the number of channels to 1 through a 1×1×1 convolution and uses the Tanh activation function to normalize the CT values to [-1,1] to match the numerical range of real CT scan data.

[0188] The 2D discriminator and the 3D discriminator adopt a multi-scale convolutional architecture. The multi-scale design enables the discriminator to capture both local texture and global anatomical consistency simultaneously, thereby comprehensively evaluating the authenticity of the generated images. The 3D discriminator is designed for three-dimensional CT scan data and can evaluate the similarity between class EOS three-dimensional data and CT standardized data. Its structure includes three levels of 3D convolutional layers. The first level uses a 4×4×4 convolutional kernel with a stride of 2, and the number of output channels is 32, followed by an IN layer, a LeakyReLU layer, and a Dropout layer. The number of channels in the subsequent levels increases to 64 and 128 in turn, while maintaining the same convolutional kernel size and stride. The last layer outputs a 32×32×32×128 probability map through a fully connected layer, and each element corresponds to the authenticity score of the input 256×256×256 voxel block. The structure of the 2D discriminator is similar to that of the 3D discriminator, but it uses 2D convolution to process the bi-plane class EOS data, with a convolutional kernel of 4×4, and the fully connected layer outputs a 32×32×128 probability map.

[0189] The generators and discriminators of the X2CT-CycleGAN network are both trained using the Adam optimizer, with initial learning rates of 2×10 -4 and 1×10 -4 , respectively. The momentum parameters β1 is set to 0.5, β2 is set to 0.999, the training period is 200 epochs, the learning rate linearly decays to zero after the 100th epoch, and the batch size is set to 4. Limited by the GPU memory and the computational complexity of 3D convolution, mixed-precision training is adopted during the training process to accelerate the calculation, and gradient clipping is used to prevent gradient explosion.

[0190] The training of the X2CT-CycleGAN network simultaneously uses dual-plane EOS-like input data to generate EOS-like 3D data and 3D CT label data to generate dual-plane EOS-like data. The training process is as follows: The 2Dto3D generator generates EOS-like 3D data based on the dual-plane EOS-like input data, and then sends it to the 3Dto2D generator to be restored to dual-plane EOS-like data. The 2D discriminator judges the similarity between the restored dual-plane EOS-like data and the dual-plane EOS-like input data; at the same time, the 3Dto2D generator generates dual-plane EOS-like data based on the 3D CT label data, and then sends it to the 2Dto3D generator to be restored to 3D CT data. The 3D discriminator judges the similarity between the restored 3D CT data and the 3D CT label data; then, the parameters of the X2CT-CycleGAN network are adjusted according to the two similarities to complete one training.

[0191] In this embodiment, the trained X2CT-CycleGAN network is only used to generate EOS-like 3D data based on dual-plane EOS-like data during use.

[0192] The loss function of the X2CT-CycleGAN network is:

[0193] L2 = L GAN (G A2B , D3, X, Y) + L GAN (G B2A , D2, Y, X) + L cyc (G A2B , G B2A )

[0194] + L identity (G A2B , G B2A , X, Y) + L D (D3, X, Y, G) + L D (D2, X, Y, G)

[0195]

[0196] Among them, L2 is the loss function of the initial 3D reconstruction network, G A2B represents the 2Dto3D generator, D3 represents the 3D discriminator, G A2B represents the 3Dto2D generator, D2 represents the 2D discriminator; X represents the dual-plane EOS-like input data set, Y represents the 3D CT label data set;

[0197] L GAN (G A2B , D3, X, Y) is the generative adversarial loss of the 3D reconstruction branch, x is the dual-plane EOS-like input data, y is the 3D CT label data, represents the expected value of the dual-plane type EOS input data under its probability distribution, represents the expected value of the 3D CT label data under its probability distribution; D3(y) represents the probability of classifying the 3D CT label data as a real image, G A2B (x) represents the EOS 3D data generated based on the dual-plane type EOS input data, D3(G A2B (x)) represents the probability of classifying the EOS 3D data generated based on the dual-plane type EOS input data as a real image;

[0198] L GAN (G B2A , D2, Y, X) represents the generative adversarial loss of the 2D reconstruction branch, D2(x) represents the probability of classifying the dual-plane type EOS input data as a real image, G B2A (y) represents the generation of dual-plane type EOS data based on the 3D CT label data, D2(G B2A (y)) represents the probability of classifying the dual-plane type EOS data generated based on the 3D CT label data as a real image;

[0199] L cyc (G A2B , G B2A ) is the cycle consistency loss, G B2A (G A2B (x)) represents the restoration of the EOS 3D data generated based on the dual-plane type EOS input data to the dual-plane type EOS data, G A2B (G B2A (y)) represents the restoration of the dual-plane type EOS data generated based on the 3D CT label data to the 3D CT data;

[0200] L identity (G A2B , G B2A , X, Y) is the identity loss, G A2B (y) represents the generation of EOS 3D data based on the 3D CT label data, G B2A (x) is the generation of dual-plane type EOS data based on the dual-plane type EOS input data;

[0201] L D (D3, X, Y, G) represents the discrimination loss of the 3D discriminator; D2(G B2A (G A2B (x))) represents the probability of classifying the restored dual-plane type EOS data of the EOS 3D data generated based on the dual-plane type EOS input data as a real image;

[0202] L D(D2,X,Y,G) represents the discriminative loss of the 2D discriminator, D3(G A2B (G B2A (y))) represents the probability of classifying the 3D CT data restored from the biplane-class EOS data generated according to the 3D CT label data as a real image.

[0203] The generative adversarial loss is used to measure the gap between the images generated by the generator and the real images. It drives the generator to generate as realistic images as possible, making it difficult for the discriminator to distinguish between the generated images and the real images. The generative adversarial loss is achieved by calculating the probability that the generated image is classified as a real image and the probability that the real image is classified as a real image. Among them, the generative adversarial loss of the 3D reconstruction branch is used to measure the authenticity of the output of the 2Dto3D generator from the biplane-class EOS input data to the class EOS 3D data, and the generative adversarial loss of the 2D reconstruction branch is used to measure the authenticity of the output of the 3Dto2D generator from the 3D CT label data to the biplane-class EOS data. The generative adversarial losses of the 3D reconstruction branch and the 2D reconstruction branch work together in the two-way training to ensure that both the generated biplane-class EOS data and the class EOS 3D data have a high degree of authenticity.

[0204] The cycle consistency loss is used to ensure the consistency during the conversion process, so that the image generated by the generator can be restored to the original image after passing through the inverse generator, and no important information will be lost during the conversion process. The cycle consistency loss is achieved by calculating the gap between the original image and the inverse generated image. The cycle consistency loss includes two parts. One part is used to measure the reconstruction error of generating the class EOS 3D data from the biplane-class EOS input data and then inversely restoring it to the biplane-class EOS data, and the other part is used to measure the reconstruction error of going from the 3D CT label data to the biplane-class EOS data and then restoring it to the CT normalized data from the biplane-class EOS data. The cycle consistency loss works together in the two-way training to ensure that the generator can maintain the consistency of the image content during the two-way conversion.

[0205] The identity loss forces the generator not to make unnecessary modifications when converting the input image to the target domain to maintain the relevance between the generated image and the input image, and is measured by the gap between the input image and the generated image. The identity loss is used to ensure that the 3D CT label data and the biplane-class EOS data can still maintain their original features after being processed by the generator.

[0206] The discriminator loss is used to measure the discriminative ability of the discriminator for real images and generated images, which is achieved by calculating the probability that real images are discriminated as real images and the probability that restored images are discriminated as real images. The discriminator loss prompts the generator to improve the output quality while ensuring that the discriminator can effectively distinguish real data from restored data. The discriminative losses of the 3D discriminator and the 2D discriminator are used to measure the discriminative ability of the 3D discriminator for dual-plane EOS input data and restored dual-plane EOS data, and the discriminative ability of the 3D discriminator for three-dimensional CT label data and restored three-dimensional CT data, respectively.

[0207] Step 6: Obtain the CT scan data to be processed, and preprocess it to obtain the standardized CT data to be processed; then, based on the projection model constructed in Step 2, project the standardized CT data to be processed to obtain the dual-plane DRR data to be processed.

[0208] Step 7: Input the dual-plane DRR data to be processed into the style transfer network to generate the dual-plane EOS data to be processed, and then input the dual-plane EOS data to be processed into the three-dimensional reconstruction network to generate the EOS-like three-dimensional data.

[0209] The present invention accurately restores the imaging characteristics of EOS equipment through physical modeling, and the generated dual-plane images have clinical diagnostic value; the innovative dual-view fusion mechanism breaks through the anatomical continuity bottleneck of traditional three-dimensional reconstruction and realizes fully automatic and high-precision three-dimensional data generation. On the one hand, the present invention proposes a physical and data dual-driven CT-EOS cross-modal mapping method. By constructing a physical projection model based on the geometric parameter system of the EOS system, the geometric projection process of EOS system imaging is accurately restored; a style transfer network with dual-channel anatomical constraints is designed, and the perspective complementary characteristics of the anteroposterior and lateral projections are used to realize the conversion of CT scan data into dual-plane EOS data. The present invention ensures the geometric accuracy of the projection through physical modeling and adapts the imaging style with the help of deep learning, effectively solving the problems of parameter generalization and feature mismatch in traditional methods.

[0210] On the other hand, the present invention proposes an X2CT-CycleGAN network for two-dimensional to three-dimensional conversion. Through the dual-view context fusion module, this network uses the channel attention mechanism to screen key anatomical features, combines spatial attention to establish cross-plane anatomical correspondence, and realizes the end-to-end reconstruction of dual-plane EOS data into EOS-like three-dimensional data. Through the multi-scale feature compensation and spatial continuity constraint mechanism, the technical problems in three-dimensional information recovery and anatomical structure integrity preservation in traditional methods are solved.

[0211] The method for generating EOS-like three-dimensional data from CT scan data according to the present invention can also form a computer program product, which includes a computer program. When the program is executed by a processor, the steps of the method for generating EOS-like three-dimensional data from CT scan data are implemented. Additionally, the present invention can also be applied to a terminal device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the method for generating EOS-like three-dimensional data from CT scan data according to the present invention are implemented. The terminal device here can be computing devices such as a computer, a notebook, a palm computer, and various cloud servers. The processor can be a general-purpose processor, a digital signal processor, an application-specific integrated circuit, or other programmable logic devices, etc.

Claims

1. A method for generating EOS-like three-dimensional data from CT scan data, characterized in that, It includes the following steps: Step 1: Obtain a CT scan data set from a publicly available medical image database and preprocess the CT scan data set to obtain a CT standardized data set; Step 2: According to the key projection parameters of the EOS system, construct a projection model based on parameter constraints, and use the projection model to project the CT standardized data in the CT standardized data set to generate biplane DRR data respectively, so as to obtain a biplane DRR data set; the key projection parameters of the EOS system are obtained through an EOS system parameter experimental calibration method based on geometric constraints; Step 3: Use the EOS system to obtain a biplane EOS image set, and preprocess the biplane EOS image set and the biplane DRR data set to obtain a biplane EOS standardized data set and a biplane DRR standardized data set respectively; Step 4: Build an initial style transfer network, use the biplane DRR standardized data set as the input and the biplane EOS standardized data set as the label to train the initial style transfer network, and obtain a biplane EOS-like data set and a style transfer network; Step 5: Build an initial 3D reconstruction network, unify the sizes of the biplane EOS-like data set and the CT standardized data set, and perform data augmentation to obtain a biplane EOS-like input data set and a 3D CT label data set; then use the biplane EOS-like input data set as the input and the 3D CT label data set as the label to train the initial 3D reconstruction network to obtain a 3D reconstruction network; Step 6: Obtain the CT scan data to be processed, and preprocess it to obtain the CT standardized data to be processed; then project the CT standardized data to be processed based on the projection model constructed in Step 2 to obtain the biplane DRR data to be processed; Step 7: Input the biplane DRR data to be processed into the style transfer network to generate the biplane EOS-like data to be processed, and then input the biplane EOS-like data to be processed into the 3D reconstruction network to generate EOS-like 3D data.

2. A method for generating EOS-like three-dimensional data from CT scan data according to claim 1, characterized in that: In Step 4, the initial style transfer network is a DC-STN network. The DC-STN network adopts a symmetric generative adversarial framework, including a two-branch generator and a discriminator; The two-branch generator includes two generator branches respectively used to independently process the anterior-posterior projection data and the lateral projection data. Both generator branches adopt an encoder-transformer-decoder architecture; A shared feature extraction module is jointly set at the back end of the encoders of the two generator branches. The output end of the shared feature extraction module is respectively connected to the input ends of the transformers of the two generator branches, and is used to fuse the anterior-posterior projection features and the lateral projection features of the biplane DRR data through a cross-view attention mechanism; The discriminator adopts a multi-scale PatchGAN structure.

3. A method for generating EOS-like three-dimensional data from CT scan data according to claim 2, characterized in that: In Step 5, the initial 3D reconstruction network is an X2CT-CycleGAN network. The X2CT-CycleGAN network includes a 3D reconstruction branch and a 2D reconstruction branch; The 3D reconstruction branch includes a 2Dto3D generator and a 3D discriminator. The 2Dto3D generator is used to generate 3D data similar to EOS, including two 2D encoders for respectively extracting the frontal view features and the lateral view features of the bi-plane EOS input data, a DV-CFM module for aligning the frontal view features and the lateral view features, and a 3D decoder for fusing the frontal view features and the lateral view features. The input ends of the two 2D encoders are respectively used to receive the frontal view and the lateral view of the bi-plane input EOS data. The output ends are respectively connected to the input end of the DV-CFM module. The output end of the DV-CFM module is connected to the input end of the 3D decoder. The output end of the 3D decoder is connected to the 3D discriminator. The 2D reconstruction branch includes a 3Dto2D generator and a 2D discriminator. The 3Dto2D generator is used to generate bi-plane EOS data, including a 3D encoder and a 2D decoder connected in sequence. The input end of the 3D encoder is used to receive the 3D CT label data. The output end of the 2D decoder is connected to the input end of the 2D discriminator.

4. A method for generating EOS-like three-dimensional data from CT scan data according to claim 3, characterized in that: The DV-CFM module includes a feature alignment module, a cross-view attention module, and a multi-scale feature fusion module. The feature alignment module includes a lateral encoding unit and an affine transformation unit. The cross-view attention module includes a deformable convolution unit, a frontal encoding unit, a first mapping unit, a second mapping unit, and an attention weight calculation unit. The input ends of the frontal encoding unit and the lateral encoding unit are respectively used to connect to the output ends of the two 2D encoders. The output end of the frontal encoding unit is respectively connected to the input end of the second mapping unit and the input end of the 3D decoder. The output end of the lateral encoding unit is connected to the input end of the affine transformation unit. The output end of the affine transformation unit is connected to the input end of the deformable convolution unit. The output end of the deformable convolution unit is connected to the input end of the first mapping unit. The input end of the attention weight calculation unit is respectively connected to the output ends of the first mapping unit and the second mapping unit. The output end is connected to the input end of the multi-scale feature fusion module. The output end of the multi-scale feature fusion module is connected to the input end of the 3D decoder. The feature alignment module is used to achieve the spatial alignment of the lateral view features and the frontal view features. The cross-view attention module is used to establish the global dependency relationship between the frontal view features and the lateral view features.

5. A method for generating EOS-like three-dimensional data from CT scan data according to claim 4, characterized in that: In step 4, the loss function of the initial style transfer network is: L1 = L DA + L BCC + L CMFC + L MSD + L CAR L DA = E A～p(A) [(D B (G(A)) - 1) 2 + E B～p(B) [(D A (F(B)) - 1) 2 ​ L BCC = λ cyc (E A～p(A) [||F(G(A)) - A||1] + E B～p(B) [||G(F(B)) - B||1]) where L1 is the loss function of the initial style transfer network. L DA is the dual-channel adversarial loss, A is the frontal view, and G(A) represents generating the lateral view according to the frontal view; B is the lateral view, and F(B) represents generating the frontal view according to the lateral view; D B (G(A)) and D A (F(B)) respectively represent the discriminative losses corresponding to G(A) and F(B); E A～p(A) represents the expected value of the frontal view under its probability distribution, and E B～p(B) represents the expected value of the lateral view under its probability distribution; L BCC is the bidirectional cycle consistency loss, λ cyc is the cycle consistency weight coefficient, F(G(A)) represents restoring the lateral view generated from the frontal view to the frontal view, G(F(B)) represents restoring the frontal view generated from the lateral view to the lateral view, and ‖‖1 represents taking the L1 norm; L CMFC is the cross-modal feature consistency loss, λ feat is the feature alignment strength, E A,B denotes taking the expectation of the joint distribution of the frontal view and the side view; denotes the shared feature map of the l-th layer of the encoder, C l is the number of channels of the shared feature map of the l-th layer, H l is the number of dimensions of the height of the shared feature map of the l-th layer, W l is the number of dimensions of the width of the shared feature map of the l-th layer, l is an integer, and 1 ≤ l ≤ L, where L is the total number of layers of the shared feature map; L MSD is the multi-scale discriminant loss, and are the s-th level discriminant losses corresponding to G(A) and F(B) respectively, where s is an integer and 1 ≤ s ≤ S, and S is the total number of levels of the discriminator; L CAR is the channel attention regularization loss, λ attn is the channel attention adjustment parameter, W c is the learnable attention weight for the c-th channel, f c is the feature of the c-th channel, c is an integer, and 1 ≤ c ≤ C, where C is the number of channels, softmax() represents the normalization function, and ReLU() represents the rectified linear unit function.

6. A method for generating EOS-like three-dimensional data from CT scan data according to claim 5, characterized in that: In step 5, the loss function of the initial 3D reconstruction network is: L2 = L GAN (G A2B , D3, X, Y) + L GAN (G B2A , D2, Y, X) + L cyc (G A2B , G B2A ) + L identity (G A2B , G B2A , X, Y) + L D (D3, X, Y, G) + L D (D2, X, Y, G) Among them, L2 is the loss function of the initial three-dimensional reconstruction network, G A2B represents the 2Dto3D generator, D3 represents the 3D discriminator, X represents the dual-plane class EOS input data set, and Y represents the three-dimensional CT label data set; G A2B represents the 3Dto2D generator, and D2 represents the 2D discriminator; L GAN (G A2B , D3, X, Y) is the generative adversarial loss of the 3D reconstruction branch, x is the biplane-like EOS input data, y is the 3D CT label data, represents the expected value of the biplane-like EOS input data under its probability distribution, represents the expected value of the 3D CT label data under its probability distribution; D3(y) represents the probability of classifying the 3D CT label data as a real image, G A2B (x) represents the EOS-like 3D data generated based on the biplane-like EOS input data, D3(G A2B (x)) represents the probability of classifying the EOS-like 3D data generated based on the biplane-like EOS input data as a real image; L GAN (G B2A , D2, Y, X) represents the generative adversarial loss of the 2D reconstruction branch. D2(x) represents the probability that the dual-plane class EOS input data is discriminated as a real image, and G B2A (y) represents generating dual-plane class EOS data according to the three-dimensional CT label data. D2(G B2A (y)) represents the probability that the dual-plane class EOS data generated according to the three-dimensional CT label data is discriminated as a real image; L cyc (G A2B ,G B2A ) is the cyclic consistency loss, G B2A (G A2B (x)) means restoring the class EOS three-dimensional data generated from the dual-plane class EOS input data to the dual-plane class EOS data, G A2B (G B2A (y)) means restoring the dual-plane class EOS data generated from the three-dimensional CT label data to the three-dimensional CT data; L identity (G A2B ,G B2A ,X,Y) is the identity loss, and G A2B (y) represents generating EOS-like three-dimensional data based on three-dimensional CT label data, and G B2A (x) is generating biplane EOS-like data based on biplane EOS-like input data; L D (D3, X, Y, G) represents the discrimination loss of the 3D discriminator; D2(G B2A (G A2B (x))) represents the probability of discriminating the bi-plane class EOS data, which restores the class EOS 3D data generated from the bi-plane class EOS input data, as a real image; L D (D2, X, Y, G) represents the discriminative loss of the 2D discriminator, D3(G A2B (G B2A (y))) represents the probability of classifying the 3D CT data restored from the biplane-class EOS data generated based on the 3D CT label data as a real image.

7. A method for generating EOS-like three-dimensional data from CT scan data according to any one of claims 2-6, characterized in that In step 4, the specific method for training the initial style transfer network is: using the Adam optimizer, alternately optimizing the parameters of the dual-branch generator and the discriminator through the gradient descent method. Among them, the specific method of alternately optimizing the parameters of the dual-branch generator and the discriminator is: First, fix the parameters of the dual-branch generator, and adjust the parameters of the discriminator according to the difference between the bi-plane EOS data generated by the dual-branch generator and the corresponding EOS bi-plane standardized data. Second, fix the parameters of the discriminator, and adjust the parameters of the dual-branch generator according to the similarity between the dual-plane class EOS data generated by the dual-branch generator and the corresponding EOS dual-plane normalized data.

8. A method for generating EOS-like three-dimensional data from CT scan data according to claim 7, characterized in that Specifically, Step 2 is as follows: Step 2.1: Obtain the key projection parameters of the EOS system by means of an experimental calibration method for the EOS system parameters based on geometric constraints; the key projection parameters of the EOS system include the distance from the projection source to the detector of the EOS system, the projection position of the projection source on the detector of the EOS system, and the distance from the projection source to the rotation center of the EOS system. Step 2.2: Iteratively optimize the key projection parameters of the EOS system through a dynamic coordinate transformation algorithm, so that the optimized key projection parameters of the EOS system satisfy the physical alignment constraint of the projection ray bundle and the detector of the EOS system as shown in the following formula: ||R·v - v′||2 < 0.1mm where R is a rotation matrix representing the rotation transformation of the detector coordinate system of the EOS system or the direction of the projection ray bundle; v is the direction vector of the projection ray bundle, v′ is the target direction vector received by the detector of the EOS system, and ‖‖2 represents taking the L2 norm. Step 2.3: Construct a projection model based on parameter constraints according to the optimized key projection parameters of the EOS system; the projection model includes an anteroposterior projection coordinate system and a lateral projection coordinate system. Step 2.4: In the anteroposterior projection coordinate system of the projection model, adopt the multi-angle ray integration method, make the X-ray source emit discrete ray bundles at multiple preset angles respectively, calculate the attenuation amount of each voxel in the CT normalized data along the path of the ray bundle passing through the CT normalized data, and obtain the multi-angle anteroposterior projection data through the following formula respectively: Wherein, I is the frontal projection data, and μ i represents the linear attenuation coefficient of the i-th voxel in the CT standardized data, which is converted from the HU value through the proportionality coefficient k = 0.85, and l i is the path length of the ray beam passing through the i-th voxel, i is an integer, and 0 ≤ i ≤ N, where N is the number of voxel layers of the CT standardized data; Step 2.5: Establish a two-parameter correction matrix, and correct the image distortion of the multi-angle anteroposterior projection data through the following formula respectively to obtain the multi-angle anteroposterior projection corrected data: Among them, (x ′ , y ′ ) are the coordinates of each pixel point in the orthographic projection correction data, (x, y) are the coordinates of each pixel point in the orthographic projection data, θ is the rotation offset angle of the EOS system detector plane, Δd x is the position deviation compensation amount of the x-axis, and Δd y is the position deviation compensation amount of the y-axis; Step 2.6: Adopt a weighted backprojection algorithm, and map the multi-angle anteroposterior projection corrected data to the detector of the EOS system through the following formula to obtain the anteroposterior DRR data: Among them, P DRR is the anterior DRR data, ω i is the angular weight factor of the i-th voxel in the CT standardized data, P j is the integration result of the anterior projection correction data at the j-th angle, and m is the number of preset angles; Step 2.7: According to the method of Step 2.4 - Step 2.6, in the lateral projection coordinate system of the projection model, obtain the lateral DRR data, so as to obtain the dual-plane DRR data. Step 2.8: Repeat Step 2.4 - Step 2.7 until all CT normalized data in the CT normalized dataset are traversed to obtain the dual-plane DRR dataset.

9. A method for generating EOS-like three-dimensional data from CT scan data according to claim 8, characterized in that: In Step 1, the specific method for preprocessing the CT scan dataset is as follows: Convert the pixel values of the CT scan data in the CT scan dataset into HU values respectively, and then perform dynamic optimization adjustment of the window width and window level on the HU values to obtain the CT scan HU dataset; then establish a two-level regional annotation system including an overall anatomical region mask and a bone region mask, and use the two-level regional annotation system to generate the overall anatomical region mask and the bone region mask by the label image method and label them in the CT scan HU dataset. In Step 3, the specific method for preprocessing the dual-plane EOS image set and the dual-plane DRR dataset is as follows: The biplane EOS images in the biplane EOS image dataset and the biplane DRR data in the biplane DRR dataset are respectively converted into 512×512 pixel resolution images through the bicubic interpolation algorithm, then a random geometric enhancement strategy is adopted, and then the pixel values are linearly mapped to the interval [0,1] through max-min normalization; In step 5, the data enhancement includes random rotation of ±5°, translation of ±10%, and gray jitter of ±15%.

10. A computer program product, comprising a computer program, characterized in that: When the computer program is executed by a processor, it implements the steps of a method for generating EOS-like three-dimensional data from CT scan data according to any one of claims 1 to 9.

11. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, it implements the steps of a method for generating EOS-like three-dimensional data from CT scan data according to any one of claims 1 to 9.

Images