Cone beam X-ray perspective deformation correction system and correction method thereof

Abstract

The invention provides an X-ray perspective deformation correction system, which is used for correcting deformation of a detected object caused by X-cone beam ray perspective projection. The X-ray perspective deformation correction system comprises a projection imaging system, a color superposition module and a learning module. The projection imaging system is a conventional CBCT imaging system. The projection imaging system carries out perspective projection on the detected object from two complementary visual angles so as to obtain a first view and a second view respectively. The first view and the second view are processed by the color superposing module to form a superposing color image. And the superposed color image is input into the learning module and is processed by the learning module to obtain a corrected image.

Description

Technical Field

[0001] The present invention relates to a perspective distortion correction system and a correction method thereof, and in particular to a cone-beam X-ray perspective distortion correction system and a correction method thereof. Background Art

[0002] X-ray imaging and flat-panel detectors are widely used in disease diagnosis, interventional treatment, and tumor radiotherapy. In cone-beam X-ray imaging, due to the divergence of X-rays, imaging structures at different depths have different magnifications on the X-ray detector. Therefore, the obtained image suffers from geometric distortion, which is called perspective distortion. In many practical applications, perspective distortion makes it difficult to perform direct and accurate geometric evaluation of the structure of interest (SOI for short), such as anatomical landmark detection, perspective image stitching, fiducial marker registration, and dual-modality image fusion. Therefore, in many X-ray applications, orthographic projection is more popular than perspective projection.

[0003] In conventional cone-beam X-ray imaging systems, such as C-arm cone-beam computed tomography (CBCT) systems, perspective distortion is a problem in applications such as chest X-ray imaging and cephalometric analysis. To reduce perspective distortion, special imaging devices have been designed, such as chest X-ray machines and cephalometric machines with long source-to-detector distances. In addition to these special imaging devices, another way to deal with cone-beam distortion is to first reconstruct a three-dimensional image and then digitally reconstruct the radiographs (DRRs) through a parallel beam projection. For example, in dental CBCT, DRRs using orthographic projections are used as synthetic lateral views, providing higher cephalometric analysis accuracy than perspective projections. However, obtaining such three-dimensional CBCT images requires obtaining hundreds of projections for three-dimensional reconstruction, which will cause additional X-ray doses to the patient. Another potential application is dual-modality imaging of magnetic resonance (MRI) and X-rays. MRI can easily obtain two-dimensional parallel beam projections because it samples in the frequency domain, while X-rays are cone beam projections, and the two modality images are not easy to align. Generating DRRs by acquiring three-dimensional MRI images is very time-consuming, taking about 30 minutes per scan. More importantly, pre-acquired three-dimensional MRI images cannot provide accurate information about daily changes in organs. Therefore, the rapid conversion of cone-beam X-ray images into parallel-beam images can be quickly registered with two-dimensional parallel-beam MRI images for real-time observation of different structures such as bones and soft tissues, which has important clinical value. Summary of the invention

[0004] An object of the present invention is to provide a cone-beam X-ray perspective deformation correction system and a correction method thereof, so as to provide a fast and accurate detection method which does not bring additional X-ray dose to the patient.

[0005] Another object of the present invention is to provide a cone-beam X-ray perspective deformation correction system and correction method thereof, wherein a two-dimensional perspective (cone-beam) projection is directly converted into a two-dimensional parallel projection.

[0006] Another object of the present invention is to provide a cone-beam X-ray perspective distortion correction system and correction method thereof to solve the difficulties caused by perspective distortion in direct and accurate geometric evaluation of anatomical structures in cone-beam X-ray transmission imaging.

[0007] Another object of the present invention is to provide a cone-beam X-ray perspective deformation correction system and a correction method thereof, wherein the perspective deformation correction problem is solved in a framework using two complementary views (with a projection angle difference of 180°), wherein the complementary view setting provides an intuitive and practical method for identifying perspective deformation structures by evaluating the deviation between the two views.

[0008] Another object of the present invention is to provide a cone-beam X-ray perspective deformation correction system and correction method thereof, wherein within the calibration accuracy range, the training model has a certain tolerance for geometric errors such as the distance from the ray source to the isocenter, the rotation angle, the detector principal point displacement and respiratory movement, so that a model trained from a CBCT imaging system is applicable to other systems of the same type.

[0009] Another object of the present invention is to provide a cone beam X-ray fluoroscopic deformation correction system and correction method thereof, whose effectiveness on real human CBCT projection data and its robustness in the presence of metal implants and surgical screws indicate prospects for practical application.

[0010] Another object of the present invention is to provide a cone-beam X-ray perspective deformation correction system and correction method thereof, wherein a simple complementary view setting is proposed to provide boundary information, thereby reducing the uncertainty and receptive field size required for learning perspective deformation in X-ray transmission imaging.

[0011] Another object of the present invention is to provide a cone-beam X-ray perspective deformation correction system and correction method thereof, wherein structures suffering from perspective deformation are identified by evaluating the deviation between two complementary views, which can be easily observed in the RGB color images of the complementary views.

[0012] According to one aspect of the present invention, the present invention further provides an X-ray perspective deformation correction system to correct the deformation of an object to be inspected caused by X-ray cone beam perspective projection, comprising:

[0013] a projection imaging system, wherein the projection imaging system is a conventional CBCT imaging system, wherein the projection imaging system performs perspective projection on the object to be detected from two complementary viewing angles to obtain a first view and a second view respectively;

[0014] a color overlay module, wherein the first view and the second view are processed by the color overlay module to form an overlay color image; and

[0015] A learning module, wherein the superimposed color image is input into the learning module and processed by the learning module to obtain a corrected image.

[0016] According to one embodiment of the present invention, the projection imaging system includes a cone-beam X-ray source, a detector and a driving module, wherein the driving module drives the cone-beam X-ray source to rotate around an isocenter so that the cone-beam X-ray source performs X-ray perspective projection on the object to be detected from two complementary viewing angles, wherein the plane where the detector is located passes through the isocenter.

[0017] According to one embodiment of the present invention, the projection imaging system includes two cone-beam X-ray sources and a detector, wherein the two cone-beam X-ray sources are symmetrical at complementary angles with an isocenter so that the cone-beam X-ray sources perform X-ray perspective projection on the object to be detected from two complementary viewing angles, respectively, and the plane where the detector is located passes through the isocenter.

[0018] According to an embodiment of the present invention, the color overlay module includes a red channel, a blue channel and a green channel, wherein the red channel and the blue channel use the first view, and the green channel uses the second view.

[0019] According to one embodiment of the present invention, the learning module includes a coordinate conversion module and a neural network, wherein the superimposed color image is converted from rectangular coordinates to polar coordinates by the coordinate conversion module to obtain a polar coordinate conversion superimposed color image, wherein the polar coordinate conversion superimposed color image is input into the neural network and is processed by the neural network to output an output image, wherein the output image is inversely transformed from polar coordinates to rectangular coordinates to obtain the corrected image.

[0020] According to one embodiment of the present invention, the neural network is selected from a CNN neural network and a Transformer neural network.

[0021] According to one embodiment of the present invention, the neural network is selected from Pix2pixGAN and TransU-Net.

[0022] According to one embodiment of the present invention, the projection imaging system includes a cone-beam X-ray source, a detector and a driving module, wherein the driving module drives the cone-beam X-ray source to rotate around an isocenter so that the cone-beam X-ray source performs X-ray perspective projection on the object to be detected from two complementary viewing angles, wherein the plane where the detector is located passes through the isocenter.

[0023] According to one embodiment of the present invention, the projection imaging system includes two cone-beam X-ray sources and a detector, wherein the two cone-beam X-ray sources are symmetrical at complementary angles with an isocenter so that the cone-beam X-ray sources perform X-ray perspective projection on the object to be detected from two complementary viewing angles, respectively, and the plane where the detector is located passes through the isocenter.

[0024] According to an embodiment of the present invention, the color overlay module includes a red channel, a blue channel and a green channel, wherein the red channel and the blue channel use the first view, and the green channel uses the second view.

[0025] According to one embodiment of the present invention, the complementary viewing angle ranges from 175° to 185°.

[0026] According to another aspect of the present invention, the present invention further provides an X-ray perspective distortion correction method, comprising the following steps:

[0027] A) performing cone-beam X-ray perspective projection on the same object to be detected from two complementary angles to form two complementary views, wherein the two complementary views include a first view and a second view;

[0028] B) resetting the two complementary images on a detector to correct the sizes of the two complementary images;

[0029] C) converting the two complementary views into color images and superimposing them to generate a superimposed color image;

[0030] D) performing polar coordinate transformation on the superimposed color image by polar coordinate transformation to obtain a polar coordinate transformed superimposed color image;

[0031] E) the polar coordinate transformed superimposed color image is input into a neural network as an input image;

[0032] F) the neural network processes the polar coordinate transformation overlay color image and outputs an output image; and

[0033] G) Inversely transforming the output image in polar coordinates to obtain a rectified image.

[0034] According to one embodiment of the present invention, step A comprises the following steps:

[0035] A1) a cone-beam X-ray source performs cone-beam X-ray perspective projection on the object to be detected at a first position to obtain the first view;

[0036] A2) rotating the cone-beam X-ray source 180° around an isocenter to move the cone-beam X-ray source to a second position;

[0037] A3) The cone-beam X-ray source performs cone-beam X-ray perspective projection on the object to be detected at the second position to obtain the second view.

[0038] According to an embodiment of the present invention, in step A, the two cone-beam X-ray sources are symmetrical at complementary angles with an isocenter as a symmetry point.

[0039] According to an embodiment of the present invention, the plane where the detector is located passes through the isocenter.

[0040] According to one embodiment of the present invention, in step C, the first view and the second view are processed by a color overlay module, wherein the color overlay module includes a red channel, a blue channel and a green channel, wherein the red channel and the blue channel use the first view, and the green channel uses the second view.

[0041] According to one embodiment of the present invention, in step D, the overlaid color image is converted from rectangular coordinates to polar coordinates, thereby obtaining the polar coordinate converted overlaid color image.

[0042] According to one embodiment of the present invention, in step G, the output image is inversely transformed from polar coordinates to rectangular coordinates to obtain the rectified image.

[0043] According to one embodiment of the present invention, step D is performed by selecting the neural network from a CNN neural network and a Transformer neural network.

[0044] According to one embodiment of the present invention, the neural network is selected from Pix2pixGAN and TransU-Net. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a flow chart of a cone-beam X-ray perspective deformation correction method according to a first preferred embodiment of the present invention.

[0046] Figure 2 4 is a block diagram of a cone-beam X-ray fluoroscopic deformation correction system according to the first preferred embodiment of the present invention.

[0047] Figure 3A A CBCT system is illustrated wherein a detector is located at the isocenter of the CBCT system.

[0048] Figure 3B The perspective deformation of a single view under the above CBCT system is explained.

[0049] Figure 3C The perspective deformation of two orthogonal views under the above CBCT system is explained.

[0050] Figure 3D The perspective deformation of two complementary views under the above CBCT system is explained.

[0051] Figure 4A and Figure 4B The perspective deformation from rectangular coordinate system to polar coordinate system is explained, where the direction and length of the arrow reflect the direction and size of the perspective deformation at the corresponding position.

[0052] Figure 5A This is a 0° perspective projection image of a small spherical phantom.

[0053] Figure 5B This is the 90° perspective projection image of the small ball phantom.

[0054] Figure 5C It is the RGB superposition of the 0° and 90° perspective projection images of the small ball phantom.

[0055] Fig. 6A It is the perspective projection image of the small ball phantom at a viewing angle of 0°.

[0056] Figure 6B It is the view difference between the 0° viewing angle perspective projection image of the small ball phantom and a reference image.

[0057] Figure 6C is an image of the reference figure, illustrating a 0° orthogonal projection image.

[0058] Fig.6D It is the 90° perspective projection image of the small ball phantom.

[0059] Fig. 6E Explained Fig. 6A and Fig.6D The view is poor.

[0060] Fig. 6F The RGB overlay of the 0° and 90° viewing angle perspective projection images of the small ball phantom is illustrated.

[0061] Figure 6G This is the 5° perspective projection image of the small ball phantom.

[0062] Figure 6H Explained Fig. 6A and Figure 6G The view is poor.

[0063] Fig.6I The RGB overlay of the 0° and 5° viewing angle perspective projection images of the small ball phantom is illustrated.

[0064] Figure 6J It is a 180° perspective projection image of the small ball phantom.

[0065] Figure 6K Explained Fig. 6A and Figure 6J The view is poor.

[0066] Figure 6L The RGB overlay of the 0° and 180° viewing angle perspective projection images of the small ball phantom is illustrated.

[0067] Fig. 7A It is the orthographic projection view of the small ball phantom at a 0° viewing angle.

[0068] Figure 7B This is the cone-beam perspective projection view of the small spherical phantom at a 0° viewing angle.

[0069] Figure 7C yes Figure 7B and Fig. 7A The view is poor.

[0070] Fig.7D It is a 180° perspective projection image of the small ball phantom.

[0071] Fig. 7E is the difference between the 0° and 180° views of the small ball phantom.

[0072] Figure 7F It is the RGB superposition of the 0° and 180° perspective projection images of the small ball phantom.

[0073] Fig. 8A The point-to-point distance distribution when Dsi=600mm is illustrated.

[0074] Figure 8B The distribution of the ratio α in complementary view settings of Dsi=600 mm and Dsi=750 mm is illustrated.

[0075] Fig.9A It is a perspective projection image of a reference image in a rectangular coordinate system.

[0076] Fig. 9B It is the perspective projection image of the reference image in the polar coordinate system.

[0077] Fig. 9C It is the RGB superposition of the 0° and 180° perspective projection images of the reference image in the polar coordinate system.

[0078] Fig.10 and Fig.11 The quantitative evaluation of different methods on small ball phantom data is explained.

[0079] FIG. 12A to FIG. 12L We illustrate Pix2pixGAN predictions in different spaces and different views.

[0080] FIG. 13A to FIG. 13OThe application of the cone-beam X-ray perspective deformation correction method and the X-ray perspective deformation correction system according to the first preferred embodiment of the present invention in human head imaging correction is explained.

[0081] FIG. 14A to FIG. 14C We demonstrate the robustness of Pix2pixGAN to geometric inaccuracies when using 0° and 180° polarity inputs on a small spherical phantom.

[0082] FIG. 15A to FIG. 15C Three DRR perspective projection images used for training are shown.

[0083] Fig.16A It is a 0° perspective projection image of the first knee joint.

[0084] Fig. 16B It is a 0° viewing angle perspective projection image of a second knee joint, wherein the second knee joint is provided with two metal implants.

[0085] Fig. 16C It is a 0° viewing angle perspective projection image of a third knee joint, wherein the third knee joint is provided with a plurality of screws.

[0086] Fig.16D It is a 180° perspective projection image of the first knee joint.

[0087] Fig.16E It is a 180° perspective projection image of the second knee joint.

[0088] Fig.16F It is a 180° perspective projection image of the third knee joint.

[0089] Figure 16G It is the perspective projection difference image of the first knee joint at 0° viewing angle and 180° viewing angle.

[0090] Fig.16H It is the perspective projection difference image of the second knee joint at 0° viewing angle and 180° viewing angle.

[0091] Fig.16I It is the perspective projection difference image of the third knee joint at 0° viewing angle and 180° viewing angle.

[0092] Fig.16J It is a color image of the superimposed perspective projection of the first knee joint at 0° and 180° viewing angles.

[0093] Figure 16K This is a color image of the second knee joint with 0° and 180° perspective projections superimposed.

[0094] Figure 16L This is a color overlay of the perspective projections of the third knee joint at 0° and 180° viewing angles.

[0095] Figure 16M is the reference image of the first knee joint.

[0096] Fig.16N is the reference image of the second knee joint.

[0097] Fig.16O is the reference image of the third knee joint.

[0098] Figure 16P is the Pix2pixGAN corrected image of the first knee joint.

[0099] Figure 16Q is the Pix2pixGAN corrected image of the second knee joint.

[0100] Figure 16R This is the Pix2pixGAN corrected image of the third knee joint.

[0101] Fig.17A yes Fig.16A A partial enlarged view of .

[0102] Fig. 17B yes Figure 16M A partial enlarged view of .

[0103] Fig. 17C yes Figure 16P A partial enlarged view of . DETAILED DESCRIPTION

[0104] The following description is used to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments described below are only examples, and those skilled in the art can think of other obvious variations. The basic principles of the present invention defined in the following description can be applied to other embodiments, variations, improvements, equivalents, and other technical solutions that do not deviate from the spirit and scope of the present invention.

[0105] Those skilled in the art should understand that, in the disclosure of the present invention, the terms "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicating the orientation or position relationship are based on the orientation or position relationship shown in the drawings, which are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation. Therefore, the above terms should not be understood as limiting the present invention.

[0106] It is to be understood that the term "one" should be understood as "at least one" or "one or more", that is, in one embodiment, the number of an element may be one, while in another embodiment, the number of the element may be multiple, and the term "one" should not be understood as a limitation on the quantity.

[0107] The drawings in the specification Figure 1 A cone-beam X-ray perspective distortion correction method according to a first preferred embodiment of the present invention is explained. The X-ray perspective distortion correction method comprises the following steps:

[0108] A) performing cone-beam X-ray perspective projection on the same object to be inspected from two complementary angles to form two complementary views;

[0109] B) resetting the two complementary images on a detector to correct the sizes of the two complementary images;

[0110] C) converting the two complementary views into color images and superimposing them to generate a superimposed color image;

[0111] D) performing polar coordinate transformation on the superimposed color image by polar coordinate transformation to obtain a polar coordinate transformed superimposed color image;

[0112] E) the polar coordinate transformed superimposed color image is input into a neural network as an input image;

[0113] F) the neural network processes the polar coordinate transformation overlay color image and outputs an output image; and

[0114] G) Inversely transform the output image in polar coordinates to obtain a rectified image.

[0115] According to the first preferred embodiment of the present invention, step A performs perspective projection of the object to be detected by two X-cone beam rays of complementary angles to form the two complementary views, namely a first view and a second view. In order to more clearly express the relationship between the first view and the second view, the first view and the second view are respectively expressed as a 0° view and a 180° view, wherein step A can perform perspective projection of the object to be detected by rotating the same X-cone beam ray projection device 180° to obtain two complementary views, namely the 0° view and the 180° view. It is worth mentioning that the angles in the 0° view and the 180° view describe the relative angular position relationship of the X-cone beam ray sources of the two views, without the limitation of absolute angles.

[0116] According to an alternative implementation of the first preferred embodiment of the present invention, the two X-cone beam ray projection devices may also be arranged at 180°, so that the two X-cone beam ray projection devices emit X-cone beam rays with a projection angle of 180° between each other to form two complementary views respectively. In this alternative implementation, the two complementary views can be formed simultaneously (for example, dual ray sources and dual detectors), so that the imaging efficiency is higher.

[0117] It is worth mentioning that in the first preferred embodiment according to the present invention, the complementary angle is 180° and an error is allowed. The best correction effect can be achieved when the complementary angle is 180°, and a better correction effect can still be achieved when there is a 5° error. In other words, the two complementary angles in step A can be selected to be 175° to 185°. Optimally, the two complementary angles are 180°.

[0118] In step C, the views are superimposed by RGB to form a color image, which is more visually pleasing and can be recognized by the human eye and neural network, thereby facilitating further correction. The two complementary views of the colorization are marked with different colors in step C, so that they can be distinguished and recognized even after being superimposed.

[0119] Step D performs polar coordinate conversion on the superimposed color image obtained in step C. According to the first preferred embodiment of the present invention, step D performs polar coordinate conversion on the superimposed color image obtained in step C from a rectangular coordinate system to a polar coordinate system.

[0120] The drawings in the specification Figure 4A and Figure 4B The perspective deformation from rectangular coordinates to polar coordinates is illustrated, where the direction and length of the arrows reflect the direction and size of the perspective deformation at the corresponding position. The depth is fixed (m>1), and the red 3×3 grid represents a 3×3 convolution kernel. Figure 4A and Figure 4B It can be seen that the polar coordinate transformation can convert the radial perspective deformation into a translation deformation, which is easier for the convolutional neural network to learn.

[0121] The polar coordinate transformed superimposed color image obtained by the polar coordinate transformation in step D is input as an input image into a neural network in step E. As an example, according to the first preferred embodiment of the present invention, the neural network is selected from Pix2pixGAN and TransU-Net.

[0122] The polar coordinate transformed superimposed color image is input into and processed by the neural network to generate the output image.

[0123] Step G further performs polar coordinate inverse conversion on the output image, that is, converts the polar coordinates into rectangular coordinates to obtain the corrected image.

[0124] It is worth mentioning that the polar coordinate transformation in step D and the corresponding polar coordinate inverse transformation in step G are not necessary steps. It is worth mentioning that the addition of step D has the following advantages: first, the correction effect is better after adding the polar coordinate transformation step; on the other hand, after the polar coordinate transformation, the registration between the two-dimensional cone-beam X-ray image and the two-dimensional parallel-beam MRI image can be achieved through the conversion between perspective and orthographic projection, so that fast two-dimensional MRI / X-ray hybrid imaging can be achieved in one examination without repositioning the patient and can be applied in interventional surgery and radiotherapy.

[0125] This cone-beam X-ray fluoroscopic distortion correction method also has the potential to accurately measure cardiothoracic ratio and head imaging.

[0126] The drawings in the specification Figure 2 A cone beam X-ray perspective deformation correction system according to the above first preferred embodiment of the present invention is explained. The cone beam X-ray perspective deformation correction system includes a projection imaging system 10, a color overlay module 20 and a learning module 30, wherein the learning module 30 includes a coordinate conversion module 31 and a neural network 32. According to the first preferred embodiment of the present invention, the projection imaging system 10 includes a cone beam X-ray source 11, a detector 12 and a driving module 13. The driving module 13 is used to drive the cone beam X-ray source 11 to rotate around a preset axis, so that the cone beam X-ray source 11 can have a preset angle, for example, two different positions of an angle of 180° to irradiate the object to be detected with cone beam X-rays, and form two X-ray perspective views. The detector 12 corrects the dimensional deformation of the two X-ray perspective views. The two X-ray perspective views corrected by the detector 12 are colored and overlaid by the color overlay module 20, thereby generating the overlaid color map.

[0127] According to the first preferred embodiment of the present invention, the color overlay module 20 colors the X-ray perspective view through RGB three channels.

[0128] It is worth mentioning that according to other embodiments of the present invention, the X-ray perspective view may also be colored by two channels or four channels (eg, RGBA four channels).

[0129] The coordinate conversion module 31 converts the superimposed color image from rectangular coordinates to polar coordinates, thereby obtaining the polar coordinate conversion superimposed color image. The polar coordinate conversion superimposed color image is input into the neural network 32, and the output image is output after being corrected by the neural network 32. The output image is converted from polar coordinates to rectangular coordinates by the coordinate conversion module 31, thereby obtaining the corrected image.

[0130] Refer to the attached drawings of the instruction manual to Figure 3AAccording to the first preferred embodiment of the present invention, the projection imaging system 10 performs X-ray projection imaging of the object to be detected through a CBCT system, wherein the detector 12 is disposed at the isocenter of the CBCT system.

[0131] Reference to the attached drawings of the specification Figure 3A In the CBCT system, the distance D from the source S to the detector D is sd The distance D from the source S to the isocenter I si The magnification factor caused by the ratio of can be eliminated by virtually moving the detector to the isocenter I. FIG. 3A to FIG. 3D , the detector 12 is placed at the isocenter I of the projection imaging system 10. After re-entering the detector, the structures in the mid-sagittal plane (where the detector 12 is located) are not magnified. However, the structures in other sagittal planes are magnified or reduced according to their depth, that is, they are subjected to perspective deformation. For the point a=(x, y, z, 1) in the homogeneous coordinate system, the perspective projection and orthographic projection of its corresponding point in perspective are respectively expressed as and express,

[0132] in,

[0133] in, and They are the perspective projection matrix and the orthographic projection matrix under 0° view. The perspective deformation (PD) is calculated as:

[0134]

[0135] When the CBCT system has Figure 3A When the orientation and position shown,

[0136]

[0137] Where s is the pixel size of the virtual detector. Given the principal point O on the detector det =[P u ,P v ,1] T , it can be clearly seen that and O det collinear, which indicates that the perspective deformation occurs along the radial direction. The amount of perspective deformation is,

[0138]

[0139] in, is the amplification factor, ρ0 is To the main point O det This shows that, at a fixed depth z, the farther the point is from the principal ray, the greater the perspective deformation. This position dependence is expressed by Figure 4A This is reflected in the different lengths and directions of the arrows.

[0140] To more clearly describe the present invention and its advantages, the present invention is illustrated by a single view and three dual-view designs respectively, and the advantages of complementary views in learning perspective deformation correction are demonstrated through numerical uncertainty analysis.

[0141] Reference to the attached drawings of the specification Figure 3B , the perspective deformation of a single view under the above CBCT system is analyzed, where the red dot is the perspective projection of the black dot at a 0° viewing angle, and the blue dot is the orthographic projection of the black dot. The perspective projection matrix is ​​underdetermined. Therefore, the same perspective projection point Corresponding to non-unique points in the imaged object, these points are located on its back-projection line, that is, in is the projection matrix For example, in Figure 3B In the figure, if there is no depth information, the black point can be located anywhere along the red real ray. Therefore, all points on the red dashed line, including the blue point, are candidate points for the orthographic projection of the red point. The red dashed line here is the orthographic projection of l on the detector (abbreviated as OPBP), that is, Therefore, learning perspective deformation from a single view is very challenging.

[0142] Reference to the attached drawings of the specification Figure 3C , the double perspective projection images (double orthogonal views) separated by 90° under the above CBCT system are analyzed, where the red dots are the perspective projections of the black dots at a 0° viewing angle, the blue dots are the orthographic projections of the black dots, and the green dots are the perspective projections of the black dots at a 90° viewing angle. In practice, dual-plane X-ray systems are widely used for depth estimation in interventional surgery. In a dual-detector CBCT system, an additional view is used, such as Figure 3C As shown. It is known that the perspective projection of a is rotated 90° along the y axis.

[0143]

[0144] Therefore, at the same detector pixel coordinates, the point-to-point distance between two orthogonal views can be calculated as

[0145] Ideally, two views of the same point of interest can determine its location in three-dimensional space, that is, the intersection of the two corresponding rays. Figure 3C The black dot in the figure is the intersection of the red and green solid lines. This ideal situation is based on the assumption that the point-to-point correspondence between the two views is certain. However, for different points a, d in the imaged object 90°has a large range of variation because it depends not only on the depth z in the 0° view, but also on the depth x in the 90° view. and The correspondence between is difficult to determine. As an example, FIG. 5A to FIG. 5C Shown are two perspective projection images of a small ball phantom taken 90° apart.

[0146] To better compare the perspective projection images of the two orthographic views, such as Figure 5C As shown, an RGB image is formed, where the red and blue channels use the image at a 0° viewing angle, while the green channel uses the image at a 90° viewing angle. The three channels are filled so that pixels with similar intensities from both views appear gray, such as the background cylinder area. In the formed RGB color image, the magenta ball phantom image from the 0° view and the green ball phantom image from the 90° view are located at different locations. Figure 5C ,Due to the large number of small ball phantoms, the point-to-point correspondence between the two views is not direct.

[0147] Let's take 5° as an example to analyze the small angle dual view. As an alternative to dual orthogonal views, in addition to the 0° view, the second view can have a small angle interval (e.g. 5°). If the perspective projection of a when the detector is rotated by angle θ is defined as The distance between two perspective projection points of point a can be defined as When the angle interval is small, such as 5°, and The distances are close, so point-to-point correspondence is easier to determine than for dual orthogonal views. However, the difference between the two views is mainly in the rotational direction, not the radial direction.

[0148] Figure 3D The perspective deformation of two dual perspective projection images (complementary views) separated by 180° of the above CBCT system is illustrated, where the red dot is the perspective projection of the black dot at a viewing angle of 0°, the blue dot is the orthographic projection of the black dot, and the green dot is the perspective projection of the black dot at a viewing angle of 180°.

[0149] The present invention proposes a complementary view (180°) for learning perspective deformation. Compared with the orthographic projection DRRs of 3D CBCT reconstruction, the complementary view scan provides low-dose exposure to the patient, and since the hundreds of views required for 3D reconstruction are reduced to two views, frequent device rotation is no longer required. In addition, since the detector pixel size of the projection image is usually smaller than the voxel spacing of the 3D reconstruction volume, and the projection image is not affected by the partial volume effect that affects the 3D reconstruction volume, the image resolution of the orthographic projection images directly acquired from these complementary views is higher than that of DRRs.

[0150] Given a 180° rotation along the y-axis and then a horizontal flip relative to the principal point, the perspective projection of a is:

[0151]

[0152] Compared with formula (3) Compared to this, only the sign of z changes. If we define is the magnification factor for a 180° view, and the point-to-point distance between two complementary views can be calculated as follows:

[0153]

[0154] In cone-beam X-ray imaging, the complementary views provide additional beneficial information about the location of the target point due to the cone angle. More specifically, under the constraint In the case of , one of m and m' is greater than or equal to 1, and the other is less than or equal to 1. Because of this, lie in and Therefore, the complementary view directly provides the interval where the orthographic projection of the target point should be. For example, in Figure 3D In the example, the blue dot is between the red dot and the green dot. We use the ratio α to represent exist and The relative position between

[0155] α=d PD / d 180° (9)

[0156] It is worth mentioning that if and only if z=0, that is, m'=m=1, d PD and d 180° are all equal to 0. This provides a basis for the human eye to directly determine which structures are subject to perspective deformation by observing the deviation between the two views of the structure of interest. Fig. 6F and Figure 6L shown.

[0157] 7A to 7F Shown are parallel-beam and cone-beam perspective projection images of the same small ball phantom in complementary viewing settings. Figure 7C Explains the cone-beam perspective projection deformation, that is, the cone-beam perspective projection of the 0° view ( Figure 7B ) and parallel beam projection ( Fig. 7A ). Figure 7C It can be seen that the size of the perspective deformation increases radially (radially) from the center to the outside. The 180° cone beam perspective projection image is as follows Fig.7D As shown, the difference with the 0° cone beam perspective projection is as follows Fig. 7E shown. Fig. 7Eand Figure 7C In the area of ​​the small ball phantom, it can be seen that the deviation between the complementary cone-beam projection views is strongly correlated with the cone-beam perspective deformation. In order to integrate this dual-view information, the perspective projection images are transformed from the 0° and 180° views to Figure 7F 3-channel RGB image in . The red and blue channels use images from the 0° view, while the green channel uses images from the 180° view. In the RGB image, the colors reveal the intensity difference between the 0° and 180° perspective projection images. The gray areas contain close intensity values ​​for the two views. In contrast, the magenta and green areas indicate larger intensity values ​​from the 0° and 180° views, respectively. They correspond to Fig. 7E Positive (bright) and negative (dark) areas in the difference image. It can be seen that perspective distortion correction is necessary. Figure 7F In the figure, the magenta ball phantom images and their corresponding green ball phantom images are close to each other, which enables a network with a limited receptive field size to capture the point-to-point dependencies.

[0158] Fig. 8A and Figure 8B Numerical uncertainty analysis for different view settings is explained. si =600 mm CBCT system, the point-to-point distance d of an exemplary cylindrical object (320 mm in diameter and 320 mm in height) PD d 5° d 90° and d 180° The distribution in Fig. 8A A numerical analysis was performed in Fig. 8A It can be seen that the perspective deformation d PD They are mainly distributed in the range of [0,55] mm, which is the minimum receptive field size required for the neural network to learn perspective deformation. For dual orthogonal views, d 90° The distribution is in the large range of [0,280] mm, which requires a larger receptive field for the neural network and brings great uncertainty to the neural network learning perspective deformation. For the 5° view, d 5° They are mainly distributed in the narrow range of [0,25] mm, which is too narrow to reflect different degrees of perspective deformation. 180° Mainly distributed in the range of [0,90] mm, in d PD The distribution of Α is as follows Figure 8B As shown. si =600mm, α is distributed in a narrow range of [0.364,0.636]. si When the diameter increases to 750 mm, the distribution range of α becomes narrower [0.395, 0.605]. The distribution of α shows that along the radial line, Located near and This greatly reduces the uncertainty. Due to the relatively short distance dependence and lower uncertainty, learning perspective deformation using dual complementary views is more effective for neural networks than learning with dual orthogonal views.

[0159] Various state-of-the-art and emerging generative neural networks can be applied to the perspective deformation correction system to learn perspective deformation. As mentioned earlier, perspective deformation depends on position. This position dependency can be effectively learned by the position encoding layer in Transformer neural networks such as VisionTransformer (ViT). Convolutional neural networks (CNNs for short) are efficient in learning translational features, but they are generally unable to effectively learn rotational change features. Instead of designing a new network architecture to address this limitation, a simple preprocessing step - polar coordinate transformation - allows CNN to effectively learn perspective deformation correction. For CNN, learning perspective deformation in polar space is more advantageous than learning directly in Cartesian space, because radial deformation in Cartesian space is converted into translational deformation in polar space, such as Figure 4B As shown. The present invention uses Pix2pixGAN as a representative of CNN and TransU-Net as a representative of the Transformer neural network to describe and verify the cone-beam X-ray perspective deformation correction method according to the first preferred embodiment of the present invention, wherein Pix2pixGAN uses U-Net as its generator and a 5-layer CNN as its discriminator, and the training loss function includes an adversarial training loss function (adversarial loss), an L1 loss function, and a perceptual loss function (perceptual loss) based on the pre-trained VGG-16 weights. TransU-Net is a CNN-Transformer hybrid network based on ViT. Like CNN, it can maintain the high resolution of the generated image and can handle long-distance dependent features like Transformer. The encoder of TransU-Net adopts the ResNet50+ViT-B / 16 configuration, and the decoder is the right half (expansive path) of the ordinary U-Net. The loss function for training TransU-Net is the L1 loss function and the perceptual loss function.

[0160] As a proof of concept, the present invention first studies the perspective deformation correction method on simulated spherical phantom image data. In order to evaluate the generalization of the proposed complementary view setting, simulated projection data experiments were further conducted on patient head CT datasets and real CBCT projection data. The real data contains 20 human knee specimens scanned by the CBCT system. The reference image of the real data is the orthographic projection image after iterative 3D reconstruction of the measured CBCT projection data.

[0161] One of the reference images is shown in rectangular and polar coordinates as 9A to 9C As shown in , the corresponding Pix2pixGAN prediction graphs of different configurations. FIG. 12A to FIG. 12L As shown, Fig. 12A is the result graph obtained by taking a single view of 0° viewing angle in a rectangular coordinate system as the network input, where Fig. 12B is the result graph obtained by superimposing the 0° and 90° views in the rectangular coordinate system as the network input, where Fig. 12C is the result graph obtained by superimposing the 0° and 180° views in the rectangular coordinate system as the network input, where Fig.12D is the result graph obtained by taking a single view of 0° viewing angle in polar coordinate system as the network input, where Fig.12E is the result graph obtained by superimposing the 0° and 90° views in the polar coordinate system as the network input, where Fig.12F is the result image obtained by superimposing the 0° and 180° views in the polar coordinate system as the network input, where Figure 12G yes Fig. 12A The error image of (compared with the parallel beam projection image), where Fig.12H yes Fig. 12B The error image is Fig.12I yes Fig. 12C The error image is Fig.12J yes Fig.12D The error image is Figure 12K yes Fig.12E The error image is Figure 12L yes Fig.12F The error image of . Fig. 9C This shows how the complementary view looks like in polar coordinates, with the neural network only having to move a small spherical phantom horizontally to a position between the corresponding magenta and green pairs. FIG. 12A to FIG. 12L The prediction results are as follows FIG. 12A to FIG. 12F As shown, the error image is FIG. 12G to FIG. 12L As shown, the error image is displayed in the [-50,50] window. In all error images, it can be seen that Figure 12L In order to compare the overall image quality, the root mean square error (RMSE) and structural similarity index (SSIM) values ​​of the prediction results under different configurations are shown in the figure. Fig.10 shown.

[0162] exist Fig.10 In addition to FIG. 12A to FIG. 12L In addition to the results of , three more results are added for comparison:

[0163] a) Combining the 0° and 90° perspective projection images to form Figure 5C , as the input of the neural network, Fig.10 In Chinese, it is indicated by “0°&90°”;

[0164] b) Use Fig. 7E The difference image of the 0° and 180° perspective projection images shown is used as the third channel of the RGB image, and the 0° perspective projection is no longer reused. Fig.10 It is indicated by "0°&180°+";

[0165] c) The direct combination of 0°, 90° and 180° perspective projection images as three channels of RGB input, denoted by “0°, 90° and 180°”.

[0166] The “0°&180°” experiment was repeated 10 times to avoid the influence of random weight initialization.

[0167] In image space, Fig.10 It is shown that learning perspective deformation in polar coordinates can significantly improve the image quality of Pix2pixGAN compared to rectangular coordinates. For the acquisition views, there is no significant improvement in RMSE value compared to a single view when combining orthogonal views. Including the difference image as the third channel (comparison of "0°&180°" vs. "0°&180°+") improves the image quality in rectangular coordinates. But in polar coordinates, it has little effect on image quality. In addition, there is no significant improvement in combining three views ("0°, 90°&180°") compared to "0°&180°". In summary, Fig.10 It shows the advantage of Pix2pixGAN using two complementary views in polar coordinates to correct perspective deformation.

[0168] The quantitative evaluation results of TransU-Net in different spatial coordinate systems and different views are shown in Fig.10 For spatial coordinates, TransU-Net performs comparably well in learning perspective deformations in rectangular and polar coordinates, although the performance is slightly better in polar coordinates. Regarding the resulting views, as with Pix2pixGAN, for TransU-Net, using dual complementary views is generally better than using dual orthogonal views or a single view.

[0169] FIG. 13A to FIG. 13OThe results of human head measurement imaging are shown as an example. In the 0° perspective projection image ( Fig. 13B ), due to perspective distortion, the anatomical structures on the left and right sides do not overlap well, especially the mandible, e.g. Fig. 13B As shown by the red arrow in the figure, it will lead to inaccurate marking of the mandibular angle point when detecting the head. Fig. 13B With reference Fig.13A The difference is as Fig. 13C As shown, Fig. 13C A 2 mm scale is shown, where 2 mm is the clinically acceptable accuracy for detecting cephalic landmarks. It is clear that many anatomical structures are displaced by more than 2 mm in the 0° perspective projection image. Fig.13D ), the perspective distortion is reduced, such as Figure 13G For example, the mandibular region has a smaller error. However, Figure 13G It also shows that many bone structures still have deviations greater than 2 mm. The results of complementary view learning in rectangular coordinates and polar coordinates are shown in Figure 2. Fig.13E and Fig.13F As shown in . The perspective deformation of both images is very small, as shown in Fig.13H and Fig.13I However, in Fig.13E In the figure, the two arrows indicate two dark areas. Fig.13H The results of TransU-Net are shown in the difference graph. FIG. 13J to FIG. 13O As shown. Figure 13M In the figure, some structures near the landmarks are distorted, such as the ear canal indicated by the arrow. The results show that using dual complementary views, TransU-Net can effectively reduce perspective distortion in both rectangular and polar coordinates.

[0170] FIG. 14A to FIG. 14C The robustness of Pix2pixGAN to geometric inaccuracies when using 0° and 180° polarity inputs on a small ball phantom is shown. Here, Pix2pixGAN is used as an example because it maintains good performance on multiple datasets mentioned above. When the geometric inaccuracies such as rotation, source-to-isocenter distance Dsi, and detector principal point position increase, the corresponding RMSE curve remains stable within a certain error range and then increases steadily, rather than increasing sharply immediately.

[0171] The complementary view setting for correcting the perspective deformation is evaluated on real CBCT projection data. In this evaluation, real CBCT projection data from a metal-implanted knee dataset are used for testing, while DRR from a metal-implanted volume CT dataset is used for training.

[0172] FIG. 15A to FIG. 15CThree examples of DRR perspective projection images used for training are shown, with synthetic metal implants placed. FIG. 16A to FIG. 16R As shown in Figure 1, the image contrast, metal image resolution and other appearances of the DRR training image are different from those of the real projection image. FIG. 16A to FIG. 16R In , results from three knee joints with and without metallic implants are shown. FIG. 16A to FIG. 16C is a 0° viewing angle projection, where FIG. 16D to FIG. 16F It is a 180° viewing angle projection, and the detector is regenerated by geometric calibration based on the projection of the respective principal point and the origin. FIG. 16G to FIG. 16I Their difference images are shown, where the magnitude of the deviation increases from the center outward, just as in DRRs with ideal scanning trajectories (e.g., Fig. 7E ), although real projection data are affected by various physical effects such as beam hardening and Poisson noise.

[0173] FIG. 16J to FIG. 16L Color overlay module showing 0° and 180° perspective projection images. Magenta and green areas indicate structures with considerable perspective distortion, e.g. Fig.16J The patella of the knee joint, Figure 16K The top of the two metal Figure 16L Two screws in the bottom middle. FIG. 16M to FIG. 16O Display the reference image, which is the orthographic projection of the iterative reconstruction of the measured CBCT projection data. In the reference image, a total of 5 marking positions are selected and marked with green dots: Figure 16M In the middle, mark two locations on the edge of the patella of the knee; Fig.16N and Fig.16O In each image, a position of the left edge of the fibula is marked. In addition, the two metal rectangular frames are represented by green dotted lines, and their width and height are represented by green solid lines, which are 29.71 mm and 111.99 mm respectively. Fig.16O In the figure, the center lines of the two screws at the bottom are indicated by green lines, the length of the middle screw is 17.14 mm, and the length of the lower screw is 19.38 mm. In the perspective projection image, the two metal corresponding rectangular boxes and the screw center lines are also marked in red. Fig. 16B In the figure, the width and height of the metal are 32.56 mm and 109.85 mm, respectively, with deviations from the reference values ​​of 2.84 mm and 1.95 mm, respectively. Fig. 16C In the figure, the centerline lengths are 18.91mm and 19.54mm, with deviations of 1.77mm and 0.16mm, respectively. Although the length deviation between the bottom screw and the reference is small, the direction deviation of the two screws is obvious.

[0174] Results for Pix2pixGAN using 0° and 180° polarity inputs show that for all markers, the green reference points are accurately located in the Pix2pixGAN image. The rectangular reference frame also accurately covers Figure 16Q The metal in Figure 16R In the figure, although the two red center lines do not completely coincide with the green center line, they are very close in length and direction. The last row shows the results of TransU-Net using 0° and 180° right-angle inputs, which are comparable to Pix2pixGAN.

[0175] refer to Fig.17A , in a true 0° perspective projection, the edges of the pore structures and bones appear sharp. The presence of Poisson noise can also be visualized to some extent. Fig. 17B , in the DRR reference image, the pore structures and bone edges appear blurred. This may be due to the partial volume effect of the intermediate 3D reconstruction volume. Fig. 17C ,In the Pix2pixGAN output, there is a slight smoothing effect. But overall, the image resolution of most anatomical structures is preserved, e.g., foram-like structures.

[0176] like Figure 7F and FIG. 16J to FIG. 16L As shown, the RGB overlay of two complementary views provides a practical way to identify which structures suffer from perspective deformation. Note that this is a sufficient condition, but not a necessary one. Color structures that deviate between the two complementary views must suffer from perspective deformation, given adequate geometric calibration. However, some structures that appear gray may also suffer from perspective deformation due to symmetry with respect to the detector. For example, the gray cylindrical background in the ball digital phantom ( Figure 7F ) are also affected by perspective distortion. Nevertheless, the neural network is still able to correct these areas, such as FIG. 12A to FIG. 12L This shows that the neural network not only exploits the geometric features provided by the complementary views, but also exploits object / task specific features to learn perspective deformation. The human anatomical structure is approximately symmetrical with respect to the midsagittal plane of the body. However, in practical applications, the midsagittal plane usually does not overlap with the virtual detection plane. Therefore, structures with perspective deformation are still represented by color. This geometric information can still be exploited by the neural network, as demonstrated by the superior performance of the complementary view setting on head data ( FIG. 13A to FIG. 13O ).

[0177] Since learning perspective deformation through dual orthogonal views in a dual-detector X-ray system does not require rotating the detector, it is more convenient than learning from dual complementary views in practice. Therefore, learning perspective deformation from dual orthogonal views is still valuable. In the present invention, the effect of learning perspective deformation using dual orthogonal views is not ideal. This is because the neural network has a large uncertainty when learning point-to-point correspondences, such as Fig. 8A and Figure 8B As shown. When the number of small ball phantoms is not large, it is easy for the human visual system to determine this correspondence based on size and intensity information. However, learning feature correspondence from two perspectives, such as point-to-point correspondence, remains a challenging task for neural networks. Currently, a lot of research is devoted to this topic. However, it remains an open solution. Discussion on correspondence learning and semantic reasoning in translation is based on Rn inappropriate perspective offset learning of dual orthogonal views. Another possible direction is to seek an intermediate 3D reconstruction from two orthogonal views. However, current algorithms are not sufficient to generate orthographic projection images from such 3D volumes. In addition, the orthographic projection DRR generated by such reconstruction usually has a lower resolution, such as Fig. 17B As shown in Figure 2 (where 400 projections are used for reconstruction). Further research on how to improve the quality of such few-viewpoint reconstructed images is necessary for our applications. For tasks requiring long-range dependencies, CNNs show limited performance due to the inherent locality of convolution operations. With large-scale pre-training, Transformers are able to learn long-range relations and transfer to downstream tasks. In this work, Pix2pixGAN and TransU-Net are studied as representatives of CNNs and Transformers, respectively. Since the use of complementary views reduces point-to-point dependencies ( Fig. 8A and Figure 8B ), Pix2pixGAN can effectively learn perspective deformation in polar coordinates. However, due to TransU-Net’s ability to learn position-dependent features and long-range dependencies, it learns perspective deformation better than Pix2pixGAN in Cartesian space. There is usually a gap between simulated X-ray images and real X-ray images. In our work, the appearance difference of image contrast and resolution is FIG. 15A to FIG. 16R As shown. Depending on the specific application, domain adaptation techniques are deep learning models trained on synthetic data must be able to generalize well to real test data. However, for various applications, deep learning models trained only on synthetic data are reported to be effective directly on real data. FIG. 15A to FIG. 15C ) have certain versatility for the real human body data in the present invention, such as FIG. 16A to FIG. 16RAs shown in the results of , the locations of landmarks such as the knee patella can be accurately predicted. Although real projection data are affected by various physical effects such as beam hardening and Poisson noise, the size of the deviation between two complementary views in the real data increases from the center outward in a similar way to DRR with an ideal scanning trajectory. This is the basis for our model trained from DRR to generalize well to real data.

[0178] FIG. 14A to FIG. 14C The effects of geometric inaccuracies such as source-to-isocenter distance error, rotation angle error, and detector principal point offset were investigated. The results show that the model with complementary view settings is robust to systematic errors within the limits of geometric calibration accuracy and mechanical manufacturing accuracy. This suggests that a model trained for one CBCT scanner can potentially be generalized to other CBCT scanners of the same type (with the same system configuration but slightly different configuration parameters). Therefore, a model trained with synthetic data in an ideal circular trajectory can still predict satisfactory results for the data obtained in real trajectories in our work ( FIG. 16A to FIG. 16R ). To achieve better generalization across different CBCT system settings, further techniques need to be explored, such as the use of conditional networks.

[0179] The out-of-distribution (OOD) problem is a major obstacle for practical applications of deep learning algorithms. This is a common problem for all data-driven models and it is not specific to our application. The current practice to alleviate this problem is to train a task-specific model with the largest possible training dataset to cover a variety of inference scenarios. Due to our limited access to real patient data, a comprehensive study of this issue is not possible at this stage. However, real data experiments can provide some hints on the robustness and generalizability of our approach in the presence of surgical tools, as metal implants such as screws and K-wires are inserted. Real data experiments show that our proposed approach is effective in the presence of metal implants and surgical screws, such as FIG. 16A to FIG. 16R shown.

[0180] This paper studies learning perspective deformation in X-ray transmission imaging using a framework with two complementary views. The color overlay of complementary views provides a practical method to identify which structures suffer from perspective deformation. Numerical simulation experiments show that complementary views have advantages over orthogonal views or single views in learning perspective deformation. Studies on spatial coordinate systems show that Pix2pixGAN as a CNN performs better in polar space than in Cartesian space. TransU-Net as a transformer-based network achieves comparable performance in Cartesian and polar spaces. This paper further studies that the method has a certain tolerance to geometric errors such as source-to-isocenter distance, rotation angle, detector principal point displacement, and respiratory motion within the calibration accuracy. This shows that a model trained from one CBCT scanner is applicable to other scanners of the same type. Experiments on chest and head data show that our method has the potential to accurately measure cardiothoracic ratio and head imaging. Experiments on real data show that the method is robust in the presence of metal implants and surgical screws. In summary, this paper provides a method for learning perspective deformation through complementary view settings, enabling traditional CBCT systems with more future applications.

Claims

1. An X-ray perspective deformation correction system, for correcting the deformation of an object to be inspected caused by the perspective projection of an X-ray beam, comprising: a projection imaging system, wherein the projection imaging system performs perspective projection on the object to be inspected from two complementary viewing angles to obtain a first view and a second view respectively; a color overlay module, wherein the first view and the second view are processed by the RGB to form an overlay color image; and A learning module, wherein the superimposed color image is input into the learning module and processed by the learning module to obtain a corrected image.

2. The X-ray perspective deformation correction system according to claim 1, wherein the projection imaging system comprises a vertebral beam X-ray source, a detector and a driving module, wherein the driving module drives the vertebral beam X-ray source to rotate around an isocenter so that the vertebral beam X-ray source performs X-ray perspective projection on the object to be detected from two complementary viewing angles, wherein the plane where the detector is located passes through the isocenter.

3. The X-ray perspective deformation correction system according to claim 1, wherein the projection imaging system comprises two vertebral beam X-ray sources and a detector, wherein the two vertebral beam X-ray sources are symmetrical at complementary angles with an isocenter as the center, so that the vertebral beam X-ray sources perform X-ray perspective projection on the object to be detected from two complementary viewing angles respectively, and the plane where the detector is located passes through the isocenter.

4. The X-ray perspective deformation correction system according to claim 1, wherein the color overlay module includes a red channel, a blue channel and a green channel, wherein the red channel and the blue channel use the first view, and the green channel uses the second view.

5. The X-ray perspective deformation correction system according to claim 1, wherein the learning module comprises a coordinate conversion module and a neural network, wherein the superimposed color image is converted from rectangular coordinates to polar coordinates by the coordinate conversion module to obtain a polar coordinate conversion superimposed color image, wherein the polar coordinate conversion superimposed color image is input into the neural network and processed by the neural network to output an output image, wherein the output image is subjected to an inverse polar coordinate conversion from polar coordinates to rectangular coordinates to obtain the corrected image.

6. The X-ray perspective deformation correction system according to claim 5, wherein the neural network is selected from a CNN neural network and a transformer neural network.

7. The X-ray perspective deformation correction system according to claim 5, wherein the neural network is selected from Pix2pixGAN and TransU-Net.

8. The X-ray perspective deformation correction system according to claim 5, wherein the projection imaging system comprises a vertebral beam X-ray source, a detector and a driving module, wherein the driving module drives the vertebral beam X-ray source to rotate around an isocenter so that the vertebral beam X-ray source performs X-ray perspective projection on the object to be detected from two complementary viewing angles, wherein the plane where the detector is located passes through the isocenter.

9. The X-ray perspective deformation correction system according to claim 5, wherein the projection imaging system comprises two vertebral beam X-ray sources and a detector, wherein the two vertebral beam X-ray sources are symmetrical at complementary angles with an isocenter as the center, so that the vertebral beam X-ray sources perform X-ray perspective projection on the object to be detected from two complementary viewing angles respectively, and the plane where the detector is located passes through the isocenter.

10. The X-ray perspective deformation correction system according to claim 8 or 9, wherein the color overlay module comprises a red channel, a blue channel and a green channel, wherein the red channel and the blue channel use the first view, and wherein the green channel uses the second view.

11. The X-ray perspective deformation correction system according to claim 10, wherein the angle range of the complementary viewing angle is 175°-185°.

12. A method for correcting X-ray perspective deformation, characterized in that: The following steps are involved: A) performing cone-beam X-ray perspective projection on the same object to be detected from two complementary angles to form two complementary views, wherein the two complementary views include a first view and a second view; B) resetting the two complementary images on a detector to correct the sizes of the two complementary images; C) converting the two complementary views into color images and superimposing them to generate a superimposed color image; D) performing polar coordinate transformation on the superimposed color image by polar coordinate transformation to obtain a polar coordinate transformed superimposed color image; E) the polar coordinate transformed superimposed color image is input into a neural network as an input image; F) the neural network processes the polar coordinate transformation overlay color image and outputs an output image; and G) Inverse polarity transformation of the output image to obtain a corrected image.

13. The X-ray perspective distortion correction method according to claim 12, wherein step A comprises the following steps: A1) a vertebral beam X-ray source performs vertebral beam X-ray perspective projection on the object to be detected at a first position to obtain the first view; A2) rotating the vertebral beam X-ray source 180° around an isocenter to move the vertebral beam X-ray source to a second position; A3) The vertebral beam X-ray source performs vertebral beam X-ray perspective projection on the object to be detected at the second position to obtain the second view.

14. The X-ray perspective deformation correction method according to claim 12, wherein in step A, the two vertebral beam X-ray sources are symmetrical at complementary angles with the first isocenter as the symmetry point.

15. The X-ray perspective distortion correction method according to claim 13 or 14, wherein the plane where the detector is located passes through the isocenter.

16. The X-ray perspective deformation correction method according to claim 15, wherein in step C, the first view and the second view are processed by a color overlay module, wherein the color overlay module includes a red channel, a blue channel and a green channel, wherein the red channel and the blue channel use the first view, and the green channel uses the second view.

17. The X-ray perspective deformation correction method according to claim 16, wherein the superimposed color image in step D is converted from rectangular coordinates to polar coordinates to obtain the polar coordinate converted superimposed color image.

18. The X-ray perspective distortion correction method according to claim 17, wherein in step G, the output image is subjected to an inverse polar coordinate conversion from polar coordinates to rectangular coordinates, thereby obtaining the corrected image.

19. The X-ray perspective deformation correction method according to claim 18, wherein step D is performed by selecting the neural network from a CNN neural network and a transformer neural network.

20. The X-ray perspective deformation correction method according to claim 19, wherein the neural network is selected from Pix2pixGAN and TransU-Net.

Images