CT (Computed Tomography) image generation method, device and equipment based on positive and lateral X-ray images

Abstract

The invention provides a CT image generation method, device and equipment based on positive and lateral X-ray images and a computer readable storage medium, and the method comprises the steps: firstly constructing a pairing data set, a non-pairing data set and a simulation data set which are needed by model training; and training and optimizing a style migration diffusion model and a diffusion model from the positive and lateral X-ray image to the CT image on the basis of the constructed data set to process the positive and lateral X-ray image of the to-be-generated simulated CT image, so that the CT image with relatively high quality can be generated.

Description

Technical Field

[0001] The present invention relates to the technical field of image generation, and in particular to a method, device, equipment and computer-readable storage medium for generating CT images based on frontal and lateral X-ray images. Background Art

[0002] Head CT images can help doctors get accurate structural information of the patient's head, but the radioactivity of CT can cause certain harm to patients. X-rays are widely used in clinical practice because there is only a small amount of radioactive hazards in the process of taking X-rays, but since they are only two-dimensional images, it is difficult for doctors to get the internal structure from them. Therefore, there are many methods for generating head CT images from X-rays, such as generating head CT images based on GAN (generative adversarial network) or CNN (convolutional neural network). However, the head CT images generated by these two methods have the problem of poor quality or low authenticity. With the rapid development of diffusion models in recent years, diffusion models have been applied to the generation of X-rays from lateral and anteroposterior films to CT images, which can generate better CT images than the generation methods based on GAN networks or CNN networks.

[0003] However, existing conditional diffusion models usually require paired data. However, in real scenarios, it is not easy to obtain paired data due to the burden of data collection and annotation. The conditional data and target data in real scenarios are usually unpaired or partially paired. In addition, the lack of X-ray frontal and lateral films and head CT image datasets in actual clinical datasets leads to low quality of generated CT images. Therefore, how to use a small amount of paired datasets and a large amount of unpaired datasets to generate high-quality CT images has become a huge challenge.

[0004] Those skilled in the art have been looking for a solution to generate high-quality head CT images using a small amount of paired data sets and a large amount of unpaired data sets. Summary of the invention

[0005] The object of the present invention is to provide a method, device, equipment and computer-readable storage medium for generating CT images based on frontal and lateral X-ray images, which can generate high-quality head CT images using a small amount of paired data sets and a large amount of unpaired data sets.

[0006] In order to solve the above technical problems, the present invention provides a method for generating a CT image based on an anteroposterior and lateral X-ray image, the method comprising:

[0007] S1. Collecting multiple frontal and lateral X-ray images and CT images of the head; wherein the frontal and lateral X-ray images include: frontal X-ray images and lateral X-ray images;

[0008] S2, constructing a paired data set, an unpaired data set and a simulated data set; wherein the anteroposterior and lateral X-ray images and the CT images in the paired data set are collected from the same head, and the anteroposterior and lateral X-ray images and the CT images in the unpaired data set are collected from different heads; the simulated anteroposterior and lateral X-ray images in the simulated data set are obtained by processing the CT images in the paired data set and the unpaired data set based on the DDR algorithm, and the simulated CT images in the simulated data set are the CT images in the paired data set and the unpaired data set;

[0009] S3, obtaining a style transfer diffusion model based on training the anteroposterior and lateral X-ray images in the paired data set and the simulated anteroposterior and lateral X-ray images;

[0010] S4, obtaining a diffusion model from the frontal and lateral X-ray images to the CT images based on training the paired data set and the unpaired data set processed by the style transfer diffusion model;

[0011] S5. Processing the anteroposterior and lateral X-ray images of the simulated CT images to be generated sequentially through the style transfer diffusion model and the diffusion model from the anteroposterior and lateral X-ray images to CT images to generate corresponding simulated CT images.

[0012] Optionally, in the method for generating a CT image based on the frontal and lateral X-ray images, the following loss function is used to perform the training of S4:

[0013] L2 loss function between the head CT images in the paired dataset and the simulated head CT images;

[0014] Perceptual loss function between head CT images in the unpaired dataset and simulated head CT images.

[0015] Optionally, in the method for generating a CT image based on the frontal and lateral X-ray images, the method further includes:

[0016] Based on the training of the paired data set and the unpaired data set, a variational autoencoder for the frontal X-ray image and a variational autoencoder for the lateral X-ray image are obtained; and

[0017] When executing S5, the anterior and lateral X-ray images of the simulated CT images to be generated are sent to the anterior X-ray image variational autoencoder and the lateral X-ray image variational autoencoder for processing. The processed results are sequentially processed by the style transfer diffusion model and the diffusion model from the anterior and lateral X-ray images to the CT images to generate the corresponding simulated CT images.

[0018] Optionally, in the method for generating a CT image based on the frontal and lateral X-ray images, the method further includes:

[0019] Based on the training of paired data sets, unpaired data sets and simulated data sets, a CT image variational autoencoder is obtained; and

[0020] The training of S4 is performed with the following loss function:

[0021] an L2 loss function between a head CT image and a simulated head CT image in a paired data set processed by the CT image variational autoencoder;

[0022] The perceptual loss function between the head CT images in the unpaired dataset and the simulated head CT images after being processed by the CT image variational autoencoder.

[0023] Optionally, in the method for generating CT images based on frontal and lateral X-ray images, each variational autoencoder performs feature extraction and data dimensionality reduction on the corresponding image.

[0024] The present invention also provides a CT image generation device based on the anteroposterior and lateral X-ray images, the CT image generation device based on the anteroposterior and lateral X-ray images comprising:

[0025] An image acquisition unit, used for acquiring multiple head frontal and lateral X-ray images and CT images; wherein the frontal and lateral X-ray images include: frontal X-ray images and lateral X-ray images;

[0026] A data set construction unit, used to construct a paired data set, an unpaired data set and a simulated data set; wherein the anteroposterior and lateral X-ray images and the CT images in the paired data set are collected from the same head, and the anteroposterior and lateral X-ray images and the CT images in the unpaired data set are collected from different heads; the simulated anteroposterior and lateral X-ray images in the simulated data set are obtained by processing the CT images in the paired data set and the unpaired data set based on the DDR algorithm, and the simulated CT images in the simulated data set are the CT images in the paired data set and the unpaired data set;

[0027] A style migration diffusion model training unit is used to obtain a style migration diffusion model based on training the anteroposterior and lateral X-ray images in the paired data set and the simulated anteroposterior and lateral X-ray images;

[0028] A diffusion model training unit for anteroposterior and lateral X-ray images to CT images, used for obtaining a diffusion model for anteroposterior and lateral X-ray images to CT images based on training on paired data sets and unpaired data sets processed by the style transfer diffusion model;

[0029] The simulated CT image generating unit is used to receive the anteroposterior and lateral X-ray images for generating simulated CT images, and the anteroposterior and lateral X-ray images are processed in sequence by the style migration diffusion model training unit and the diffusion model training unit from the anteroposterior and lateral X-ray images to CT images to generate simulated CT images corresponding to the anteroposterior and lateral X-ray images.

[0030] Optionally, the CT image generation device based on anteroposterior and lateral X-ray images also includes an image variational autoencoder training unit, which is used to obtain an anteroposterior X-ray image variational autoencoder, a lateral X-ray image variational autoencoder and a CT image variational autoencoder based on training of paired data sets, unpaired data sets and simulated data sets.

[0031] Optionally, in the CT image generation device based on the frontal and lateral X-ray images, the simulated CT image generation unit includes:

[0032] A display unit, used for displaying the generated simulated CT image;

[0033] A receiving unit, used for receiving the frontal and lateral X-ray images of the simulated CT images to be generated input by the user;

[0034] The transmission unit is used to transmit the anteroposterior and lateral X-ray images received by the receiving unit to the image variational autoencoder training unit, and return the results processed by the diffusion model training unit from the anteroposterior and lateral X-ray images to CT images to the display unit for display.

[0035] The present invention also provides an electronic device, the electronic device comprising: a processor and a memory storing computer program instructions;

[0036] When the processor executes the computer program instructions, the CT image generation method based on the frontal and lateral X-ray images as described above is implemented.

[0037] The present invention also provides a computer-readable storage medium, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the method for generating a CT image based on the frontal and lateral X-ray images as described above is implemented.

[0038] In the method, device, equipment and computer-readable storage medium for generating CT images based on the anteroposterior and lateral X-ray images provided by the present invention, firstly, a paired data set, an unpaired data set and a simulated data set required for model training are constructed, and then based on the constructed data set, an optimized style transfer diffusion model and a diffusion model from the anteroposterior and lateral X-ray images to CT images are trained to process the anteroposterior and lateral X-ray images of the simulated CT images to be generated, so as to generate higher quality CT images.

[0039] On the other hand, when training the diffusion model from anteroposterior and lateral X-ray images to CT images, the L2 loss function and perceptual loss function were introduced, which effectively improved the quality and authenticity of the simulated CT images generated by the model and is practical. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The above and other objects, features and advantages of the present disclosure will become more apparent through a more detailed description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, wherein like reference numerals generally represent like components throughout the exemplary embodiments of the present disclosure.

[0041] Figure 1 is a flow chart of a method for generating a CT image based on frontal and lateral X-ray images in one embodiment of the present invention;

[0042] Figure 2 It is a principle block diagram of S6 in the CT image generation method based on the frontal and lateral X-ray images;

[0043] Figure 3 It is a schematic diagram of a CT image generating device based on frontal and lateral X-ray images in one embodiment of the present invention.

[0044] In the figure:

[0045] 1- Image acquisition unit;

[0046] 2-Dataset construction unit;

[0047] 3-Image variational autoencoder training unit;

[0048] 4-Style migration diffusion model training unit;

[0049] 5- Diffusion model training unit from anteroposterior and lateral X-ray images to CT images;

[0050] 6-Simulated CT image generation unit. DETAILED DESCRIPTION

[0051] The following is a further detailed description of the CT image generation method, device, equipment and computer-readable storage medium based on the frontal and lateral X-ray images proposed by the present invention in combination with the accompanying drawings and specific embodiments. The advantages and features of the present invention will become clearer according to the following description and claims. It should be noted that the drawings are all in a very simplified form and are not in precise proportions, and are only used to conveniently and clearly assist in explaining the purpose of the embodiments of the present invention.

[0052] The following is an explanation of the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the present embodiment only illustrate the basic concept of the present invention in a schematic manner, and only the components related to the present invention are shown in the figure instead of being drawn according to the number, shape and size of the components in the actual implementation. The type, quantity and proportion of each component in the actual implementation can be a random change, and the component layout type may also be more complicated.

[0053] Certain terms are used throughout the specification and claims to refer to specific system components. As will be appreciated by those skilled in the art, different companies may refer to a component by different names. It is not intended herein to distinguish between components that have different names but the same function. In the specification and claims, the terms "include" and "comprising" are used in an open-ended manner and, therefore, should be interpreted as "including, but not limited to ...".

[0054] As shown in this application and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0055] In addition, each embodiment of the following description has one or more technical features, but this does not mean that the user of the present invention must implement all the technical features in any embodiment at the same time, or can only implement part or all of the technical features in different embodiments separately. In other words, under the premise that implementation is possible, those skilled in the art can selectively implement part or all of the technical features in any embodiment according to the disclosure of the present invention and according to the design specifications or implementation requirements, or selectively implement a combination of part or all of the technical features in multiple embodiments, thereby increasing the flexibility of the implementation of the present invention.

[0056] In order to make the purpose and features of the present invention more obvious and understandable, the specific implementation methods of the present invention are further described below in conjunction with the accompanying drawings. However, the present invention can be implemented in different forms and should not be considered to be limited to the described embodiments.

[0057] In the first aspect, the present application embodiment provides a method for generating a CT image based on an anterior and lateral X-ray image. Figure 1The CT image generation method based on the frontal and lateral X-ray images of the present invention is described in detail. Figure 1 As shown, the CT image generation method based on the frontal and lateral X-ray images includes the following steps:

[0058] First, step S1 is executed to collect multiple head frontal and lateral X-ray images and CT images; wherein the frontal and lateral X-ray images include: frontal X-ray images and lateral X-ray images.

[0059] Next, step S2 is performed to construct a paired data set, an unpaired data set, and a simulated data set; wherein the anteroposterior and lateral X-ray images and the CT images in the paired data set are collected from the same head, and the anteroposterior and lateral X-ray images and the CT images in the unpaired data set are collected from different heads; the simulated anteroposterior and lateral X-ray images in the simulated data set are obtained by processing the CT images in the paired data set and the unpaired data set based on the DDR algorithm, and the simulated CT images in the simulated data set are the CT images in the paired data set and the unpaired data set.

[0060] For the DDR algorithm, DRR is the abbreviation of Digitally Reconstructed Radiograph. DRR is to generate a 2D image (equivalent to a DR image) by using a mathematical simulation algorithm on 3D CT volume data (multiple cross-sectional image data). Here, the real CT image of the head is simulated and calculated based on the DDR algorithm to generate simulated frontal and lateral X-ray images.

[0061] Next, step S3 is performed to obtain a style transfer diffusion model based on training the anteroposterior and lateral X-ray images in the paired data set and the unpaired data set and the simulated anteroposterior and lateral X-ray images.

[0062] Next, step S4 is performed to obtain a diffusion model from the frontal and lateral X-ray images to the CT images based on training the paired data set and the unpaired data set processed by the style transfer diffusion model.

[0063] In order to improve the quality and authenticity of the generated images, the following loss function is used when training S4:

[0064] L2 loss function between the head CT images in the paired dataset and the simulated head CT images;

[0065] Perceptual loss function between head CT images in the unpaired dataset and simulated head CT images.

[0066] Specifically, when executing S4, the L2 loss function is introduced to train the paired data set; the perceptual loss function is introduced to train the unpaired data set. Based on the introduction of two loss functions, the L2 loss function and the perceptual loss function, when training and optimizing the diffusion model parameters from the frontal and lateral X-ray images to the CT images, the quality and authenticity of the generated images are effectively improved.

[0067] Next, please refer to Figure 2 , execute step S5, and process the anteroposterior and lateral X-ray images of the simulated CT images to be generated by the style transfer diffusion model and the diffusion model from the anteroposterior and lateral X-ray images to the CT images in sequence to generate the corresponding simulated CT images. In other words, the doctor inputs the real anteroposterior and lateral X-ray images of the current patient, and plans to obtain the corresponding simulated CT images based on executing S5, which can be obtained without taking CT scans, effectively avoiding the impact of CT radioactivity on the patient.

[0068] Furthermore, the CT image generation method based on the anteroposterior and lateral X-ray images also includes: obtaining an anteroposterior X-ray image variational autoencoder and a lateral X-ray image variational autoencoder based on a paired data set and an unpaired data set; preferably, when executing S5, the anteroposterior and lateral X-ray images for generating simulated CT images are sent to the anteroposterior X-ray image variational autoencoder and the lateral X-ray image variational autoencoder for processing, and the processed results are sequentially processed by the style transfer diffusion model and the diffusion model from the anteroposterior and lateral X-ray images to CT images to generate the corresponding simulated CT images.

[0069] Furthermore, the method for generating CT images based on frontal and lateral X-ray images further includes: obtaining a CT image variational autoencoder based on training of a paired data set, an unpaired data set, and a simulated data set;

[0070] In order to further optimize the training results of S4, the following loss function is used for S4 training:

[0071] an L2 loss function between a head CT image and a simulated head CT image in a paired data set processed by the CT image variational autoencoder;

[0072] The perceptual loss function between the head CT images in the unpaired dataset and the simulated head CT images after being processed by the CT image variational autoencoder.

[0073] Specifically, the process of processing the anteroposterior X-ray image by the variational autoencoder includes: extracting features and reducing data dimension of the anteroposterior X-ray image; the process of processing the lateral X-ray image by the variational autoencoder includes: extracting features and reducing data dimension of the lateral X-ray image; the process of processing the CT image by the variational autoencoder includes: extracting features and reducing data dimension of the CT image. Here, the variational autoencoder (VAE) can learn the important features of the input data and use them for subsequent classification, clustering and other tasks, map the high temperature data to a low-dimensional latent space, and achieve dimensionality reduction of the data, thereby accelerating the subsequent model training speed and improving the quality of the generated image.

[0074] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0075] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0076] In a second aspect, the present application provides a CT image generation device based on frontal and lateral X-ray images. Figures 1 to 3 The CT image generating device based on the frontal and lateral X-ray images described in this embodiment is described in detail.

[0077] like Figure 3As shown, the CT image generation device based on the anteroposterior and lateral X-ray images includes: an image acquisition unit 1, a data set construction unit 2, a style migration diffusion model training unit 4, a diffusion model training unit 5 from anteroposterior and lateral X-ray images to CT images, and a simulated CT image generation unit 6, wherein the image acquisition unit 1 is used to acquire anteroposterior and lateral X-ray images and CT images of multiple heads; wherein the anteroposterior and lateral X-ray images include: anteroposterior X-ray images and lateral X-ray images; the data set construction unit is used to construct a paired data set, an unpaired data set and a simulated data set based on the data acquired by the image acquisition unit 1; wherein the anteroposterior and lateral X-ray images and CT images in the paired data set are acquired from the same head, and the anteroposterior and lateral X-ray images and CT images in the unpaired data set are acquired from different heads; the simulated anteroposterior and lateral X-ray images in the simulated data set are images of the paired data set and the unpaired data set based on the DDR algorithm. The simulated CT images are obtained by processing the CT images, wherein the simulated CT images in the simulated data set are the CT images in the paired data set and the unpaired data set; the style transfer diffusion model training unit 4 is used to obtain the style transfer diffusion model based on the training of the anteroposterior and lateral X-ray images in the paired data set and the unpaired data set and the simulated anteroposterior and lateral X-ray images; the diffusion model training unit 5 from the anteroposterior and lateral X-ray images to the CT images is used to obtain the diffusion model from the anteroposterior and lateral X-ray images to the CT images based on the training of the paired data set and the unpaired data set processed by the style transfer diffusion model; the simulated CT image generating unit 6 is used to receive the anteroposterior and lateral X-ray images for generating the simulated CT images, and the anteroposterior and lateral X-ray images are sequentially processed by the style transfer diffusion model training unit 4 and the diffusion model training unit 5 from the anteroposterior and lateral X-ray images to the CT images to generate the simulated CT images corresponding to the anteroposterior and lateral X-ray images.

[0078] like Figure 3 and Figure 2As shown, the CT image generation device based on the anteroposterior and lateral X-ray images also includes: an image variational autoencoder training unit, which is used to obtain the anteroposterior X-ray image variational autoencoder, the lateral X-ray image variational autoencoder and the CT image variational autoencoder based on the training of the paired data set, the unpaired data set and the simulated data set. Here, the anteroposterior X-ray image variational autoencoder and the lateral X-ray image variational autoencoder are obtained based on the training of the paired data set and the unpaired data set; the CT image variational autoencoder is obtained based on the training of the paired data set, the unpaired data set and the simulated data set. At this time, the anteroposterior and lateral X-ray images to be generated by the simulated CT image generation unit 6 are sent to the anteroposterior X-ray image variational autoencoder and the lateral X-ray image variational autoencoder for processing, and the processed results are sequentially processed by the style transfer diffusion model and the diffusion model from the anteroposterior and lateral X-ray images to the CT images to generate the corresponding simulated CT images.

[0079] Specifically, Figure 2 As shown, the working process of the CT image generation device based on the anteroposterior and lateral X-ray images is as follows: the anteroposterior X-ray image and the lateral X-ray image (collectively referred to as the real anteroposterior and lateral X-ray images) of the same head are respectively processed by the anteroposterior X-ray image variational autoencoder and the lateral X-ray image variational autoencoder and input into the style transfer diffusion model to generate a simulated anteroposterior X-ray image and a simulated lateral X-ray image; then, the simulated anteroposterior X-ray image and the simulated lateral X-ray image generated by the style transfer diffusion model are input into the diffusion model from the anteroposterior and lateral X-ray images to the CT image to generate a simulated CT image of the corresponding head. The entire process of generating the simulated CT image is simple and fast, and the generated simulated CT image is of high quality and has clinical practicality.

[0080] Furthermore, the simulated CT image generating unit 6 includes: a display unit, a receiving unit and a transmission unit, wherein the display unit is used to display the generated simulated CT image; the receiving unit is used to receive the anteroposterior and lateral X-ray images of the simulated CT image to be generated input by the user; the transmission unit is used to transmit the anteroposterior and lateral X-ray images received by the receiving unit to the image variational autoencoder training unit 3, and return the results processed by the diffusion model training unit 5 from the anteroposterior and lateral X-ray images to the CT images to the display unit for display.

[0081] It can be understood that since the method embodiment and the device embodiment are different presentation forms of the same technical concept, the content of the method embodiment part in this application should be synchronously adapted to the device embodiment part and will not be repeated here.

[0082] In a third aspect, an embodiment of the present application provides an electronic device, including a processor and a memory storing computer program instructions;

[0083] When the processor executes the computer program instructions, it implements the CT image generation method based on the frontal and lateral X-ray images as described in any one of the first aspects of the embodiments of the present application.

[0084] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which computer program instructions are stored. When the computer program instructions are executed by a processor, a CT image generation method based on frontal and lateral X-ray images as described in any one of the first aspects of the embodiments of the present application is implemented.

[0085] In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in a processor or an instruction in the form of software. The steps of the method disclosed in conjunction with the embodiment of the present application can be directly embodied as a hardware processor for execution, or a combination of hardware and software modules in a processor for execution. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory and completes the steps of the above method in conjunction with its hardware. To avoid repetition, it is not described in detail here.

[0086] In various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0087] Those skilled in the art will appreciate that the various illustrative logical blocks (ILBs) and steps described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or in a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0088] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0089] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0090] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0091] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more available media integration. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk), etc.

[0092] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0093] In summary, in the method, device, equipment and computer-readable storage medium for generating CT images based on the anteroposterior and lateral X-ray images provided by the present invention, firstly, a paired data set, an unpaired data set and a simulated data set required for model training are constructed, and then, based on the constructed data set, a style transfer diffusion model and a diffusion model from the anteroposterior and lateral X-ray images to CT images are trained and optimized to process the anteroposterior and lateral X-ray images of the simulated CT images to be generated, so as to generate higher quality CT images.

[0094] On the other hand, when training the diffusion model from anteroposterior and lateral X-ray images to CT images, the L2 loss function and perceptual loss function were introduced, which effectively improved the quality and authenticity of the simulated CT images generated by the model and is practical.

[0095] The above description is only a description of the preferred embodiments of the present invention, and is not intended to limit the scope of the present invention. Any changes or modifications made by a person skilled in the art in the field of the present invention based on the above disclosure shall fall within the scope of protection of the claims.

Claims

1. A method for generating a CT image based on an anteroposterior and lateral X-ray image, characterized in that: include: S1. Collecting multiple frontal and lateral X-ray images and CT images of the head; wherein the frontal and lateral X-ray images include: frontal X-ray images and lateral X-ray images; S2, constructing a paired data set, an unpaired data set and a simulated data set; wherein the anteroposterior and lateral X-ray images and the CT images in the paired data set are collected from the same head, and the anteroposterior and lateral X-ray images and the CT images in the unpaired data set are collected from different heads; the simulated anteroposterior and lateral X-ray images in the simulated data set are obtained by processing the CT images in the paired data set and the unpaired data set based on the DDR algorithm, and the simulated CT images in the simulated data set are the CT images in the paired data set and the unpaired data set; S3, obtaining a style transfer diffusion model based on training the anteroposterior and lateral X-ray images in the paired data set and the simulated anteroposterior and lateral X-ray images; S4, obtaining a diffusion model from the anteroposterior and lateral X-ray images to the CT images based on training the anteroposterior and lateral X-ray images and their corresponding CT images in the paired data set and the unpaired data set processed by the style transfer diffusion model; S5. Processing the anteroposterior and lateral X-ray images of the simulated CT images to be generated sequentially through the style transfer diffusion model and the diffusion model from the anteroposterior and lateral X-ray images to CT images to generate corresponding simulated CT images.

2. The method for generating a CT image based on an anteroposterior and lateral X-ray image according to claim 1, characterized in that: The training of S4 is performed with the following loss function: L2 loss function between the head CT images in the paired dataset and the simulated head CT images; Perceptual loss function between head CT images in the unpaired dataset and simulated head CT images.

3. The method for generating a CT image based on an anteroposterior and lateral X-ray image according to claim 2, characterized in that: Also includes: Based on the training of paired data sets and unpaired data sets, variational autoencoders for frontal X-ray images and lateral X-ray images are obtained; as well as When executing S5, the anterior and lateral X-ray images of the simulated CT images to be generated are sent to the anterior X-ray image variational autoencoder and the lateral X-ray image variational autoencoder for processing. The processed results are sequentially processed by the style transfer diffusion model and the diffusion model from the anterior and lateral X-ray images to the CT images to generate the corresponding simulated CT images.

4. The method for generating a CT image based on an anteroposterior and lateral X-ray image according to claim 3, characterized in that: Also includes: Based on the training of paired data sets, unpaired data sets and simulated data sets, a CT image variational autoencoder is obtained; as well as The training of S4 is performed with the following loss function: an L2 loss function between a head CT image and a simulated head CT image in a paired data set processed by the CT image variational autoencoder; The perceptual loss function between the head CT images in the unpaired dataset and the simulated head CT images after being processed by the CT image variational autoencoder.

5. The method for generating a CT image based on an anteroposterior and lateral X-ray image according to claim 4, characterized in that: Each variational autoencoder performs feature extraction and data dimension reduction on the corresponding image.

6. A CT image generation device based on frontal and lateral X-ray images, characterized in that: include: An image acquisition unit, used for acquiring multiple head frontal and lateral X-ray images and CT images; wherein the frontal and lateral X-ray images include: frontal X-ray images and lateral X-ray images; A data set construction unit, used to construct a paired data set, an unpaired data set and a simulated data set; wherein the anteroposterior and lateral X-ray images and the CT images in the paired data set are collected from the same head, and the anteroposterior and lateral X-ray images and the CT images in the unpaired data set are collected from different heads; the simulated anteroposterior and lateral X-ray images in the simulated data set are obtained by processing the CT images in the paired data set and the unpaired data set based on the DDR algorithm, and the simulated CT images in the simulated data set are the CT images in the paired data set and the unpaired data set; A style migration diffusion model training unit is used to obtain a style migration diffusion model based on training the anteroposterior and lateral X-ray images in the paired data set and the simulated anteroposterior and lateral X-ray images; A diffusion model training unit for anteroposterior and lateral X-ray images to CT images, used for obtaining a diffusion model for anteroposterior and lateral X-ray images to CT images based on training on paired data sets and unpaired data sets processed by the style transfer diffusion model; The simulated CT image generating unit is used to receive the anteroposterior and lateral X-ray images for generating simulated CT images, and the anteroposterior and lateral X-ray images are processed in sequence by the style migration diffusion model training unit and the diffusion model training unit from the anteroposterior and lateral X-ray images to CT images to generate corresponding simulated CT images.

7. The CT image generation device based on the anteroposterior and lateral X-ray images according to claim 6, characterized in that: It also includes an image variational autoencoder training unit, which is used to obtain an anteroposterior X-ray image variational autoencoder, a lateral X-ray image variational autoencoder and a CT image variational autoencoder based on training of paired data sets, unpaired data sets and simulated data sets.

8. The CT image generation device based on the anteroposterior and lateral X-ray images according to claim 6, characterized in that: The simulated CT image generating unit comprises: A display unit, used for displaying the generated simulated CT image; A receiving unit, used for receiving the frontal and lateral X-ray images of the simulated CT images to be generated input by the user; The transmission unit is used to transmit the anteroposterior and lateral X-ray images received by the receiving unit to the image variational autoencoder training unit, and return the results processed by the diffusion model training unit from the anteroposterior and lateral X-ray images to CT images to the display unit for display.

9. An electronic device, characterized in that: The electronic device comprises: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, the method for generating a CT image based on the frontal and lateral X-ray images as described in any one of claims 1 to 5 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, and when the computer program instructions are executed by the processor, the method for generating a CT image based on the anteroposterior and lateral X-ray images according to any one of claims 1 to 5 is implemented.

Images