Method, device and equipment for generating CT image based on fronto-lateral x-ray image

Abstract

The application provides a CT image generation method and device based on posteroanterior X-ray images, equipment and a computer readable storage medium. First, paired data sets, unpaired data sets and simulation data sets required for model training are constructed, and then the diffusion model of the posteroanterior X-ray images to the CT images is trained based on the constructed data sets to process the posteroanterior X-ray images of the simulation CT images to be generated, so that a CT image with high quality can be generated.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image generation, in particular to a CT image generation method and device based on anteroposterior and lateral X-ray images, equipment and computer readable storage medium. BACKGROUND

[0002] The head CT image can help doctors obtain accurate structural information of the patient's head, but the radioactivity of CT will cause certain harm to the patient. The X-ray anteroposterior and lateral film shooting process only exists a little radioactivity harm, which is widely used in the clinic by doctors, but since it is only a two-dimensional image, doctors are difficult to obtain the internal structure from it. Therefore, there are many generation methods from X-ray anteroposterior and lateral film to head CT image at present, such as generating head CT image based on GAN (generative adversarial network) or CNN (convolutional neural network). However, the generated head CT images of the two methods have the problems of poor quality or low authenticity. With the rapid development of diffusion model in recent years, the diffusion model is applied to the generation of X-ray anteroposterior and lateral film to CT image, and compared with the generation method based on GAN network or CNN network, the diffusion model can generate better CT image.

[0003] However, the existing conditional diffusion model usually requires paired data, but in the real scene, due to the burden of data collection, labeling and other burdens, it is not easy to obtain paired data, and the conditional data and target data in the real scene are usually unpaired or partially paired, plus the lack of X-ray anteroposterior and lateral film and head CT image data set in the actual clinical data set, resulting in low quality of the generated CT image, therefore, how to use a small amount of paired data set and a large amount of unpaired data set to generate high-quality CT image becomes a great challenge.

[0004] How to use a small amount of paired data set and a large amount of unpaired data set to generate high-quality head CT image is a method that the person skilled in the art has been looking for to solve. SUMMARY

[0005] The purpose of the present application is to provide a CT image generation method and device based on anteroposterior and lateral X-ray images, equipment and computer readable storage medium, which can generate high-quality head CT image by using a small amount of paired data set and a large amount of unpaired data set.

[0006] To solve the above technical problems, the present application provides a CT image generation method based on anteroposterior and lateral X-ray images, which comprises:

[0007] S1, collecting anteroposterior and lateral X-ray images and CT images of a plurality of heads; wherein the anteroposterior and lateral X-ray images comprise anteroposterior X-ray images and lateral X-ray images;

[0008] S2, construct a paired data set, an unpaired data set, and a simulation data set; wherein the frontal-lateral X-ray image and the CT image in the paired data set are collected from the same head, the frontal-lateral X-ray image and the CT image in the unpaired data set are collected from different heads; the simulated frontal-lateral X-ray image in the simulation data set is obtained by processing the CT image in the paired data set and the unpaired data set based on the DDR algorithm, and the simulated CT image in the simulation data set is the CT image in the paired data set and the unpaired data set;

[0009] S3, based on training of the frontal-lateral X-ray image in the paired data set and the unpaired data set and the simulated frontal-lateral X-ray image, a style transfer diffusion model is obtained;

[0010] S4, based on training of the paired data set and the unpaired data set processed by the style transfer diffusion model, a diffusion model of frontal-lateral X-ray image to CT image is obtained;

[0011] S5, the frontal-lateral X-ray image to be generated for the simulated CT image is sequentially processed by the style transfer diffusion model and the diffusion model of frontal-lateral X-ray image to CT image, and the corresponding simulated CT image is generated.

[0012] Optionally, in the CT image generation method based on frontal-lateral X-ray image, the training of S4 adopts the following loss function:

[0013] L2 loss function between the head CT image in the paired data set and the simulated head CT image;

[0014] perception loss function between the head CT image in the unpaired data set and the simulated head CT image.

[0015] Optionally, in the CT image generation method based on frontal-lateral X-ray image, further comprising:

[0016] based on training of the paired data set and the unpaired data set, a frontal X-ray image variational autoencoder and a lateral X-ray image variational autoencoder are obtained; and

[0017] when S5 is executed, the frontal-lateral X-ray image to be generated for the simulated CT image is sent to the frontal X-ray image variational autoencoder and the lateral X-ray image variational autoencoder for processing, and the result of the processing is sequentially processed by the style transfer diffusion model and the diffusion model of frontal-lateral X-ray image to CT image, and the corresponding simulated CT image is generated.

[0018] Optionally, in the CT image generation method based on frontal-lateral X-ray image, further comprising:

[0019] obtain a CT image variational autoencoder based on training of the paired data set, the unpaired data set, and the simulated data set; and

[0020] The training of S4 adopts the following loss function:

[0021] L2 loss function between the head CT image in the paired data set processed by the CT image variational autoencoder and the simulated head CT image;

[0022] perceptual loss function between the head CT image in the unpaired data set processed by the CT image variational autoencoder and the simulated head CT image.

[0023] Optionally, in the CT image generation method based on the frontal and lateral X-ray images, each variational autoencoder performs feature extraction and data dimension reduction on the corresponding image.

[0024] The application further provides a CT image generation device based on frontal and lateral X-ray images, which comprises:

[0025] an image acquisition unit configured to acquire frontal and lateral X-ray images and CT images of multiple heads; wherein the frontal and lateral X-ray images comprise frontal X-ray images and lateral X-ray images;

[0026] a data set construction unit configured to construct a paired data set, an unpaired data set, and a simulated data set; wherein the frontal and lateral X-ray images and the CT images in the paired data set are acquired from the same head, the frontal and lateral X-ray images and the CT images in the unpaired data set are acquired from different heads, the simulated frontal and lateral X-ray images in the simulated data set are obtained by processing the CT images in the paired data set and the unpaired data set based on a DDR algorithm, and the simulated CT images in the simulated data set are the CT images in the paired data set and the unpaired data set;

[0027] a style transfer diffusion model training unit configured to train a style transfer diffusion model based on the frontal and lateral X-ray images in the paired data set and the unpaired data set and the simulated frontal and lateral X-ray images;

[0028] a frontal and lateral X-ray image to CT image diffusion model training unit configured to train a frontal and lateral X-ray image to CT image diffusion model based on the paired data set and the unpaired data set processed by the style transfer diffusion model, to obtain the frontal and lateral X-ray image to CT image diffusion model;

[0029] The analog CT image generation unit is configured to receive a posteroanterior X-ray image to be used to generate an analog CT image, and generate the analog CT image corresponding to the posteroanterior X-ray image after the posteroanterior X-ray image is processed by the style transfer diffusion model training unit and the diffusion model training unit for converting the posteroanterior X-ray image to a CT image in sequence.

[0030] Optionally, in the CT image generation device based on posteroanterior X-ray images, the image variational autoencoder training unit is configured to obtain a posteroanterior X-ray image variational autoencoder, a lateral X-ray image variational autoencoder, and a CT image variational autoencoder based on training of the paired data set, the unpaired data set, and the simulated data set.

[0031] Optionally, in the CT image generation device based on posteroanterior X-ray images, the analog CT image generation unit comprises:

[0032] The display unit is configured to display the generated analog CT image.

[0033] The receiving unit is configured to receive a posteroanterior X-ray image to be used to generate an analog CT image input by a user.

[0034] The transmission unit is configured to transmit the posteroanterior X-ray image received by the receiving unit to the image variational autoencoder training unit, and return the result processed by the diffusion model training unit for converting the posteroanterior X-ray image to a CT image to the display unit for display.

[0035] The electronic device comprises a processor and a memory storing computer program instructions.

[0036] The processor implements the CT image generation method based on posteroanterior X-ray images when executing the computer program instructions.

[0037] The computer readable storage medium stores computer program instructions, and the computer program instructions are executed by the processor to implement the CT image generation method based on posteroanterior X-ray images.

[0038] In the CT image generation method, device, equipment, and computer readable storage medium based on posteroanterior X-ray images provided by the application, the paired data set, the unpaired data set, and the simulated data set required for model training are first constructed, and then the style transfer diffusion model, the diffusion model for converting the posteroanterior X-ray image to a CT image are trained based on the constructed data set to process the posteroanterior X-ray image to be used to generate an analog CT image, so that a CT image with higher quality can be generated.

[0039] On the other hand, in the diffusion model training of the posteroanterior X-ray image to the CT image, the L2 loss function and the perception loss function are introduced, which effectively improves the quality and authenticity of the simulated CT image generated by the model, and has practicality. BRIEF DESCRIPTION OF DRAWINGS

[0040] The above and other objects, features and advantages of the present disclosure will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings in which like reference characters refer to like parts throughout the figures, in which exemplary embodiments of the present disclosure are shown.

[0041] Figure 1 is a flow chart of a CT image generation method based on posteroanterior X-ray images in an embodiment of the present application;

[0042] Figure 2 is a principle block diagram of S6 in the CT image generation method based on posteroanterior X-ray images;

[0043] Figure 3 is a schematic diagram of a CT image generation device based on posteroanterior X-ray images in an embodiment of the present application.

[0044] In the drawings:

[0045] 1 - image acquisition unit;

[0046] 2 - data set construction unit;

[0047] 3 - image variational autoencoder training unit;

[0048] 4 - style transfer diffusion model training unit;

[0049] 5 - posteroanterior X-ray image to CT image diffusion model training unit;

[0050] 6 - simulated CT image generation unit. DETAILED DESCRIPTION

[0051] The CT image generation method, device, equipment and computer readable storage medium based on posteroanterior X-ray images proposed by the present application are further described in detail below in combination with the drawings and specific embodiments. The advantages and features of the present application will be clearer according to the following description and claims. It should be noted that the drawings are very simplified and use non-precise proportions, only to facilitate and clarify the purpose of assisting the description of the embodiments of the present application.

[0052] The advantages and effects of the present application can be easily understood by those skilled in the art from the description of the present application. The present application can also be implemented or applied in other different embodiments, and various modifications or changes can be made to the details in the description based on different views and applications without departing from the spirit of the present application. It should be noted that the drawings provided in the embodiments only schematically illustrate the basic concepts of the present application, and only the components related to the present application are shown in the drawings, rather than the number, shape and size of the components in actual implementation. The actual implementation of each component can be arbitrarily changed, and the component layout pattern can be more complex.

[0053] Certain terminology is used throughout the description and claims to refer to particular components. As one skilled in the art will appreciate, different companies can refer to a component by different names. What is important to the art is that the names used are consistent with the context in which they are used. In the specification and claims, the terms "include" and "comprise" are used in an open-ended fashion, and thus should be interpreted to mean "including, but not limited to....".

[0054] As used in this application and the claims, "a," "an," and "the" include both singular and plural referents unless the context clearly dictates otherwise. Generally, the terms "comprise," "comprises" and "comprising," when used in this specification and in the following claims, specify the presence of stated features, elements, and / or components, but do not preclude the presence or addition of one or more other features, elements, and / or components.

[0055] In addition, each of the embodiments of the following description has one or more technical features, but this does not mean that the user must simultaneously implement all the technical features in any embodiment, or can only separately implement one or more technical features in different embodiments. In other words, under the premise of implementation, the skilled in the art can selectively implement part or all of the technical features in any embodiment, or selectively implement a combination of part or all of the technical features in multiple embodiments, thereby increasing the flexibility of the implementation of the present application, according to the disclosure of the present application, and according to the design specification or implementation requirements.

[0056] In order to make the purpose and features of the present application more obvious and easy to understand, the specific embodiments of the present application are further described below in conjunction with the drawings. However, the present application can be implemented in different forms, and should not be considered as being limited to the described embodiments.

[0057] In a first aspect, the embodiments of the present application provide a CT image generation method based on a posteroanterior X-ray image. Please refer to the following Figure 1The CT image generation method based on the frontal-lateral X-ray image is described in detail. Figure 1 As shown in the figure, the CT image generation method based on the frontal-lateral X-ray image comprises the following steps:

[0058] Firstly, step S1 is performed to collect frontal-lateral X-ray images and CT images of multiple heads; wherein the frontal-lateral X-ray images comprise frontal X-ray images and lateral X-ray images.

[0059] Then, step S2 is performed to construct a paired data set, an unpaired data set, and a simulation data set; wherein the frontal-lateral X-ray images and CT images in the paired data set are collected from the same head, and the frontal-lateral X-ray images and CT images in the unpaired data set are collected from different heads; the simulated frontal-lateral X-ray images in the simulation data set are obtained by processing the CT images in the paired data set and the unpaired data set based on the DDR algorithm, and the simulated CT images in the simulation data set are the CT images in the paired data set and the unpaired data set.

[0060] For the DDR algorithm, DRR is the abbreviation of Digitally Reconstructed Radiograph, and the full name is digital reconstructed radiograph. DRR is a 2D image (equivalent to a DR image) generated by a mathematical simulation algorithm based on 3D CT volume data (multiple cross-sectional image data). Here, the real CT image of the head is simulated and calculated based on the DDR algorithm to generate a simulated frontal-lateral X-ray image.

[0061] Then, step S3 is performed to obtain a style transfer diffusion model based on training of the frontal-lateral X-ray images in the paired data set and the unpaired data set and the simulated frontal-lateral X-ray images.

[0062] Then, step S4 is performed to obtain a diffusion model from frontal-lateral X-ray images to CT images based on training of the paired data set and the unpaired data set processed by the style transfer diffusion model.

[0063] In order to improve the quality and authenticity of the generated image, the following loss function is used when performing the training of S4:

[0064] L2 loss function between the head CT image in the paired data set and the simulated head CT image;

[0065] Perceptual loss function between the head CT image in the unpaired data set and the simulated head CT image.

[0066] Specifically, in the execution of S4, the paired data set is trained by introducing the L2 loss function, and the unpaired data set is trained by introducing the perception loss function. Based on introducing the L2 loss function and the perception loss function when training and optimizing the diffusion model parameters of the frontal and lateral X-ray images to CT images, the quality and authenticity of the generated images are effectively improved.

[0067] Next, please refer to Figure 2 , execute step S5, sequentially process the frontal and lateral X-ray images to be generated into simulated CT images through the style transfer diffusion model and the frontal and lateral X-ray image to CT image diffusion model, and generate the corresponding simulated CT images. In other words, the doctor inputs the real frontal and lateral X-ray images of the current patient, and plans to obtain the corresponding simulated CT images based on the execution of S5, without the need to take CT, effectively avoiding the impact of the radioactivity of CT on the patient.

[0068] Further, the CT image generation method based on frontal and lateral X-ray images further comprises: obtaining a frontal X-ray image variational autoencoder and a lateral X-ray image variational autoencoder based on the paired data set and the unpaired data set; preferably, in the execution of S5, the frontal and lateral X-ray images to be generated into simulated CT images are sent to the frontal X-ray image variational autoencoder and the lateral X-ray image variational autoencoder for processing, and the processed results are sequentially processed through the style transfer diffusion model and the frontal and lateral X-ray image to CT image diffusion model to generate the corresponding simulated CT images.

[0069] Further, the CT image generation method based on frontal and lateral X-ray images further comprises: obtaining a CT image variational autoencoder based on the training of the paired data set, the unpaired data set and the simulated data set;

[0070] In order to further optimize the training result of S4, the following loss functions are used in the execution of S4 training:

[0071] The L2 loss function between the head CT image in the paired data set processed by the CT image variational autoencoder and the simulated head CT image;

[0072] The perception loss function between the head CT image in the unpaired data set processed by the CT image variational autoencoder and the simulated head CT image.

[0073] Specifically, the process of the frontal X-ray image variational autoencoder processing the frontal X-ray image includes feature extraction and data dimension reduction of the frontal X-ray image; the process of the lateral X-ray image variational autoencoder processing the lateral X-ray image includes feature extraction and data dimension reduction of the lateral X-ray image; and the process of the CT image variational autoencoder processing the CT image includes feature extraction and data dimension reduction of the CT image. Here, the variational autoencoder (VAE) can learn important features of input data and use the features for subsequent classification, clustering and other tasks, map high-temperature data to a low-dimensional latent space, realize data dimension reduction, thereby accelerating the subsequent model training speed and improving the quality of generated images.

[0074] It should be noted that, for the foregoing method embodiments, in order to simply describe, they are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited to the action sequence described, because according to the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions and modules involved are not necessarily necessary for the present application.

[0075] In the above embodiments, the description of each embodiment has its own emphasis, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0076] In a second aspect, the embodiments of the present application provide a CT image generation device based on frontal and lateral X-ray images. The following refers to the accompanying drawings to describe the CT image generation device. Figures 1 to 3 The CT image generation device based on frontal and lateral X-ray images is described in detail.

[0077] As Figure 3As shown, the CT image generation device based on the frontal and lateral X-ray images comprises an image acquisition unit 1, a data set construction unit 2, a style transfer diffusion model training unit 4, a frontal and lateral X-ray image to CT image diffusion model training unit 5, and a simulated CT image generation unit 6. The image acquisition unit 1 is configured to acquire frontal and lateral X-ray images and CT images of a plurality of heads. The frontal and lateral X-ray images include frontal X-ray images and lateral X-ray images. The data set construction unit is configured to construct paired data sets, unpaired data sets, and simulated data sets based on the data acquired by the image acquisition unit 1. In the paired data sets, the frontal and lateral X-ray images and the CT images are acquired from the same head. In the unpaired data sets, the frontal and lateral X-ray images and the CT images are acquired from different heads. The simulated frontal and lateral X-ray images in the simulated data sets are obtained by processing the CT images in the paired data sets and the unpaired data sets based on the DDR algorithm. The simulated CT images in the simulated data sets are the CT images in the paired data sets and the unpaired data sets. The style transfer diffusion model training unit 4 is configured to train a style transfer diffusion model based on the frontal and lateral X-ray images in the paired data sets and the unpaired data sets and the simulated frontal and lateral X-ray images. The frontal and lateral X-ray image to CT image diffusion model training unit 5 is configured to train a frontal and lateral X-ray image to CT image diffusion model based on the paired data sets and the unpaired data sets processed by the style transfer diffusion model. The simulated CT image generation unit 6 is configured to receive frontal and lateral X-ray images to be used to generate simulated CT images. After the frontal and lateral X-ray images are sequentially processed by the style transfer diffusion model training unit 4 and the frontal and lateral X-ray image to CT image diffusion model training unit 5, the simulated CT image generation unit 6 generates simulated CT images corresponding to the frontal and lateral X-ray images.

[0078] As Figure 3 and Figure 2As shown, the CT image generation device based on the frontal and lateral X-ray images further comprises an image variational autoencoder training unit configured to obtain a frontal X-ray image variational autoencoder, a lateral X-ray image variational autoencoder, and a CT image variational autoencoder based on training on the paired data set, the unpaired data set, and the simulated data set. Here, the frontal X-ray image variational autoencoder and the lateral X-ray image variational autoencoder are obtained based on training on the paired data set and the unpaired data set; and the CT image variational autoencoder is obtained based on training on the paired data set, the unpaired data set, and the simulated data set. At this time, the frontal and lateral X-ray images of the simulated CT image to be generated received by the simulated CT image generation unit 6 are sent to the frontal X-ray image variational autoencoder and the lateral X-ray image variational autoencoder for processing, and the processed results are sequentially processed by the style transfer diffusion model and the diffusion model from the frontal and lateral X-ray images to CT images, to generate the corresponding simulated CT image.

[0079] Specifically, as shown in Figure 2 As shown, the working process of the CT image generation device based on the frontal and lateral X-ray images is as follows: the frontal X-ray image and the lateral X-ray image (collectively referred to as real frontal and lateral X-ray images) of the same head are input into the style transfer diffusion model after being processed by the frontal X-ray image variational autoencoder and the lateral X-ray image variational autoencoder, to generate simulated frontal X-ray images and simulated lateral X-ray images; then, the simulated frontal X-ray images and the simulated lateral X-ray images generated by the style transfer diffusion model are input into the diffusion model from the frontal and lateral X-ray images to CT images, to generate simulated CT images of the corresponding head. The entire process of generating simulated CT images is simple and fast, and the generated simulated CT images have high quality and are practical in clinical applications.

[0080] Further, the simulated CT image generation unit 6 comprises a display unit, a receiving unit, and a transmission unit, wherein the display unit is configured to display the generated simulated CT image; the receiving unit is configured to receive the frontal and lateral X-ray images of the simulated CT image input by a user; and the transmission unit is configured to transmit the frontal and lateral X-ray images received by the receiving unit to the image variational autoencoder training unit 3, and return the results processed by the diffusion model from the frontal and lateral X-ray images to CT images to the display unit for display.

[0081] It can be understood that, since the method embodiment and the device embodiment are different presentation forms of the same technical concept, the content of the method embodiment part in the present application should be adapted to the device embodiment part at the same time, which will not be described here again.

[0082] In a third aspect, the embodiments of the present application provide an electronic device comprising a processor and a memory storing computer program instructions.

[0083] The processor implements the CT image generation method based on the fronto-lateral X-ray image according to any one of the first aspect of the embodiments of the present application when executing the computer program instructions.

[0084] In a fourth aspect, the embodiments of the present application provide a computer readable storage medium, and the computer readable storage medium stores computer program instructions. The computer program instructions are executed by a processor to implement the CT image generation method based on the fronto-lateral X-ray image according to any one of the first aspect of the embodiments of the present application.

[0085] In the implementation process, each step of the above method can be completed by integrated logic circuits of hardware in the processor or instructions in the form of software. The steps of the method disclosed in the embodiments of the present application can be directly embodied as execution completed by a hardware processor, or executed by a combination of hardware and software modules in the processor. The software module can be located in a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, a register, or other mature storage media in the art. The storage medium is located in the memory, and the processor reads information in the memory and combines the hardware to complete the steps of the above method. To avoid repetition, it will not be described in detail here.

[0086] In various embodiments of the present application, the size of the serial number of each process described above does not mean the order of execution. The execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0087] Those of ordinary skill in the art can realize that the various illustrative logical blocks (ILB) and steps described in connection with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether the functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0088] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented by other manners. For example, the above-described device embodiments are merely illustrative, for example, the division of the units is merely a logical function division, and in actual implementation, another division manner can be adopted, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0089] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place or can be distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0090] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit.

[0091] In the above embodiments, all or part can be realized by software, hardware, firmware or any combination thereof. When realized by software, all or part can be realized in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instructions can be stored in a computer readable storage medium or transferred from one computer readable storage medium to another, for example, the computer instructions can be transferred from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line) or wireless (such as infrared, wireless, microwave, etc.) manner. The computer readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. integrated with one or more available media. The available media can be magnetic media (such as floppy disk, hard disk, magnetic tape), optical media (such as DVD), or semiconductor media (such as solid state disk) and the like.

[0092] The above merely describes a specific implementation of the present application, but the protection scope of the present application is not limited thereto, any person skilled in the art can easily think of changes or replacements within the technical scope disclosed by the present application, which should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

[0093] To sum up, in the CT image generation method and device based on the posteroanterior X-ray image, the equipment and the computer readable storage medium provided by the application, firstly, the paired data set, the unpaired data set and the simulation data set required for model training are constructed, then the simulation CT image generation method based on the constructed data set is trained, and the posteroanterior X-ray image of the simulation CT image is processed by the diffusion model of the posteroanterior X-ray image to CT image, so that a CT image with higher quality can be generated.

[0094] On the other hand, when the diffusion model of the posteroanterior X-ray image to CT image is trained, the L2 loss function and the perception loss function are introduced, which effectively improves the quality and authenticity of the simulation CT image generated by the model, and has practicality.

[0095] The above description is merely a description of the preferred embodiments of the present application, and does not limit the scope of the present application in any way, and any changes or modifications made by a person skilled in the art according to the above disclosure are within the protection scope of the claims.

Claims

1. A CT image generation method based on postero-lateral X-ray images, characterized by, Comprising: S1, collecting a plurality of head's frontal and lateral X-ray images and CT images; wherein the frontal and lateral X-ray images include: frontal X-ray images and lateral X-ray images; S2, constructing a paired data set, an unpaired data set, and a simulated data set; wherein the frontal and lateral X-ray images in the paired data set are collected from the same head, and the frontal and lateral X-ray images in the unpaired data set are collected from different heads; the simulated frontal and lateral X-ray images in the simulated data set are obtained by processing the CT images in the paired data set and the unpaired data set based on a DDR algorithm, and the simulated CT images in the simulated data set are the CT images in the paired data set and the unpaired data set; S3, training based on the frontal and lateral X-ray images in the paired data set and the unpaired data set and the simulated frontal and lateral X-ray images to obtain a style transfer diffusion model; S4, training based on the frontal and lateral X-ray images in the paired data set and the unpaired data set processed by the style transfer diffusion model and their corresponding CT images to obtain a diffusion model from frontal and lateral X-ray images to CT images, and the following loss functions are used for training: L2 loss function between head CT images in the paired data set and simulated head CT images; Perceptual loss function between head CT images in the unpaired data set and simulated head CT images; S5, sequentially processing the frontal and lateral X-ray images of the to-be-generated simulated CT images by the style transfer diffusion model and the diffusion model from frontal and lateral X-ray images to CT images to generate corresponding simulated CT images.

2. The CT image generation method based on postero-lateral X-ray images according to claim 1, characterized in that, Further comprising: Based on the training of the paired data set and the unpaired data set, a frontal X-ray image variational autoencoder and a lateral X-ray image variational autoencoder are obtained; And When performing S5, the frontal and lateral X-ray images of the to-be-generated simulated CT images are sent to the frontal X-ray image variational autoencoder and the lateral X-ray image variational autoencoder for processing, and the results are sequentially processed by the style transfer diffusion model and the diffusion model from frontal and lateral X-ray images to CT images to generate corresponding simulated CT images.

3. The CT image generation method based on the postero-lateral X-ray image according to claim 2, characterized by, Further comprising: Based on the training of the paired data set, the unpaired data set, and the simulated data set, a CT image variational autoencoder is obtained; And The following loss functions are used for training in S4: L2 loss function between head CT images in the paired data set processed by the CT image variational autoencoder and simulated head CT images; Perceptual loss function between head CT images in the unpaired data set processed by the CT image variational autoencoder and simulated head CT images.

4. The CT image generation method based on frontal and lateral X-ray images according to claim 3, wherein each variational autoencoder performs feature extraction and data dimension reduction on the corresponding images. Comprising:

5. A CT image generation apparatus based on postero-lateral X-ray images, characterized by, An image acquisition unit for acquiring a plurality of head's frontal and lateral X-ray images and CT images; wherein the frontal and lateral X-ray images include: frontal X-ray images and lateral X-ray images; ​ The data set construction unit is configured to construct a paired data set, an unpaired data set, and a simulation data set; the paired data set is constructed by using the same head to collect the frontal and lateral X-ray images and the CT image; the unpaired data set is constructed by using different heads to collect the frontal and lateral X-ray images and the CT image; the simulation data set is constructed by using the DDR algorithm to process the CT images in the paired data set and the unpaired data set to obtain simulation frontal and lateral X-ray images and simulation CT images; The style transfer diffusion model training unit is configured to train the style transfer diffusion model based on the frontal and lateral X-ray images in the paired data set and the unpaired data set and the simulation frontal and lateral X-ray images. The frontal and lateral X-ray image to CT image diffusion model training unit is configured to train the frontal and lateral X-ray image to CT image diffusion model based on the paired data set and the unpaired data set processed by the style transfer diffusion model training unit, and the training is performed by using the following loss functions: L2 loss function between the head CT image in the paired data set and the simulation head CT image; Perceptual loss function between the head CT image in the unpaired data set and the simulation head CT image; The simulation CT image generation unit is configured to receive the frontal and lateral X-ray image to be used to generate the simulation CT image, and sequentially process the frontal and lateral X-ray image by using the style transfer diffusion model training unit and the frontal and lateral X-ray image to CT image diffusion model training unit to generate the corresponding simulation CT image.

6. The CT image generation apparatus based on the postero-lateral X-ray image according to Claim 5, wherein The image variational autoencoder training unit is configured to train the frontal X-ray image variational autoencoder, the lateral X-ray image variational autoencoder, and the CT image variational autoencoder based on the paired data set, the unpaired data set, and the simulation data set.

7. The CT image generation apparatus based on the postero-lateral X-ray image according to Claim 6, wherein The simulation CT image generation unit includes: The display unit is configured to display the generated simulation CT image. The receiving unit is configured to receive the frontal and lateral X-ray image to be used to generate the simulation CT image input by a user. The transmission unit is configured to transmit the frontal and lateral X-ray image received by the receiving unit to the image variational autoencoder training unit, and return the result processed by the frontal and lateral X-ray image to CT image diffusion model training unit to the display unit for display.

8. An electronic device, comprising: The electronic device includes a processor and a memory storing computer program instructions; The processor executes the computer program instructions to implement the CT image generation method based on the frontal and lateral X-ray image according to any one of claims 1-4.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer program instructions, and the computer program instructions are executed by the processor to implement the CT image generation method based on the frontal and lateral X-ray image according to any one of claims 1-4.

Images