CT thin layer image reconstruction method, device and equipment based on diffusion model

Abstract

The application provides a CT thin layer image reconstruction method and device based on a diffusion model, equipment and a computer readable storage medium. The CT thin layer image reconstruction method based on the diffusion model comprises the following steps: performing feature extraction on a random noise image to obtain first feature information; performing feature extraction on a CT thick layer image to obtain second feature information; fusing the first feature information and the second feature information to obtain feature fusion information; and performing feature extraction on the feature fusion information to obtain a CT thin layer image. According to the embodiment of the application, the image reconstruction can be more accurate.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of deep learning intelligent identification, and particularly relates to a CT thin layer image reconstruction method and device based on a diffusion model, equipment and a computer readable storage medium. BACKGROUND

[0002] CT images are an effective auxiliary tool for doctors in recent years. CT images with high clarity and resolution play a very good boosting role for both doctors' diagnosis and AI algorithm lesion segmentation. However, if a CT image with high clarity is to be obtained, it means that the patient's time in the CT room increases when the image is taken, that is, the radiation received increases, and that the computer space required for the subsequent storage of the CT image increases. Because many hospitals are reluctant to store CT images with high clarity (thin layer).

[0003] The existing thick layer CT image reconstruction method using deep learning mainly uses a supervised learning method. The main problem of this method is that it needs to find corresponding paired CT images for training, that is, it needs to find 2 sets of corresponding images of a patient, one set of thick layer images and one set of thin layer images. Then the thick layer images are taken as training images and the thin layer images are taken as labels for training.

[0004] The problem of this method is that it is difficult to find a large amount of paired data. Therefore, in actual operation, either the corresponding training data cannot be found and has to be abandoned, or the training data is too little, resulting in poor final effect.

[0005] Therefore, how to more accurately perform image reconstruction is a technical problem that those skilled in the art need to solve. SUMMARY

[0006] The application embodiment provides a CT thin layer image reconstruction method and device based on a diffusion model, equipment and a computer readable storage medium, which can more accurately perform image reconstruction.

[0007] In a first aspect, the application embodiment provides a CT thin layer image reconstruction method based on a diffusion model, comprising:

[0008] performing feature extraction on the random noise image to obtain first feature information;

[0009] performing feature extraction on the CT thick layer image to obtain second feature information;

[0010] fusing the first feature information and the second feature information to obtain feature fusion information;

[0011] performing feature extraction on the feature fusion information to obtain a CT thin layer image.

[0012] Optionally, the obtaining of the random noise image comprises:

[0013] adding noise to the original image multiple times to obtain the random noise image;

[0014] wherein the multiple times of added noise are all subject to Gaussian distribution.

[0015] Optionally, the adding of noise to the original image multiple times to obtain the random noise image comprises:

[0016] adjusting the size of the original image after adding noise each time;

[0017] normalizing and synthesizing multiple images of which the size has been adjusted to obtain the random noise image.

[0018] Optionally, the feature extraction of the random noise image to obtain the first feature information comprises:

[0019] extracting features of the random noise image by semantic segmentation coding to obtain the first feature information.

[0020] Optionally, the feature extraction of the CT thick layer image to obtain the second feature information comprises:

[0021] extracting features of the CT thick layer image by conditional coding to obtain the second feature information.

[0022] Optionally, the fusing of the first feature information and the second feature information to obtain the feature fusion information comprises:

[0023] fusing the first feature information and the second feature information by multi-modal image fusion to obtain the feature fusion information.

[0024] Optionally, the feature extraction of the feature fusion information to obtain the CT thin layer image comprises:

[0025] extracting features of the feature fusion information by semantic segmentation decoding to obtain the CT thin layer image.

[0026] In a second aspect, an embodiment of the present application provides a CT thin layer image reconstruction device based on a diffusion model, comprising:

[0027] a noise image feature extraction module configured to extract features of the random noise image to obtain the first feature information;

[0028] a thick layer image feature extraction module configured to extract features of the CT thick layer image to obtain the second feature information;

[0029] The feature information fusion module is configured to fuse the first feature information and the second feature information to obtain feature fusion information.

[0030] The thin layer image acquisition module is configured to perform feature extraction on the feature fusion information to obtain a CT thin layer image.

[0031] In a third aspect, an electronic device is provided. The electronic device includes a processor and a memory storing computer program instructions.

[0032] The processor executes the computer program instructions to implement the CT thin layer image reconstruction method based on a diffusion model.

[0033] In a fourth aspect, a computer readable storage medium is provided. The computer readable storage medium stores computer program instructions. The computer program instructions are executed by a processor to implement the CT thin layer image reconstruction method based on a diffusion model.

[0034] The CT thin layer image reconstruction method based on a diffusion model, the device, the electronic device and the computer readable storage medium can more accurately perform image reconstruction.

[0035] The CT thin layer image reconstruction method based on a diffusion model includes: performing feature extraction on a random noise image to obtain first feature information; performing feature extraction on a CT thick layer image to obtain second feature information; fusing the first feature information and the second feature information to obtain feature fusion information; and performing feature extraction on the feature fusion information to obtain a CT thin layer image. BRIEF DESCRIPTION OF DRAWINGS

[0036] In order to more clearly illustrate the specific embodiments of the present application or the technical solutions in the prior art, the following will briefly introduce the drawings needed to be used in the specific embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present application. Those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0037] Figure 1 is a flowchart of the CT thin layer image reconstruction method based on a diffusion model provided by an embodiment of the present application;

[0038] Figure 2 is a flowchart of the CT thin layer image reconstruction method based on a diffusion model provided by an embodiment of the present application;

[0039] Figure 3 is a structural diagram of the CT thin layer image reconstruction device based on a diffusion model provided by an embodiment of the present application;

[0040] Figure 4 is a structural schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0041] Features and exemplary embodiments of various aspects of the present application will be described below in detail, in order to make the purposes, technical solutions and advantages of the present application more clear and apparent, the present application will be further described in detail below in combination with the drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, but not to limit the present application. For those skilled in the art, the present application can be implemented without some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by showing examples of the present application.

[0042] It should be noted that, in this paper, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. Without more limitations, the elements defined by the statement "include" do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0043] In order to solve the problems in the prior art, the embodiments of the present application provide a CT thin layer image reconstruction method based on a diffusion model, a device, an equipment and a computer readable storage medium. First, the CT thin layer image reconstruction method based on a diffusion model provided by the embodiments of the present application is introduced.

[0044] Figure 1 The flowchart of the CT thin layer image reconstruction method based on a diffusion model provided by an embodiment of the present application is shown. As shown in Figure 1 The CT thin layer image reconstruction method based on a diffusion model includes:

[0045] S101, feature extraction is performed on a random noise image to obtain first feature information;

[0046] S102, feature extraction is performed on a CT thick layer image to obtain second feature information;

[0047] S103, the first feature information and the second feature information are fused to obtain feature fusion information;

[0048] S104, feature extraction is performed on the feature fusion information to obtain a CT thin layer image.

[0049] In one embodiment, the acquisition of the random noise image comprises:

[0050] The noise is added to the original image multiple times to obtain the random noise image.

[0051] The multiple added noises are all subject to Gaussian distribution.

[0052] In one embodiment, the multiple addition of noise to the original image to obtain the random noise image comprises:

[0053] After adding noise to the original image each time, the size is adjusted.

[0054] The multiple size-adjusted images are normalized and synthesized to obtain the random noise image.

[0055] Specifically, the diffusion model is a model for image generation. It is mainly used for image generation in the field of multi-modal. Its main theoretical basis has two points:

[0056] a. Markov Chain: In the time sequence relationship and strategy, the behavior of the n+1 step is only related to the n step, and has nothing to do with other steps.

[0057] b. Bayes rule: P(A, B, C) = P(A)P(B, C|A) = P(AB)P(C|AB).

[0058] The main theoretical assumption is that all the noise added in the training process is subject to Gaussian distribution. The training process of the entire diffusion model, the original image is converted through the Forward process T times, and each conversion adds noise subject to Gaussian distribution to the image. After T times of conversion, the image becomes a completely random noise image.

[0059] In one embodiment, the feature extraction is performed on the random noise image to obtain first feature information, comprising:

[0060] The feature extraction is performed on the random noise image by a semantic segmentation coding manner to obtain the first feature information.

[0061] In one embodiment, the feature extraction is performed on the CT thick layer image to obtain second feature information, comprising:

[0062] The feature extraction is performed on the CT thick layer image by a conditional coding manner to obtain the second feature information.

[0063] In one embodiment, the first feature information and the second feature information are fused to obtain feature fusion information, including:

[0064] The first feature information and the second feature information are fused by a multi-modal image fusion manner to obtain feature fusion information.

[0065] In one embodiment, the feature fusion information is feature extracted to obtain a CT thin layer image, including:

[0066] The feature fusion information is feature extracted by a semantic segmentation decoding manner to obtain a CT thin layer image.

[0067] Specifically, the diffusion model for image generation is first applied to the task of CT high-definition image reconstruction. The specific method flow is as shown in the figure. Figure 2 The entire structure adds a conditional branch based on the diffusion model, that is, three 3mm CT images, so as to form supervision for the extended model, and finally generate three 1mm CT images.

[0068] The input Xt of the entire model is random noise data, the feature information is extracted by the Segmentation Encoder, the Condition Encoder is used as a feature extractor to extract the information of three 3mm thick CT images, the features extracted by the Segmentation Encoder and the features extracted by the Condition Encoder are fused by addition, and then the Segmentation Decoder is used for feature extraction, and finally three 1mm thick CT images are generated.

[0069] Figure 3 It is a structure diagram of a CT thin layer image reconstruction device based on a diffusion model provided by one embodiment of the application. The CT thin layer image reconstruction device based on the diffusion model includes:

[0070] The noise image feature extraction module 301 is configured to extract features of a random noise image to obtain first feature information.

[0071] The thick layer image feature extraction module 302 is configured to extract features of a CT thick layer image to obtain second feature information.

[0072] The feature information fusion module 303 is configured to fuse the first feature information and the second feature information to obtain feature fusion information.

[0073] The thin layer image acquisition module 304 is configured to extract features of the feature fusion information to obtain a CT thin layer image.

[0074] Figure 4A structural schematic diagram of an electronic device provided by an embodiment of the present application is shown.

[0075] The electronic device can include a processor 401 and a memory 402 storing computer program instructions.

[0076] Specifically, the processor 401 described above can include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or can be configured as one or more integrated circuits that implement the embodiments of the present application.

[0077] The memory 402 can include a mass storage for data or instructions. By way of example and not limitation, the memory 402 can include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. Where appropriate, the memory 402 can include removable or non-removable (or fixed) media. Where appropriate, the memory 402 can be internal or external to the electronic device. In certain embodiments, the memory 402 can be a non-volatile solid-state memory.

[0078] In one embodiment, the memory 402 can be a read-only memory (ROM). In one embodiment, the ROM can be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically alterable ROM (EAROM), or a flash memory, or a combination of two or more of these.

[0079] The processor 401 reads and executes the computer program instructions stored in the memory 402 to implement any of the CT thin layer image reconstruction methods based on the diffusion model described above.

[0080] In one example, the electronic device can further include a communication interface 403 and a bus 410. Wherein, as shown, the processor 401, the memory 402, and the communication interface 403 are connected through the bus 410 and complete communication among each other. Figure 4

[0081] The communication interface 403 is mainly used to realize the communication between the modules, devices, units and / or equipment in the embodiments of the present application.

[0082] ​Bus 410 includes a hardware, software, or both that couples components of electronic device to each other. As an example and not by way of limitation, bus can include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand (IB) interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or another suitable bus or a combination of two or more of these. Where appropriate, bus 410 can include one or more buses. Although this application describes and shows a particular bus, this application contemplates any suitable bus or interconnect.

[0083] In addition, in combination with the CT thin layer image reconstruction method based on the diffusion model in the above embodiments, the embodiments of the present application can provide a computer readable storage medium to implement. The computer readable storage medium has computer program instructions stored thereon; the computer program instructions are executed by the processor to implement any one of the CT thin layer image reconstruction methods based on the diffusion model in the above embodiments.

[0084] It needs to be clear that the present application is not limited to the specific configurations and processes described above and shown in the drawings. For the sake of brevity, detailed descriptions of well-known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between steps, after understanding the spirit of the present application.

[0085] The functional modules shown in the structure block diagram described above can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a functional card, etc. When implemented in software, the elements of the present application are program or code segment used to perform the required tasks. The program or code segment can be stored in a machine readable medium or transmitted through a data signal carried in a carrier wave on a transmission medium or communication link. The "machine readable medium" can include any medium capable of storing or transmitting information. Examples of machine readable medium include electronic circuit, semiconductor memory device, ROM, flash memory, erasable ROM (EROM), floppy disk, CD-ROM, optical disk, hard disk, optical fiber medium, radio frequency (RF) link, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0086] It should also be noted that the example embodiments mentioned in the present application describe some methods or systems based on a series of steps or devices. However, the present application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiments, or in an order different from the embodiments, or several steps can be performed simultaneously.

[0087] The above generally describes aspects of the present application with reference to a flowchart and / or a block diagram of the method, the apparatus (system) and computer program product according to embodiments of the present application. It should be understood that each block of the flowchart and / or the block diagram, as well as combinations of blocks in the flowchart and / or the block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing apparatus implement the functions / acts specified in the flowchart and / or the block diagram block or blocks. The processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field programmable logic circuit. It should also be understood that each block of the flowchart and / or the block diagram, as well as combinations of blocks in the flowchart and / or the block diagram, can also be implemented by dedicated hardware, or a combination of computer instructions and dedicated hardware.

[0088] The above is only a specific implementation of the present application, and those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described system, module and unit can refer to the corresponding process in the foregoing method embodiments, which will not be described here. It should be understood that the protection scope of the present application is not limited to this, and any skilled person in the art can easily think of various equivalent modifications or replacements within the technical range disclosed in the present application, and these modifications or replacements should be covered within the protection scope of the present application.

Claims

1. A method for reconstructing thin-slice CT images based on a diffusion model, characterized in that, include: Feature extraction is performed on the random noise image to obtain the first feature information; Feature extraction was performed on three 3mm thick CT slice images to obtain the second feature information; The three 3mm CT images are used to form conditional branches to supervise the diffusion model; The first feature information and the second feature information are fused to obtain feature fusion information; Feature extraction was performed on the feature fusion information to obtain three 1mm thin-slice CT images; The acquisition of random noise images includes: adding noise to the original image multiple times to obtain random noise images; wherein the noise added multiple times all follows a Gaussian distribution; Adding noise to the original image multiple times to obtain a random noise image includes: adjusting the size of the original image after each addition of noise; normalizing and synthesizing multiple resized images to obtain a random noise image; Extracting features from a random noise image to obtain first feature information includes: extracting features from a random noise image through semantic segmentation coding to obtain first feature information; Feature extraction is performed on CT thick-slice images to obtain second feature information, including: feature extraction is performed on CT thick-slice images through conditional coding to obtain second feature information; The first feature information and the second feature information are fused to obtain feature fusion information, including: fusing the first feature information and the second feature information through multimodal image fusion to obtain feature fusion information; The process of extracting features from the feature fusion information to obtain a CT thin-slice image includes: extracting features from the feature fusion information through semantic segmentation decoding to obtain a CT thin-slice image.

2. A CT thin-slice image reconstruction device based on a diffusion model, characterized in that, The device includes: The noise image feature extraction module is used to extract features from random noise images to obtain the first feature information; The thick-slice image feature extraction module is used to extract features from three 3mm thick-slice CT images to obtain second feature information; the three 3mm CT images are used to form conditional branches to supervise the diffusion model. The feature information fusion module is used to fuse the first feature information and the second feature information to obtain feature fusion information; The thin-slice image acquisition module is used to extract features from the feature fusion information to obtain three 1mm CT thin-slice images; The acquisition of random noise images includes: adding noise to the original image multiple times to obtain random noise images; wherein the noise added multiple times all follows a Gaussian distribution; Adding noise to the original image multiple times to obtain a random noise image includes: adjusting the size of the original image after each addition of noise; normalizing and synthesizing multiple resized images to obtain a random noise image; Extracting features from a random noise image to obtain first feature information includes: extracting features from a random noise image through semantic segmentation coding to obtain first feature information; Feature extraction is performed on CT thick-slice images to obtain second feature information, including: feature extraction is performed on CT thick-slice images through conditional coding to obtain second feature information; The first feature information and the second feature information are fused to obtain feature fusion information, including: fusing the first feature information and the second feature information through multimodal image fusion to obtain feature fusion information; The process of extracting features from the feature fusion information to obtain a CT thin-slice image includes: extracting features from the feature fusion information through semantic segmentation decoding to obtain a CT thin-slice image.

3. An electronic device, characterized in that, The electronic device includes: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, it implements the CT thin-slice image reconstruction method based on the diffusion model as described in claim 1.

4. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement the CT thin-slice image reconstruction method based on the diffusion model as described in claim 1.

Images