Angiography artifact removal method and device based on conditional diffusion and storage medium
Through the source conditional restriction, forward noise training, style-oriented guidance and reverse noise-denoising training of the conditional diffusion model, the problem of information loss caused by striped artifact removal in OCTA technology is solved, and efficient blood flow without pairing the data set is achieved Information recovery is better than existing methods.
Patent Information
- Application Number
- CN202410972258.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-19
- Publication Date
- 2024-11-12
AI Technical Summary
The existing OCTA angiography technology can easily lead to information loss when removing striped artifacts, and requires a high-cost paired data set, which cannot effectively restore blood flow reflex information in abnormal B-scans.
Using a conditional diffusion-based method, through source conditional restriction, forward noise training, style directional guidance and reverse denoising training, the conditional diffusion model is used to restore abnormal B-scans to normal B-scans to maximize the blood flow information .
Without pairing datasets, it is able to effectively remove stripes and maximize blood flow information, which is better than existing methods, especially when dealing with bold and low transparency stripes.
Smart Images

Figure CN118941664A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the field of angiography artifact removal, and in particular to angiography artifact removal method, device and storage medium based on conditional diffusion. Background Art
[0002] Optical coherence tomography (OCT) angiography (OCTA) is a non-invasive imaging technique that produces high-resolution three-dimensional images by measuring the coherence of light beam reflections for observing the microvascular structures of the retina and choroid. OCTA visualizes microvasculature by detecting the motion contrast of flowing blood. Specifically, OCTA performs multiple repetitive scans at the same tissue location to obtain structural B-scans and then moves to the next location, creating 3D data in the process. The enface image of OCTA can be obtained by signal summation or projection of the 3D data. By comparing the signal changes between adjacent B-scans, it can detect the movement of red blood cells within the blood vessels. However, in addition to the movement of blood cells, differences between B-scans can be caused by pulsations associated with the heartbeat cycle, breathing, and trembling. This motion appears as periodic streaking noise in the axial or longitudinal direction of the entire image, also known as motion artifact (BMA).
[0003] At present, most methods for removing OCTA stripes focus on enface image processing, which we refer to as enface-based methods in this paper. These methods regard the OCTA artifact removal problem as a learning-based image filling problem, and they have made effective progress due to the convenience of end-to-end. However, their limitations are that they have a high demand for paired datasets and are very expensive. The use of deep learning methods for OCTA stripe removal has been proven. This type of work removes the pixel columns corresponding to the stripes and uses the stripe mask data to train the image complementation model, thereby achieving pixel complementation at the stripes. However, these methods do not consider the actual reasons for the generation of stripes, which can easily lead to information loss in the image restoration process. Fundamentally speaking, simply removing stripes cannot meet actual needs. Abnormal B-scans also provide key tissue structure information and blood flow signals. Repairing the original vascular information masked by stripes is the ultimate goal.
[0004] Therefore, in order to restore the blood flow reflection information in abnormal B-scans, developing a method that can learn the style of normal B-scans around abnormal B-scans to restore the normal blood flow grayscale distribution has become an urgent problem to be solved in current research. Summary of the invention
[0005] The main purpose of the present invention is to overcome the above-mentioned defects in the prior art and propose a method, device and storage medium for removing angiography artifacts based on conditional diffusion, so as to restore abnormal B-scans to normal B-scans and retain OCTA blood flow information to the maximum extent.
[0006] The present invention adopts the following technical solution:
[0007] The method for removing artifacts from angiography based on conditional diffusion is characterized in that:
[0008] Source condition restriction: the B-scans image data with signal loss and the adjacent normal B-scans image data are input into the conditional diffusion model to predict pure noise;
[0009] Forward noise training: the conditional diffusion model gradually adds random Gaussian noise to the input image data, so that the model can learn the distribution of image data under different noise levels to obtain a pure noise data result;
[0010] Style-oriented guidance, the conditional diffusion model guides the reverse denoising process by calculating the Gram matrix loss of the internal latent space of the adjacent normal B-scans image data and the signal-missing B-scans image data;
[0011] In the reverse denoising training, the conditional diffusion model gradually restores the original blood flow signal characteristics of the signal-missing B-scans image according to the directional guidance for the pure noise data results, and reconstructs the enface image in 3D to obtain the OCTA enface image with stripes removed.
[0012] In the source condition restriction, the signal-missing B-scans image data is paired with the nearest normal B-scans image data to create a matching data set, and the data set is input into the conditional diffusion model to learn the grayscale difference between the two types of B-scans images, and the grayscale distribution loss is introduced into the target function to predict pure noise.
[0013] In the forward noise training, Gaussian noise is added to the input data image under given conditions to generate noisy data, and the final pure noise is obtained by gradually adding Gaussian noise multiple times.
[0014] In the style-oriented guidance, the conditional diffusion model extracts semantic features from the B-scans image data with signal loss and the adjacent normal B-scans image data respectively, and then calculates the Gram matrix loss of the internal latent space of the semantic features of the two, thereby achieving the effect of conditional style guidance.
[0015] The Gram matrix loss of the internal latent space of the semantic features of the two is calculated. Specifically, the reference image and the predicted image in the process will pass through a pre-trained VGG network (Ref.). The VGG network outputs the feature map corresponding to each layer. The Gram loss is calculated between the two feature maps, and the generation process of the conditional diffusion model is introduced in the form of gradient to achieve conditional guidance.
[0016] In the reverse denoising training, the conditional diffusion model is able to generate normal B-scans images by gradually removing noise, and the B-scans images with stripes and signal loss are replaced with the normal B-scans after directional guidance, and then 3D reconstruction is performed to obtain the OCTA enface image with stripes removed.
[0017] A device comprises: a processor and a memory, wherein the memory stores program instructions for implementing an artifact removal method for angiography based on conditional diffusion, and when the program instructions are executed by the processor, the device implements the method.
[0018] A storage medium stores computer-readable instructions for implementing an artifact removal method for angiography based on conditional diffusion. When the computer-readable instructions are executed by one or more processors, the method is implemented.
[0019] It can be seen from the above description of the present invention that, compared with the prior art, the present invention has the following beneficial effects:
[0020] The method of the present invention is implemented using a directional guided diffusion model framework, including source condition restriction, forward noise training, style directional guidance and reverse denoising training; by making a matching data set from abnormal B-scans and adjacent normal B-scans, the stripes of abnormal B-scans are removed in combination with the diffusion model. During the sampling process, normal B-scans are generated from pure noise data using directional style guidance to replace abnormal B-scans, the stripes are removed from the root cause of the stripes, and a 3D enface image is reconstructed, which retains the blood flow information to the maximum extent.
[0021] The method of the present invention does not require the collection of difficult-to-obtain paired OCTA images as training sets, nor does it require manual stripe creation. Both quantitative and qualitative results show that the present invention outperforms many state-of-the-art existing methods on downstream tasks, especially when dealing with bold, low-transparency stripes, which has practical significance in image analysis and medical diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] Figure 1 It is a composition diagram of the conditional diffusion model of the present invention;
[0023] Figure 2 is a schematic diagram of the stripe removal effect of the B-scans image of the present invention compared with other methods;
[0024] Figure 3 is a schematic diagram of the OCTA enface image stripe removal effect of the present invention compared with other methods;
[0025] Figure 4 It is a schematic diagram of the downstream task of OCTA enface image stripe removal of the present invention compared with other methods.
[0026] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. DETAILED DESCRIPTION
[0027] The present invention is further described below through specific implementation modes.
[0028] The present invention provides a novel framework for optical coherence tomography angiography (OCTA) artifact removal, which belongs to the field of artificial intelligence-assisted medical diagnosis. OCTA is a non-invasive blood flow detection technology that recognizes vascular regions by detecting the signal of blood flow motion contrast. The method of the present invention uses the signal recovery of B-scans as the entry point, the structural information of adjacent B-scans and the style information of a given reference image as the guiding conditions, and restores the original blood flow structure and motion contrast in B-scans with missing signals based on a conditional diffusion model.
[0029] The angiography artifact removal method based on conditional diffusion of the present invention is implemented based on the conditional diffusion model, see Figure 1 The conditional diffusion model includes several parts, including the forward process, the reverse process, the source domain guidance and the style guidance. The forward process is the model training process. At this stage, the task of the model is to predict the noise at each step, and the objective function is the mean square error between the prediction result and the Gaussian distribution output. The reverse process is to output the model prediction result and gradually remove the predicted noise. The source domain guidance refers to the concat result of the B-scan and the image to be converted near the normal structure as input to introduce the source domain structure information. The style guidance is to give a reference style image, calculate two Gram matrix losses and introduce style feature information in the form of gradients. The training process is the forward process, the feature space is the reverse style guidance process, and the semantic extractor is the Pre-trained VGG network.
[0030] The method of the present invention specifically comprises the following steps:
[0031] 1) Source condition restriction: B-scans image data with missing signals and adjacent normal B-scans image data are input into the conditional diffusion model to predict pure noise, so that the model can learn the data distribution characteristics of two different styles of images. Among them, B-scans image data with missing signals are abnormal B-scans images.
[0032] Among them, the signal-missing B-scans image data are paired with the nearest normal B-scans image data to create a matching data set, the data set is input into the conditional diffusion model to learn the grayscale difference between the two types of B-scans images, and the grayscale distribution loss is introduced into the target function to predict pure noise.
[0033] 2) Forward noise training: The conditional diffusion model gradually adds random Gaussian noise to the input image data, so that the model can learn the distribution of image data under different noise levels to obtain a pure noise data result.
[0034] The forward noise training process is equivalent to a Markov chain, which is to add Gaussian noise to the input data image under given conditions to generate noisy data, and then gradually add Gaussian noise multiple times to obtain the final pure noise.
[0035] 3) Style-oriented guidance, the conditional diffusion model guides the reverse denoising process by calculating the Gram matrix loss of the internal latent space of the adjacent normal B-scans image data and the signal-missing B-scans image data.
[0036] In style-oriented guidance, the conditional diffusion model extracts semantic features from the B-scans image data with signal loss and the adjacent normal B-scans image data respectively, and then calculates the Gram matrix loss of the internal latent space of the semantic features of the two, thereby achieving the effect of conditional style guidance.
[0037] Among them, for the Gram matrix loss of the internal latent space of the semantic features of both, first, the reference image and the predicted image in the process will pass through a pre-trained VGG network (Ref.), the VGG network outputs the feature map corresponding to each layer, and the Gram loss is calculated between the two feature maps. The generation process of the conditional diffusion model is introduced in the form of gradient to realize conditional guidance.
[0038] 4) Reverse denoising training: the conditional diffusion model gradually restores the original blood flow signal characteristics of the B-scans image with missing signals according to the directional guidance for the pure noise data results, and reconstructs the enface image in 3D to obtain the OCTA enface image with stripes removed.
[0039] In the reverse denoising training, the conditional diffusion model is able to generate normal B-scans images by gradually removing noise, replacing the B-scans images with stripes and signal loss with normal B-scans after directional guidance, and then performing 3D reconstruction to obtain the OCTAenface image with stripes removed.
[0040] The method of the present invention uses the structural information of adjacent normal B-scans image data and the style features of designated B-scans as guidance, generates B-scans after blood vessel restoration through a conditional diffusion model, and restores the enface image through a 3D reconstruction technique.
[0041] Based on this, the present invention also proposes a device, including a processor and a memory, wherein the memory stores program instructions for implementing an artifact removal method for angiography based on conditional diffusion. When the program instructions are executed by the processor, the device implements the above-mentioned artifact removal method for angiography based on conditional diffusion.
[0042] A storage medium stores computer-readable instructions for implementing an artifact removal method for angiography based on conditional diffusion. When the computer-readable instructions are executed by one or more processors, the above-mentioned artifact removal method for angiography based on conditional diffusion is implemented.
[0043] Reference Figure 1 The present invention first uses the structure of adjacent normal B-scans frames as a guide, and uses abnormal B-scans as source condition constraints, and puts them into the conditional diffusion model for training to obtain pure noise data results. Then, based on the style guidance of the reference frame, the Gram matrix loss of the internal latent space of normal B-scans and abnormal B-scans in the matching data set is calculated as a directional style guide to guide the sampling process, that is, gradually denoising the pure noise data to finally generate normal B-scans.
[0044] Reference Figure 2 , the directional guided diffusion method (B2E-LDG) proposed in this paper effectively removes stripes from abnormal B-scans, and is compared with neighboring B-scans replacement method, CycleGAN and other methods. It can be seen that B2E-LDG can learn fine grayscale style features and background noise, and restore correct blood flow information on abnormal B-scans, which is difficult for CycleGAN to do.
[0045] Reference Figure 3, the directional guided diffusion method proposed in the present invention effectively removes stripes from abnormal B-scans. After 3D reconstruction of the enface image, this method is also compared with neighboring B-scans replacement method, MW-FFTA, CycleGAN and other methods, and analyzed in the region of interest of the enface A image. It can be seen that directly replacing the neighboring B-scans will produce a duplicate area as wide as the stripe, which may cause abnormal details of the blood vessel direction. MW-FFTA uses smoothing to propagate stripe noise to other normal areas and is not very robust in being sensitive to stripe features. CycleGAN can learn overall coloring features at the B-scans level, but the details after reconstruction on the enface are not perfect enough. B2E-LDG not only achieves stripe removal, but also effectively restores blood flow information at the stripes.
[0046] Reference Figure 4 ,In order to compare the effects of neighboring B-scans replacement, MW-FFTA, CycleGAN and B2E-LDG on downstream tasks, the results obtained by different methods are subjected to the same downstream task processing. For example, when segmenting blood vessels, stripes significantly interfere with the morphology of blood vessels, and stripes will be mistakenly considered as part of blood vessels. This phenomenon occurs not only on thick stripes, but also on thin stripes. Compared with other algorithms, the method proposed in the present invention can significantly eliminate stripe interference. The results of blood vessel extraction also prove that the algorithm of the present invention can achieve stripe removal without damaging the information of other regions.
[0047] The present invention proposes a new stripe removal framework B2E-LDG for OCTAenface images, which uses feature space intervention to achieve conditional B-scans generation. The feature space of adjacent normal B-scans contains the correct grayscale style and rich vascular morphology and topology information to help convert B-scans. In addition, specifying the style feature space of B-scans can achieve conversion across OCTA devices and datasets. In the present invention, evaluation indicators for stripe removal that better meet practical applications are further proposed, without the need to collect difficult-to-obtain paired OCTA images as training sets, nor the need to manually produce stripes. Both quantitative and qualitative results show that the present invention outperforms many state-of-the-art existing methods on downstream tasks, especially when dealing with bold, low-transparency stripes, which has practical significance in image analysis and medical diagnosis.
[0048] The above is only a specific implementation of the present invention, but the design concept of the present invention is not limited thereto. Any non-substantial changes to the present invention using this concept shall be deemed as an infringement of the protection scope of the present invention.
Claims
1. Angiography artifact removal method based on conditional diffusion, It is characterized in that Source condition restriction: the B-scans image data with signal loss and the adjacent normal B-scans image data are input into the conditional diffusion model to predict pure noise; Forward noise training: the conditional diffusion model gradually adds random Gaussian noise to the input image data, so that the model can learn the distribution of image data under different noise levels to obtain a pure noise data result; Style-oriented guidance, the conditional diffusion model guides the reverse denoising process by calculating the Gram matrix loss of the internal latent space of the adjacent normal B-scans image data and the signal-missing B-scans image data; In the reverse denoising training, the conditional diffusion model gradually restores the original blood flow signal characteristics of the B-scans image with missing signals according to the directional guidance for the pure noise data results, and reconstructs the enface image in 3D to obtain the OCTA enface image with stripes removed.
2. The method for removing artifacts from angiography based on conditional diffusion as claimed in claim 1, It is characterized in that In the source condition restriction, the signal-missing B-scans image data is paired with the nearest normal B-scans image data to create a matching data set, and the data set is input into the conditional diffusion model to learn the grayscale difference between the two types of B-scans images, and the grayscale distribution loss is introduced into the target function to predict pure noise.
3. The method for removing artifacts from angiography based on conditional diffusion as claimed in claim 1, It is characterized in that In the forward noise training, Gaussian noise is added to the input data image under given conditions to generate noisy data, and the final pure noise is obtained by gradually adding Gaussian noise multiple times.
4. The method for removing artifacts from angiography based on conditional diffusion as claimed in claim 1, It is characterized in that In the style-oriented guidance, the conditional diffusion model extracts semantic features from the B-scans image data with signal loss and the adjacent normal B-scans image data respectively, and then calculates the Gram matrix loss of the internal latent space of the semantic features of the two, thereby achieving the effect of conditional style guidance.
5. The method for removing artifacts from angiography based on conditional diffusion as claimed in claim 4, It is characterized in that The Gram matrix loss of the internal latent space of the semantic features of the two is calculated. Specifically, the reference image and the predicted image in the process will pass through a pre-trained VGG network (Ref.). The VGG network outputs the feature map corresponding to each layer. The Gram loss is calculated between the two feature maps, and the generation process of the conditional diffusion model is introduced in the form of gradient to achieve conditional guidance. The method for removing artifacts from angiography based on conditional diffusion as described in claim 1 is characterized in that, in the reverse denoising training, the conditional diffusion model is enabled to generate normal B-scans images by gradually removing noise, the B-scans images with stripes and signal loss are replaced with the normal B-scans after directional guidance, and then 3D reconstruction is performed to obtain an OCTA enface image with stripes removed.
6. A device, include: A processor and a memory, wherein the memory stores program instructions for implementing a method for removing artifacts from angiography based on conditional diffusion, and when the program instructions are executed by the processor, the device implements the method according to any one of claims 1 to 5.
7. A storage medium storing computer-readable instructions for implementing a method for removing artifacts from angiography based on conditional diffusion, wherein when the computer-readable instructions are executed by one or more processors, the method according to any one of claims 1 to 5 is implemented.
