Medical image anomaly detection method based on diffusion model

Abstract

The invention relates to a medical image anomaly detection method based on a diffusion model, and belongs to the field of medical image anomaly detection. The method comprises the following steps: S1, acquiring a medical image which comprises a health sample and an abnormal sample; s2, constructing a Mem-DDPM model, wherein the Mem-DDPM model comprises a diffusion model block and a memory module; s3, training a Mem-DDPM model: training a diffusion model and a memory module at the same time only by using a healthy sample in an unsupervised manner; and S4, in a test, the parameters of the memory module are fixed, a to-be-detected medical image is input to the trained Mem-DDPM model, and abnormal places in the to-be-detected image are screened out. According to the invention, the accuracy of medical image anomaly detection can be improved.

Description

Technical Field

[0001] The invention belongs to the field of medical image anomaly detection and relates to a medical image anomaly detection method based on a diffusion model. Background Art

[0002] With the development of modern medical imaging technologies, such as X-ray, computed tomography (CT), magnetic resonance imaging (MRI), ultrasound imaging, and nuclear medicine imaging, the amount of medical data has increased rapidly, and medical imaging has become an important tool for clinical diagnosis, treatment planning, and efficacy evaluation. Although these technologies provide non-invasive means for early detection of diseases, the large amount of imaging data also brings a huge workload to medical experts. The emergence of anomaly detection technology aims to help medical imaging experts quickly and accurately identify lesions in an automated way and improve work efficiency. Traditional medical image processing methods rely on steps such as image segmentation, feature extraction, and classification. These methods strongly rely on expert experience and manually designed features, and show limitations for abnormal situations in complex images. Considering the privacy and cost of medical images, it is impossible to annotate images on a large scale. Supervised learning relies on a large amount of annotated data. In the field of medical images, annotating abnormal areas or lesions often requires professional medical knowledge. The annotation process is time-consuming, labor-intensive, and expensive. At the same time, supervised methods are usually trained for specific types of abnormalities. In practice, unseen abnormalities or rare diseases are often not in the training data, resulting in the inability of the model to effectively identify them. A common problem in medical imaging is data imbalance, that is, normal samples far outnumber abnormal samples, resulting in poor detection of abnormalities by supervised models. With breakthroughs in artificial intelligence, especially deep learning technology, methods such as convolutional neural networks (CNNs), generative adversarial networks (GANs), and contrastive learning have been introduced into the field of medical image processing. These algorithms no longer rely on manually designed features, but can automatically learn effective feature representations from a large amount of image data, greatly improving the accuracy of anomaly detection.

[0003] Autoencoders (AE) and variational autoencoders (VAE) show stability during training and fast inference speed, but their blurred reconstruction effect limits their effectiveness in unsupervised anomaly detection. AnoGAN is one of the earliest GANs used for anomaly detection. AnoGAN learns the distribution of normal images through a generative adversarial network (GAN), then generates "normal" images similar to the test image through backpropagation, and detects anomalies through pixel and feature differences. The refinement process requires hundreds of backpropagation iterations, resulting in slow inference speed. The DDPM (Denoising Diffusion Model) method introduces noise to the input image, then uses the trained model to remove the noise and reconstruct the original image. Unlike most autoencoder-based methods, DDPM retains spatial information in its hidden representation, which is crucial for the image generation process. In the latest research on medical imaging, DDPM shows scalable and stable training characteristics, and generates high-quality and clear images through classifier guidance. However, the quality of samples generated by the DDPM diffusion model is not stable enough, and there may be problems such as excessive noise or blurred details. At the same time, the performance of anomaly segmentation detection is not satisfactory enough. Therefore, the present invention urgently needs a new anomaly detection method that can be used for medical images, which can not only generate clear and high-quality reconstructed images in the reconstruction to improve the accuracy of anomaly detection, but also deal with the sample imbalance problem encountered when training the model. Summary of the invention

[0004] In view of this, an object of the present invention is to provide a medical image anomaly detection method based on a diffusion model to improve the accuracy of medical image anomaly detection.

[0005] In order to achieve the above object, the present invention provides the following technical solutions:

[0006] A method for detecting abnormalities in medical images based on a diffusion model specifically comprises the following steps:

[0007] S1: Obtain medical images, including healthy samples and abnormal samples;

[0008] S2: Construct the Mem-DDPM model, including the diffusion model and the memory module, where Mem-DDPM represents the diffusion model enhanced by the memory module;

[0009] S3: Training Mem-DDPM model: Train the diffusion model and memory module simultaneously using only healthy samples in an unsupervised manner;

[0010] S4: In the test, the parameters of the memory module are fixed and the medical image to be detected is input into the trained Mem-DDPM model to screen out abnormal areas in the image to be detected.

[0011] Furthermore, in step S2, the diffusion model constructed is: for the input image x, by gradually adding simplex noise at multiple time steps T, a series of noise images {x0, x1, ..., x T}, image x t The noise level t gradually increases from 0 to T.

[0012] Further, in step S2, the diffusion model includes a residual block, and the residual block includes multiple convolutional layers, activation function layers and regularization layers.

[0013] Further, in step S2, the memory module includes a memory unit for storing the prototype encoding pattern, and an attention-based addressing mechanism.

[0014] Further, in step S3, the Mem-DDPM model is trained, specifically including: using only healthy samples to train the diffusion model and memory module at the same time, gradually adding simplex noise to the input x0 to convert it into x T , the model will predict the noise added at time T, denoise it, and generate x after denoising T-1 Repeat T steps to generate the restored and reconstructed image x′0. Calculate the reconstruction loss function through x0 and x′0 and combine it with the loss function of the memory module to train the entire diffusion model and memory module.

[0015] The training objectives of the MemDDPM model are:

[0016]

[0017] in, is the loss function for training the MemDDPM model, and δ is a hyperparameter in training; MSE loss function for training diffusion model is expressed as:

[0018] with∈∈Simplex(v=2 -6 ,N=6,γ=0.8)

[0019] Among them, ∈ θ represents the noise predicted by the model, ∈ represents the real noise; α t =1-β t , β t represents the noise intensity added in time step t, which gradually increases; v represents the initial frequency, which determines the level of detail of the noise; N represents the octave of the noise; γ represents the decay rate, which controls how the amplitude of each frequency component decreases as the frequency increases.

[0020] E(w′ i ) represents minimizing w′i The entropy of is expressed as:

[0021]

[0022] Among them, w′ i Represents sample x i The memory addressing weight.

[0023] Further, in step S3, sample x i The memory addressing weight w′ i The expression is:

[0024]

[0025] Among them, max(.,0) is the ReLU activation function, η is a very small positive scalar, λ is the shrinkage threshold, and w i Represents the i-th element of w; w is a row vector containing non-negative elements whose sum is 1.

[0026] Furthermore, step S4 specifically includes: in the test, the parameters of the fixed memory module remain unchanged, and when predicting the noise for the abnormal sample model, the abnormal area will be gradually removed, so that the restored image is more biased towards the normal image; finally, E is calculated sq =(x′0-x0) 2 , filter out E sq >0.5 is where the anomaly predicted by the Mem-DDPM model occurs, where E sq represents the mean square error, x′0 represents the reconstructed image, and x0 represents the input image.

[0027] The beneficial effects of the present invention are as follows: the present invention can generate clear and high-quality reconstructed images during reconstruction, can deal with the sample imbalance problem encountered when training models, does not require pixel-level annotation of images, reduces the cost of training models, has sufficient generalization capabilities, and improves the accuracy of medical image anomaly detection.

[0028] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0030] Figure 1This is the architecture diagram of the Mem-DDPM proposed in the present invention;

[0031] Figure 2 It is a schematic diagram of some details of the MemDDPM model;

[0032] Figure 3 is a schematic diagram of the structure of the memory module;

[0033] Figure 4 This is a schematic diagram of the effect of using the Mem-DDPM model of the present invention for detection;

[0034] Figure 5 is the impact of time step T on sample anomaly detection performance. DETAILED DESCRIPTION

[0035] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0036] See also Figure 1 to Figure 5 The present invention provides a medical image anomaly detection model (Mem-DDPM) based on a diffusion model, which consists of a diffusion model and a memory module. Mem-DDPM uses only a data set containing healthy samples to simultaneously train the diffusion model and the memory module in an unsupervised manner. In the forward training process, the diffusion model continuously adds simplex noise instead of Gaussian noise to the healthy samples, and learns to predict and remove the previously added noise based on the current image distribution during the denoising process. The training is not targeted at specific anomalies, so it has better generalization. The memory module can remember the typical patterns of healthy samples during training, and can guide the diffusion model to find the best prediction noise that is closer to the real noise during the prediction process, thereby improving the accuracy of the overall model in predicting abnormal areas of medical images.

[0037] like Figure 1 As shown in the figure, Mem-DDPM mainly consists of two parts: diffusion model and memory module. During training, only healthy samples are used to train the diffusion model and memory module at the same time, and the input x0 is gradually added with simplex noise to become x T , the model will predict the noise added at time T, denoise it, and generate x after denoising T-1 Loop T steps to generate the restored and reconstructed image x ′0, through x0 and x ′ 0 Calculate the reconstruction loss function and combine it with the loss function of the memory module to train the entire diffusion model and memory module. In the test, the parameters of the fixed memory module remain unchanged. When predicting the noise for the abnormal sample model, the abnormal area will be gradually removed, so that the restored image is more biased towards the normal image. Finally, calculate E sq =(x′0-x0) 2 , filter out E sq >0.5 is where the anomaly predicted by the model of the present invention occurs.

[0038] 1. Diffusion model

[0039] The basic idea of ​​the diffusion model is: for an input image x, a series of noisy images {x0, x1, …, x T}, image x t The noise level t gradually increases from 0 to T. The diffusion model θ is trained to obtain the value from x according to formula (5). t Predict x t-1 , for any time t∈{1,…,T}. During the training process, we know x t-1 The model is trained using the mean square error (MSE) loss. T Start by predicting x t-1 For t∈{1,…,T}. Through this iterative denoising process, a reconstructed image x′0 can be generated. The forward denoising process is a non-homogeneous Markov chain, which can be defined by formula (1);

[0040]

[0041] Among them, q(x t |x t-1 ) indicates that given the state x at the previous moment t-1 Under the condition that the current state x t The conditional probability distribution of ; represents Gaussian distribution, β tTable represents the noise intensity added at time step t, I represents the identity matrix, as part of the defense difference matrix. The noise sampled here is not Gaussian noise but simplex noise. Perlin noise generates smooth noise by generating random gradients on the grid, calculating the inner product between these gradients and the offsets of the candidate points, and then interpolating the results. More advanced simplex noise uses a simplex grid of equilateral triangles, reducing the complexity of dimensions and reducing directional artifacts in traditional gradient Perlin noise. Compared with standard Gaussian perturbations, simplex noise introduces more structured anomalies, allowing the denoising process to better "repair" these structured anomalies. So this recursive formula can be explicitly written as:

[0042]

[0043] Formula (2) allows to directly calculate x at any time step t t Fast sampling without the need to compute intermediate steps {x t-1 ,…,x1}. By directly defining x t In this process, the model can directly skip the need for sequential sampling of earlier time steps, thereby improving efficiency in practice. Among them, ∈ represents the real noise, v represents the initial frequency, which determines the level of detail of the noise, N represents the number of octaves of the noise, γ represents the decay rate, which controls how the amplitude of each frequency component decreases as the frequency increases, and α t =1-β t and Denoising process θ Learning is done by optimizing the model parameters θ, which can be expressed as:

[0044]

[0045] in,

[0046] ∈ θ Represents the noise predicted by the model, and the MSE loss function used to train the model is expressed as:

[0047]

[0048] Using formula (5) from x t Predict x t-1 ;

[0049]

[0050] 2. Some details of the MemDDPM model are as follows Figure 2As shown in Figure 1, the input x passes through multiple ResBlocks (residual blocks) and then enters the memory module, which can store and retrieve patterns at different stages, helping to retain key information. ResBlock consists of multiple convolutional layers, activation function layers, and regularization layers.

[0051] 3. Memory module

[0052] The memory module structure is as follows Figure 3 As shown in Figure 1, it consists of two main parts: a memory unit (Memory M) for storing prototype encoding patterns, and an attention-based addressing mechanism (Memory Addressing) for efficiently accessing the stored content. The attention mechanism plays a crucial role in selecting relevant information from the memory for further processing.

[0053] The input shape is C*H*W tensor, which can be divided into H*W Z vectors after deformation. The memory cells are designed as a matrix Contains N real-valued vectors of fixed dimension C. The dimension C is the same as the number of input channels, that is, Each row vector m in the matrix M i represents a memory item, where i∈[N] represents the i-th row and [N] is a set of integers from 1 to N. Given The addressing mechanism is based on addressing vectors To obtain z′;

[0054]

[0055] Where w is a row vector containing non-negative elements whose sum is 1, w i represents the i-th element of w. The weight vector w is calculated based on the encoding z. As shown in formula (6), the addressing weight w is the key to access the memory module. The hyperparameter N defines the maximum capacity of the memory unit. The impact of the size of N on the experiment will be explored later.

[0056] Each weight w i It is calculated by the softmax operation

[0057]

[0058] Among them, d(·,·) represents a similarity calculation method, and the cosine similarity used here;

[0059]

[0060] The memory module obtains the representation z by retrieving the memory item that best matches z. ′Due to the limited capacity of the memory unit and the sparse addressing technique, only a small number of memory items can be accessed at a time. Therefore, the effective behavior of the memory module can be explained as extracting the most representative patterns from the limited prototype memory to improve the model's prediction of noise during inference.

[0061] Reconstructing with limited memory items of normal modes helps to introduce larger reconstruction errors on abnormal data, and the attention-based addressing mechanism helps to achieve this goal. However, some abnormal data may still be well reconstructed by combining multiple memory items with a dense weight vector w containing many small elements. To address this problem, a hard shrinkage operation is used to promote the sparsity of w and reduce the contribution of unnecessary memory items to the reconstruction.

[0062]

[0063] Among them, w′ i represents the i-th element of the memory addressing weight vector w′ after contraction, and λ represents the contraction threshold. Since it is difficult to directly implement the back propagation of discontinuous functions in formula (9), considering that all elements in the weight vector w are non-negative numbers, in order to simplify the calculation, a continuous ReLU activation function is used to redefine the hard contraction operation.

[0064]

[0065] where max(·,0) is the ReLU activation function and η is a very small positive scalar. In practice, setting the shrinkage threshold λ in the interval [1 / N,3 / N] can produce ideal results. After shrinkage, w′ is obtained by the following operation i =w′ i / ‖w′‖ is valid for all i. Finally, the reconstructed z′ is obtained through z′=w′M.

[0066] w′ i Represents each sample x i To further promote the sparsity of w′, in addition to the shrinkage operation, a sparse regularization term is applied to w′ during training. For all terms of w′ to be non-negative, we minimize the entropy of w′:

[0067]

[0068] Combining formula (4) and formula (11), the training objective constructed for MemDDPM is as follows

[0069]

[0070] Among them, δ is a hyperparameter in training. In practice, different weights are assigned according to the location of the memory module. During the training process, the memory module M is updated through back propagation and gradient descent. During back propagation, only the non-zero addressing weight w′ t The gradient of the associated memory term will be non-zero.

[0071] 4. Verification experiment:

[0072] In order to verify whether the medical abnormality detection method of the present invention can effectively detect and segment brain abnormalities such as brain tumors, the present invention uses the NFBS dataset to train the model, and compares it with AE, VAE, AnoGan, DDPM, and pDDPM methods on the abnormal dataset of the Edinburgh University Clinical Brain Science Center and the MSLUB dataset. The present invention tests the performance indicators of Dice and AURPC on the two datasets, and the results are shown in Table 1.

[0073] Table 1 Performance indicators of different methods tested on two datasets

[0074]

[0075] According to the results in Table 1, the model method of the present invention achieved the best results in both data sets. In the Edinburgh University Clinical Brain Science Center Abnormal Dataset (anoDataset), our method Dice reached 44.46%, an increase of 1.48% compared to the previous pDDPM method, and AURPC reached 45.86%, an increase of 1.87%; on the MSLUB data set, Dice reached 10.37%, and AURPC reached 11.87%, both better than other methods. In summary, the anomaly detection method of the present invention has achieved better performance results on different medical image brain MRI data sets and compared with different anomaly detection algorithms.

[0076] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A medical image anomaly detection method based on a diffusion model, characterized in that: The method specifically comprises the following steps: S1: Obtain medical images, including healthy samples and abnormal samples; S2: Construct the Mem-DDPM model, including the diffusion model and the memory module, where Mem-DDPM represents the diffusion model enhanced by the memory module; S3: Training Mem-DDPM model: Train the diffusion model and memory module simultaneously using only healthy samples in an unsupervised manner; S4: In the test, the parameters of the memory module are fixed and the medical image to be detected is input into the trained Mem-DDPM model to screen out abnormal areas in the image to be detected.

2. The medical image anomaly detection method according to claim 1, characterized in that: In step S2, the diffusion model constructed is: for the input image x, by gradually adding simplex noise at multiple time steps T, a series of noise images {x0, x1, ..., x T }, image x t The noise level t gradually increases from 0 to T.

3. The medical image anomaly detection method according to claim 1, characterized in that: In step S2, the diffusion model includes a residual block, and the residual block includes multiple convolutional layers, activation function layers and regularization layers.

4. The medical image anomaly detection method according to claim 1, characterized in that: In step S2, the memory module includes a memory unit for storing the prototype encoding pattern and an attention-based addressing mechanism.

5. The medical image anomaly detection method according to claim 2, characterized in that: In step S3, the Mem-DDPM model is trained, which specifically includes: using only healthy samples to train the diffusion model and memory module at the same time, gradually adding simplex noise to the input x0 to convert it into x T , the model will predict the noise added at time T, denoise it, and generate x after denoising T-1 Repeat T steps to generate the restored and reconstructed image x′0. Calculate the reconstruction loss function through x0 and x′0 and combine it with the loss function of the memory module to train the entire diffusion model and memory module. The training objectives of the Mem-DDPM model are: in, is the loss function for training the MemDDPM model, and δ is a hyperparameter in training; MSE loss function for training diffusion model is expressed as: Among them, ∈ θ represents the noise predicted by the model, ∈ represents the real noise; α t =1-β t , β t represents the noise intensity added in time step t, which gradually increases; v represents the initial frequency, which determines the level of detail of the noise; N represents the number of octaves of the noise; γ represents the decay rate, which controls how the amplitude of each frequency component decreases as the frequency increases; E(w i ′) represents the minimization of w i The entropy of ′ is expressed as: Among them, w i ′ represents the sample x i The memory addressing weight.

6. The medical image anomaly detection method according to claim 5, characterized in that: In step S3, sample x i The memory addressing weight w i The expression of ′ is: Among them, max(.,0) is the ReLU activation function, η is a very small positive scalar, λ is the shrinkage threshold, and w i Represents the i-th element of w; w is a row vector containing non-negative elements whose sum is 1.

7. The medical image anomaly detection method according to claim 1, characterized in that: Step S4 specifically includes: in the test, the parameters of the fixed memory module remain unchanged, and when predicting the noise for the abnormal sample model, the abnormal area will be gradually removed, so that the restored image is more biased towards the normal image; finally, E is calculated sq =(x′0-x0) 2 , filter out E sq >0.5 is where the anomaly predicted by the Mem-DDPM model occurs, where E sq represents the mean square error, x′0 represents the reconstructed image, and x0 represents the input image.

Images