A medical image anomaly detection method based on autoencoder network

Abstract

The present invention discloses a method for detecting abnormality of medical images using an autoencoder network, comprising: S1, selecting a set image as a data set for a medical image abnormality detection task, dividing the foreground of the set image into superpixel blocks, and obtaining a superpixel block mask of the set image; S2, randomly selecting a number of superpixel block masks of the set image, and using the superpixel block mask to cover the original image of the set image; S3, inputting the covered set image into an autoencoder network for training, and the autoencoder network uses the weighted sum of the mean square error and the structural similarity error as a loss function to adjust the network parameters; S4, when performing abnormality detection on the medical image, inputting the image into the trained autoencoder network, obtaining a difference image from the input image and the output image, and calculating the abnormal area according to the difference image. The present invention can improve the reconstruction capability of the autoencoder network for normal samples, and better distinguish abnormal images through reconstruction errors.

Description

Technical Field

[0001] The present invention relates to the field of medical detection technology, and more specifically, to a self-encoding network medical image anomaly detection method. Background Art

[0002] In the field of computer vision, many relatively mature algorithms have been put into practical applications, such as face recognition, target tracking, and target detection systems, etc., which are inseparable from the core machine learning algorithm. However, through analysis, it is found that many algorithms are easily affected by out-of-distribution samples, which will lead to errors and overconfident decisions. In real applications, these out-of-distribution samples are often regarded as abnormal samples. If they can be detected, then deep learning algorithms can be deployed more safely and robustly in various application fields.

[0003] Therefore, how to connect a large amount of medical image data, organize mining and application, and improve the application ability of computer-aided diagnosis is an important issue in the field of medical image research. The human eye has perception gaps, also known as "inattentional blindness". Due to the complexity of medical images, the human eye sometimes cannot pay attention to all abnormalities, so computers can play an auxiliary role in diagnosis.

[0004] Most of the existing deep learning-based medical image anomaly detection algorithms rely on a large amount of pixel-level labeled data, that is, the labeled data contains abnormal areas, and the annotation of medical images often requires a lot of manpower and material resources. At the same time, medical images with abnormal conditions are diverse, and it is impossible to list all abnormal conditions for supervised learning. Therefore, it is necessary to develop a self-encoding network medical image anomaly detection method. Summary of the invention

[0005] The purpose of the present invention is to provide a self-encoding network medical image anomaly detection method to overcome the defects of the prior art.

[0006] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0007] A method for detecting abnormalities in medical images using an autoencoder network, comprising the following steps:

[0008] S1. Select a set image as a data set for the medical image anomaly detection task, and use a simple linear iterative clustering algorithm to divide the foreground of the set image into superpixel blocks to obtain a superpixel block mask of the set image;

[0009] S2, randomly selecting a number of superpixel block masks of a set image, and using the superpixel block masks to cover the original image of the set image;

[0010] S3, inputting the masked set image into the autoencoder network for training, the autoencoder network uses the weighted sum of the mean square error and the structural similarity error between the output image and the original image of the masked set image as a loss function to adjust the network parameters, so that the difference image between the output image of the autoencoder network and the input image reaches a first preset threshold;

[0011] S4. When performing abnormality detection on a medical image, the image is input into a trained autoencoder network, a difference image is obtained from the input image and the output image, and the abnormal area is calculated based on the difference image.

[0012] Furthermore, the pixel value filled in the masked area in step S2 is the average value of the pixel value of the corresponding original image area.

[0013] Furthermore, in step S3:

[0014] The expression of the mean square error is:

[0015]

[0016] In the formula, I and K represent the input image and the reconstructed image respectively, and m and n represent the length and width of the image respectively;

[0017] The structural similarity error expression is:

[0018] SSIM(x,y)=f(l(x,y),c(x,y),s(x,y)

[0019] Where l is brightness, c is contrast, and s is structure, which are defined as follows:

[0020]

[0021]

[0022]

[0023] In the formula, μ x and μ y are the mean values ​​of all pixels in the image block, σ x , σ y is the variance of the pixel values ​​of the image block, σ xy is the corresponding covariance. In order to prevent the denominator from being 0, C1, C2, and C3 are added to the formula. If C3 = C2 / 2, the simplified structural similarity error expression is:

[0024]

[0025] The expression of the loss function is:

[0026] Loss(x, y) = w * SIMM(x,y)+(1-w) * MSE(x,y).

[0027] Furthermore, the abnormal area is calculated according to the difference image in step S4 as follows: if the pixels in the difference image that are smaller than the second set threshold are set as normal pixels, the number of pixels in the connected area is used to distinguish the noise area in the remaining area.

[0028] Furthermore, the pixel value in the noise area is reduced by one order of magnitude.

[0029] Furthermore, the set image is a brain magnetic resonance imaging image.

[0030] Compared with the prior art, the advantages of the present invention are: an autoencoder network medical image anomaly detection method provided by the present invention adds a superpixel block foreground masking strategy to the semi-supervised anomaly detection framework based on the autoencoder network, which can improve the autoencoder network's ability to reconstruct normal samples and better distinguish abnormal areas through reconstruction errors. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0032] Figure 1 It is a flow chart of the self-encoding network medical image anomaly detection method of the present invention.

[0033] Figure 2 It is a schematic diagram of the superpixel block mask divided by a simple linear iterative clustering algorithm in the present invention.

[0034] Figure 3 It is a reconstruction training strategy diagram based on image processing in the present invention.

[0035] Figure 4 It is a schematic diagram of using the super pixel block mask to cover the original image of the set image in the present invention.

[0036] Figure 5 It is a function diagram of the mean square error loss in the present invention. DETAILED DESCRIPTION

[0037] The preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings so that the advantages and features of the present invention can be more easily understood by those skilled in the art, thereby making a clearer and more definite definition of the protection scope of the present invention.

[0038] See also Figure 1 As shown, this embodiment discloses a self-encoding network medical image anomaly detection method, comprising:

[0039] Step S1: select a set image (brain MRI image is selected in this embodiment) as the data set for the medical image anomaly detection task, and use a simple linear iterative clustering algorithm to divide the foreground of the set image into superpixel blocks to obtain a superpixel block mask of the set image, as shown in the schematic diagram. Figure 2 shown.

[0040] In this embodiment, superpixel division utilizes the similarity of features between pixels to group pixels to form superpixels, and uses a relatively small number of superpixels to replace a large number of pixels in the original image to represent the features of the image, thereby reducing the complexity of image post-processing. Therefore, superpixel generation is usually used as a preprocessing step of the image segmentation algorithm.

[0041] In this embodiment, a simple linear iterative clustering algorithm SLIC (Simple Linear Iterative Cluster) is used, which is a very classic super pixel generation algorithm, and its idea is as follows: first, the image in the RGB color space is converted to the CIE-Lab space, and the image in this space consists of three channels, a brightness channel and two color channels. In the CIE-Lab color space, the meanings of each component of L, a, and b are brightness, components from green to red, and components from blue to yellow. This color space is designed based on people's perception of color. Compared with the RGB color space, it is more in line with human visual characteristics and is easier to change and adjust. If the brightness needs to be adjusted, the value of the corresponding L channel is adjusted. If the color is to be adjusted, the value of the a and b channels is adjusted, and the color gamut range is greater than the color gamut in the RGB space. Each pixel can be regarded as a five-dimensional vector (L, a, b, x, y). A distance metric is constructed for the five-dimensional vector so that the similarity between two pixels can be measured by the distance between their corresponding vectors. The smaller the distance, the more similar the pixels are, and the larger the distance, the smaller the similarity.

[0042] In this embodiment, all pixels in the foreground of the image are grouped into superpixel blocks, which can be regarded as a clustering problem at the pixel level. The SLIC algorithm refers to the idea of ​​the K-means clustering algorithm. First, K seed points are generated, and then the distances from the remaining pixels to the seed points are calculated. The seed points with the closest distance are selected and added to the set corresponding to the seed points until all the pixels are searched. Then the average vector value between all pixels in the K sets is calculated, and K cluster centers are obtained again. Then, these K centers are used to search for several pixels with the closest vector distances around them, and they are added to the category. After all the pixels are classified, the cluster centers are updated, and iterated again, and so on until convergence. We use the SLIC algorithm to generate masks corresponding to superpixel divisions of brain magnetic resonance images.

[0043] The advantage of using the SLIC algorithm in this embodiment is that the generated superpixel blocks are compact and neat, and can not only segment color images, but also grayscale images. It can be applied to brain magnetic resonance images. At the same time, the algorithm requires fewer parameters to be set. In the default case, only the number of pre-segmented superpixels needs to be given. The SLIC algorithm is ideal in terms of the compactness and running speed of generating superpixels.

[0044] Step S2, randomly select several superpixel block masks of the set image, and use the superpixel block masks to cover the original image of the set image, and the pixel value filled in the covered area is the average value of the pixel value of the corresponding original image area.

[0045] The main idea of ​​this embodiment is to use the reconstruction ability of the autoencoder network and train the network using only normal samples, so that the network model can reconstruct the normal samples well. When the abnormal samples are input into the autoencoder network, there will be a large reconstruction error, so the abnormality can be determined by the size of the error. There can be different reconstruction strategies for images in different application fields, in which the input image can be preprocessed and then input into the autoencoder network. The framework diagram under this strategy is shown in Figure 3.

[0046] In combination with the characteristics of medical images, this embodiment designs a superpixel foreground masking strategy. The masking method in the reconstruction strategy will be introduced below. According to the superpixel block mask obtained in step S1, a number of superpixel blocks are randomly selected, and only the masks of the selected superpixel blocks are retained. The mask is used to mask the corresponding area of ​​the original image. The pixel value filled in the masked area is the average value of the pixel value of the corresponding original image area, such as Figure 4 shown.

[0047] Step S3, inputting the masked set image into the autoencoder network for training, the autoencoder network uses the weighted sum of the mean square error and the structural similarity error between the output image and the original image of the masked set image as the loss function to adjust the network parameters so that the difference image between the output image of the autoencoder network and the input image reaches a first preset threshold.

[0048] In this embodiment, the superpixel foreground masking strategy is mainly inspired by the context-autoencoder. Similar to the classic autoencoder network, it also first encodes the input image into a shallow feature space, learns a compact feature representation, and then uses the decoder to reconstruct the original image. However, the general autoencoder network only compresses the image information and does not fully learn the semantic information of the image content. Therefore, this embodiment proposes a new reconstruction method that allows the autoencoder network to learn to fill in the large area missing parts of the input image, so that it cannot obtain prompt information through nearby pixels, but must understand deeper semantic information. Through the study of context-autoencoder and the analysis of brain magnetic resonance images, this embodiment proposes an autoencoder network reconstruction strategy based on superpixel foreground masking.

[0049] Since each region of a single complete organ in the human body has similar physiological structures, parts of the same structure show similar colors in medical images, and lesion areas are generally more likely to appear in connected areas with the same physiological structure. The SLIC algorithm clusters pixels based on the color similarity and distance between pixels. Each pixel block generated is a continuous area with relatively relevant colors, so it can cover the corresponding area of ​​the superpixel block, simulating pathological abnormalities to a certain extent. At the same time, the network is trained to fill in the occluded part through the context of the occluded area, so that the autoencoder network can better learn the semantic features of normal samples.

[0050] During the network model training process, this embodiment uses the weighted sum of the mean square error and the structural similarity error between the output image and the original uncovered image as the training loss function to adjust the model parameters.

[0051] In this embodiment, the mean square error loss (MSE) is often used as the loss function of the regression model, which calculates the mean of the sum of squares of the differences between the predicted value and the label. When the network model outputs multidimensional data, it can be regarded as a point in the vector space. When the Euclidean distance is used to describe the distance between two points in the multidimensional space, the mean square error loss can be regarded as the L2 distance between the two points in the Euclidean space. Its expression is as follows:

[0052]

[0053] The function graph is Figure 5 shown.

[0054] In the above expression, I and K mainly represent two images, which can represent the input image and the reconstructed image respectively in the autoencoder network, and m and n represent the length and width of the image respectively. The mean square error curve is smooth, continuous and can be differentiated everywhere. It is convenient to use the gradient descent algorithm in the neural network model and is a common loss function. As the error decreases, the gradient also decreases, which is conducive to convergence and can converge to the minimum value quickly. The smaller the value of the mean square error, the better the overall image reconstruction effect of the autoencoder network model.

[0055] In this embodiment, the mean square error loss focuses on the deviation corresponding to each pixel in the image. The large deviation of a single pixel may have a greater impact on the overall loss, and the structural similarity loss can make up for the shortcomings of the mean square error loss. The structural similarity index (SSIM) pays more attention to the regional characteristics of the image. In some cases, the two pictures are only slightly different in brightness, and the important content in the image is not much different. At this time, the mean square error of the two pictures may be large. Therefore, unlike the mean square error measuring absolute error, the structural similarity loss is more in line with the intuitive feeling of the human eye. The structural similarity index mainly considers the image from three perspectives: brightness (Luminance), contrast (Contrast), structure (Structure), which can be expressed as the following expression:

[0056] SSIM(x,y)=f(l(x,y),c(x,y),s(x,y))

[0057] In the expression, l, c, and s are defined as follows:

[0058]

[0059]

[0060]

[0061] In the formula, μ x and μ y are the mean values ​​of all pixels in the image block, σ x , σ y is the variance of the pixel values ​​of the image block, σ xy is the corresponding covariance. In order to prevent the denominator from being 0, C1, C2, and C3 are added to the formula. If C3 = C2 / 2, the simplified structural similarity error expression is:

[0062]

[0063] The weighted sum of the mean square error and the structural similarity error is used as the loss function for autoencoder network training. The expression of the loss function is as follows:

[0064] Loss(x, y) = w * SIMM(x,y)+(1-w) * MSE(x,y)

[0065] Step S4: When performing abnormality detection on a medical image, the image is input into a trained autoencoder network, a difference image is obtained from the input image and the output image, and the abnormal area is calculated based on the difference image.

[0066] Compared with some anomaly detection methods that can only perform binary classification on samples, this embodiment can obtain pixel-level abnormal area detection results. The anomaly detection method based on image segmentation can directly output the classification results of pixels, but usually requires label data to train the network, while this embodiment has certain structural advantages. It does not require pixel-level label data during the training stage. The autoencoder network can use the difference image between the reconstructed image and the input image to obtain the sample pixel-level abnormality detection results, that is, to determine whether the pixel is abnormal. Generally speaking, pixels with higher pixel values ​​in the difference image may be abnormal pixels, and multiple connected abnormal pixels constitute abnormal areas. Since there may be some noise in the medical image during the reconstruction process, there may also be a corresponding small range of connected areas on the difference image, but it is not abnormal, so the abnormal area calculation method used in this embodiment is introduced below.

[0067] This embodiment first sets a pixel value threshold, and considers that the pixel points in the difference image whose pixel values ​​are lower than the threshold are not abnormal. In an absolutely ideal case, the reconstructed image and the input image are exactly the same, and the pixel values ​​of all the pixels in the difference image are 0. Since this situation almost never occurs, this embodiment first sets a smaller pixel value as the threshold, and the pixels corresponding to the values ​​smaller than the threshold are regarded as normal pixels by default. Next, the interference of noise is eliminated on the remaining pixels. By analyzing the characteristics of medical images, the abnormal areas are usually larger connected areas, so the number of pixels in the connected areas can be used to distinguish between abnormal areas and noise areas. After dividing the noise area, the pixel values ​​corresponding to the pixels in the noise area are reduced by one order of magnitude, and the pixel values ​​are divided by 10 to reduce the interference of noise when dividing the image pixels into abnormal areas.

[0068] This embodiment introduces a superpixel foreground masking strategy in the semi-supervised image anomaly detection framework based on the autoencoder network, trains the network's ability to use contextual information to fill in the masked area, and improves the network's ability to extract and reconstruct semantic information of normal images. In addition, the superpixel foreground masking simulates pathological abnormalities to a certain extent, which helps to make the reconstruction error of normal images smaller and the reconstruction error of abnormal images larger during the test process, so that abnormal images can be distinguished by the reconstruction error between the output image and the input image.

[0069] The mean square error loss MSE and the structural similarity index loss SSIM consider the two images from the overall and local perspectives respectively. When training the autoencoder network, the weighted sum of the mean square error loss MSE and the structural similarity index loss SSIM is used as the loss function to constrain the network training. The reconstructed medical image is made as close to the control image as possible both overall and locally, which makes up for the deficiency that the network is too easily affected by a single pixel error when only using MSE as the loss function.

[0070] Although the embodiments of the present invention are described in conjunction with the accompanying drawings, the patent owner may make various variations or modifications within the scope of the appended claims. As long as they do not exceed the protection scope described in the claims of the present invention, they should be within the protection scope of the present invention.

Claims

1. A self-encoding network medical image anomaly detection method, characterized in that: The following steps are involved: S1. Select a set image as a data set for the medical image anomaly detection task, and use a simple linear iterative clustering algorithm to divide the foreground of the set image into superpixel blocks to obtain a superpixel block mask of the set image; S2, randomly selecting a number of superpixel block masks of a set image, and using the superpixel block masks to cover the original image of the set image; S3, inputting the masked set image into the autoencoder network for training, the autoencoder network uses the weighted sum of the mean square error and the structural similarity error between the output image and the original image of the masked set image as a loss function to adjust the network parameters, so that the difference image between the output image of the autoencoder network and the input image reaches a first preset threshold; S4. When performing abnormality detection on a medical image, the image is input into a trained autoencoder network, a difference image is obtained from the input image and the output image, and the abnormal area is calculated based on the difference image; In step S3: The expression of the mean square error is: In the formula, I and K represent the input image and the reconstructed image respectively, and m and n represent the length and width of the image respectively; The structural similarity error expression is: SSIM(x,y)=f(l(x,y),c(x,y),s(x,y)) Where l is brightness, c is contrast, and s is structure, which are defined as follows: In the formula, μ x and μ y are the mean values ​​of all pixels in the image block, σ x , σ y is the variance of the pixel values ​​of the image block, σ xy is the corresponding covariance. In order to prevent the denominator from being 0, C1, C2, and C3 are added to the formula. If C3 = C2 / 2, the simplified structural similarity error expression is: The expression of the loss function is: Loss(x,y)=w * SIMM(x,y)+(1-w) * MSE(x,y)。 2. The method for detecting anomalies in medical images using an autoencoder network according to claim 1, characterized in that: The pixel value filled in the masked area in step S2 is the average value of the pixel value of the corresponding original image area.

3. The method for detecting anomalies in medical images using a self-encoding network according to claim 1, characterized in that: The specific method of calculating the abnormal area according to the difference image in step S4 is as follows: if the pixels in the difference image that are smaller than the second set threshold are set as normal pixels, the number of pixels in the connected area is used to distinguish the noise area in the remaining area.

4. The method for detecting anomalies in medical images using a self-encoding network according to claim 3, characterized in that: The pixel values ​​in the noise area are reduced by an order of magnitude.

5. The method for detecting anomalies in medical images using a self-encoding network according to claim 1, characterized in that: The set image is a brain magnetic resonance image.

Images