A fundus blood vessel segmentation domain generalization method based on frequency domain analysis

Abstract

A generalization method for fundus vessel segmentation based on frequency domain analysis includes the following steps: 1) preprocessing fundus images, including numerical processing and size adjustment; 2) extracting features from the preprocessed fundus images, including constructing and training a neural network model; 3) using the trained model to segment fundus vessels and the retinal background. This invention employs a feature normalization algorithm based on frequency domain analysis. By learning a unified semantic frequency distribution space, it enhances the model's feature representation and generalization capabilities, effectively overcoming the data dependence of supervised learning methods and the decrease in segmentation algorithm accuracy when facing data from different domains, further improving the generalization ability of the fundus vessel segmentation algorithm.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of medical image analysis, relates to the domain generalization problem in fundus blood vessel segmentation, uses a frequency domain analysis method based on Discrete Cosine Transform (DCT), is a kind of deep learning technology, and belongs to a labeled data learning method. BACKGROUND

[0002] The fundus is distributed with rich capillaries, and the capillaries can moisten the eyeball. Modern imaging technology can help doctors directly observe the vascular structure of the retina through fundus images, and then judge the condition of the patient. For example, capillary tumors, bleeding and small bleeding points can often be seen in the fundus images of diabetic patients, and arteriosclerosis can be observed in the fundus blood vessels of hypertensive patients. These eye blood vessel lesions also reflect the vascular condition of the whole body to some extent, and doctors can analyze and judge the type and severity of the disease through fundus examination. Therefore, fundus examination is an important means for diagnosing fundus diseases, and fundus blood vessel segmentation can provide an auxiliary quantitative tool for disease analysis and diagnosis.

[0003] The fundus image is rich in details, and if relying on manual analysis by doctors, it has a high requirement on the observation ability and medical experience of doctors. Computer-aided diagnosis technology can help doctors improve the efficiency and accuracy of diagnosis. Traditional computer-aided diagnosis technology will extract some manually pre-set features by an algorithm, and doctors will make judgments according to these features. In recent years, the rapid development of deep learning has made the visual recognition accuracy of computers exceed that of humans in some specific fields. A trained deep neural network automatically outputs the fundus blood vessel segmentation result at the network output end by extracting features layer by layer and learning the semantic information in the image, which will significantly reduce the workload of doctors and improve the efficiency of screening and diagnosis.

[0004] In actual application scenarios, the samples of the training set and the test set of deep learning differ in terms of illumination, contrast, sampling equipment and the like, that is, the data source domain in the training process and the data target domain at the time of testing produce a domain shift problem, which further affects the generalization performance of the model, and thus the deep neural network often cannot achieve satisfactory blood vessel segmentation results in the testing stage. At the same time, a model with good generalization ability needs a large amount of and rich labeled data for training, however, the fundus blood vessel segmentation dataset is difficult to collect and label, and the public dataset often has the problems of small sample size and insufficient diversity. SUMMARY

[0005] In order to overcome the data dependence of the supervised learning method and the problem of accuracy reduction of the target domain data segmentation algorithm, and further improve the generalization ability of the fundus blood vessel segmentation algorithm, the application proposes a fundus blood vessel segmentation domain generalization method based on frequency domain analysis, adopts a feature normalization algorithm based on frequency domain analysis, the algorithm is an improvement and generalization of the batch normalization (Batch Normalization, BN) algorithm, mainly composed of a frequency domain normalization module and a spatial domain normalization module, by learning a more unified semantic frequency distribution space, the expression ability and generalization ability of the model are enhanced, and the trained network model can obtain more accurate fundus blood vessel analysis results on the target domain.

[0006] The technical scheme adopted by the application to solve the above problems is:

[0007] A fundus blood vessel segmentation domain generalization method based on frequency domain analysis, the method comprises the following steps:

[0008] 1) pre-processing the fundus image;

[0009] 2) feature extraction is performed on the pre-processed fundus image;

[0010] 3) according to the threshold value, the features are segmented into blood vessels and retinal background.

[0011] Further, the step 1) comprises the following steps:

[0012] 11) using random rotation, horizontal random flip, vertical random flip, random brightness and saturation to enhance the original picture, and using Gamma nonlinearization coding to enhance the contrast effect of the image;

[0013] 12) using the maximum and minimum normalization to map the pixel value from 0-255 to 0-1;

[0014] 13) cutting off the redundant pixels in the fundus image, only retaining the retinal region in the image, and then, in order to ensure the input scale invariance, the cut fundus image is unified to 512*512 pixels by the method of bilinear interpolation. In order to fully utilize the image information, and also to introduce randomness, each 512*512 pixel fundus image is randomly cropped 64 times, and the cropped image block size is 64*64.

[0015] Further, the step 2) comprises the following steps:

[0016] 21) constructing a neural network model, and taking the pre-processed data as the input of the model training;

[0017] 22) training the neural network, and updating the model parameters through the algorithm

[0018] 23) The predicted probability output by the model is extracted as a feature from the input data.

[0019] Further, in the step 21), the network construction process is as follows:

[0020] 211) Constructing a frequency-space normalization layer;

[0021] 212) Setting an encoder, and each layer module is composed of a residual block, a frequency-space normalization layer, and a down-sampling layer;

[0022] 213) Setting a decoder, and each layer module is composed of a residual block, a frequency-space normalization layer, and an up-sampling layer;

[0023] 214) Connecting the encoder and the decoder in a cross-layer jumping manner.

[0024] Further, in the step 22), the network training process is as follows:

[0025] 221) Setting a loss function;

[0026] 222) Setting model parameter initialization;

[0027] 223) Setting an optimizer.

[0028] The process of the step 3) is: segmenting the blood vessels and the retinal background according to the extracted features, setting 0.5 as a threshold, and marking the feature pixels greater than 0.5 as blood vessel features and the feature pixels less than 0.5 as fundus background features.

[0029] Compared with the prior art, the beneficial effects of the present application are that: by preprocessing the fundus dataset, a model fusing frequency domain information and spatial domain information is established, and by introducing an attention mechanism, the features in different domains are normalized to a unified hidden space. The present application can improve the robustness of the blood vessel segmentation model to data distribution disturbance, and further realize the generalization ability to fundus images of different sources. BRIEF DESCRIPTION OF DRAWINGS

[0030] Figure 1 is a flowchart of a fundus blood vessel segmentation domain generalization method based on frequency domain analysis.

[0031] Figure 2 is a preprocessing schematic diagram of a fundus image.

[0032] Figure 3 is a frequency normalization module schematic diagram.

[0033] Figure 4 is a space normalization schematic diagram.

[0034] Figure 5 is a fundus blood vessel segmentation model schematic diagram.

[0035] Figure 6 is the fundus blood vessel segmentation result, wherein (a) is the original picture, (b) is the expert annotation, and (c) is the segmentation result. DETAILED DESCRIPTION

[0036] The application will be further described below in combination with a flow chart.

[0037] Reference Figures 1-5 A fundus blood vessel segmentation domain generalization method based on frequency domain analysis, comprising the following steps:

[0038] 1) Reference Figure 2 , the fundus image is preprocessed, and the image preprocessing comprises the following steps: 11) using random rotation, horizontal random flip, vertical random flip, random brightness and saturation to enhance the original picture, and using Gamma nonlinearization coding to enhance the contrast effect of the image; 12) using the maximum and minimum normalization to map the pixel value from 0-255 to 0-1; 13) cutting off the redundant pixels in the fundus image, and only retaining the retinal region in the image, and then, to ensure the input scale invariance, the cut fundus image is unified to 512*512 pixels by the method of bilinear interpolation. In order to fully utilize the image information and also introduce randomness, each 512*512 pixel fundus image is randomly cropped 64 times, and the size of the cropped image block is 64*64;

[0039] 2) Reference Figure 3 , 4 , 5, according to the extracted features, the blood vessels and the retinal background are segmented, wherein the feature extraction comprises the following steps: 21) constructing a neural network model, and taking the preprocessed data as the input of the model training; 22) training the neural network, and updating the model parameters by algorithm; 23) the prediction probability output by the model is taken as the feature extracted from the input data;

[0040] The network construction process is as follows:

[0041] 211) Reference Figure 2 , 3 , a frequency domain-space domain normalization layer is constructed, the frequency domain-space domain normalization layer is composed of a frequency domain normalization module and a space domain normalization module, wherein the frequency domain normalization module obtains a normalized frequency domain hidden space by DCT feature decomposition and introducing a learnable weight; and the space domain normalization module dynamically generates a scaling coefficient γ s and a translation coefficient β s from the input image to obtain a normalized space domain feature.

[0042] 212) Reference Figure 5An encoder is set, and the purpose of the encoder is to obtain the latent space features. Each layer module is composed of a residual block, a frequency-space domain normalization layer, and a down-sampling layer.

[0043] 213) Reference Figure 5 A decoder is set, and the purpose of the decoder is to reconstruct the segmented image from the feature vector z. Each layer module is composed of a residual block, a frequency-space domain normalization layer, and an up-sampling layer.

[0044] 214) The encoder and the decoder are connected in a cross-layer jump manner. The purpose of this step is to connect the features of different levels and resolutions.

[0045] Further, in the step 22), the training process of the network is as follows:

[0046] 221) A loss function is set to calculate the binary cross entropy between the network output features and the segmentation labels.

[0047] 222) Model parameter initialization is set. The parameters of the fundus blood vessel segmentation network are randomly initialized by Kaiming.

[0048] 223) An optimizer is set. The Adam optimization algorithm is adopted, the momentum is set to 0.99, the default learning rate is 0.002, the network is trained in a loop, and the parameters are updated for 4000 rounds.

[0049] 3) The features extracted according to the threshold are used to segment the blood vessels and the retinal background. 0.5 is set as the threshold. The feature pixels greater than 0.5 are marked as blood vessel features, and the feature pixels less than 0.5 are marked as fundus background features.

[0050] The above only describes the preferred embodiments of the present application and does not limit the present application. Any modification, equivalent replacement and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A frequency domain analysis-based fundus blood vessel segmentation domain generalization method, characterized in that, The method comprises the following steps: 1) preprocessing the fundus image; 2) feature extraction on the preprocessed fundus image; 3) segmenting blood vessels and retinal background according to the extracted features; The step 2) comprises the following steps: 21) constructing a neural network model, taking the preprocessed data as the input of model training; 22) training the neural network to update the model parameters through algorithm; 23) the prediction probability output by the model is taken as the feature extracted from the input data; In the step 21), the network construction process is as follows: 211) constructing a frequency domain-spatial domain normalization layer, the frequency domain-spatial domain normalization layer being composed of a frequency domain normalization module and a spatial domain normalization module, wherein the frequency domain normalization module obtains a normalized frequency domain hidden space by DCT feature decomposition and introduction of a learnable weight; and the spatial domain normalization module dynamically generates scaling coefficients γ s and translation coefficients β s from an input image to obtain normalized spatial domain features; 212) setting an encoder, and each layer module is composed of a residual block, a frequency-space domain normalization layer and a down-sampling layer; 213) setting a decoder, and each layer module is composed of a residual block, a frequency-space domain normalization layer and an up-sampling layer; 214) connecting the encoder and the decoder in a cross-layer jumping manner; The step 1) comprises the following steps: 11) using random rotation, horizontal random flip, vertical random flip, random brightness and saturation to enhance the original picture, and using Gamma nonlinear coding to enhance the contrast effect of the image; 12) using the maximum value normalization to map the pixel value from 0-255 to 0-1; 13) cutting off the redundant pixels in the fundus image, only retaining the retinal area in the image, and then performing bilinear interpolation on the cut fundus image to ensure the input scale invariance.

2. The frequency domain analysis based fundus blood vessel segmentation domain generalization method according to claim 1, characterized in that, In the step 22), the network training process is as follows: 221) setting a loss function; 222) setting the model parameter initialization; 223) setting the optimizer.

3. The frequency domain analysis based fundus blood vessel segmentation domain generalization method according to claim 1, characterized in that, The process of the step 3) is: according to the extracted features, setting 0.5 as the threshold, marking the feature pixels greater than 0.5 as blood vessel features, and marking the feature pixels less than 0.5 as fundus background features.

Images