An automatic measurement method for blood vessel diameter of fundus OCT image

Abstract

The application introduces a kind of blood vessel diameter automatic measurement method for fundus OCT image.Retinal blood vessel diameter measurement is an important clinical examination method, which can be used for the diagnosis and monitoring of ophthalmic diseases and cardiovascular diseases.The traditional half-peak full-width measurement method has higher requirements for the medical level of operators and limited applicability, while the present method realizes efficient and accurate measurement through a two-stage model.In the first stage, the approximate area of the blood vessel is obtained by using U-Net network, and in the second stage, the edge attention network (EAT) is used for accurate segmentation of the blood vessel area.The feature fusion module uses different fusion methods to meet the requirements of measurement speed and accuracy.The method has high efficiency and accuracy, and reduces the influence of human error.The neural network model trained by a large amount of data can accurately identify and measure retinal blood vessels of different shapes and diameter ranges, and is suitable for the diagnosis and monitoring needs of various ophthalmic and cardiovascular diseases.

Description

Technical Field

[0001] This invention relates to the fields of medical image analysis, artificial intelligence, and machine vision, and is an automatic method for measuring blood vessel diameter in fundus OCT images. Background Technology

[0002] Morphological examination of retinal vessels plays a crucial role in the diagnosis and monitoring of ophthalmic and cardiovascular diseases. Studies have shown that diseases such as retinal vascular occlusion, retinitis, and glaucoma lead to changes in retinal arterioles, and these changes can assist physicians in accurate diagnosis and monitoring disease progression. Furthermore, abnormal morphology of retinal arterioles, such as stenosis and sclerosis, is associated with an increased risk of cardiovascular diseases like coronary heart disease and stroke. Regularly monitoring morphological changes in retinal arterioles can help physicians assess patients' cardiovascular disease risk and implement appropriate preventative and treatment measures. Currently, spectral-domain optical coherence tomography (SD-OCT) imaging technology is widely used to obtain cross-sectional images of retinal vessels.

[0003] In previous studies, researchers used the full width at half maximum (FMHW) method to measure blood vessel diameter. However, this method requires a certain level of medical expertise from the measurer and is not applicable to large patient populations. In recent years, with the rapid development of neural networks, various network architectures have been used for fully automated segmentation and measurement of medical images, with lower user requirements, greatly facilitating the automation of blood vessel diameter measurement. However, many problems still need to be solved. For example, the OCT images used are high-resolution imaging, containing significant noise; furthermore, this task requires high accuracy in diameter measurement, necessitating pixel-level precision.

[0004] In response to the aforementioned medical needs and technical challenges, this invention provides a highly efficient and accurate method for automatically measuring the diameter of retinal vessels on OCT images, providing strong support for the diagnosis and treatment monitoring of ophthalmic diseases, reducing manpower input, and improving medical efficiency. Summary of the Invention

[0005] Based on the above problems, this invention proposes a full-width half-peak algorithm implementation process, provides a mathematical model of blood vessel diameter, and theoretically solves the problem of inaccurate blood vessel boundaries in blurred images; it uses a two-stage deep learning method to segment OCT images, and algorithmically solves the problem of difficulty in segmenting precise boundaries in current image processing; it proposes a complete development and algorithm implementation process to automate the technology, and solves the current supply-demand imbalance problem in clinical practice where doctors have limited diagnostic efficiency but patients have high demand.

[0006] To achieve the above objectives, the present invention employs the following technical means:

[0007] An automated method for measuring blood vessel diameter in fundus OCT images includes the following steps:

[0008] Step 1, Data Labeling: The full width at half maximum (FWHM) measurement method is used to pre-estimate the inner and outer diameters of the blood vessel. At the same time, the pixels within 70% of the width to the left and right of the center of the blood vessel are scanned column by column to obtain the edge information of the core position of the blood vessel. The relevant peripheral blood vessel contour information is then labeled to obtain the labeled image.

[0009] Step 2, Data Pre-processing: Cropping out the regions containing retinal blood vessels from the original image, and then normalizing the cropped OCT image;

[0010] Considering that the acquired images are single-channel grayscale images, grayscale normalization is adopted, and the grayscale values ​​of the pixels are distributed between 0 and 255 to avoid insufficient image contrast, i.e., unbalanced distribution of image pixel brightness, and at the same time to speed up the training speed of the network.

[0011] Step 3, Data Augmentation: Perform affine transformations on the labeled images obtained after Step 2 to augment the dataset. The data augmentation methods include random horizontal / vertical flipping, random rotation, and edge sharpening.

[0012] Step 4, Two-Stage Modeling: A two-stage model is used to complete the coarse segmentation and fine segmentation tasks of the OCT image. In the first stage, the approximate region of the blood vessels is obtained through the U-Net neural network; in the second stage, the Edge Attention Transformer (EAT) network is used to achieve accurate segmentation of the blood vessel region.

[0013] The Edge Attention Transformer (EAT) network has an overall encoder-decoder structure. The encoder uses a PVT model to divide the image into patches and initially extract image features, resulting in four low- to high-dimensional multi-scale features. The decoder introduces an Edge Aware Block (EAB) module, which uses edge information as a supervision signal to further enhance the encoded image features, thereby strengthening the edge information and obtaining enhanced multi-scale features. The supervision signal for edge information is extracted using distance transform. Simultaneously, the final step of decoding uses a Pyramid Future Fusion Model (PFFM) module to fuse the enhanced multi-scale features from bottom to top, obtaining the final segmentation result. The data fusion method can be either convolutional fusion (Conv Fusion) or deep convolutional fusion (DW-Conv Fusion), with the specific implementation choosing different fusion methods based on the requirements of running speed and accuracy.

[0014] Step 5, Vessel Diameter Measurement: The segmentation results from the first stage are filtered using digital image processing methods to remove irrelevant noise; the final segmentation results enable the measurement of the inner and outer diameters of blood vessels and the calculation of the vessel wall area, and the results can be used for subsequent clinical analysis.

[0015] In the above technical solution, the specific implementation algorithm of the full width at half maximum (FWHM) measurement method is as follows:

[0016] Step a1. Differentiation: Obtain a column of pixel values ​​vertically along the middle of the blood vessel, and use backward difference to subtract the data of the previous row from the data of the next row to obtain the difference information at the center.

[0017] Step a2. Differential sequence positive and negative grouping: For the differential information in step a1, divide it into several groups of positive and negative adjacent sequences with 0 as the discontinuity point;

[0018] Step a3. Pair up the positive and negative groups, and form a group with the positive value interval to the right of the negative value interval as the starting point. If the starting point is positive, discard it.

[0019] Step a4. The point with the highest difference: Calculate the sum of the absolute values ​​of each group of positive and negative values. You will be able to find the group with the largest difference in the left and right halves of the image respectively. The coordinates of the peak value are the boundary where the blood vessel wall is located in step 1, that is, the edge information is obtained.

[0020] Step a5. Repeat the above algorithm within a suitable range on the left and right sides of the central region to obtain the edge information of the blood vessel wall in the key area.

[0021] In the above technical solution, the OCT vascular image normalization formula in step 2 is:

[0022]

[0023] In the formula, I(i,j) and N(i,j) represent the gray values ​​of the original image and the transformed image, respectively, and min and max represent the minimum and maximum gray values ​​of the original image, respectively.

[0024] In the above technical solution, step 4:

[0025] The U-Net network is used to perform coarse segmentation on the input OCT image. The U-Net model consists of an encoder and a decoder. The core algorithm of the encoder is convolution, and the core algorithm of the decoder is interpolation. The OCT image is fed into the encoder, which generates high-level features of the image step by step. In the decoder, the generated high-level features are gradually fused with the features of the previous layer to reconstruct the segmentation result, and finally the OCT image is coarsely segmented. The encoder and decoder have the same feature dimension. The model uses identity mapping to connect features of the same dimension together (concatenate operation) to deepen the model's memory of shallow features.

[0026] In the above technical solution, the edge-aware block adopts an encoder-decoder structure. Through self-attention mechanism, cross-attention mechanism and linear neural network algorithm, it can further extract features. The decoder part of the edge-aware block adds an edge supervision signal with the same dimension as the input features. The decoding part considers the impact of edge information on the final segmentation result, thereby enhancing the image features and improving the accuracy of image segmentation.

[0027] In the above technical solution, the hierarchical feature fusion module, in order to fully utilize the image features at each stage, uses two fusion methods to fuse the multi-scale image features obtained from the edge-aware block from bottom to top. After fusion, the segmentation result of the OCT image is obtained, namely two binary images marked with the location of blood vessels. The two fusion methods are convolutional fusion and depthwise convolutional fusion. Because the present invention adopts the above technical means, it has the following beneficial effects: 1) In terms of OCT image data annotation, the present invention uses the full width at half maximum (FWHM) method to define the key parameters of fundus blood vessels and provides specific algorithm implementation steps. For most OCT images, it can accurately find the coordinates of the internal and external blood vessels at the center position. 2) In terms of model design, while ensuring appropriate model complexity, the present invention designs a two-stage neural network. First, the coarse segmentation result of the image is extracted by the classic medical segmentation model U-Net; the second-stage EAT network uses an attention mechanism to refine the edge of the result, effectively using the edge correlation transformation map of the original image as a supervision signal to better achieve edge information extraction. 3) In terms of data fusion (feature fusion), convolutional fusion (Conv Fusion) can achieve higher accuracy, while deep convolutional fusion (DW-Conv Fusion) can achieve faster running speed. Different fusion methods can be selected according to different needs in the actual scenario.

[0028] 4) A convenient method for fine segmentation of OCT images is proposed. In clinical use, simply taking a screenshot of the OCT image acquired by the SD-OCT scanning device and feeding it into the neural network can achieve end-to-end segmentation results, displaying the inner and outer diameters of blood vessels and the area of ​​the vessel wall. The entire process only requires the doctor to send the screenshot from the scanning device's host computer to the model inference interface, offering advantages such as convenient testing, fast operation, and accurate results. Attached Figure Description

[0029] Figure 1 The OCT images designed in this invention, from left to right, are: a schematic diagram of cross-sectional extraction of region B of the OCT image, a schematic diagram of the location of arteries and veins, and a schematic diagram of the method for measuring the inner and outer diameters of blood vessels;

[0030] Figure 2 This is a basic flowchart of the OCT fundus vessel diameter measurement method in this invention;

[0031] Figure 3 This is a data curve diagram of the full width at half maximum (FMHW) method in this invention;

[0032] Figure 4 This is a schematic diagram of the full width at half maximum (FMHW) results in this invention;

[0033] Figure 5This is a structural diagram of the stage-one network model (U-Net) in this invention;

[0034] Figure 6 This is a structural diagram of the Stage Two Network Model (EAT) in this invention;

[0035] Figure 7 This is a structural diagram of the EAB module in the network model EAT of this invention. Detailed Implementation

[0036] The embodiments of the present invention will be described in detail below. Although the present invention will be described and illustrated in conjunction with some specific embodiments, it should be noted that the present invention is not limited to these embodiments. On the contrary, any modifications or equivalent substitutions made to the present invention should be covered within the scope of the claims of the present invention.

[0037] Furthermore, to better illustrate the present invention, numerous specific details are set forth in the following detailed embodiments. Those skilled in the art will understand that the present invention can be practiced without these specific details.

[0038] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0039] The device used for OCT image acquisition is the Spectralis SD-OCT scanning device from Heidelberg GmbH, Germany. It is a frequency-domain optical coherence tomography (OCT) system capable of high-resolution, high-speed cross-sectional imaging of the retina.

[0040] like Figure 1 As shown, OCT images can be used to measure the lumen diameter and outer diameter of retinal arterioles.

[0041] The hardware environment used for algorithm testing is an Intel(R) Xeon(R) CPU E5-1650 v4 @ 3.60GHz GPU NVIDIA TITAN XP, and the runtime environment is Python 3.6 and related extension packages PyTorch.

[0042] The core algorithms of the model include deep convolutional neural networks, edge feature extraction, self-attention mechanism, and data fusion.

[0043] In particular, experiments have demonstrated that the Transformer model exhibits powerful feature extraction and discovery capabilities in both Natural Language Processing (NLP) and Computer Vision, particularly regarding self-attention mechanisms. The core algorithm of the Transformer lies in its extensive use of self-attention mechanism modules to fully extract information from the input. Therefore, our method utilizes numerous self-attention structures in both the two-stage EAT model, from the PVT architecture adopted by the encoder to the design of the EAT decoder.

[0044] like Figure 2 As shown, the basic process of this invention first involves calibrating fundus blood vessel data based on the half-peak full width (FMCW) algorithm, then training the U-Net and EAT models, performing two-stage extraction on the fundus OCT images, and finally performing digital image processing and analysis on the segmentation results to obtain the final blood vessel diameter. This includes data preprocessing, FWHM method data calibration, convolutional neural networks, Transformer networks and their variants, and digital image processing.

[0045] The specific steps are as follows:

[0046] The original OCT image is cropped to obtain a grayscale image with a size of 384*384. The pixel values ​​of the data are then normalized to the 0 and 1 range to accelerate model training.

[0047] The formula for grayscale transformation normalization is:

[0048]

[0049] In the formula, I(i,j) and N(i,j) represent the gray values ​​of the original image and the transformed image, respectively, and min and max represent the minimum and maximum gray values ​​of the original image, respectively.

[0050] To obtain a more precise edge region of the blood vessel center, this invention uses the full width at half maximum (FMHW) method, referencing... Figure 3 , 4 The specific implementation algorithm is as follows:

[0051] 1. Difference (differentiation), such as Figure 3 As shown in the topmost curve, this curve is a series of pixel values ​​obtained longitudinally along the effective location of the blood vessel. The data of the next time series is subtracted from the data of the previous time series using backward difference to obtain the difference (derivation) information at the center of the blood vessel.

[0052] 2. Differential sequence positive and negative grouping: For the differential information in step 1, divide it into several groups of positive and negative adjacent sequences with 0 as the discontinuity point.

[0053] 3. Pair up the negative and positive groups, starting with the negative interval (if the starting point is positive, discard it) and forming a group with the positive interval on the right (if the last interval is negative, discard it).

[0054] 4. Calculate the highest point of difference, and sum the absolute values ​​of each group of positive and negative peaks, such as... Figure 3 As shown by the curve below. Then, in the left and right halves of the image, we can find the pair with the largest difference. The coordinates corresponding to the peak value are the coordinates of the vessel wall edge in step 1, as shown below. Figure 4 As shown.

[0055] 5. By selecting appropriate intervals in the left and right regions of the center and repeating the above algorithm, the edge information of the blood vessel wall in the key area can be obtained.

[0056] This method expands the dataset by introducing affine transformations, thereby improving the model's performance and generalization ability. Affine transformations involve changes in the shape, position, and angle of an image, including operations such as scaling, rotation, shearing, flipping, and translation. Scaling changes the image size, rotation adjusts the image angle, shearing changes the image shape, flipping produces a mirror effect, and translation moves the image position. We apply these transformation operations to the original image to generate diverse image samples, thus improving the model's robustness and generalization ability.

[0057] In the first stage of the model, we use the U-Net model based on a convolutional neural network to perform coarse segmentation of the blood vessel segmentation image. For example... Figure 5 As shown, the U-Net architecture consists of an encoder and a decoder. The former generates high-level features of the image, while the latter reconstructs the segmentation result based on this representation. For the same spatial resolution from encoder to decoder, the model uses two mappings connected together (gray arrows) to deepen the model's memory of shallow features.

[0058] Edge Attention Transformer (EAT). The overall design of EAT is as follows: Figure 6 As shown, the model consists of PVT (Pyramid Vision Transformer) as the encoder, EAB (Edge-Aware Block) as the feature enhancement part, and PFFM (Pyramid Future Fusion Model) as the multi-scale feature fusion part. The core of the first two is the self-attention mechanism.

[0059] Edge-Aware Block (EAB). For example... Figure 7As shown, EAB adopts an encoder-decoder approach. The main algorithms used within the module are self-attention mechanism, cross-attention mechanism, and linear neural network algorithm, which can further extract features. In particular, the decoder part of EAB adds edge supervision signals with the same dimension as the input features. In the decoding part, the model considers the impact of edge information on the final segmentation result, thereby enhancing image features and improving the accuracy of image segmentation.

[0060] Self-attention mechanisms, also known as scaled dot product attention structures, are used in this context. Figure 5 As shown, this structure has three input features: the query vector Q, the key K, and the value V. Q and K can be used to calculate an attention score matrix, which is used to refine the data value V. The formula is as follows:

[0061]

[0062] In the formula, d is the dimension of the query vector Q and the value vector K.

[0063] If K, Q, and V are not exactly the same, the above formula evolves into a cross-attention mechanism. In this application, K and Q are image features extracted by the model, while V is the edge feature of the blood vessel.

[0064] The essence of a linear neural network (also known as a multilayer perceptron, MLP) is linear matrix operations. The main forward computation formula is as follows:

[0065]

[0066]

[0067] In the formula w i Here, x represents the model weights, z represents the intermediate variables, y represents the output matrix, and sign is an activation function that can be replaced by other nonlinear functions.

[0068] The hierarchical feature fusion module PFFM (Pyramid Future Fusion Model) is used to fully utilize image features at each stage. Two fusion methods are employed to fuse the multi-scale image features obtained from the EAB module from bottom to top, resulting in a segmentation result of the OCT image: two binary images (internal and external vessels) labeled with vessel locations. The fusion methods are convolutional fusion (Conv Fusion) and depthwise convolutional fusion (DW-Conv Fusion).

[0069] The formula for Conv Fusion is as follows:

[0070] F fusion =Conv{Concat{F Higgh ,φ(F Low )}}

[0071] In the formula, F High F Low These represent high-dimensional and low-dimensional features, respectively. φ is the interpolation algorithm, Concat is the concatenation operation between two vectors, and Conv is the convolution operator.

[0072] The formula for Deep Convolutional Fusion (DW-Conv Fusion) is as follows:

[0073] F fusion =f{F′ High +F′ Low}

[0074]

[0075]

[0076] In the formula, F High F Low These are high-dimensional and low-dimensional features, respectively. These are two different 1*1 convolutional layers, where f is the activation function, φ is the interpolation algorithm, and DWConv is the depthwise convolution operator.

[0077] After training the model, simply input the portion of the OCT image containing the vascular region into the model for inference to obtain fine segmentation maps of the intravascular and extravascular regions and the vascular wall region. Useful noise in the results is filtered out using digital image processing methods such as connected component calculation, closing operations, and feature filtering, and contour information is calculated. Finally, the clinically required vascular-related parameters are provided.

[0078] like Figure 3 As shown, the topmost curve is a series of pixel change curves obtained vertically from the center position of the OCT image, and below it is the result after second-order difference of pixel values. The four pixel ordinates determined by FMHW have been marked in the image.

[0079] like Figure 4 As shown, the inner and outer diameter edges of a certain column of pixels can be calculated mathematically using the FMHW algorithm.

[0080] like Figure 5 As shown, the model used in the first stage generally presents a "U" shape, performs coarse segmentation of the blood vessel region, and the dimensions of the images before and after the model do not change.

[0081] like Figure 6As shown, the model used in the second stage is EAT. In the overall architecture of the EAT model, the PVT block extracts features at different levels, the EAB block uses prior supervision signals to enhance high-dimensional image features, and PFFM is used to fuse features of different dimensions.

[0082] like Figure 7 As shown, the EAB module adopts an overall encoding-decoding architecture, but the decoding module adds edge information of the image to enhance image features.

[0083] The above description is only a preferred embodiment of the present invention. It should be noted that those skilled in the art can make several improvements and modifications without departing from the concept of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. An automatic method for measuring blood vessel diameter in fundus OCT images, comprising the following steps: Step 1, Data Labeling: The full width at half maximum (FWHM) measurement method is used to pre-estimate the inner and outer diameters of the blood vessel. At the same time, the pixels within 70% of the width from the center of the blood vessel to the left and right are scanned column by column to obtain the edge information of the core position of the blood vessel. The relevant peripheral blood vessel contour information is then labeled to obtain the labeled image. Step 2, Data Preprocessing: Cropping out the regions containing retinal blood vessels from the original image, and then normalizing the cropped OCT image; Considering that the acquired images are single-channel grayscale images, grayscale image normalization is adopted, and the grayscale values ​​of the pixels are distributed between 0 and 255 to avoid insufficient image contrast, i.e., unbalanced distribution of image pixel brightness, and at the same time to speed up the training speed of the network. Step 3, Data Augmentation: Perform affine transformations on the labeled images obtained after Step 2 to augment the dataset. The data augmentation methods include random horizontal / vertical flipping, random rotation, and edge sharpening. Step 4, Two-stage model: A two-stage model is used to complete the coarse segmentation task and the fine segmentation task of the OCT image respectively. In the first stage, the approximate area of ​​the blood vessels is obtained through the U-Net neural network, which completes the coarse segmentation task. In the second stage, the edge attention network is used to achieve accurate segmentation of the blood vessel area, which completes the fine segmentation task. The edge attention network has an overall architecture of an encoder-decoder structure. The encoder uses a PVT model to divide the image into blocks and initially extract image features, resulting in four multi-scale features ranging from low-dimensional to high-dimensional. The decoder incorporates an edge-aware block (EAB), which employs an encoder-decoder structure. Through self-attention, cross-attention, and linear neural network algorithms, it enables further feature extraction. The decoder portion of the edge-aware block incorporates an edge supervision signal of the same dimension as the input features. This considers the impact of edge information on the final segmentation result, using edge information as a supervision signal to further enhance the encoded image features, resulting in enhanced multi-scale features. The supervision signal for edge information is extracted using distance transform. Simultaneously, the final step of decoding uses a hierarchical data fusion module to fuse the enhanced multi-scale features from bottom to top, yielding the final segmentation result. Step 5: Blood vessel diameter measurement. The second-stage segmentation result is filtered using digital image processing methods to remove irrelevant noise. The final segmentation result enables measurement of the inner and outer diameters of blood vessels and calculation of the vessel wall area. The results are used for subsequent clinical analysis.

2. The method according to claim 1, characterized in that, The full width at half maximum (FWHM) measurement method, specifically implemented using the following algorithm: Step a1. Differentiation: Obtain a column of pixel values ​​vertically along the middle of the blood vessel, and use backward difference to subtract the data of the previous row from the data of the next row to obtain the difference information at the center. Step a2. Differential sequence positive and negative grouping: For the differential information in step a1, divide it into several groups of positive and negative adjacent sequences with 0 as the discontinuity point; Step a3. Pair up the positive and negative groups, and form a group with the positive value interval to the right of the negative value interval as the starting point. If the starting point is positive, discard it. Step a4. The point with the highest difference: Calculate the sum of the absolute values ​​of each group of positive and negative values. You will be able to find the largest group of differences in the left and right halves of the image. The coordinates of the peak value are the boundary where the blood vessel wall is located in step 1, which is to obtain the edge information. Step a5. Repeat the above algorithm within a suitable range on the left and right sides of the central region to obtain the edge information of the blood vessel wall in the key region.

3. The method according to claim 1, characterized in that, The normalization formula for OCT vascular images in step 2 is: In the formula, I(i, j) and N(i, j) represent the gray values ​​of the original image and the transformed image, respectively, and min and max represent the minimum and maximum gray values ​​of the original image, respectively.

4. The method according to claim 1, characterized in that, In step 4: The U-Net network is used to coarsely segment the input OCT image. The structure of the U-Net model consists of an encoder and a decoder. The core algorithm of the encoder is convolution operation, and the core algorithm of the decoder is interpolation algorithm. The OCT image is fed into the encoder to generate high-level features of the image step by step. In the decoder, the generated high-level features are gradually fused with the features of the previous layer to reconstruct the segmentation result, and finally coarse segmentation of the OCT image is achieved. The encoder and decoder have the same feature dimension, and the model uses identity mapping to connect features of the same dimension together, deepening the model's memory of shallow features.

5. The method according to claim 1, characterized in that, In order to utilize the image features at each stage, the hierarchical feature fusion module uses two fusion methods to fuse the multi-scale image features obtained from the edge sensing block from bottom to top. After fusion, the segmentation result of the OCT image is obtained, which is two binary images that respectively mark the positions of the internal and external blood vessels. The two fusion methods are convolutional fusion and depthwise convolutional fusion.

Images