Direction-guided OCTA image vessel segmentation method

Abstract

This invention relates to a direction-guided OCTA image vessel segmentation method, belonging to the fields of computer vision and medical image processing. The method first constructs a directional convolution, which extracts the directional information of the target, enhancing the contrast between small blood vessels and noise. Then, a direction-guided module is constructed to aggregate features based on directional information, enhancing the feature representation of small blood vessels and noise. Finally, a direction-guided network is constructed, which can accurately segment blood vessels under the guidance of vessel orientation. This invention fully utilizes the directional characteristics of blood vessels in OCTA images, effectively addressing the challenges commonly encountered in OCTA images, such as complex vascular networks, variable orientations, and difficulty in distinguishing between small blood vessels and noise. This method effectively distinguishes small blood vessels from noise, while enabling local vascular regions to perceive the overall structure of the blood vessels, alleviating the problems of vessel discontinuity, missed detections, and false detections.

Description

Technical Field

[0001] This invention relates to a direction-guided OCTA image vessel segmentation method, belonging to the fields of computer vision and medical image processing. This method effectively addresses the challenges commonly encountered in OCTA images, such as complex vascular networks, variable orientations, and difficulty in distinguishing small vessels from noise. This invention designs a direction-guided network that extracts the target's orientation information based on directional convolution and aggregates target features with similar orientation information based on the direction-guided module. This enhances the contrast between small vessels and noise, enabling local vascular regions to perceive the overall structure of the blood vessels and mitigating the problems of vessel discontinuity, missed detections, and false detections. Background Technology

[0002] Optical coherence tomography (OCTA) is a non-invasive imaging technique widely used to display high-resolution retinal vessels and provide detailed visualization of the vascular system. OCTA is of great significance for the diagnosis and treatment of various vision-related diseases. Retinal vessel segmentation is one of the important research topics in computer vision, as it can assist doctors in diagnosing retinal and blood-related diseases. However, due to the low contrast between small vessels and noise, segmentation results may result in vessel discontinuities, missed detections, and false detections. Furthermore, the complex structure of retinal vessels exhibits different orientations and sizes, posing a significant challenge to segmentation. These variations in orientation and size make accurate detection of the boundaries of each vessel difficult. While existing methods analyze the characteristics of vessels in OCTA images, most focus only on semantic feature extraction, neglecting specific structural and feature information such as tubular structures and orientation. Some methods use deformable convolution and its variants to introduce additional offset parameters during convolution to adapt to vessel deformation and pose changes. However, these methods may not be able to describe the deformation of small vessels. On the other hand, some methods use shape-constrained loss functions to guarantee vessel connectivity. These loss functions only guarantee the overall connectivity of the vascular network and cannot establish connections between locally disconnected small vessels.

[0003] Existing processing methods have achieved good results in extracting multi-scale features, feature fusion, and large blood vessel segmentation. However, they perform poorly in noisy images and images with signal occlusion, and suffer from problems such as missed detection, false detection, and discontinuity of small blood vessels. Summary of the Invention

[0004] The purpose of this invention is to overcome the aforementioned shortcomings of the prior art and provide a direction-guided OCTA image vessel segmentation method. To this end, this invention adopts the following technical solution:

[0005] 1. This invention provides a direction-guided OCTA image vessel segmentation method, the method comprising the following steps:

[0006] Step 1: Construct the dataset using the public datasets OCTA-500 and ROSE-1, and divide the data into training, validation, and test sets;

[0007] Step 2: Perform data augmentation using random rotations;

[0008] Step 3: Construct directional convolution to extract directional information and enhance the contrast between fine blood vessels and noise;

[0009] Step 4: Construct a direction-guided module to aggregate features by direction, enhancing the feature representation of small blood vessels and noise;

[0010] Step 5: Construct a directional guidance network to achieve vessel segmentation in OCTA images;

[0011] Step 6: Train the network model, using Dice and cross-entropy as loss functions, Adam as the optimizer, and update the network parameters through the backpropagation algorithm;

[0012] Step 7: Input the OCTA image into the trained directional guidance network to obtain the output segmentation result.

[0013] 2. Based on the directional distribution of blood vessels, we constructed directional convolution to extract the directional information of the target, increasing the inter-class difference between small blood vessels and noise, and reducing the intra-class difference between small blood vessels and large blood vessels. The directional convolution includes average pooling, directional filtering, normalization, and depthwise separable convolution. First, average pooling is performed on the input:

[0014] X m =Avgpool(X) (1)

[0015] Where X is the input of the directional convolution, Avgpool is the average pooling operation along the channel dimension, and then X is... m Perform the following operations:

[0016]

[0017] Where d i Let i be the directional filter corresponding to direction i, where i takes values ​​from 1, 2, ..., 12. The directional filter value consists of 0 and 1. * represents the convolution operation, and ~ represents the logical NOT operation, thus obtaining the feature of direction i. Then perform the following operations on it:

[0018]

[0019] Where concat represents the concatenation operation along the channel dimension, and then... Perform the following operations:

[0020]

[0021] Norm represents the normalization operation, and DSConv represents depthwise separable convolution, which includes 3×3 depthwise convolution and 1×1 pointwise convolution, ultimately yielding orientation information D.

[0022] After directional convolution, we obtain the orientation information of the target. By extracting the orientation information of the target, we can effectively distinguish between small blood vessels and noise.

[0023] 3. Construct a direction guidance module to aggregate features by direction. First, extract direction information from the input:

[0024] D = DC(X) (5)

[0025] Where DC represents directional convolution, and to facilitate feature aggregation, the dimensions of D and X are transformed:

[0026] D′,X′=Img2Seq(D,X) (6)

[0027] Where Img2Seq represents the Reshape operation, which converts the 2D feature map into a 1D vector, and then further maps D′ and X′:

[0028] Q, K = Linear(D′) (7)

[0029] V = Linear(X′) (8)

[0030] Where Linear represents a fully connected layer, the orientation information and input features are mapped to Q, K and V respectively, and then it performs the following operations:

[0031]

[0032] Where Softmax represents the Softmax function, fa represents the length of Q, T represents the transpose operation, and then V′ is dimensionally transformed and added to the input X to obtain the final output:

[0033] output = Seq2Img(V′) + X (10)

[0034] Seq2Img represents the Reshape operation, which converts a 1D vector into a 2D feature map.

[0035] The beneficial effects of this invention are:

[0036] 1. This invention extracts the directional information of the target through directional convolution, increases the contrast between small blood vessels and noise, and can effectively distinguish between small blood vessels and noise.

[0037] 2. This invention uses a direction guidance module to aggregate features of similar directions, enabling local vascular regions to perceive the overall structure of blood vessels. This allows the network to make judgments based on the overall structure of blood vessels rather than local vascular regions, while also enhancing the feature representation of small blood vessels and noise.

[0038] 3. This invention can not only improve the continuity of blood vessels and alleviate the problem of missed detection of blood vessels, but also reduce the false detection of blood vessels and effectively improve the blood vessel segmentation effect. Attached Figure Description

[0039] Figure 1 : Overall network structure diagram of the OCTA image vessel segmentation method based on direction guidance.

[0040] Figure 2 : Schematic diagram of directional convolution in a directional-guided OCTA image vessel segmentation method.

[0041] Figure 3 : Schematic diagram of the direction guidance module in the direction-guided OCTA image vessel segmentation method.

[0042] Figure 4 : Overall flowchart of the OCTA image vessel segmentation method based on direction guidance. Detailed Implementation

[0043] The overall network structure diagram of this invention is as follows: Figure 1 As shown, this invention provides a direction-guided network with a U-shaped structure, including an encoder and a decoder. The network first extracts preliminary features through basic convolutional blocks composed of convolutions, Dropout, batch normalization, Swish activation functions, and residual connections. Then, it extracts the target's directional information through directional convolutions, enhancing the contrast between the foreground and background. Next, the direction-guided module aggregates features according to direction, enabling local features to perceive the features of the entire blood vessel, enhancing the feature representation of blood vessels and noise. Upsampling is performed through transposed convolutions, and the segmentation result is output using a sigmoid function. Cross-entropy and Dice are used as loss functions, and the network parameters are updated through backpropagation. The specific implementation process of the technical solution of this invention is described below with reference to the accompanying drawings.

[0044] 1. The network has a U-shaped structure, including an encoder and a decoder.

[0045] Network structure and partial module structure, such as Figure 1As shown, the network has a U-shaped structure, including an encoder and a decoder. In the encoder, the input image first passes through a basic convolutional layer to obtain the encoder features of the first layer. The basic convolutional layer includes a 3×3 convolution, Dropout, normalization, and a Swish activation function immediately following the residual connection. The encoder features of the first layer are then passed through a convolutional block and a direction-guided module to obtain the encoder features of the second layer. The convolutional block includes a 2×2 max-pooling layer, two basic convolutional layers, and a Swish activation function. The encoder features of the second layer are then passed through a convolutional block and a direction-guided module to obtain the encoder features of the third layer. The encoder features of the third layer are then passed through a convolutional block to obtain the encoder features of the fourth layer.

[0046] In the decoder, the features of the fourth-layer encoder and the third-layer encoder are concatenated along the channel dimension. The concatenated features are then passed through an upsampling convolutional block and a directional guidance module to obtain the third-layer decoder features. The upsampling convolution includes a transposed convolution, two basic convolutional layers, and a Swish activation function. The features of the third-layer decoder and the second-layer encoder are concatenated along the channel dimension and then passed through an upsampling convolutional block and a directional guidance module to obtain the second-layer decoder features. The features of the second-layer decoder and the first-layer encoder are then concatenated and passed through an upsampling convolutional block to obtain the first-layer decoder features. The first-layer decoder features are then passed through an output layer to obtain the final predicted segmentation result. The output layer includes two 1×1 convolutions, batch normalization, and a Sigmoid activation function.

[0047] 2. Extract the target's orientation information through oriented convolution.

[0048] Even low-contrast, minute blood vessels retain directional characteristics. Regions where blood vessels are almost severed also exhibit a similar orientation. This invention utilizes directional convolution to effectively extract this feature. The directional convolution structure is as follows: Figure 2 As shown, directional convolution first undergoes an average pooling operation along one channel dimension:

[0049] X m =Avgpool(X) (11)

[0050] Where X is the input to the directional convolution, and Avgpool is the average pooling operation along the channel dimension; the directional convolution includes directional filtering to extract directional information, with 12 directional filters corresponding to 12 directions, and the angle between any two consecutive directions is 15°, with each directional filter having a size of 7×7; for each directional filter d... i Pixel values ​​along a specific directional angle (i×15°) are set to 1, while others are set to 0; directional filtering d is used. i For X m Perform a convolution operation and divide the result by the directional filter d.i The number of pixels with a median value of 1 is used to calculate the average value, which is defined as the blood vessel feature in the i-th direction; directional filtering d is applied. i Perform a logical NOT operation and repeat the above steps to obtain non-vascular features. To distinguish them from omnidirectional noise, the non-vascular features need to be subtracted from the vascular features; directional features... The calculation is as follows:

[0051]

[0052] Where d i This represents the directional filtering corresponding to the i-th direction, * represents the convolution operation, and ~ represents the logical NOT operation; concatenating the 12 directional features forms a 12-dimensional directional feature. It contains information from 12 directions:

[0053]

[0054] Where concat represents concatenation within the channel dimension; for Standardization is performed to ensure data stability. During orientation information extraction, some pixels in blood vessels may be assigned incorrect orientations that differ from their true orientations. Depthwise separable convolution (DSConv) is used to correct misaligned pixels. DSConv learns spatial relationships in a certain direction through 3×3 depthwise convolution and adjusts the relationships between different directions using 1×1 pointwise convolution, thereby reorganizing and adjusting the representation of different directions in the orientation map to correct incorrect orientations and enhance generalization ability. The orientation map D is calculated as follows:

[0055]

[0056] Norm represents normalization, and DSConv represents depthwise separable convolution. DC can transform the feature representation of a feature map into the directional representation of a direction map, thereby enhancing the contrast between small blood vessels and noise.

[0057] 3. Aggregate features by direction using the direction guidance module.

[0058] Blood vessels are distributed in a certain direction, while noise is generally randomly distributed. Therefore, this invention constructs a direction guidance module, the structure of which is as follows: Figure 3 As shown, the orientation guidance module establishes the correlation between blood vessels and noise from a directional perspective, enabling the network to distinguish between blood vessels and noise while also considering the directional features of blood vessels. The orientation guidance module can aggregate vascular features with similar orientations, thereby enhancing the feature representation of low-contrast vascular regions and improving vascular continuity. The orientation guidance module obtains directional features by performing directional convolution on the input and then performs shape transformation on the directional features and the input.

[0059] D = DC(X) (15)

[0060] D′,X′=Img2Seq(D,X) (16)

[0061] Where DC represents directional convolution, and Img2Seq represents the reshaping operation that converts 2D feature maps into 1D labels; directional features are mapped to query and key feature representations, and input is mapped to value feature representations.

[0062] Q, K = Linear(D′) (17)

[0063] V = Linear(X′) (18)

[0064] Where Linear represents a fully connected layer, and Q, K, and V represent query, key, and value, respectively; the mapping of directional features is calculated by using a dot product and the Soffmax function to obtain a directional similarity matrix. The directional similarity matrix is ​​multiplied by the input mapping to re-represent the input features. The features are then shape-transformed and added to the input. The calculation process is as follows:

[0065]

[0066] output = Seq2Img(V′) + X (20)

[0067] Where Softmax represents the Softmax function, and Seq2Img represents the reshaping operation that changes 1D labels to 2D feature maps.

[0068] This invention proposes a direction-guided OCTA image vessel segmentation method. Based on the directional characteristics of blood vessels, this method uses directional convolution to extract directional information and uses a direction-guided module to aggregate features by direction, effectively distinguishing small blood vessels from noise and alleviating the problems of blood vessel discontinuity, missed detection, and false detection.

Claims

1. A method for vessel segmentation in OCTA images based on direction guidance, characterized in that, The method includes the following steps: Step 1: Construct the dataset using the public datasets OCTA-500 and ROSE-1, and divide the data into training, validation, and test sets; Step 2: Perform data augmentation using random rotations; Step 3: Construct directional convolution to extract directional information and enhance the contrast between fine blood vessels and noise; Step 4: Construct a direction-guided module to aggregate features by direction, enhancing the feature representation of small blood vessels and noise; Step 5: Construct a directional guidance network to achieve vessel segmentation in OCTA images; Step 6: Train the network model, using Dice and cross-entropy as loss functions, Adam as the optimizer, and update the network parameters through the backpropagation algorithm; Step 7: Input the OCTA image into the trained directional guidance network to obtain the output segmentation result; In step 3, based on the directional distribution characteristics of blood vessels, directional convolution is constructed to extract the directional information of the target. Directional convolution includes average pooling, directional filtering, normalization, and depthwise separable convolution. First, average pooling is performed on the input: （1） in The input for directional convolution is... For channel-level average pooling, then perform... Perform the following operations: （2） in For direction The corresponding directional filter, The value of is 1, 2...12, and the value of the directional filter consists of 0 and 1. The convolution operation is represented by ~, which represents the logical NOT operation, yielding the direction. Features Then perform the following operations on it: （3） in This indicates a concatenation operation along the channel dimension, followed by... Perform the following operations: （4） in This indicates a normalization operation. This represents depthwise separable convolution, which includes 3×3 depthwise convolution and 1×1 pointwise convolution, ultimately yielding orientation information D; In step 4, the direction guidance module aggregates features by direction, first extracting direction features from the input: （5） in This represents directional convolution, used to facilitate feature aggregation. and Transformation Dimensions: （6） in Convert the 2D feature map into a 1D vector, and then... and Further mapping: （7） （8） in This represents a fully connected layer where directional information and input features are mapped to... , and Then perform the following operations on it: （9） in, This represents the softmax function. express Length, This indicates the transpose operation, and then... After dimensional transformation and input Adding them together gives the final output: （10） in This indicates that a 1D vector is converted into a 2D feature map.

Images