System for arteriovenous segmentation of retinal vascular images

Abstract

The application provides an arteriovenous segmentation system suitable for a retinal blood vessel image, a feature extraction module is used for extracting first high-dimensional features of the retinal blood vessel image; a deformable feature propagation block is used for receiving the first high-dimensional features and outputting third high-dimensional features; a classification module is used for obtaining a target probability distribution map according to the third high-dimensional features; and a morphology reconstruction module is used for obtaining an arterial blood vessel region and a venous blood vessel region based on the target probability distribution map. The system processes the first high-dimensional features of the retinal blood vessel image through the deformable feature propagation block, so that the third high-dimensional features outputted contain more abundant local topological information, the accuracy of the target probability distribution map is improved, and the fidelity of the blood vessel region obtained through reconstruction is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image data processing, and in particular to a system for arteriole and venule segmentation of retinal blood vessel images. BACKGROUND

[0002] The morphological features of retinal blood vessels can provide evidence for the pathogenesis, disease characteristics and treatment effect monitoring of eye diseases and systemic diseases. More and more longitudinal evidence shows that the morphological changes of retinal blood vessels are early physiological markers of cardiometabolic risk and outcomes. At the same time, many diseases have different effects on retinal arteries and veins: narrow arterial vessels are related to hypertension; wider veins are related to diabetes, elevated glycosylated hemoglobin, smoking and obesity; retinal arterioles and venules are also involved in different retinal vascular disease spectrum. Therefore, it is particularly important to distinguish arteries and veins in retinal images. In order to obtain robust and reliable features, image segmentation algorithms need to be designed to predict arterial and venous vessel regions. Most existing methods are based on general image segmentation neural networks and use loss functions based on pixel-by-pixel comparison for training, ignoring the fidelity of the vascular topology. Therefore, using image segmentation neural networks for arteriole and venule segmentation often leads to topological errors, which are often more serious when OCTA images have breaks, blurring or signal attenuation caused by artifacts. SUMMARY

[0003] The present application aims to provide a system for arteriole and venule segmentation of retinal blood vessel images to improve the fidelity of the reconstructed blood vessel regions.

[0004] The present application provides a system for arteriole and venule segmentation of retinal blood vessel images, which is generated by optical coherence tomography blood vessel imaging. The system comprises a feature extraction module, a feature enhancement module, a classification module and a morphological reconstruction module connected in sequence. The feature enhancement module includes a deformable feature propagation block. The feature extraction module is used to extract features from the received retinal blood vessel image to obtain the first high-dimensional features of the retinal blood vessel image. The deformable feature propagation block is used to receive the first high-dimensional features and output the third high-dimensional features. The classification module is used to map the target probability distribution map according to the third high-dimensional features. The target probability distribution map includes an arterial probability distribution map and a venous probability distribution map. The morphological reconstruction module is used to reconstruct the target blood vessel region of the retina based on the target probability distribution map. The target blood vessel region includes an arterial blood vessel region and a venous blood vessel region.

[0005] Further, the number of the deformable feature propagation blocks is multiple; the multiple deformable feature propagation blocks are sequentially connected; each of the deformable feature propagation blocks is configured to perform deformable convolution operation on the received second high-dimensional feature, and output a third high-dimensional feature; wherein the second high-dimensional feature received by the first deformable feature propagation block is the first high-dimensional feature; the second high-dimensional feature received by each deformable feature propagation block other than the first deformable feature propagation block is the third high-dimensional feature output by the previous deformable feature propagation block; and the classification module is configured to map the third high-dimensional feature output by the last deformable feature propagation block to obtain a target probability distribution map.

[0006] Further, each of the deformable feature propagation blocks in the feature enhancement module is further connected with an attention gate module; for each deformable feature propagation block connected with an attention gate module, the attention gate module is configured to determine a weight coefficient according to the second high-dimensional feature received by the deformable feature propagation block and the fourth high-dimensional feature output by the deformable feature propagation block; and the second high-dimensional feature and the fourth high-dimensional feature received by the deformable feature propagation block are weighted and calculated according to the weight coefficient to obtain the third high-dimensional feature.

[0007] Further, the morphology reconstruction module is further configured to: assign a weight value to each pixel point in the target probability distribution map based on a preset weight function and a probability value corresponding to each pixel point in the target probability distribution map; wherein the greater the probability value of a pixel point, the smaller the corresponding weight value; for each seed point in a preset seed point set, calculate a minimum weight path from the seed point to each pixel point in the target probability distribution map using a shortest path algorithm; wherein the seed point set includes: a boundary pixel point of the target probability distribution map, and / or a pixel point in a cup region; determine a pixel dependency graph based on each minimum weight path; and determine a target blood vessel region based on the pixel dependency graph.

[0008] Further, the weight function is as follows:

[0009]

[0010] wherein, p ij represents a probability value corresponding to a pixel point with pixel coordinates (i, j); w ij represents a weight value corresponding to a pixel point with pixel coordinates (i, j); θ represents a preset threshold; and ∈ is a preset positive value.

[0011] Further, the morphological reconstruction module is further configured to: assign a selection return value to each pixel point in the target probability distribution map based on a preset threshold and a probability value corresponding to each pixel point in the target probability distribution map; wherein the greater the selection return value, the greater the probability of the pixel point being classified as a blood vessel; and solve a target closed subgraph from the pixel dependency graph based on the pixel dependency graph and the selection return value corresponding to each pixel point, by using a min-cut algorithm; wherein the sum of the selection return values of the pixel points corresponding to the target closed subgraph is the largest; and determine the target blood vessel region based on the pixel points corresponding to the target closed subgraph.

[0012] Further, the morphological reconstruction module is further configured to: calculate a difference value between the probability value corresponding to each pixel point in the target probability distribution map and the preset threshold, to obtain a difference result; and determine the selection return value corresponding to the pixel point as the difference result.

[0013] Further, each deformable feature propagation block comprises a deformable convolution layer, and a batch normalization layer, a nonlinear activation layer and an ordinary convolution layer connected with the deformable convolution layer, respectively.

[0014] Further, each attention gate module comprises an ordinary convolution layer and a nonlinear activation layer connected in sequence.

[0015] Further, the feature extraction module is further configured to: perform feature extraction on the received retinal blood vessel image by using a pre-trained neural network model, to obtain a first high-dimensional feature of the retinal blood vessel image.

[0016] The system for segmenting arterioles and venules of a retinal blood vessel image provided by the application comprises a feature extraction module configured to extract a first high-dimensional feature of a retinal blood vessel image; a deformable feature propagation block configured to receive the first high-dimensional feature and output a third high-dimensional feature; a classification module configured to obtain a target probability distribution map based on the third high-dimensional feature; and a morphological reconstruction module configured to obtain an arteriole blood vessel region and a venule blood vessel region based on the target probability distribution map. The system processes the first high-dimensional feature of the retinal blood vessel image by using the deformable feature propagation block, so that the third high-dimensional feature output contains more abundant local topological information, improves the accuracy of the target probability distribution map, and further improves the fidelity of the blood vessel region obtained by reconstruction. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the specific embodiments of the application or the technical solutions in the prior art, the following will briefly introduce the drawings needed to be used in the specific embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the application, and those skilled in the art can also obtain other drawings according to these drawings without any creative effort.

[0018] Figure 1A structure schematic diagram of a system for arteriovenous segmentation of retinal blood vessel images is provided for an embodiment of the present application.

[0019] Figure 2 A structure schematic diagram of a feature enhancement module is provided for an embodiment of the present application.

[0020] Figure 3 A structure schematic diagram of a morphology reconstruction module is provided for an embodiment of the present application.

[0021] FIG. 4(a) is a schematic diagram of an arteriovenous segmentation result based on a retinal blood vessel image in the prior art provided for an embodiment of the present application.

[0022] FIG. 4(b) is a schematic diagram of an arteriovenous segmentation result based on U-Net in the prior art provided for an embodiment of the present application.

[0023] FIG. 4(c) is a schematic diagram of an arteriovenous segmentation result provided for an embodiment of the present application.

[0024] FIG. 4(d) is a schematic diagram of a real arteriovenous region provided for an embodiment of the present application.

[0025] Figure 5 A flowchart of a method for arteriovenous segmentation of retinal blood vessel images is provided for an embodiment of the present application. DETAILED DESCRIPTION

[0026] The technical solutions of the present application will be described clearly and completely below in conjunction with embodiments. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative efforts fall within the scope of the present application.

[0027] In related art, from the imaging modalities, most of the traditional methods rely on fundus color photography for arteriovenous segmentation. However, fundus color photography is not sensitive enough to fine blood vessels, especially in the fovea and parafovea. Using fluorescence angiography technology can capture finer blood vessels, but as an invasive technology, it has side effects. Optical coherence tomography angiography (OCTA) technology can non-invasively image retinal microvessels; existing technologies combine fundus color photography and OCTA to segment retinal arteriovenous, but the simultaneous shooting of fundus color photography and OCTA images is cumbersome and requires subsequent complex registration operations, which is not conducive to portable acquisition and analysis of retinal images.

[0028] From the perspective of segmentation algorithm, most of the existing methods are based on general image segmentation neural network and use loss function based on pixel-by-pixel comparison for training, ignoring the fidelity of the blood vessel topology. Therefore, using the image segmentation neural network for arteriovenous segmentation often leads to topological errors (including blood vessel rupture and arteriovenous misconnection); these topological errors are often more serious when the OCTA image has a rupture, blur or signal attenuation caused by artifacts. Based on this, the embodiment of the present application provides an arteriovenous segmentation system suitable for retinal blood vessel images, which can be applied to applications requiring arteriovenous region segmentation of retinal blood vessel images.

[0029] In order to facilitate the understanding of the present embodiment, first, a kind of arteriovenous segmentation system suitable for retinal blood vessel image disclosed in the embodiment of the present application is introduced, and the retinal blood vessel image is generated by optical coherence tomography blood vessel imaging, which can be understood as the image obtained by scanning the retinal blood vessel region based on OCT technology;The retinal blood vessel image is usually a retinal OCTA image, which is a kind of clinical retinal image, which can realize non-invasive three-dimensional retinal imaging of high resolution and blood vessel front view of retinal blood vessels by using the motion signal of flowing blood as internal contrast. Retinal OCTA image is not easy to be affected by medium turbidity and lesion, and can provide high-resolution observation of retinal microvascular network at capillary level. As shown in the figure, Figure 1 As shown, the system comprises: a feature extraction module 10, a feature enhancement module 11, a classification module 12 and a morphology reconstruction module 13 connected in turn;Among them, the feature enhancement module 11 includes a deformable feature propagation block;The feature extraction module 10 is used for feature extraction of the received retinal blood vessel image, to obtain the first high-dimensional feature of the retinal blood vessel image;In actual implementation, the above retinal blood vessel image is usually scanned in advance, and the acquisition process of the retinal blood vessel image usually includes original retinal blood vessel image acquisition, calculation, stratification and other processing, which can refer to the acquisition method of the retinal blood vessel image in the prior art, and will not be described here;The retinal blood vessel image is input into the feature extraction module 10, and the corresponding high-dimensional feature of each pixel point in the retinal blood vessel image is calculated by the feature extraction module 10, which can be understood as some important information or attribute extracted from each pixel point, and the feature dimension is high;The feature extraction module 10 can map the retinal blood vessel image into the first high-dimensional feature, and the dimension of the first high-dimensional feature can be: high of image x width of image x feature dimension.

[0030] The deformable feature propagation block is configured to receive the first high-dimensional feature and output a third high-dimensional feature. Specifically, the deformable feature propagation block performs a deformable convolution operation on the received first high-dimensional feature and outputs the third high-dimensional feature. The deformable feature propagation block learns a displacement field based on the received first high-dimensional feature, and obtains a deformed receptive field using the displacement field. The deformable feature propagation block performs a deformable convolution operation on the received first high-dimensional feature using the deformed receptive field to obtain a third high-dimensional feature with enhanced features corresponding to the deformable feature propagation block. The displacement field can be understood as the displacement of each pixel in the retinal blood vessel image in the x, y, and z directions learned based on the first high-dimensional feature. The receptive field can be understood as the size of the region mapped on the input image by the pixel in the feature map output by each layer of the convolutional neural network of the deformable feature propagation block. The deformed receptive field can be output according to the displacement field. The displacement field and the receptive field can be determined in a manner known in the art. By adding an adaptive learning displacement field (relative to the offset of a regular convolution kernel), the convolution kernel with the displacement field can better adapt to various shapes of input. Furthermore, the pixels are connected to each other, i.e., a larger receptive field is obtained. When searching for image features, more image positions can be associated, and the global information of the input data can be better captured, thereby improving the recognition and detection capabilities of the deformable feature propagation block.

[0031] The classification module 12 is configured to map the third high-dimensional feature to obtain a target probability distribution map. The target probability distribution map includes an arterial probability distribution map and a venous probability distribution map. The third high-dimensional feature with enhanced features output by the deformable feature propagation block is input to the classification module 12, which typically includes a normal convolution layer and a nonlinear sigmoid activation function. The normal convolution layer and the nonlinear sigmoid activation function in the classification module 12 can map the third high-dimensional feature with enhanced features to a two-dimensional feature map, i.e., obtain the arterial probability distribution map and the venous probability distribution map, and pass them to the morphology reconstruction module 13. The mapping method can be known in the art and will not be described here. The probability value corresponding to each pixel in the obtained arterial probability distribution map and venous probability distribution map is typically a value between 0 and 1. The probability value corresponding to each pixel in the arterial probability distribution map can be used to indicate the probability of the pixel being an artery. The probability value corresponding to each pixel in the venous probability distribution map can be used to indicate the probability of the pixel being a vein.

[0032] The morphological reconstruction module 13 is configured to reconstruct a target blood vessel region of the retina based on the target probability distribution map; the target blood vessel region includes an arterial blood vessel region and a venous blood vessel region. The morphological reconstruction module 13 can reconstruct a final arterial blood vessel region based on the arterial probability distribution map output by the classification module 12 under the connectivity constraint; and can reconstruct a final venous blood vessel region based on the venous probability distribution map output by the classification module 12 under the connectivity constraint. The connectivity constraint is described below. Taking two pixel points as an example, two judgment conditions for the connectivity of the two pixel points include whether the positions of the two pixel points are adjacent and whether the gray values of the two pixel points satisfy a specific similarity criterion. The connectivity constraint condition to be met corresponds to the connectivity constraint described above. However, the connectivity constraint condition to be met can be set according to actual requirements, and specific reference can be made to prior art, which is not limited herein.

[0033] The above-described arteriovenous segmentation system for retinal blood vessel images processes the first high-dimensional feature of the retinal blood vessel image through the deformable feature propagation block, so that the third high-dimensional feature output contains more rich local topological information, improves the accuracy of the target probability distribution map, and further improves the fidelity of the blood vessel region reconstructed.

[0034] Further, the number of the deformable feature propagation blocks is multiple; the multiple deformable feature propagation blocks are sequentially connected; that is, the output of a previous deformable feature propagation block is taken as the input of a subsequent deformable feature propagation block, and the deformable feature propagation blocks are sequentially connected in order. The number of the deformable feature propagation blocks can be set according to actual requirements.

[0035] Each deformable feature propagation block is configured to perform deformable convolution operation on the received second high-dimensional feature and output a third high-dimensional feature; wherein the first deformable feature propagation block receives the first high-dimensional feature as the second high-dimensional feature; and the other deformable feature propagation blocks receive the third high-dimensional feature output by the previous deformable feature propagation block as the second high-dimensional feature; and the classification module is configured to map the third high-dimensional feature output by the last deformable feature propagation block to obtain the target probability distribution map.

[0036] The received first high-dimensional features can be enhanced by a plurality of deformable feature propagation blocks in the feature enhancement module 11. Generally, the more the number of deformable feature propagation blocks, the richer the local topological information contained in the third high-dimensional features output by the last deformable feature propagation block. For each deformable feature propagation block, the deformable feature propagation block learns a displacement field based on the received second high-dimensional features, and obtains a deformed receptive field using the displacement field, and performs deformable convolution operation on the received second high-dimensional features using the deformed receptive field to obtain the third high-dimensional features after feature enhancement corresponding to the deformable feature propagation block. The above displacement field can be understood as the displacement amount of each pixel point in the retinal blood vessel image learned based on the second high-dimensional features in the x, y, and z directions. In actual implementation, the third high-dimensional features after feature enhancement output by the last deformable feature propagation block can be input into the classification module 12 to obtain the target probability distribution map.

[0037] Further, each deformable feature propagation block in the feature enhancement module 11 is also connected with an attention gate module. For each deformable feature propagation block connected with an attention gate module, the attention gate module is used to determine a weight coefficient according to the second high-dimensional features received by the deformable feature propagation block and the fourth high-dimensional features output by the deformable feature propagation block; and the second high-dimensional features and the fourth high-dimensional features received by the deformable feature propagation block are weighted and calculated according to the weight coefficient to obtain the third high-dimensional features.

[0038] Specifically, the calculation method of the feature enhancement module 11 can be defined as follows:

[0039]

[0040]

[0041]

[0042] wherein k = 1, 2, 3 … K; represents the kth deformable feature propagation block; represents the fourth high-dimensional features output by the kth deformable feature propagation block, which can also be referred to as candidate features; x (k-1) represents the second high-dimensional features received by the kth deformable feature propagation block, wherein if k = 1, the second high-dimensional features received by the first deformable feature propagation block are the first high-dimensional features output by the feature extraction module 10, and the first high-dimensional features can be represented as x (0) represents that if k = 2, 3 … K, the second high-dimensional features received by the kth deformable feature propagation block are the third high-dimensional features output by the k-1th deformable feature propagation block; G() represents 3x3 convolution plus a Sigmoid activation function, and the output g (k)This represents the weighting coefficient, which is typically a value between 0 and 1; x (k) This represents the third high-dimensional feature output by the attention gate module connected to the k-th deformable feature propagation block.

[0043] The input to G() above consists of two parts: x (k-1) and in: This represents the k-th deformable feature propagation block for x. (k-1) The depth features extracted after processing such as deformable convolution are the aforementioned fourth high-dimensional features. In other words, the third high-dimensional feature output from the (k-1)th deformable feature propagation block and the fourth high-dimensional feature extracted from the current kth deformable feature propagation block are concatenated and input into G(). G() can automatically find a suitable weight coefficient g through training. (k) After obtaining the weight coefficient, the third high-dimensional feature output by the attention gate module connected to the kth deformable feature propagation block can be calculated.

[0044] like Figure 2 The diagram shows a structural schematic of a feature enhancement module 11. This module includes K sequentially connected sub-modules. Each sub-module includes a deformable feature propagation block and an attention gate module, which are interconnected and can be used to control the feature flow between the deformable feature propagation blocks. The feature enhancement module 11 can enhance the first high-dimensional features extracted by the feature extraction module 10, thereby enabling the learned third high-dimensional features to contain richer local topological information.

[0045] Furthermore, the morphological reconstruction module 13 is also used to: assign weights to each pixel in the target probability distribution map based on a preset weight function and the probability value corresponding to each pixel in the target probability distribution map; wherein, the larger the probability value of a pixel, the smaller the corresponding weight; for each seed point in the preset seed point set, use the shortest path algorithm to calculate the minimum weight path from the seed point to each pixel in the target probability distribution map; wherein, the seed point set includes: boundary pixels of the target probability distribution map, and / or, pixels in the optic cup region; determine a pixel dependency map based on each minimum weight path; and determine the target blood vessel region based on the pixel dependency map.

[0046] The aforementioned weighting function can be set according to actual needs. Typically, this weighting function is a decreasing function, meaning that the probability value of each pixel and the weight calculated using this function are negatively correlated. Pixels with higher probability values ​​have smaller weights, and vice versa. A preferred weighting function is as follows:

[0047]

[0048] wherein p ij represents the probability value corresponding to the pixel point with pixel coordinates (i, j); w ij represents the weight value corresponding to the pixel point with pixel coordinates (i, j); θ represents a preset threshold; ∈ is a preset positive value.

[0049] The ∈ is usually a very small positive value close to 0, for example, ∈ = 10 -8 The preset threshold θ can be set according to actual needs, for example, taking θ = 0.5 as an example, when p ij < 0.5, the weight value corresponding to the pixel point is calculated as -log p ij , and when p ij ≥ 0.5, the preset ∈ is taken as the weight value corresponding to the pixel point.

[0050] The boundary pixel points of the target probability distribution map and / or the pixel points in the cup region can be taken as a seed point set. For each seed point, the minimum weight path from the seed point to each pixel point in the target probability distribution map is calculated using a shortest path algorithm. For example, there can be multiple paths from seed point A to pixel point B, each path usually connecting multiple pixel points. The weight values of the pixel points connected by each path can be calculated to determine the minimum weight path from seed point A to pixel point B. The edges on the minimum weight path are found and taken as the edges of the pixel dependency graph. The pixel dependency graph is a graphical representation method for describing the mutual dependency relationship between pixels in an image. It can capture the structure and features of an image by considering the spatial relationship and similarity between pixel points.

[0051] In the pixel dependency graph, each pixel point is represented as a node in the graph, and the dependency relationship between pixel points is represented by edges. These edges can represent the spatial relationship between pixel points, such as the connection between adjacent pixel points, or represent the similarity between pixel points, such as color, texture, or brightness similarity.

[0052] Through the pixel dependency graph, the relationship between pixel points in an image can be analyzed and play an important role in image processing and computer vision tasks. For example, in image segmentation tasks, the similarity between pixel points can be modeled using the pixel dependency graph to better segment the image into different regions. In image generation tasks, the pixel dependency graph can be used to generate realistic images with spatial continuity and consistency. According to the pixel dependency graph, the target blood vessel region can be determined.

[0053] Further, the morphology reconstruction module 13 is further configured to: assign a selection reward value to each pixel point in the target probability distribution map based on a preset threshold and the probability value corresponding to each pixel point in the target probability distribution map; wherein the greater the selection reward value, the greater the probability of the pixel point being classified as a blood vessel; and solve a target closed subgraph from the pixel dependency graph based on the pixel dependency graph and the selection reward value corresponding to each pixel point, wherein the selection reward value corresponding to the pixel points of the target closed subgraph is the maximum sum; and determine the target blood vessel region based on the pixel points corresponding to the target closed subgraph.

[0054] In actual implementation, the selection reward value of each pixel point can be determined according to the formula r ij =r(p ij ) where the function r needs to satisfy p ij >θ is positive, and p ij <θ is negative; further, the morphology reconstruction module 13 is further configured to: calculate the difference between the probability value corresponding to each pixel point in the target probability distribution map and the preset threshold to obtain a difference result; and determine the difference result as the selection reward value corresponding to the pixel point. In actual implementation, to simplify the calculation process and improve the calculation efficiency, the selection reward value of each pixel point in the target probability distribution map can be determined according to the formula r ij =p ij -θ, wherein the difference between the probability value corresponding to each pixel point and the threshold θ is calculated, and the selection reward value corresponding to each pixel point is obtained, for example, the probability value corresponding to the pixel point C is 0.8, and the threshold θ is 0.5, then the selection reward value corresponding to the pixel point C is 0.8-0.5=0.3; the selection reward value corresponding to each pixel point can be used to indicate the probability of the pixel point being classified as a blood vessel, generally, the greater the selection reward value, the greater the probability of the pixel point being classified as a blood vessel, and vice versa, the smaller the selection reward value, the smaller the probability of the pixel point being classified as a blood vessel. Finally, the target closed subgraph is solved from the pixel dependency graph based on the pixel dependency graph and the selection reward value corresponding to each pixel point by using the minimum cut algorithm, so that the total selection reward value is the maximum, which can ensure that the blood vessel region connectivity is strictly met, and the pixel points corresponding to the target closed subgraph are taken as the target blood vessel region obtained by the final segmentation. The minimum cut algorithm can be used to find the minimum cut in the graph, which means that the graph is divided into two disjoint parts, and the sum of the edge weights between the two parts is the minimum; the specific implementation of the minimum cut algorithm can refer to related technologies, which will not be described here; in another implementation, each path in the pixel dependency graph can be traversed, and the selection reward values of the pixel points on each path can be summed to obtain the total sum of the selection reward values corresponding to each path, and the target closed subgraph can be determined according to the path with the maximum total sum of the selection reward values.

[0055] As Figure 3As shown in a structural schematic diagram of the morphology reconstruction module, the morphology reconstruction module 13 receives a target probability distribution map, assigns a value to each pixel point in the target probability distribution map, and constructs a pixel dependency graph for each seed point in a seed point set by using a shortest path algorithm; based on a preset threshold and a probability value corresponding to each pixel point in the target probability distribution map, a selection reward value is assigned to each pixel point in the target probability distribution map, and based on the pixel dependency graph and the selection reward value corresponding to each pixel point, a target closed subgraph is solved from the pixel dependency graph by using a min-cut algorithm, and then a predicted blood vessel region is obtained.

[0056] Further, as shown in the embodiment, each deformable feature propagation block comprises a deformable convolution layer, and a batch normalization layer, a nonlinear activation layer and an ordinary convolution layer connected with the deformable convolution layer respectively. Figure 2

[0057] The deformable convolution layer, the batch normalization layer, the nonlinear activation layer and the ordinary convolution layer can all adopt a common basic structure in a deep learning network. The deformable convolution layer learns a displacement field based on the input high-dimensional features, and obtains a deformed receptive field by using the displacement field, and then performs convolution operation on the input high-dimensional features by using the deformed receptive field. The batch normalization can standardize the received input to have zero mean and unit variance, and can help accelerate the training process of the neural network. The batch normalization layer calculates the mean and variance in each training batch, and standardizes the input using these statistical data. In this way, the neural network is more robust to the scale and distribution changes of the input data, which helps to reduce the problems of gradient vanishing and gradient explosion, and alleviate the overfitting problem.

[0058] The nonlinear activation layer can introduce nonlinear operation by using a nonlinear activation function (such as ReLU, Sigmoid, etc.), so that the neural network can learn more complex and expressive function relationships. This can improve the representation ability of the network, and also helps to solve the problem of gradient vanishing, so that information can flow better in the network. The ordinary convolution layer is mainly used for feature extraction of the received input. The input of the batch normalization layer, the nonlinear activation layer and the ordinary convolution layer is the output of the deformable convolution layer, each layer processes the received input accordingly, and outputs the corresponding calculation result.

[0059] Further, each attention gate module comprises an ordinary convolution layer and a nonlinear activation layer connected in sequence.

[0060] The functions of the ordinary convolution layer and the nonlinear activation layer can be referred to the related descriptions in the foregoing embodiments, which will not be described herein. In this embodiment, the convolution kernel size of all convolution operations can be set as needed, for example, it can be 3x3, etc.

[0061] ​Further, the feature extraction module 10 is further configured to: adopt a pre-trained neural network model to perform feature extraction on the received retinal blood vessel image to obtain first high-dimensional features of the retinal blood vessel image.

[0062] The neural network model can be a convolutional neural network (CNN), a recurrent neural network (RNN), etc. A suitable neural network model can be selected according to actual needs. The neural network model can automatically and quickly extract first high-dimensional features meaningful for arteriovenous segmentation from the retinal blood vessel image. The training process of the neural network model can refer to related technologies, which will not be described here.

[0063] Fig. 4(a) shows a schematic diagram of an arteriovenous segmentation result based on a blood vessel image in the prior art, Fig. 4(b) shows a schematic diagram of an arteriovenous segmentation result based on a U-Net in the prior art, Fig. 4(c) shows a schematic diagram of an arteriovenous segmentation result, and Fig. 4(d) shows a schematic diagram of a real arteriovenous region. Each of the figures includes a large figure and three small figures. Each of the small figures corresponds to a selected region in the corresponding large figure. As can be seen from Figs. 4(a) and 4(b), compared with the real arteriovenous region in Fig. 4(d), it can be seen from the three small figures below each of the figures in Figs. 4(a) and 4(b) that there are problems such as blood vessel rupture and arteriovenous misconnection. Fig. 4(c) shows an arteriovenous segmentation result obtained by using the present scheme. Compared with the real arteriovenous region in Fig. 4(d), it can be seen that, compared with Figs. 4(a) and 4(b), the segmentation result of the present scheme has higher topological fidelity, the phenomenon of blood vessel rupture and arteriovenous misconnection is significantly suppressed, and the predicted arteriovenous segmentation result is more consistent with the real situation.

[0064] The system generally needs to be trained using multiple retinal blood vessel images and labeled arteriovenous region masks, and can predict the arteriovenous region of a new retinal blood vessel image. The feature extraction module 10, the feature enhancement module 11, and the classification module 12 are trained as a whole deep learning model, and a segmentation task loss function can be used during training. After the feature extraction module 10, the feature enhancement module 11, and the classification module 12 are trained, they participate in the prediction process of a new image together with the morphological reconstruction module 13. The above arteriovenous segmentation system for retinal blood vessel images can accurately predict the arteriovenous regions and restore the topological structure of the corresponding blood vessels (especially small blood vessels), thereby achieving accurate segmentation of the arteriovenous regions.

[0065] The system can only segment arteriovenous based on the retinal blood vessel image, which facilitates image acquisition and analysis. Moreover, the deformable convolution used by the feature enhancement module 11 can help the neural network better capture the local topological features of the blood vessels and adapt to blood vessels of different shapes; the morphological reconstruction module 13 predicts the blood vessel region based on connectivity constraints, which ensures that the obtained blood vessel region is strictly connected. Experimental results show that the arteriovenous blood vessel region segmented by the method has higher topological fidelity, and the phenomenon of blood vessel rupture and arteriovenous misconnection is significantly inhibited, realizing more accurate arteriovenous blood vessel segmentation. In addition, the arteriovenous blood vessel segmentation scheme provided by the present application can provide stable, reliable and sensitive indicators for future quantitative evaluation of retinal blood vessel structure.

[0066] The embodiment of the present application also provides an arteriovenous segmentation method suitable for a retinal blood vessel image, wherein the retinal blood vessel image is generated by optical coherence tomography blood vessel imaging, as shown in the following formula (I): Figure 5 The method comprises the following steps:

[0067] In step S502, a feature extraction module extracts features of the received retinal blood vessel image to obtain first high-dimensional features of the retinal blood vessel image.

[0068] In step S504, a deformable feature propagation block receives the first high-dimensional features and outputs third high-dimensional features.

[0069] In step S506, a classification module maps target probability distribution maps according to the third high-dimensional features; wherein the target probability distribution maps comprise arteriovenous probability distribution maps.

[0070] In step S508, a morphological reconstruction module reconstructs target blood vessel regions of the retina based on the target probability distribution maps; wherein the target blood vessel regions comprise arteriovenous blood vessel regions.

[0071] The arteriovenous segmentation method suitable for the retinal blood vessel image processes the first high-dimensional features of the retinal blood vessel image through the deformable feature propagation block, so that the third high-dimensional features outputted contain more local topological information, which improves the accuracy of the target probability distribution maps and further improves the fidelity of the reconstructed blood vessel regions.

[0072] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. An arteriovenous segmentation system for retinal vascular images, wherein the retinal vascular images are generated by optical coherence tomography (OCT) vascular imaging, characterized in that, The system comprises, in sequence, a feature extraction module, a feature enhancement module, a classification module, and a morphology reconstruction module; wherein the feature enhancement module comprises a deformable feature propagation block; The feature extraction module is configured to perform feature extraction on a received retinal blood vessel image to obtain a first high-dimensional feature of the retinal blood vessel image; The deformable feature propagation block is configured to receive the first high-dimensional feature and output a third high-dimensional feature; The classification module is configured to map the third high-dimensional feature to obtain a target probability distribution map; wherein the target probability distribution map comprises an arterial probability distribution map and a venous probability distribution map; The morphology reconstruction module is configured to reconstruct a target blood vessel region of the retina based on the target probability distribution map; the target blood vessel region comprises an arterial blood vessel region and a venous blood vessel region; Each deformable feature propagation block in the feature enhancement module is further connected to an attention gate module; For each deformable feature propagation block connected to an attention gate module, the attention gate module is configured to determine a weight coefficient based on a second high-dimensional feature received by the deformable feature propagation block and a fourth high-dimensional feature output by the deformable feature propagation block; The second high-dimensional feature received by the deformable feature propagation block and the fourth high-dimensional feature are weighted and calculated based on the weight coefficient to obtain a third high-dimensional feature; The morphology reconstruction module is further configured to: assign a weight value to each pixel point in the target probability distribution map based on a preset weight function and a probability value corresponding to each pixel point in the target probability distribution map; wherein the greater the probability value of the pixel point, the smaller the corresponding weight value; for each seed point in a preset seed point set, calculate a minimum weight path from the seed point to each pixel point in the target probability distribution map using a shortest path algorithm; wherein the seed point set comprises boundary pixel points of the target probability distribution map and / or pixel points in a cup region; determine a pixel dependency graph based on each minimum weight path; determine a target blood vessel region based on the pixel dependency graph; The morphology reconstruction module is further configured to: assign a selection return value to each pixel point in the target probability distribution map based on a preset threshold and a probability value corresponding to each pixel point in the target probability distribution map; wherein the greater the selection return value, the greater the probability that the pixel point is classified as a blood vessel; solve a target closed subgraph from the pixel dependency graph using a minimum cut algorithm based on the pixel dependency graph and the selection return value corresponding to each pixel point; wherein the sum of the selection return values of the pixel points corresponding to the target closed subgraph is maximum; determine a target blood vessel region based on the pixel points corresponding to the target closed subgraph; The morphology reconstruction module is further configured to: for each pixel point in the target probability distribution map, calculate a difference value between the probability value corresponding to the pixel point and the preset threshold to obtain a difference result; determine the difference result as the selection return value corresponding to the pixel point.

2. The system of claim 1, wherein, The number of deformable feature propagation blocks is multiple; the multiple deformable feature propagation blocks are connected in sequence. Each deformable feature propagation block is configured to perform a deformable convolution operation on a received second high-dimensional feature to output a third high-dimensional feature; wherein the second high-dimensional feature received by a first deformable feature propagation block is the first high-dimensional feature; and the second high-dimensional feature received by each deformable feature propagation block other than the first deformable feature propagation block is the third high-dimensional feature output by a previous deformable feature propagation block. The classification module is configured to map the third high-dimensional feature output by the last deformable feature propagation block to obtain a target probability distribution map.

3. The system of claim 1, wherein, The weight function is as follows: ； wherein, represents a probability value corresponding to a pixel point with pixel coordinates (i, j); represents a weight value corresponding to a pixel point with pixel coordinates (i, j); represents a preset threshold value; is a preset positive value.

4. The system of claim 2, wherein, Each deformable feature propagation block comprises a deformable convolution layer, and a batch normalization layer, a nonlinear activation layer and a normal convolution layer connected to the deformable convolution layer, respectively.

5. The system of claim 1, wherein, Each attention gate module comprises a normal convolution layer and a nonlinear activation layer connected in sequence.

6. The system of claim 1, wherein, The feature extraction module is further configured to: extract features from the received retinal blood vessel image using a pre-trained neural network model to obtain a first high-dimensional feature of the retinal blood vessel image.

Images