Retinal vascular imaging segmentation method, device, equipment and medium

Abstract

The present application discloses a retinal vascular imaging segmentation method, device, equipment and medium, which relates to the field of image segmentation technology, including: obtaining a retinal vascular image to be segmented generated after retinal vascular imaging is performed using a preset vascular imaging method; inputting the retinal vascular image to be segmented into a trained target vascular segmentation model, so as to perform full vascular recognition, large vascular recognition, arterial vascular recognition and venous vascular recognition on the retinal vascular image to be segmented respectively through a target full vascular recognition sub-model, a target large vascular recognition sub-model, a target arterial vascular recognition sub-model and a target venous vascular recognition sub-model in the target vascular segmentation model, and obtain corresponding process vascular recognition images respectively; binarizing the process vascular recognition image through the target vascular recognition segmentation model, so as to obtain a binarized full vascular image, a binarized large vascular image, a binarized arterial vascular image and a binarized venous vascular image.

Description

Technical Field

[0001] The present invention relates to the technical field of image segmentation, and in particular to a retinal blood vessel imaging segmentation method, device, equipment and medium. Background Art

[0002] At present, the retinal blood vessels have complex morphology, with bifurcations and tortuosity, which makes the segmentation of blood vessel images more difficult than that of other parts of the human body. However, there are many ways to implement retinal blood vessel segmentation, such as threshold-based blood vessel segmentation, filter-based blood vessel segmentation, blood vessel segmentation image segmentation and other methods.

[0003] (1) Threshold-based vascular segmentation: It is to use a global threshold or an adaptive threshold to directly binarize the OCTA (Optical Coherence Tomography Angiography) image. The premise of this method is that the difference between the boundary pixels of the object and the adjacent pixels is large, and then the rules set by the researchers are used to separate it from the background, and then the mask is obtained. However, OCTA images contain a large number of blood vessels mixed with surrounding tissues and (or) noise, so using this method will result in the loss of many microvascular signals.

[0004] (2) Filter-based vascular segmentation: It can suppress noise in OCTA images and better display microvessels, but its quantitative performance is far inferior to the threshold method. In addition, its performance depends on the researchers' adjustment of the filter parameters, which requires extremely high learning costs.

[0005] In summary, when faced with retinal vascular images, how to detect the tiny vascular structures in the image to the greatest extent possible, retain the vascular information, and accurately segment large blood vessels and microvessels, and accurately distinguish between arteries and veins are technical problems to be solved in this field. Summary of the invention

[0006] In view of this, the purpose of the present invention is to provide a retinal vascular imaging segmentation method, device, equipment and medium, which can detect the tiny vascular structure in the image to the greatest extent when facing the retinal vascular image, retain the vascular information, and accurately segment the large blood vessels and microvessels, and accurately distinguish between arteries and veins. The specific scheme is as follows:

[0007] In a first aspect, the present application discloses a retinal vascular imaging segmentation method, comprising:

[0008] Acquire a retinal blood vessel image to be segmented generated after imaging the retinal blood vessels using a preset blood vessel imaging method;

[0009] Inputting the retinal blood vessel image to be segmented into the trained target blood vessel segmentation model, so as to respectively perform full blood vessel recognition, large blood vessel recognition, arterial blood vessel recognition and venous blood vessel recognition on the retinal blood vessel image to be segmented through the target full blood vessel recognition sub-model, the target large blood vessel recognition sub-model, the target arterial blood vessel recognition sub-model and the target venous blood vessel recognition sub-model in the target blood vessel segmentation model, and obtain corresponding process blood vessel recognition images respectively;

[0010] The process blood vessel recognition image is binarized by the target blood vessel recognition segmentation model to obtain a binarized whole blood vessel image, a binarized large blood vessel image, a binarized arterial blood vessel image, and a binarized venous blood vessel image.

[0011] Optionally, before inputting the retinal blood vessel image to be segmented into the trained target blood vessel segmentation model, the method further includes:

[0012] Using retinal vascular training images with vascular label data to U 2 -Net model to perform model training so that the U 2 The convolution layer of the -Net model uses a convolution kernel to slide on the retinal vascular training image based on the vascular label data, and calculates the convolution feature of each image position to obtain a feature map;

[0013] Through the U 2 The pooling layer of the -Net model performs a pooling operation on the feature map to reduce the size of the feature map and extract target features including the center position of the blood vessel and the edge position of the blood vessel;

[0014] Through the U 2 -The U-shaped residual block structure of the Net model extracts local features of the feature map including vascular microstructures and vascular branches;

[0015] Through the U 2 -Net model and performs a deconvolution operation on the target features and local features based on the blood vessel label data to update the U 2 -Net model parameters to obtain the trained target blood vessel segmentation model.

[0016] Optionally, the retinal blood vessel training image with blood vessel label data is used to train U 2 -Net model to perform model training so that the U 2 The convolution layer of the -Net model uses a convolution kernel to slide on the retinal vascular training image based on the vascular label data, and calculates the convolution feature of each image position to obtain a feature map, including:

[0017] The label information of each blood vessel type, each blood vessel position, each blood vessel shape, and each blood vessel boundary in the training image marked in the blood vessel label data is used as a supervisory signal, and the U 2 The convolution layer of the -Net model performs a convolution operation on the retinal vascular training image and calculates the convolution features of each image position to obtain a feature map.

[0018] Optionally, through the U 2 The pooling layer of the -Net model extracts target features of the feature map including the center position of the blood vessel and the edge position of the blood vessel, including:

[0019] Through the U 2 The pooling layer of the -Net model extracts the target features of the feature map including the center position of the blood vessel and the edge position of the blood vessel to train the U 2 -Net model understands the location and relationship of blood vessels in vascular images and obtains image context information.

[0020] Optionally, the U 2 -Net model and performs a deconvolution operation on the target features and local features based on the blood vessel label data to update the U 2 -Net model parameters to obtain the trained target vessel segmentation model, including:

[0021] Through the U 2 -Net model and performs deconvolution operations on target features and local features based on each blood vessel type and each blood vessel boundary in the blood vessel label data to update the U 2 -Net model parameters, and obtain the trained target vessel segmentation model including the target full vessel recognition sub-model, the target large vessel recognition sub-model, the target arterial vessel recognition sub-model, and the target venous vessel recognition sub-model.

[0022] Optionally, the U 2 -Net model and performs a deconvolution operation on the target features and local features based on the blood vessel label data to update the U 2 -Net model parameters to obtain the trained target vessel segmentation model, including:

[0023] Through the U 2 -Net model and performing a deconvolution operation on the target features and the local features based on the blood vessel label data to obtain a target feature map after the deconvolution operation;

[0024] Enlarging the image size of the target feature map after the deconvolution operation to the same size as the retinal blood vessel training image to obtain an enlarged target feature map;

[0025] Calculate the loss value between the enlarged target feature map and the blood vessel label data to reversely update the U according to the loss value 2 -Net model parameters to obtain the trained target blood vessel segmentation model.

[0026] Optionally, the loss value between the enlarged target feature map and the blood vessel label data is calculated to reversely update the U according to the loss value. 2 -Net model parameters to obtain the trained target vessel segmentation model, including:

[0027] Calculating a cross entropy loss value between a probability distribution of the predicted blood vessel type information corresponding to the amplified target feature map and a probability distribution of a standard blood vessel type in the blood vessel label data by a cross entropy loss function;

[0028] According to the cross entropy loss value, the gradient information of each model parameter with respect to the cross entropy loss value is calculated by a back propagation algorithm;

[0029] Use the preset gradient descent algorithm to update the U according to the gradient information 2 -Model parameters of the Net model;

[0030] Repeat the process of updating the U according to the gradient information using the preset gradient descent algorithm. 2 -Net model parameters to obtain the trained target blood vessel segmentation model.

[0031] In a second aspect, the present application discloses a retinal vascular imaging segmentation device, comprising:

[0032] An image acquisition module, used for acquiring a retinal blood vessel image to be segmented generated after imaging the retinal blood vessels using a preset blood vessel imaging method;

[0033] an image recognition module, used for inputting the retinal blood vessel image to be segmented into a trained target blood vessel segmentation model, so as to respectively perform full blood vessel recognition, large blood vessel recognition, arterial blood vessel recognition and venous blood vessel recognition on the retinal blood vessel image to be segmented through a target full blood vessel recognition sub-model, a target large blood vessel recognition sub-model, a target arterial blood vessel recognition sub-model and a target venous blood vessel recognition sub-model in the target blood vessel segmentation model, and obtain corresponding process blood vessel recognition images respectively;

[0034] The image segmentation module is used to perform binarization processing on the process blood vessel identification image through the target blood vessel identification segmentation model to obtain a binarized whole blood vessel image, a binarized large blood vessel image, a binarized arterial blood vessel image, and a binarized venous blood vessel image.

[0035] In a third aspect, the present application discloses an electronic device, comprising:

[0036] Memory, used to store computer programs;

[0037] The processor is used to execute the computer program to implement the steps of the retinal vascular imaging segmentation method disclosed above.

[0038] In a fourth aspect, the present application discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the steps of the retinal vascular imaging segmentation method disclosed above are implemented.

[0039] It can be seen that the present application discloses a retinal vascular imaging segmentation method, including: obtaining a retinal vascular image to be segmented generated after imaging the retinal vascular using a preset vascular imaging method; inputting the retinal vascular image to be segmented into a trained target vascular segmentation model, so as to perform full vessel recognition, large vessel recognition, arterial vessel recognition and venous vessel recognition on the retinal vascular image to be segmented through a target full vessel recognition sub-model, a target large vessel recognition sub-model, a target arterial vessel recognition sub-model and a target venous vessel recognition sub-model in the target vascular segmentation model, and obtain corresponding process vessel recognition images respectively; binarizing the process vessel recognition image through the target vessel recognition segmentation model to obtain a binarized full vessel image, a binarized large vessel image, a binarized arterial vessel image and a binarized venous vessel image. It can be seen that the target vessel segmentation model can capture more subtle vascular features in the image and provide more accurate segmentation results. The trained target vessel segmentation model can handle the complex morphology and changes of the blood vessels and improve the accuracy of segmentation. Compared with the prior art, which uses the direct qualitative and quantitative segmentation method of the processed image obtained after image processing of the original image when performing vascular segmentation, the trained target vessel segmentation model reduces the dependence on manually set thresholds or filters, has better adaptability to retinal images of different individuals, and improves the degree of automation. In addition, the target vessel recognition segmentation model can accurately segment the original input image and directly output different types of binarized vascular images to provide more comprehensive information. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0041] Figure 1A flow chart of a retinal vascular imaging segmentation method disclosed in this application;

[0042] Figure 2 A flow chart of a target blood vessel segmentation model training method disclosed in this application;

[0043] Figure 3 This is a schematic diagram of the structure of a retinal vascular imaging and segmentation device disclosed in this application;

[0044] Figure 4 This is a structural diagram of an electronic device disclosed in this application. DETAILED DESCRIPTION

[0045] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0046] At present, the retinal blood vessels have complex morphology, with bifurcations and tortuosity, which makes the segmentation of blood vessel images more difficult than that of other parts of the human body. However, there are many ways to implement retinal blood vessel segmentation, such as threshold-based blood vessel segmentation, filter-based blood vessel segmentation, blood vessel segmentation image segmentation and other methods.

[0047] (1) Threshold-based blood vessel segmentation: This method uses a global threshold or an adaptive threshold to directly binarize the OCTA image. The premise of this method is that the difference between the boundary pixels of the object and the adjacent pixels is large, and then the rules set by the researchers are used to separate it from the background, and then the mask is obtained. However, there are a large number of blood vessels mixed with surrounding tissues and (or) noise in OCTA images, so using this method will result in the loss of many microvascular signals.

[0048] (2) Filter-based vascular segmentation: It can suppress noise in OCTA images and better display microvessels, but its quantitative performance is far inferior to the threshold method. In addition, its performance depends on the researchers' adjustment of the filter parameters, which requires extremely high learning costs.

[0049] To this end, the present invention provides a retinal vascular imaging segmentation solution, which can detect the tiny vascular structure in the image to the greatest extent when facing the retinal vascular image, retain the vascular information, and accurately segment the large blood vessels and microvessels, and accurately distinguish between arteries and veins.

[0050] Reference Figure 1 As shown, an embodiment of the present invention discloses a retinal vascular imaging segmentation method, comprising:

[0051] Step S11: obtaining a retinal blood vessel image to be segmented which is generated after imaging the retinal blood vessels using a preset blood vessel imaging method.

[0052] In this embodiment, optical coherence tomography vascular imaging (OCTA) technology is used to image the retinal blood vessels to achieve non-invasive blood flow imaging examination. The retinal vascular images under this technology are not easily affected by fundus anatomical variations and retinal nerve fiber reflections, and the measurement of vascular density does not rely on high-precision retinal layer segmentation. Therefore, the obtained OCTA images provide corresponding preparation for image segmentation for subsequent vascular segmentation.

[0053] Step S12: inputting the retinal vascular image to be segmented into the trained target vascular segmentation model, so as to perform full vessel recognition, large vessel recognition, arterial vessel recognition and venous vessel recognition on the retinal vascular image to be segmented respectively through the target full vessel recognition sub-model, target large vessel recognition sub-model, target arterial vessel recognition sub-model and target venous vessel recognition sub-model in the target vascular segmentation model, and obtain corresponding process vessel recognition images respectively.

[0054] In this embodiment, the OCTA image is input as the retinal vascular image to be segmented into the trained target vascular segmentation model, so that each target full vascular recognition sub-model, target large vascular recognition sub-model, target arterial vascular recognition sub-model, and target venous vascular recognition sub-model in the trained target vascular segmentation model can respectively identify the corresponding category of vascular images. In this way, the corresponding category of vascular features can be identified by each trained sub-model to obtain a process vascular recognition image, thereby achieving the image segmentation purpose of the entire retinal vascular image to be segmented. However, the output vascular recognition image at this time can present different gray values ​​or colors because the OCTA image (original image) can represent the intensity, distribution and other information of blood flow, rather than just two states. The specific image features will also vary depending on the equipment, scanning parameters and the tissue conditions being detected. Therefore, the segmented image at this time does not meet the final binarization requirements. Therefore, the current image segmentation obtains the process vascular recognition images corresponding to the full blood vessels, large blood vessels, arterial blood vessels, and venous blood vessels.

[0055] Step S13: binarizing the process vessel recognition image by using the target vessel recognition and segmentation model to obtain a binarized whole vessel image, a binarized large vessel image, a binarized arterial vessel image, and a binarized venous vessel image.

[0056] In this embodiment, after each sub-model outputs a process vessel identification image after image segmentation of the retinal vessel image to be segmented, the target vessel identification and segmentation model further performs binarization processing on each process vessel identification image, and converts it from a process vessel identification image containing different gray values ​​or colors into a binary black-and-white image. The reason is that, since the image segmentation process has been completed, the current process vessel identification image determines its corresponding category, and the process vessel identification image only contains a certain category of identified vessel images. Therefore, in order to meet the actual binary image requirements, it is only necessary to perform binarization processing on each process vessel identification image. Specifically, the process vessel identification image is subjected to image preprocessing including denoising, contrast enhancement and other operations to improve the quality of the process vessel identification image. A fixed threshold can be set manually, or an automatic threshold selection algorithm, such as the Otsuka algorithm, the maximum inter-class variance method OTSU, etc., can be used to select a fixed threshold. The specific method for setting the fixed threshold is not specifically limited to this. The value of each pixel in the process blood vessel recognition image after the image quality is improved is compared with the fixed threshold, and the pixels greater than the fixed threshold are set to white (or 1), and the pixels less than or equal to the fixed threshold are set to black (or 0), so as to obtain the whole blood vessel image after binarization, the large blood vessel image after binarization, the arterial blood vessel image after binarization, and the venous blood vessel image after binarization. It should be noted that after the binarization processing, it is also possible to choose whether to perform image post-processing to optimize the binarization result according to actual needs. Specifically, some morphological processing, such as corrosion, expansion and other post-processing operations, can be performed according to actual needs.

[0057] It can be seen that the present application discloses a retinal vascular imaging segmentation method, including: obtaining a retinal vascular image to be segmented generated after imaging the retinal vascular using a preset vascular imaging method; inputting the retinal vascular image to be segmented into a trained target vascular segmentation model, so as to perform full vessel recognition, large vessel recognition, arterial vessel recognition and venous vessel recognition on the retinal vascular image to be segmented through a target full vessel recognition sub-model, a target large vessel recognition sub-model, a target arterial vessel recognition sub-model and a target venous vessel recognition sub-model in the target vascular segmentation model, and obtain corresponding process vessel recognition images respectively; binarizing the process vessel recognition image through the target vessel recognition segmentation model to obtain a binarized full vessel image, a binarized large vessel image, a binarized arterial vessel image and a binarized venous vessel image. It can be seen that the target vessel segmentation model can capture more subtle vascular features in the image and provide more accurate segmentation results. The trained target vessel segmentation model can handle the complex morphology and changes of the blood vessels and improve the accuracy of segmentation. Compared with the prior art, which uses the direct qualitative and quantitative segmentation method of the processed image obtained after image processing of the original image when performing vascular segmentation, the trained target vessel segmentation model reduces the dependence on manually set thresholds or filters, has better adaptability to retinal images of different individuals, and improves the degree of automation. In addition, the target vessel recognition segmentation model can accurately segment the original input image and directly output different types of binarized vascular images to provide more comprehensive information.

[0058] Reference Figure 2 As shown, the embodiment of the present invention discloses a target blood vessel segmentation model training method. Compared with the previous embodiment, this embodiment further illustrates and optimizes the technical solution. Specifically:

[0059] Step S21: Using the retinal blood vessel training image with blood vessel label data to train U 2 -Net model to perform model training so that the U 2 The convolution layer of the -Net model uses a convolution kernel to slide on the retinal vascular training image based on the vascular label data, calculates the convolution feature of each image position, and obtains a feature map.

[0060] In this embodiment, U-Net is named for its U-shaped symmetrical structure design, which can make full use of the context information and local details of the input image and greatly reduce the amount of training data. However, since it adopts a design of first downsampling encoder to compress the image and then extract high-level features to identify specific structures in complex content images, in fact, the microvascular features in OCTA images cannot be completely extracted. Therefore, the present invention proposes a two-level nested U-shaped structure U 2-Net(U square Net,U 2 Network) semantic segmentation model, U 2 The framework structure of the U-Net semantic segmentation model is U-Net. 2 -Net uses a decoding method similar to the U-Net semantic segmentation model, but its convolutional layer or deconvolutional layer embeds a complete U-shaped residual block structure, rather than a single convolution operation. This embedded structure can better extract local details. 2 -Net model has the following structure and functions: Input layer: used to input images into the model; Convolution layer: used to extract features from images through convolution operations. The convolution kernel slides on the image to calculate the features of each position; Pooling layer: used to perform pooling operations on the feature map after convolution, such as maximum pooling or average pooling, to reduce the size of the feature map and extract the main features at the same time; Residual block: U 2 -The convolution layer or deconvolution layer in the Net model embeds a complete U-shaped residual block structure, which can better extract local details; Deconvolution layer: used to gradually enlarge the feature map to the size of the original image through deconvolution operation in the decoding stage. Output layer: used to generate segmentation results or other required outputs in the final output layer. Among them, the convolution layer, pooling layer, and residual block constitute the residual U block RSU.

[0061] In this embodiment, the label information of each blood vessel type, each blood vessel position, each blood vessel shape, and each blood vessel boundary in the training image annotated in the blood vessel label data is used as a supervisory signal, and the U 2 The convolution layer of the -Net model performs convolution operations on the retinal vascular training image and calculates the convolution features of each image position to obtain a feature map. It can be understood that in determining U 2 -Net semantic segmentation model is used as the pre-trained model for retinal blood vessel segmentation. First, the retinal blood vessel training image carrying the blood vessel label data is obtained through the input layer. Then the input layer inputs the retinal blood vessel training image into the convolution layer so that U 2-Net model's convolutional layer uses the convolution kernel to slide on the retinal vascular training image to calculate the convolution features of each image position to obtain a feature map. At the same time, the label information of each vascular type, vascular position, vascular shape, and vascular boundary in the vascular label data is used as a supervisory signal to supervise the convolution operation of the convolution layer to calculate the convolution features of each image position in the retinal vascular training image and obtain a feature map. It should be noted that the convolution layer guides the model to learn how to identify and segment different types of blood vessels from the image under the guidance of the supervisory signal. During the training process, the model adjusts the parameters of the convolution kernel according to the label information so that the model can accurately predict the vascular structure in the image. Through continuous training and optimization, the performance of the model will gradually improve, so that it can better complete the vascular segmentation task. U 2 -Net model can generate sub-models of different blood vessel types, namely, full blood vessel identification sub-model, large blood vessel identification sub-model, arterial blood vessel identification sub-model, and venous blood vessel identification sub-model.

[0062] It should be noted that the convolution features calculated by the convolution layer at each image position may specifically include but are not limited to: the convolution layer calculates the shape, diameter, curvature, etc. of the blood vessels in the retinal blood vessel training image to obtain the vascular morphological features; the convolution layer captures the texture information of the blood vessels, such as: the smoothness of the blood vessels, the thickness change, and other vascular texture features; the convolution layer analyzes the distribution of blood vessels in the retinal blood vessel training image, including: the density of the blood vessels, the direction, and other vascular distribution features. The feature map of the retinal blood vessel training image is constructed based on the above convolution features.

[0063] Step S22: Through the U 2 The pooling layer of the -Net model performs a pooling operation on the feature map to reduce the size of the feature map and extract target features including the center position of the blood vessel and the edge position of the blood vessel.

[0064] In this embodiment, through the U 2 The pooling layer of the -Net model extracts the target features of the feature map including the center position of the blood vessel and the edge position of the blood vessel to train the U 2 -Net model understands the position and relationship of blood vessels in the blood vessel image and obtains the image context information. It can be understood that after extracting features in the convolution layer and obtaining the feature map, the feature map is subjected to the maximum pooling or average pooling pooling operation. The pooling operation can reduce the size of the feature map to facilitate the extraction of the main features in the feature map, that is, the key information of the retinal blood vessel training image. Among them, the main features extracted by the pooling layer specifically include the target features of the blood vessel center position and the blood vessel edge position. Then the extracted target features are used to train U 2-Net model understands the position and relationship of blood vessels in the vascular image and obtains image context information. Among them, the pooling layer plays a role of downsampling, the purpose is to reduce the resolution and data volume of the image, and at the same time extract important feature information for more efficient calculation and representation in subsequent processing.

[0065] It should be noted that the specific model structure and the location of the downsampling encoder may vary depending on the actual application scenario. 2 -Net models may downsample at multiple levels to extract progressively higher-level features, for example, a downsampling encoder may be arranged on top of a convolutional layer.

[0066] Step S23: Through the U 2 The U-shaped residual block structure of the -Net model extracts local features of the feature map including vascular microstructures and vascular branches.

[0067] In this embodiment, the local features of the feature map are extracted by using a U-shaped residual block structure, wherein the local features include: microstructures of blood vessels, blood vessel branches, etc. In this way, the method of extracting local features by using a U-shaped residual block structure supplements the target feature training U 2 -Net model understands the location and relationship of blood vessels in vascular images. Among them, since the U-shaped residual block structure can be directly embedded in the above convolutional layer, so that the convolutional layer only performs a single convolution operation, and simultaneously performs the operations of convolution to extract target features and extract local features.

[0068] Step S24: Through the U 2 -Net model and performs a deconvolution operation on the target features and local features based on the blood vessel label data to update the U 2 -Net model parameters to obtain the trained target blood vessel segmentation model.

[0069] In this embodiment, through the U 2 -Net model and performs deconvolution operations on target features and local features based on each blood vessel type and each blood vessel boundary in the blood vessel label data to update the U 2 -Net model parameters, and obtain the trained target vessel segmentation model including the target full vessel recognition sub-model, the target large vessel recognition sub-model, the target arterial vessel recognition sub-model, and the target venous vessel recognition sub-model. It can be understood that the deconvolution operation is performed on the target features and local features using the vessel types and vessel boundaries in the vessel label data again to update U 2-Net model parameters, it should be noted that the deconvolution layer can also be embedded in the U-shaped residual block structure, so that the low-resolution feature map containing target features and local features obtained after convolution pooling is upsampled by the deconvolution layer and restored to the original resolution of the OCTA image, thereby obtaining a more refined image representation, and then the embedded U-shaped residual module can fuse features at different levels to obtain a richer image representation. Among them, the deconvolution layer can enhance local features, thereby improving the image segmentation accuracy. Specifically, in the deconvolution layer, the degree of enhancement of local features can be controlled by setting different convolution kernel sizes and step sizes. In addition, in the U-shaped residual module, low-level features can be transferred to high levels through jump connections, thereby enhancing the representation ability of features. Therefore, after the deconvolution operation of the deconvolution layer, the U 2 -Net model to update model parameters to obtain a trained target vessel segmentation model including a target full vessel recognition sub-model, a target large vessel recognition sub-model, a target arterial vessel recognition sub-model, and a target venous vessel recognition sub-model.

[0070] In this embodiment, the deconvolution process is as follows: 2 -Net model and performing a deconvolution operation on the target feature and the local feature based on the blood vessel label data to obtain a target feature map after the deconvolution operation; enlarging the image size of the target feature map after the deconvolution operation to the same size as the retinal blood vessel training image to obtain an enlarged target feature map; calculating a loss value between the enlarged target feature map and the blood vessel label data to reversely update the U according to the loss value 2 -Net model parameters to obtain the trained target blood vessel segmentation model.

[0071] In this embodiment, in order to classify each blood vessel in the image and improve the segmentation accuracy of the trained target blood vessel segmentation model, it is necessary to calculate the loss between the predicted value and the true label in the training process, and reversely calculate the gradient of the model parameter corresponding to the current loss to the loss according to the calculated loss to update the model parameter. The specific execution steps are as follows: calculate the cross entropy loss value between the probability distribution of the corresponding predicted blood vessel type information of the enlarged target feature map and the probability distribution of the standard blood vessel type in the blood vessel label data through the cross entropy loss function; calculate the gradient information of each model parameter to the cross entropy loss value through the back propagation algorithm according to the cross entropy loss value; and update the U according to the gradient information using the preset gradient descent algorithm. 2 -Net model parameters; repeatedly executing the preset gradient descent algorithm to update the U according to the gradient information 2-Net model parameters to obtain the trained target vessel segmentation model. It can be understood that the cross entropy loss function is used to compare the probability distribution of the model's corresponding predicted vascular type information for the amplified target feature map with the probability distribution of the standard vascular type in the vascular label data to measure the difference between the two, that is, to substitute the two into the cross entropy loss function calculation formula to obtain the cross entropy loss value. Then, based on the cross entropy loss value, the gradient of each model parameter to the cross entropy loss is calculated through the back propagation algorithm. Use an optimization algorithm (such as stochastic gradient descent, etc.) to update U according to the gradient information 2 -Net model parameters, so that the cross entropy loss value gradually decreases until the cross entropy loss value reaches the preset loss value threshold, stop iterative update, and set the current U 2 -Net model is used as the target blood vessel segmentation model.

[0072] It can be seen that by embedding the U-shaped residual block structure, the features of receptive fields of different scales are integrated, and more contextual information can be captured from different scales. Then, through the method of local features + target features, the features of tiny blood vessel structures are also taken into account in the model training process. When the model segments blood vessel images, it can detect tiny blood vessel structures to the greatest extent, retain blood vessel information, accurately segment large blood vessels and microvessels, and accurately distinguish arteries and veins. The use of pooling operations increases the depth of the entire architecture, but does not significantly increase the computational cost. These characteristics make U 2 -Net networks are able to better utilize contextual information when processing images, thereby improving the performance of tasks such as image segmentation.

[0073] Reference Figure 3 As shown, the present invention also discloses a retinal vascular imaging segmentation device, comprising:

[0074] An image acquisition module 11 is used to acquire a retinal blood vessel image to be segmented generated after imaging the retinal blood vessels using a preset blood vessel imaging method;

[0075] The image recognition module 12 is used to input the retinal blood vessel image to be segmented into the trained target blood vessel segmentation model, so as to perform full blood vessel recognition, large blood vessel recognition, arterial blood vessel recognition and venous blood vessel recognition on the retinal blood vessel image to be segmented respectively through the target full blood vessel recognition sub-model, target large blood vessel recognition sub-model, target arterial blood vessel recognition sub-model and target venous blood vessel recognition sub-model in the target blood vessel segmentation model, and obtain corresponding process blood vessel recognition images respectively;

[0076] The image segmentation module 13 is used to perform binarization processing on the process vessel recognition image through the target vessel recognition segmentation model to obtain a binarized whole vessel image, a binarized large vessel image, a binarized arterial vessel image, and a binarized venous vessel image.

[0077] It can be seen that the present application discloses obtaining a retinal vascular image to be segmented after imaging the retinal vascular using a preset vascular imaging method; inputting the retinal vascular image to be segmented into a trained target vascular segmentation model, so as to perform full vascular recognition, large vascular recognition, arterial vascular recognition and venous vascular recognition on the retinal vascular image to be segmented through a target full vascular recognition sub-model, a target large vascular recognition sub-model, a target arterial vascular recognition sub-model and a target venous vascular recognition sub-model in the target vascular segmentation model, and obtain corresponding process vascular recognition images respectively; binarizing the process vascular recognition image through the target vascular recognition segmentation model to obtain a binarized full vascular image, a binarized large vascular image, a binarized arterial vascular image and a binarized venous vascular image. It can be seen that the target vessel segmentation model can capture more subtle vascular features in the image and provide more accurate segmentation results. The trained target vessel segmentation model can handle the complex morphology and changes of the blood vessels and improve the accuracy of segmentation. Compared with the prior art, which uses the direct qualitative and quantitative segmentation method of the processed image obtained after image processing of the original image when performing vascular segmentation, the trained target vessel segmentation model reduces the dependence on manually set thresholds or filters, has better adaptability to retinal images of different individuals, and improves the degree of automation. In addition, the target vessel recognition segmentation model can accurately segment the original input image and directly output different types of binarized vascular images to provide more comprehensive information.

[0078] Furthermore, the present application also discloses an electronic device. Figure 4 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content in the diagram cannot be regarded as any limitation on the scope of use of the present application.

[0079] Figure 4 The present invention provides a schematic diagram of the structure of an electronic device 20 provided in an embodiment of the present application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is used to store a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the retinal vascular imaging segmentation method disclosed in any of the above embodiments. In addition, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0080] In this embodiment, the power supply 23 is used to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and the external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0081] Among them, the processor 21 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 21 can be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 21 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 21 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 21 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0082] In addition, the memory 22, as a carrier for storing resources, can be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon can include an operating system 221, a computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0083] Among them, the operating system 221 is used to manage and control the hardware devices and computer programs 222 on the electronic device 20, so as to realize the operation and processing of the processor 21 on the massive data 223 in the memory 22, which can be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program that can be used to complete the retinal vascular imaging segmentation method performed by the electronic device 20 disclosed in any of the aforementioned embodiments, the computer program 222 can further include a computer program that can be used to complete other specific tasks. In addition to data transmitted from an external device received by the electronic device, the data 223 can also include data collected by its own input and output interface 25.

[0084] Furthermore, the present application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the retinal vascular imaging segmentation method disclosed above is implemented. For the specific steps of the method, reference may be made to the corresponding contents disclosed in the above embodiments, and no further description will be given here.

[0085] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0086] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application. The steps of the method or algorithm described in conjunction with the embodiments disclosed herein can be implemented directly with hardware, a software module executed by a processor, or a combination of the two. The software module can be placed in a random access memory RAM (Random Access Memory), memory, read-only memory ROM (Read Only Memory), electrically programmable EPROM (Electrically Programmable Read Only Memory), electrically erasable programmable EEPROM (ElectricErasable Programmable Read Only Memory), register, hard disk, removable disk, CD-ROM (CompactDisc-Read Only Memory), or any other form of storage medium known in the technical field.

[0087] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0088] The scheme provided by the present invention is introduced in detail above. Specific examples are used in this article to illustrate the principle and implementation mode of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation mode and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A retinal vascular imaging segmentation method, characterized in that: include: Acquire a retinal blood vessel image to be segmented generated after imaging the retinal blood vessels using a preset blood vessel imaging method; The retinal blood vessel image to be segmented is input into the trained target blood vessel segmentation model, so that the target full blood vessel recognition sub-model, the target large blood vessel recognition sub-model, the target arterial blood vessel recognition sub-model, and the target venous blood vessel recognition sub-model in the target blood vessel segmentation model respectively perform full blood vessel recognition, large blood vessel recognition, micro-vessel recognition, arterial blood vessel recognition, and venous blood vessel recognition on the retinal blood vessel image to be segmented, and obtain corresponding process blood vessel recognition images respectively; Binarizing the process blood vessel recognition image by using the target blood vessel segmentation model to obtain a binarized whole blood vessel image, a binarized large blood vessel image, a binarized microvascular image, a binarized arterial blood vessel image, and a binarized venous blood vessel image; Before inputting the to-be-segmented retinal vessel image into the trained target vessel segmentation model, the method further includes: Using retinal vascular training images with vascular label data to U 2 -Net model to perform model training so that the U 2 The convolution layer of the -Net model uses a convolution kernel to slide on the retinal vascular training image based on the vascular label data, and calculates the convolution feature of each image position to obtain a feature map; Through the U 2 The pooling layer of the -Net model performs a pooling operation on the feature map to reduce the size of the feature map and extract target features including the center position of the blood vessel and the edge position of the blood vessel; Through the U 2 -The U-shaped residual block structure of the Net model extracts local features of the feature map including vascular microstructures and vascular branches; Through the U 2 -Net model and performs a deconvolution operation on the target features and local features based on the blood vessel label data to update the U 2 -Net model parameters to obtain the trained target blood vessel segmentation model; The U 2 -Net model and performs a deconvolution operation on the target features and local features based on the blood vessel label data to update the U 2 -Net model parameters to obtain the trained target vessel segmentation model, including: Through the U 2 -Net model and performs deconvolution operations on target features and local features based on each blood vessel type and each blood vessel boundary in the blood vessel label data to update the U 2 -Net model parameters, and obtain the trained target vessel segmentation model including the target full vessel recognition sub-model, the target large vessel recognition sub-model, the target arterial vessel recognition sub-model, and the target venous vessel recognition sub-model.

2. The retinal vascular imaging segmentation method according to claim 1, characterized in that: The retinal blood vessel training image with blood vessel label data is used to train U 2 -Net model to perform model training so that the U 2 The convolution layer of the -Net model uses a convolution kernel to slide on the retinal vascular training image based on the vascular label data, and calculates the convolution feature of each image position to obtain a feature map, including: The label information of each blood vessel type, each blood vessel position, each blood vessel shape, and each blood vessel boundary in the training image marked in the blood vessel label data is used as a supervisory signal, and the U 2 The convolution layer of the -Net model performs a convolution operation on the retinal vascular training image and calculates the convolution features of each image position to obtain a feature map.

3. The retinal vascular imaging segmentation method according to claim 1, characterized in that: Through the U 2 The pooling layer of the -Net model extracts target features of the feature map including the center position of the blood vessel and the edge position of the blood vessel, including: Through the U 2 The pooling layer of the -Net model extracts the target features of the feature map including the center position of the blood vessel and the edge position of the blood vessel to train the U 2 -Net model understands the location and relationship of blood vessels in vascular images and obtains image context information.

4. The retinal vascular imaging segmentation method according to claim 1, characterized in that: The U 2 -Net model and performs a deconvolution operation on the target features and local features based on the blood vessel label data to update the U 2 -Net model parameters to obtain the trained target vessel segmentation model, including: Through the U 2 -Net model and performing a deconvolution operation on the target features and the local features based on the blood vessel label data to obtain a target feature map after the deconvolution operation; Enlarging the image size of the target feature map after the deconvolution operation to the same size as the retinal blood vessel training image to obtain an enlarged target feature map; Calculate the loss value between the enlarged target feature map and the blood vessel label data to reversely update the U according to the loss value 2 -Net model parameters to obtain the trained target blood vessel segmentation model.

5. The retinal vascular imaging segmentation method according to claim 4, characterized in that: The loss value between the enlarged target feature map and the blood vessel label data is calculated to reversely update the U according to the loss value. 2 -Net model parameters to obtain the trained target vessel segmentation model, including: Calculating a cross entropy loss value between a probability distribution of the predicted blood vessel type information corresponding to the amplified target feature map and a probability distribution of a standard blood vessel type in the blood vessel label data by a cross entropy loss function; According to the cross entropy loss value, the gradient information of each model parameter with respect to the cross entropy loss value is calculated by a back propagation algorithm; Use the preset gradient descent algorithm to update the U according to the gradient information 2 -Model parameters of the Net model; Repeat the process of updating the U according to the gradient information using the preset gradient descent algorithm. 2 -Net model parameters to obtain the trained target blood vessel segmentation model.

6. A retinal vascular imaging segmentation device, characterized in that: include: An image acquisition module, used for acquiring a retinal blood vessel image to be segmented generated after imaging the retinal blood vessels using a preset blood vessel imaging method; an image recognition module, used for inputting the retinal blood vessel image to be segmented into a trained target blood vessel segmentation model, so as to respectively perform full blood vessel recognition, large blood vessel recognition, microvessel recognition, arterial blood vessel recognition and venous blood vessel recognition on the retinal blood vessel image to be segmented through a target full blood vessel recognition sub-model, a target large blood vessel recognition sub-model, a target arterial blood vessel recognition sub-model and a target venous blood vessel recognition sub-model in the target blood vessel segmentation model, and obtain corresponding process blood vessel recognition images respectively; An image segmentation module, used for binarizing the process blood vessel recognition image by using the target blood vessel segmentation model to obtain a binarized whole blood vessel image, a binarized large blood vessel image, a binarized microvascular image, a binarized arterial blood vessel image, and a binarized venous blood vessel image; The retinal vascular imaging segmentation device is used to use the retinal vascular training image carrying vascular label data to segment the retinal vascular image. 2 -Net model to perform model training so that the U 2 The convolution layer of the -Net model uses a convolution kernel to slide on the retinal vascular training image based on the vascular label data to calculate the convolution feature of each image position to obtain a feature map; 2 -Net model performs a pooling operation on the feature map to reduce the size of the feature map and extract target features including the center position of the blood vessel and the edge position of the blood vessel; 2 -Net model's U-shaped residual block structure extracts local features of the feature map including vascular microstructures and vascular branches; 2 -Net model and performs a deconvolution operation on the target features and local features based on the blood vessel label data to update the U 2 -Net model parameters to obtain the trained target blood vessel segmentation model; The retinal vascular imaging segmentation device is also used to 2 -Net model and performs deconvolution operations on target features and local features based on each blood vessel type and each blood vessel boundary in the blood vessel label data to update the U 2 -Net model parameters, and obtain the trained target vessel segmentation model including the target full vessel recognition sub-model, the target large vessel recognition sub-model, the target arterial vessel recognition sub-model, and the target venous vessel recognition sub-model.

7. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the steps of the retinal vascular imaging segmentation method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: Used to store computer programs; wherein, when the computer program is executed by a processor, the steps of the retinal vascular imaging segmentation method according to any one of claims 1 to 5 are implemented.

Images