Eye fundus image blood vessel segmentation method and related equipment

Through the vascular segmentation model trained by global convolution network and multi-scale convolution network, data augmentation technology is used to automatically segment blood vessels in fundus images, solving the problem of doctors' accuracy and efficiency in manually segmenting blood vessels, and improving the accuracy of quantitative disease analysis Sex and efficiency.

CN119068181APending Publication Date: 2024-12-03HE UNIV +1
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202310632129.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-30
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

In the prior art, doctors manually segment blood vessels in fundus images have subjective judgment effects, resulting in low segmentation accuracy and efficiency, affecting the accuracy and efficiency of quantitative disease analysis.

Method used

The global convolution network is adopted to train the vascular segmentation model through multi-scale convolutional neural network and multi-scale graph convolutional network, and use labeled fundus images to enhance data on unlabeled fundus images to generate sufficient image data sets to realize automated vascular segmentation.

Benefits of technology

It improves the accuracy and efficiency of fundus image vascular segmentation, reduces the impact of doctors' subjective judgment, and enhances the reliability and accuracy of quantitative disease analysis.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119068181A_ABST
    Figure CN119068181A_ABST
Patent Text Reader

Abstract

The invention provides an eye fundus image blood vessel segmentation method and related equipment, and aims to improve the blood vessel segmentation effect in an eye fundus image, a plurality of new marked eye fundus images can be synthesized by using a single marked eye fundus image in the stage of training a blood vessel segmentation model, so that enough image data sets are obtained, and the image segmentation efficiency is improved. The blood vessel segmentation model is obtained by performing model training on a global convolutional network formed by a multi-scale convolutional neural network and a multi-scale image convolutional network, so that after an eye fundus image of a to-be-recognized eye is sliced into a plurality of to-be-segmented eye fundus images with a preset size, the to-be-segmented eye fundus images can be input into the blood vessel segmentation model, and the blood vessel segmentation efficiency is improved. And the blood vessel segmentation image of the to-be-identified eye is rapidly and accurately output.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application mainly relates to the field of medical image processing technology, and in particular to a fundus image blood vessel segmentation method and related equipment. Background Art

[0002] Fundus images can usually reflect symptoms of diseases such as diabetes and hypertension. The vascular image can be first segmented from the fundus image, and then the doctor can observe the changes in the vascular structure in the vascular image to assist the doctor in achieving quantitative analysis of the disease.

[0003] Among them, due to the complex morphology of fundus blood vessels, if doctors manually segment blood vessel images based on experience, the accuracy of segmentation can easily be affected by the doctor's subjective judgment. At the same time, the segmentation efficiency of this manual segmentation of blood vessel images is also very low, which in turn affects the accuracy and efficiency of the quantitative analysis results of the disease determined by the assistant doctor. Summary of the invention

[0004] In order to solve the above problems, this application proposes the following technical solutions:

[0005] On the one hand, the present application proposes a fundus image blood vessel segmentation method, the method comprising:

[0006] Obtaining a fundus image of an eye to be identified;

[0007] Slicing the fundus image to obtain a plurality of fundus images to be segmented of a preset size;

[0008] Inputting the plurality of fundus images to be segmented into a trained blood vessel segmentation model, and outputting a blood vessel segmentation image of the eye to be identified;

[0009] Among them, the blood vessel segmentation model is obtained by training a global convolutional network through an image data set, and the global convolutional network is composed of a multi-scale convolutional neural network and a multi-scale graph convolutional network. The image data set is obtained by performing data enhancement processing on multiple unlabeled fundus images through a first labeled fundus image and a first label of the first labeled fundus image.

[0010] Optionally, using a labeled fundus image to perform data enhancement processing on multiple unlabeled fundus images to obtain the image data set includes:

[0011] Obtaining a first labeled fundus image, a first label of the first labeled fundus image, and a plurality of unlabeled fundus images;

[0012] According to the spatial and appearance transformation model, obtaining the spatial difference and appearance difference between the first labeled fundus image and each of the unlabeled fundus images, so as to synthesize the corresponding second labeled fundus image and the second label;

[0013] An image data set is constructed using the first labeled fundus image, the first label, a plurality of the second labeled fundus images and the corresponding second labels.

[0014] Optionally, the space and appearance transformation model includes learned independent space transformation network and appearance transformation network;

[0015] The step of obtaining the spatial difference and appearance difference between the first labeled fundus image and each of the unlabeled fundus images based on the spatial and appearance transformation model to synthesize the corresponding second labeled fundus image and the second label includes:

[0016] Inputting the first labeled fundus image and the plurality of unlabeled fundus images into the appearance transformation network to obtain an appearance difference between the first labeled fundus image and each of the unlabeled fundus images;

[0017] Inputting the first labeled fundus image and the plurality of unlabeled fundus images into the spatial transformation network to obtain a spatial difference between the first labeled fundus image and each of the unlabeled fundus images;

[0018] Obtaining a corresponding second labeled fundus image through transformation processing according to the appearance difference and the spatial difference corresponding to the first labeled fundus image and the same unlabeled fundus image;

[0019] The first label and the spatial difference corresponding to the same unlabeled fundus image are transformed to obtain a second label corresponding to the second labeled fundus image.

[0020] Optionally, inputting the plurality of fundus images to be segmented into a trained blood vessel segmentation model and outputting the blood vessel segmentation image of the eye to be identified includes:

[0021] Input each of the fundus images to be segmented into a multi-scale convolutional neural network for feature extraction to obtain a first feature map of a corresponding scale; the first feature map of any scale is obtained by processing the last convolution layer of the corresponding scale in the multi-scale convolutional neural network;

[0022] Inputting the first feature maps of different scales into the corresponding multi-scale graph convolutional network to perform global feature extraction, so as to obtain second feature maps of corresponding scales;

[0023] The second feature maps at multiple scales are decoded and processed to output a blood vessel segmentation image of the eye to be identified.

[0024] Optionally, the decoding process of the second feature maps at multiple scales to obtain the blood vessel segmentation image of the eye to be identified includes:

[0025] Upsampling the second feature maps of adjacent smaller scales in the ascending order of the scales of the plurality of second feature maps, and fusing the obtained third feature maps with the second feature maps of adjacent larger scales, so as to increase the semantic information of the second feature maps of the adjacent larger scales lost during the acquisition process;

[0026] It is determined that the adjacent larger-scale second feature map of this fusion is the second feature map of the maximum scale, and based on the fourth feature map obtained by this fusion, the blood vessel segmentation image of the eye to be identified is output.

[0027] Optionally, the step of inputting the first feature maps of different scales into the corresponding multi-scale graph convolution network for global feature extraction to obtain the second feature maps of the corresponding scales includes:

[0028] The first feature map of any scale is input into the first convolution group and the second convolution layer for processing, respectively, to obtain the corresponding fifth feature map; the first convolution group and the second convolution group are formed by splitting the convolution kernel of the first scale, and the deployment positions of the convolution layers of the same scale of the first convolution group and the second convolution group are opposite;

[0029] The fifth feature maps are respectively input into the convolutional layer of the second scale for processing, and the processed feature maps are fused to obtain a second feature map with the same scale as the first feature map; the second scale is smaller than the first scale.

[0030] Optionally, in the process of training the global convolutional network through the image dataset to obtain the blood vessel segmentation model, a loss value between the predicted segmented image obtained in each training and the label of the corresponding input marked eye image in the image dataset is obtained through a binary cross entropy function;

[0031] It is determined that the loss value obtained from this training satisfies the training termination condition, and the global convolutional network obtained from this training constitutes the blood vessel segmentation model.

[0032] On the other hand, the present application also proposes a fundus image blood vessel segmentation device, the device comprising:

[0033] A fundus image acquisition module, used to obtain a fundus image of an eye to be identified;

[0034] A slicing processing module, used for slicing the fundus image to obtain a plurality of fundus images to be segmented of a preset size;

[0035] A blood vessel segmentation processing module, used for inputting the plurality of fundus images to be segmented into a trained blood vessel segmentation model, and outputting a blood vessel segmentation image of the eye to be identified;

[0036] Among them, the blood vessel segmentation model is obtained by training a global convolutional network through an image data set, and the global convolutional network is composed of a multi-scale convolutional neural network and a multi-scale graph convolutional network. The image data set is obtained by performing data enhancement processing on multiple unlabeled fundus images through a first labeled fundus image and a first label of the first labeled fundus image.

[0037] On the other hand, the present application also proposes a computer device, comprising:

[0038] Communication interface;

[0039] A memory for storing a program for implementing the fundus image blood vessel segmentation method as described above;

[0040] The processor is used to load and execute the program stored in the memory to implement the fundus image blood vessel segmentation method as described above.

[0041] On the other hand, the present application also proposes a storage medium on which at least one set of computer instruction sets is stored. The computer instruction sets are loaded and executed by a processor to implement the fundus image blood vessel segmentation method as described above.

[0042] It can be seen that the present application proposes a fundus image blood vessel segmentation method and related equipment. In order to improve the blood vessel segmentation effect in the fundus image, in the stage of training the blood vessel segmentation model, a single labeled fundus image can be used to synthesize multiple new labeled fundus images, so as to obtain a sufficient number of image data sets, and a global convolutional network composed of a multi-scale convolutional neural network and a multi-scale graph convolutional network is trained to obtain a model. In this way, after the fundus image of the eye to be identified is sliced ​​into multiple fundus images to be segmented of preset sizes, the fundus image to be segmented can be input into the blood vessel segmentation model, and the blood vessel segmentation image of the eye to be identified can be output quickly and accurately. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0044] Figure 1 A flowchart of an optional example of the fundus image blood vessel segmentation method proposed in this application;

[0045] Figure 2 A flowchart of another optional example of the fundus image blood vessel segmentation method proposed in the present application;

[0046] Figure 3 A flowchart of another optional example of the fundus image blood vessel segmentation method proposed in the present application;

[0047] Figure 4 A schematic diagram of a process for obtaining an image data set for training a blood vessel segmentation model in the fundus image blood vessel segmentation method proposed in the present application;

[0048] Figure 5 A flowchart diagram of another optional example of the fundus image blood vessel segmentation method proposed in the present application;

[0049] Figure 6 A schematic diagram of the network structure of a blood vessel segmentation model applicable to the fundus image blood vessel segmentation method proposed in this application;

[0050] Figure 7 A schematic diagram of the structure of a multi-scale graph convolutional network in a blood vessel segmentation model applicable to the fundus image blood vessel segmentation method proposed in this application;

[0051] Figure 8 A schematic structural diagram of an optional example of the fundus image blood vessel segmentation device proposed in the present application;

[0052] Fig. 9 A schematic diagram of the hardware structure of an optional example of a computer device suitable for the fundus image blood vessel segmentation method proposed in the present application;

[0053] Fig.10 The figure is a schematic diagram of the hardware structure of another optional example of a computer device suitable for the fundus image blood vessel segmentation method proposed in the present application. DETAILED DESCRIPTION

[0054] Regarding the description of the background technology part, with the development and application of artificial intelligence AI technology, in order to improve the efficiency and accuracy of blood vessel segmentation, it is proposed that the blood vessel image can be quickly and accurately segmented from the collected fundus image based on image segmentation technology, that is, through image segmentation technology and a set of image data (i.e., labeled image data) labeled for the vascular image, an image segmentation model is trained, and the actually collected fundus image is input into the image segmentation model to obtain the required blood vessel image, which solves the technical problem caused by the doctor's subjective judgment in the manual method of segmenting the blood vessel image from the fundus image.

[0055] However, in the field of medical image applications, the above-mentioned labeled image data is more scarce than natural images, resulting in a small amount of labeled image data, which will affect the output accuracy of the image segmentation model trained therefrom, that is, reduce the accuracy of vascular segmentation of fundus images. In response to this problem, the present application proposes to model the unlabeled image data based on limited labeled image data (such as a labeled fundus image, i.e., a fundus image annotated with a vascular segmentation image), obtain image features in the unlabeled image data, and generate new labeled image data, thereby obtaining a sufficient amount of labeled image data to train the image segmentation model and improve the segmentation performance of the image segmentation model.

[0056] Based on this, the present application can perform data enhancement processing on multiple unlabeled fundus images through a labeled fundus image, that is, a fundus image marked with a vascular segmentation image, to obtain a sufficient number of labeled fundus images to constitute an image data set for training a vascular segmentation model, and the vascular segmentation model can adopt a global convolutional network, which can be composed of a multi-scale convolutional neural network and a multi-scale graph convolutional network to improve the vascular segmentation effect, that is, to improve the accuracy and reliability of vascular segmentation of fundus images, and better assist doctors in determining the patient's fundus lesions and their treatment plans.

[0057] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. For the convenience of description, only the parts related to the relevant inventions are shown in the drawings. In the absence of conflict, the embodiments in the present application and the features in the embodiments can be combined with each other. That is to say, based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0058] Wherein, flow chart is used in the present application to illustrate the operation performed by the system according to the embodiment of the present application.It should be understood that the preceding or following operation is not necessarily performed accurately in order.On the contrary, each step can be processed in reverse order or simultaneously.At the same time, other operations can also be added to these processes, or a certain step or several steps of operation can be removed from these processes.

[0059] Reference Figure 1, is a flowchart of an optional example of the fundus image blood vessel segmentation method proposed in this application. The method can be applied to a computer device, which can be a server such as an independent physical server, a server cluster integrated with multiple physical servers, or a cloud server with cloud computing capabilities, and / or a terminal device with a certain data processing capability, such as a smart medical device, a robot, a desktop computer, etc. Of course, in some scenarios, the terminal device can also interact with the server to implement the fundus image blood vessel segmentation and recognition method proposed in this application. Based on this, Figure 1 As shown, the fundus image blood vessel segmentation and recognition method proposed in this embodiment may include:

[0060] Step S11, obtaining a fundus image of the eye to be identified;

[0061] In the actual application of this application, medical equipment such as fundus cameras can be used to collect images of the eye to be identified, and obtain fundus images of the eye to be identified, that is, fundus retinal images, which can usually characterize the morphological changes of the entire retina, including but not limited to the morphology of retinal blood vessels, whether there are bleeding, exudation, hemangioma, new blood vessels, etc. on the retina, so as to infer the disease of the eye to be identified by determining whether it is abnormal through tissue structure, morphology, and blood vessel changes. This application is explained by taking the scene where the structure and morphology of fundus blood vessels need to be observed as an example.

[0062] Optionally, if the medical device for collecting fundus images has data processing capabilities, the medical device can be used as a computer device to execute the fundus image blood vessel segmentation method proposed in this application; if the medical device does not have data processing capabilities, the fundus image collected of the eye to be identified can be sent to the computer device through wired communication or wireless communication, so that the computer device obtains the fundus image of the eye to be identified. The implementation method of step S11 is not limited to these two implementation methods.

[0063] Step S12, slicing the fundus image to obtain a plurality of fundus images to be segmented of a preset size;

[0064] In practical applications, due to the limited memory of the GPU (Graphics Processing Unit) in the computer device, the collected fundus images of larger size can be sliced ​​first to obtain fundus image slices of preset size, such as multiple fundus images to be segmented of size 256×256, and then sent to the GPU for subsequent processing. The preset size based on the slicing process can be determined according to the input image size requirements of the trained vascular segmentation model, and this application does not limit its value and image slicing processing method.

[0065] Step S13, input multiple fundus images to be segmented into the trained vascular segmentation model, and output the vascular segmentation image of the eye to be identified; the vascular segmentation model is obtained by training a global convolutional network through an image data set, and the global convolutional network is composed of a multi-scale convolutional neural network and a multi-scale graph convolutional network. The image data set is obtained by performing data enhancement processing on multiple unlabeled fundus images through a first labeled fundus image and its first label.

[0066] In combination with the above description of the technical solution of the present application, in order to improve the effect of blood vessel segmentation in fundus images, in the stage of training the blood vessel segmentation model, a single labeled fundus image can be used to synthesize multiple new labeled fundus images, that is, to obtain a sufficient number of image data sets, and to train a global convolutional network composed of a multi-scale convolutional neural network and a multi-scale graph convolutional network. The implementation process can refer to the description of the corresponding embodiment below, and this embodiment will not be described in detail here.

[0067] Among them, the multi-scale convolutional neural network is used as a feature extractor. Through the convolution layer with convolution kernels of different sizes, each input fundus image to be segmented is subjected to feature extraction, and the corresponding feature maps of multiple sizes are output. In this way, image features of different dimensions in the fundus image can be extracted, which improves the comprehensiveness of the image features. After feature extraction by the graph convolution network containing multi-scale convolution layers, a larger receptive field is obtained, which helps to obtain better blood vessel segmentation effect. This application does not restrict the network structures of the multi-scale convolutional neural network and the multi-scale graph convolutional network, which can be determined according to the situation.

[0068] Reference Figure 2 , is a flowchart of another optional example of the fundus image blood vessel segmentation method proposed in the present application. This embodiment can describe an optional implementation method of the fundus image blood vessel segmentation method described above, such as describing the acquisition process of the above-mentioned image data set. Figure 2 As shown, the method may include:

[0069] Step S21, obtaining a first labeled fundus image and its first label, and a plurality of unlabeled fundus images;

[0070] As described above for the vascular segmentation model, in order to quickly obtain sufficient training image data, realize supervised training of the vascular segmentation model, and provide the segmentation effect of the vascular segmentation model, a labeled fundus image (i.e., a labeled fundus image, which can be recorded as a first labeled fundus image to distinguish it from the subsequently synthesized labeled fundus image) and its label (such as a vascular segmentation image, which can be a vascular mask) are obtained. Specifically, they can be obtained from practice, and the present application does not limit the method for obtaining the first labeled fundus image and its first label.

[0071] Since it is often difficult to obtain a large number of annotated fundus images, i.e., labeled fundus images and their labels, in practice, this application proposes to use a labeled fundus image to synthesize a new labeled fundus image. Therefore, a preset number (which can be a large value, determined according to the model supervision training requirements) of multiple unlabeled fundus images can also be obtained to generate a corresponding number of new labeled fundus images and their labels according to the method described below. These unlabeled fundus images can be directly obtained by the above-mentioned medical equipment for various types of eyes (such as eyes of patients with fundus lesions caused by diabetes, hypertension, etc., eyes without any fundus lesions, etc., to improve the diversity of sample images). This application does not limit the acquisition method and order of step S21, which can be determined according to the circumstances.

[0072] Step S22, obtaining the spatial difference and appearance difference between the first labeled fundus image and each unlabeled fundus image according to the spatial and appearance transformation model, so as to synthesize the corresponding second labeled fundus image and the second label;

[0073] In the actual application of the present application, data enhancement can be performed by learning image transformation, that is, learning the difference between the first labeled fundus image and each unlabeled fundus image, and synthesizing a new labeled fundus image and its label based on this, which is recorded as the second labeled fundus image and the corresponding second label. Specifically, an independent spatial and appearance transformation model can be learned to capture the distribution of spatial and appearance differences between the labeled fundus image and each unlabeled fundus image, realize image spatial transformation (spatialtransform) and appearance transformation (appearance transform), synthesize the second labeled image, and distort the first label map through spatial transformation, synthesize the corresponding second label, and ensure that the newly synthesized second labeled image has the correct label. The present application does not limit the network structure of the spatial and appearance transformation model and its learning implementation method.

[0074] Based on the above analysis, the above-mentioned spatial and appearance transformation model may include learned independent spatial transformation networks and appearance transformation networks, which may be composed of convolutional neural networks, and are used to perform appearance transformation and spatial transformation on the first marked fundus image and the unmarked fundus image, respectively, to obtain corresponding appearance difference distribution and spatial difference distribution, and then combine the first marked fundus image and the first label image to synthesize a new second marked fundus image and its second label, respectively. The implementation process of this embodiment will not be described in detail here.

[0075] Step S23, using the first marked fundus image, the first label, a plurality of second marked fundus images and corresponding second labels to form an image data set;

[0076] Step S24, training a global convolutional network including a multi-scale convolutional neural network and a multi-scale graph convolutional network using the image data set to obtain a blood vessel segmentation model.

[0077] Following the above analysis, the present application can directly use the synthesized second labeled fundus image set and its corresponding second label set to form an annotated image data set. Of course, it can also be combined with the first labeled fundus image and its first label to form an image data set, and then the training set can be used to implement the training of the vascular segmentation model. Optionally, the image data set can be divided into a training set and a test set according to a certain ratio, and the training set can be used to implement the training of the vascular segmentation model. The trained vascular segmentation model is tested using the test set, and optimized according to the test results to obtain a high-performance vascular segmentation model. The implementation process can refer to the description of the corresponding part of the embodiment below.

[0078] It can be seen that for medical images such as fundus images, a large number of annotated fundus images are obtained through the above-mentioned method, and the supervised training of the vascular segmentation model is realized, which ensures the segmentation effect of the trained vascular segmentation model, and further ensures the reliability and accuracy of assisting doctors in realizing quantitative analysis of diseases based on vascular segmentation images.

[0079] Reference Figure 3 , which is a flowchart of another optional example of the fundus image blood vessel segmentation method proposed in the present application. This embodiment can describe an optional refined implementation method of the acquisition process of the image data set for training the blood vessel segmentation model proposed above, such as Figure 3 As shown, in the data preparation stage before model training, the computer device can perform but is not limited to the following steps:

[0080] Step S31, obtaining a first labeled fundus image and its first label, and a plurality of unlabeled fundus images;

[0081] The implementation process of step S31 can refer to the description of the corresponding part of the above embodiment, and this embodiment will not be described in detail here. In order to facilitate the description of the data enhancement process, the first marked fundus image can be denoted as x, and its label, that is, the blood vessel segmentation image, can be denoted as l x , so, (x,l x ) can represent a labeled fundus image and its segmentation map, i.e., label, and the unlabeled fundus image can be recorded as y (i) , that is, the i-th unlabeled fundus image, the value of i is determined by the number of unlabeled fundus images obtained, so a group of unlabeled fundus images containing multiple unlabeled fundus images can be {y (i)}

[0082] Step S32, inputting the first labeled fundus image and the multiple unlabeled fundus images into an appearance transformation network to obtain the appearance difference between the first labeled fundus image and each unlabeled fundus image;

[0083] Combined with the above analysis, refer to Figure 4 The data enhancement process diagram shown in FIG. 1 is a schematic diagram showing that the transformation τ(·) from one atlas to another atlas can be defined as the spatial transformation τ s (·) and appearance transformation τ a The combination of (·), that is, the above space and appearance transformation model, can be recorded as τ(·) = τ s (τ a (·)). For the learning of independent appearance transformation network and space transformation network, such as Figure 4 As shown, a convolutional neural network based on the U-Net architecture can be used, and the specific network structure is not described in detail in this application.

[0084] Among them, the appearance change τ a (·) Modeling can be done by adding each pixel in the atlas spatial structure, which can be done by This function is used to calculate each pixel to obtain the appearance difference between the first labeled fundus image and each unlabeled fundus image. It can be said that the learned spatial transformation model is mapped to the atlas space, and then it can be used This function realizes the appearance transformation data ψ of the first labeled image x and the i-th unlabeled fundus image (i) The fusion process is input into the image transformation processor Warp for subsequent processing.

[0085] For the function used to calculate each pixel in the above appearance transformation process It can be estimated based on the convolutional neural network of the U-Net architecture, in which each convolution layer can be configured with a 3×3 convolution kernel to achieve feature extraction, and processed through the LeakyReLU activation layer to avoid the activation value being too small to cause the neuron parameters to fail to update. In the feature extraction process of the encoder, the maximum pooling layer can be used to process the extracted feature map to reduce the spatial resolution. In the decoder, an upsampling layer can be configured to supplement the features lost in the feature extraction process and improve the output accuracy.

[0086] Step S33, inputting the first labeled fundus image and the multiple unlabeled fundus images into a spatial transformation network to obtain a spatial difference between the first labeled fundus image and each unlabeled fundus image;

[0087] The formation of the spatial transformation network can be a smooth pixel-level displacement field u, so the deformation function φ = id + u can be defined, where id is the identity mapping. This function realizes the deformation application of φ to x. In order to obtain the spatial transformation distribution of multiple unlabeled images, we can use This function represents the distortion of the first marked fundus image x to y (i) Therefore, y (i) The approximate inverse deformation of is written as the approximate inverse deformation to x is Regarding the first labeled fundus image x and each unlabeled fundus image y (i) The spatial difference between The specific implementation process is not described in detail in this embodiment.

[0088] Step S34, obtaining a corresponding second marked fundus image through transformation processing according to the appearance difference and spatial difference corresponding to the first marked fundus image and the same unmarked fundus image;

[0089] Step S35, transforming the first label and the spatial difference corresponding to the same unlabeled fundus image to obtain a second label corresponding to the second labeled fundus image;

[0090] like Figure 4 As shown, when obtaining the same unlabeled fundus image y (i) After the appearance difference and spatial difference are obtained, the appearance difference can be added and fused with the first labeled fundus image x. Then, as analyzed above, the fusion processing result and the spatial difference are transformed by warp to synthesize a new labeled fundus image, which is recorded as the second labeled fundus image.

[0091] At the same time, in order to avoid being affected by uncertainties or errors in the spatial transformation network, the above-mentioned spatial difference and the first label can be used for warp transformation processing to determine that the newly synthesized labeled image is correctly labeled, that is, to ensure that the synthesized label for the second labeled fundus image, that is, the second label, is correct. This application does not describe in detail the implementation process of the warp transformation processing.

[0092] Step S36, constructing an image data set using the first labeled fundus image, the first label, a plurality of second labeled fundus images and the corresponding second labels.

[0093] Reference Figure 5 , which is a flow chart of another optional example of the fundus image blood vessel segmentation method proposed in the present application, can describe another optional refined implementation of the fundus image blood vessel segmentation method proposed in the above embodiment, such as Figure 5 As shown, the method may include:

[0094] Step S51, obtaining a fundus image of the eye to be identified;

[0095] Step S52, slicing the fundus image to obtain a plurality of fundus images to be segmented of a preset size;

[0096] Step S53, inputting each fundus image to be segmented into a multi-scale convolutional neural network for feature extraction to obtain a first feature map of the corresponding scale; the first feature map of any scale is obtained by processing the last convolution layer of the corresponding scale in the multi-scale convolutional neural network;

[0097] Reference Figure 6 The network structure diagram of the blood vessel segmentation model shown in the figure can use an encoder and decoder structure to obtain a preset size (this application uses 256×256 as an example) of the fundus image slice to be segmented, which can be input into the encoder for feature extraction. Figure 6 As shown in FIG. 1 , the fundus image to be segmented of size 256×256×3 is convolved with a convolution kernel of size 3×3 in the first convolution layer according to a step size of 1, and a first feature map of size 256×256×48 is output.

[0098] Then, after passing through the 11×11 convolution kernel in the second convolution layer, convolution processing is performed according to a step size of 2, and a first feature map of 128×128×48 is output. After being processed by the maximum pooling layer, such as the fourth convolution layer processing, a feature map of 64×64×48 is output. The feature of the fifth convolution layer with a convolution kernel of 3×3 and a step size of 1 is extracted, and a first feature map of 64×64×128 is output. After being processed by the maximum pooling layer, such as the processing of the sixth convolution layer, a feature map of 32×32×128 is output.

[0099] After that, two convolutional layers with a convolution kernel size of 3×3 and a step size of 1, namely the seventh convolutional layer and the eighth convolutional layer, continue to extract features from the input feature map, and output the first feature map of size 32×32×128. After being processed by the maximum pooling layer, as shown in Figure 6 The ninth convolutional layer in the output image is processed and the first feature map of size 16×16×128 is output.

[0100] It can be seen that the present application uses multiple convolutional layers of multiple scales to form an encoder. In the process of extracting features from the segmented fundus image, image features of different levels / dimensions can be extracted, which enriches the output feature map and helps improve the segmentation effect. It should be noted that the number of convolutional layers contained in the above-mentioned multi-scale convolutional neural network, as well as the convolutional layer size and step size of each convolutional layer, can be adaptively adjusted according to actual needs and are not limited to Figure 6 The convolution scales shown.

[0101] Step S54, inputting the first feature map of any scale into the first convolution group and the second convolution layer for processing respectively, to obtain the corresponding fifth feature map; the first convolution group and the second convolution group are formed by splitting the convolution kernel of the first scale, and the deployment positions of the convolution layers of the same scale of the first convolution group and the second convolution group are opposite;

[0102] Step S55, inputting the fifth feature map into the convolution layer of the second scale for processing, fusing the processed feature maps to obtain a second feature map of the same scale as the first feature map; the second scale is smaller than the first scale;

[0103] Following the above analysis, after obtaining the first feature maps of multiple scales of the same fundus image to be segmented, the first feature maps of different scales can be input into the corresponding multi-scale graph convolutional network GCN (Graph Convolutional Network) for global feature extraction to obtain the second feature map of the corresponding scale. Among them, since larger-sized convolution kernels are more efficient than stacked small-sized convolution kernels, each GCN can use a larger-sized convolution layer to obtain a larger receptive field and better segmentation effect. This application does not limit the network structure of GCN.

[0104] Optional, see Figure 7 The schematic diagram of the structure of the multi-scale graph convolutional network shown in FIG. If the first two convolutional layer structures on the left and right sides of the network used to process the input feature map are respectively recorded as the first convolution group and the second convolution layer, as shown in FIG. Figure 7 As shown, the first convolution group and the second convolution group can be split by convolution kernels of a first scale (such as 11×11 size), and the deployment positions of the convolution layers of the same scale of the first convolution group and the second convolution group are opposite.

[0105] Therefore, for the first feature map of any scale (which can be recorded as w×h×c, image width×height×number of channels) output by the encoder, after inputting the multi-scale graph convolutional network GCN, it can be input into the first convolution group and the second convolution layer for processing respectively, and the obtained fifth feature map can be input into the convolution layer of the second scale (such as Figure 7 The feature map is fused by the Sum function to obtain a feature map with the same scale as the first feature map (i.e., w×h), as shown in Figure 6 As shown, from small to large (i.e., from low level to high level), there are second feature maps of size 16×16×2, second feature maps of size 32×32×2, second feature maps of size 64×64×2, second feature maps of size 128×128×2, second feature maps of size 256×256×2, etc. The implementation process is not described in detail in this application.

[0106] Step S56, upsampling the adjacent second feature maps of smaller scales in the ascending order of the scales of the multiple second feature maps, and fusing the obtained third feature map into the adjacent second feature map of larger scale, so as to increase the semantic information lost in the acquisition process of the adjacent second feature map of larger scale;

[0107] Step S57, determining that the adjacent second feature map of larger scale fused this time is the second feature map of maximum scale, and outputting the blood vessel segmentation image of the eye to be identified according to the fourth feature map obtained by this fusion.

[0108] As analyzed above, the multi-scale feature map of the fundus image of the eye to be identified is extracted from different stages of the encoder. The decoder of the global convolutional network proposed in this application can generate a multi-scale semantic map for each category. After the low-level feature map is upsampled, it is added to the high-level feature map to supplement the lost semantic information and generate a new feature map, that is, the adjacent smaller-scale second feature map is upsampled, and the obtained third feature map is fused to the adjacent larger-scale second feature map. The decoding is carried out in this way until it is determined that the adjacent larger-scale second feature map fused this time is the second feature map of the largest scale. The final feature map can be classified and predicted by the final semantic score map to obtain a vascular segmentation image.

[0109] Reference Figure 6 In the decoding process shown in the figure, the first feature map output by the last layer of the encoder is processed by GCN to obtain a second feature map of size 16×16×2, and after upsampling, a third feature map of size 32×32×2 is output, which is added to the second feature map of size 32×32×2 output by GCN connected to the adjacent previous layer, that is, the eighth convolutional layer, i.e., fusion processing is performed, and the fused feature map is continued to be upsampled to obtain a feature map of size 64×64×2 and a feature map of size 64×64×2 output by GCN connected to the corresponding fifth convolutional layer. The second feature map of size 256×256×2 is fused, and the fused feature map is upsampled to obtain a feature map of size 128×128×2 and a second feature map of size 128×128×2 output by the corresponding GCN connected to the second convolutional layer. The fused feature map is upsampled to obtain a feature map of size 256×256×2 and a second feature map of size 256×256×2 output by the corresponding GCN connected to the first convolutional layer to obtain a semantic score map, and a vessel segmentation image is output based on the semantic score map.

[0110] When a multi-task image recognition model processes an eye image to be recognized, the task reference data of each of the multiple tasks obtained can be directly displayed on the display of a computer device, such as displaying the multiple task reference data on the eye image to be recognized output by the display, or outputting the multiple task reference data in another display area different from the display area where the eye image to be recognized is located, or displaying the multiple task reference data alone, etc. The present application does not impose any restrictions on the implementation method of outputting multiple task reference data on the display.

[0111] In actual applications, for the obtained vascular segmentation image of the eye to be identified, a fundus detection report including the vascular segmentation image and the fundus image can be generated according to a preset output format, and then the fundus detection report can be output, such as display output through a display, or print output through a printing device connected to the computer device for easy viewing by patients or other doctors, etc. Of course, the fundus image and vascular segmentation image can also be directly displayed on a display configured by the computer device to assist doctors in achieving quantitative analysis of fundus diseases. This application does not limit the output method of the vascular segmentation image, which can be determined according to actual needs.

[0112] In the fundus image blood vessel segmentation method described in the above embodiments, the training of the blood vessel segmentation model used can be combined with the above Figure 6 and Figure 7 The network structure shown and its corresponding processing process are implemented, that is, after replacing the fundus image of the eye to be identified with each marked fundus image in the image data set, the vascular segmentation image of the marked fundus image can be predicted according to the fundus image vascular segmentation method described above, and recorded as the predicted segmentation image. After that, the loss value between the predicted segmentation image and the label of the marked fundus image is calculated, and the parameter adjustment of the vascular segmentation model is implemented accordingly. The vascular segmentation model with adjusted parameters is continuously trained by the marked fundus image until the training termination condition is met, such as the loss value converges, and the training can be stopped. The vascular segmentation model is constituted by the global convolutional network obtained by the final training.

[0113] Optionally, use the binary cross entropy function (binary_crossentropy) to obtain the loss value between the predicted segmented image obtained for each training and the label of the corresponding input labeled eye image in the image dataset The binary cross entropy function is as follows:

[0114]

[0115] In the binary cross entropy function, o can represent the label of the marked fundus image, that is, the real segmented image, and s can represent the predicted segmented image of the marked fundus image. It should be noted that the method for calculating the loss value between the predicted segmented image and the label of the marked fundus image includes but is not limited to the binary cross entropy calculation method described above, and other loss functions can also be used for implementation. The implementation process is not described in detail in this application.

[0116] Reference Figure 8 , is a schematic diagram of an optional exemplary structure of a fundus image blood vessel segmentation device proposed in this application, and the device may include:

[0117] A fundus image acquisition module 81, used to obtain a fundus image of an eye to be identified;

[0118] A slicing processing module 82, used for slicing the fundus image to obtain a plurality of fundus images to be segmented of a preset size;

[0119] The blood vessel segmentation processing module 83 is used to input the multiple fundus images to be segmented into the trained blood vessel segmentation model, and output the blood vessel segmentation image of the eye to be identified;

[0120] Among them, the blood vessel segmentation model is obtained by training a global convolutional network through an image data set, and the global convolutional network is composed of a multi-scale convolutional neural network and a multi-scale graph convolutional network. The image data set is obtained by performing data enhancement processing on multiple unlabeled fundus images through a first labeled fundus image and a first label of the first labeled fundus image.

[0121] Optionally, the image data set obtaining module in the above device for performing data enhancement processing on multiple unlabeled fundus images using a labeled fundus image to obtain the image data set may include:

[0122] A data acquisition unit, configured to acquire a first labeled fundus image, a first label of the first labeled fundus image, and a plurality of unlabeled fundus images;

[0123] A space and appearance transformation processing unit, configured to obtain a space difference and an appearance difference between the first labeled fundus image and each of the unlabeled fundus images according to a space and appearance transformation model, so as to synthesize a corresponding second labeled fundus image and a second label;

[0124] The image data set forming unit is used to form an image data set by using the first marked fundus image, the first label, a plurality of the second marked fundus images and the corresponding second labels.

[0125] The above-mentioned space and appearance transformation model includes a learned independent space transformation network and appearance transformation network. Based on this, the above-mentioned space and appearance transformation processing unit may include:

[0126] an appearance difference obtaining unit, configured to input the first labeled fundus image and the plurality of unlabeled fundus images into the appearance transformation network, and obtain an appearance difference between the first labeled fundus image and each of the unlabeled fundus images;

[0127] A spatial difference obtaining unit, configured to input the first labeled fundus image and the plurality of unlabeled fundus images into the spatial transformation network, and obtain a spatial difference between the first labeled fundus image and each of the unlabeled fundus images;

[0128] A first transformation processing unit, configured to obtain a corresponding second labeled fundus image through transformation processing according to the appearance difference and the spatial difference corresponding to the first labeled fundus image and the same unlabeled fundus image;

[0129] The second transformation processing unit is used to transform the first label and the spatial difference corresponding to the same unlabeled fundus image to obtain a second label corresponding to the second labeled fundus image.

[0130] Based on the description of the above embodiment, the blood vessel segmentation processing module 83 may include:

[0131] A first feature map obtaining unit is used to input each of the fundus images to be segmented into a multi-scale convolutional neural network for feature extraction to obtain a first feature map of a corresponding scale; the first feature map of any scale is obtained by processing the last convolution layer of the corresponding scale in the multi-scale convolutional neural network;

[0132] A second feature map obtaining unit, used for inputting the first feature maps of different scales into the corresponding multi-scale graph convolution network for global feature extraction to obtain a second feature map of the corresponding scale;

[0133] A decoding processing unit is used to decode the second feature maps at multiple scales and output a blood vessel segmentation image of the eye to be identified.

[0134] Optionally, the decoding processing unit may include:

[0135] A first processing subunit is used to upsample the second feature maps of adjacent smaller scales in the ascending order of the scales of the plurality of second feature maps, and fuse the obtained third feature map into the second feature map of adjacent larger scale, so as to increase the semantic information of the second feature map of adjacent larger scale lost during the acquisition process;

[0136] The blood vessel segmentation image output subunit is used to determine that the adjacent larger-scale second feature map fused this time is the second feature map of the maximum scale, and output the blood vessel segmentation image of the eye to be identified based on the fourth feature map obtained by this fusion.

[0137] Optionally, the second feature map obtaining unit may include:

[0138] The fifth feature map obtaining subunit is used to input the first feature map of any scale into the first convolution group and the second convolution layer for processing, so as to obtain the corresponding fifth feature map; the first convolution group and the second convolution group are formed by splitting the convolution kernel of the first scale, and the deployment positions of the convolution layers of the same scale of the first convolution group and the second convolution group are opposite;

[0139] The second feature map obtaining subunit is used to input the fifth feature map into the convolution layer of the second scale for processing, fuse the processed feature maps, and obtain a second feature map with the same scale as the first feature map; the second scale is smaller than the first scale.

[0140] Based on the device structure described in the above embodiments, the model training module for training the global convolutional network through the image data set to obtain the blood vessel segmentation model may include:

[0141] A loss value obtaining unit, used for obtaining, in the process of obtaining the blood vessel segmentation model by training the global convolutional network through the image data set, a loss value between the predicted segmented image obtained in each training and the label of the corresponding input marked eye image in the image data set by using a binary cross entropy function;

[0142] The model obtaining unit is used to determine whether the loss value obtained in this training satisfies the training termination condition, and the global convolutional network obtained in this training constitutes the blood vessel segmentation model.

[0143] It should be noted that the various modules, units, etc. in the above-mentioned device embodiments can be stored in the memory as program modules, and the processor executes the above-mentioned program modules stored in the memory to implement the corresponding functions. For the functions implemented by each program module and its combination, as well as the technical effects achieved, please refer to the description of the corresponding parts of the above-mentioned method embodiments, which will not be repeated in this embodiment.

[0144] The present application also provides a storage medium on which a computer program can be stored. The computer program can be called and loaded by a processor to implement the various steps of the fundus image blood vessel segmentation method described in the above embodiment. The specific implementation process can refer to the description of the corresponding part of the above embodiment, which will not be repeated in this embodiment.

[0145] Reference Fig. 9 , is a schematic diagram of the hardware structure of an optional example of a computer device suitable for the fundus image blood vessel segmentation method proposed in this application, such as Fig. 9 As shown, the computer device may include: a communication interface 91, a memory 92 and a processor 93, wherein:

[0146] The number of communication interface 91, memory 92 and processor 93 can be at least one, and the communication interface 91, memory 92 and processor 93 can all be connected to a communication bus to achieve data interaction between each other through the communication bus. The specific implementation process can be determined according to the requirements of the specific application scenario and will not be described in detail in this application.

[0147] The communication interface 91 may include a communication interface that can realize data interaction by using a wireless communication network, such as a WIFI module, a 5G / 6G (fifth generation mobile communication network / sixth generation mobile communication network) module, a GPRS module, and other corresponding communication interfaces, so as to realize wireless communication between the computer device and other devices; of course, the communication interface 91 may also be a data interface for realizing wired communication, so that the computer device can be connected to other devices through a corresponding type of data cable to realize data transmission between different devices, such as the collected fundus images, the segmented blood vessel segmentation images, etc.

[0148] It should be understood that the above-mentioned communication interface 91 includes a communication interface for realizing data interaction between internal components of a computer device, such as a USB interface, a serial / parallel port, a multimedia transmission interface, etc. This application does not limit the type and number of the communication interface 91.

[0149] In an embodiment of the present application, the memory 9 can be used to store a program for implementing the fundus image blood vessel segmentation method described in any of the above method embodiments; the processor 93 can load and execute the program stored in the memory 92 to implement the various steps of the fundus image blood vessel segmentation method proposed in any of the above method embodiments of the present application. The specific implementation process can refer to the description of the corresponding part of the corresponding embodiment above and will not be repeated here.

[0150] In practical applications, the memory 92 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device or other volatile solid-state storage device. The processor 93 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, etc. The types of the memory 92 and the processor 93 may be determined according to actual needs.

[0151] It should be understood that Fig. 9 The structure of the computer device shown does not constitute a limitation on the computer device in the embodiment of the present application. In practical applications, the computer device may include Fig. 9 More components shown, or a combination of certain components, may be determined according to the product type of the computer device. In the case where the computer device is a terminal device, such as Fig.10 As shown, the computer device may also include: at least one output component such as an image acquisition component 94, a display 95, a speaker 96, and may also include at least one input component such as a microphone 97, a function button, etc. as needed to meet different input operations on the computer device, as well as a sensor module containing multiple sensors, a power supply component, etc., which can be determined according to the application scenario and are not listed one by one in this application.

[0152] Finally, it should be noted that the "system", "device", "unit" and / or "module" used in this application is a method for distinguishing different components, elements, parts, parts or assemblies at different levels. However, if other words can achieve the same purpose, the word can be replaced by other expressions.

[0153] As shown in this application and claims, unless the context clearly indicates an exception, the words "a", "an", "a kind" and / or "the" do not refer to the singular, but also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of clearly identified steps and elements, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements. The elements defined by the sentence "includes a..." do not exclude the existence of other identical elements in the process, method, commodity or device that includes the elements.

[0154] Among them, in the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" refers to two or more than two. The following terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features.

[0155] In addition, the various embodiments in this specification are described in a progressive or parallel manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the various embodiments can be referred to each other. For the devices and computer equipment disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0156] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for segmenting blood vessels in fundus images, characterized in that: The method comprises: Obtaining a fundus image of an eye to be identified; Slicing the fundus image to obtain a plurality of fundus images to be segmented of a preset size; Inputting the plurality of fundus images to be segmented into a trained blood vessel segmentation model, and outputting a blood vessel segmentation image of the eye to be identified; Among them, the blood vessel segmentation model is obtained by training a global convolutional network through an image data set, and the global convolutional network is composed of a multi-scale convolutional neural network and a multi-scale graph convolutional network. The image data set is obtained by performing data enhancement processing on multiple unlabeled fundus images through a first labeled fundus image and a first label of the first labeled fundus image.

2. The method according to claim 1, characterized in that: The method of using a labeled fundus image to perform data enhancement processing on a plurality of unlabeled fundus images to obtain the image data set includes: Obtaining a first labeled fundus image, a first label of the first labeled fundus image, and a plurality of unlabeled fundus images; According to the spatial and appearance transformation model, obtaining the spatial difference and appearance difference between the first labeled fundus image and each of the unlabeled fundus images, so as to synthesize the corresponding second labeled fundus image and the second label; An image data set is constructed using the first labeled fundus image, the first label, a plurality of the second labeled fundus images and the corresponding second labels.

3. The method according to claim 2, characterized in that The spatial and appearance transformation model includes learned independent spatial transformation networks and appearance transformation networks; The step of obtaining the spatial difference and appearance difference between the first labeled fundus image and each of the unlabeled fundus images based on the spatial and appearance transformation model to synthesize the corresponding second labeled fundus image and the second label includes: Inputting the first labeled fundus image and the plurality of unlabeled fundus images into the appearance transformation network to obtain an appearance difference between the first labeled fundus image and each of the unlabeled fundus images; Inputting the first labeled fundus image and the plurality of unlabeled fundus images into the spatial transformation network to obtain a spatial difference between the first labeled fundus image and each of the unlabeled fundus images; Obtaining a corresponding second labeled fundus image through transformation processing according to the appearance difference and the spatial difference corresponding to the first labeled fundus image and the same unlabeled fundus image; The first label and the spatial difference corresponding to the same unlabeled fundus image are transformed to obtain a second label corresponding to the second labeled fundus image.

4. The method according to any one of claims 1 to 3, characterized in that: The step of inputting the plurality of fundus images to be segmented into a trained blood vessel segmentation model and outputting the blood vessel segmentation image of the eye to be identified comprises: Input each of the fundus images to be segmented into a multi-scale convolutional neural network for feature extraction to obtain a first feature map of a corresponding scale; the first feature map of any scale is obtained by processing the last convolution layer of the corresponding scale in the multi-scale convolutional neural network; Inputting the first feature maps of different scales into the corresponding multi-scale graph convolutional network to perform global feature extraction, so as to obtain second feature maps of corresponding scales; The second feature maps at multiple scales are decoded and processed to output a blood vessel segmentation image of the eye to be identified.

5. The method according to claim 4, characterized in that The decoding process of the second feature maps at multiple scales to obtain the blood vessel segmentation image of the eye to be identified includes: Upsampling the second feature maps of adjacent smaller scales in the ascending order of the scales of the plurality of second feature maps, and fusing the obtained third feature maps with the second feature maps of adjacent larger scales, so as to increase the semantic information of the second feature maps of the adjacent larger scales lost during the acquisition process; It is determined that the adjacent larger-scale second feature map of this fusion is the second feature map of the maximum scale, and based on the fourth feature map obtained by this fusion, the blood vessel segmentation image of the eye to be identified is output.

6. The method according to claim 4, characterized in that The step of inputting the first feature maps of different scales into the corresponding multi-scale graph convolution network to perform global feature extraction to obtain second feature maps of corresponding scales includes: The first feature map of any scale is input into the first convolution group and the second convolution layer for processing, respectively, to obtain the corresponding fifth feature map; the first convolution group and the second convolution group are formed by splitting the convolution kernel of the first scale, and the deployment positions of the convolution layers of the same scale of the first convolution group and the second convolution group are opposite; The fifth feature maps are respectively input into the convolutional layer of the second scale for processing, and the processed feature maps are fused to obtain a second feature map with the same scale as the first feature map; the second scale is smaller than the first scale.

7. The method according to any one of claims 1 to 3, characterized in that: In the process of training the global convolutional network through the image data set to obtain the blood vessel segmentation model, a loss value between the predicted segmented image obtained in each training and the label of the corresponding input marked eye image in the image data set is obtained through a binary cross entropy function; It is determined that the loss value obtained from this training meets the training termination condition, and the global convolutional network obtained from this training constitutes the blood vessel segmentation model.

8. A fundus image blood vessel segmentation device, characterized in that: The device comprises: A fundus image acquisition module, used to obtain a fundus image of an eye to be identified; A slicing processing module, used for slicing the fundus image to obtain a plurality of fundus images to be segmented of a preset size; A blood vessel segmentation processing module, used for inputting the plurality of fundus images to be segmented into a trained blood vessel segmentation model, and outputting a blood vessel segmentation image of the eye to be identified; Among them, the blood vessel segmentation model is obtained by training a global convolutional network through an image data set, and the global convolutional network is composed of a multi-scale convolutional neural network and a multi-scale graph convolutional network. The image data set is obtained by performing data enhancement processing on multiple unlabeled fundus images through a first labeled fundus image and a first label of the first labeled fundus image.

9. A computer device, characterized in that: The computer device comprises: Communication interface; A memory, used for storing a program for implementing the fundus image blood vessel segmentation method according to any one of claims 1 to 7; A processor is used to load and execute the program stored in the memory to implement the fundus image blood vessel segmentation method according to any one of claims 1 to 7.

10. A storage medium, characterized in that: At least one set of computer instruction sets is stored thereon, and the computer instruction sets are loaded and executed by a processor to implement the fundus image blood vessel segmentation method according to any one of claims 1 to 7.

Citation Information

Patent Citations

  • Remote sensing image semantic segmentation method based on regional attention multi-scale feature fusion

    CN111797779A

  • Eye fundus image processing method, segmentation model training method and device

    CN115705643A