Eye fundus image optic disc segmentation method and related equipment

Abstract

The invention provides a fundus image optic disk segmentation method and related equipment, and aims to improve the optic disk segmentation effect in a fundus image and accurately position the optic disk in the fundus image. In a trained optic disk segmentation model, a coding network and a decoding network are connected through a graph convolutional network; inputting an optic disk contour graph obtained according to a class activation mapping graph of an optic disk region of the fundus image to be segmented into the graph convolutional network, so that the feature representation corresponding to the feature graph output by the coding network is adjusted according to the optic disk contour graph, and decoding the adjusted feature graph by the decoding network to obtain a segmented optic disk contour graph; and accurately obtaining an optic disc segmentation image in the fundus image to be segmented.

Description

Technical Field

[0001] The present invention mainly relates to the field of digital medical technology, and in particular to a fundus image optic disc segmentation method and related equipment. Background Art

[0002] The optic disc, abbreviated as optic disc, is the gathering place of visual fibers on the retina and the beginning of the optic nerve. As an important feature and component of the retinal fundus image, its shape, size and other parameters are usually used as important indicators to measure the health status and lesions of the fundus, assisting doctors in diagnosing and treating diseases such as diabetic retinopathy, hypertensive retinopathy, and glaucoma.

[0003] However, in the process of optic disc analysis of fundus images, it is easily interfered by factors such as blood vessel occlusion, uneven brightness, and lesions, which makes it impossible to accurately locate the optic disc, reduce the accuracy of fundus optic disc segmentation, and cannot ensure that the extracted optic disc parameters are accurate and reliable, thereby affecting the accuracy and efficiency of assisting doctors in determining disease analysis results. Summary of the invention

[0004] In order to solve the above problems, this application proposes the following technical solutions:

[0005] On the one hand, the present application proposes a method for optic disc segmentation of fundus images, characterized in that the method comprises:

[0006] Obtaining a fundus image to be segmented;

[0007] Acquire a class activation map of the optic disc area in the fundus image to be segmented;

[0008] According to the class activation map, obtaining an optic disc contour figure in the fundus image to be segmented;

[0009] Inputting the fundus image to be segmented and the optic disc contour graphic into a trained optic disc segmentation model, and outputting an optic disc segmentation image;

[0010] Among them, the optic disc segmentation model includes an encoding network, a graph convolutional network and a decoding network connected in sequence. The encoding network and the decoding network are both constructed based on convolutional neural networks. The graph convolutional network can adjust the feature representation corresponding to the feature map output by the encoding network according to the optic disc contour graphic.

[0011] Optionally, the step of inputting the fundus image to be segmented and the optic disc contour graphic into a trained optic disc segmentation model and outputting an optic disc segmentation image comprises:

[0012] Inputting the fundus image to be segmented into the encoding network to extract features with different resolutions to obtain a first feature map;

[0013] Inputting the first feature map and the optic disc outline graph into the graph convolutional network, and adjusting the feature representation corresponding to the first feature map using the graphic information in the optic disc outline graph to obtain a second feature map;

[0014] Inputting the second feature map into the decoding network, performing decoding processing according to the feature maps of different resolutions extracted by the encoding network, to obtain a third feature map; the feature maps of different resolutions refer to feature maps extracted by the encoding network before the first feature map;

[0015] The third feature map is processed by a unit convolution layer to output an optic disc segmentation image of the fundus image to be segmented.

[0016] Optionally, the encoding network includes a plurality of encoding blocks, each of the encoding blocks includes two convolutional layers and a maximum pooling layer connected in sequence, and each of the convolutional layers has a convolution kernel of the same size so as to perform convolution processing according to the same step length;

[0017] Inputting the fundus image to be segmented into the encoding network to extract features of different resolutions to obtain a first feature map, including:

[0018] Inputting the fundus image to be segmented into the coding network, performing convolution processing of the convolution layers of the plurality of coding blocks connected in sequence and processing of the maximum pooling layer, performing feature extraction on the maximum pooling layer output feature map of the last coding block, and obtaining a first feature map;

[0019] The proportional relationship between the resolutions of the feature maps output by the second convolutional layers of the multiple encoding blocks is determined according to the step size.

[0020] Optionally, the decoding network includes the same number of decoding blocks as the encoding blocks included in the encoding network, each of the decoding blocks includes an upsampling layer, a feature fusion layer and a convolution layer connected in sequence; the convolution kernel size and step size of the convolution layer in the decoding block are the same as those of the convolution layer in the encoding block;

[0021] The step of inputting the second feature map into the decoding network and performing decoding processing according to the feature maps of different resolutions extracted by the encoding network to obtain a third feature map includes:

[0022] Inputting the second feature map into the first decoding block of the decoding network, and obtaining a third feature map output by the convolution layer of the last decoding block according to the feature map output by the second convolution layer of the encoding block in the encoding network, after decoding by a plurality of decoding blocks;

[0023] Among them, in the decoding processing process of each of the decoding blocks, the feature fusion layer of the decoding block adds the feature map output by the upsampling layer and the feature map output by the second convolution layer corresponding to the encoding block in the encoding network, and inputs the obtained fused feature map into the convolution layer of the decoding block for feature extraction, so that the extracted feature map is input into the next decoding block to continue decoding processing.

[0024] Optionally, the step of obtaining a class activation map of the optic disc area in the fundus image to be segmented includes:

[0025] The fundus image to be segmented is cropped according to a preset size to obtain a corresponding plurality of slice images; the preset size is smaller than the size of the optic disc area in the fundus image to be segmented;

[0026] Obtaining image category labels of various slice images; the image category labels are used to indicate whether the corresponding slice image contains at least a part of the optic disc area;

[0027] According to the image category label, a class activation mapping process is performed on the plurality of slice images including at least a portion of the optic disc area to obtain a class activation map of the optic disc area in the fundus image to be segmented.

[0028] Optionally, obtaining the optic disc contour graphic in the fundus image to be segmented according to the class activation map includes:

[0029] Performing point cloud sampling on the class activation map by farthest point sampling to obtain a preset number of sampling point positions for the optic disc area; wherein the initial sampling point during point cloud sampling is a random sampling point or a sampling point with the largest distance from the center of the point cloud;

[0030] Using the preset number of sampling point positions to form first optic disc contour information of the optic disc area;

[0031] Downsampling the first optic disc contour information to obtain second optic disc contour information having a resolution matching a trained optic disc segmentation model;

[0032] An optic disc contour graphic for the fundus image to be segmented is constructed using the second optic disc contour information.

[0033] Optionally, each convolution layer of the encoding network and the decoding network is composed of a convolution kernel of size 3×3, and a convolution operation is performed after the input feature map is moved at intervals of one pixel;

[0034] The graph convolution network includes multiple superimposed graph convolution layers, and each feature point in the output feature map can be adjusted according to information from a high-order neighborhood, and the information of the high-order neighborhood is determined according to the connection relationship between different feature points of the optic disc contour represented by the optic disc contour graphic.

[0035] On the other hand, the present application also proposes a fundus image optic disc segmentation device, the device comprising:

[0036] A fundus image acquisition module to be segmented, used for acquiring the fundus image to be segmented;

[0037] A class activation map acquisition module, used to acquire a class activation map of the optic disc area in the fundus image to be segmented;

[0038] An optic disc contour image acquisition module is used to obtain an optic disc contour image in the fundus image to be segmented according to the class activation map;

[0039] An optic disc segmentation module, used for inputting the fundus image to be segmented and the optic disc contour graphic into a trained optic disc segmentation model, and outputting an optic disc segmentation image;

[0040] Among them, the optic disc segmentation model includes an encoding network, a graph convolutional network and a decoding network connected in sequence. The encoding network and the decoding network are both constructed based on convolutional neural networks. The graph convolutional network can adjust the feature representation corresponding to the feature map output by the encoding network according to the optic disc contour graphic.

[0041] On the other hand, the present application also proposes an electronic device, comprising:

[0042] Communication module;

[0043] A memory for storing a program for implementing the above-mentioned method for segmenting the optic disc of fundus images;

[0044] The processor is used to load and execute the program stored in the memory to implement the above-mentioned method for optic disc segmentation of fundus images.

[0045] On the other hand, the present application further proposes a computer-readable storage medium on which at least one set of computer instruction sets is stored. The computer instruction sets are loaded and executed by a processor to implement the above-mentioned fundus image optic disc segmentation method.

[0046] It can be seen that the present application proposes a method for optic disc segmentation in fundus images and related equipment. In order to improve the effect of optic disc segmentation in fundus images and accurately locate the optic disc in fundus images, in the optic disc segmentation model trained by the present application, the encoding network and the decoding network are connected through a graph convolutional network, and the optic disc contour graphic obtained according to the class activation map of the optic disc area of ​​the fundus image to be segmented is input into the graph convolutional network, so that the feature representation corresponding to the feature map output by the encoding network is adjusted accordingly, and the adjusted feature map is decoded by the decoding network to accurately obtain the optic disc segmentation image in the fundus image to be segmented, thereby effectively assisting doctors to accurately and quickly determine the fundus health status and lesions. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0048] Figure 1 A schematic flow chart of an optional example of the method for optic disc segmentation of fundus images proposed in this application;

[0049] Figure 2 A flowchart diagram of another optional example of the method for optic disc segmentation of fundus images proposed in the present application;

[0050] Figure 3 A schematic flow chart of an optional method for obtaining a class activation map in the method for optic disc segmentation of fundus images proposed in this application;

[0051] Figure 4 A schematic diagram of a process for obtaining an image data set for training an optic disc segmentation model in the fundus image optic disc segmentation method proposed in the present application;

[0052] Figure 5 It is a schematic diagram of an optional structure of an optic disc segmentation model applicable to the optic disc segmentation method of fundus image proposed in the present application;

[0053] Figure 6 A schematic diagram of a graph convolutional network structure in an optic disc segmentation model applicable to the optic disc segmentation method for fundus images proposed in this application;

[0054] Figure 7 A flowchart diagram of another optional example of the method for optic disc segmentation of fundus images proposed in this application;

[0055] Figure 8 is another optional structural schematic diagram of an optic disc segmentation model applicable to the optic disc segmentation method of fundus images proposed in the present application;

[0056] Fig. 9 A schematic structural diagram of an optional example of the fundus image optic disc segmentation device proposed in the present application;

[0057] Fig.10 A schematic structural diagram of another optional example of the fundus image optic disc segmentation device proposed in the present application;

[0058] Fig.11 The figure is a schematic diagram of the hardware structure of an optional example of an electronic device applicable to the method for optic disc segmentation of fundus images proposed in the present application. DETAILED DESCRIPTION

[0059] Regarding the description in the background technology section, with the development and application of artificial intelligence AI technology, in order to improve the efficiency and accuracy of optic disc segmentation, this application proposes a joint convolutional neural network and a graph convolutional network to realize optic disc segmentation of fundus images. Specifically, a graph convolutional network will be used to replace the deep convolution in the codec network. In this way, the local features of the fundus image and the global information based on graph connections can be learned. The graph convolutional network is used to learn graphic features in depth, such as the optic disc contour, and the extracted feature map is corrected accordingly to improve the optic disc segmentation effect.

[0060] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. For the convenience of description, only the parts related to the relevant inventions are shown in the drawings. In the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other. That is to say, based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0061] Among them, the present invention uses a flow chart to illustrate the operations performed by the system according to an embodiment of the present invention. It should be understood that the previous or subsequent operations are not necessarily performed accurately in order. On the contrary, each step can be processed in reverse order or simultaneously. At the same time, other operations can also be added to these processes, or one or more operations can be removed from these processes.

[0062] Reference Figure 1, is a flowchart of an optional example of the method for optic disc segmentation of fundus images proposed in this application. The method can be applied to electronic devices, which can be servers such as independent physical servers, server clusters integrated with multiple physical servers, or cloud servers with cloud computing capabilities, and / or terminal devices with certain data processing capabilities, such as smart medical devices, robots, desktop computers, etc. Of course, in some scenarios, the terminal device can also interact with the server to implement the method for optic disc segmentation of fundus images proposed in this application. Based on this, Figure 1 As shown, the fundus image optic disc segmentation method proposed in this embodiment may include:

[0063] Step S11, obtaining a fundus image to be segmented;

[0064] In actual application of the present application, medical equipment such as a fundus camera can be used to capture images of the patient's eye to be identified to obtain a fundus image of the eye to be identified, that is, a retinal fundus image, so as to measure the fundus health status and lesions by analyzing parameters such as the shape and size of the optic disc. The present application does not limit the method for implementing the acquisition of fundus images.

[0065] Optionally, if the medical device for collecting fundus images has data processing capabilities, the medical device can be used as an electronic device to execute the fundus image optic disc segmentation method proposed in this application; if the medical device does not have data processing capabilities, the fundus image collected of an eye to be identified can be sent to an electronic device through wired communication or wireless communication, so that the electronic device determines each received fundus image as a fundus image to be segmented. The implementation method of step S11 is not limited to these two implementation methods.

[0066] Step S12, obtaining a class activation map of the optic disc area of ​​the fundus image to be segmented;

[0067] In practical applications, class activation mapping (CAM), also known as heat map, saliency map, etc., is used to obtain the approximate position and shape information of the optic disc area in the fundus image to be segmented. CAM can remove the last fully connected layer of the residual network ResNet, use the global average pooling (GAP) layer and the fully connected layer to obtain a larger feature map, avoid the flattening of the feature map caused by the use of the fully connected layer, and embed the spatial position information of the optic disc into the feature map to accurately obtain the position and shape of the optic disc contour in the fundus image to be segmented.

[0068] Step S13, obtaining an optic disc contour graphic in the fundus image to be segmented according to the class activation map;

[0069] Following the above analysis, the class activation map can include the optic disc for the fundus image to be segmented, which can represent the correlation between each pixel in the fundus image to be segmented and the optic disc by different colors. The higher the correlation, the darker the corresponding color. The optic disc area of ​​concern in the entire fundus image to be segmented is determined, that is, the optic disc contour position and graphic information contained in the fundus image to be segmented are determined. The implementation process can be determined with reference to the operation principle of the class activation mapping, and the embodiments of the present application will not be described in detail here.

[0070] Step S14, inputting the fundus image to be segmented and the optic disc contour graphic into the trained optic disc segmentation model, and outputting the optic disc segmentation image.

[0071] In an embodiment of the present application, the optic disc segmentation model may include an encoding network, a graph convolutional network and a decoding network connected in sequence, the encoding network and the decoding network are both constructed based on a convolutional neural network, the convolution block between the two is replaced by a graph convolutional network, and the feature map output by the encoding network and the above-mentioned optic disc contour graphic are used as input, that is, the graph convolutional network can adjust the corresponding features of the feature map output by the encoding network according to the optic disc contour graphic, such as extracting the graphic information of the optic disc contour graphic, multiplying the graphic information with the feature map output by the encoding network, and correcting the corresponding feature representation of the feature map, so that the feature map output by the encoding network contains not only low-level rich feature representations, but also high-level graphic information, and inputs it into the decoding network for decoding processing, so as to accurately obtain the optic disc segmentation image in the fundus image to be segmented.

[0072] It can be seen that the optic disc segmentation model of the above network structure can not only extract local features in the fundus image to be segmented through the encoding network, but also can deeply learn graphic features through the graph convolutional network. It can correct the feature map extracted by the encoding network based on the extracted graphic information of the optic disc contour, improve the optic disc segmentation effect of the decoding network, and improve the segmentation performance of the optic disc segmentation model. This application does not limit the training implementation method of the optic disc segmentation model.

[0073] In this way, when the quality of the fundus image to be segmented is low, and the optic disc is disturbed by factors such as blood vessel occlusion, uneven brightness, and lesions, the present application will obtain the optic disc contour graphic during the optic disc segmentation process, thereby correcting the feature map directly extracted from the image to obtain complete and rich optic disc features, so that the decoding network can accurately locate and segment the optic disc, thereby improving the accuracy of optic disc segmentation for such fundus images to be segmented, so as to assist doctors in accurately determining the health status of the patient's eye to be identified and accurately locating lesions through various indicator parameters of the optic disc.

[0074] Reference Figure 2, which is a flowchart of another optional example of the method for optic disc segmentation of fundus images proposed in the present application. This embodiment can provide a detailed description of an optional implementation process of how to obtain an optic disc contour graphic in the fundus image to be segmented in the method for optic disc segmentation of fundus images described above, such as Figure 2 As shown, the method may include:

[0075] Step S21, obtaining a fundus image to be segmented;

[0076] Step S22, cropping the fundus image to be segmented according to a preset size to obtain a corresponding plurality of slice images; the preset size is smaller than the size of the optic disc area in the fundus image to be segmented;

[0077] The fundus image to be segmented actually collected often contains various components of the fundus, such as the retina, optic disc, blood vessels, etc. In this application, only the optic disc area is needed to obtain the optic disc contour and avoid affecting the image processing efficiency due to the limited memory of the GPU (Graphics Processing Unit) in the electronic device. Figure 3 The schematic diagram of the process of obtaining the optic disc contour shown in FIG. 1 can cut slices of the fundus image to be segmented according to the preset size to obtain multiple slice images, such as Figure 3 As shown, each slice image is a partial image of the fundus image to be segmented, so that some slice images contain part of the optic disc area, and some slice images do not contain the optic disc area. This application needs to analyze the slice images containing the optic disc area to determine the optic disc contour.

[0078] In the embodiment of the present application, in order to improve the cropping efficiency and reduce the interference content of the fundus image on the optic disc contour, the size of each slice image cropped in the present embodiment can be smaller than the size of the optic disc area, and the preset size can have a multiple relationship with the size of the fundus image to be segmented. Figure 3 As shown, the fundus image to be segmented can be directly averagely cropped according to the preset size to obtain multiple slice images with preset sizes. The present application does not limit the image cropping implementation method and the value of the preset size.

[0079] Step S23, obtaining various image category labels of the plurality of slice images; the image category labels are used to indicate whether the corresponding slice image contains at least a portion of the optic disc area;

[0080] As analyzed above, the present application only needs to perform optic disc contour recognition on slice images that contain at least part of the optic disc area. Therefore, the present application can classify each slice image, determine the slice image that contains at least part of the optic disc area, and record it as the slice image to be processed. The present application does not limit the recognition implementation method of this type of slice image.

[0081] Optionally, the present application can train a class discrimination model (such as a binary classifier, etc., and the present application does not impose any restrictions on its model network structure) based on sample slice images with annotated image category labels, and can input multiple slice images obtained by cropping the actually obtained fundus images to be segmented into the class discrimination model for processing to determine whether the corresponding slice image contains at least part of the optic disc area, and obtain the image category label of the corresponding slice image. The image category label indicates whether the corresponding slice image contains at least part of the optic disc area, so that the slice image to be processed that contains at least part of the optic disc area can be determined subsequently based on the image list label.

[0082] In some other embodiments, the present application determines whether the slice image contains a part of the optic disc area. The category judgment is at the image level. According to the content of each slice image displayed, the category input operation can be performed on each slice image to obtain the image category label of the corresponding slice image, such as a Positive label with optic disc or a Negative label without optic disc, so that the electronic device can accurately filter out the slice images to be processed with optic discs. It should be noted that the implementation method of the category judgment of whether the slice image contains a optic disc includes but is not limited to the content described in this embodiment.

[0083] Step S24, performing class activation mapping processing on a plurality of slice images including at least a portion of the optic disc area according to the image category label, to obtain a class activation map of the optic disc area in the fundus image to be segmented;

[0084] Combined with the description of the class activation mapping CAM, a discrimination positioning technology, in the above embodiment, the following formula can be used to obtain the class activation mapping CAM map of the slice image containing the optic disc area, that is, the CAM of the Positive class, which can be recorded as CAM positive :

[0085]

[0086] In the above formula (1), CAM positive It can be a CAM image of a slice image of the optic disc Positive class. F indicates the feature map processed by the pool5 layer after the last fully connected layer of the residual network is removed. F i is the i-th slice of the channel dimension, N channel Indicates the size of the channel dimension, W positive It can represent the weight of the slice image with the optic disc Positive class in the fully connected layer, It can represent F i About CAM positive The contribution weight of .

[0087] Step S25, performing point cloud sampling on the class activation map by using the farthest point sampling method to obtain a preset number of sampling point positions for the optic disc area;

[0088] After obtaining the class activation mapping CAM image of each slice image containing at least part of the optic disc area, each CAM image can be thresholded to obtain the approximate position and shape information of the optic disc, and the point cloud image can be sampled through the farthest point sampling (FPS) method to obtain the entire optic disc contour, but it is not limited to this optic disc contour acquisition method.

[0089] Among them, the farthest point sampling FPS is to uniformly sample M points (i.e., sampling points) on the N-point view disk point cloud, and the positions of these M sampling points represent the overall contour of the view disk point cloud. In the actual process, M sampling points can be iteratively selected on the N-point point cloud, and each selection can be combined with the currently selected sampling point set S = {P0, P1, ..., P i}, the sampling point with the largest minimum distance among all sampling points is added to the set and the next iteration is performed.

[0090] For example, if the input optic disc point cloud of the slice image containing at least part of the optic disc area has N points, a starting sampling point P0 is selected from the optic disc point cloud, such as randomly selecting or selecting the sampling point with the largest distance from the center of the point cloud as P0, thereby obtaining an initial sampling point set S = {P0}, and then, the array L method can be used to record the minimum distance from each of the N points to each sampling point in the sampling point set S, and the distance from each of the N points to the initial sampling point P0 is determined accordingly, and the point P1 corresponding to the maximum distance value can be selected and added to the sampling point set S = {P0, P1}, and the points to be calculated are Calculate the distance from N points to the sampling point P1. If the distance is less than L[i], the distance can be assigned to L[i]. In this way, the array L always maintains the sampling point set S = {P0, P1} to record the corresponding distances. The array L records the minimum distance from all N points to all sampling points in the sampling point set S. Select the point P2 among the N points corresponding to the maximum value, and continue to add it to the sampling point set S = {P0, P1, P2}. In this way, the sampling point selection is recommended until the sampling point set obtains M sampling points. The positions of these M sampling points constitute the optic disc contour in the fundus image to be segmented.

[0091] It should be noted that this application does not limit the numerical value of the preset number M, which can be determined based on actual conditions.

[0092] Step S26, using a preset number of sampling point positions to form first optic disc contour information of the optic disc area;

[0093] Step S27, down-sampling the first optic disc contour information to obtain second optic disc contour information having a resolution matching the trained optic disc segmentation model;

[0094] Step S28, constructing an optic disc contour graphic for the fundus image to be segmented using the second optic disc contour information;

[0095] Following the above analysis, the optic disc contour in the fundus image to be segmented is obtained according to the above method, and the contour information can be recorded as the first optic disc contour information, whose resolution is usually different from the resolution of the input image of the graph convolutional network in the optic disc segmentation model. The present application can downsample the first optic disc contour information according to the resolution of the input image of the graph convolutional network, such as downsampling the sampling points that constitute the first optic disc contour, to obtain the second optic disc contour information that matches the resolution of the input image of the graph convolutional network, such as the positions of the sampling points that constitute the optic disc contour, and the contour graphic information, etc. In this way, the optic disc contour graphic G(V,E) can be constructed accordingly.

[0096] in, It can represent the N vertices constituting the outline graph of the disc, and E can represent the edge connecting two adjacent vertices. This application does not describe the graph construction implementation process in detail.

[0097] It should be noted that during the training process of the optic disc segmentation model, for each sample fundus image in the data set, the optic disc contour graphic in the sample fundus image can be determined according to the method described above, and each group of sample fundus images and optic disc contour graphics can be used to implement the training of the optic disc segmentation model as described in the context. This application does not describe the model training implementation process in detail.

[0098] Step S29, input the fundus image to be segmented and the optic disc contour graphic into the trained optic disc segmentation model, and output the optic disc segmentation image of the fundus image to be segmented through the processing of the encoding network, graph convolution network and decoding network connected in sequence.

[0099] In this embodiment, in combination with the relevant description of the optic disc segmentation model proposed in the present application in the context embodiment, the fundus image to be segmented is input into the encoding network of the optic disc segmentation model for feature extraction, feature maps of different resolutions can be extracted, and the feature map of a certain resolution finally obtained by the encoding network is input into the graph convolution network. At the same time, the optic disc contour graphic obtained by the above method is input into the graph convolution network. For the problem that the optic disc features in the feature map directly extracted are not complete and accurate due to factors such as the optic disc being blocked by blood vessels, uneven brightness, and lesions, the present application can extract local features from the feature map through the graph convolution network, and learn the optic disc contour graphic features in deep layer, that is, based on the global information of graph connection, so as to correct the feature map using the graphic features, thereby improving the completeness and accuracy of the feature map input to the decoding network, thereby improving the accuracy of optic disc segmentation prediction.

[0100] Reference Figure 4 , which is a flowchart of another optional example of the method for optic disc segmentation of fundus images proposed in the present application. This embodiment can provide a detailed description of an optional implementation process of how to use the optic disc segmentation model to obtain an optic disc segmentation image in the fundus image to be segmented in the method for optic disc segmentation of fundus images described above. Regarding the process of obtaining the optic disc contour graphic, reference can be made to the description of the corresponding part of the above embodiment, such as Figure 4 As shown, the method may include:

[0101] Step S41, obtaining a fundus image to be segmented;

[0102] Step S42, obtaining a class activation map of the optic disc area in the fundus image to be segmented;

[0103] Step S43, obtaining an optic disc contour graphic in the fundus image to be segmented according to the class activation map;

[0104] Regarding the implementation process of step S41-step S42, reference may be made to the description of the corresponding parts of step S21-step S28 in the above embodiment, and this embodiment will not be elaborated here.

[0105] Step S44, inputting the fundus image to be segmented into a coding network to extract features at different resolutions to obtain a first feature map;

[0106] As described in the above embodiment, the network structure of the optic disc segmentation model trained in this application is as follows: Figure 5 The structural diagram of the optic disc segmentation model shown in the figure includes an encoding network, a graph convolutional network and a decoding network connected in sequence. The fundus image to be segmented is input into the encoding network for feature extraction. In this process, feature maps of different resolutions can be obtained through different encoding blocks, which are used to input into the decoding network to help the decoding network reconstruct spatial information. The implementation process can refer to the description of the corresponding part below.

[0107] Step S45, inputting the first feature map and the optic disc contour graph into a graph convolutional network, and adjusting the feature representation corresponding to the first feature map using the graphic information in the optic disc contour graph to obtain a second feature map;

[0108] In combination with the context of the relevant description of the optic disc segmentation scheme of the present application, the deep convolution between the encoding network and the decoding network will be replaced by a graph convolutional network, and the first feature map from the encoding network and the obtained optic disc contour image can be used as its input. By extracting the high-level contour graphic information of the optic disc, multiplying it with the first feature map, correcting the feature representation corresponding to the first feature map, a second feature map with rich features and accuracy is obtained.

[0109] In the actual application of this application, the above-mentioned graph convolution GCN network may include multiple superimposed graph convolution GCN layers, such as 4 graph convolution layers, etc. This application does not limit the number of GCN layers constituting the GCN network. Figure 6 The structural diagram of the graph convolutional network shown in Figure 1 is as follows: j The input eigenvector of each vertex of the disc contour graph can be represented, and the matrix of all eigenvectors can be represented as C can represent the dimension of the input feature vector, and the adjacency matrix of the graph can be expressed as After graph convolution processing, this application needs to learn the feature map of each node, and the feature matrix can be expressed as F can represent the dimension of the desired feature map.

[0110] Among them, in the above graph convolution processing process, operations such as feature representation, feature aggregation on the graph and nonlinear activation can be included, and the feedforward propagation in the GCN layer can be:

[0111]

[0112] In the above formula (2), H l Can represent the hidden features of the lth layer; is the normalized adjacency matrix, is the corresponding degree matrix of A+I, I can represent the unit matrix; σ(·) can represent an activation function, such as ReLU function, etc. This application does not limit the type of activation function used in the graph convolutional network, which can be determined according to the situation; W l It can represent the learnable weight matrix of the lth layer.

[0113] Combined with the above analysis, in the multiple graph convolution layers contained in the graph convolution network, several graph convolutions can be superimposed together according to the operation represented by the above formula (2) to obtain more capabilities for learning complex features, so that each feature point in its output feature map can be adjusted according to the information from the high-order neighborhood. The information of the high-order neighborhood is determined according to the connection relationship between different feature points of the optic disc contour represented by the optic disc contour graph, thereby improving the optic disc segmentation decision and improving the subsequent optic disc segmentation effect.

[0114] Step S46, inputting the second feature map into the decoding network, performing decoding processing according to the feature maps of different resolutions extracted by the encoding network, and obtaining a third feature map; the feature maps of different resolutions refer to the feature maps extracted by the encoding network before the first feature map;

[0115] As analyzed above, during the feature extraction process of the input fundus image to be segmented, the encoding network will extract features from different resolutions to obtain feature maps of different resolutions, and connect them with the corresponding blocks in the decoding network by skipping connections to assist the decoding network in reconstructing spatial information. The implementation process can refer to the description of the corresponding part of the embodiment below.

[0116] Among them, in order to improve the learning ability of the convolutional neural network, the above-mentioned encoding network and decoding network can adopt densely connected convolution blocks as basic components, combined with other network layers. Each convolution layer can use a convolution kernel of size 3×3 and perform a convolution operation with a step size of 1, that is, the convolution kernel performs a convolution operation after moving the input feature map by one pixel to obtain a feature map of a new resolution. The composition structure of the encoding network and the decoding network can refer to but is not limited to the network structure described in the corresponding part of the following embodiment.

[0117] Step S47, processing the third feature map through the unit convolution layer, and outputting the optic disc segmentation image of the fundus image to be segmented.

[0118] For the third feature map output by the last decoding block of the decoding network, it can be processed through a 1×1 convolution layer with a sigmoid activation function to obtain the optic disc segmentation image of the fundus image to be segmented. The implementation process is not described in detail in this application.

[0119] Reference Figure 7 , which is a flow chart of another optional example of the method for optic disc segmentation of fundus images proposed in the present application. This embodiment can describe an optional refinement implementation method of the optic disc segmentation model for the optic disc segmentation of the fundus image to be segmented in the method for optic disc segmentation of fundus images described above, such as Figure 7 As shown, the method may include:

[0120] Step S71, inputting the fundus image to be segmented into the encoding network, after feature extraction by two connected convolutional layers in the first encoding block, inputting the obtained feature map with the first resolution into the maximum pooling layer for downsampling, and inputting the obtained feature map with the second resolution into the next encoding block to continue feature extraction;

[0121] Step S72, determining that multiple coding blocks in the coding network complete feature extraction in sequence, and inputting the first feature map output by the maximum pooling layer of the last coding block into the graph convolution network;

[0122] In the embodiment of the present application, the coding network may include multiple coding blocks, each coding block includes two convolutional layers and a maximum pooling layer connected in sequence, and each convolutional layer has a convolution kernel of the same size (such as 3×3) to perform convolution processing according to the same step length (such as 1, etc.), referring to Figure 8 The schematic diagram of the structure of another optional instance of the optic disc segmentation model shown in the first row shows the encoding network. Starting from the left, 4 (but not limited to this number, depending on the situation) encoding blocks are connected in sequence, and each encoding block includes two 3×3×1 convolutional layers and one maximum pooling layer.

[0123] Therefore, after the fundus image to be segmented is input into the coding network, it is subjected to feature extraction of two convolutional layers of size 3×3 and step size 1 in the first coding block, and a feature map of size 256×256×32 (image size×number of channels) is output. After being processed by the maximum pooling layer, the obtained feature map is input into the second coding block, and the feature map of size 128×128×64 is output after being subjected to feature extraction of two convolutional layers of size 3×3 and step size 1 in the second coding block. After being processed by the maximum pooling layer, the obtained feature map is input into the third coding block, and the feature map of size 128×128×64 is output. Two convolutional layers with a size of 3×3 and a step size of 1 are used for feature extraction, and the output feature map size is 64×64×128. It is further input into the maximum pooling layer for processing, and the resulting feature map of size 32×32×128 is input into the fourth encoding block. After two convolutional layers with a size of 3×3 and a step size of 1, a feature map of size 32×32×256 is output, which is then input into the maximum pooling layer for processing. After that, the resulting feature map is input into a convolutional layer with a size of 3×3 and a step size of 1 for feature extraction, and the first feature map of size 16×16×512 is output.

[0124] It can be seen that in the feature extraction process of the input fundus image to be segmented by the coding network, after the convolution processing of the convolution layers of the multiple coding blocks connected in sequence and the maximum pooling layer processing are performed, the feature map of the maximum pooling output of the last coding block can be obtained, and then the convolution layer with the same size convolution kernel can be input for feature extraction to obtain the first feature map. It should be understood that the proportional relationship between the resolutions of the feature maps output by the second convolution layer of each of the multiple coding blocks is determined according to the step size of the convolution processing, including but not limited to the coding processing method described in this embodiment.

[0125] Step S73, using a graph convolutional network and the graphic information of the optic disc contour graphic of the fundus image to be segmented, adjusting the corresponding feature representation of the first feature map to obtain a second feature map;

[0126] Regarding the processing process of the graph convolutional network, please refer to the description of the corresponding part of the above embodiment, and this embodiment will not be described in detail here.

[0127] Step S74, inputting the second feature map into the decoding network, upsampling the second feature map through the upsampling layer of the first decoding block, and obtaining an upsampled feature map;

[0128] Step S75, adding the upsampled feature map to the feature map output by the second convolutional layer of the last coding block in the coding network to obtain a fused feature map;

[0129] Step S76, input the fused feature map into the convolution layer of the first decoding block for feature extraction, input the extracted feature map into the next decoding block, and continue to decode the input feature map according to the feature map output by the second convolution layer of the penultimate encoding block in the encoding network;

[0130] Step S77, determining that multiple decoding blocks in the decoding network are completed in sequence, processing the third feature map output by the convolution layer of the last decoding block through the unit convolution layer, and outputting the optic disc segmentation image of the fundus image to be segmented.

[0131] In the embodiment of the present application, the decoding network may include the same number of decoding blocks as the encoding blocks included in the encoding network, and each decoding block includes an upsampling layer, a feature fusion layer, and a convolution layer connected in sequence; the convolution kernel size and step size of the convolution layer in the decoding block are the same as those of the convolution layer in the encoding block. Figure 8 The network structure of the optic disc segmentation model shown in the figure is used as an example to illustrate. Figure 8 In the decoding network shown in the second row, multiple decoding blocks are connected in sequence starting from the right. The first decoding block is connected to the second convolutional layer of the last encoding block, and the second decoding block is connected to the second convolutional layer of the second-to-last encoding block, and so on, until the last decoding block is connected to the second convolutional layer of the first encoding block.

[0132] Still Figure 8 The decoding network shown includes 4 decoding blocks as an example for explanation. The second feature map of the size of 16×16×512 output by the graph convolutional network is extracted by a convolutional layer of size 3×3 and step size 1 (which corresponds to the last network layer in the encoding network). After the feature map of size 16×16×256 is output, it is upsampled by the upsampling layer in the first decoding block, and the obtained feature map is added and fused with the feature map output by the second convolutional layer of the fourth decoding block of the encoding network. After feature extraction by a convolutional layer of size 3×3 and step size 1, the obtained feature map of size 32×32×128 is input into the second decoding block. After the upsampling operation on the feature map by the upsampling layer of the second decoding block, the obtained feature map is added and fused with the feature map output by the second convolutional layer in the third encoding block. The convolution layer of the fused feature map is used to extract features, and the obtained feature map of size 64×64×64 is input into the third decoding block. After the upsampling operation on the input feature map, the obtained feature map is added and fused with the feature map output by the second convolution layer in the second encoding block. After processing by a convolution layer of size 3×3 and step size 1, a feature map of size 128×128×32 is output, which is input into the fourth decoding block. After the upsampling operation, the obtained feature map is added and fused with the feature map output by the second convolution layer of the first encoding block. After feature processing by a convolution layer of size 3×3 and step size 1, the obtained fused feature map is obtained to obtain a third feature map of size 256×256×32. After being processed by the last connected 1×1 unit convolution layer with sigmoid activation function, the optic disc segmentation image of the fundus image to be segmented is output.

[0133] It can be seen that the present application adopts a graph convolutional network as a deep network between the encoding network and the decoding network, and the graph convolutional network also has a corresponding number of multiple convolutional layers to extract high-level optic disc graphics information, multiply it with the first feature map output by the encoding network, correct the corresponding feature representation in the first feature map, and then send the output second feature map to the decoding network. During the decoding process of each decoding block, it is connected to the convolutional layer of the corresponding encoding block in the encoding network, and its bottom / shallow basic feature representation is used to process the high-level graphics information captured by the graph convolutional network, so as to accurately obtain the optic disc segmentation image.

[0134] Combined with the above analysis, in the training process of the optic disc segmentation model, for each sample fundus image in the data set, the optic disc segmentation image in the sample fundus image can be predicted according to the method described above, and then the optic disc segmentation model is trained by minimizing the loss value of the optic disc segmentation image to obtain an optic disc segmentation model that meets the constraint conditions, so as to realize the optic disc segmentation of the fundus image to be segmented that is actually collected. The implementation process of the model training is not described in detail in this application. Regarding the method of obtaining the loss value of the prediction result of the optic disc segmentation image, a reasonable loss function can be selected according to actual needs, and this embodiment is not described in detail here.

[0135] Reference Fig. 9 , is a schematic structural diagram of an optional example of the fundus image optic disc segmentation device proposed in this application, such as Fig. 9 As shown, the device may include:

[0136] A fundus image to be segmented obtaining module 91, used for obtaining a fundus image to be segmented;

[0137] A class activation map acquisition module 92 is used to acquire a class activation map of the optic disc area in the fundus image to be segmented;

[0138] An optic disc contour image acquisition module 93 is used to obtain an optic disc contour image in the fundus image to be segmented according to the class activation map;

[0139] An optic disc segmentation module 94 is used to input the fundus image to be segmented and the optic disc contour graphic into a trained optic disc segmentation model, and output an optic disc segmentation image;

[0140] Among them, the optic disc segmentation model includes an encoding network, a graph convolutional network and a decoding network connected in sequence. The encoding network and the decoding network are both constructed based on convolutional neural networks. The graph convolutional network can adjust the graphic information in the feature map output by the encoding network according to the optic disc contour graphic.

[0141] Optional, such as Fig.10 As shown, the above-mentioned class activation map acquisition module 92 may include:

[0142] The slice image obtaining unit 921 is used to crop the fundus image to be segmented according to a preset size to obtain a corresponding plurality of slice images; the preset size is smaller than the size of the optic disc area in the fundus image to be segmented;

[0143] An image category label obtaining unit 922 is used to obtain image category labels of various slice images; the image category labels are used to indicate whether the corresponding slice image contains at least a part of the optic disc area;

[0144] The class activation map obtaining unit 923 is used to perform class activation mapping processing on the plurality of slice images including at least part of the optic disc area according to the image category label to obtain a class activation map of the optic disc area in the fundus image to be segmented.

[0145] Optional, such as Fig.10 As shown, the above-mentioned disc outline graph obtaining module 93 may include:

[0146] The sampling point position obtaining unit 931 is used to perform point cloud sampling on the class activation map by the farthest point sampling method to obtain a preset number of sampling point positions for the optic disc area; wherein the initial sampling point during the point cloud sampling is a random sampling point or a sampling point with the largest distance from the point cloud center;

[0147] A first optic disc contour information forming unit 932, configured to form first optic disc contour information of the optic disc area by using the preset number of sampling point positions;

[0148] A second optic disc contour information obtaining unit 933 is used to downsample the first optic disc contour information to obtain second optic disc contour information having a resolution matching the trained optic disc segmentation model;

[0149] The optic disc contour graph construction unit 934 is configured to construct an optic disc contour graph for the fundus image to be segmented by using the second optic disc contour information.

[0150] In some embodiments, Fig.10 As shown, the above-mentioned video disc segmentation module 94 may include:

[0151] A first feature map obtaining unit 941 is used to input the fundus image to be segmented into the encoding network to extract features with different resolutions to obtain a first feature map;

[0152] A second feature map obtaining unit 942 is used to input the first feature map and the optic disc contour graph into the graph convolution network, and adjust the feature representation corresponding to the first feature map using the graphic information in the optic disc contour graph to obtain a second feature map;

[0153] A third feature map obtaining unit 943 is used to input the second feature map into the decoding network, and perform decoding processing according to the feature maps of different resolutions extracted by the encoding network to obtain a third feature map; the feature maps of different resolutions refer to feature maps extracted by the encoding network before the first feature map;

[0154] The optic disc segmentation image output unit 944 is used to process the third feature map through a unit convolution layer to output an optic disc segmentation image of the fundus image to be segmented.

[0155] Optionally, the encoding network may include multiple encoding blocks, each of the encoding blocks includes two convolutional layers and a maximum pooling layer connected in sequence, and each of the convolutional layers has a convolution kernel of the same size to perform convolution processing according to the same step length. Based on this, the first feature map obtaining unit 941 may include:

[0156] A first processing unit is used to input the fundus image to be segmented into the coding network, and after the convolution processing of the convolution layers of the plurality of coding blocks connected in sequence and the processing of the maximum pooling layer, perform feature extraction on the maximum pooling layer output feature map of the last coding block to obtain a first feature map;

[0157] The proportional relationship between the resolutions of the feature maps output by the second convolutional layers of the multiple encoding blocks is determined according to the step size.

[0158] Optionally, the decoding network includes the same number of decoding blocks as the encoding blocks included in the encoding network, each of the decoding blocks includes an upsampling layer, a feature fusion layer and a convolution layer connected in sequence; the convolution kernel size and step size of the convolution layer in the decoding block are the same as those of the convolution layer in the encoding block; based on this, the third feature map obtaining unit 943 may include:

[0159] A second processing unit is configured to input the second feature map into the first decoding block of the decoding network, and obtain a third feature map output by the convolution layer of the last decoding block through decoding processing of a plurality of decoding blocks according to the feature map output by the second convolution layer of the encoding block in the encoding network;

[0160] Among them, in the decoding processing process of each of the decoding blocks, the feature fusion layer of the decoding block adds the feature map output by the upsampling layer and the feature map output by the second convolution layer corresponding to the encoding block in the encoding network, and inputs the obtained fused feature map into the convolution layer of the decoding block for feature extraction, so that the extracted feature map is input into the next decoding block to continue decoding processing.

[0161] In the above embodiments, each convolution layer of the encoding network and the decoding network is composed of a convolution kernel of size 3×3, and a convolution operation is performed after the input feature map is moved at intervals of one pixel; in addition, the graph convolution network includes a plurality of superimposed graph convolution layers, and each feature point in the output feature map can be adjusted according to information from a high-order neighborhood, and the information of the high-order neighborhood is determined according to the connection relationship between different feature points of the optic disc contour represented by the optic disc contour graph.

[0162] It should be noted that the various modules, units, etc. in the above-mentioned device embodiments can be stored in the memory as program modules, and the processor executes the above-mentioned program modules stored in the memory to implement the corresponding functions. For the functions implemented by each program module and its combination, as well as the technical effects achieved, please refer to the description of the corresponding parts of the above-mentioned method embodiments, which will not be repeated in this embodiment.

[0163] The present invention also provides a computer-readable storage medium on which a computer program can be stored. The computer program can be called and loaded by a processor to implement the various steps of the fundus image optic disc segmentation method described in the above embodiment. The specific implementation process can refer to the description of the corresponding part of the above embodiment, and will not be repeated in this embodiment.

[0164] Reference Fig.11 , is a schematic diagram of the hardware structure of an optional example of an electronic device suitable for the fundus image optic disc segmentation method proposed in this application, such as Fig.11 As shown, the electronic device may include: a communication module 111, a memory 112 and a processor 113, wherein:

[0165] The number of the communication module 111, the memory 112 and the processor 113 can be at least one, and the communication module 111, the memory 112 and the processor 113 can all be connected to a communication bus to achieve data interaction with each other through the communication bus. The specific implementation process can be determined according to the requirements of the specific application scenario, and the present invention will not elaborate on it.

[0166] The communication module 111 may include a communication module that can realize data interaction by using a wireless communication network, such as a WIFI module, a 5G / 6G (fifth generation mobile communication network / sixth generation mobile communication network) module, a GPRS module, etc., to realize wireless communication between the electronic device and other devices; of course, the communication module 111 may also be a data interface for realizing wired communication, so that the electronic device can be connected to other devices through a corresponding type of data cable to realize data transmission between different devices, such as the collected fundus image, the segmented optic disc segmentation image, etc.

[0167] It should be understood that the above-mentioned communication module 111 includes a communication interface for realizing data interaction between internal components of the electronic device, such as a USB interface, a serial / parallel port, a multimedia transmission interface, etc. This application does not limit the type and number of the communication module 111.

[0168] In an embodiment of the present application, the memory 112 can be used to store a program for implementing the method for optic disc segmentation of fundus images described in any of the above method embodiments; the processor 113 can load and execute the program stored in the memory 112 to implement the various steps of the method for optic disc segmentation of fundus images proposed in any of the above method embodiments of the present application. The specific implementation process can refer to the description of the corresponding part of the corresponding embodiment above and will not be repeated here.

[0169] In practical applications, the memory 112 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device or other volatile solid-state storage device. The processor 113 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, etc. The types of the memory 112 and the processor 113 may be determined according to actual needs.

[0170] It should be understood that Fig.11 The structure of the electronic device shown does not constitute a limitation on the electronic device in the embodiment of the present invention. In practical applications, the electronic device may include Fig.11 More components shown, or a combination of certain components, can be determined based on the product type of the electronic device. When the electronic device is a terminal device, the electronic device may also include: an image acquisition component, a display, a speaker and at least one output component. It may also include a pickup, a function button and at least one input component as needed to meet different input operations on the electronic device, as well as a sensor module containing multiple sensors, a power supply component, etc., which can be determined based on the application scenario and are not listed one by one in this application.

[0171] Finally, it should be noted that the "system", "device", "unit" and / or "module" used in the present invention is a method for distinguishing different components, elements, parts, portions or assemblies at different levels. However, if other words can achieve the same purpose, the word can be replaced by other expressions.

[0172] As shown in the present invention and claims, unless the context clearly indicates an exception, the words "a", "an", "a kind" and / or "the" do not specifically refer to the singular, but may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of clearly identified steps and elements, and these steps and elements do not constitute an exclusive list, and the method or device may also include other steps or elements. The elements defined by the sentence "includes a..." do not exclude the existence of other identical elements in the process, method, commodity or device that includes the elements.

[0173] Among them, in the description of the embodiments of the present invention, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present invention, "multiple" refers to two or more than two. The following terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features.

[0174] In addition, the various embodiments in this specification are described in a progressive or parallel manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the various embodiments can be referred to each other. For the devices and electronic devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0175] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for optic disc segmentation of fundus images, characterized in that: The method comprises: Obtaining a fundus image to be segmented; Acquire a class activation map of the optic disc area in the fundus image to be segmented; According to the class activation map, obtaining an optic disc contour figure in the fundus image to be segmented; Inputting the fundus image to be segmented and the optic disc contour graphic into a trained optic disc segmentation model, and outputting an optic disc segmentation image; Among them, the optic disc segmentation model includes an encoding network, a graph convolutional network and a decoding network connected in sequence. The encoding network and the decoding network are both constructed based on convolutional neural networks. The graph convolutional network can adjust the feature representation corresponding to the feature map output by the encoding network according to the optic disc contour graphic.

2. The method according to claim 1, characterized in that The step of inputting the fundus image to be segmented and the optic disc contour graphic into a trained optic disc segmentation model and outputting an optic disc segmentation image comprises: Inputting the fundus image to be segmented into the encoding network to extract features with different resolutions to obtain a first feature map; Inputting the first feature map and the optic disc outline graph into the graph convolutional network, and adjusting the feature representation corresponding to the first feature map using the graphic information in the optic disc outline graph to obtain a second feature map; Inputting the second feature map into the decoding network, performing decoding processing according to the feature maps of different resolutions extracted by the encoding network, to obtain a third feature map; the feature maps of different resolutions refer to feature maps extracted by the encoding network before the first feature map; The third feature map is processed by a unit convolution layer to output an optic disc segmentation image of the fundus image to be segmented.

3. The method according to claim 2, characterized in that The encoding network includes a plurality of encoding blocks, each of the encoding blocks includes two convolutional layers and a maximum pooling layer connected in sequence, and each of the convolutional layers has a convolution kernel of the same size so as to perform convolution processing according to the same step length; Inputting the fundus image to be segmented into the encoding network to extract features of different resolutions to obtain a first feature map, including: Inputting the fundus image to be segmented into the coding network, performing convolution processing of the convolution layers of the plurality of coding blocks connected in sequence and processing of the maximum pooling layer, performing feature extraction on the maximum pooling layer output feature map of the last coding block, and obtaining a first feature map; The proportional relationship between the resolutions of the feature maps output by the second convolutional layers of the multiple encoding blocks is determined according to the step size.

4. The method according to claim 3, characterized in that The decoding network includes the same number of decoding blocks as the encoding blocks included in the encoding network, each of the decoding blocks includes an upsampling layer, a feature fusion layer and a convolution layer connected in sequence; the convolution kernel size and step size of the convolution layer in the decoding block are the same as those of the convolution layer in the encoding block; The step of inputting the second feature map into the decoding network and performing decoding processing according to the feature maps of different resolutions extracted by the encoding network to obtain a third feature map includes: Inputting the second feature map into the first decoding block of the decoding network, and obtaining a third feature map output by the convolution layer of the last decoding block according to the feature map output by the second convolution layer of the encoding block in the encoding network, after decoding by a plurality of decoding blocks; Among them, in the decoding processing process of each of the decoding blocks, the feature fusion layer of the decoding block adds the feature map output by the upsampling layer and the feature map output by the second convolution layer corresponding to the encoding block in the encoding network, and inputs the obtained fused feature map into the convolution layer of the decoding block for feature extraction, so that the extracted feature map is input into the next decoding block to continue decoding processing.

5. The method according to any one of claims 1 to 4, characterized in that: The step of obtaining a class activation map of the optic disc area in the fundus image to be segmented comprises: The fundus image to be segmented is cropped according to a preset size to obtain a corresponding plurality of slice images; the preset size is smaller than the size of the optic disc area in the fundus image to be segmented; Obtaining image category labels of various slice images; the image category labels are used to indicate whether the corresponding slice image contains at least a part of the optic disc area; According to the image category label, a class activation mapping process is performed on the plurality of slice images including at least a portion of the optic disc area to obtain a class activation map of the optic disc area in the fundus image to be segmented.

6. The method according to claim 5, characterized in that The step of obtaining the optic disc contour graphic in the fundus image to be segmented according to the class activation map comprises: Performing point cloud sampling on the class activation map by farthest point sampling to obtain a preset number of sampling point positions for the optic disc area; wherein the initial sampling point during point cloud sampling is a random sampling point or a sampling point with the largest distance from the center of the point cloud; Using the preset number of sampling point positions to form first optic disc contour information of the optic disc area; Downsampling the first optic disc contour information to obtain second optic disc contour information having a resolution matching a trained optic disc segmentation model; An optic disc contour graphic for the fundus image to be segmented is constructed using the second optic disc contour information.

7. The method according to any one of claims 1 to 4, characterized in that: Each convolution layer of the encoding network and the decoding network is composed of a convolution kernel of size 3×3, and a convolution operation is performed after the input feature map is moved at intervals of one pixel; The graph convolution network includes multiple superimposed graph convolution layers, and each feature point in the output feature map can be adjusted according to information from a high-order neighborhood, and the information of the high-order neighborhood is determined according to the connection relationship between different feature points of the optic disc contour represented by the optic disc contour graphic.

8. A fundus image optic disc segmentation device, characterized in that: The device comprises: A fundus image acquisition module to be segmented, used for acquiring the fundus image to be segmented; A class activation map acquisition module, used to acquire a class activation map of the optic disc area in the fundus image to be segmented; An optic disc contour image acquisition module, used for acquiring an optic disc contour image in the fundus image to be segmented according to the class activation map; An optic disc segmentation module, used for inputting the fundus image to be segmented and the optic disc contour graphic into a trained optic disc segmentation model, and outputting an optic disc segmentation image; Among them, the optic disc segmentation model includes an encoding network, a graph convolutional network and a decoding network connected in sequence. The encoding network and the decoding network are both constructed based on convolutional neural networks. The graph convolutional network can adjust the feature representation corresponding to the feature map output by the encoding network according to the optic disc contour graphic.

9. An electronic device, characterized in that: The electronic device comprises: Communication module; A memory for storing a program for implementing the method for optic disc segmentation of fundus images according to any one of claims 1 to 7; A processor is used to load and execute the program stored in the memory to implement the fundus image optic disc segmentation method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: At least one set of computer instruction sets is stored thereon, and the computer instruction sets are loaded and executed by a processor to implement the method for optic disc segmentation of fundus images according to any one of claims 1 to 7.

Images