Method, device, equipment and storage medium for processing images

Abstract

The present application discloses a method, device, equipment and storage medium for processing images, which relate to the field of artificial intelligence, specifically computer vision, deep learning, and smart medical fields, and can be applied to medical image analysis scenarios. The specific implementation scheme is: acquiring a fundus image; determining a vascular image based on the fundus image; segmenting the vascular image to obtain a vascular segment image; determining the arterial vascular segment and the venous vascular segment in the vascular segment image; determining the arteriovenous ratio based on the arterial vascular segment and the venous vascular segment, and outputting it. This implementation method can overcome the error of the pixel-level classification algorithm by segmenting the vascular image to obtain a vascular segment image, and calculating the arteriovenous ratio based on the arterial vascular segment and the venous vascular segment in the vascular segment image, thereby improving the accuracy of the arteriovenous classification and the calculation of the arteriovenous ratio. It is convenient and fast to operate, and has a wide range of applications.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, specifically computer vision, deep learning, and smart medicine, and can be applied to medical image analysis scenarios, and in particular to a method, apparatus, device, and storage medium for processing images. Background Art

[0002] As the only internal vascular tissue that can be observed without trauma, retinal blood vessels have always attracted extensive attention from researchers. By classifying retinal blood vessels into arteries and veins, we can have a deeper understanding of the structure of the retina. Currently, there are two main methods for classifying retinal blood vessels into arteries and veins: semi-automatic classification method and fully automatic classification method.

[0003] The methods for classifying retinal arteries and veins based on retinal fundus images mainly use the color information, geometry, topology information and morphology of blood vessels for semi-automatic and automatic classification, but they are often inefficient and inaccurate. Summary of the invention

[0004] The present disclosure provides a method, apparatus, device and storage medium for processing an image.

[0005] According to one aspect of the present disclosure, a method for processing an image is provided, comprising: acquiring a fundus image; determining a vascular image based on the fundus image; segmenting the vascular image to obtain a vascular segment image; determining arterial vascular segments and venous vascular segments in the vascular segment image; determining an arteriovenous vascular ratio based on the arterial vascular segments and the venous vascular segments, and outputting the ratio.

[0006] According to another aspect of the present disclosure, there is provided an apparatus for processing an image, comprising: an acquisition unit configured to acquire a fundus image; a vascular image determination unit configured to determine a vascular image based on the fundus image; a vascular segment image determination unit configured to segment the vascular image to obtain a vascular segment image; an arteriovenous segment determination unit configured to determine an arterial segment and a venous segment in the vascular segment image; and an output unit configured to determine an arteriovenous ratio based on the arterial segment and the venous segment, and output the result.

[0007] According to another aspect of the present disclosure, there is provided an electronic device for processing an image, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method for processing an image as described above.

[0008] According to yet another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided. The computer instructions are used to enable a computer to execute the above-mentioned method for processing an image.

[0009] The technology according to the present application solves the problem of low efficiency and accuracy in retinal arteriovenous classification. By segmenting the vascular image to obtain vascular segment images, and calculating the arteriovenous ratio based on the arterial segments and venous segments in the vascular segment images, the pixel-level classification algorithm error can be overcome, and the accuracy of arteriovenous classification and arteriovenous ratio calculation can be improved. The operation is convenient and fast, and the application range is wide.

[0010] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present application.

[0012] Figure 1 is an exemplary system architecture diagram in which an embodiment of the present application may be applied;

[0013] Figure 2 is a flow chart of an embodiment of a method for processing an image according to the present application;

[0014] Figure 3 is a schematic diagram of an application scenario of the method for processing an image according to the present application;

[0015] Figure 4 is a flowchart of another embodiment of a method for processing an image according to the present application;

[0016] Figure 5 is a structural schematic diagram of an embodiment of a device for processing an image according to the present application;

[0017] Figure 6 It is a block diagram of an electronic device used to implement the method for processing an image in an embodiment of the present application. DETAILED DESCRIPTION

[0018] The following is a description of exemplary embodiments of the present application in conjunction with the accompanying drawings, including various details of the embodiments of the present application to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0019] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0020] Figure 1 An exemplary system architecture 100 is shown to which embodiments of the method for processing an image or the apparatus for processing an image of the present application can be applied.

[0021] like Figure 1 As shown, the system architecture 100 may include cameras 101, 102, a network 103 and a terminal device 104. The network 103 is used to provide a medium for a communication link between the cameras 101, 102 and the terminal device 104. The network 103 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0022] The cameras 101 and 102 can interact with the terminal device 104 through the network 103 to receive or send messages. The cameras 101 and 102 can capture images and send the captured images to the terminal device 104, or store them locally. The cameras 101 and 102 can be fixed on the medical scanning device.

[0023] The terminal device 104 can obtain the collected images from the cameras 101 and 102, process the images, and output the processing results. Various communication client applications, such as image processing applications, can be installed on the terminal device 104.

[0024] The terminal device 104 can be hardware or software. When the terminal device 104 is hardware, it can be various electronic devices, including but not limited to smart phones, tablet computers, e-book readers, car computers, laptop computers, desktop computers, etc. When the terminal device 104 is software, it can be installed in the electronic devices listed above. It can be implemented as multiple software or software modules, or as a single software or software module. No specific limitation is made here.

[0025] It should be noted that the method for processing an image provided in the embodiment of the present application may be executed by the terminal device 104. Accordingly, the apparatus for processing an image may be disposed in the terminal device 104.

[0026] It should be understood that Figure 1 The number of cameras, networks, and terminal devices in the embodiment is only for illustration. Any number of cameras, networks, and terminal devices may be provided as required.

[0027] Continue to refer Figure 2, shows a process 200 of an embodiment of a method for processing an image according to the present application. The method for processing an image in this embodiment includes the following steps:

[0028] Step 201, acquiring a fundus image.

[0029] In this embodiment, the execution subject (eg Figure 1 The terminal device 104 in the apparatus can obtain the collected fundus images from the fundus camera, digital ophthalmoscope and other devices through wired connection or wireless connection. Specifically, the fundus camera can use an image acquisition card and a CCD camera. The film area (26×22mm) imaged by the fundus camera is much larger than the imaging area (6.4×4.8mm) of the CCD camera, so an optical interface is required for optical conversion and imaging, and the fundus image is clearly imaged on the CCD receiving target surface according to the required magnification. The optical interface mainly plays the role of optical conversion and imaging. Specifically, the digital ophthalmoscope can be used in conjunction with a mobile terminal to collect fundus images using the mobile terminal.

[0030] Step 202: Determine a blood vessel image based on the fundus image.

[0031] After acquiring the fundus image, the executing entity may further process the fundus image. Specifically, the executing entity may first set the length of the longest side of the original fundus image to 1024 pixels, place the set fundus image in the center of the background image of 1024*1024 pixels, form an image of fixed pixel size (1024*1024), and then automatically segment the image to obtain a vascular image. The fundus image may include optic disc information, fundus vascular information, and background information around the fundus blood vessels. The vascular image contains fundus vascular information. The "segmentation" here may be to separate the optic disc in the fundus image, the background around the fundus blood vessels, and the fundus blood vessels, and extract the vascular image corresponding to the fundus blood vessels in the fundus image.

[0032] Specifically, the execution subject can use the deep learning segmentation algorithm framework to automatically classify and segment the vascular images in the fundus image. The deep learning segmentation algorithm framework, for example, can use the Unet framework model. The Unet framework model includes two parts. The first part is a feature extraction structure, which can use a pre-trained residual neural network, such as Resnet-50 and other networks for feature extraction, and the second part is an upsampling part, which uses deconvolution processing. Specifically, the Resnet-50 network is used to extract features corresponding to the blood vessels in the fundus image, such as color features, size features, etc., so as to automatically segment the vascular image. For example, first input an original fundus image of any size, such as 1024*1024 pixels, into the Unet framework model structure; the original fundus image is forward propagated to the upsampling layer through the feature extraction Resnet-50 network of the first part, and propagated to the last shared convolution layer, which is used to generate a higher-dimensional feature map on the one hand, and the original image information is fused into the upsampling layer on the other hand, so that the information for the final pixel classification is richer and the important information in the original image can be retained. Finally, it reaches the convolution layer that classifies each pixel, and after this layer, the classification score of the pixel can be output. The Unet framework model training uses the binary weighted cross entropy loss function combined with the Dice loss function as the model's optimization loss function to optimize the Unet framework model. Finally, the score map output by the Unet framework model is thresholded to obtain a binary segmentation image of the fundus blood vessels.

[0033] Step 203, segmenting the blood vessel image to obtain blood vessel segment images.

[0034] After obtaining the vascular image, the execution subject may segment the vascular image to obtain vascular segment images. Specifically, the "segmentation" here may be to disconnect an entire blood vessel in the vascular image to obtain separated vascular segment images. The execution subject may call a cropping tool or an erasing tool to crop the vascular image or erase part of the vascular image area according to a preset length, and finally segment to obtain vascular segment images of equal length. Of course, it is understandable that the vascular segment images may not be of equal length, and the vascular image may be segmented into vascular segment images of different lengths according to actual needs. The present application does not specifically limit the segmentation method of the vascular segment and the length of the vascular segment obtained by segmentation.

[0035] Step 204: Determine the arterial blood vessel segments and the venous blood vessel segments in the blood vessel segment image.

[0036] After obtaining the vascular segment image, the execution subject can determine the arterial vascular segment and the venous vascular segment in the vascular segment image. Specifically, based on the difference in color between the arterial and venous blood vessels, a pre-trained deep learning classification network model can be used to extract the color features of the vascular segment, and the vascular segment can be classified according to the extracted features. The present application does not limit the extracted features. For example, the deep learning classification network model can use a pre-trained ResNet-50, etc. to perform classification judgments on the arterial vascular segment and the venous vascular segment to determine the arterial vascular segment and the venous vascular segment in the vascular segment image.

[0037] Step 205, determining the arteriovenous blood vessel ratio according to the arterial blood vessel segment and the venous blood vessel segment, and outputting the ratio.

[0038] After determining the arterial blood vessel segments and the venous blood vessel segments in the blood vessel segment image, the execution subject can determine the arteriovenous blood vessel ratio according to the arterial blood vessel segments and the venous blood vessel segments, and output it. Specifically, the execution subject can call a measurement tool to measure the diameters of the arterial blood vessel segments and the venous blood vessel segments, respectively obtain the blood vessel diameters of the arterial blood vessel segments and the venous blood vessel segments, determine the ratio of the arteriovenous blood vessel diameters as the arteriovenous blood vessel ratio, and output it.

[0039] Continue to refer Figure 3 , which shows a schematic diagram of an application scenario of the method for processing an image according to the present application. Figure 3 In the application scenario of analyzing medical images in classroom teaching, the teacher can use a camera 301 to collect fundus images 302, and a computer 304 can obtain fundus images 302 through a wired or wireless network 303. The computer 304 determines a blood vessel image 305 based on the fundus image 302. The computer 304 performs segmentation processing on the blood vessel image 305 to obtain a blood vessel segment image 306. The computer 304 determines the arterial blood vessel segments and the venous blood vessel segments in the blood vessel segment image 306. The computer 304 determines the arteriovenous blood vessel ratio based on the arterial blood vessel segments and the venous blood vessel segments, and outputs the determined arterial blood vessel segments, venous blood vessel segments, arteriovenous blood vessel ratio, and the calculation process of the arteriovenous blood vessel ratio on the classroom teaching screen, so as to make students understand arteriovenous blood vessels and the arteriovenous blood vessel ratio more intuitively.

[0040] This embodiment can overcome the pixel-level classification algorithm error by segmenting the blood vessel image to obtain a blood vessel segment image, and improve the accuracy of arteriovenous classification and calculation of the arteriovenous ratio. The operation is convenient and fast, and the application range is wide.

[0041] Continue to refer Figure 4, which shows a process 400 of another embodiment of a method for processing an image according to the present application. Figure 4 As shown, the method for processing an image in this embodiment may include the following steps:

[0042] Step 401, acquiring a fundus image.

[0043] The principle of step 401 is similar to that of step 201 and will not be described in detail here.

[0044] Step 402: Determine a blood vessel image based on the fundus image.

[0045] The principle of step 402 is similar to that of step 202 and will not be described in detail here.

[0046] Specifically, step 402 can be implemented by following steps 4021 to 4023:

[0047] Step 4021, identifying the fundus image to obtain the position of the optic disc area.

[0048] After acquiring the fundus image, the execution subject can identify the fundus image and obtain the position of the optic disc area. Specifically, due to the differences in brightness and color within the fundus image and between different fundus images, the difference between arteriovenous vessels decreases after a certain distance from the optic disc. At the same time, the complex entanglement of blood vessels in the optic disc is not conducive to differentiation. More importantly, the calculation of the arteriovenous ratio mainly focuses on the trunk blood vessels. The farther away from the optic disc, the lower the value of the calculation. Therefore, only the blood vessels around the optic disc area are obtained and calculated. Specifically, the execution subject can use the pre-trained Faster-RCNN deep learning network model to realize the acquisition of blood vessel information around the optic disc area. Specifically, first input an image of any size into the Faster-RCNN deep learning network model, such as an image of 1024*1024 pixels. The image is forward propagated to the shared convolutional layer through the Faster-RCNN network, and on the one hand, a feature map for input to the Region Proposal Network (RPN) is obtained, and on the other hand, it continues to forward propagate to the unique convolutional layer of the Faster-RCNN deep learning network model to generate a higher-dimensional feature map, and the higher-dimensional feature map is input to the RoI pooling layer of the Faster-RCNN deep learning network model. The feature map for input to the RPN network passes through the RPN network to obtain region proposals and region scores, and non-maximum suppression is used for the region scores (the threshold can be 0.5), and the region proposals with the top N scores (i.e., the top-N scored region proposals) are output to the RoI pooling layer of the Faster-RCNN deep learning network model. At this time, the higher-dimensional feature map and the top-N scored region proposals are jointly input to the RoI pooling layer of the Faster-RCNN deep learning network model. The features of the corresponding region proposals are extracted by the pooling layer and input to the fully connected layer of the Faster-RCNN deep learning network model to obtain the classification score of the region and the position of the optic disc region. Among them, the RPN network can be obtained through end-to-end training of back propagation and stochastic gradient descent.

[0049] Step 4022, determining a target fundus image according to the fundus image, the position of the optic disc area and the preset step length.

[0050] After obtaining the position of the optic disc area, the execution subject can determine the target fundus image according to the fundus image, the position of the optic disc area and the preset step length. Specifically, the execution subject can take the optic disc area as the center in the fundus image and obtain a quadrilateral area surrounding the optic disc area with a length and a width of the preset step length. Specifically, the preset step length can be 6 times the optic disc diameter or 3 times the optic disc diameter; the quadrilateral area can be a rectangular area with a length of 6 times the optic disc diameter and a width of 3 times the optic disc diameter. The execution subject can determine the obtained quadrilateral area as the target fundus image.

[0051] Step 4023, performing binary segmentation on the target fundus image to obtain a blood vessel image.

[0052] In this embodiment, the target fundus image is binarized and then segmented. Image binarization is the process of setting the grayscale value of the pixel on the image to 0 or 255, that is, the whole image presents an obvious black and white effect. In digital image processing, binary images occupy a very important position. The binarization of the image greatly reduces the amount of data in the target fundus image, thereby highlighting the outline of the vascular image. Specifically, the execution subject can use the Unet framework to classify the target fundus image, obtain the classification score map of the target fundus image, thereby determining the color vascular image in the target fundus image, and then perform threshold segmentation on the classification score map, and then perform image binarization on the image after threshold segmentation to obtain a vascular image. The white area in the vascular image represents the blood vessel, and the black area represents the picture background around the blood vessel.

[0053] This embodiment uses deep learning to conveniently and quickly determine the position of the optic disc area in the fundus image. Since the image features extracted by deep learning are better, it is more robust than the traditional segmentation algorithm and has a higher accuracy in determining the position of the optic disc area. By obtaining a vascular image based on the optic disc area position, a preset step size, and a binary segmentation method, the amount of data processed for the target fundus image can be reduced and the image processing speed can be improved. By setting a preset step size to obtain the target fundus image, the quality of the acquired vascular data to be classified around the optic disc can be improved, thereby improving the accuracy of image processing.

[0054] Step 403: segment the blood vessel image to obtain a blood vessel segment image.

[0055] The principle of step 403 is similar to that of step 203 and will not be described again here.

[0056] Specifically, step 403 can be implemented by following steps 4031 to 4033:

[0057] Step 4031, extracting the blood vessel centerline in the blood vessel image.

[0058] After obtaining the vascular image, the execution subject can extract the vascular centerline in the vascular image. Specifically, the vascular image is first subjected to a dilation algorithm, that is, the vascular image is merged with the pixels of the background points at the corresponding positions in the original fundus image, that is, the pixel values ​​at the corresponding positions are "OR" operated to expand the boundary of the vascular image outward to obtain a dilated vascular image, thereby avoiding problems such as vascular breakage and discontinuity caused by incomplete segmentation. Then, the dilated vascular image is thinned using a thinning algorithm, that is, the lines of the dilated vascular image are reduced from a multi-pixel width to a unit pixel width, and the topology is kept unchanged to obtain the vascular centerline.

[0059] Step 4032, determine the intersection point of the blood vessel centerline.

[0060] After obtaining the centerline of the blood vessel, the execution subject can call the point capture tool to determine the intersection of the centerline of the blood vessel. The intersection can be the intersection of an artery and an artery, the branch point of an artery, the intersection of a vein and a vein, the branch point of a vein, or the intersection of an artery and a vein. This application does not specifically limit the type of intersection.

[0061] Step 4033, segment the blood vessel image according to the intersection points to obtain a blood vessel segment image.

[0062] After obtaining the intersection, the execution subject can segment the blood vessel image according to the intersection to obtain a blood vessel segment image. Specifically, the execution subject can call the erasing tool to erase the circular area with the intersection position as the center and R (the maximum radius of the blood vessel * 1.2) as the radius, so that the blood vessels connected to each other are disconnected to form a large number of independent blood vessel segments. Then the large number of independent blood vessel segments are merged with the pixels of the background points at the corresponding positions in the original fundus image, that is, an "or" operation is performed with the pixel values ​​in the original fundus image. If the pixel values ​​of the corresponding positions of each blood vessel segment and the original fundus image are both 0, the result of the "or" operation is 0, and the color of the corresponding position is black. If one of the pixel values ​​of the corresponding positions of each blood vessel segment and the original fundus image is not 0, the result of the "or" operation is 1, and the color of the corresponding position is white. As a result, the boundaries of each blood vessel segment are expanded outward to obtain an expanded blood vessel segment image, and the expanded blood vessel segment image is determined as the blood vessel segment image to be classified after the blood vessel image is segmented.

[0063] This embodiment converts the problem of classifying arteries and veins for each pixel into the problem of classifying arteries and veins for each blood vessel segment, which can not only overcome the problem that a blood vessel may have arterial pixels and venous pixels due to the error of the pixel-level classification algorithm, but also avoid the problem of arterial and venous intersections and blood vessel branches that are prone to cause misjudgment of blood vessels. This embodiment can directly remove the intersection without affecting the calculation of the arterial and venous diameters, avoiding the influence of the arterial and venous classification errors in the area near the intersection on the calculation of the arterial and venous ratio. By extracting the blood vessel centerline and determining the intersection of the blood vessel centerline, the accuracy of segmenting the blood vessel image to obtain the blood vessel segment image can be improved. And by merging the pixels of the independent blood vessel segment with the background points at the corresponding position in the original fundus image, the merged expanded blood vessel segment image is determined as the blood vessel segment image to be classified. In this way, high-dimensional and deeper imaging features can be extracted by deep learning. When classifying the imaging features, the optimization function is used to automatically learn the important features that can distinguish between arteries and veins, overcoming the influence of human factors on the traditional method. At the same time, a large number of practices in the field of images have proved that deep learning classification methods have higher classification accuracy and better robustness than traditional artificial description features and traditional SVM, random forest and other classifiers, and can also identify low-quality images well. At the same time, it can avoid the increase in the difficulty of classifying arteries and veins due to the lack of contrast with background information caused by the use of independent vascular information, making the classification between arteries and veins simpler and more accurate.

[0064] Step 404: determine the arterial blood vessel segments and the venous blood vessel segments in the blood vessel segment image.

[0065] The principle of step 404 is similar to that of step 204 and will not be described in detail here.

[0066] Specifically, step 404 can be implemented by following the steps 4041 to 4044:

[0067] Step 4041: extract image channel information of each blood vessel segment in the blood vessel segment image.

[0068] Step 4042: Determine the target data to be classified corresponding to each blood vessel segment according to the image channel information, the preset blood vessel segment size, and the preset number of blood vessel segments.

[0069] In this embodiment, the blood vessel segment images all belong to the same person. After obtaining the blood vessel segment images, the execution subject can extract the image channel information of each blood vessel segment in the blood vessel segment images. Specifically, before this, the execution subject can also sort the blood vessel segment images to be classified according to the perimeter of the blood vessel segment area. For example, the 20 blood vessel segment images in the first sorting are selected, and the cutout tool is called to cut out the area with a pixel size of 128*64 for each selected blood vessel segment image. Specifically, the cutout is performed with the center of each selected blood vessel segment image as the center point. If the size of the blood vessel segment image is less than 128*64, the insufficient area is supplemented with 0. If there are less than 20 blood vessels, an image with all 0 is used as the blood vessel segment image for supplementation. Then, the execution subject extracts the image channel information of each cut blood vessel segment, for example, the green channel information, and finally forms a three-dimensional data indicating the green channel information with a dimension of 128*64*20. Among them, "128*64" is the preset blood vessel segment size, and 20 is the preset number of blood vessel segments. This three-dimensional data is used as the target data to be classified corresponding to each blood vessel segment and input into the pre-trained classification model for classification.

[0070] Step 4043: Determine the score of each blood vessel segment corresponding to the target data to be classified according to the target data to be classified and the pre-trained classification model.

[0071] In this embodiment, the pre-trained classification model is used to characterize the corresponding relationship between the target data to be classified and the score. The execution subject can input the target data to be classified into the pre-trained classification model, and output the score of each blood vessel segment corresponding to the target data to be classified.

[0072] Step 4044: determining the arterial blood vessel segments and the venous blood vessel segments in the blood vessel segment image according to the scores and the preset threshold.

[0073] In this embodiment, after obtaining the score of each blood vessel segment output by the pre-trained classification model, such as ResNet, the execution subject can determine the arterial blood vessel segment and the venous blood vessel segment in the blood vessel segment image according to the score and the preset threshold. Of course, the pre-trained classification model can also continue to determine the arterial blood vessel segment and the venous blood vessel segment in the blood vessel segment image according to the output score and the preset threshold. Specifically, the preset threshold may include a first preset threshold and a second preset threshold. The execution subject can determine the size relationship between each score and the preset threshold to determine the type of the blood vessel segment. Specifically, for each score, the execution subject determines that the blood vessel segment corresponding to the score is an arterial blood vessel segment in response to determining that the score is less than the first preset threshold. The execution subject determines that the blood vessel segment corresponding to the score is a venous blood vessel segment in response to determining that the score is greater than the second preset threshold. The execution subject determines that the blood vessel segment corresponding to the score is an "other" blood vessel segment in response to determining that the score is greater than the first preset threshold and less than the second preset threshold. When pre-training the classification model, the training set used is part of the target data to be classified and the scores of the part of the target data to be classified that are labeled. For example, the score can be 0 (representing vein), 1 (representing artery), 2 (representing others, because there may be some blood vessel segments with less than 20 blood vessels supplemented with 0, this part is marked as "others"). In this way, the operation is simple, very convenient and fast, and it is easier to implement and promote in practice.

[0074] This embodiment determines the target data to be classified by extracting the image channel information, which can make the target data to be classified the most robust, least affected by light, acquisition equipment, etc., and have good data quality, which can improve the accuracy of vascular segment classification. In addition, this application can retain the contrast difference between arteries and veins by using the target data to be classified of the same person as the input of the classification model. Compared with the traditional classification based only on the characteristics of arteries and veins, it is more in line with the information used by the human eye to distinguish arteries and veins, and the information is retained more completely.

[0075] Step 405: Determine the arteriovenous blood vessel ratio based on the arterial blood vessel segment and the venous blood vessel segment, and output it.

[0076] The principle of step 405 is similar to that of step 205 and will not be described in detail here.

[0077] Specifically, step 405 can be implemented by following steps 4051 to 4052:

[0078] Step 4051, obtaining the mean value of the arterial vessel diameter of the arterial vessel segment and the mean value of the venous vessel diameter of the venous vessel segment.

[0079] After determining the arterial and venous segments in the vascular segment image, the execution subject can obtain the mean arterial diameter of the arterial segment and the mean venous diameter of the venous segment. Specifically, the execution subject can use the distanceTransform() function of opencv, which is used to calculate the distance between each non-zero pixel and its nearest zero pixel in the image, and the output is to save the distance information between each non-zero point and the nearest zero point; the brighter the point on the image, the farther away from the zero point. The function is used to calculate the distance between each non-zero point and its nearest zero point in the binary image of the arterial and venous segments. The average distance is set as a threshold for each vascular segment, and the mean distance of all points greater than the threshold is counted as the vascular radius of the vascular segment, and the vascular diameter is radius*2. Then, according to the calculation method, the mean arterial diameter Da of all arterial segments is calculated, and the mean venous diameter Dv of all venous segments is calculated.

[0080] Step 4052, determine the arteriovenous ratio based on the mean arterial diameter and the mean venous diameter, and output it.

[0081] After obtaining the mean value Da of the arterial blood vessel diameter and the mean value Dv of the venous blood vessel diameter, the execution subject may determine the ratio Da / Dv as the arteriovenous blood vessel ratio and output it.

[0082] This embodiment can make the calculation result more accurate by determining the arteriovenous ratio according to the average arterial blood vessel diameter and the average venous blood vessel diameter.

[0083] Further references Figure 5 As an implementation of the methods shown in the above figures, the present application provides an embodiment of a device for processing an image. Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0084] like Figure 5 As shown, the apparatus 500 for processing an image in this embodiment includes: an acquisition unit 501 , a blood vessel image determination unit 502 , a blood vessel segment image determination unit 503 , an arteriovenous blood vessel segment determination unit 504 , and an output unit 505 .

[0085] The acquisition unit 501 is configured to acquire a fundus image.

[0086] The blood vessel image determining unit 502 is configured to determine a blood vessel image based on the fundus image.

[0087] The blood vessel segment image determining unit 503 is configured to segment the blood vessel image to obtain a blood vessel segment image.

[0088] The arteriovenous blood vessel segment determination unit 504 is configured to determine the arterial blood vessel segment and the venous blood vessel segment in the blood vessel segment image.

[0089] The output unit 505 is configured to determine the arteriovenous blood vessel ratio according to the arterial blood vessel segment and the venous blood vessel segment, and output it.

[0090] In some optional implementations of this embodiment, the vascular image determination unit 502 is further configured to: identify the fundus image to obtain the position of the optic disc area; determine the target fundus image based on the fundus image, the position of the optic disc area and a preset step size; and perform binary segmentation on the target fundus image to obtain a vascular image.

[0091] In some optional implementations of this embodiment, the blood vessel segment image determination unit 503 is further configured to: extract blood vessel centerlines in the blood vessel image; determine intersections of the blood vessel centerlines; and segment the blood vessel image according to the intersections to obtain blood vessel segment images.

[0092] In some optional implementations of this embodiment, the arteriovenous blood vessel segment determination unit 504 is further configured to: extract image channel information of each blood vessel segment in the blood vessel segment image; determine the target data to be classified corresponding to each blood vessel segment according to the image channel information, a preset blood vessel segment size, and a preset number of blood vessel segments; determine the score of each blood vessel segment corresponding to the target data to be classified according to the target data to be classified and a pre-trained classification model, wherein the pre-trained classification model is used to characterize the correspondence between the target data to be classified and the score; and determine the arterial blood vessel segment and the venous blood vessel segment in the blood vessel segment image according to the score and a preset threshold.

[0093] In some optional implementations of the present embodiment, the output unit 505 is further configured to: obtain the mean arterial vessel diameter of the arterial vessel segment and the mean venous vessel diameter of the venous vessel segment; determine the arteriovenous vessel ratio based on the mean arterial vessel diameter and the mean venous vessel diameter, and output it.

[0094] It should be understood that the units 501 to 505 described in the apparatus 500 for processing an image are respectively the same as those described in the reference Figure 2 Therefore, the operations and features described above for the method for processing an image are also applicable to the device 500 and the units contained therein, and will not be described in detail here.

[0095] According to an embodiment of the present application, the present application also provides an electronic device for processing an image and a readable storage medium.

[0096] like Figure 6, is a block diagram of an electronic device for a method for processing an image according to an embodiment of the present application. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.

[0097] like Figure 6 As shown, the electronic device includes: one or more processors 601, memory 602, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses 605, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the electronic device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In other embodiments, if necessary, multiple processors and / or multiple buses 605 can be used together with multiple memories and multiple memories. Similarly, multiple electronic devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 6 A processor 601 is taken as an example.

[0098] The memory 602 is a non-transitory computer-readable storage medium provided in the present application. The memory stores instructions executable by at least one processor to enable at least one processor to perform the method for processing an image provided in the present application. The non-transitory computer-readable storage medium of the present application stores computer instructions, which are used to enable a computer to perform the method for processing an image provided in the present application.

[0099] The memory 602 is a non-transitory computer-readable storage medium that can be used to store non-transitory software programs, non-transitory computer executable programs and units, such as program instructions / units corresponding to the method for processing an image in the embodiment of the present application (for example, the attached Figure 5 The processor 601 executes various functional applications and data processing of the server by running the non-transient software programs, instructions and modules stored in the memory 602, that is, implements the method for processing images in the above method embodiment.

[0100] The memory 602 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required by at least one function; the data storage area may store data created according to the use of the electronic device for processing images, etc. In addition, the memory 602 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory 602 may optionally include a memory remotely arranged relative to the processor 601, and these remote memories may be connected to the electronic device for processing images via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0101] The electronic device for the method of processing an image may further include: an input device 603 and an output device 604. The processor 601, the memory 602, the input device 603 and the output device 604 may be connected via a bus 605 or other means. Figure 6 The connection via bus 605 is taken as an example.

[0102] The input device 603 can receive input digital or character information, and generate key signal input related to user settings and function control of the electronic device for processing images, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator rod, one or more mouse buttons, a trackball, a joystick and other input devices. The output device 604 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The display device may include, but is not limited to, a liquid crystal display (LCD), a light emitting diode (LED) display and a plasma display. In some embodiments, the display device may be a touch screen.

[0103] Various implementations of the systems and techniques described herein can be realized in digital electronic circuit systems, integrated circuit systems, dedicated ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0104] These computer programs (also referred to as programs, software, software applications, or code) include machine instructions for programmable processors and can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and / or means (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0105] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0106] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0107] A computer system may include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship to each other.

[0108] According to the technical solution of the embodiment of the present application, by segmenting the blood vessel image to obtain a blood vessel segment image, and calculating the arteriovenous ratio based on the arterial blood vessel segments and venous blood vessel segments in the blood vessel segment image, the pixel-level classification algorithm error can be overcome, and the accuracy of arteriovenous classification and arteriovenous ratio calculation can be improved. The operation is convenient and fast, and the application range is wide.

[0109] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this application can be executed in parallel, sequentially or in different orders, as long as the expected results of the technical solution disclosed in this application can be achieved, and this document is not limited here.

[0110] The above specific implementations do not constitute a limitation on the protection scope of this application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of this application should be included in the protection scope of this application.

Claims

1. A method for processing an image, comprising: Acquire fundus images; determining a blood vessel image according to the fundus image; Segmenting the blood vessel image to obtain a blood vessel segment image; extracting image channel information of each blood vessel segment in the blood vessel segment image; Determining target data to be classified corresponding to each blood vessel segment according to the image channel information, a preset blood vessel segment size, and a preset number of blood vessel segments; Determining the score of each blood vessel segment corresponding to the target data to be classified according to the target data to be classified and the pre-trained classification model, wherein the pre-trained classification model is used to characterize the corresponding relationship between the target data to be classified and the score; determining an arterial blood vessel segment and a venous blood vessel segment in the blood vessel segment image according to the score and a preset threshold; An arteriovenous blood vessel ratio is determined based on the arterial blood vessel segment and the venous blood vessel segment and outputted.

2. The method according to claim 1, wherein: The step of determining a blood vessel image according to the fundus image includes: Identifying the fundus image to obtain the position of the optic disc area; Determining a target fundus image according to the fundus image, the position of the optic disc area and a preset step size; The target fundus image is binarized and segmented to obtain a blood vessel image.

3. The method according to claim 1, wherein: The segmenting of the blood vessel image to obtain a blood vessel segment image includes: extracting a blood vessel centerline in the blood vessel image; determining an intersection point of the centerlines of the blood vessels; The blood vessel image is segmented according to the intersection points to obtain a blood vessel segment image.

4. The method according to any one of claims 1 to 3, wherein: The determining the arteriovenous blood vessel ratio according to the arterial blood vessel segment and the venous blood vessel segment and outputting the ratio comprises: Obtaining a mean value of the arterial vessel diameters of the arterial vessel segments and a mean value of the venous vessel diameters of the venous vessel segments; The arteriovenous ratio is determined based on the mean arterial diameter and the mean venous diameter, and is output.

5. A device for processing an image, comprising: an acquisition unit configured to acquire a fundus image; a blood vessel image determination unit configured to determine a blood vessel image based on the fundus image; a blood vessel segment image determination unit, configured to segment the blood vessel image to obtain a blood vessel segment image; an arteriovenous blood vessel segment determination unit, configured to extract image channel information of each blood vessel segment in the blood vessel segment image; determine target data to be classified corresponding to each blood vessel segment according to the image channel information, a preset blood vessel segment size, and a preset number of blood vessel segments; determine a score of each blood vessel segment corresponding to the target data to be classified according to the target data to be classified and a pre-trained classification model, wherein the pre-trained classification model is used to characterize the correspondence between the target data to be classified and the score; determine the arterial blood vessel segment and the venous blood vessel segment in the blood vessel segment image according to the score and a preset threshold; The output unit is configured to determine an arteriovenous blood vessel ratio according to the arterial blood vessel segment and the venous blood vessel segment, and output the ratio.

6. The device according to claim 5, wherein: The blood vessel image determination unit is further configured to: Identifying the fundus image to obtain the position of the optic disc area; Determining a target fundus image according to the fundus image, the position of the optic disc area and a preset step size; The target fundus image is binarized and segmented to obtain a blood vessel image.

7. The device according to claim 5, wherein: The blood vessel segment image determination unit is further configured to: extracting a blood vessel centerline in the blood vessel image; determining an intersection point of the centerlines of the blood vessels; The blood vessel image is segmented according to the intersection points to obtain a blood vessel segment image.

8. The device according to any one of claims 5 to 7, wherein: The output unit is further configured to: Obtaining a mean value of the arterial vessel diameters of the arterial vessel segments and a mean value of the venous vessel diameters of the venous vessel segments; The arteriovenous ratio is determined based on the mean arterial diameter and the mean venous diameter, and is output.

9. An electronic device for processing an image, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 4.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 4.

Images