Wide-angle image segmentation method and device, storage medium and electronic equipment
By determining the intersection of the optic disc and blood vessels in the wide-angle fundus image and performing arterial and venous regions, the problem of poor arteriovenous segmentation results in the wide-angle fundus image is solved, and higher segmentation accuracy is achieved.
Patent Information
- Application Number
- CN202411943964.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-26
- Publication Date
- 2025-05-30
AI Technical Summary
The difference in the diameter and color of the fundus vessels in wide-angle fundus images is small, resulting in poor arteriovenous segmentation results, especially in a wider viewing angle range, where arteriovascular and venous blood vessels cannot be accurately segmented.
By determining the intersection of the optic disk and blood vessels in the target fundus image, arterial area growth and venous area growth are performed based on this information to obtain a more accurate second arteriovenous segmentation result.
The accuracy of arteriovenous segmentation in wide-angle fundus images is improved, and the second arteriovenous segmentation result with high accuracy can be obtained based on the first arteriovenous segmentation result.
Smart Images
Figure CN120070309A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer technology, and particularly to a wide-angle image segmentation method, apparatus, storage medium, and electronic device. Background Art
[0002] At present, in order to record and detect retinal diseases more comprehensively, the captured fundus images are usually wide-angle fundus images. The field of view angle of a conventional fundus image is usually 45 degrees, while the field of view angle of a wide-angle fundus image can reach more than 135 degrees, or even 220 degrees. Therefore, compared with conventional fundus images, wide-angle fundus images can record a wider viewing angle range, which is beneficial for medical staff to diagnose diseases.
[0003] In related technologies, arteriovenous segmentation mainly relies on the vessel diameter and vessel color in the fundus image. However, the differences in the vessel diameter and color of the fundus vessels in the fundus image are small, especially for wide-angle fundus images with a larger imaging range, where the differences in the vessel diameter and color of the fundus vessels are even smaller. This results in poor segmentation results for small vessels far from the optic disc, and thus the arterial and venous vessels cannot be accurately segmented. Summary of the Invention
[0004] To solve the above technical problems, the present application is proposed. Embodiments of the present application provide a wide-angle image segmentation method, apparatus, storage medium, and electronic device.
[0005] In a first aspect, an embodiment of the present application provides a wide-angle image segmentation method, including: determining the optic disc in a target fundus image; determining a first arteriovenous segmentation result corresponding to the target fundus image; determining the vessel crossing points in the target fundus image, where the vessel crossing points include vessel branch crossing points and arteriovenous crossing points; and respectively performing arterial region growth and venous region growth based on the optic disc, vessel crossing points, and the first arteriovenous segmentation result to obtain a second arteriovenous segmentation result corresponding to the target fundus image.
[0006] In a possible implementation manner, the first arteriovenous segmentation result includes a first arterial segmentation result and a first venous segmentation result. Among them, respectively performing arterial region growth and venous region growth based on the optic disc, vessel crossing points, and the first arteriovenous segmentation result to obtain a second arteriovenous segmentation result corresponding to the target fundus image includes: performing arterial region growth based on the optic disc, vessel branch crossing points, arteriovenous crossing points, and the first arterial segmentation result according to a preset vessel growth rule to obtain a second arterial segmentation result; performing venous region growth based on the optic disc, vessel branch crossing points, arteriovenous crossing points, and the first venous segmentation result according to a preset vessel growth rule to obtain a second venous segmentation result; and determining a second arteriovenous segmentation result corresponding to the target fundus image based on the second arterial segmentation result and the second venous segmentation result.
[0007] In a possible implementation, based on the blood vessel growth rule, artery region growth / vein region growth is performed, including: based on the first seed point, the second seed point, and the artery-vein crossing point, artery region growth / vein region growth is performed to obtain the second artery segmentation result / the second vein segmentation result; wherein, the first seed point is a pixel point corresponding to an artery / vein within a target radius determined based on the first artery segmentation result / the first vein segmentation result with the optic disc as the center, and the second seed point is a blood vessel branch crossing point.
[0008] In a possible implementation, the first seed point grows until it reaches the second seed point and then stops growing, and the second seed point grows in the opposite direction towards the optic disc.
[0009] In a possible implementation, the method further includes: during the process of artery region growth / vein region growth based on the first seed point and the second seed point, if the growth reaches the artery-vein crossing point, determine two tangent directions extending in the direction of the optic disc for the two crossing blood vessels corresponding to the artery-vein crossing point, and determine the target tangent direction among the two tangent directions where there is a grown artery region / vein region; using the artery-vein crossing point as the third seed point, continue artery region growth / vein region growth based on the blood vessel growth rule in the opposite direction of the target tangent direction.
[0010] In a possible implementation, the method further includes: for each blood vessel branch crossing point, if it is determined that there is no grown artery region connected to the artery region corresponding to the blood vessel branch crossing point in the direction from the blood vessel branch crossing point to the optic disc, delete the artery region corresponding to the blood vessel branch crossing point from the second artery segmentation result, where the artery region corresponding to the blood vessel branch crossing point is the region obtained by performing artery region growth with the blood vessel branch crossing point as the seed point; and / or, for each blood vessel branch crossing point, if it is determined that there is no grown vein region connected to the vein region corresponding to the blood vessel branch crossing point in the direction from the blood vessel branch crossing point to the optic disc, delete the vein region corresponding to the blood vessel branch crossing point from the second vein segmentation result, where the vein region corresponding to the blood vessel branch crossing point is the region obtained by performing vein region growth with the blood vessel branch crossing point as the seed point.
[0011] In a possible implementation, the method further includes: determining the artery-vein segmentation result corresponding to the target fundus image according to the first artery-vein segmentation result and the second artery-vein segmentation result.
[0012] In a possible implementation, the target fundus image includes multiple local images. Determining the arteriovenous segmentation result corresponding to the target fundus image according to the first arteriovenous segmentation result and the second arteriovenous segmentation result includes: for each local image, determining a first pixel set of the local image in the first arteriovenous segmentation result and a second pixel set of the local image in the second arteriovenous segmentation result; based on the first pixel set and the second pixel set, determining the average difference of the matching pixels in the first pixel set and the second pixel set; when the average difference is greater than the target threshold, taking the arteriovenous segmentation result of the local image in the second arteriovenous segmentation result as the arteriovenous segmentation result corresponding to the local image; when the average difference is not greater than the target threshold, determining the arteriovenous segmentation result corresponding to the local image based on the arteriovenous segmentation result of the local image in the first arteriovenous segmentation result and the arteriovenous segmentation result of the local image in the second arteriovenous segmentation result.
[0013] In a second aspect, an embodiment of the present application provides a wide-angle image segmentation device, including: a first determination module, configured to determine the optic disc in the target fundus image; a second determination module, configured to determine the first arteriovenous segmentation result corresponding to the target fundus image; a third determination module, configured to determine the vascular crossing points in the target fundus image, where the vascular crossing points include vascular branch crossing points and arteriovenous crossing points; a region growing module, configured to perform arterial region growing and venous region growing respectively based on the optic disc, the vascular crossing points and the first arteriovenous segmentation result to obtain the second arteriovenous segmentation result corresponding to the target fundus image.
[0014] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program for executing the method described in the first aspect.
[0015] In a fourth aspect, an embodiment of the present application provides an electronic device, including: a processor; a memory for storing instructions executable by the processor; the processor is configured to execute the method described in the first aspect.
[0016] The wide-angle image segmentation method proposed by the embodiments of the present application may first determine the optic disc in the target fundus image; then determine the first arteriovenous segmentation result corresponding to the target fundus image; then determine the vascular crossing points in the target fundus image, where the vascular crossing points include vascular branch crossing points and arteriovenous crossing points; and finally perform arterial region growing and venous region growing respectively based on the optic disc, the vascular crossing points and the first arteriovenous segmentation result to obtain the second arteriovenous segmentation result corresponding to the target fundus image.
[0017] Since the segmentation results of the optic disc and the vessel crossing points in the target fundus image are accurate, regional growth based on the optic disc and the vessel crossing points can obtain a second arteriovenous segmentation result with higher accuracy on the basis of the first arteriovenous segmentation result. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] The above and other objects, features, and advantages of the present application will become more apparent by describing the embodiments of the present application in more detail with reference to the accompanying drawings. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation to the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.
[0019] Figure 1 Shown is a schematic diagram of a scenario applicable to an exemplary embodiment of the present application.
[0020] Figure 2 Shown is a schematic flowchart of a wide-angle image segmentation method provided by an exemplary embodiment of the present application.
[0021] Figure 3 Shown is a schematic flowchart of a method for determining a second arteriovenous segmentation result provided by an exemplary embodiment of the present application.
[0022] Figure 4 Shown is a schematic flowchart of a method for determining an arteriovenous segmentation result corresponding to a target fundus image provided by an exemplary embodiment of the present application.
[0023] Figure 5 Shown is a schematic structural diagram of a wide-angle image segmentation device provided by an exemplary embodiment of the present application.
[0024] Figure 6 Shown is a schematic structural diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0025] Next, the technical solutions in the embodiments of the present application will be clearly and completely described with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.
[0026] Application Overview
[0027] Fundus images are the only images of the human body that can simply, non-invasively, and clearly observe capillary changes. Through fundus images, the basic structures of the fundus can be clearly observed, such as arteriovenous blood vessels, optic discs, optic cups, maculas, etc. According to incomplete statistics, more than 60 kinds of fundus diseases or systemic chronic diseases will have corresponding lesions in the fundus, such as high myopia, diabetic retinopathy, glaucoma, hypertensive retinopathy. Therefore, fundus images are usually the clinical diagnostic reference basis for various diseases.
[0028] Currently, more and more medical staff choose to use wide-angle fundus images with a wider viewing angle during diagnosis and treatment, which is more conducive to medical staff's diagnosis of diseases. However, compared with conventional fundus images, the differences in the diameters and colors of fundus blood vessels in wide-angle fundus images are smaller, resulting in a lower accuracy rate for segmenting wide-angle fundus images. And in actual tests, when segmenting wide-angle fundus images, the segmentation speed is also slower.
[0029] In view of this, this application proposes a wide-angle image segmentation method, which can first determine the optic disc in the target fundus image; then determine the first arteriovenous segmentation result corresponding to the target fundus image; then determine the blood vessel intersection points in the target fundus image, where the blood vessel intersection points include blood vessel branch intersection points and arteriovenous intersection points; finally, based on the optic disc, blood vessel intersection points and the first arteriovenous segmentation result, perform arterial region growth and venous region growth respectively to obtain the second arteriovenous segmentation result corresponding to the target fundus image.
[0030] Since the segmentation results of the optic disc and blood vessel intersection points in the target fundus image are accurate, region growth based on the optic disc and blood vessel intersection points can obtain a second arteriovenous segmentation result with higher accuracy on the basis of the first arteriovenous segmentation result.
[0031] Exemplary Scenarios
[0032] The wide-angle image segmentation method proposed in the embodiments of this application can be executed by an electronic device, which can be a terminal, such as a smart phone, a tablet computer, a desktop computer, etc., or the electronic device can also be a server.
[0033] Figure 1 The figure shows a schematic diagram of the application scenario of the wide-angle image segmentation method provided by an exemplary embodiment of this application. Figure 1 The shown scenario includes: a server 101 and a client 102 communicatively connected to the server 101. Among them, the server 101 is used to execute the method mentioned in the embodiments of this application.
[0034] Exemplarily, in the actual application process, the user issues an image segmentation instruction through the user terminal 102. After receiving the image segmentation instruction, the server 101 can perform arteriovenous segmentation on the target fundus image. Specifically, when performing arteriovenous segmentation on the target fundus image, the server 101 is used to: determine the optic disc in the target fundus image; determine the first arteriovenous segmentation result corresponding to the target fundus image; determine the vascular crossing points in the target fundus image, where the vascular crossing points include vascular branch crossing points and arteriovenous crossing points; and respectively perform arterial region growth and venous region growth based on the optic disc, vascular crossing points, and the first arteriovenous segmentation result to obtain the second arteriovenous segmentation result corresponding to the target fundus image.
[0035] Exemplarily, after determining the second arteriovenous segmentation result of the target fundus image, the server 101 can also determine the arteriovenous segmentation result corresponding to the target fundus image according to the first arteriovenous segmentation result and the second arteriovenous segmentation result.
[0036] Exemplarily, after determining the second arteriovenous segmentation result or the arteriovenous segmentation result corresponding to the target fundus image, the server 101 can send the second arteriovenous segmentation result or the arteriovenous segmentation result corresponding to the target fundus image to the user terminal 102 or other display devices to display the second arteriovenous segmentation result or the arteriovenous segmentation result corresponding to the target fundus image on the user terminal 102 or other display devices, thereby assisting medical personnel in diagnosis.
[0037] Exemplarily, the above-mentioned user terminal 102 includes, but is not limited to, computer terminals such as desktop computers and laptop computers, and mobile terminals such as tablet computers and mobile phones.
[0038] Exemplary Method
[0039] See Figure 2 shown, which is a schematic flowchart of a wide-angle image segmentation method provided by an exemplary embodiment of the present application. As Figure 2 shown, the wide-angle image segmentation method provided by the embodiment of the present application includes the following steps 201 to step 204.
[0040] Step 201, determine the optic disc in the target fundus image.
[0041] Exemplarily, the target fundus image can be a wide-angle fundus image captured by a fundus camera, such as a fundus image with a field of view angle of 135 degrees. The target fundus image includes, but is not limited to, all image data saved by a medical institution during the treatment of the same or different patients, relevant fundus image data of one or more patients input by a user, and fundus image data captured by one or more patients according to requirements during a medical visit. It should be noted that the fundus image can be an image captured centered on the optic disc or an image captured centered on the fovea. This application is mainly applicable to wide-angle fundus images, and this method is also applicable to ordinary fundus images. The server 101 can directly receive the target fundus image or obtain it from a data storage device.
[0042] In addition, the optic disc is a structure with obvious features in the eye. Therefore, a relatively accurate optic disc positioning result can be obtained in step 201. Exemplarily, the optic disc in the target fundus image can be determined through an optic disc detection model, and the optic disc detection model can be trained based on training images pre-annotated with optic disc position information.
[0043] Step 202: Determine the first arteriovenous segmentation result corresponding to the target fundus image.
[0044] The first arteriovenous segmentation result includes the segmented arterial region and venous region.
[0045] Exemplarily, the target fundus image can be input into a segmentation model, and the segmentation model outputs the first arteriovenous segmentation result corresponding to the target fundus image, where the segmentation model can be trained based on training images marked with arterial blood vessel regions and / or venous blood vessel regions.
[0046] Exemplarily, the target fundus image can be cut into multiple local images first, then the arteriovenous segmentation results corresponding to each local image are determined respectively, and then the first arteriovenous segmentation result is composed of the arteriovenous segmentation results corresponding to the multiple local images.
[0047] Step 203: Determine the blood vessel crossing points in the target fundus image.
[0048] The blood vessel crossing points include blood vessel branch crossing points and arteriovenous crossing points. The blood vessel branch crossing points are used to represent arterial blood vessel branch crossing points where a single arterial blood vessel bifurcates into two arterial blood vessels and / or venous blood vessel branch crossing points where a single venous blood vessel bifurcates into two venous blood vessels, and the arteriovenous crossing points are used to represent the crossing points of arterial blood vessels and venous blood vessels.
[0049] Exemplarily, the target fundus image can be input into a vascular crossing detection model, and the vascular crossings can be output by the vascular crossing detection model, where the vascular crossing detection model can be trained with training images marked with vascular branch crossings and / or arteriovenous crossings.
[0050] Step 204: Based on the optic disc, vascular crossings, and the first arteriovenous segmentation result, perform arterial region growth and venous region growth respectively to obtain the second arteriovenous segmentation result corresponding to the target fundus image.
[0051] The second arteriovenous segmentation result includes the segmented arterial region and the segmented venous region.
[0052] Specifically, the seed points corresponding to the arterial vessels and the seed points corresponding to the venous vessels can be determined based on the optic disc, vascular crossings, and the first arteriovenous segmentation result, and arterial region growth and venous region growth are respectively performed based on the seed points corresponding to the arterial vessels and the seed points corresponding to the venous vessels, and finally the segmented arterial region and venous region are obtained.
[0053] In a possible implementation manner, when performing arterial region growth and venous region growth respectively based on the optic disc, vascular crossings, and the first arteriovenous segmentation result to obtain the second arteriovenous segmentation result corresponding to the target fundus image, the target channel image corresponding to the target fundus image can be determined in advance, and then arterial region growth and venous region growth are respectively performed on the target channel image based on the optic disc, vascular crossings, and the first arteriovenous segmentation result to obtain the second arteriovenous segmentation result corresponding to the target fundus image.
[0054] Specifically, the target channel image can be a single-channel image or a combined channel image of multiple channels. In a specific example, if the target fundus image is an RGB (Red, Green, Blue, red, green, blue) channel image, the green channel image in the target fundus image can be extracted as the target channel image. Exemplarily, after extracting the green channel image, image enhancement processing can also be performed on the green channel image to improve the accuracy of region growth.
[0055] In a possible implementation manner, the first arteriovenous segmentation result includes a first arterial segmentation result and a first venous segmentation result. When performing arterial region growth and venous region growth respectively based on the optic disc, vascular crossings, and the first arteriovenous segmentation result to obtain the second arteriovenous segmentation result corresponding to the target fundus image, as Figure 3 shown in the flowchart of the method for determining the second arteriovenous segmentation result provided by an exemplary embodiment of the present application, the following steps 301 to 303 can be included, where the execution order of steps 301 and 302 is not in sequence.
[0056] Step 301: Based on the optic disc, blood vessel branch intersection points, arteriovenous crossing points, and the first artery segmentation result, perform artery region growth based on a preset blood vessel growth rule to obtain a second artery segmentation result.
[0057] Exemplarily, the first artery segmentation result can be the artery region segmented by a segmentation model. In addition, the preset blood vessel growth rule can include at least one of a region growth direction, a region growth condition, and a stop region growth condition. The specific blood vessel growth rule will be described in detail later. The second artery segmentation result is the artery region segmented based on Step 301.
[0058] Step 302: Based on the optic disc, blood vessel branch intersection points, arteriovenous crossing points, and the first vein segmentation result, perform vein region growth based on a preset blood vessel growth rule to obtain a second vein segmentation result.
[0059] Exemplarily, the first vein segmentation result can be the vein region segmented by a segmentation model, and the second vein segmentation result is the vein region segmented based on Step 302.
[0060] Step 303: Based on the second artery segmentation result and the second vein segmentation result, determine the second arteriovenous segmentation result corresponding to the target fundus image.
[0061] Specifically, the second arteriovenous segmentation result can include the second artery segmentation result and the second vein segmentation result, such as the segmented artery image and the segmented vein image. Or, the second arteriovenous segmentation result can be the fused second artery segmentation result and the second vein segmentation result, such as an image containing the segmented artery blood vessels and vein blood vessels.
[0062] In a possible implementation manner, when performing artery region growth / vein region growth based on the blood vessel growth rule, artery region growth / vein region growth can be performed based on the first seed point, the second seed point, and the arteriovenous crossing points to obtain the second artery segmentation result / the second vein segmentation result. Among them, the first seed point is the pixel point corresponding to the artery / vein within the target radius determined based on the first artery segmentation result / the first vein segmentation result with the optic disc as the center, and the second seed point is the blood vessel branch intersection point.
[0063] Specifically, when performing artery region growth, the pixel points of the artery region in the first artery segmentation result within the target radius from the optic disc with the optic disc as the center can be used as the first seed point, and the blood vessel branch intersection point can be used as the second seed point.
[0064] Since the blood vessels closer to the optic disc have obvious features, the artery regions segmented within the target radius of the optic disc in the first artery segmentation result are accurate. The blood vessel branch intersection points mentioned here can be artery blood vessel branch intersection points, or can be artery blood vessel branch intersection points and vein blood vessel branch intersection points.
[0065] When any seed point (including the first seed point and the second seed point) grows the artery region, the adjacent pixel points of the seed point (such as 8-neighborhood pixel points) can be determined, and then it is determined whether the adjacent pixel points meet the region growth conditions. If so, the adjacent pixel points are incorporated into the segmented artery region; if not, the adjacent pixel points are not incorporated into the segmented artery region.
[0066] Similarly, when growing the vein region, the pixel points of the vein region in the first vein segmentation result within the target radius of the optic disc centered on the optic disc can be used as the first seed point, and the blood vessel branch intersection point can be used as the second seed point. Among them, the blood vessel branch intersection point can be a vein blood vessel branch intersection point, or can be an artery blood vessel branch intersection point and a vein blood vessel branch intersection point.
[0067] When any seed point (including the first seed point and the second seed point) grows the vein region, the adjacent pixel points of the seed point (such as 8-neighborhood pixel points) can be determined, and then it is determined whether the adjacent pixel points meet the region growth conditions. If so, the adjacent pixel points are incorporated into the segmented vein region; if not, the adjacent pixel points are not incorporated into the segmented vein region.
[0068] Exemplarily, the above region growth conditions may include at least one of the following: the gradient information of the adjacent pixel points is less than the gradient threshold, and the gray difference value between the seed point and the adjacent pixel points is less than the gray difference threshold. It can be understood that since the gradient information of the blood vessel region is lower than that of the background region, the gray information can accurately distinguish whether the adjacent seed points are seed points of the blood vessel region. In addition, the gray values of the pixel points within the same blood vessel have small differences, so the gray difference value can accurately determine whether the seed point and the adjacent pixel points are pixel points within the same blood vessel.
[0069] In a possible implementation manner, the growth of the first seed point stops when it reaches the second seed point, and the second seed point grows in the opposite direction of the optic disc. That is, the artery pixel points / vein pixel points near the optic disc stop growing when they reach the blood vessel branch intersection point, while the blood vessel branch intersection point grows in the direction away from the optic disc. Here, since the first seed point and the second seed point can grow in different directions simultaneously, using this method can greatly shorten the time required for region growth, thereby improving the time required for arteriovenous segmentation of wide-angle fundus images.
[0070] In a possible implementation, during the process of artery region growth / vein region growth based on the first seed point and the second seed point, if the growth reaches the arteriovenous crossing point, determine the two tangent directions of the two crossed blood vessels corresponding to the arteriovenous crossing point extending towards the optic disc direction, and determine the target tangent direction among the two tangent directions where the already grown artery region / vein region exists; take the arteriovenous crossing point as the third seed point, and continue the artery region growth / vein region growth along the reverse direction of the target tangent direction based on the blood vessel growth rule.
[0071] Specifically, since the arteriovenous crossing point is the crossing point of the artery blood vessel and the vein blood vessel, and when performing artery region growth, only the artery blood vessel needs to be included in the artery region, and when performing vein region growth, only the vein blood vessel needs to be included in the vein region. Therefore, it is necessary to determine which blood vessel at the arteriovenous crossing point needs to be included in the already segmented artery region / vein region. Since the first seed point determined based on the optic disc and the first arteriovenous segmentation result is accurate, the artery region / vein region determined based on the first seed point in the optic disc direction is accurate. If there is an already grown artery region / vein region in the (towards the optic disc) tangent direction of any crossed blood vessel, it means that this crossed blood vessel is an artery blood vessel / vein blood vessel. Therefore, growth can continue only along the direction of this crossed blood vessel (i.e., the reverse direction of the target tangent direction).
[0072] Through this method, the direction in which region growth needs to be performed among the two crossed blood vessels can be accurately identified, so as to accurately perform the region growth of the artery region / vein region.
[0073] Among them, when taking the arteriovenous crossing point as the third seed point and continuing the artery region growth / vein region growth along the reverse direction of the tangent direction corresponding to the already grown artery region / vein region based on the blood vessel growth rule, the third seed point can be used as a new seed point to perform region growth according to the growth rule of the above-mentioned seed point. For example, determine whether the adjacent pixel points of this seed point meet the region growth condition. If so, include this adjacent pixel point in the already segmented artery region / vein region. If not, do not include this adjacent pixel point in the already segmented artery region / vein region. Another example is that during the process of artery region growth / vein region growth based on the second seed point, if the growth reaches the arteriovenous crossing point, determine the tangent direction corresponding to the already grown artery region / vein region among the tangent directions of the two crossed blood vessels corresponding to the arteriovenous crossing point; take the arteriovenous crossing point as the new third seed point, and continue the artery region growth / vein region growth along the reverse direction of the tangent direction corresponding to the already grown artery region / vein region based on the blood vessel growth rule.
[0074] Similarly, in any of the above embodiments, for the pixel points included in the segmented artery region / vein region, they are used as new second seed points and region growing is performed according to the growing rules of any of the above second seed points. This process is repeated until there are no new second seed points in the target fundus image, and a second artery segmentation result / a second vein segmentation result is obtained.
[0075] In a possible implementation, for each blood vessel branch crossing point, if it is determined that there is no artery region obtained by growing and connected to the artery region corresponding to the blood vessel branch crossing point in the direction from the blood vessel branch crossing point to the optic disc, the artery region corresponding to the blood vessel branch crossing point is deleted from the second artery segmentation result, where the artery region corresponding to the blood vessel branch crossing point is the region obtained by growing the artery region with the blood vessel branch crossing point as the seed point; and / or, for each blood vessel branch crossing point, if it is determined that there is no vein region obtained by growing and connected to the vein region corresponding to the blood vessel branch crossing point in the direction from the blood vessel branch crossing point to the optic disc, the vein region corresponding to the blood vessel branch crossing point is deleted from the second vein segmentation result, where the vein region corresponding to the blood vessel branch crossing point is the region obtained by growing the vein region with the blood vessel branch crossing point as the seed point.
[0076] Specifically, since it is not possible to accurately distinguish whether the blood vessel branch crossing point is an artery blood vessel branch crossing point or a vein blood vessel branch crossing point (such as detected by a blood vessel crossing point detection model), during the artery region growing / vein region growing, the region grown based on the blood vessel branch crossing point includes both the vein region and the artery region. Since the first seed points determined based on the optic disc and the first artery-vein segmentation result are accurate, the artery region / vein region determined based on the first seed points in the direction of the optic disc is accurate. If there is no artery region / vein region obtained by growing and connected to the artery region / vein region corresponding to the blood vessel branch crossing point in the direction from the blood vessel branch crossing point to the optic disc, it indicates that during the artery region / vein region growing, this blood vessel branch crossing point is a vein blood vessel branch crossing point / artery blood vessel branch crossing point. Therefore, the region grown based on this blood vessel branch crossing point needs to be deleted from the second artery segmentation result / second vein segmentation result.
[0077] By this method, the vein region grown based on the vein blood vessel branch crossing point / the artery region grown based on the artery blood vessel branch crossing point can be accurately deleted from the second artery segmentation result / second vein segmentation result, solving the problem of incorrect detection of the artery region / vein region caused by the inability to accurately identify the type of blood vessel branch crossing point.
[0078] In a possible implementation manner, the arteriovenous segmentation result corresponding to the target fundus image may also be determined according to the first arteriovenous segmentation result and the second arteriovenous segmentation result. It can be understood that by combining the first arteriovenous segmentation result and the second arteriovenous segmentation result, the arteriovenous segmentation result corresponding to the target fundus image can be determined more accurately. Exemplarily, it may be to take the intersection of the first arteriovenous segmentation result and the second arteriovenous segmentation result, the union of the first arteriovenous segmentation result and the second arteriovenous segmentation result, take the result with a larger / smaller arterial / venous region from the first arteriovenous segmentation result and the second arteriovenous segmentation result, etc.
[0079] In a specific example, the target fundus image includes multiple local images. When determining the arteriovenous segmentation result corresponding to the target fundus image according to the first arteriovenous segmentation result and the second arteriovenous segmentation result, as Figure 4 shown in the flowchart of the method for determining the arteriovenous segmentation result corresponding to the target fundus image provided by an exemplary embodiment of the present application, the following steps 401 to step 403 may be included.
[0080] Step 401: For each local image, determine a first pixel set of the local image in the first arteriovenous segmentation result and a second pixel set of the local image in the second arteriovenous segmentation result.
[0081] Specifically, in the case where the first arteriovenous segmentation result includes the arteriovenous segmentation results corresponding to multiple local images, for the arteriovenous segmentation result corresponding to any local image, the arteriovenous segmentation result corresponding to the local image at the corresponding position can be found from the second arteriovenous segmentation result, and then the first pixel set of the local image in the first arteriovenous segmentation result and the second pixel set in the second arteriovenous segmentation result can be determined. Among them, the first pixel set includes the pixel values of multiple pixels of the local image in the first arteriovenous segmentation result, and the second pixel set includes the pixel values of multiple pixels of the local image in the second arteriovenous segmentation result.
[0082] Step 402: Based on the first pixel set and the second pixel set, determine the average difference of the matching pixels in the first pixel set and the second pixel set.
[0083] The matching pixels in the first pixel set and the second pixel set are the pixels with the same position information in the first pixel set and the second pixel set.
[0084] Exemplarily, the difference of each pixel in the first pixel set and the second pixel set can be calculated, then the differences of each pixel are added up, and then divided by the number of pixels to obtain the average difference of the matching pixels in the first pixel set and the second pixel set.
[0085] Step 403: Determine whether the average difference value is greater than the target threshold.
[0086] If yes, execute Step 404; if no, execute Step 405.
[0087] Step 404: Use the arteriovenous segmentation result of the local image in the second arteriovenous segmentation result as the arteriovenous segmentation result corresponding to the local image.
[0088] Step 405: Determine the arteriovenous segmentation result corresponding to the local image based on the arteriovenous segmentation result of the local image in the first arteriovenous segmentation result and the arteriovenous segmentation result of the local image in the second arteriovenous segmentation result.
[0089] It can be understood that the second arteriovenous segmentation result obtained by using the region growing algorithm is more accurate than the first arteriovenous segmentation result. Therefore, when the average difference value is greater than the target threshold (that is, when there is a large difference between the first arteriovenous segmentation result and the second arteriovenous segmentation result), the arteriovenous segmentation result of the local image in the second arteriovenous segmentation result can be selected as the arteriovenous segmentation result corresponding to the final local image.
[0090] If the average difference value is not greater than the target threshold, it indicates that the arteriovenous segmentation result of the local image in the first arteriovenous segmentation result is not much different from the arteriovenous segmentation result of the local image in the second arteriovenous segmentation result. Then, the arteriovenous segmentation result of the local image in the first arteriovenous segmentation result and the arteriovenous segmentation result of the local image in the second arteriovenous segmentation result can be fused to obtain the arteriovenous segmentation result corresponding to the final local image. Exemplarily, the union of the arteriovenous segmentation result of the local image in the first arteriovenous segmentation result and the arteriovenous segmentation result of the local image in the second arteriovenous segmentation result can be taken as the arteriovenous segmentation result corresponding to the final local image to obtain a more complete arteriovenous segmentation result.
[0091] Through the above method, different methods can be selected to determine the arteriovenous segmentation result corresponding to the local image according to the pixel value difference of the local image in the first arteriovenous result and the second arteriovenous result, so as to improve the accuracy of the arteriovenous segmentation result corresponding to the local image.
[0092] In the above embodiment, the optic disc in the target fundus image can be determined first; then the first arteriovenous segmentation result corresponding to the target fundus image can be determined; then the blood vessel crossing points in the target fundus image can be determined, and the blood vessel crossing points include blood vessel branch crossing points and arteriovenous crossing points; finally, based on the optic disc, the blood vessel crossing points and the first arteriovenous segmentation result, arterial region growing and venous region growing are respectively performed to obtain the second arteriovenous segmentation result corresponding to the target fundus image.
[0093] By this method, since the segmentation results of the optic disc and the vascular crossing points in the target fundus image are accurate, region growing can be performed based on the optic disc and the vascular crossing points to obtain a second arteriovenous segmentation result with higher accuracy on the basis of the first arteriovenous segmentation result.
[0094] Exemplary Device
[0095] As described above in conjunction with Figures 2 to 4 , the method embodiments of the present application have been described in detail. Next, in conjunction with Figure 5 and Figure 6 , the apparatus embodiments of the present application will be described in detail. It should be understood that the descriptions of the method embodiments and the apparatus embodiments correspond to each other. Therefore, for the parts not described in detail, reference may be made to the previous method embodiments.
[0096] Figure 5 FIG. shows a schematic structural diagram of an apparatus provided by an exemplary embodiment of the present application. As Figure 5 shown, the wide-angle image segmentation apparatus 500 provided by the embodiment of the present application includes:
[0097] A first determination module 501, configured to determine the optic disc in the target fundus image;
[0098] A second determination module 502, configured to determine the first arteriovenous segmentation result corresponding to the target fundus image;
[0099] A third determination module 503, configured to determine the vascular crossing points in the target fundus image, where the vascular crossing points include vascular branch crossing points and arteriovenous crossing points;
[0100] A region growing module 504, configured to perform artery region growing and vein region growing respectively based on the optic disc, the vascular crossing points, and the first arteriovenous segmentation result to obtain the second arteriovenous segmentation result corresponding to the target fundus image.
[0101] In a possible implementation manner, the first arteriovenous segmentation result includes a first artery segmentation result and a first vein segmentation result; the region growing module 504 is further configured to: perform artery region growing based on the optic disc, the vascular branch crossing points, the arteriovenous crossing points, and the first artery segmentation result according to a preset vascular growth rule to obtain a second artery segmentation result; perform vein region growing based on the optic disc, the vascular branch crossing points, the arteriovenous crossing points, and the first vein segmentation result according to a preset vascular growth rule to obtain a second vein segmentation result; and determine the second arteriovenous segmentation result corresponding to the target fundus image based on the second artery segmentation result and the second vein segmentation result.
[0102] In a possible implementation, the region growing module 504 is further configured to: perform artery region growing / vein region growing based on the first seed point, the second seed point, and the arteriovenous crossing point to obtain a second artery segmentation result / a second vein segmentation result; wherein, the first seed point is a pixel point corresponding to an artery / vein within a target radius determined based on the first artery segmentation result / the first vein segmentation result with the optic disc as the center, and the second seed point is a blood vessel branch crossing point.
[0103] In a possible implementation, the first seed point stops growing when it reaches the second seed point, and the second seed point grows in the opposite direction towards the optic disc.
[0104] In a possible implementation, the region growing module 504 is further configured to: during the process of performing artery region growing / vein region growing based on the first seed point and the second seed point, if it grows to the arteriovenous crossing point, determine two tangent directions extending towards the optic disc direction of the two crossing blood vessels corresponding to the arteriovenous crossing point, and determine a target tangent direction in which there is a grown artery region / vein region among the two tangent directions; use the arteriovenous crossing point as the third seed point, and continue to perform artery region growing / vein region growing based on the blood vessel growth rule in the opposite direction of the target tangent direction.
[0105] In a possible implementation, the region growing module 504 is further configured to: for each blood vessel branch crossing point, if it is determined that there is no grown artery region connected to the artery region corresponding to the blood vessel branch crossing point in the direction from the blood vessel branch crossing point to the optic disc, delete the artery region corresponding to the blood vessel branch crossing point from the second artery segmentation result, where the artery region corresponding to the blood vessel branch crossing point is the region obtained by performing artery region growing with the blood vessel branch crossing point as the seed point; and / or, for each blood vessel branch crossing point, if it is determined that there is no grown vein region connected to the vein region corresponding to the blood vessel branch crossing point in the direction from the blood vessel branch crossing point to the optic disc, delete the vein region corresponding to the blood vessel branch crossing point from the second vein segmentation result, where the vein region corresponding to the blood vessel branch crossing point is the region obtained by performing vein region growing with the blood vessel branch crossing point as the seed point.
[0106] In a possible implementation, the apparatus is further configured to: determine the arteriovenous segmentation result corresponding to the target fundus image according to the first arteriovenous segmentation result and the second arteriovenous segmentation result.
[0107] In a possible implementation, the target fundus image includes a plurality of local images, and the apparatus is further configured to: for each local image, determine a first pixel set of the local image in the first arteriovenous segmentation result and a second pixel set of the local image in the second arteriovenous segmentation result; based on the first pixel set and the second pixel set, determine the average difference of the matching pixels in the first pixel set and the second pixel set; in the case where the average difference is greater than the target threshold, use the arteriovenous segmentation result of the local image in the second arteriovenous segmentation result as the arteriovenous segmentation result corresponding to the local image; in the case where the average difference is not greater than the target threshold, based on the arteriovenous segmentation result of the local image in the first arteriovenous segmentation result and the arteriovenous segmentation result of the local image in the second arteriovenous segmentation result, determine the arteriovenous segmentation result corresponding to the local image.
[0108] Next, an electronic device according to an embodiment of the present application will be described with reference to Figure 6 FIG. Figure 6 FIG. 6 is a schematic structural diagram of an electronic device provided by an exemplary embodiment of the present application.
[0109] As Figure 6 shown in FIG. 6, the electronic device 600 includes one or more processors 601 and a memory 602.
[0110] The processor 601 may be a central processing unit (CPU) or other form of processing unit having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 600 to perform desired functions.
[0111] The memory 602 may include one or more computer program products, and the computer program products may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory, etc. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 601 may run the program instructions to implement the wide-angle image segmentation method of various embodiments of the present application described above and / or other desired functions. Various contents such as the target fundus image, the first arteriovenous segmentation result, the second arteriovenous segmentation result, etc. may also be stored in the computer-readable storage medium.
[0112] In one example, the electronic device 600 may further include: an input device 603 and an output device 604, and these components are interconnected through a bus system and / or other forms of connection mechanisms (not shown).
[0113] The input device 603 may include, for example, a keyboard, a mouse, and the like.
[0114] The output device 604 may output various information to the outside, including the first arteriovenous segmentation result, the second arteriovenous segmentation result, and the like. The output device 604 may include, for example, a display, a speaker, a printer, a communication network, and remote output devices connected thereto, and the like.
[0115] Of course, for simplicity, Figure 6 only some of the components related to the present application in the electronic device 600 are shown in [reference], and components such as a bus, an input / output interface, and the like are omitted. In addition, according to specific application scenarios, the electronic device 600 may further include any other appropriate components.
[0116] In addition to the above methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions that, when run by a processor, cause the processor to execute the steps in the wide-angle image segmentation method according to various embodiments of the present application described above in this specification.
[0117] The computer program product may be written in any combination of one or more programming languages for programming code to perform the operations of the embodiments of the present application. The programming languages include object-oriented programming languages such as Java, C++, etc., and also include conventional procedural programming languages such as the "C" language or similar programming languages. The programming code may be executed entirely on the user's computing device, partially on the user's device, executed as a stand-alone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.
[0118] Furthermore, an embodiment of the present application may also be a computer-readable storage medium, on which computer program instructions are stored, and the computer program instructions, when run by a processor, cause the processor to execute the steps in the image detection method according to various embodiments of the present application described above in this specification.
[0119] The computer-readable storage medium may adopt any combination of one or more readable media. The readable media may be a readable signal medium or a readable storage medium. The readable storage medium may include, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.
[0120] The basic principles of the present application have been described above in conjunction with specific embodiments. However, it should be noted that the advantages, benefits, effects, etc. mentioned in the present application are only examples and not limitations. It cannot be considered that these advantages, benefits, effects, etc. are essential for each embodiment of the present application. In addition, the above-disclosed specific details are only for illustrative and easy-to-understand purposes and are not limitations. The above details do not limit the present application to necessarily adopt the above specific details for implementation.
[0121] The block diagrams of the devices, apparatuses, equipment, and systems involved in the present application are only illustrative examples and do not intend to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open-ended words, meaning "including but not limited to", and can be used interchangeably with each other. The words "or" and "and" used herein refer to the word "and / or" and can be used interchangeably with each other, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to" and can be used interchangeably with each other.
[0122] It should also be noted that in the devices, equipment, and methods of the present application, each component or each step can be decomposed and / or recombined. These decompositions and / or recombinations should be regarded as equivalent solutions of the present application.
[0123] The above description of the disclosed aspects enables any person skilled in the art to make or use the present application. Various modifications to these aspects are very obvious to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of the present application. Therefore, the present application is not intended to be limited to the aspects shown herein, but rather to the broadest scope consistent with the principles and novel features disclosed herein.
[0124] The foregoing description has been presented for purposes of illustration and description. In addition, this description is not intended to limit the embodiments of the present application to the forms disclosed herein. Although several example aspects and embodiments have been discussed above, those skilled in the art will recognize some of their variations, modifications, alterations, additions, and sub-combinations.
Claims
1. A wide-angle image segmentation method, characterized in that: include: identifying the optic disc in the target fundus image; Determine a first artery and vein segmentation result corresponding to the target fundus image; Determine a blood vessel intersection in the target fundus image, wherein the blood vessel intersection includes a blood vessel branch intersection and an arteriovenous intersection; Based on the optic disc, the blood vessel intersection and the first artery and vein segmentation result, artery region growth and vein region growth are performed respectively to obtain a second artery and vein segmentation result corresponding to the target fundus image.
2. The wide-angle image segmentation method according to claim 1, characterized in that: The first artery and vein segmentation result includes a first artery segmentation result and a first vein segmentation result; The step of performing arterial region growth and venous region growth based on the optic disc, the vascular intersection and the first arteriovenous segmentation result to obtain a second arteriovenous segmentation result corresponding to the target fundus image includes: According to the optic disc, the intersection of the blood vessel branches, the intersection of the artery and vein, and the first artery segmentation result, based on a preset blood vessel growth rule, artery region growth is performed to obtain a second artery segmentation result; According to the optic disc, the blood vessel branch intersection, the arteriovenous intersection and the first vein segmentation result, based on a preset blood vessel growth rule, vein region growth is performed to obtain a second vein segmentation result; Based on the second artery segmentation result and the second vein segmentation result, a second artery and vein segmentation result corresponding to the target fundus image is determined.
3. The wide-angle image segmentation method according to claim 2, characterized in that: Based on the blood vessel growth rule, arterial region growth / venous region growth is performed, including: Based on the first seed point, the second seed point and the arteriovenous intersection point, arterial region growing / venous region growing is performed to obtain the second artery segmentation result / the second vein segmentation result; The first seed point is a pixel point corresponding to an artery / vein within a target radius determined based on the first artery segmentation result / the first vein segmentation result with the optic disc as the center, and the second seed point is a blood vessel branch intersection.
4. The wide-angle image segmentation method according to claim 3, characterized in that: The first seed point grows to the second seed point and stops growing, and the second seed point grows in a direction away from the optic disc.
5. The wide-angle image segmentation method according to claim 4, characterized in that: The method further comprises: In the process of performing arterial region growth / venous region growth based on the first seed point and the second seed point, if the growth reaches the arteriovenous intersection, two tangent directions extending toward the optic disc of two intersecting blood vessels corresponding to the arteriovenous intersection are determined, and a target tangent direction of the arterial region / venous region obtained by the growth is determined in the two tangent directions; The arteriovenous intersection is used as the third seed point, and the arterial region growth / venous region growth is continued based on the blood vessel growth rule along the opposite direction of the target tangent direction.
6. The wide-angle image segmentation method according to claim 5, characterized in that: The method further comprises: For each of the vascular branch intersections, if it is determined that there is no arterial region obtained by growth that is connected to the arterial region corresponding to the vascular branch intersection in the direction from the vascular branch intersection to the optic disc, the arterial region corresponding to the vascular branch intersection is deleted from the second arterial segmentation result, wherein the arterial region corresponding to the vascular branch intersection is a region obtained by growing the arterial region using the vascular branch intersection as a seed point; and / or, For each of the vascular branch intersections, if it is determined that there is no grown vein area that is connected to the vein area corresponding to the vascular branch intersection in the direction from the vascular branch intersection to the optic disc, the vein area corresponding to the vascular branch intersection is deleted from the second vein segmentation result, wherein the vein area corresponding to the vascular branch intersection is the area obtained by growing the vein area with the vascular branch intersection as the seed point.
7. The wide-angle image segmentation method according to claim 1, characterized in that: The method further comprises: The artery and vein segmentation result corresponding to the target fundus image is determined according to the first artery and vein segmentation result and the second artery and vein segmentation result.
8. The wide-angle image segmentation method according to claim 7, characterized in that: The target fundus image includes a plurality of local images, and determining the arteriovenous segmentation result corresponding to the target fundus image according to the first arteriovenous segmentation result and the second arteriovenous segmentation result includes: For each of the partial images, determining a first pixel set of the partial image in the first artery and vein segmentation result, and a second pixel set of the partial image in the second artery and vein segmentation result; Determine, based on the first pixel set and the second pixel set, an average of differences between matching pixels in the first pixel set and the second pixel set; When the difference average value is greater than the target threshold, taking the arteriovenous segmentation result of the local image in the second arteriovenous segmentation result as the arteriovenous segmentation result corresponding to the local image; When the difference average value is not greater than the target threshold, the arteriovenous segmentation result corresponding to the local image is determined based on the arteriovenous segmentation result of the local image in the first arteriovenous segmentation result and the arteriovenous segmentation result of the local image in the second arteriovenous segmentation result.
9. A wide-angle image segmentation device, characterized in that: include: A first determination module is used to determine the optic disc in the target fundus image; A second determination module is used to determine a first artery and vein segmentation result corresponding to the target fundus image; A third determination module is used to determine the blood vessel intersection points in the target fundus image, wherein the blood vessel intersection points include blood vessel branch intersection points and arteriovenous intersection points; A region growing module is used to perform arterial region growing and venous region growing respectively based on the optic disc, the blood vessel intersection and the first arteriovenous segmentation result to obtain a second arteriovenous segmentation result corresponding to the target fundus image.
10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and the computer program is used to execute the wide-angle image segmentation method according to any one of claims 1 to 8.
11. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is used to execute the wide-angle image segmentation method described in any one of claims 1 to 8.