Image dark spot elimination method based on multi-threshold shape measurement

Abstract

The embodiment of the invention relates to the technical field of eye fundus image processing, in particular to an image dark spot elimination method based on multi-threshold shape measurement, and the method comprises the steps: carrying out the ROI region extraction of a to-be-processed eye fundus image, and carrying out the graying processing, and obtaining a gray image of the ROI region; performing multi-threshold segmentation on the grayscale image to obtain a plurality of binary images; candidate dark spot detection is carried out on each binary image based on dark spot shape measurement, and candidate dark spots in each binary image are determined; voting based on the candidate dark spots in each binary image, screening out real dark spots, and determining a real center point and a real radius of the real dark spots based on the center point and the radius of the real dark spots in each binary image; and defining a reference region based on the real radius of the real dark spot, and eliminating the real dark spot based on the average gray value of the reference region. According to the method, the dark spots in the eye fundus image can be quickly and conveniently eliminated with high quality, and the required computing resources are few.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of fundus image processing, and in particular to a method for removing dark spots from an image based on multi-threshold shape measurement. Background Art

[0002] The structure of the human eyeball is very delicate, very similar to a traditional camera, and the fundus is like the film of a camera. The fundus is the only part of the human body where the arteries, veins and capillaries can be directly and centrally observed with the naked eye. These blood vessels can reflect the dynamics of the blood circulation and health status of the whole body. Many diseases can be reflected in the fundus, such as fundus hemorrhage, which is a serious complication of diabetes. Hypertension, coronary heart disease and kidney disease will also leave "clues" on the fundus. Fundus photography is a way to examine the fundus. Fundus images allow us to see the organizational structure of the fundus, analyze whether there are abnormalities, and finally give a specific treatment plan based on detailed screening and diagnosis.

[0003] Normal fundus images are orange-red and consist of structures such as the retina, fundus arteries, fundus veins, optic disc, optic nerve fibers, macula on the retina, fovea, and choroid behind the retina. Lesions in these parts are collectively referred to as fundus diseases. It can be seen that the quality of fundus images is directly related to the discovery and further detection of fundus diseases.

[0004] Dark spots (stray light) are typical image defects in fundus images, and the presence of dark spots seriously affects the clarity of fundus images. At present, technicians mostly use image enhancement technology to reduce the interference of weak or partial distortion defects, and use image repair technology to repair severely distorted areas. However, due to the limitations of built-in hardware resources of fundus cameras, the applicability of deep learning methods is poor, and traditional image processing methods have better applicability, but traditional image processing methods are not specifically aimed at the processing of dark spots in fundus images, and customized dark spot elimination methods need to be further explored. Summary of the invention

[0005] In view of this, an embodiment of the present application proposes an image dark spot removal method based on multi-threshold shape measurement, which can quickly, conveniently and with high quality remove dark spots in fundus images, requires fewer computing resources, and can be perfectly deployed on common fundus cameras.

[0006] In a first aspect, an embodiment of the present application proposes a method for removing dark spots from an image based on a multi-threshold shape metric, comprising the following steps: extracting a ROI (Region Of Interest) region from an image to be processed, and graying the extracted ROI region to obtain a grayscale image of the ROI region; performing multi-threshold segmentation on the grayscale image of the ROI region to obtain a number of binary images of the ROI region; performing candidate dark spot detection on each binary image based on the dark spot shape metric to determine the candidate dark spots in each binary image; wherein the dark spot shape metric includes color, area, roundness, convexity and inertia rate; voting based on the candidate dark spots in each binary image to screen out the real dark spots, and determining the real center point and real radius of the real dark spots based on the center point and radius of the real dark spots in each binary image; defining a reference area based on the real radius of the real dark spot, and eliminating the real dark spots in the image to be processed based on the average grayscale value of the reference area.

[0007] Optionally, the grayscale image of the ROI area is subjected to multi-threshold segmentation to obtain a number of binary images of the ROI area, including: traversing each grayscale value threshold value preset during multi-threshold segmentation, and determining in turn whether the grayscale value of each pixel in the grayscale image is greater than the current grayscale value threshold value; assigning the grayscale value of the pixel in the grayscale image whose grayscale value is less than the current grayscale value threshold value to 0, and assigning the grayscale value of the pixel in the grayscale image whose grayscale value is greater than or equal to the current grayscale value threshold value to 1, to obtain a binary image obtained based on the current grayscale value threshold segmentation; after completing the traversal of each grayscale value threshold value, a number of binary images of the ROI area are obtained.

[0008] Optionally, the method of performing candidate dark spot detection on each binary image based on the dark spot shape measurement to determine the candidate dark spots in each binary image includes: traversing each binary image, performing connected area detection on the current binary image, and judging in turn whether each connected area in the current binary image satisfies five dark spot shape measurement conditions; determining the connected area in the current binary image that simultaneously satisfies the five dark spot shape measurement conditions as the candidate dark spot; after completing the traversal of each binary image, obtaining the candidate dark spots in each binary image; wherein the five dark spot shape measurement conditions are respectively a color measurement condition, an area measurement condition, a roundness measurement condition, a convexity measurement condition, and an inertia rate measurement condition.

[0009] Optionally, the color measurement condition is met when the grayscale values ​​of the pixels in the connected area are all 0; the area measurement condition is met when the area S of the connected area is within a preset area range; the roundness measurement condition is met when the roundness C of the connected area is greater than a preset roundness threshold, C = 4πS / p 2, p is the perimeter of the connected area; the convexity measurement condition is met when the convexity V of the connected area is within the preset convexity range, V=S / H, H is the area of ​​the convex hull of the connected area; the inertia rate measurement condition is met when the inertia rate of the connected area is greater than the preset inertia rate threshold.

[0010] Optionally, voting is performed based on the candidate dark spots in each binary image to screen out the real dark spots, including: based on the center points of the candidate dark spots in each binary image, combined with the principle that center point deviations less than a preset deviation threshold are regarded as the same candidate dark spot, voting is performed in units of each binary image, and the number of times each candidate dark spot appears is counted; and the candidate dark spot whose number of appearances is greater than the preset appearance threshold is determined as the real dark spot.

[0011] Optionally, the real center point and the real radius of the real dark spot are determined based on the center point and the radius of the real dark spot in each binary image by the following formula:

[0012]

[0013] Among them, (x i ,y i ) represents the coordinates of the center point of the real dark spot in the i-th binary image, For voting symbols, Indicates that the real dark spot has never appeared in the i-th binary image. Indicates that the real dark spot has appeared in the i-th binary image, β i represents the inertia rate of the real dark spot in the i-th binary image, n represents the total number of binary images, (x t ,y t ) represents the coordinates of the real center point of the real dark spot, r i represents the radius of the real dark spot in the i-th binary image, r t Indicates the true radius of the true dark spot.

[0014] Optionally, the step of defining a reference area based on a real radius of the real dark spot, and eliminating the real dark spot in the fundus image to be processed based on the average grayscale value of the reference area includes: defining a circular area in the grayscale image with a real center point of the real dark spot as the center and 3 times the real radius as the radius, and subtracting a circular ring formed by the real dark spot from the defined circular area as the reference area; calculating the average grayscale value of every other pixel point in the four directions of due east, due west, due south and due north in the circular area starting from the real center point to obtain a grayscale value curve L;

[0015] L=[l1,l2,…,l j ,…,l 3r ];

[0016]

[0017] j∈[0,3r t ];

[0018] Calculate j∈[r t ,3r t All l in ] i The average value R, taking R as the reference value, calculates R and j∈[0,r t All l in ] i The difference is obtained by performing polynomial fitting on the non-smooth curve to obtain the fitted difference grayscale data, and the fitted difference grayscale data is superimposed on the grayscale value curve L to finally obtain the result image with the real dark spots eliminated.

[0019] Through the above method, the ROI area is first determined in the fundus image to be processed, and the uninteresting and unimportant areas are excluded from the calculation range, which effectively saves computing resources. Then, the extracted ROI area is grayed to obtain a grayscale image of the ROI area, and the grayscale image of the ROI area is multi-threshold segmented to obtain several binary images of the ROI area. Multi-threshold segmentation is more reasonable than single threshold segmentation, which improves the accuracy of subsequent determination of dark spots. Next, based on the dark spot shape metrics including color, area, roundness, convexity and inertia rate, candidate dark spots are detected for each binary image, and the candidate dark spots in each binary image are determined. Based on the dark spot shape metric, non-dark spot areas can be well eliminated. Then, based on the candidate dark spots in each binary image, voting is performed to screen out the real dark spots, and based on the center point and radius of the real dark spots in each binary image, the real center point and real radius of the real dark spots are determined. Finally, the reference area is delineated based on the real radius, and the real dark spots in the fundus image to be processed are eliminated based on the average grayscale value of the reference area, thereby eliminating the dark spots in the fundus image quickly, conveniently and with high quality. The required computing resources are relatively small and can be perfectly deployed on common fundus cameras. At the same time, the color consistency between the dark spots after elimination and the surrounding areas is guaranteed.

[0020] In the second aspect, an embodiment of the present application proposes an image dark spot removal system based on multi-threshold shape measurement, the system comprising: a preprocessing module, used to extract the ROI area of ​​the fundus image to be processed, and grayscale the extracted ROI area to obtain a grayscale image of the ROI area; a multi-threshold segmentation module, used to perform multi-threshold segmentation on the grayscale image of the ROI area to obtain a number of binary images of the ROI area; a detection module, used to perform candidate dark spot detection on each binary image based on the dark spot shape measurement, and determine the candidate dark spots in each binary image, wherein the dark spot shape measurement includes color, area, roundness, convexity and inertia rate; a voting module, used to vote based on the candidate dark spots in each binary image, screen out the real dark spots, and determine the real center point and real radius of the real dark spots based on the center point and radius of the real dark spots in each binary image; an elimination module, used to delineate a reference area based on the real radius of the real dark spot, and eliminate the real dark spots in the fundus image to be processed based on the average grayscale value of the reference area.

[0021] In a third aspect, an embodiment of the present application proposes an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute a method for removing dark spots in an image based on multi-threshold shape measurement as described in the first aspect above.

[0022] In a fourth aspect, an embodiment of the present application proposes a computer-readable storage medium storing a computer program, which, when executed by a processor, can implement an image dark spot removal method based on multi-threshold shape measurement as described in the first aspect above.

[0023] It can be understood that the beneficial effects of the second to fourth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the related technologies, the drawings required for use in the embodiments of the present application or the related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0025] Figure 1 is a flow chart of a method for removing dark spots from an image based on multi-threshold shape measurement provided in one embodiment of the present application;

[0026] Figure 2 It is a schematic diagram of the working principle of an image dark spot removal method based on multi-threshold shape measurement provided in an embodiment of the present application;

[0027] Figure 3 is a schematic diagram of roundness provided in one embodiment of the present application;

[0028] Figure 4 is a schematic diagram of convexity provided in one embodiment of the present application;

[0029] Figure 5 is a schematic diagram of the reference area and grayscale fitting provided in one embodiment of the present application;

[0030] Figure 6 is a curve diagram of grayscale fitting of each channel provided in an embodiment of the present application;

[0031] Figure 7 is a structural schematic diagram of an image dark spot removal system based on multi-threshold shape measurement provided in another embodiment of the present application;

[0032] Figure 8 It is a structural schematic diagram of an electronic device provided in another embodiment of the present application. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the embodiments of the present application will be described in detail below in conjunction with the accompanying drawings. In the various embodiments of the present application, many technical details are proposed in order to make the reader better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical scheme claimed in the present application can also be implemented. The division of the following embodiments is only for the convenience of description, and the specific implementation mode of the present application should not constitute any limitation. The various embodiments can be combined with each other and referenced to each other under the premise of no contradiction.

[0034] The visual characteristics of stray light in fundus images have certain similarities with the common image fogging phenomenon, which can be used as a reference for removing stray light. At present, technicians mostly choose to use image enhancement technology to reduce the interference of weak or partial distortion defects, and use image repair technology to repair severely distorted areas.

[0035] In order to facilitate the explanation of the technical solution proposed in this application, the relevant contents of image enhancement and image restoration are first introduced here.

[0036] Typical image enhancement methods include: histogram equalization, adaptive histogram equalization, contrast-limited adaptive histogram equalization, Retinex method, wavelet transform method, etc. Histogram equalization mainly changes the gray level of each pixel in the image by changing the histogram of the image to enhance the contrast of images with a small dynamic range. Adaptive histogram equalization can obtain more image details by changing the local histogram. Contrast-limited adaptive histogram equalization overcomes the problem of amplified noise in adaptive histogram equalization by limiting the contrast of each small area.

[0037] The Retinex-based method uses the consistency of color perception to eliminate the influence of ambient light and enhance the reflection information of the image. The two-color reflection model is used to remove the highlight of the object.

[0038] The dehazing method based on image enhancement mainly adopts the atmospheric degradation model. Typical methods include dark channel dehazing method and dehazing method based on guided filtering. Other representative methods include but are not limited to: single image dehazing method, fast image restoration method, and Bayesian dehazing method.

[0039] The dark channel prior theory states that in a haze-free image, one color channel of each pixel always has a very low grayscale value, so the grayscale values ​​of all pixels in the entire dark primary are approximately 0. The dehazing model can be expressed as:

[0040] I(x)=J(x)t(x)+A[1-t(x)];

[0041] Among them, I(x) represents the existing image to be dehazed, J(x) represents the original haze-free image to be restored, A represents the global atmospheric optical component, and t(x) represents the perspective.

[0042] The traditional natural environment defogging method can be used as a reference, but it is based on the assumption that the fog is uniformly distributed. Therefore, image enhancement technology cannot be directly used to eliminate the unevenly distributed stray light haze phenomenon in most cases.

[0043] Commonly used image restoration techniques can be divided into two categories. The first category includes traditional diffusion-based or patch-based low-dimensional feature methods, and the second category is deep learning-based methods.

[0044] Traditional diffusion or patch-based methods usually use variational methods or patch similarity to propagate information from background areas to missing areas, such as using known areas to fill missing areas by prioritizing pixel blocks and using fast nearest neighbor methods.

[0045] In recent years, deep learning-based image restoration methods have made great progress in the field of image restoration. The initial research was to train convolutional neural networks to denoise and repair small areas. Since the context encoder uses a fully connected layer in image restoration work. On this basis, residual learning and PatchGAN were introduced to improve performance. Using the contextual attention mechanism, dilated convolution, partial convolution, and gated convolution help the network learn features better.

[0046] However, traditional image processing methods (i.e., image enhancement and image restoration) do not specifically target the processing of dark spots in fundus images, and customized dark spot removal methods need further exploration.

[0047] An embodiment of the present application proposes a method for removing dark spots from an image based on multi-threshold shape metrics, which is applied to an electronic device, wherein the electronic device may be a terminal or a server. In this embodiment and the following embodiments, the electronic device is described using a server as an example. The implementation details of the method for removing dark spots from an image based on multi-threshold shape metrics proposed in this embodiment are described in detail below. The following content is only the details provided for ease of understanding and is not necessary for the implementation of this solution.

[0048] The specific process of the image dark spot removal method based on multi-threshold shape measurement proposed in this embodiment can be as follows: Figure 1 As shown, the specific worker principle can be as follows Figure 2 As shown, including:

[0049] Step 101 : extracting the ROI region from the fundus image to be processed, and graying the extracted ROI region to obtain a grayscale image of the ROI region.

[0050] In the specific implementation, after the server obtains the fundus image to be processed, it first needs to extract the ROI area of ​​the fundus image to be processed and only retain the region of interest. This operation effectively reduces the amount of calculation and saves computing resources, while avoiding the adverse effects of the content in unimportant areas on the elimination of dark spots. Subsequently, the server grayscales the extracted ROI area to obtain a grayscale image of the ROI area, and grayscales the ROI area, reducing the difficulty of subsequent processing.

[0051] Step 102, performing multi-threshold segmentation on the grayscale image of the ROI region to obtain a plurality of binary images of the ROI region.

[0052] In the specific implementation, after obtaining the grayscale image of the ROI area, the server immediately performs multi-threshold segmentation on the grayscale image of the ROI area to obtain several binary images of the ROI area. Multi-threshold segmentation is more scientific and reasonable than single-threshold segmentation, and improves the accuracy of subsequent determination of dark spots.

[0053] In one example, when the server performs multi-threshold segmentation on the grayscale image of the ROI area to obtain several binary images of the ROI area, it is necessary to traverse the various grayscale value thresholds preset during the multi-threshold segmentation, and determine in turn whether the grayscale value of each pixel in the grayscale image is greater than the current grayscale value threshold, assign the grayscale value of the pixel in the grayscale image whose grayscale value is less than the current grayscale value threshold to 0, and assign the grayscale value of the pixel in the grayscale image whose grayscale value is greater than or equal to the current grayscale value threshold to 1, and obtain the binary image obtained by segmentation based on the current grayscale value threshold. After completing the traversal of each grayscale value threshold, several binary images of the ROI area can be finally obtained.

[0054] Step 103 , performing candidate dark spot detection on each binary image based on the dark spot shape measurement, and determining the candidate dark spots in each binary image.

[0055] In a specific implementation, after obtaining a number of binary images, the server can perform candidate dark spot detection on each binary image based on the dark spot shape metric to determine the candidate dark spots in each binary image. The dark spot shape metric includes color, area, roundness, convexity and inertia rate.

[0056] There are a series of connected areas in each binary image, and dark spots are hidden in these connected areas, but not every connected area can be considered a dark spot. Therefore, the process of candidate dark spot detection is the process of screening these connected areas. The server traverses each binary image, performs connected area detection on the current binary image, and determines whether each connected area in the current binary image satisfies the five dark spot shape measurement conditions in turn, and determines the connected areas in the current binary image that simultaneously meet the five dark spot shape measurement conditions as candidate dark spots. After completing the traversal of each binary image, the candidate dark spots in each binary image are obtained.

[0057] Among them, the five dark spot shape measurement conditions are color measurement condition, area measurement condition, roundness measurement condition, convexity measurement condition and inertia rate measurement condition. The five dark spot shape measurement conditions are introduced in detail below.

[0058] Color measurement conditions: binary images have only two spot colors, namely white spots and black spots. The colors of the spots can be distinguished by determining their grayscale values. According to the principle of binarization, only black spots can be candidate dark spots. Therefore, it is necessary to determine whether the connected areas in the binary image meet the color measurement conditions, that is, to determine whether the grayscale values ​​of the pixels in the connected areas are all 0. If the color measurement conditions are met, the grayscale values ​​of the pixels in the connected areas are all 0.

[0059] Area measurement conditions, the area of ​​the connected region cannot be called a spot if it is too large or too small, that is, only a connected region with a moderate area can be a candidate dark spot, so it is necessary to calculate the area S of the connected region, and determine whether the connected region in the binary image meets the area measurement conditions, that is, to determine whether the area S of the connected region is within the preset area range. If the area measurement conditions are met, that is, the area S of the connected region is within the preset area range, only when the area S is between the set maximum area and the minimum area, the connected region is retained as a candidate. The preset area range can be set by a technician in this field based on actual needs, and the embodiments of the present application do not specifically limit this.

[0060] Roundness measurement condition, the roundness of the connected area is represented by C, C = 4πS / p 2 , p is the perimeter of the connected region. Figure 3 As shown, when C=1, the shape of the connected area is a perfect circle, and when C=1, the connected area is a gradually elongated polygon. The closer the connected area is to a circle, the more likely it is to be a candidate dark spot. Therefore, it is determined whether the connected area in the binary image meets the roundness measurement condition, that is, it is determined whether the roundness C of the connected area is greater than the preset roundness threshold. If the roundness measurement condition is met, the roundness C of the connected area is greater than the preset roundness threshold. The preset roundness threshold can be set by a technician in this field based on actual needs, and the embodiments of the present application do not specifically limit this.

[0061] Convexity measurement condition, the convexity of the connected region is represented by V, V = S / H, H is the area of ​​the convex hull of the connected region. Figure 4 As shown, the convex hull (convex graph) means that all parts of the figure are inside the area surrounded by the tangent of the figure. The convexity can be used to well characterize the degree of convexity of the connected area. When C=1, the connected area is a square. A connected area that is too concave or too convex cannot be called a spot. In other words, only a connected area with moderate convexity can be a candidate dark spot. Therefore, it is judged whether the connected area in the binary image meets the convexity measurement condition, that is, it is judged whether the convexity V of the connected area is within the preset convexity range. If the convexity measurement condition is met, that is, the convexity V of the connected area is within the preset convexity range, the connected area is retained as a candidate only when the convexity V is between the set maximum convexity and the minimum convexity. The preset convexity range can be set by a technician in this field based on actual needs, and the embodiments of the present application do not specifically limit this.

[0062] Inertia rate measurement condition, the inertia rate is used to describe the degree of "flatness" of the contour. The inertia rate of a circle is 1, the inertia rate of an ellipse is between 0 and 1, and the inertia rate of a straight line is 0. Connected areas that are too flat cannot be called spots. Therefore, it is necessary to determine whether the connected areas in the binary image meet the inertia rate measurement condition, that is, to determine whether the inertia rate of the connected areas is greater than a preset inertia rate threshold. If the inertia rate measurement condition is met, the inertia rate of the connected areas is greater than the preset inertia rate threshold. The preset inertia rate threshold can be set by a technician in this field based on actual needs, and the embodiments of the present application do not specifically limit this.

[0063] Step 104 , voting is performed based on the candidate dark spots in each binary image to screen out the real dark spots, and based on the center point and radius of the real dark spots in each binary image, the real center point and the real radius of the real dark spots are determined.

[0064] In a specific implementation, after determining the candidate dark spots in each binary image, the server then votes based on the candidate dark spots in each binary image to screen out the real dark spots, and determines the real center point and real radius of the real dark spots based on the center point and radius of the real dark spots in each binary image.

[0065] In one example, the server votes on each binary image as a unit based on the center point of the candidate dark spot in each binary image and the principle that the center point deviation is less than a preset deviation threshold and is regarded as the same candidate dark spot. The number of times each candidate dark spot appears is counted, and the candidate dark spot whose number of appearances is greater than the preset occurrence threshold is determined as the real dark spot. The principle that the center point deviation is less than the preset deviation threshold and is regarded as the same candidate dark spot is that the candidate dark spots with close center points are regarded as the same spot.

[0066] Theoretically, a real dark spot is obtained by voting from multiple binary images. Therefore, when determining the real center point and real radius of the real dark spot, the center point and radius of the corresponding candidate dark spots in all binary images that voted for it must be considered. Based on this, the real center point and real radius of the real dark spot can be determined based on the center point and radius of the real dark spot in each binary image, which can be achieved by the following formula:

[0067]

[0068] Among them, (x i ,y i ) represents the coordinates of the center point of the real dark spot in the i-th binary image, For voting symbols, Indicates that the real dark spot has never appeared in the i-th binary image. Indicates that the real dark spot has appeared in the i-th binary image, β irepresents the inertia rate of the real dark spot in the i-th binary image, that is, the weight corresponding to each binary image is the square of its inertia rate, n represents the total number of binary images, (x t ,y t ) represents the coordinates of the real center point of the real dark spot, r i represents the radius of the real dark spot in the i-th binary image, r t Indicates the true radius of the true dark spot.

[0069] Step 105 , defining a reference area based on the real radius of the real dark spot, and eliminating the real dark spot in the fundus image to be processed based on the average gray value of the reference area.

[0070] In a specific implementation, after determining the real dark spot, the server may define a reference area based on the real radius of the real dark spot, and eliminate the real dark spot in the fundus image to be processed based on the average gray value of the reference area.

[0071] In one example, the server takes the real center point of the real dark spot as the center and the real radius r t As the radius, a circular area is delineated in the grayscale image, and the circle formed by subtracting the real dark spot from the delineated circular area is used as the reference area, such as Figure 5 Then, the average gray value of every other pixel in the four directions of east, west, south and north of the circular area is calculated, and the gray value curve L is obtained;

[0072] L=[l1,l2,…,l j ,…,l 3r ];

[0073]

[0074] j∈[0,3r t ];

[0075] Then calculate j∈[r t ,3r t All l in ] i The average value r of R is taken as the reference value, and the difference between R and j∈[0,r t All l in ] i The difference is obtained by fitting the non-smooth curve with a polynomial, and the fitted difference grayscale data is obtained. The fitted difference grayscale data is superimposed on the grayscale value curve L, and finally the result image with the real dark spots eliminated is obtained.

[0076] In one example, the grayscale fitting curve of each channel can be as follows: Figure 6 shown.

[0077] In this embodiment, the ROI area is first determined in the fundus image to be processed, and the area of ​​no interest is excluded from the calculation range, which effectively saves computing resources. Then, the extracted ROI area is grayed to obtain a gray image of the ROI area, and the gray image of the ROI area is multi-threshold segmented to obtain several binary images of the ROI area. Multi-threshold segmentation is more reasonable than single threshold segmentation, which improves the accuracy of determining dark spots. Next, based on the dark spot shape metrics including color, area, roundness, convexity and inertia rate, candidate dark spots are detected for each binary image, and candidate dark spots in each binary image are determined. Based on the dark spot shape metric, non-dark spot areas can be well eliminated. Then, based on the candidate dark spots in each binary image, voting is performed to screen out the real dark spots, and based on the center point and radius of the real dark spots in each binary image, the real center point and real radius of the real dark spots are determined. Finally, the reference area is delineated based on the real radius of the real dark spots, and the real dark spots in the fundus image to be processed are eliminated based on the average grayscale value of the reference area. In this way, the dark spots in the fundus image can be eliminated quickly, conveniently and with high quality, with less computing resources required, and can be perfectly deployed on common fundus cameras. At the same time, the color consistency between the dark spots and the surrounding areas is guaranteed after the dark spots are eliminated.

[0078] The step division of the above methods is only for clear description. When implemented, they can be combined into one step or some steps can be split and decomposed into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this application; adding insignificant modifications to the algorithm or process or introducing insignificant designs without changing the core design of the algorithm and process are all within the scope of protection of this application.

[0079] Another embodiment of the present application proposes an image dark spot removal system based on multi-threshold shape measurement. The implementation details of the image dark spot removal system based on multi-threshold shape measurement proposed in this embodiment are specifically described below. The following content is only for the convenience of understanding the implementation details provided, and is not necessary for implementing this example. Figure 7 1 is a schematic diagram of the structure of an image dark spot removal system based on multi-threshold shape measurement proposed in this embodiment, including: a preprocessing module 201, a multi-threshold segmentation module 202, a detection module 203, a voting module 204 and an elimination module 205.

[0080] The pre-processing module 201 is used to extract the ROI region from the fundus image to be processed, and grayscale the extracted ROI region to obtain a grayscale image of the ROI region.

[0081] The multi-threshold segmentation module 202 is used to perform multi-threshold segmentation on the grayscale image of the ROI region to obtain a plurality of binary images of the ROI region.

[0082] The detection module 203 is used to perform candidate dark spot detection on each binary image based on the dark spot shape metric to determine the candidate dark spots in each binary image, wherein the dark spot shape metric includes color, area, roundness, convexity and inertia rate.

[0083] The voting module 204 is used to vote based on the candidate dark spots in each binary image to screen out the real dark spots, and determine the real center point and real radius of the real dark spots based on the center point and radius of the real dark spots in each binary image.

[0084] The elimination module 205 is used to define a reference area based on the real radius of the real dark spot, and eliminate the real dark spot in the fundus image to be processed based on the average gray value of the reference area.

[0085] It is worth mentioning that all modules involved in this embodiment are logic modules. In practical applications, a logic unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. In addition, in order to highlight the innovative part of this application, this embodiment does not introduce units that are not closely related to solving the technical problems proposed by this application, but this does not mean that there are no other units in this embodiment.

[0086] It is not difficult to find that this embodiment is a system embodiment corresponding to the above method embodiment, and this embodiment can be implemented in conjunction with the above method embodiment. The relevant technical details and technical effects mentioned in the above embodiments are still valid in this embodiment, and in order to reduce repetition, they are not repeated here. Accordingly, the relevant technical details mentioned in this embodiment can also be applied in the above embodiments.

[0087] Another embodiment of the present application provides an electronic device, whose specific structure is as follows: Figure 8 As shown, it includes: at least one processor 301; and a memory 302 that is communicatively connected to the at least one processor 301; wherein the memory 302 stores instructions that can be executed by the at least one processor 301, and the instructions are executed by the at least one processor 301 so that the at least one processor 301 can execute an image dark spot removal method based on multi-threshold shape measurement described in the above embodiments.

[0088] Among them, the memory and the processor are connected in a bus manner, and the bus may include any number of interconnected buses and bridges, and the bus connects various circuits of one or more processors and memories together. The bus can also connect various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and will not be further described in this article. The bus interface is responsible for providing an interface between the bus and the transceiver. The transceiver can be one component or multiple components, such as multiple receivers and transmitters, providing a unit for communicating with various other devices on a transmission medium. The data processed by the processor is transmitted on a wireless medium through an antenna, and further, the antenna also receives data and transmits the data to the processor.

[0089] The processor is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory can be used to store data used by the processor when performing operations.

[0090] Another embodiment of the present application provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it can implement the image dark spot removal method based on multi-threshold shape measurement described in the above embodiments.

[0091] That is, those skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a program, and the program is stored in a storage medium, including a number of instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, ROM (Read-Only Memory), RAM (Random Access Memory), disk or optical disk and other media that can store program codes.

[0092] Those skilled in the art will appreciate that the above embodiments are specific embodiments for implementing the present application, and in actual applications, various changes may be made thereto in form and detail without departing from the spirit and scope of the present application.

Claims

1. A method for removing dark spots from an image based on multi-threshold shape metrics, characterized in that: include: Extracting the ROI region from the fundus image to be processed, and graying the extracted ROI region to obtain a grayscale image of the ROI region; Perform multi-threshold segmentation on the grayscale image of the ROI area to obtain several binary images of the ROI area; Based on the dark spot shape metric, candidate dark spots are detected in each binary image to determine the candidate dark spots in each binary image; wherein the dark spot shape metric includes color, area, roundness, convexity and inertia rate; Voting is performed based on the candidate dark spots in each binary image to screen out the real dark spots, and based on the center point and radius of the real dark spots in each binary image, the real center point and the real radius of the real dark spots are determined; A reference area is defined based on the real radius of the real dark spot, and the real dark spot in the fundus image to be processed is eliminated based on the average gray value of the reference area.

2. The image dark spot removal method based on multi-threshold shape measurement according to claim 1 is characterized in that: The grayscale image of the ROI region is subjected to multi-threshold segmentation to obtain a plurality of binary images of the ROI region, including: Traverse each gray value threshold preset during multi-threshold segmentation, and determine in turn whether the gray value of each pixel in the gray image is greater than the current gray value threshold; The grayscale values ​​of the pixels in the grayscale image whose grayscale values ​​are less than the current grayscale value threshold are assigned to 0, and the grayscale values ​​of the pixels in the grayscale image whose grayscale values ​​are greater than or equal to the current grayscale value threshold are assigned to 1, thereby obtaining a binary image segmented based on the current grayscale value threshold; After completing the traversal of each gray value threshold, several binary images of the ROI area are obtained.

3. The image dark spot removal method based on multi-threshold shape measurement according to claim 2 is characterized in that: The method of performing candidate dark spot detection on each binary image based on the dark spot shape measurement to determine the candidate dark spots in each binary image includes: Traverse each binary image, perform connected area detection on the current binary image, and determine in turn whether each connected area in the current binary image meets the five dark spot shape measurement conditions; Determine the connected area in the current binary image that satisfies five dark spot shape measurement conditions as a candidate dark spot; After completing the traversal of each binary image, the candidate dark spots in each binary image are obtained; Among them, the five dark spot shape measurement conditions are color measurement condition, area measurement condition, roundness measurement condition, convexity measurement condition and inertia rate measurement condition.

4. The method for removing dark spots from an image based on multi-threshold shape metrics according to claim 3, characterized in that: The color measurement condition is specifically that the grayscale values ​​of the pixels in the connected area are all 0; The area measurement condition is specifically that the area S of the connected region is within a preset area range; The circularity measurement condition is that the circularity C of the connected area is greater than the preset circularity threshold, C = 4πS / p 2 , p is the perimeter of the connected region; The convexity measurement condition is specifically that the convexity V of the connected region is within a preset convexity range, V = S / H, H is the area of ​​the convex hull of the connected region; The inertia rate measurement condition is specifically satisfied when the inertia rate of the connected area is greater than a preset inertia rate threshold.

5. The image dark spot removal method based on multi-threshold shape measurement according to any one of claims 1 to 4, characterized in that: The method of voting for candidate dark spots in each binary image to screen out real dark spots includes: Based on the center point of the candidate dark spot in each binary image, combined with the principle that the center point deviation is less than the preset deviation threshold and is considered to be the same candidate dark spot, voting is performed in units of each binary image to count the number of times each candidate dark spot appears; The candidate dark spots whose occurrence times are greater than a preset occurrence threshold are determined as real dark spots.

6. The method for removing dark spots from an image based on multi-threshold shape metrics according to claim 5, characterized in that: The real center point and real radius of the real dark spot are determined based on the center point and radius of the real dark spot in each binary image by the following formula: Among them, (x i ,y i ) represents the coordinates of the center point of the real dark spot in the i-th binary image, For voting symbols, Indicates that the real dark spot has never appeared in the i-th binary image. Indicates that the real dark spot has appeared in the i-th binary image, β i represents the inertia rate of the real dark spot in the i-th binary image, n represents the total number of binary images, (x t ,y t ) represents the coordinates of the real center point of the real dark spot, r i represents the radius of the real dark spot in the i-th binary image, r t Indicates the true radius of the true dark spot.

7. The method for removing dark spots from an image based on multi-threshold shape metrics according to any one of claims 1 to 4, characterized in that: The method of defining a reference area based on the real radius of the real dark spot and eliminating the real dark spot in the fundus image to be processed based on the average gray value of the reference area includes: With the real center point of the real dark spot as the center and 3 times the real radius as the radius, a circular area is delineated in the grayscale image, and the circle formed by subtracting the real dark spot from the delineated circular area is used as the reference area; Calculate the average gray value of every other pixel in the four directions of east, west, south and north in the circular area starting from the true center point to obtain the gray value curve L; L＝[l1,l2,…,l j ,…,l 3r ]； j∈[0,3r t ]； Calculate j∈[r t ,3r t All l in ] i The average value R, taking R as the reference value, calculates R and j∈[0,r t All l in ] i The difference between , and we get a non-smooth curve; The non-smooth curve is fitted with a polynomial to obtain the fitted difference grayscale data, which is then superimposed on the grayscale value curve L to finally obtain a result image with the real dark spots eliminated.

8. An image dark spot removal system based on multi-threshold shape measurement, characterized in that: include: The preprocessing module is used to extract the ROI region of the fundus image to be processed, and grayscale the extracted ROI region to obtain a grayscale image of the ROI region; A multi-threshold segmentation module is used to perform multi-threshold segmentation on the grayscale image of the ROI area to obtain several binary images of the ROI area; A detection module, used for performing candidate dark spot detection on each binary image based on dark spot shape measurement to determine the candidate dark spots in each binary image, wherein the dark spot shape measurement includes color, area, roundness, convexity and inertia rate; A voting module is used to vote based on the candidate dark spots in each binary image to screen out the real dark spots, and determine the real center point and real radius of the real dark spots based on the center point and radius of the real dark spots in each binary image; The elimination module is used to define a reference area based on the real radius of the real dark spot, and eliminate the real dark spot in the fundus image to be processed based on the average gray value of the reference area.

9. An electronic device, characterized in that: include: at least one processor; and, a memory communicatively coupled to the at least one processor; The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the image dark spot removal method based on multi-threshold shape measurement as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the image dark spot removal method based on multi-threshold shape measurement as claimed in any one of claims 1 to 7 can be implemented.

Images