A method and system for detecting photoreceptors in fundus images

Abstract

The present invention discloses a method and system for detecting photoreceptors in fundus images, which relates to the technical field of medical image processing. The method uses a chained filtering algorithm to process the fundus image to obtain a filtered image, and determines the type of each pixel point in the filtered image based on an adaptive threshold. The types include photoreceptor pixel points and background pixel points, so as to be able to realize the automatic detection of photoreceptors in fundus images and improve the detection efficiency and detection accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and particularly to a method and system for detecting photoreceptors in fundus images. Background Art

[0002] The human eye is an extremely complex and precise human organ. As the most important part of the visual optical path of the human eye, the integrity of the function and structure of the retina is the guarantee of good visual function. Systemic diseases (such as hypertension, diabetes) and eye diseases (such as retinal arteriovenous occlusion) are manifested in the retina, making retinal examination very important in the process of disease diagnosis and treatment. The retinal arteriovenous is the only vascular structure that can be directly observed in the human body. Its display of the fundus conditions can not only help clinicians observe and judge eye diseases, but also indirectly show the manifestations of various systems of the whole body. Fundus images are currently the only in-vivo vascular images that can directly show the internal structure of the human eye retina by direct shooting, and can image various structures such as the optic disc, blood vessels, macula, nerve fibers, and photoreceptor cells. It provides an important window for clinicians to observe the retina. If you want to check the retina, you need to take fundus images. In the past, when reading fundus images, it was done manually by doctors, with low efficiency and heavy workload. Especially when it comes to annotating photoreceptors in fundus images, it is extremely time-consuming and laborious, and the accuracy is also very poor.

[0003] Based on this, there is an urgent need for a technology that can automatically detect photoreceptors in fundus images. Summary of the Invention

[0004] The purpose of the present invention is to provide a method and system for detecting photoreceptors in fundus images, which can realize the automatic detection of photoreceptors in fundus images and improve the detection efficiency and accuracy.

[0005] To achieve the above purpose, the present invention provides the following solutions:

[0006] A method for detecting photoreceptors in fundus images, the detection method comprising:

[0007] Obtain a fundus image;

[0008] Process the fundus image by using a chain filtering algorithm to obtain a filtered image;

[0009] Determine the type of each pixel point of the filtered image based on an adaptive threshold; the type includes photoreceptor pixel points and background pixel points.

[0010] In some embodiments, before processing the fundus image using the chain filtering algorithm, the detection method further includes: interpolating and magnifying the fundus image using the bicubic interpolation algorithm to obtain a preprocessed image, and using the preprocessed image as the new fundus image.

[0011] In some embodiments, processing the fundus image using the chain filtering algorithm to obtain a filtered image specifically includes:

[0012] For each pixel point in the fundus image, using the pixel point as a starting point, performing chain growth in multiple preset directions respectively to obtain a chain corresponding to each preset direction; for each chain, determining the minimum value of the gray values of all pixel points included in the chain, and calculating the difference between the gray value of the pixel point and the minimum value to obtain a gray difference corresponding to each chain; calculating the filtered gray value of the pixel point based on the gray differences corresponding to all chains to obtain a filtered image.

[0013] In some embodiments, using the pixel point as a starting point and performing chain growth in multiple preset directions respectively to obtain a chain corresponding to each preset direction specifically includes:

[0014] For each preset direction, perform the following steps:

[0015] Using the pixel point as a starting point;

[0016] Determining a candidate pixel set according to the starting point and the search direction of the current iteration; based on the positions and gray values of all candidate pixel points in the candidate pixel set, selecting a candidate pixel point as the target pixel point; connecting the starting point and the target pixel point, and performing chain growth on the intermediate chain of the previous iteration to obtain the intermediate chain of the current iteration;

[0017] Judging whether the number of pixel points included in the intermediate chain of the current iteration reaches a preset number;

[0018] If so, end the iteration, and use the intermediate chain of the current iteration as the chain corresponding to the preset direction;

[0019] If not, calculate the growth direction of the current iteration according to the position of the starting point and the position of the target pixel point, and calculate the search direction of the next iteration according to the growth direction of the current iteration; using the target pixel point as the starting point of the next iteration, return to the step of "determining a candidate pixel set according to the starting point and the search direction of the current iteration".

[0020] In some embodiments, the calculation formula of the search direction is:

[0021]

[0022] Wherein, Ds[i] is the search direction of the i-th iteration; Ds[i-1] is the search direction of the (i-1)-th iteration; x is an intermediate parameter; m is a sensitivity coefficient; Dg[i-1] is the growth direction of the (i-1)-th iteration.

[0023] In some embodiments, the calculation formula of the filtered gray value is:

[0024]

[0025] Wherein, R(P) is the filtered gray value of pixel point P; min{MAD j} is the minimum value of the gray value differences corresponding to all chains; MAD j is the gray value difference corresponding to the j-th chain; k is an amplification factor.

[0026] In some embodiments, determining the type of each pixel point of the filtered image based on the adaptive threshold specifically includes:

[0027] For each pixel point of the filtered image, compare the gray value of the pixel point with the adaptive threshold; if the gray value is greater than or equal to the adaptive threshold, then the pixel point is a photoreceptor pixel point; otherwise, the pixel point is a background pixel point.

[0028] In some embodiments, the calculation formula of the adaptive threshold is:

[0029] T = μ + a × σ;

[0030] Wherein, T is the adaptive threshold; μ is the average value of the gray values of all pixel points of the filtered image; a is a relative decision threshold; σ is the standard deviation of the gray values of all pixel points of the filtered image.

[0031] In some embodiments, after determining the type of each pixel point, the detection method further includes:

[0032] Use the region growing algorithm to count the photoreceptor pixel points of the filtered image to obtain the number of photoreceptors and the position of each photoreceptor; the photoreceptor includes a plurality of adjacent photoreceptor pixel points;

[0033] Calculate the density of the photoreceptors according to the number of photoreceptors;

[0034] Draw a Voronoi diagram based on the positions of all the photoreceptors; determine the number of Voronoi diagram neighbors of each photoreceptor according to the Voronoi diagram.

[0035] A fundus image photoreceptor detection system, the detection system includes:

[0036] An acquisition module for acquiring fundus images;

[0037] A processing module for processing the fundus image using a chained filtering algorithm to obtain a filtered image;

[0038] A detection module for determining the type of each pixel point in the filtered image based on an adaptive threshold; the types include photoreceptor pixel points and background pixel points.

[0039] According to the specific embodiments provided by the present invention, the following technical effects are disclosed:

[0040] The present invention provides a method and system for detecting photoreceptors in fundus images, which processes the fundus image using a chained filtering algorithm to obtain a filtered image, and determines the type of each pixel point in the filtered image based on an adaptive threshold. The types include photoreceptor pixel points and background pixel points, thereby enabling automatic detection of photoreceptors in fundus images and improving detection efficiency and accuracy. Description of the Drawings

[0041] To more clearly illustrate the technical solutions in the embodiments of the present invention or in the prior art, the following will briefly introduce the drawings required for use in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention, and those of ordinary skill in the art can also obtain other drawings based on these drawings without creative efforts.

[0042] Figure 1 A schematic diagram of the fundus image provided by the present invention;

[0043] Figure 2 A labeled schematic diagram of the fundus image provided by the present invention;

[0044] Figure 3 A flowchart of the detection method provided in Embodiment 1 of the present invention;

[0045] Figure 4 A principle block diagram of the detection method provided in Embodiment 1 of the present invention;

[0046] Figure 5 A schematic diagram of the preprocessed image provided in Embodiment 1 of the present invention;

[0047] Figure 6 A schematic diagram of the preset direction provided in Embodiment 1 of the present invention;

[0048] Figure 7 A schematic diagram of the candidate pixel set provided in Embodiment 1 of the present invention;

[0049] Figure 8 Schematic flowchart of the chain filtering algorithm provided in Embodiment 1 of the present invention;

[0050] Figure 9 Schematic diagram of the annotation of photoreceptor pixel points provided in Embodiment 1 of the present invention;

[0051] Figure 10 Schematic diagram of the Voronoi diagram provided in Embodiment 1 of the present invention;

[0052] Figure 11 Schematic diagram of the comparison between the original fundus image and the processed image provided in Embodiment 1 of the present invention;

[0053] Figure 12 System block diagram of a detection system provided in Embodiment 2 of the present invention;

[0054] Figure 13 Another system block diagram of the detection system provided in Embodiment 2 of the present invention. Detailed implementation manners

[0055] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0056] The purpose of the present invention is to provide a method and system for detecting photoreceptors in fundus images, which can realize the automatic detection of photoreceptors in fundus images and improve the detection efficiency and accuracy.

[0057] To make the above objects, features, and advantages of the present invention more obvious and understandable, the present invention will be further described in detail below in conjunction with the accompanying drawings and specific implementation manners.

[0058] Photoreceptors are a type of visual cells, including cone cells and rod cells, and their function is to receive light stimuli and convert light energy into nerve impulses. In the fovea centralis of the retina, only photoreceptors exist, which allows light to directly reach the photoreceptors, so the light-sensing and color-discriminating abilities are the most sensitive here. Photoreceptors are an important feature of fundus images, and their quantity, density, and distribution characteristics are important indicators for diagnosing various diseases. An especially important indicator in fundus images is the photoreceptor quantity / density. Figure 1 is an 8° fundus image collected clinically using a confocal fundus microscope, Figure 1 and the white bright spots in it are photoreceptors.

[0059] In the past, when reading fundus images, it was done manually by doctors, which was inefficient and involved a large workload. Especially when it came to the annotation and counting of photoreceptors, it was extremely time-consuming and laborious, and the accuracy was also very poor. For example Figure 2 (a) is a partial area intercepted from Figure 1 and Figure 2 (b) is the result of manual annotation, and it can be seen that the annotation workload is very large. Moreover, the number and density of photoreceptors are relatively important indicators, which can be used for the clinical diagnosis and early screening of various eye diseases (such as retinal arteriovenous occlusion, etc.) and systemic diseases (such as hypertension and diabetes, etc.). How to speed up the entire detection process, reduce the workload of researchers and medical staff, and thus obtain the index data for clinical diagnosis and disease screening faster and better is an urgent problem to be solved at present.

[0060] Based on this, the purpose of the present invention is to design a computer vision algorithm according to this usage scenario, automatically realize the detection and counting of photoreceptors, and initially generate some diagnostic indicators, so as to reduce the workload of medical staff in reading films during clinical medical treatment and in image annotation during scientific research work, and obtain the index data better and faster. Therefore, the present invention specifically relates to a technology for processing fundus images and detecting photoreceptors using a chain filtering algorithm for confocal fundus images.

[0061] Embodiment 1:

[0062] This embodiment is used to provide a method for detecting photoreceptors in fundus images. As shown in Figure 3 and Figure 4 , the detection method includes:

[0063] S1: Obtain a fundus image;

[0064] The fundus image in this embodiment can be obtained by shooting with an image acquisition device such as a confocal fundus microscope.

[0065] S2: Process the fundus image using a chain filtering algorithm to obtain a filtered image;

[0066] Due to factors such as the limitations of the image acquisition device, the original fundus image obtained in S1 itself has the problem of insufficient clarity, which may mask the morphology and quantity information of photoreceptors. Therefore, it is necessary to use a preprocessing method of image interpolation and magnification to increase the clarity of the fundus image for subsequent processing. Therefore, before S2, this embodiment can also perform preprocessing operations such as interpolation and magnification on the fundus image. Specifically, use the bicubic interpolation algorithm to interpolate and magnify the fundus image to obtain a preprocessed image, and use the preprocessed image as the new fundus image to execute S2. Figure 5 That is, the preprocessed image obtained by interpolating and magnifying the fundus image shown in Figure 2 (a).

[0067] S3: Determine the type of each pixel point in the filtered image based on an adaptive threshold; the types include photoreceptor pixel points and background pixel points.

[0068] In fundus images, photoreceptors appear as brighter light spots compared to the surrounding background. In this embodiment, this characteristic is utilized to perform image processing on the fundus image using a chain filtering algorithm (also known as a chain growth filtering algorithm). The idea of the chain filtering algorithm is as follows: Process each pixel point of the fundus image in sequence. From multiple preset directions of the current pixel point, extend a chain that grows according to a rule to obtain information about the current pixel point and its surrounding pixel points. Compare the gray value of the current pixel point with the gray values of its surrounding pixel points. If the comparison result shows that the current pixel point is significantly brighter (i.e., has a higher gray value), it is considered to be part of a photoreceptor, and its brightness is enhanced (i.e., the gray value is increased), so that the pixel points belonging to the photoreceptor become bright points. Conversely, if the current pixel point is considered to be part of the background, its brightness is suppressed (i.e., the gray value is decreased). Specifically, using the chain filtering algorithm to process the fundus image, the obtained filtered image may include: For each pixel point in the fundus image, using this pixel point as the starting point, perform chain growth in multiple preset directions to obtain the chain corresponding to each preset direction; for each chain, determine the minimum value of the gray values of all pixel points included in the chain, and calculate the difference between the gray value of this pixel point and the minimum value to obtain the gray difference corresponding to each chain; calculate the filtered gray value of this pixel point based on the gray differences corresponding to all chains to obtain the filtered image.

[0069] To enable those skilled in the art to better understand the chain filtering algorithm described in this embodiment, here, the terms involved in the chain filtering algorithm are introduced first:

[0070] A chain is a set composed of multiple pixel points that follows a specific region growth rule. A chain has a chain head and a chain tail. At the initial moment, a chain has only one pixel point, and at this time, this pixel point is both the chain head and the chain tail. During the chain growth process, in each iteration, only the pixel point adjacent to the pixel point serving as the chain tail (subsequently referred to as the chain tail pixel point) can be added to the chain and serve as the new chain tail.

[0071] The preset directions can be 8, as Figure 6 shown, Figure 6 the numbers in Figure 6 represent the directions. Directions 0, 1, 2, 3, 4, 5, 6, and 7 respectively represent above, upper right, right, lower right, below, lower left, left, and upper left of the pixel point (i.e., the gray grid in

[0072] During the growth process of the chain, there are a search direction and a growth direction. Each of the search direction and the growth direction also has 8 directions, which are the same as the 8 directions of the above-mentioned preset directions. That is, the search direction and the growth direction also include Figure 6 the directions 0, 1, 2, 3, 4, 5, 6, and 7 shown, which respectively represent above, upper right, right, lower right, below, lower left, left, and upper left of the pixel point.

[0073] The search direction is used to determine the set of candidate pixels to be added to the chain. The set of candidate pixels starts from the chain-end pixel point ( Figure 7 the light gray grid) and consists of three adjacent pixel points ( Figure 7 the dark gray grids in) in the search direction. The pixel points to be added to the chain during the growth process will be generated from the set of candidate pixels. As Figure 7 shown, it is the set of candidate pixels corresponding to each of the 8 search directions. Figure 7 (a) is the set of candidate pixels corresponding to direction 0. Figure 7 (b) is the set of candidate pixels corresponding to direction 1. Figure 7 (c) is the set of candidate pixels corresponding to direction 2. Figure 7 (d) is the set of candidate pixels corresponding to direction 3. Figure 7 (h) is the set of candidate pixels corresponding to direction 4. Figure 7 (g) is the set of candidate pixels corresponding to direction 5. Figure 7 (f) is the set of candidate pixels corresponding to direction 6. Figure 7 (e) is the set of candidate pixels corresponding to direction 7.

[0074] The growth direction is used to represent the orientation of the pixel point added to the chain relative to the current chain-end pixel point.

[0075] Here, based on the above term introduction, the chain growth process of the chain filtering algorithm in this embodiment is further introduced. As Figure 8 shown, starting from a pixel point, chain growth is performed in multiple preset directions respectively. The chains obtained for each preset direction may include: for each preset direction, the following steps are performed to obtain the chain corresponding to each preset direction.

[0076] (1) Take a pixel point as the starting point.

[0077] (2) Determine the set of candidate pixels according to the starting point and the search direction of the current iteration; based on the positions and gray values of all candidate pixel points in the set of candidate pixels, select a candidate pixel point as the target pixel point; connect the starting point and the target pixel point, and perform chain growth on the intermediate chain of the previous iteration to obtain the intermediate chain of the current iteration.

[0078] Among them, the search direction of the first iteration can be given manually. According to Figure 7 , the candidate pixel set can be determined based on the starting point and the search direction of the current iteration. It should be noted that the candidate pixel points in the candidate pixel set are all pixel points in the fundus image. The process of selecting the target pixel point may include: taking the candidate pixel point with the largest gray value in the candidate pixel set as the target pixel point. If there are multiple candidate pixel points with the largest gray value, select the leftmost one as the target pixel point. It should be noted that the left side here is defined according to the search direction of the current iteration. Since the starting point of the current iteration is the tail pixel point of the middle chain of the previous iteration, connecting the starting point and the target pixel point is equivalent to connecting the tail pixel point of the middle chain of the previous iteration and the target pixel point. At this time, it is equivalent to performing one-step chain growth on the middle chain of the previous iteration, and the middle chain of the current iteration can be obtained.

[0079] (3) Determine whether the number of pixel points (i.e., the chain length) included in the middle chain of the current iteration reaches a preset number.

[0080] The chain length in this embodiment can be manually defined according to the size of the photoreceptor in the specific scenario. In this embodiment, the chain length is preferably set to 8.

[0081] (4) If so, end the iteration, and use the middle chain of the current iteration as the chain corresponding to the preset direction.

[0082] (5) If not, calculate the growth direction of the current iteration according to the position of the starting point and the position of the target pixel point, and calculate the search direction of the next iteration according to the growth direction of the current iteration; use the target pixel point as the starting point of the next iteration, and return to the step of "determining the candidate pixel set according to the starting point and the search direction of the current iteration".

[0083] In this embodiment, compare the orientation of the position of the target pixel point relative to the position of the starting point. According to Figure 6 the growth direction of the current iteration can be determined.

[0084] In order to automatically adapt to the complex background environment in the fundus image, the chain growth process has an adaptive ability. During the growth process, it can gradually and automatically select the search direction to ensure comprehensive collection of information in the surrounding area for the judgment of the current pixel point. Therefore, in order to make the growth direction of the chain adapt to the complex background information, in the process of chain growth, the following formula is used in this embodiment to determine the search direction of the i-th iteration in the chain growth process:

[0085]

[0086] Among them, Ds[i] is the search direction of the i-th iteration; Ds[i - 1] is the search direction of the (i - 1)-th iteration; x is an intermediate parameter; m is the sensitivity coefficient, that is, the sensitivity coefficient of the chain growth turning speed. In this embodiment, the speed of turning during chain growth can be adjusted by modifying m. To balance the flexibility and ductility of the chain, m is preferably set to 3 in this embodiment; Dg[i - 1] is the growth direction of the (i - 1)-th iteration. Based on the above formula, in this embodiment, the search direction of the next iteration can be calculated according to the growth direction of the current iteration.

[0087] For each pixel point P of the fundus image, the above steps can be used to obtain 8 chains extending from the pixel point P in 8 preset directions respectively to the surrounding. For each chain, determine the minimum value of the gray values of all pixel points included in the chain, and calculate the difference between the gray value of the pixel point P and the minimum value to obtain the gray difference corresponding to each chain.

[0088] Specifically, the formula for calculating the gray difference is:

[0089] Q ∈ C j , j = 0, 1, 2, 3, 4, 5, 6, 7;

[0090] MAD j = G[P] - min(G[Q]);

[0091] Among them, Q is the pixel point in the j-th chain; C j is the chain corresponding to the j-th preset direction, that is, the j-th chain; MAD j is the gray difference corresponding to the j-th chain; G[P] is the gray value of the pixel point P; min(G[Q]) is the minimum value of the gray values of the j-th chain.

[0092] After obtaining the gray difference corresponding to each chain, calculate the filtered gray value of the pixel point P based on the gray differences corresponding to all chains to obtain the filtered image. The formula for calculating the filtered gray value is:

[0093]

[0094] Among them, R(P) is the filtered gray value of the pixel point P; min{MAD j} is the minimum value of the gray differences corresponding to all chains; MAD j is the gray difference corresponding to the j-th chain; k is the amplification coefficient. To make the filtering result more intuitive, k is taken as [25, 50] in this embodiment.

[0095] When there are 8 preset directions, the formula for calculating the filtered gray value is:

[0096]

[0097] Through the above process, growth filtering can be performed on each pixel point in the fundus image to obtain the filtered gray value of each pixel point. The photoreceptor information can be highlighted and the background information can be suppressed through the above formula. All pixel points in the fundus image take the corresponding filtered gray value, and at this time, the fundus image is converted into a filtered image.

[0098] In this embodiment, determining the type of each pixel point of the filtered image based on the adaptive threshold may include: for each pixel point of the filtered image, comparing the gray value of the pixel point with the adaptive threshold; if the gray value is greater than or equal to the adaptive threshold, then the pixel point is a photoreceptor pixel point and belongs to a part of the photoreceptor; otherwise, the pixel point is a background pixel point. Finally, all pixel points in the filtered image with a gray value greater than or equal to T are regarded as pixel points constituting the target area (i.e., the photoreceptor).

[0099] Among them, the calculation formula of the adaptive threshold is:

[0100] T = μ + a × σ;

[0101] Among them, T is the adaptive threshold; μ is the average value of the gray values of all pixel points of the filtered image; a is the relative decision threshold, usually taking 2 to 3; σ is the standard deviation of the gray values of all pixel points of the filtered image.

[0102] Through the above calculation formula of the adaptive threshold, this embodiment can use the adaptive threshold method to dynamically detect the target area (i.e., the area including photoreceptor pixel points) and the background area (i.e., the area including background pixel points). As Figure 9 shown, it is the target area and the background area determined based on the adaptive threshold, Figure 9 and the white area where the black dots are located in it is the target area.

[0103] After determining the type of each pixel point, the detection method of this embodiment further includes calculating the number of photoreceptors in the area based on the filtered image obtained after chain filtering, and calculating a series of parameters and indicators accordingly. Specifically, this step may include:

[0104] (1) Using the region growing algorithm to count the photoreceptor pixel points of the filtered image to obtain the number of photoreceptors and the position of each photoreceptor. The photoreceptor includes multiple adjacent photoreceptor pixel points.

[0105] In this embodiment, the specific points (i.e., photoreceptors) in the filtered image are identified, and information such as their coordinates (positions) and quantities is recorded to obtain the quantity of photoreceptors and the position of each photoreceptor. Considering that a photoreceptor often includes multiple photoreceptor pixel points, the region growing algorithm is used for counting in this embodiment, which specifically includes: creating a matrix M, where a pixel point in the filtered image corresponds to an element in matrix M, and setting all elements in M to 0; setting a photoreceptor quantity counter (Number Counter) and initializing the value of Number Counter to 0; traversing all pixel points in the filtered image in sequence. In this embodiment, the filtered image can be traversed in the order from left to right and from top to bottom. For pixel point P(x, y), the following operations are performed:

[0106] 1) If the grayscale value of pixel point P(x, y) ≥ T and the value of the element M(x, y) corresponding to P(x, y) in matrix M is 0, jump to 2); otherwise, jump to 3).

[0107] 2) Set M(x, y) to 1, set the photoreceptor area counter (Area Counter) to 1, determine all adjacent neighboring pixel points within the 3*3 region centered on P(x, y) whose corresponding elements in M have a value of 0. For each neighboring pixel point, set the value of the corresponding element in M for that neighboring pixel point to 1, and determine whether the grayscale value of the neighboring pixel point is ≥ T. If so, increment Area Counter by 1, and use this neighboring pixel point as the next iterative P(x, y), then return to the step of "determining all adjacent neighboring pixel points within the 3*3 region centered on P(x, y) whose corresponding elements in M have a value of 0". If not, stop the iteration, jump to 4), and the current value of Area Counter can be used to obtain the quantity of all photoreceptor pixel points that jointly form a target region with P(x, y). The quantity of all photoreceptor pixel points included in this target region is the area of this target region.

[0108] 3) Set M(x, y) to 1 and traverse the next pixel point in the traversal order.

[0109] 4) Increment Number Counter by one and record the coordinate position (x, y) of pixel point P(x, y).

[0110] After the traversal is completed, the current value of Number Counter is the quantity of photoreceptors, the coordinate position (x, y) of each pixel point P(x, y) is the position of the photoreceptor to which P(x, y) belongs, and the current value of Area Counter corresponding to each pixel point P(x, y) in 2) is the area of the photoreceptor to which P(x, y) belongs.

[0111] Preferably, the target region composed of multiple photoreceptor pixel points is identified by using the region growing algorithm, that is, after each photoreceptor is identified, the photoreceptors with too small area are removed, and the photoreceptors with too small area are regarded as the background region. Whether the area is too small can be achieved by manually setting a preset value. If the area is less than the preset value, it is considered that the area is too small.

[0112] (2) Calculate the density of the photoreceptors according to the number of the photoreceptors.

[0113] In this embodiment, the number of the photoreceptors determined in (1) and the position coordinate information of each photoreceptor can be further used to calculate medical indicators for assisting clinical diagnosis, such as calculating indicators such as the density of the photoreceptors and the number of neighbors of the photoreceptors.

[0114] In this embodiment, the density of the photoreceptors is calculated according to the following formula:

[0115] D = N / S;

[0116] Wherein, D is the density of the photoreceptors; N is the number of the photoreceptors; S is the total number of pixel points in the filtered image.

[0117] (3) Draw a Voronoi diagram based on the positions of all the photoreceptors; determine the number of Voronoi diagram neighbors of each photoreceptor according to the Voronoi diagram.

[0118] The steps for calculating the number of Voronoi diagram neighbors are specifically as follows:

[0119] 1) Generate a Delaunay triangulation

[0120] 1.1) Take any coordinate point as the starting point, find the coordinate point closest to the starting point, and connect the starting point and the coordinate point closest to the starting point to form a side of the Delaunay triangle, that is, the baseline. The coordinate points in this embodiment refer to the photoreceptors, and the distance can be calculated according to the positions of the photoreceptors.

[0121] 1.2) According to the discrimination rules of the Delaunay triangulation (i.e., the two basic properties of the Delaunay triangulation), find the third coordinate point that forms a Delaunay triangle with the baseline.

[0122] 1.3) Connect both endpoints of the baseline to the third coordinate point to become a new baseline, and obtain a Delaunay triangle.

[0123] 1.4) Iterate the above steps, that is, execute the above steps for each coordinate point until all baselines are processed to generate a Delaunay triangulation.

[0124] 2) Generate a Voronoi diagram

[0125] 2.1) Traverse the set of Delaunay triangles (i.e., the Delaunay triangulation network), and record which three coordinate points each triangle is composed of.

[0126] 2.2) Calculate the circumcenter of each Delaunay triangle and record it.

[0127] 2.3) Traverse the set of Delaunay triangles to find the adjacent Delaunay triangles ta, tb, and tc that share sides with the three sides of the current Delaunay triangle t.

[0128] 2.4) If adjacent Delaunay triangles corresponding to the sides are found, connect the circumcenters of the found adjacent Delaunay triangles to the circumcenter of the current Delaunay triangle t and store them in the Voronoi edge set. If not found, find the intersection point of the perpendicular bisector ray corresponding to the side and the edge, and store the connection between the circumcenter of the current Delaunay triangle t and this intersection point in the Voronoi edge set.

[0129] Based on the Voronoi edge set, a Voronoi diagram can be generated, as Figure 10 shown.

[0130] 3) According to the generated Voronoi diagram, obtain the number of Voronoi diagram neighbors of each coordinate point.

[0131] After calculating the metrics, this embodiment can further display and store the processing results. To more intuitively compare the changes before and after processing, convert the grayscale images of the original fundus image and the processed image obtained after processing using S1 - S3 into 3D stereoscopic images, and display and save them together with various metric data such as the number, position, density, and number of Voronoi diagram neighbors of the photoreceptors calculated in the above process. As Figure 11 shown,[[]] Figure 11 (a) is the original fundus image, Figure 11 (b) is the 3D stereoscopic image corresponding to the original fundus image, Figure 11 (c) is the processed image, Figure 11 (d) is the 3D stereoscopic image corresponding to the processed image. Obviously, the photoreceptors in the original fundus image are not prominent, and the photoreceptors in the 3D stereoscopic image corresponding to the original fundus image cannot be effectively distinguished from the background. However, the photoreceptors (i.e., the white bright spots in the figure) in the processed image are very prominent, and the photoreceptors in the 3D stereoscopic image corresponding to the processed image are also effectively distinguished from the background.

[0132] To overcome the problems existing in the processing of fundus images in the prior art, this embodiment provides a method for detecting photoreceptors in fundus images based on a chain filtering algorithm. First, the fundus image is interpolated and enlarged to optimize the image quality and create more space for subsequent processing. Then, chain filtering is performed to highlight the specific points that may belong to photoreceptors and suppress the irrelevant background. Next, an adaptive threshold and region growing algorithm are used to identify the specific points in the filtered image and record information such as their coordinates and quantities. Then, based on the obtained information such as coordinates and quantities, indicators such as the density of photoreceptors and the number of Voronoi diagram neighbors are calculated. Finally, the processing results are displayed and stored, so as to be able to automatically process the fundus images obtained by a confocal fundus microscope, automatically detect the photoreceptors in the fundus images, and simultaneously generate multiple indicators related to photoreceptors to assist clinical diagnosis and research. The detection method of this embodiment can automatically detect photoreceptors, count the photoreceptors, automatically calculate relevant indicators such as the number and density of photoreceptors, without manual statistical work, realizing the automation of photoreceptor detection and statistics, automatically generating diagnostic indicators and results, liberating a large amount of time and energy required for manual photoreceptor detection and counting, and ensuring the accuracy of photoreceptor detection and counting.

[0133] Embodiment 2:

[0134] This embodiment is used to provide a system for detecting photoreceptors in fundus images. As Figure 12 shown, the detection system includes:

[0135] An acquisition module M1, configured to acquire fundus images;

[0136] A processing module M2, configured to process the fundus image by using a chain filtering algorithm to obtain a filtered image;

[0137] A detection module M3, configured to determine the type of each pixel point in the filtered image based on an adaptive threshold; the types include photoreceptor pixel points and background pixel points.

[0138] Of course, this embodiment can also divide the functional modules of the detection system in other ways. As Figure 13 shown, the detection system of this embodiment may further include:

[0139] (1) An image preprocessing module for optimizing the image quality.

[0140] The clarity and size of the fundus image itself are difficult to meet the requirements of photoreceptor detection. As a preference, this embodiment uses a bicubic interpolation algorithm for interpolating and enlarging the fundus image, which can make the enlarged image smoother and the enlargement result more accurate.

[0141] (2) A chain filtering module for performing image filtering processing.

[0142] This module will process each pixel point in the fundus image in sequence, extending a chain that grows according to rules from 8 directions of the current pixel point to obtain information about the current pixel point and its surrounding environment. If it is found that the gray value of the current pixel point is higher than that of the surrounding environment, the current pixel point is considered to belong to the target area, and its brightness should be retained or even appropriately enhanced; if it is found that the gray value of the current pixel point is the same as or even lower than that of the surrounding environment, it is considered to belong to the background area, and its brightness is suppressed, that is, its gray value is reduced.

[0143] (3) A photoreceptor counting module for counting the number and coordinates of photoreceptors.

[0144] This module will, based on the chain filtering module, detect photoreceptors in the filtered image and record their coordinate and quantity information.

[0145] (4) A result display and storage module for displaying and storing the processing results.

[0146] This module is used to store the processed image results, and by analyzing the coordinate and quantity information of the obtained photoreceptors, metrics such as the density and number of neighbors of the photoreceptors are obtained.

[0147] As an optimization, this embodiment can also automatically generate a folder for the time when the current result data is generated. The photoreceptor information in xlsx format is independent of the system. As an independent file, it can circulate between different terminals and be read and processed by program software on different devices.

[0148] (5) A human-machine interaction module for interacting with the user of the detection system.

[0149] This module will serve as a medium for the software and hardware system to communicate information and transfer files to the user, and play a role in the following two aspects:

[0150] 1) As the input and output interface of the detection system, transfer the data to be processed and the processing results.

[0151] The system user will, through the human-machine interaction module, input the data obtained from the confocal fundus microscope device into the system. After the system completes the processing of the data and generates the processing results, the results will also be presented to the user through this module.

[0152] 2) As the input interface for the debugging instructions of the detection system, help the user flexibly adjust the details of data processing.

[0153] During the operation of the detection system, many detailed parameters will affect the data processing process and ultimately affect and change the processing results. Through the human-computer interaction module, the user can flexibly adjust multiple parameters according to the characteristics and requirements of the current data. In this embodiment, by providing a human-computer interaction module in the detection system, it is convenient for users to intervene in the detection process.

[0154] (6) An evolutionary iteration module for self-iterative optimization.

[0155] This module uses evolutionary computing methods, taking multiple parameters used in the entire processing flow as optimization objects, with the goal of forming a set of parameters that can achieve the best results within a limited time.

[0156] The iteration process is as follows:

[0157] 1) Based on the initially set parameter values manually, automatically perform random mutations to form multiple different parameter combinations. The parameters in the parameter combinations include chain length, sensitivity coefficient m, amplification coefficient k, and relative decision threshold a.

[0158] 2) Each parameter combination is separately tested through the entire process to obtain multiple index data of the photoreceptor.

[0159] 3) Record the experimental results of each parameter combination and compare them, retaining the parameter combination with the best performance. In this embodiment, it can be compared one by one with the photoreceptor coordinates manually marked to obtain the index of "accuracy rate", and determine the parameter combination with the best performance according to the accuracy rate. It can also, after obtaining the distribution law of the overall number density of photoreceptors, check whether these parameters match the medical prior knowledge to determine the parameter combination with the best performance.

[0160] 4) Using the best parameter group in 3) as the benchmark, repeat 1), 2), and 3) until the experimental performance meets the criteria manually input or the experimental time exceeds the set value.

[0161] The fundus image photoreceptor detection system provided in this embodiment includes an image preprocessing module, a chain filtering module, a photoreceptor counting module, a result display and storage module, a human-computer interaction module, and an evolutionary iteration module. The image preprocessing module is used to optimize the image quality, the chain filtering module is used to perform image filtering processing, the photoreceptor counting module is used to count the number and coordinates of photoreceptors, the result display and storage module is used to display and store the processing results, the human-computer interaction module is used to help the detection system interact with its users, and the evolutionary iteration module is used to automatically iterate and generate parameter combinations with better performance, thereby improving the performance of the detection system.

[0162] In each embodiment described in this specification, the focus is on the differences from other embodiments. For the same or similar parts among the embodiments, reference can be made to each other. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple. For the relevant parts, reference can be made to the description in the method section.

[0163] In this article, specific examples are used to elaborate on the principles and implementation manners of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those of ordinary skill in the art, based on the idea of the present invention, there will be changes in the specific implementation manners and application scopes. In summary, the content of this specification should not be construed as a limitation to the present invention.

Claims

1. A method for detecting photoreceptors in fundus images, characterized in that, The detection method includes: Obtaining a fundus image; Processing the fundus image by using a chained filtering algorithm to obtain a filtered image; Determining the type of each pixel point of the filtered image based on an adaptive threshold; the types include photoreceptor pixel points and background pixel points; The processing of the fundus image by using the chained filtering algorithm to obtain a filtered image specifically includes: For each pixel point in the fundus image, taking the pixel point as a starting point, performing chain growth in multiple preset directions respectively to obtain a chain corresponding to each preset direction; for each chain, determining the minimum value of the gray values of all pixel points included in the chain, and calculating the difference between the gray value of the pixel point and the minimum value to obtain a gray difference corresponding to each chain; calculating the filtered gray value of the pixel point based on the gray differences corresponding to all the chains to obtain a filtered image; The determining the type of each pixel point of the filtered image based on the adaptive threshold specifically includes: For each pixel point of the filtered image, comparing the gray value of the pixel point with the adaptive threshold; if the gray value is greater than or equal to the adaptive threshold, then the pixel point is a photoreceptor pixel point; otherwise, the pixel point is a background pixel point; The calculation formula of the adaptive threshold is: ； Among them, T is the adaptive threshold; is the average value of the gray values of all pixel points of the filtered image; a is the relative decision threshold; is the standard deviation of the gray values of all pixel points of the filtered image.

2. The detection method according to claim 1, wherein, Before processing the fundus image by using the chained filtering algorithm, the detection method further includes: interpolating and magnifying the fundus image by using a bicubic interpolation algorithm to obtain a preprocessed image, and using the preprocessed image as the new fundus image.

3. The detection method according to claim 1, wherein The taking the pixel point as a starting point and performing chain growth in multiple preset directions respectively to obtain a chain corresponding to each preset direction specifically includes: For each preset direction, perform the following steps: Taking the pixel point as a starting point; Determining a candidate pixel set according to the starting point and the search direction of the current iteration; based on the positions and gray values of all candidate pixel points in the candidate pixel set, selecting a candidate pixel point as a target pixel point; connecting the starting point and the target pixel point, and performing chain growth on the intermediate chain of the previous iteration to obtain the intermediate chain of the current iteration; Judging whether the number of pixel points included in the intermediate chain of the current iteration reaches a preset number; If so, end the iteration, and use the intermediate chain of the current iteration as the chain corresponding to the preset direction; If not, calculating the growth direction of the current iteration according to the position of the starting point and the position of the target pixel point, and calculating the search direction of the next iteration according to the growth direction of the current iteration; taking the target pixel point as the starting point of the next iteration, and returning to the step of "determining a candidate pixel set according to the starting point and the search direction of the current iteration".

4. The detection method according to claim 3, characterized in that The calculation formula of the search direction is: ； Among them, is the search direction for the i th iteration; is the search direction for the i -1th iteration; x is an intermediate parameter; m is the sensitivity coefficient; is the growth direction for the i -1th iteration.

5. The detection method according to claim 1, wherein The calculation formula of the filtered gray value is: ； Among them, is the gray value after filtering of the pixel point P ; is the minimum value of the gray difference corresponding to all chains; is the j gray difference corresponding to the k th chain; is the magnification factor.

6. The detection method according to claim 1, wherein After determining the type of each pixel point, the detection method further includes: Counting the photoreceptor pixel points of the filtered image by using a region growing algorithm to obtain the number of photoreceptors and the position of each photoreceptor; the photoreceptor includes multiple adjacent photoreceptor pixel points; Calculate the density of the photoreceptors according to the number of the photoreceptors; Draw a Voronoi diagram based on the positions of all the photoreceptors; determine the number of Voronoi diagram neighbors of each photoreceptor according to the Voronoi diagram.

7. A fundus image photoreceptor detection system, characterized in that The detection system includes: An acquisition module, configured to acquire a fundus image; A processing module, configured to process the fundus image by using a chain filtering algorithm to obtain a filtered image; A detection module, configured to determine the type of each pixel point of the filtered image based on an adaptive threshold; the type includes photoreceptor pixel points and background pixel points; The processing of the fundus image by using the chain filtering algorithm to obtain a filtered image specifically includes: For each pixel point in the fundus image, taking the pixel point as a starting point, performing chain growth in a plurality of preset directions respectively to obtain a chain corresponding to each preset direction; for each chain, determining the minimum value of the gray values of all the pixel points included in the chain, and calculating the difference between the gray value of the pixel point and the minimum value to obtain a gray difference corresponding to each chain; calculating the filtered gray value of the pixel point based on the gray differences corresponding to all the chains to obtain a filtered image; The determining the type of each pixel point of the filtered image based on an adaptive threshold specifically includes: For each pixel point of the filtered image, comparing the gray value of the pixel point with the adaptive threshold; if the gray value is greater than or equal to the adaptive threshold, the pixel point is a photoreceptor pixel point; otherwise, the pixel point is a background pixel point; The calculation formula of the adaptive threshold is: ； Among them, T is the adaptive threshold; is the average value of the gray values of all pixel points of the filtered image; a is the relative decision threshold; is the standard deviation of the gray values of all pixel points of the filtered image.

Images