Eye disease analysis method based on eye image

Abstract

The invention discloses an eye disease analysis method based on an eye image, and the method comprises the steps: carrying out the image segmentation of a sclera region through employing a convolutional neural network architecture u-net network model of image segmentation, carrying out the recognition of a hyperemia part of the sclera region, and calculating the proportion of the hyperemia part through employing FrangiFilter blood vessel recognition in combination with the color recognition of an LAB color space; by combining the image algorithm with a traditional image processing and deep learning model, the segmentation precision of the sclera region, the identification precision of sclera congestion, the robustness, the high accuracy and the rating accuracy are improved under the condition that the operation speed is not reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of eye disease analysis, and specifically provides a method for analyzing eye diseases based on eye images. Background Art

[0002] Dry eye is a multi-factorial ocular surface disease characterized by the imbalance of the tear film homeostasis and accompanied by ocular discomfort symptoms. The main pathophysiological mechanisms include unstable tear film, increased tear osmotic pressure, ocular surface inflammatory reaction and damage, and nerve abnormalities. Common symptoms of dry eye include ocular dryness, foreign body sensation, burning sensation, eye itching, pain, eye redness, visual fatigue, blurred vision, vision fluctuation, etc. Eye redness analysis is an important feature for the judgment of dry eye.

[0003] Currently, traditional image algorithms are mostly used for image segmentation in the scleral region and segmentation of the scleral congestion region in current devices, such as global binaryzation, edge detection and other algorithms; however, traditional image algorithms are greatly affected by light, resulting in inaccurate scleral segmentation, low robustness, and the grading of eye redness analysis generally depends on the experience of medical staff.

[0004] Therefore, it is necessary to provide a method for analyzing eye diseases based on eye images. Summary of the Invention

[0005] The purpose of the present invention is to provide a method for analyzing eye diseases based on eye images to solve the problems raised in the above background art.

[0006] To solve the above technical problems, the present invention provides the following technical solutions:

[0007] A method for analyzing eye diseases based on eye images, comprising the following steps:

[0008] S100. Obtain a batch of images for eye redness analysis, manually label the scleral region in the images, and input the labeled images into the built u-net neural network model for training to finally obtain a trained u-net neural network model;

[0009] S200. Input the real-time captured eye redness image into the pre-trained u-net neural network model to obtain a mask image with the scleral region segmented;

[0010] S300. Convert the RGB image to an LAB image. Lab consists of a luminance channel and two color channels. L represents luminance, a represents the component from green to red, and b represents the component from blue to yellow. The Lab color space is more in line with human vision and can better segment red;

[0011] The S400, by combining the Lab image and using the filtering function Frangi of the Hessian matrix to process the image, can better segment the congested part of the scleral region;

[0012] S500, for the enhanced blood vessel map after Frangi filtering, filters out the interference information in the non-blood vessel area, calculates the proportion value of the congested area in the entire scleral region, and judges the severity of congestion according to the depth of the congested color;

[0013] S600, grades the degree of red eyes based on the proportion of congestion and the severity of congestion.

[0014] According to the above technical solution, the Hessian matrix of S400 is a square matrix of the second partial derivatives of a scalar function, which describes the local curvature of a multivariable function, and its basic form is:

[0015]

[0016] Second partial differential in the X direction:

[0017]

[0018] Second partial differential in the Y direction:

[0019]

[0020] Mixed partial differential in the X and Y directions:

[0021]

[0022] For an image, it only has two directions, X and Y, so the Hessian matrix is a binary matrix, and correspondingly:

[0023]

[0024] Corresponding to the image convolution operation respectively, to form the Hessian matrix of the image:

[0025]

[0026] According to the above technical solution, the filtering function Frangi is the Frangi2D filtering, which is an edge detection and enhancement filtering algorithm constructed based on the Hessian matrix:

[0027] Since f xy = f yx , and H is a real symmetric matrix, two eigenvalues λ1, λ2 can be used to construct the enhancement filtering. That is, in the two-dimensional image space, the two eigenvalues λ1, λ2 of the Hessian matrix can be calculated by the following formula:

[0028]

[0029] Where: K = (f xx +f yy ) / 2,

[0030] Congested blood vessels generally appear in a linear form. The difference operation of the Hessian matrix is ​​combined with the Gaussian function. The linear enhancement filter at different scales σ is obtained by changing the standard deviation offset of the Gaussian function. The half-width of the window rectangle for local characteristic analysis is generally taken as 3σ. At the current scale, the diameter of the blood vessel is smaller than the width and height of the window rectangle of the current size response. Then the eigenvalue of the Hessian matrix of the tubular blood vessel satisfies:

[0031] |λ|≈0,|λ1|<<|λ2|.

[0032] Use two operators to define the relationship between these two eigenvalues:

[0033]

[0034] At the current scale, the responsivity of a point p in the image belonging to a blood vessel region is defined as:

[0035]

[0036] Among them, β is used to adjust the difference between linear and blocky objects, and c and γ are parameters that control the overall smoothness of linear objects;

[0037] The adjustment range of the vascular linear response function is β∈[0.3,2], c∈[10 -5 ,10 -6 ],γ∈[3,6], when the windows corresponding to different scales match the blood vessel diameter, v(p) will have the largest response:

[0038]

[0039] The maximum responsiveness is obtained by selecting the iteration factor σ. The range of the iteration factor [σ min ,σ max ], the iteration step increment is step, the smaller the scale factor, the better the enhancement effect of the blood vessels with the smallest diameter, and the larger the scale factor, the better the enhancement effect of the blood vessels with the largest diameter.

[0040] According to the above technical solution, the specific steps of calculating the scale factor are as follows:

[0041] a. Input corneal conjunctival area image;

[0042] b. Set the range of the scale factor σ as [σ min , σ max , and the iteration step size step. For each pixel of the image I, repeat from c to j;

[0043] c. Initialize the spatial scale factor σ and the tubular response V(σ, p) max

[0044] d. If the scale σ satisfies the extreme value, stop the loop and jump to i

[0045] e. Calculate the convolution of the element I ij with the second-order differential of the Gaussian function

[0046] f. Generate the Hessian matrix H, and calculate its two eigenvalues λ1 and λ2 and eigenvectors e1 and e2

[0047] g. Calculate the output value V(σ, p) of the tubular response for the enhanced filter

[0048] h. Update the iteration factor σ = σ + step, and jump to d

[0049] i. Output the maximum enhanced filter output value V(σ, p) of the high pixels max and the corresponding eigenvalues and eigenvectors

[0050] j. Perform non-linear anisotropic diffusion on the blood vessel image with V(σ, p) max and its eigenvectors

[0051] k. Perform adaptive gray-scale morphological operations on different directions and sizes of structures with V(σ, p) max , the corresponding σ, eigenvalues, and eigenvectors.

[0052] According to the above technical solution, the specific calculation method for S500 to calculate the proportion value of the congested area in the entire scleral area is as follows:

[0053] 1) Calculate the number of non-zero pixels Rpixels in the blood vessel enhancement map after Frangi filtering. Calculate the number of pixels Spixel in the scleral area according to the mask of the scleral area. Calculate the proportion of red eye:

[0054]

[0055] 2) Statistically analyze the gray-scale distribution of the non-zero pixels in the blood vessel enhancement map after Frangi filtering. Among them, the larger the gray-scale value, the deeper the red color, indicating severe congestion, and the smaller the gray-scale value, the lighter the red color, indicating less severe congestion. Calculate the proportion weights according to the gray-scale distribution for different gray-scales:

[0056]

[0057] where gray is the grayscale value, n_gray is the number of pixels with the grayscale value of gray, and the weighted proportion of red eyes (the severity of congestion) is

[0058]

[0059] According to the above technical solution, the specific classification and discrimination method of S600 is as follows:

[0060] Calculate according to the proportion of red eyes ratio1 and the weighted proportion of red eyes ratio2:

[0061] ratio = (ratio1 * k1 + ratio2 * (1 - k1)) * 100

[0062] k1 generally takes 0.5 and can be adjusted according to the actual situation. Finally, use the fractional grading system to obtain the corresponding red eye degree level.

[0063] Compared with the prior art, the beneficial effects achieved by the present invention are:

[0064] Through this image algorithm combined with the models of traditional image processing and deep learning, without reducing the operation speed, the segmentation accuracy of the scleral region, the recognition accuracy of scleral congestion are improved, the robustness is enhanced, and the high accuracy and the accuracy of rating are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 is a flowchart of the present invention;

[0066] Figure 2 is an example diagram of the image processing of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0067] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0068] The present invention provides a technical solution: as Figure 1-2 shown, a method for analyzing eye diseases based on eye images includes the following steps:

[0069] S100. Obtain a batch of images for red eye analysis, manually label the scleral region in the images, and input the labeled images into the built u-net neural network model for training, and finally obtain a trained u-net neural network model;

[0070] S200. Input the red-eye image captured in real time into the pre-trained u-net neural network model to obtain a mask image with the scleral region segmented.

[0071] S300. Convert the RGB image to an LAB image. Lab consists of a luminance channel and two color channels. L represents luminance, a represents the component from green to red, and b represents the component from blue to yellow. The Lab color space is more in line with human vision and can better segment red.

[0072] S400. Process the Lab image using the Frangi filtering function of the Hessian matrix to better segment the congested part of the scleral region.

[0073] S500. For the blood vessel enhanced image after Frangi filtering, filter out the interference information in the non-vessel region, calculate the proportion value of the congested area in the entire scleral region, and at the same time judge the severity of congestion according to the depth of the congested color.

[0074] S600. Grade the degree of red eye based on the proportion of congestion and the severity of congestion.

[0075] Specifically, the Hessian matrix in S400 is the square matrix of the second-order partial derivatives of a scalar function, which describes the local curvature of a multivariable function. Its basic form is:

[0076]

[0077] Second-order partial differential in the X direction:

[0078]

[0079] Second-order partial differential in the Y direction:

[0080]

[0081] Mixed partial differential in the X and Y directions:

[0082]

[0083] For an image, it only has two directions, X and Y. So the Hessian matrix is a binary matrix, corresponding to:

[0084]

[0085] Corresponding to the image convolution operation respectively, to form the Hessian matrix of the image:

[0086]

[0087] Specifically, the filtering function Frangi is the Frangi2D filter, which is an edge detection and enhancement filtering algorithm constructed based on the Hessian matrix:

[0088] Since f xy = f yx , and H is a real symmetric matrix, two eigenvalues λ1 and λ2 can be used to construct the enhancement filter. That is, in the two-dimensional image space, the two eigenvalues λ1 and λ2 of the Hessian matrix can be calculated by the following formula:

[0089]

[0090] where: K = (f xx + f yy ) / 2,

[0091] For congested blood vessels, they generally appear in a linear form. By combining the differential operation of the Hessian matrix with the Gaussian function and changing the standard deviation offset of the Gaussian function, linear enhancement filters at different scales σ can be obtained. The half-width of the window rectangle for local feature analysis is generally taken as 3σ. At the current scale, if the diameter of the blood vessel is smaller than the width and height of the window rectangle corresponding to the current size response, the eigenvalues of the Hessian matrix of the tubular blood vessel satisfy:

[0092] |λ| ≈ 0, |λ1| << |λ2|.

[0093] Two operators are used to define the relationship between these two eigenvalues:

[0094]

[0095] At the current scale, the response degree of a point p in the image belonging to the blood vessel area is defined as:

[0096]

[0097] where β is used to adjust the difference between linear and blocky objects, and c and γ are parameters that control the overall smoothness of the linear object;

[0098] The adjustment range of the blood vessel linear response function is β ∈ [0.3, 2], c ∈ [10 -5 , 10 -6 , γ ∈ [3, 6]. When the windows corresponding to different scales match the blood vessel diameter, v(p) will have the maximum response:

[0099]

[0100] The maximum response degree is obtained by selecting the σ of the iteration factor, and the change range of the iteration factor is [σmin , σ max , the iteration step increment is step. The smaller the scale factor, the better the enhancement effect of the blood vessels with the smallest diameter. The larger the scale factor, the better the enhancement effect of the blood vessels with the largest diameter.

[0101] Specifically, the specific steps for calculating the scale factor are as follows:

[0102] a. Input the image of the corneal limbus region;

[0103] b. Set the range of the scale factor σ as [σ min , σ max and the iteration step step. For each pixel of the image I, repeat c to j;

[0104] c. Initialize the spatial scale factor σ and the tubular response V(σ, p) max

[0105] d. If the scale σ satisfies the extreme value, stop the loop and jump to i

[0106] e. Calculate the convolution of the element I ij with the second-order differential of the Gaussian function

[0107] f. Generate the Hessian matrix H, and calculate its two eigenvalues λ1 and λ2 and eigenvectors e1 and e2

[0108] g. Calculate the output value V(σ, p) of the enhanced filtering tubular response

[0109] h. Update the iteration factor σ = σ + step, and jump to d

[0110] i. Output the maximum enhanced filtering output value V(σ, p) of the high pixels max and the corresponding eigenvalues and eigenvectors

[0111] j. Perform non-linear anisotropic diffusion on the blood vessel image with V(σ, p) max and its eigenvectors

[0112] k. Perform adaptive gray-scale morphological operations on different directions and sizes of structures with V(σ, p) max , the corresponding σ, eigenvalues and eigenvectors.

[0113] Specifically, the specific calculation method for S500 to calculate the proportion value of the congested area in the entire scleral area is as follows:

[0114] 1) Calculate the number of non-zero pixels Rpixels in the blood vessel enhancement map after Frangi filtering. Calculate the number of pixels Spixel in the scleral area according to the mask of the scleral area. The calculation of the proportion of red eyes is:

[0115]

[0116] 2) Calculate the distribution of gray levels among the non-zero pixels in the blood vessel enhancement map after Frangi filtering. Among them, the larger the gray level value, the darker the red color, indicating severe congestion, and the smaller the gray level value, the lighter the red color, indicating less severe congestion. Different gray levels calculate the proportion weights according to their distributions:

[0117]

[0118] where gray is the gray level value, n_gray is the number of pixels with the gray level value of gray, and the weighted proportion of red eyes (severity of congestion) is

[0119]

[0120] Specifically, the specific classification and discrimination method of S600 is as follows:

[0121] Calculate according to the proportion of red eyes ratio1 and the weighted proportion of red eyes ratio2:

[0122] ratio = (ratio1 * k1 + ratio2 * (1 - k1)) * 100

[0123] k1 is generally taken as 0.5 and can be adjusted according to the actual situation. Finally, using the fractional grading system, the corresponding levels of red eye degree are obtained.

[0124] It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variation thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or device.

[0125] Finally, it should be noted that the above are only the preferred embodiments of the present invention and are not used to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for analyzing eye diseases based on eye images, characterized in that: The following steps are involved: S100, obtaining a batch of images for red eye analysis, manually annotating the sclera area in the image, inputting the annotated image into the constructed u-net neural network model for training, and finally obtaining a trained u-net neural network model; S200, inputting the red eye image captured in real time into a pre-trained u-net neural network model to obtain a mask image with a segmented sclera area; S300, convert the RGB image to LAB image. Lab is composed of a brightness channel and two color channels. L represents brightness, a represents the component from green to red, and b represents the component from blue to yellow. The Lab color space is more consistent with human vision and can better segment red. S400 and Lab images are combined with Frangi's filter function using the Hessian matrix to process the image, which can better segment the congested part of the sclera area; The blood vessel enhancement image after S500 and Frangi filtering filters out the interference information of non-vascular areas, calculates the ratio of the congested area to the entire sclera area, and judges the severity of congestion according to the depth of the congestion color; S600. The degree of eye redness is graded according to the proportion of congestion and the severity of congestion.

2. The eye disease analysis method based on eye images according to claim 1, characterized in that: The Hessian matrix of S400 is the square matrix of the second-order partial derivatives of the scalar function, which describes the local curvature of a multivariable function. The basic form is: Second order partial differential in the X direction: Second order partial differential in the Y direction: Mixed partial differentials in the X and Y directions: For an image, it only has two directions, X and Y, so the Hessian matrix is ​​a binary matrix, corresponding to: They correspond to the image convolution operations respectively, forming the Hessian matrix of the image:

3. The eye disease analysis method based on eye images according to claim 2, characterized in that: The filter function Frangi is a Frangi2D filter, which is an edge detection enhancement filter algorithm constructed based on the Hessian matrix: Because f xy =f yx , H is a real symmetric matrix, so the two eigenvalues ​​λ1 and λ2 can be used to construct the enhancement filter, that is, in the two-dimensional image space, the two eigenvalues ​​λ1 and λ2 of the Hessian matrix can be calculated by the following formula: Where: K = (f xx +f yy ) / 2, Congested blood vessels generally appear in a linear form. The difference operation of the Hessian matrix is ​​combined with the Gaussian function. The linear enhancement filter at different scales σ is obtained by changing the standard deviation offset of the Gaussian function. The half-width of the window rectangle for local characteristic analysis is generally taken as 3σ. At the current scale, the diameter of the blood vessel is smaller than the width and height of the window rectangle of the current size response. Then the eigenvalue of the Hessian matrix of the tubular blood vessel satisfies: |λ|≈0,|λ1|<<|λ2|. Use two operators to define the relationship between these two eigenvalues: At the current scale, the responsivity of a point p in the image belonging to a blood vessel region is defined as: Among them, β is used to adjust the difference between linear and blocky objects, and c and γ are parameters that control the overall smoothness of linear objects; The adjustment range of the vascular linear response function is β∈[0.3,2], c∈[10 -5 ,10 -6 ],γ∈[3,6], when the windows corresponding to different scales match the blood vessel diameter, v(p) will have the largest response: The maximum responsiveness is obtained by selecting the iteration factor σ. The range of the iteration factor [σ min ,σ max ], the iteration step increment is step.

4. The eye disease analysis method based on eye images according to claim 3, characterized in that: The specific steps for calculating the scale factor are as follows: a. Input corneal conjunctival area image; b. Set the scale factor σ range [σ min ,σ max ] and iteration step size step, for each pixel of image I, repeat c~j; c. Initialize the spatial scale factor σ and the tubular response V(σ,p) max d. If the scale σ meets the extreme value, stop the loop and jump to i e. Computational Element I ij Convolution with the second derivative of a Gaussian function f. Generate the Hessian matrix H and calculate its two eigenvalues ​​λ1 and λ2 and eigenvectors e1 and e2 g. Calculate the tubular response output value V(σ,p) of the enhanced filter h. Update iteration factor σ = σ + step, jump to d i. Output the maximum enhanced filter output value V(σ,p) of high pixels max and the corresponding eigenvalues ​​and eigenvectors j. V(σ,p) max Nonlinear anisotropic diffusion of vascular images using its eigenvector k. Take V(σ,p) max , the corresponding σ, eigenvalues ​​and eigenvectors select structures of different directions and sizes for adaptive grayscale morphological operations.

5. The eye disease analysis method based on eye images according to claim 1, characterized in that: The specific calculation method of S500 to calculate the ratio of the hyperemia area to the entire sclera area is as follows: 1) Calculate the number of non-zero pixels Rpixels in the blood vessel enhancement image after Frangi filtering, calculate the number of pixels Spixel in the sclera area according to the mask of the sclera area, and calculate the redness ratio: 2) Calculate the distribution of grayscale in non-zero pixels in the Frangi filtered blood vessel enhancement image. The larger the grayscale value, the darker the red color and the more severe the congestion. The smaller the grayscale value, the lighter the red color and the less severe the congestion. The weights of different grayscales are calculated according to the distribution of grayscale: Among them, gray is the gray value, n_gray is the number of pixels with gray value, and the weighted red eye ratio (the severity of congestion) is 6. The eye disease analysis method based on eye images according to claim 1, characterized in that: The specific classification method of S600 is as follows: Calculated based on the redness ratio ratio1 and the weighted redness ratio ratio2: ratio=(ratio1*k1+ratio2*(1-k1))*100 k1 is generally set to 0.5, which can be adjusted according to actual conditions. Finally, the score grading system is used to obtain the corresponding level of envy.

Images