Meibomian gland image analysis method

Abstract

The present invention particularly relates to a meibomian gland image analysis method that enables detailed visualization of the glands within the eyelid in the diagnosis and treatment of meibomian gland disease, cleaning of image regions that cause errors, analysing the images obtained for disease diagnosis and increasing the accuracy of disease diagnosis by comparing the analysis results.

Description

[0001] DESCRIPTION

[0002] MEIBOMIAN GLAND IMAGE ANALYSIS METHOD

[0003] Field of the Invention

[0004] The present invention relates to an image analysis method that enables the analysis of the glands in the eyelid in the diagnosis and treatment of meibomian gland disease.

[0005] The present invention particularly relates to a meibomian gland image analysis method that enables detailed visualization of the glands within the eyelid in the diagnosis and treatment of meibomian gland disease, cleaning of image regions that cause errors, analysing the images obtained for disease diagnosis and increasing the accuracy of disease diagnosis by comparing the analysis results.

[0006] State of the Art

[0007] Meibomian glands, located in the upper and lower eyelids, lubricate the surface of the eye to maintain its moist and slippery structure, and this oil secreted from the meibomian gland prevents the rapid evaporation of tears. Meibomian gland dysfunction (MGD), also called meibomianitis, occurs when there is blockage and / or insufficient oil secretion into the tears, causing rapid evaporation of tears and conditions such as MGD, dry eye syndrome, and blepharitis. With the imaging method called meibography, adverse conditions such as obstruction, shortening, irregularity or glandular atrophy in the glands can be seen. Various treatment methods are used in the treatment of meibomian gland disease, such as massage that the patient can apply on his / her own, medical treatments, and applying rhythmic pressure on the eyelids (Thermal Pulsation System-LipiFlow) after softening the solidified fat in the meibomian ducts by warming the eyelids. Infrared imaging systems (Lipiscan) that allow imaging of the meibomian glands are used to diagnose the disease before treatment and to determine the degree of disease, progression and effectiveness of the treatment applied while treatment applications such as Lipiflow are in progress. However, in current practices, postprocedural evaluations are generally qualitative, and a quantitative analysis is needed to clearly determine the disease status after imaging and to make an accurate treatment evaluation.

[0008] In some of the existing meibomian devices (e.g., the Sirius device), approximately 200 points are manually placed on the edges of the glands by user selection, and an analysis system roughly calculates the gland area within this manually determined area by the user. However, this process takes approximately 20-30 minutes to complete, even for the most experienced operator, and if any of the points in question are placed incorrectly by the user, the entire operation is disrupted and the process must be repeated from the beginning. In addition, it becomes difficult to detect different errors and faulty areas that occur during imaging, thus making it difficult to select the correct area. Infrared meibography images obtained from the Oculus Keratograph 5M device, another device used for the process in question, are manually classified based on the observations made by two operators, and a third operator checks the classifications of the first two operators, which are incompatible with each other. In this method, due to manual selection and control, various errors occur depending on the experience of the operator, and the processes must be repeated from the beginning to correct the errors. In order to eliminate these problems to a certain extent, the software called Imaged is used, which accelerates the analysis process of meibography images. Imaged is a dava-based image processing program, and in this program, the area around the glands and the area around the eyelid are marked manually. However, it is quite difficult to process and mark the walls and details of the meibomian glands with this program; sharpening, reflection removal or precise selection cannot be done, and therefore the program can only give a rough result by calculating the ratio of the area where the glands are concentrated to the total selected area.

[0009] A state of the art patent document numbered CN111528789A describes a meibomian gland image detection method for evaluating meibomian gland definition. In the mentioned method, the patient's upper eyelid must first be turned inside out to completely expose the upper eyelid layer. Using the infrared meibomian gland photography system, an image of the patient's upper eyelid layer is taken and the measurement values of the measurement points on the patient's nasal side, center and temple side in the image are measured. The average value of the three measurement values is taken as the specific meibomian gland definition of the relevant patient. The result obtained after two independent measurers perform the determination twice is averaged as the final measurement result. According to the method, meibomian gland definition is measured and morphological changes of different patients can be compared objectively with this measured definition. Visual comparison of previous and current data changes of the same patient is made. In said document, there is no mention of a solution for quickly and accurately selecting the area to be analysed by clearing defects such as reflections and blurring, calculating the lost area, performing image sharpening and ignoring / deleting operations.

[0010] In another patent document numbered CN111145155A belonging to the state of the art, an image development method for the meibomian oil gland is mentioned. The method described herein describes obtaining and processing a meibomian gland image, enhancing the gland image, extracting gland edge information and creating masks surrounding the glands, extracting a gland image, removing noise points and discrete points and calculating the average width of the gland gaps and the average width of the gland gaps, calculating the ratio of the glands to obtain the centerline of each gland, outlining the glands, calculating the length of each centerline and concluding that glands are missing if the length of each centerline is less than the set minimum length value. In said document, there is no mention of a solution for quickly and accurately selecting the area to be analysed by clearing defects such as reflections and blurring, calculating the lost area, performing image sharpening and ignoring / deleting operations.

[0011] As a result, there is a need to develop a meibomian gland image analysis method that enables detailed visualization of the glands within the eyelid, removal of image regions containing blur and reflection that cause errors, clarification and analysis of images obtained for disease diagnosis, and increasing the accuracy of disease diagnosis by comparing analysis results in the diagnosis and treatment of meibomian gland disease.

[0012] Object of the Invention

[0013] The present invention is related to a meibomian gland image analysis method which fulfils the above-mentioned requirements, eliminates all disadvantages and brings some additional advantages.

[0014] The main object of the meibomian gland image analysis method, which is the subject of the invention, is to obtain a meibomian gland image analysis method that enables the imaging of meibomian glands, the selection of the area to be measured on the image and the making of changes on the selected area, the cleaning of image defects such as blur and reflection that create errors during measurement and the increase of image clarity, thus increasing the speed and accuracy of the diagnosis and analysis of the disease condition.

[0015] Another object of the present invention is to provide a practical meibomian gland image analysis method that allows a single user to select the area to be image measured and to perform changes such as deleting, ignoring, repositioning on the selected area without the need to perform the operations from the beginning after selection.

[0016] Another object of the present invention is to obtain an efficient meibomian gland image analysis method that enables the removal of unwanted regions or regions that may cause measurement errors from the selected image area and increases the measurement consistency after selecting the image area to be measured / analysed. Another object of the present invention is to obtain a functional meibomian gland image analysis method that shortens the measurement time and simplifies the measurement settings.

[0017] Another object of the present invention is to obtain a meibomian gland image analysis method that increases the effectiveness of diagnosing the disease and monitoring it after diagnosis or during treatment.

[0018] Another object of the present invention is to obtain a useful meibomian gland image analysis method that enables the calculation of the missing area that does not contribute to the measurement process or creates measurement errors and allows rearrangement over the total area first selected by the user.

[0019] Another object of the present invention is to obtain a functional meibomian gland image analysis method that is suitable for working with systems containing different features of electronic components for imaging and image processing.

[0020] A meibomian gland image analysis method that is suitable for use in an analysis system comprising at least one processing unit, at least one image processing unit, at least one display unit, at least one input unit and at least one storage unit in order to achieve the above objectives in the most general way, and that enables the selection of the measurement area on the eyelid image, the making of changes such as deletion, addition, ignoring, etc. on the selected area without cancelling the same, and the elimination of image defects that affect the measurement, comprising the steps of; generating the image mask by means of the storage unit; increasing the contrast value of the image mask by means of the image processing unit; determining the first mask area by the user by means of the input unit; determining the second mask area by the user by means of the input unit; determining the third mask area by the user by means of the input unit; obtaining the measurement mask that contains the actual measurement area and will be used to detect meibomian gland disease.

[0021] The structural and characteristic features of the present invention will be understood clearly by the following detailed description made with reference to these drawings and therefore the evaluation shall be made by taking these figures and the detailed description into consideration.

[0022] Detailed Description of the Invention In this detailed description, the preferred embodiments of the meibomian gland analysis method is described only for clarifying the subject matter in a manner such that no limiting effect is created.

[0023] Meibomian gland image analysis method, which is suitable for use in an analysis system developed with the present invention comprising at least one processing unit that provides numerical calculation and process control, preferably a processor suitable for running at least one operating system; at least one image processing unit that provides processing and analysis of the image, preferably a display card and / or processor suitable for running OpenCV (Open Computer Vision Library); at least one display unit that provides transmission of the image to the user, preferably in the form of a screen; at least one input unit that provides information input and editing, preferably in the form of a keyboard, controller and / or mouse; and at least one storage unit that provides recording of the information, which allows the selection of the measurement area on the eyelid image, preferably with an infrared image, and the removal of image defects that affect the measurement, without cancelling the selected area, such as deletion, addition, ignoring, repositioning, etc., comprises the process steps of; recording the eyelid image via the said storage unit and creating an image mask with all pixel values black / RGB (0) by selecting the area on the eyelid image to be measured by the user; obtaining the histogram of pixel distribution by determining the pixel values of the image mask by means of said image processing unit, converting the pixel distribution intensities of the image mask into a graph, and increasing the contrast value of the image mask by processing the image so as to expand the pixel distribution density range in said graph; determining the Gaussian distribution around the pixels of the image mask whose contrast value is increased, preferably by means of said image processing unit, determining the Gaussian mean value by averaging the determined Gaussian distribution, increasing the sharpness value of the image mask by equating the pixel values of the image mask to the said Gaussian mean value; preferably saving the eyelid image and image mask by converting them to SDL_Texture or pixel data format; preferably converting said image mask into three separate masks with all pixel values being black / RGB (0) to create a first mask providing calculation, a second mask providing loss / deletion display and a third mask providing display of the ignored area, saving them and then presenting them to the user via said display unit; converting the pixel value of the area selected on the image mask by the user via the said input unit to a first color other than black, preferably green color / RGB(0, 255,0), and determining the first mask area to be entered into the calculation in the meibomian gland identification process; converting the pixel value of the area selected on the image mask by the user, via the said input unit, to the first or a second color different from the first color, preferably red color / RGB(255,0,0), and determining the second mask area to be taken as a loss in the calculation in the meibomian gland identification process; converting the pixel value of the area selected on the image mask by the user via said input unit to the first or second color or a third color different from the second color, preferably blue color / RGB(0, 0, 255), and determining the third mask area to be ignored in the meibomian gland identification process; preferably, by means of said image processing unit, dividing the first, second and third masks into more than one processing area and converting each area into a form suitable for analysis and image processing for meibomian gland identification; preferably performing histogram analysis showing the pixel intensity values graph of the mask images using the masking regional otsu threshold method by means of the said processing unit of the first, second and third masks and by determining an optimal first threshold value in the histogram chart, converting the image into two separate images, preferably in Matlb format, positioned on top of each other, with the pixel values above this threshold value at the top and the pixel values below it at the bottom; combining the first, second and third masks in accordance with the pixel positions and numbers by means of the said processing unit, performing histogram analysis showing the pixel intensity values graph of the image obtained using the masking regional otsu threshold method and determining an optimal first threshold value in the histogram chart, converting the image to white-on-black form, preferably in Matlb format, with pixel values above this threshold value at the top and pixel values below it at the bottom and obtaining the measurement mask that will be used to detect meibomian gland disease and that contains the actual measurement area created by subtracting the areas removed from the calculation and ignored from the said calculation area.

[0024] In an embodiment of the meibomian gland image analysis method developed with the present invention, the eyelid image in JPG, PNG etc. format that is desired to be processed is selected through the interface presented to the user through the mentioned imaging unit and saved to the storage unit in Mat format with the OpenCV library. In order to display the area selections to the user for feedback, an image mask is created with all pixel values initially black (0) in the image dimensions opened in OpenCV Mat format. The image in the Mat format in the storage unit and the feedback mask in the Mat format are combined with the alpha values of 0.9 and 0.1 , converted to SDL_Texture format and copied to the "Screen to be Marked for the Application of ROTM (Regional Otsu Threshold Method of Masking)" section. Three masks in the OpenCV Matlb format, with all pixel values initially black (0), in the size of the image to be used as the first mask providing calculation, the second mask providing loss display and the third mask providing display of the ignored area, are created by the processing unit.

[0025] In the "Screen to be Marked for ROTM Application" section, the coordinate information marked by the user via the input unit is taken and the value of the pixel, as much as the size of the selected area on the 3 masks created in the storage unit, is converted to white (255). In other masks, pixel values are converted to black (0) in the same position as the size of the selected area. Afterwards, in the area size marked on the image mask, the calculation area pixel values are colored green / RGB(0, 255,0), the lost area is colored red / RGB(255,0,0) and the ignored area is colored blue / RGB(0, 0,255). Thus, the “User Feedback Screen” step is executed; the ROTM method is called using the image and masks as arguments. The processed image in Matlb format obtained as a result of the ROTM method is converted to SDL_Texture format and presented through the display element mentioned in the “ROTM Method Applied Image” screen, and the loss percentage obtained is presented in the “Current MGD Loss Rate” section.

[0026] In the process of determining the third mask area to be ignored in the meibomian gland identification process on the image mask by the user via the said input unit, respectively:

[0027] • A Matlb variable named processArea is created.

[0028] • The select, ignore and delete / loss masks taken as arguments are combined to this mask with the OpenCV bitwise_or function.

[0029] • Using the obtained mask, the processed area coordinates (roi) are obtained with the OpenCV boundingRect function.

[0030] • A Matlb variable named processed is created.

[0031] • The part of the image taken as an argument to the processed variable that falls within the processed area coordinates (roi) is copied in black and white format.

[0032] • A Gaussian blur filter is applied to the processed variable with OpenCV at a kernel size of 9,9 to remove noise.

[0033] • An OpenCV Mat variable named hist is created and the histogram value of the processed in the roi is written to this variable with the OpenCV calcHist function.

[0034] • The thresh variable is calculated with the brightSpotThresh variable I 200 whose processed length * processed width * are taken as argument.

[0035] • A variable named max with a value of 255 is created.

[0036] • As long as the max and thresh variables are greater than 0, the loop is repeated, and in each loop, the max element of hist is subtracted from the thresh variable and max is decreased by 1.

[0037] • OpenCV threshold function and processed is processed with the thresh value obtained as a result of the loop.

[0038] • With the OpenCV dilate function, the white pixels of the processed are expanded to the size of 19,19.

[0039] • The ignore mask taken as argument is combined with the select mask received as argument by the OpenCV bitwise_or function. • The processed variable is copied to the coordinates falling within the roi of the ignore mask taken as an argument.

[0040] In the ROTM application, a mask called processArea is created in OpenCV Matlb format, and the selection, ignore and delete / loss masks taken as arguments are collected into this mask with the OpenCV bitwise_or function. Using the obtained mask afterwards, the processed area coordinates (roi) are obtained with the OpenCV boundingRect function. Afterwards, masks called discardMask and ignoreMask are created in OpenCV Matlb format, and the parts of the ignore and delete / loss masks taken as arguments that fall within the processed area coordinates (roi) are copied to these masks. These masks are then inverted using the OpenCV bitwise_not function. A mask called Mask is created in OpenCV Matlb format, and the part of the calculation mask taken as an argument that falls within the processed area coordinates (roi) is copied to the mask called Mask by combining discardMask and ignoreMask with the OpenCV bitwise_and function. The number of pixels whose value is not 0 in the calculation mask and the delete / loss mask taken as arguments is calculated with the OpenCV countNonZero function and collected into the eyelidArea variable. If the pointer of the image received as an argument is not the same as the last globally held image pointer (recordPtr), the following steps are performed:

[0041] • recordPtr is replaced with the image pointer taken as argument.

[0042] • The part of the image taken as an argument to the last image (hostSource) in the globally held OpenCV Matlb format that falls within the processed area coordinates (roi) is copied in black and white format.

[0043] • A Gaussian blur filter is applied to the hostSource variable with OpenCV at a kernel size of 9,9 to remove noise.

[0044] • A histogram equalization filter is applied to hostSource with OpenCV.

[0045] The function used in the mask-dependent application of the Otsu method takes the mean intensity value (mu), the number of pixels falling within the mask (M) and the image histogram array (hist) as arguments. As a result, it gives the threshold value. Variables attached to the foreground within the function are called class 1 , and variables attached to the background are called class 2. In the mentioned function:

[0046] In order to normalize the Mu value, the scale value is calculated as 1 / M.

[0047] Mu is normalized by multiplying it by scale.

[0048] Class 1 weighted mean (mu1) and probability of class 1 (q1) are set to 0. For comparison purposes, the maximum sigma value (maxSigma) and the maximum threshold value (maxVal) are set to 0.

[0049] In the next steps, the histogram array is repeated in loops as much as its size (256) and the number of loops is kept in the variable "i":

[0050] • Class 1 probability (pj) is calculated as the histogram value * scale at position i.

[0051] • The mu1 value is scaled by multiplying it by q1 to reflect the newly classified pixels.

[0052] • The p_i value is added to the q1 value.

[0053] • Probability of class 2 (q2) is calculated as the complement of q1 (1 - q1).

[0054] • Weighted mean is calculated for both classes; the value of mu1 is changed to (mu1 + i * pj) I q1 , mu2 is created with the value of (mu - q1 * mu1) I q2.

[0055] • The sigma value is calculated as q1 * q2 * (mu1 - mu2) * (mu1 - mu2).

[0056] • If the calculated sigma value is greater than maxSigma, maxSigma is updated and maxVal is replaced with i.

[0057] When the loop is completed, the maxVal value is returned.

[0058] The following steps are applied to measure the loss value of the image taken as argument:

[0059] • A Matlb variable named Processed is created with the dimensions of hostSource.

[0060] • The width of the image is divided by the globally held chunkSize and the resulting number is rounded up to obtain the width division number (divideByX).

[0061] • The length of the image is divided by the globally held chunkSize and the resulting number is rounded up to obtain the length division number (divideByY).

[0062] • If divideByX and divideByY are 0, they are changed to 1.

[0063] • The width of the Processed Matlb variable is divided by the divideByX number and rounded down to calculate the part width (xLen).

[0064] • The part length (yLen) is calculated by dividing the length of the Processed Matlb variable by the divideByY number and rounding down.

[0065] The next steps are performed in nested loops, where the number of the outer loop is kept in the variable “i” and the number of the inner loop is kept in the variable “j”. The outer loop “i” is repeated as many times as divideByX, while the inner loop “j” is repeated as many times as divideByY:

[0066] • The starting point of the part on the x-axis is calculated as i * xLen (beginX). • The end point of the part on the x-axis is calculated as xLen if the loop is not the last outer loop, and as the width of the processed Matlb variable - is calculated as beginX if it is the last outer loop.

[0067] • The starting point of the part on the y-axis is calculated as j * yLen (beginY).

[0068] • The end point of the part on the y-axis is calculated as yLen if the loop is not the last inner loop, and as the length of the processed Matlb variable - is calculated as beginY if it is the last inner loop.

[0069] • OpenCV Rect value is created at the beginX, beginY, endX, endY coordinates that determine the region of the part (region of interest, chunkRoi).

[0070] • An OpenCV Mat variable named Hist is created and the histogram value in the hostSource's chunkRoi is written to this variable using the OpenCV calcHist function and the chunkRoi region of the Mask.

[0071] • The total value of the histogram is calculated with the OpenCV sum function (M).

[0072] • If M is 0, the loop continues.

[0073] • The mean intensity value (mu) is calculated; hist is iterated and the histogram value * iteration number is added to mu at each iteration.

[0074] • The calculateThresh function is called using M, mu, and hist as arguments and its result is stored (maxVal).

[0075] • With the OpenCV threshold function, the chunkRoi region of the hostSource is written to the chunkRoi region of the processed using the maxVal threshold - 4.

[0076] At the end of the loop, processed and mask are combined with bitwise_and and the area outside the mask is turned black.

[0077] The number of non-zero pixels in the processed variable is calculated as glandArea with the OpenCV countNonZero function and the gland loss rate is calculated as 1 - (glandArea / eyelidArea) and the processed and gland loss rate are returned.

[0078] With the help of the meibomian gland image analysis method developed with the present invention, detailed imaging of the glands within the eyelid is provided in the diagnosis and treatment of meibomian gland disease, removal of image areas containing blur and reflection that cause errors, and clarification and analysis of the images obtained for disease diagnosis. In addition, by separating the resulting image masks into multiple analysis areas and layers, an effective, practical and reliable meibomian gland image analysis method is obtained, which enables the refinement of image analysis and meibomian gland identification processes and the increase of disease diagnosis accuracy by comparing the analysis results.

Claims

CLAIMS1. A meibomian gland image analysis method which enables the selection of the measurement area on the eyelid image, makes changes such as deletion, addition, ignoring, repositioning without cancelling the selected area, and eliminates image defects affecting the measurement, wherein the method is suitable for use in an analysis system comprising at least one processing unit providing digital calculation and process control, at least one image processing unit providing image processing and analysis, at least one display unit providing image transmission to the user, at least one input unit providing information input and editing, and at least one storage unit providing information recording, comprising the steps of:- recording the eyelid image via the storage unit and creating an image mask with all pixel values black / RGB (0) by selecting the area on the eyelid image to be measured by the user;- obtaining the histogram of pixel distribution by determining the pixel values of the image mask by means of the image processing unit, converting the pixel distribution intensities of the image mask into a graph, and increasing the contrast value of the image mask by processing the image so as to expand the pixel distribution density range in the graph;- converting the pixel value of the area selected on the image mask by the user via the the input unit to a first color other than black and determining the first mask area to be entered into the calculation in the meibomian gland identification process;- converting the pixel value of the area selected on the image mask by the user via the input unit to the first color or a second color different from the first color and determining the second mask area to be excluded from the calculation in the meibomian gland identification process;- converting the pixel value of the area selected on the image mask by the user, via the input unit, to the first or second color or a third color different from the second color and determining the third mask area to be ignored in the meibomian gland identification process;- combining the first, second and third masks in accordance with the pixel positions and numbers by means of the processing unit, performing histogram analysisshowing the pixel intensity values graph of the image obtained by using the masking regional otsu threshold method and determining an optimal first threshold value in the histogram chart, converting the image into white-on-black form with pixel values above this threshold value at the top and pixel values below it at the bottom and obtaining the measurement mask that will be used to detect meibomian gland disease and that contains the actual measurement area created by subtracting the areas removed from the calculation and ignored from the calculation area.

2. A meibomian gland image analysis method according to claim 1 , wherein the processing unit is a processor suitable for running at least one operating system.

3. A meibomian gland image analysis method according to claim 1, wherein the image processing unit is a graphics card and / or processor suitable for running OpenCV.

4. A meibomian gland image analysis method according to claim 1 , wherein the imaging unit is a screen.

5. A meibomian gland image analysis method according to claim 1, wherein the input unit is in the form of a keyboard, controller and / or mouse.

6. A meibomian gland image analysis method according to claim 1, wherein the eyelid image is an infrared image.

7. A meibomian gland image analysis method according to claim 1, wherein the method comprising the steps of, determining the Gaussian distribution around the pixels of the image mask whose contrast value is increased by means of the image processing unit, determining the Gaussian mean value by averaging the determined Gaussian distribution, increasing the sharpness value of the image mask by equating the pixel values of the image mask to the Gaussian mean value.

8. A meibomian gland image analysis method according to claim 1, wherein the method comprising the step of saving the eyelid image and image mask by converting it to SDL_Texture or pixel data format.

9. A meibomian gland image analysis method according to claim 1, wherein the method comprising the step of converting the image mask into three separate masks with all pixel values being black / RGB (0) to create the first mask providing calculation, the second mask providing loss / deletion display and the third mask providing display of the ignored area, recording the same and then presenting to the user through the display unit.

10. A meibomian gland image analysis method according to claim 1, wherein the eyelid image is an infrared image.

11. A meibomian gland image analysis method according to claim 1, wherein the first color is green / RGB(0, 255,0).

12. A meibomian gland image analysis method according to claim 1 , wherein the second color is red / RGB(255,0,0).

13. A meibomian gland image analysis method according to claim 1, wherein the third color is blue / RGB(0, 0, 255).

14. A meibomian gland image analysis method according to claim 1, wherein the method comprising the step of dividing the first, second and third masks into multiple processing areas, and analysing each area within itself for meibomian gland identification and converting the same into a form suitable for image processing by means of the image processing unit.

15. A meibomian gland image analysis method according to claim 1, wherein the method comprising the step of performing histogram analysis showing the pixel intensity values graph of the mask images using the masking regional otsu threshold method by means of the processing unit of the first, second and third masks and by determining an optimal first threshold value in the histogram chart, converting the image into two separate images, preferably in Matlb format, positioned on top of each other, with the pixel values above this threshold value at the top and the pixel values below it at the bottom.

16. A meibomian gland image analysis method according to claim 1, wherein the white-on-black image is in Matlb format.