A dynamic tear meniscus parameter measurement method for dry eye assessment

Abstract

The application relates to a dynamic tear river parameter measurement method for dry eye evaluation, which comprises the following steps: collecting an initial video of one-time blinking of a user; wherein the initial video content comprises the eyes of the user, and the first frame image of the initial video is collected after the user closes the eyes; determining a measurement video segment from the initial video; segmenting a lower tear river in the measurement video segment through a deep learning model; smoothing the upper boundary and the lower boundary of the segmented lower tear river through a polynomial fitting method; and evaluating the height of the lower tear river according to the smoothed lower tear river. The method can measure the height of the lower tear river without relying on an operator, and improves the accuracy and repeatability of the measurement.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical image processing, and in particular to a dynamic tear meniscus parameter measurement method for dry eye evaluation. BACKGROUND

[0002] Dry eye disease is a common chronic ocular surface disease characterized by tear film homeostasis imbalance and related ocular surface symptoms.

[0003] Existing tear meniscus height (TMH) measurement methods are divided into invasive and non-invasive methods, but these methods usually rely on operators, have poor repeatability, and can lead to false results. SUMMARY

[0004] (I) Technical problems to be solved

[0005] In order to solve the above problems, the present application provides a dynamic tear meniscus parameter measurement method for dry eye evaluation.

[0006] (II) Technical solutions

[0007] In order to achieve the above purpose, the main technical solutions adopted by the present application include:

[0008] A dynamic tear meniscus parameter measurement method for dry eye evaluation, the method comprising:

[0009] Collecting an initial video of a user's blink process; wherein the initial video includes the user's eyes, and the first frame image of the initial video is collected within the first time duration after the user closes his eyes;

[0010] Determining a measurement video segment from the initial video;

[0011] Segmenting the tear meniscus in the measurement video segment by a deep learning model;

[0012] Smoothing the upper and lower boundaries of the segmented tear meniscus by a polynomial fitting method;

[0013] According to the smoothed tear meniscus, evaluating the tear meniscus height TMH.

[0014] Optionally, determining the measurement video segment from the initial video comprises:

[0015] Starting from the first frame image of the initial video, intercepting the closed-eye video segment;

[0016] In the closed-eye video segment, the last frame image clearly containing the eye is determined as the first frame image of the measurement video segment;

[0017] Starting from the first frame image of the initial video, a plurality of frame images in the initial video are continuously intercepted in reverse direction to form the measurement video segment.

[0018] Optionally, starting from the first frame image of the initial video, the closed-eye video segment is intercepted, including:

[0019] The first frame image of the initial video is determined as a current comparison image, and the second frame image of the initial video is selected as a current processing image;

[0020] The comparison minimum pixel coordinates are determined according to the current comparison image , the comparison outer side pixel coordinates and the comparison inner side pixel coordinates ; the current minimum pixel coordinates , the previous outer side pixel coordinates and the previous inner side pixel coordinates in the current processing image are identified; wherein, the origin of the coordinate system where the coordinates are located is the upper left corner of the image, the positive direction of the horizontal axis is horizontal right, and the positive direction of the vertical axis is vertical down; the minimum pixel coordinates are the pixel point coordinates with the maximum vertical coordinate of the lower boundary of the upper eyelid, the outer side pixel coordinates are the pixel point coordinates corresponding to the outer corner of the eye, and the inner side pixel coordinates are the pixel point coordinates corresponding to the inner corner of the eye;

[0021] The distance between and , the distance between and , the distance between and , the first included angle between the first straight line and the second straight line , the second included angle between the third straight line and the fourth straight line ; wherein, and are located on the first straight line, and are located on the second straight line, and are located on the third straight line, and are located on the fourth straight line;

[0022] The comparison value is determined according to , , , ;

[0023] If the comparison value is , it is determined that the current processing image is the end frame image of the closed-eye video segment, otherwise it is determined that the current processing image is not the end frame image of the closed-eye video segment.

[0024] If the current processing image is not the end frame image of the closed-eye video segment, the current processing image and the current comparison image are both taken as the current comparison image, the next frame image of the current processing image is taken as the current processing image, and the step of determining the comparison minimum pixel coordinate according to the current comparison image and the subsequent steps are re-executed; if the current processing image is the end frame image of the closed-eye video segment, all frame images from the first frame of the initial video to the end frame image of the closed-eye video segment form the closed-eye video segment.

[0025] Optionally, the comparison minimum pixel coordinate , the comparison outside pixel coordinate and the comparison inside pixel coordinate are determined according to the current comparison image, and the method comprises the steps of:

[0026] determining the minimum pixel coordinate , the outside pixel coordinate and the inside pixel coordinate in each current comparison image respectively; wherein, is the current comparison image identifier;

[0027] calculating a first difference value between the minimum pixel coordinates of any two current comparison images, a second difference value between the outside pixel coordinates, and a third difference value between the inside pixel coordinates;

[0028] determining a minimum first difference value among all the first difference values, a minimum second difference value among all the second difference values, and a minimum third difference value among all the third difference values;

[0029] the center coordinate of the triangle formed by the two current comparison images corresponding to the minimum first difference value and the minimum pixel coordinate in the first frame of the initial video is determined as ; the center coordinate of the triangle formed by the two current comparison images corresponding to the minimum second difference value and the minimum pixel coordinate in the first frame of the initial video is determined as ; and the center coordinate of the triangle formed by the two current comparison images corresponding to the minimum third difference value and the minimum pixel coordinate in the first frame of the initial video is determined as .

[0030] Optionally, the comparison value is determined according to , , , , and the method comprises the steps of:

[0031] determining the distance deviation and the angle deviation .

[0032] ​According to and determine the comparison value.

[0033] Optionally, the comparison value is determined according to and , comprising:

[0034] determining the comparison value as .

[0035] wherein, is a pixel difference threshold value, is a maximum value function.

[0036] Optionally, the comparison value is determined according to and , comprising:

[0037] determining the comparison value as .

[0038] wherein, is a pixel difference threshold value, is a maximum value function.

[0039] Optionally, the fitting formula used by the polynomial fitting method is: .

[0040] wherein, is a polynomial coefficient, is a pre-set polynomial order, is a polynomial variable.

[0041] Optionally, the tear meniscus height TMH is evaluated according to the smoothed tear meniscus, comprising:

[0042] evaluating the tear meniscus height TMH .

[0043] wherein, is a pixel-millimeter magnification, is a tear meniscus length, is an upper boundary of the tear meniscus, is a lower boundary of the tear meniscus.

[0044] Optionally, after the tear meniscus in the measurement video segment is segmented by the deep learning model, further comprising:

[0045] determining the TMH area TMA .

[0046] wherein, is a total number of pixels of the tear meniscus mask.

[0047] (Three) beneficial effects

[0048] The present application relates to a kind of dynamic tear river parameter measurement methods for dry eye evaluation, the method comprises: the initial video of the process of one blink of user is collected;Wherein, initial video includes user's eye, the first frame image of initial video is collected in the first time duration after user closes eyes;Determine the measurement video segment from initial video;The tear river in measurement video segment is segmented by deep learning model;The upper boundary and lower boundary of tear river segmented are smoothed by polynomial fitting method;According to the tear river after smoothing, the height of tear river is evaluated.The method of the present application can measure the height of tear river without relying on operator, improve the accuracy and repeatability of measurement. BRIEF DESCRIPTION OF DRAWINGS

[0049] Figure 1 A flowchart of a dynamic tear river parameter measurement method for dry eye evaluation provided by an embodiment of the present application is shown.

[0050] Figure 2 A coordinate system diagram provided by an embodiment of the present application is shown.

[0051] Figure 3 An upper eyelid diagram provided by an embodiment of the present application is shown.

[0052] Figure 4 A measurement video segment diagram provided by an embodiment of the present application is shown.

[0053] Figure 5 A tear river segmentation diagram provided by an embodiment of the present application is shown.

[0054] Figure 6 A tear river polynomial fitting diagram provided by an embodiment of the present application is shown. DETAILED DESCRIPTION

[0055] In order to better explain the present application, so as to be understood, the present application is described in detail by specific embodiments, combined with the drawings.

[0056] Existing tear meniscus height (TMH) measurement methods are divided into invasive and non-invasive methods, but these methods usually rely on operators, have poor repeatability, and may lead to false results.

[0057] To solve the problem, the application relates to a dynamic tear river parameter measurement method for dry eye evaluation, which comprises the following steps: collecting an initial video of a user's one-time blinking process; wherein the initial video comprises the user's eyes, and the first frame image of the initial video is collected within a first time length after the user closes the eyes; determining a measurement video segment from the initial video; segmenting the tear river in the measurement video segment through a deep learning model; smoothing the upper boundary and the lower boundary of the segmented tear river through a polynomial fitting method; and evaluating the tear river height according to the smoothed tear river. The method can measure the tear river height without relying on an operator, and improves the measurement accuracy and repeatability.

[0058] Referring to Figure 1 The embodiment provides a dynamic tear river parameter measurement method for dry eye evaluation, and the implementation process of the method is as follows:

[0059] 101. Collecting an initial video of a user's one-time blinking process.

[0060] The initial video comprises the user's eyes, and the first frame image of the initial video is collected within a first time length after the user closes the eyes.

[0061] For example, the first time length is 5 seconds. That is, the step 101 starts the collection of the initial video within 5 seconds after the user closes the eyes, in this process, the user opens the eyes and maximally keeps the eyes open, and then closes the eyes. The initial video is the video from within 5 seconds after the user closes the eyes to the time when the user closes the eyes again. During the collection of the initial video, the user keeps still, so that the positions of the user's eyes in each frame of the initial video are unchanged and comparable.

[0062] 102. Determining a measurement video segment from the initial video.

[0063] The implementation process of the step is as follows:

[0064] 102-1. Starting from the first frame image of the initial video, a closed-eye video segment is intercepted.

[0065] Because the initial video is the whole process from the time when the user closes the eyes to the time when the user opens the eyes and then closes the eyes again, the closed-eye video segment is the video from the time when the user closes the eyes to the time when the user opens the eyes. The implementation process of the step 102-1 is shown in steps 201-206.

[0066] 201. Determining the first frame image of the initial video as a current comparison image, and selecting the second frame image of the initial video as a current processing image.

[0067] 202. Determining a comparison minimum pixel coordinate a comparison outer pixel coordinate and a comparison inner pixel coordinate The minimum pixel coordinate in the current processing image is identified The front-lateral pixel coordinate and the front-medial pixel coordinate .

[0068] Wherein, the origin of the coordinate system is the upper left corner of the image, the positive direction of the horizontal axis is horizontal right, and the positive direction of the vertical axis is vertical down, as shown in Figure 2 The minimum pixel coordinate is the pixel coordinate of the lower boundary of the upper eyelid with the maximum vertical coordinate, the front-lateral pixel coordinate is the pixel coordinate corresponding to the outer corner of the eye, and the front-medial pixel coordinate is the pixel coordinate corresponding to the inner corner of the eye.

[0069] That is, in step 202, the lowest point of the lower boundary of the upper eyelid (such as point 1 in Figure 3 ), the pixel coordinate corresponding to the outer corner of the eye (such as point 2 in Figure 3 ), and the pixel coordinate corresponding to the inner corner of the eye (such as point 3 in Figure 3 ) in each frame of the current comparison image and the current processing image are identified.

[0070] If there is only one current comparison image, the pixel coordinate of the lower boundary of the upper eyelid with the maximum vertical coordinate in the current comparison image is , the pixel coordinate corresponding to the outer corner of the eye is , and the pixel coordinate corresponding to the inner corner of the eye is .

[0071] If there are multiple current comparison images, the comparison minimum pixel coordinate , the comparison front-lateral pixel coordinate , and the comparison front-medial pixel coordinate are determined by the following steps:

[0072] 1. Determine the minimum pixel coordinate , the front-lateral pixel coordinate , and the front-medial pixel coordinate in each current comparison image, respectively.

[0073] Wherein, is the identification of the current comparison image.

[0074] 2. Calculate the first difference value between the minimum pixel coordinates of any two current comparison images, the second difference value between the front-lateral pixel coordinates, and the third difference value between the front-medial pixel coordinates.

[0075] For example, there are three current comparison images, and in 2, the first difference value, the second difference value, and the third difference value between the first current comparison image and the second current comparison image are determined. The first difference value, the second difference value, and the third difference value between the first current comparison image and the third current comparison image are determined. The first difference value, the second difference value, and the third difference value between the second current comparison image and the third current comparison image are determined.

[0076] 3. Determine the minimum first difference among all the first differences, the minimum second difference among all the second differences, and the minimum third difference among all the third differences.

[0077] 4. The center coordinate of the triangle formed by the two current compared images corresponding to the minimum first difference and the minimum pixel coordinate in the first frame image of the initial video is determined as The center coordinate of the triangle formed by the two current compared images corresponding to the minimum second difference and the minimum pixel coordinate in the first frame image of the initial video is determined as The center coordinate of the triangle formed by the two current compared images corresponding to the minimum third difference and the minimum pixel coordinate in the first frame image of the initial video is determined as .

[0078] It should be noted that since the positive direction of the longitudinal axis of the coordinate system is downward, the maximum longitudinal coordinate corresponds to the lowest point of the lower boundary of the upper eyelid in the image, so the pixel coordinate of the lower boundary of the upper eyelid with the maximum longitudinal coordinate is determined as the minimum pixel coordinate.

[0079] 203, determine the distance between and , the distance between and , and the distance between and , the first angle between the first straight line and the second straight line , the second angle between the third straight line and the fourth straight line . Among them, and are located on the first straight line, and are located on the second straight line, and are located on the third straight line, and are located on the fourth straight line.

[0080] That is, and form the first straight line, and form the second straight line, and form the third straight line, and form the fourth straight line.

[0081] 204, according to ,​ 、 、 determine the comparison value.

[0082] The execution process of step 204 is to determine the distance deviation and the angle deviation , and determine the comparison value according to and .

[0083] Wherein, the implementation process of determining the comparison value according to and is various, for example, determining the comparison value as . Or, determining the comparison value as .

[0084] Wherein, is the pixel difference threshold, is the maximum value function.

[0085] 205, if the comparison value, then determining the current processing image as the end frame image of the closed-eye video segment, otherwise determining the current processing image as the non-end frame image of the closed-eye video segment.

[0086] Since the user keeps still during the initial video acquisition process of step 1, the pixel point coordinate of the maximum lower boundary vertical coordinate of the upper eyelid will change, but the pixel point coordinates corresponding to the outer and inner corners of the eye will not change. However, due to the inevitable shaking of the user and the deviation of the acquisition device itself, the pixel point coordinates corresponding to the outer and inner corners of the eye may change a little, which can be considered as a system error. 、 、 、 The point deviation and line deviation of the outer and inner corners of the eye represent the system error (i.e. is the point deviation, is the line deviation), and the comparison value is the final system deviation obtained by comprehensively considering the point deviation and the line deviation.

[0087] represent the change of the pixel point coordinate of the maximum lower boundary vertical coordinate of the upper eyelid. If the change is greater than the comparison value, it means that the change of the upper eyelid is not caused by the system error, and it is considered that the user starts to open the eyes and the closed eyes end, so the current processing image is determined as the end frame image of the closed-eye video segment.

[0088] 206, if the current processing image is not the end frame image of the closed-eye video segment, then the current processing image and the current comparison image are both regarded as the current comparison image, the next frame image of the current processing image is regarded as the current processing image, and the step of determining the minimum pixel coordinates of the comparison according to the current comparison image and the subsequent steps (i.e., repeating step 202 and the subsequent steps) are re-executed until the current processing image is obtained as the end frame image of the closed-eye video segment. If the current processing image is the end frame image of the closed-eye video segment, then all frame images from the first frame of the initial video to the end frame image of the closed-eye video segment form the closed-eye video segment.

[0089] 102-2, in the closed-eye video segment, the last frame image clearly containing the eye is determined as the first frame image of the measurement video segment.

[0090] Figure 4 Frame0 in the closed-eye video segment, the last frame image clearly containing the eye.

[0091] 102-3, starting from the first frame image of the measurement video segment, a plurality of frame images in the initial video are continuously intercepted backward to form the measurement video segment.

[0092] For example, the plurality of frame images are 14 frame images. That is, the measurement video segment starts from the frame image of the last clear eye after the closed eye and is composed of 14 frame images backward. Figure 4 As shown in the figure, the first frame image of the measurement video segment is Frame0, and the subsequent 14 frame images (i.e., Frame1 to Frame14) are added to obtain a total of 15 continuous frame images of the measurement video segment.

[0093] 103, segmenting the tear river in the measurement video segment by using a deep learning model.

[0094] This step can be implemented by using an existing deep learning model, and the segmentation effect is not described in detail here, as shown in the figure. Figure 5

[0095] After performing step 103 of segmenting the tear river in the measurement video segment by using the deep learning model, the effective parameters are also measured.

[0096] The effective parameters include but are not limited to: the overall TMH of the tear river, the TMH of the center, the nasal side and the temporal side, the average upper boundary curvature of the TMH, the TMH area, the 1, 2, 3 and 4 millimeter tear river length, etc.

[0097] For example, the TMH area TMA is determined.

[0098] wherein, is the total number of pixels of the tear river mask,​​ pixel-millimeter magnification.

[0099] 104, smoothing the upper boundary and the lower boundary of the tear river obtained by segmentation by a polynomial fitting method.

[0100] wherein the fitting formula used by the polynomial fitting method is: .

[0101] wherein, is a polynomial coefficient, is a pre-set polynomial order, is a polynomial variable.

[0102] For example, by linear fitting the effective parameters by the fitting formula above, the slope of linear fitting is calculated, and then the smoothing processing is realized.

[0103] The details of fitting and smoothing processing can adopt the existing implementation process, which will not be described in detail here, and the fitting effect is shown in Figure 6 .

[0104] 105, evaluating the tear meniscus height TMH according to the smoothed tear river.

[0105] For example, evaluating the tear meniscus height TMH .

[0106] wherein, is a pixel-millimeter magnification, is a tear river length, is a tear river upper boundary, is a tear river lower boundary.

[0107] In addition, after step 104 is performed, the difference in dynamic parameter slope between different dry eye groups can also be statistically analyzed and compared to evaluate the existence and severity of dry eye.

[0108] The embodiment relates to a dynamic tear river parameter measurement method for dry eye evaluation, which collects an initial video of a user's one-time blinking process; wherein the initial video includes the user's eyes, and the first frame image of the initial video is collected within the first time length after the user closes the eyes; a measurement video segment is determined from the initial video; a tear river in the measurement video segment is segmented by a deep learning model; the upper boundary and the lower boundary of the tear river obtained by segmentation are smoothed by a polynomial fitting method; and the tear meniscus height is evaluated according to the smoothed tear river. The method can measure the tear meniscus height without relying on an operator, and improves the measurement accuracy and repeatability.

[0109] It is to be understood that the application is not limited to particular configurations and processes described herein and as shown in the figures. For the sake of brevity, conventional methods and apparatus will not be described in detail. In the above embodiments, numerous specific details are set forth in order to provide a thorough understanding of the present application. However, it will be apparent to one skilled in the art that the present application can be practiced without these specific details. In other instances, well-known methods have not been described in detail in order to avoid obscuring the present application. It will be appreciated that the scope of the application encompasses all solutions falling within the spirit and scope of the claims.

[0110] It is also to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting. It is also possible in the practice of the application to utilize various concepts, alternatives, equivalents, modifications, and / or additions to the details discussed and illustrated herein.

[0111] Finally, it should be noted that the above-described embodiments are merely exemplary of the application, and that they should not be taken as limiting the scope of the application. Although the present application has been described with reference to particular means, materials, and embodiments, from the information contained herein, one of ordinary skill in the art having benefit of this disclosure can effect equivalent means, materials, and embodiments without departing from the scope of the disclosed application. Accordingly, the application is not to be construed as limited to the specific forms shown as defining the scope of equivalence to be accorded as within the scope of the appended claims.

Claims

1. A dynamic tear meniscus parameter measurement method for dry eye assessment, characterized by, The method comprises: collecting an initial video of a user's one-time blinking process; wherein the initial video includes the user's eyes, and a first frame image of the initial video is collected after the user closes the eyes; determining a measurement video segment from the initial video; segmenting a lower tear river in the measurement video segment by a deep learning model; smoothing the upper boundary and the lower boundary of the segmented lower tear river by a polynomial fitting method; evaluating a tear river height TMH according to the smoothed tear river; determining the measurement video segment from the initial video comprises: starting from the first frame image of the initial video, intercepting a closed-eye video segment; determining the last frame image clearly containing the eyes in the closed-eye video segment as the first frame image of the measurement video segment; starting from the first frame image of the measurement video segment, continuously intercepting multiple frame images in the initial video to form the measurement video segment; starting from the first frame image of the initial video, intercepting a closed-eye video segment comprises: determining the first frame image of the initial video as a current comparison image, and selecting a second frame image of the initial video as a current processing image; determining a matching minimum pixel coordinate according to the current matching image , a matching outside pixel coordinate , and a matching inside pixel coordinate ; identifying a current minimum pixel coordinate in the current processing image , a current outside pixel coordinate , and a current inside pixel coordinate ; wherein the origin of the coordinate system is the upper left corner of the image, the positive direction of the horizontal axis is to the right horizontally, and the positive direction of the vertical axis is downward vertically; the minimum pixel coordinate is the pixel point coordinate with the maximum vertical coordinate of the lower boundary of the upper eyelid, the outside pixel coordinate is the pixel point coordinate corresponding to the outer corner of the eye, and the inside pixel coordinate is the pixel point coordinate corresponding to the inner corner of the eye; determining the distance between the distance between , the distance between the distance between and the distance between the distance between a first included angle between the first straight line and the second straight line a second included angle between the third straight line and the fourth straight line ; wherein and are located on the first straight line, and are located on the second straight line, and are located on the third straight line, and are located on the fourth straight line; According to , , , determining the alignment value; If If the comparison value is greater than 0, it is determined that the current processing image is an end frame image of the closed-eye video segment, otherwise it is determined that the current processing image is not an end frame image of the closed-eye video segment. If the currently processed image is not the end frame of a video segment with eyes closed, then both the currently processed image and the currently compared image are used as the current comparison image, and the next frame of the currently processed image is used as the current processed image. The process of determining the minimum pixel coordinates for comparison based on the current comparison image is then repeated. The steps and subsequent steps; if the currently processed image is the end frame image of the closed-eye video segment, then all frame images from the first frame of the initial video to the end frame image of the closed-eye video segment form the closed-eye video segment.

2. The method of claim 1, wherein, determining a matching minimum pixel coordinate according to the current matching image a matching outside pixel coordinate and a matching inside pixel coordinate comprising: determining minimum pixel coordinates in each current comparison image respectively , outer pixel coordinates and inner pixel coordinates ; wherein, is a current comparison image identifier; calculating a first difference value between the minimum pixel coordinates of any two current comparison images, a second difference value between the outer pixel coordinates, and a third difference value between the inner pixel coordinates; determining the minimum first difference value among all first difference values, the minimum second difference value among all second difference values, and the minimum third difference value among all third difference values; The center coordinate of a triangle formed by the two current comparison images corresponding to the minimum first difference and the minimum pixel coordinate in the first frame image of the initial video is determined as The center coordinate of a triangle formed by the two current comparison images corresponding to the minimum second difference and the minimum pixel coordinate in the first frame image of the initial video is determined as The center coordinate of a triangle formed by the two current comparison images corresponding to the minimum third difference and the minimum pixel coordinate in the first frame image of the initial video is determined as ​ 3. The method of claim 1, wherein, The according to , , , determining the comparison value comprises: determining a distance deviation and an angle deviation ; According to and determined alignment values.

4. The method of claim 3, wherein, The according to And Determining the alignment value comprises: determining the comparison value as ; wherein, is a pixel difference threshold value, is a max function.

Images