Dynamic lacrimal river parameter measurement method for dry eye assessment

Abstract

The invention relates to a dynamic lacrimal river parameter measurement method for dry eye assessment. The method comprises the following steps: acquiring an initial video of a user in a blinking process; wherein the initial video content comprises user eyes, and the first frame image of the initial video is collected after the user closes the eyes; determining a measured video segment from the initial video; segmenting a lower tear river in the measurement video clip through a deep learning model; smoothing the upper boundary and the lower boundary of the lower tear river obtained by segmentation through a polynomial fitting method; and evaluating the height of the lower lacrimal river according to the smoothed lower lacrimal river. According to the method, the height of the lower lacrimal river can be measured on the premise of not depending on operators, and the accuracy and repeatability of measurement are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to a dynamic tear meniscus parameter measurement method for dry eye assessment. Background Art

[0002] Dry eye disease is a common chronic ocular surface disease characterized by an imbalance in tear film homeostasis and related ocular surface symptoms.

[0003] Existing methods for measuring Tear Meniscus Height (TMH) are divided into invasive and non-invasive methods, but these methods are usually operator-dependent, have poor repeatability, and may lead to erroneous results. Summary of the invention

[0004] 1. Technical issues to be resolved

[0005] In order to solve the above problems, the present invention provides a dynamic tear meniscus parameter measurement method for dry eye assessment.

[0006] (II) Technical solution

[0007] In order to achieve the above object, the main technical solutions adopted by the present invention include:

[0008] A method for measuring dynamic tear meniscus parameters for dry eye assessment, the method comprising:

[0009] Collecting an initial video of a user's blinking process; wherein the initial video includes the user's eyes, and the first frame image of the initial video is collected within the first time period after the user closes his eyes;

[0010] determining a measurement video segment from the initial video;

[0011] The tear river in the measured video clips was segmented using a deep learning model;

[0012] The upper and lower boundaries of the tear river obtained by segmentation are smoothed by polynomial fitting method;

[0013] Based on the smoothed tear meniscus, the tear meniscus height TMH is evaluated.

[0014] Optionally, determining a measurement video segment from the initial video includes:

[0015] Starting from the first frame of the initial video, capture the closed-eye video segment;

[0016] In the closed-eye video segment, the last frame image that clearly contains the eyes is determined as the first frame image of the measurement video segment;

[0017] Starting from the first frame image of the measurement video segment, multiple frame images in the initial video are continuously intercepted backward to form the measurement video segment.

[0018] Optionally, starting from the first frame image of the initial video, a closed-eye video segment is captured, including:

[0019] The initial image is determined as the current comparison image, and the second frame image of the initial video is selected as the current processing image;

[0020] Determine the minimum pixel coordinates for comparison based on the current comparison image Compare outer pixel coordinates and compare the inner pixel coordinates Identify the current minimum pixel coordinates in the current processed image Anterior lateral pixel coordinates and the anterior medial pixel coordinates The origin of the coordinate system is the upper left corner of the image, the positive direction of the horizontal axis is horizontal to the right, and the positive direction of the vertical axis is vertically downward; the minimum pixel coordinate is the pixel coordinate with the largest vertical coordinate at the lower boundary of the upper eyelid, the outer pixel coordinate is the pixel coordinate corresponding to the outer corner of the eye, and the inner pixel coordinate is the pixel coordinate corresponding to the inner corner of the eye;

[0021] Sure and The distance between and The distance between as well as and The distance between A first angle between the first straight line and the second straight line The second angle between the third straight line and the fourth straight line in, and Located on the first straight line,

[0022] and Located on the second straight line, and Located on the third straight line, and Located on the fourth straight line;

[0023] according to determining a comparison value;

[0024] like If the comparison value is 0, it is determined that the current processed image is the end frame image of the closed-eye video segment, otherwise it is determined that the current processed image is not the end frame image of the closed-eye video segment;

[0025] If the current processing image is not the end frame image of the closed-eye video clip, the current processing image and the current comparison image are both used as the current comparison image, and the next frame image of the current processing image is used as the current processing image, and the minimum pixel coordinates of the comparison are determined according to the current comparison image again. if the currently processed image is the end frame image of the closed-eye video segment, all frame images starting from the first frame of the initial video to the end frame image of the closed-eye video segment form a closed-eye video segment.

[0026] Optionally, the minimum pixel coordinates for comparison are determined based on the current comparison image. Compare outer pixel coordinates and compare the inner pixel coordinates include:

[0027] Determine the minimum pixel coordinates in each current comparison image Outer pixel coordinates and the inner pixel coordinates Wherein, i is the current comparison image identifier;

[0028] Calculate a first difference between the minimum pixel coordinates of any two currently compared images, a second difference between the outer pixel coordinates, and a third difference between the inner pixel coordinates;

[0029] Determine a minimum first difference among all the first differences, determine a minimum second difference among all the second differences, and determine a minimum third difference among all the third differences;

[0030] The center coordinates of the triangle formed by the minimum pixel coordinates in the two current comparison images and the initial image corresponding to the minimum first difference are determined as The center coordinates of the triangle formed by the minimum pixel coordinates in the two current comparison images and the initial image corresponding to the minimum second difference are determined as The center coordinates of the triangle formed by the minimum pixel coordinates in the two current comparison images and the initial image corresponding to the minimum third difference are determined as

[0031] Optionally, according to Determine the comparison value, including:

[0032] Determine distance deviation and angular deviation

[0033] The alignment value was determined based on DE and DD.

[0034] Optionally, determining a comparison value based on DE and DD includes:

[0035] Determine the alignment value as (1+max{DE, DD})×Pe0;

[0036] Where Pe0 is the pixel difference threshold, and max{·} is the maximum value function.

[0037] Optionally, determining a comparison value based on DE and DD includes:

[0038] Determine the comparison value

[0039] Where Pe0 is the pixel difference threshold, and max{·} is the maximum value function.

[0040] Optionally, the fitting formula used by the polynomial fitting method is: z(x)=w0+w1x+w2x 2 +…+w N x N ;

[0041] Among them, w0, w1, w2, …, w N are polynomial coefficients, N is the preset polynomial order, and x is the polynomial variable.

[0042] Optionally, based on the smoothed tear manifold, evaluating the tear manifold height TMH includes:

[0043]

[0044] Where M is the pixel-millimeter magnification, D is the tear river length, e up For the upper boundary of the River of Tears, e low It is the lower boundary of the River of Tears.

[0045] Optionally, after segmenting the tear river in the measured video clip by a deep learning model, the method further includes:

[0046] Determination of TMH area

[0047] Among them, P is the total number of pixels of the tear river mask.

[0048] (III) Beneficial effects

[0049] The present invention relates to a dynamic tear river parameter measurement method for dry eye assessment, the method comprising: collecting an initial video of a user's blinking process; wherein the initial video includes the user's eyes, and the first frame image of the initial video is collected within the first time period after the user closes his eyes; determining a measurement video segment from the initial video; segmenting the tear river in the measurement video segment through a deep learning model; smoothing the upper and lower boundaries of the segmented tear river through a polynomial fitting method; and assessing the tear river height based on the smoothed tear river. The method of the present invention can measure the tear river height without relying on an operator, thereby improving the accuracy and repeatability of the measurement. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 A schematic diagram of a flow chart of a method for measuring dynamic tear meniscus parameters for dry eye assessment provided in one embodiment of the present application;

[0051] Figure 2 A schematic diagram of a coordinate system provided for an embodiment of the present application;

[0052] Figure 3 A schematic diagram of the upper eyelid provided in one embodiment of the present application;

[0053] Figure 4 A schematic diagram of a measurement video clip provided in an embodiment of the present application;

[0054] Figure 5 A schematic diagram of tear river segmentation provided in an embodiment of the present application;

[0055] Figure 6 A schematic diagram of the tear river polynomial fitting provided in an embodiment of the present application. DETAILED DESCRIPTION

[0056] In order to better explain the present invention and facilitate understanding, the present invention is described in detail below through specific implementation modes in conjunction with the accompanying drawings.

[0057] Existing methods for measuring Tear Meniscus Height (TMH) are divided into invasive and non-invasive methods, but these methods are usually operator-dependent, have poor repeatability, and may lead to erroneous results.

[0058] In order to solve this problem, the present invention relates to a dynamic tear river parameter measurement method for dry eye assessment, the method comprising: collecting an initial video of a user's blinking process; wherein the initial video includes the user's eyes, and the first frame image of the initial video is collected within the first time period after the user closes his eyes; determining a measurement video segment from the initial video; segmenting the tear river in the measurement video segment through a deep learning model; smoothing the upper and lower boundaries of the segmented tear river through a polynomial fitting method; and assessing the tear river height based on the smoothed tear river. The method of the present invention can measure the tear river height without relying on an operator, thereby improving the accuracy and repeatability of the measurement.

[0059] See also Figure 1 This embodiment provides a dynamic tear meniscus parameter measurement method for dry eye assessment. The implementation process of the method is as follows:

[0060] 101, collect an initial video of a user's blinking process.

[0061] The initial video includes the user's eyes, and the first frame image of the initial video is collected within the first time period after the user closes his eyes.

[0062] For example, the first duration is 5 seconds. That is, step 101 starts collecting the initial video within 5 seconds after the user closes his eyes. During this process, the user opens his eyes and keeps them open as much as possible, and then closes his eyes. The initial video is the video from 5 seconds after the user closes his eyes to the time when the user closes his eyes again. During the process of the user collecting the initial video, the user remains still to ensure that the position of the user's eyes in each frame of the initial video remains unchanged and is comparable.

[0063] 102, determining a measurement video segment from an initial video.

[0064] The implementation process of this step is:

[0065] 102-1, starting from the first frame image of the initial video, capture the closed-eye video segment.

[0066] Because the initial video is the entire process from the user closing eyes to opening eyes and then closing eyes again, the closed-eye video segment is the video from the user closing eyes to opening eyes. The implementation process of step 102-1 is shown in steps 201 to 206.

[0067] 201 , determining an initial image as a current comparison image, and selecting a second frame image of the initial video as a current processing image.

[0068] 202, determine the minimum pixel coordinates for comparison according to the current comparison image Compare outer pixel coordinates and compare the inner pixel coordinates Identify the current minimum pixel coordinates in the current processed image Anterior lateral pixel coordinates and the anterior medial pixel coordinates

[0069] The origin of the coordinate system is the upper left corner of the image, the positive direction of the horizontal axis is horizontal to the right, and the positive direction of the vertical axis is vertically downward. Figure 2 The minimum pixel coordinate is the pixel coordinate of the lower boundary of the upper eyelid with the largest vertical coordinate, the outer pixel coordinate is the pixel coordinate corresponding to the outer corner of the eye, and the inner pixel coordinate is the pixel coordinate corresponding to the inner corner of the eye.

[0070] That is, in step 202, the lowest point of the lower boundary of the upper eyelid in each frame of the current comparison image and the current processing image (such as Figure 3 Point 1), outer corner of eye (such as Figure 3 Point 2) and the inner corner of the eye (such as Figure 3 The coordinates corresponding to point 3) in .

[0071] If there is only one current comparison image, the pixel coordinate of the maximum ordinate of the lower boundary of the upper eyelid in the current comparison image is The pixel coordinates corresponding to the outer corner of the eye are The pixel coordinates corresponding to the inner corner of the eye are

[0072] If there are multiple images to be compared, the minimum pixel coordinates for comparison are determined by the following steps: Compare outer pixel coordinates and compare the inner pixel coordinates

[0073] 1. Determine the minimum pixel coordinates in each current comparison image Outer pixel coordinates and the inner pixel coordinates

[0074] Wherein, i is the current comparison image identifier.

[0075] 2. Calculate the first difference between the minimum pixel coordinates of any two currently compared images, the second difference between the outer pixel coordinates, and the third difference between the inner pixel coordinates.

[0076] For example, there are three current comparison images, and the first difference, second difference, and third difference between the first current comparison image and the second current comparison image are determined in 2. The first difference, second difference, and third difference between the first current comparison image and the third current comparison image are determined. The first difference, second difference, and third difference between the second current comparison image and the third current comparison image are determined.

[0077] 3. Determine the minimum first difference among all the first differences, determine the minimum second difference among all the second differences, and determine the minimum third difference among all the third differences.

[0078] 4. The center coordinates of the triangle formed by the minimum pixel coordinates in the two current comparison images and the initial image corresponding to the minimum first difference are determined as The center coordinates of the triangle formed by the minimum pixel coordinates in the two current comparison images and the initial image corresponding to the minimum second difference are determined as The center coordinates of the triangle formed by the minimum pixel coordinates in the two current comparison images and the initial image corresponding to the minimum third difference are determined as

[0079] It should be noted that, since the positive direction of the vertical axis of the coordinate system is downward, the vertical coordinate corresponding to the lowest point of the lower boundary of the upper eyelid in the image is the largest, so the coordinates of the pixel point with the largest vertical coordinate of the lower boundary of the upper eyelid are determined as the minimum pixel coordinates.

[0080] 203, OK and The distance between and The distance between as well as and The distance between A first angle between the first straight line and the second straight line The second angle between the third straight line and the fourth straight line in, and Located on the first straight line, and Located on the second straight line, and Located on the third straight line, and Located on the fourth straight line.

[0081] That is to say, and Form the first straight line, and Form a second straight line, and Forming the third straight line, and A fourth straight line is formed.

[0082] 204, according to Determine the comparison value.

[0083] The execution process of step 204 is: determining the distance deviation and angular deviation The alignment value was determined based on DE and DD.

[0084] There are many implementation processes for determining the comparison value based on DE and DD. For example, the comparison value is determined as (1+max{DE, DD})×Pe0. Or, the comparison value is determined as

[0085] Where Pe0 is the pixel difference threshold, and max{·} is the maximum value function.

[0086] 205, if If the comparison value is 0, it is determined that the current processed image is the end frame image of the closed-eye video segment, otherwise it is determined that the current processed image is not the end frame image of the closed-eye video segment.

[0087] Since the user remains motionless during the acquisition of the initial video in step 1, the coordinates of the pixel point with the largest ordinate at the lower boundary of the upper eyelid will change, but the coordinates of the pixel points corresponding to the outer corner of the eye and the inner corner of the eye will not change. However, due to the inevitable jitter of the user and the deviation of the acquisition device itself, the coordinates of the pixel points corresponding to the outer corner of the eye and the inner corner of the eye may change slightly, which can be considered as a system error. The system error is characterized by the point deviation and line deviation of the outer and inner canthus (ie, DE is the point deviation and DD is the line deviation). The comparison value is the final system deviation obtained by combining the point deviation and the line deviation.

[0088] It represents the change in the coordinates of the pixel point with the largest ordinate at the lower boundary of the upper eyelid. If the change is greater than the comparison value, it means that the change in the upper eyelid is not caused by a system error. It is considered that the user starts to open his eyes and ends to close his eyes. Therefore, the currently processed image is the end frame image of the closed-eye video clip.

[0089] 206, if the current processing image is not the end frame image of the closed-eye video clip, the current processing image and the current comparison image are both used as the current comparison image, and the next frame image of the current processing image is used as the current processing image, and the determination of the minimum pixel coordinates for comparison based on the current comparison image is re-executed. and subsequent steps (i.e., repeatedly executing step 202 and subsequent steps) until a current processed image is obtained that is the end frame image of the eye-closed video segment. If the current processed image is the end frame image of the eye-closed video segment, all frame images starting from the first frame of the initial video to the end frame image of the eye-closed video segment form the eye-closed video segment.

[0090] 102-2, in the closed-eye video segment, determine the last frame image that clearly contains the eyes as the first frame image of the measurement video segment.

[0091] Figure 4 Frame0 in the closed-eye video clip will clearly contain the last frame image of the eyes.

[0092] 102-3, starting from the first frame image of the measurement video segment, multiple frame images in the initial video are continuously intercepted backward to form the measurement video segment.

[0093] For example, the multiple frames are 14 frames. That is to say, the measured video segment is composed of 14 frames starting from the last frame image of the clear eye collected after the eyes are closed. Figure 4 As shown, the first frame image of the measurement video segment is Frame0, and together with the subsequent 14 frames (ie, Frame1 to Frame14), a total of 15 frames of continuous images of the measurement video segment are obtained.

[0094] 103,segmentation of tear rivers in measured video clips via deep learning models.

[0095] This step can be implemented using an existing deep learning model, which will not be described in detail here. The segmentation effect is as follows Figure 5 shown.

[0096] After executing step 103 to segment the tear river in the measured video clip using the deep learning model, effective parameters are also measured.

[0097] Among them, the effective parameters include but are not limited to: overall TMH of the tear meniscus, central, nasal and temporal TMH, average upper boundary curvature of TMH, TMH area, 1, 2, 3, 4 mm tear meniscus length, etc.

[0098] For example, determine the TMH area

[0099] Where P is the total number of pixels of the tear river mask, and M is the pixel-millimeter magnification.

[0100] 104, smoothing the upper and lower boundaries of the tear river obtained by segmentation by a polynomial fitting method.

[0101] The fitting formula used by the polynomial fitting method is: z(x)=w0+w1x+w2x 2 +…+w N x N .

[0102] Among them, w0, w1, w2, …, w N are polynomial coefficients, N is the preset polynomial order, and x is the polynomial variable.

[0103] For example, a linear fit is performed on the effective parameters using the above fitting formula, and the slope of the linear fit is calculated, thereby achieving smoothing processing.

[0104] The details of fitting and smoothing can be implemented using the existing process, which will not be described in detail here. The fitting effect is as follows: Figure 6 shown.

[0105] 105, based on the smoothed tear meniscus, the tear meniscus height TMH is evaluated.

[0106] For example, assessing the height of the tear mantle

[0107] Where M is the pixel-millimeter magnification, D is the tear river length, e up For the upper boundary of the River of Tears, e low It is the lower boundary of the River of Tears.

[0108] In addition, after executing step 104, the differences in the slopes of the dynamic parameters between different dry eye disease groups may be statistically analyzed and compared to assess the presence and severity of dry eye disease.

[0109] This embodiment relates to a dynamic tear mane parameter measurement method for dry eye assessment, which collects an initial video of a user's blinking process; wherein the initial video includes the user's eyes, and the first frame image of the initial video is collected within the first time period after the user closes his eyes; a measurement video segment is determined from the initial video; the tear mane in the measurement video segment is segmented by a deep learning model; the upper and lower boundaries of the segmented tear mane are smoothed by a polynomial fitting method; and the tear mane height is evaluated based on the smoothed tear mane. This method can measure the tear mane height without relying on an operator, thereby improving the accuracy and repeatability of the measurement.

[0110] It should be clear that the present invention is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present invention.

[0111] It should also be noted that the exemplary embodiments mentioned in the present invention describe some methods or systems based on a series of steps or devices. However, the present invention is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiments, or in a different order from the embodiments, or several steps can be performed simultaneously.

[0112] Finally, it should be noted that the above-described embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some or all of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for measuring dynamic tear meniscus parameters for dry eye assessment, characterized in that: The method comprises: Collecting an initial video of a user's blinking process; wherein the initial video includes the user's eyes, and the first frame image of the initial video is collected after the user closes his eyes; determining a measurement video segment from the initial video; Segmenting the Lower Tear River in the measurement video clip by a deep learning model; The upper and lower boundaries of the segmented Lower Tear River were smoothed by polynomial fitting method; Based on the smoothed tear meniscus, the tear meniscus height TMH is evaluated.

2. The method according to claim 1, characterized in that Determining a measurement video segment from the initial video includes: Starting from the first frame image of the initial video, capturing a closed-eye video segment; Determine the last frame image clearly containing the eyes in the closed-eye video segment as the first frame image of the measurement video segment; Starting from the first frame image of the measurement video segment, multiple frame images in the initial video are continuously intercepted backward to form a measurement video segment.

3. The method according to claim 2, characterized in that Starting from the first frame image of the initial video, a closed-eye video segment is captured, including: Determine the initial image as the current comparison image, and select the second frame image of the initial video as the current processing image; Determine the minimum pixel coordinate for comparison according to the current comparison image Compare outer pixel coordinates and compare the inner pixel coordinates Identify the current minimum pixel coordinates in the current processed image Anterior lateral pixel coordinates and the anterior medial pixel coordinates The origin of the coordinate system is the upper left corner of the image, the positive direction of the horizontal axis is horizontal to the right, and the positive direction of the vertical axis is vertically downward; the minimum pixel coordinate is the pixel coordinate with the largest vertical coordinate at the lower boundary of the upper eyelid, the outer pixel coordinate is the pixel coordinate corresponding to the outer corner of the eye, and the inner pixel coordinate is the pixel coordinate corresponding to the inner corner of the eye; Sure and The distance between and The distance between as well as and The distance between A first angle between the first straight line and the second straight line The second angle between the third straight line and the fourth straight line in, and Located on the first straight line, and Located on the second straight line, and Located on the third straight line, and Located on the fourth straight line; according to determining a comparison value; like then determine that the current processed image is the end frame image of the closed-eye video segment, otherwise determine that the current processed image is not the end frame image of the closed-eye video segment; If the current processing image is not the end frame image of the closed-eye video clip, the current processing image and the current comparison image are both used as the current comparison image, and the next frame image of the current processing image is used as the current processing image, and the minimum pixel coordinates of the comparison are determined according to the current comparison image again. if the currently processed image is the end frame image of the closed-eye video segment, all frame images starting from the first frame of the initial video to the end frame image of the closed-eye video segment form a closed-eye video segment.

4. The method according to claim 3, characterized in that Determining the minimum pixel coordinates for comparison according to the current comparison image Compare outer pixel coordinates and compare the inner pixel coordinates include: Determine the minimum pixel coordinates in each current comparison image Outer pixel coordinates and the inner pixel coordinates Wherein, i is the current comparison image identifier; Calculate a first difference between the minimum pixel coordinates of any two currently compared images, a second difference between the outer pixel coordinates, and a third difference between the inner pixel coordinates; Determine a minimum first difference among all the first differences, determine a minimum second difference among all the second differences, and determine a minimum third difference among all the third differences; The center coordinates of the triangle formed by the minimum pixel coordinates in the two current comparison images corresponding to the minimum first difference and the initial image are determined as The center coordinates of the triangle formed by the minimum pixel coordinates in the two current comparison images corresponding to the minimum second difference and the initial image are determined as The center coordinates of the triangle formed by the minimum pixel coordinates in the two current comparison images corresponding to the minimum third difference and the initial image are determined as 5. The method according to claim 3, characterized in that: The basis Determine the comparison value, including: Determine distance deviation and angular deviation The alignment value was determined based on DE and DD.

6. The method according to claim 5, characterized in that Determining the comparison value according to DE and DD includes: Determine the alignment value as (1+max{DE, DD})×Pe0; Where Pe0 is the pixel difference threshold, and max{·} is the maximum value function.

7. The method according to claim 5, characterized in that Determining the comparison value according to DE and DD includes: Determine the comparison value Where Pe0 is the pixel difference threshold, and max{·} is the maximum value function.

8. The method according to claim 1, characterized in that The fitting formula used in the polynomial fitting method is: z(x)=w0+w1x+w2x 2 +…+w N x N ; Among them, w0,w1,w2,…,w N are polynomial coefficients, N is the preset polynomial order, and x is the polynomial variable.

9. The method according to claim 1, characterized in that: The step of evaluating the tear meniscus height TMH according to the smoothed tear meniscus comprises: Assessing the height of the tear meniscus Where M is the pixel-millimeter magnification, D is the tear river length, and e up For the upper boundary of the River of Tears, e low It is the lower boundary of the River of Tears.

10. The method according to claim 1, characterized in that After segmenting the tear river in the measured video clip by the deep learning model, the method further includes: Determination of TMH area Among them, P is the total number of pixels of the tear river mask.

Images