Tear film detection system and method based on video data

Abstract

The present disclosure describes a tear film detection system and method based on video data, which comprises an acquisition module configured to acquire a plurality of detection frames corresponding to video data of a tear film as a tear film data queue; a state analysis module configured to determine an eye state of a middle detection frame based on a change in shape and size of a corneal region in three consecutive detection frames in the tear film data queue; a detection segment analysis module configured to acquire a plurality of detection segments corresponding to the video data based on the eye state of the detection frame obtained by the state analysis module, and determine a detection result of each detection segment based on the eye state of the detection frame being open; and a result analysis module configured to determine a target detection result based on a plurality of detection results corresponding to a plurality of detection segments. Thus, the segmentation accuracy of tear film detection can be high, real-time analysis can be supported, and convenience can be achieved.

Description

Technical Field

[0001] This disclosure generally relates to intelligent medical systems, and more specifically to a tear film detection system and method based on video data. Background Technology

[0002] The tear film is a dynamic and complex structure, composed of an anterior lipid layer, a middle aqueous layer, and an inner mucous layer. The tear film structure prevents tear evaporation and provides a smooth optical surface for the cornea. Clinical studies have shown that research on tear film status (e.g., tear film stability) is of great significance. For example, tear film status can help assess some eye diseases (e.g., dry eye syndrome). Furthermore, tear film status can guide the wearing of some eye products (e.g., contact lenses).

[0003] Currently, two common methods are used to assess tear film status: one involves projecting a Placido ring (concentric rings) while the subject's eyes remain open and analyzing the image features of the rings to detect projection deformation and determine tear film status (e.g., tear film breakup time); the other is the fluorescein tear breakup time (FTBUT) test, which involves instilling a pre-set dose of sodium fluorescein into the eye and analyzing the time the tear film remains stable without blinking to determine tear film status. These two methods can generally be combined with video analysis technology to obtain tear film status. For example, patent document (CN111062443A) provides a deep learning-based tear film breakup time detection system. This system extracts all frames from a captured tear film breakup time examination video and inputs them into a classification model, outputting a classification label for each frame to form a 0 / 1 sequence; it extracts the longest open-eye segment in the sequence as the detection segment; and it inputs each detection segment into a dry spot segmentation model for segmentation prediction, ultimately obtaining the overall tear film breakup time of the subject.

[0004] However, the detection process based on Placido ring image analysis requires the subject to keep their eyes open continuously, and the detection results are easily affected by noise such as eyelashes in the anterior segment. The recording needs to be restarted after the subject blinks or makes a momentary eye movement, so the convenience needs to be improved. The classification model in the patent document (CN111062443A) directly identifies the opening and closing state, without considering the impact of the dynamic changes of the cornea on the classification accuracy, which in turn affects the segmentation accuracy of the detection segment. Summary of the Invention

[0005] This disclosure is made in view of the above-mentioned situation, and its purpose is to provide a tear film detection system and method based on video data that enables high segmentation accuracy of tear film detection, supports real-time analysis, and is convenient.

[0006] To this end, the first aspect of this disclosure provides a tear film detection system based on video data, comprising: an acquisition module, a state analysis module, a detection segment analysis module, and a result analysis module; the acquisition module is configured to acquire multiple detection frames corresponding to the video data for the tear film that change over time and to use them as a tear film data queue; the state analysis module is configured to receive a detection frame to be analyzed, acquire three consecutive detection frames in the middle of the detection frame to be analyzed in the tear film data queue, and determine the eye state of the detection frame to be analyzed based on the changes in the shape and size of the corneal region in the three detection frames, wherein the eye state includes an open state, an open-eye process state, and a closed-eye process state. In the process state, the shape and size of the corneal region are represented by relative size; the detection segment analysis module is configured to read detection frames in the tear film data queue, obtain multiple detection segments corresponding to the video data based on the eye state of the detection frames obtained by the state analysis module, and determine the detection result of each detection segment based on the detection frames where the eye state is open, wherein the start frame and end frame of each detection segment are determined by the detection frames where the eye state is open and the detection frames where the eye state is closed in the tear film data queue, respectively; and the result analysis module is configured to determine the target detection result based on the multiple detection results corresponding to the multiple detection segments.

[0007] In this disclosure, the eye state of the tear film in the video data is accurately determined based on the dynamic changes of the cornea. Multiple detection segments are then determined based on the eye state of the detection frames, and these segments are analyzed to obtain the target detection result. In this approach, the eye state determined based on the dynamic changes of the cornea enables high segmentation accuracy for tear film detection, and the more accurate segmentation supports real-time analysis and provides interpretability. Furthermore, the shape and size of the corneal region are represented using relative sizes, which reduces the negative impact of potential offsets between multiple detection frames on the changes, thus ensuring compatibility with various complex video data acquisition environments.

[0008] Additionally, in the system according to the first aspect of this disclosure, optionally, the video data includes at least one of video files and video buffer data, wherein the video data is acquired after staining the tear film with a contrast agent. This enables the analysis of non-real-time and / or real-time video data. Furthermore, the contrast agent can improve the contrast of images in the video data, which is beneficial for identifying the tear film state.

[0009] Additionally, in the system disclosed in the first aspect, optionally, the three detection frames sequentially include a previous frame, an intermediate frame, and a next frame, wherein the intermediate frame is the detection frame to be analyzed, and the state analysis module is configured to: if the change is that the shape and size of the corneal region in the previous frame is greater than a preset size, the shape and size of the corneal region in the intermediate frame is greater than the preset size, and the shape and size of the corneal region in the next frame is greater than the preset size, then the eye state of the intermediate frame is open; if the change is that the shape and size of the corneal region in the previous frame is less than the preset size, the shape and size of the corneal region in the intermediate frame is greater than the preset size, and the shape and size of the corneal region in the next frame is greater than the shape and size of the corneal region in the intermediate frame, then the eye state of the intermediate frame is in the process of opening eyes; if the change is that the shape and size of the corneal region in the previous frame is greater than the preset size, the shape and size of the corneal region in the intermediate frame is less than the shape and size of the corneal region in the previous frame, and the shape and size of the corneal region in the next frame is less than the preset size, then the eye state of the intermediate frame is in the process of closing eyes.

[0010] Additionally, in the system according to the first aspect of this disclosure, optionally, the relative size is the ratio of the size of the corneal region of the detection frame to the size of a reference object, wherein the reference object is at least one of the whole image corresponding to the detection frame and the region of interest. In this case, the negative impact of possible offsets between multiple detection frames on variations can be reduced, thereby enabling compatibility with various complex environments for acquiring video data.

[0011] Furthermore, in the system disclosed in the first aspect, optionally, the tear film data queue is a buffered data queue, and the detection segment analysis module is configured to: read a detection frame from the buffered data queue in chronological order and use it as the current frame; determine the eye state of the current frame based on the changes in the shape and size of the corneal region between the previous frame, the current frame, and the next frame; if the eye state of the current frame is an open-eye state, then use the current frame as the starting frame of the detection segment; if the eye state of the current frame is a closed-eye state, then use the current frame as the ending frame of the detection segment with a determined starting frame, thereby determining a detection segment; and update the current frame as the next detection frame in the buffered data queue and continue to make judgments based on the current frame until the buffered data queue is completely read. In this case, the eye state determined based on the dynamic changes of the cornea can obtain a more accurate detection segment in real time, and the more accurate detection segment can support real-time analysis.

[0012] Additionally, in the system disclosed in the first aspect, optionally, the detection result includes at least one of tear film breakup time and tear film breakup morphology, and the target detection result includes at least one of target tear film breakup time and target tear film breakup morphology.

[0013] In addition, in the system disclosed in the first aspect, optionally, if the eye state of a detection frame is open, it is determined whether the corneal region of the detection frame has ruptured. If the corneal region of the detection frame has ruptured, the tear film rupture time corresponding to the detection segment is calculated based on the position of the detection frame in the detection segment, and / or the tear film rupture morphology corresponding to the detection segment is obtained based on the detection frame.

[0014] Additionally, in the system disclosed in the first aspect, optionally, the result analysis module is configured to average the multiple tear film breakup times corresponding to the multiple detection segments to determine the target tear film breakup time.

[0015] Additionally, in the system disclosed in the first aspect, optionally, it further includes a pre-trained deep learning-based corneal region segmentation model, which is used to acquire the corneal region of the detection frame; and / or includes a pre-trained deep learning-based tear film breakage detection model, which is used to detect whether the corneal region of the detection frame has broken. In this case, the deep learning-based method can learn as much as possible the high-level semantic feature information of the corneal region in the video data through the annotation information of the training data, and can be compatible with video data under different acquisition conditions, thereby obtaining more accurate results.

[0016] The second aspect of this disclosure provides a tear film detection method based on video data, comprising: acquiring multiple detection frames corresponding to the video data for the tear film that change over time and using them as a tear film data queue; reading the detection frames in the tear film data queue; acquiring multiple detection segments corresponding to the video data based on the eye state of the detection frames; and determining the detection results of each detection segment based on the detection frames where the eye state is open, wherein the eye state includes open state, open-eye process state, and closed-eye process state; determining the start frame and end frame of each detection segment based on the detection frames in the tear film data queue where the eye state is open-eye process state and the detection frames where the eye state is closed-eye process state; in analyzing the eye state of the detection frames, acquiring three consecutive detection frames in the middle of the detection frame to be analyzed in the tear film data queue; determining the eye state of the detection frame to be analyzed based on the changes in the shape and size of the corneal region in the three detection frames, wherein the shape and size of the corneal region are represented by relative size; and determining a target detection result based on the multiple detection results corresponding to the multiple detection segments.

[0017] In this disclosure, the eye state of the tear film in the video data is accurately determined based on the dynamic changes of the cornea. Multiple detection segments are then determined based on the eye state of the detection frames, and these segments are analyzed to obtain the target detection result. In this approach, the eye state determined based on the dynamic changes of the cornea enables high segmentation accuracy for tear film detection, and the more accurate segmentation supports real-time analysis and provides interpretability. Furthermore, the shape and size of the corneal region are represented using relative sizes, which reduces the negative impact of potential offsets between multiple detection frames on the changes, thus ensuring compatibility with various complex video data acquisition environments.

[0018] According to this disclosure, a tear film detection system and method based on video data is provided, which enables high segmentation accuracy of tear film detection, supports real-time analysis, and is convenient. Attached Figure Description

[0019] This disclosure will now be explained in further detail by way of example only with reference to the accompanying drawings, in which:

[0020] Figure 1 This is an exemplary schematic diagram illustrating the detection environment involved in the examples of this disclosure.

[0021] Figure 2 This is an exemplary block diagram illustrating a detection system as described in this disclosure.

[0022] Figure 3 This is an exemplary flowchart illustrating the determination of eye state for three consecutive detection frames as described in this disclosure.

[0023] Figure 4A This is a schematic diagram showing a related image of a detection frame in an open and unbroken state, as described in the examples of this disclosure.

[0024] Figure 4B This is a schematic diagram showing a related image of a detection frame that is in an open state and has been broken, as described in the example of this disclosure.

[0025] Figure 4C This is a schematic diagram showing a related image of a detection frame that illustrates the closing process state involved in the example of this disclosure and where the breakage state cannot be identified.

[0026] Figure 5 This is an exemplary flowchart illustrating the detection segment recognition involved in the examples of this disclosure.

[0027] Figure 6 This is an exemplary flowchart illustrating a training method for a corneal region segmentation model as described in this disclosure.

[0028] Figure 7This is an exemplary flowchart illustrating a training method for a tear film breakage detection model as described in this disclosure.

[0029] Figure 8 This is an exemplary flowchart illustrating a tear film detection method based on video data as described in this disclosure. Detailed Implementation

[0030] The preferred embodiments of this disclosure are described in detail below with reference to the accompanying drawings. In the following description, the same reference numerals are used for the same components, and repeated descriptions are omitted. Furthermore, the drawings are merely schematic diagrams, and the proportions of the components or the shapes of the components may differ from actual figures. It should be noted that the terms "comprising" and "having," and any variations thereof, in this disclosure, do not necessarily limit the process, method, system, product, or apparatus to the explicitly listed steps or units, but may include or have other steps or units not explicitly listed or inherent to these processes, methods, products, or apparatuses. All methods described in this disclosure may be performed in any suitable order unless otherwise indicated herein or clearly contradicted by the context.

[0031] As mentioned above, existing technologies have some limitations. Furthermore, the detection process based on Placido ring image analysis, due to the projection of a ring, is not conducive to capturing the morphology of the tear film itself. Existing methods generally divide the process into determining the start time and the breakup time, without considering dynamic judgment conditions, thus failing to support real-time analysis. Other methods employ post-processing, i.e., after obtaining video data, uniform preprocessing is performed to initially obtain detection segments before further analysis to filter detection segments; these methods also cannot support real-time analysis.

[0032] To address all or part of the aforementioned problems, this disclosure proposes a scheme for identifying detection segments and / or tear film detection. This scheme enables high segmentation accuracy in tear film detection, supports real-time analysis, and is convenient. Specifically, the scheme can accurately determine the eye state of the detection frame in the tear film video data based on the dynamic changes of the cornea, determine multiple detection segments based on the eye state of the detection frame, and then analyze these multiple detection segments to obtain target detection results. In this case, the eye state determined based on the dynamic changes of the cornea enables high segmentation accuracy in tear film detection, and the more accurate segmentation supports real-time analysis. Furthermore, the scheme involved in this disclosure supports real-time analysis, thereby enabling timely acquisition of detection results for each detection segment and / or target detection results.

[0033] Furthermore, the scheme described in this disclosure is more convenient and easier to implement than the detection process based on Placido ring image analysis. The scheme described in this disclosure can employ a lightweight deep learning model to perform real-time computation on video data in a GPU (graphics processing unit) environment.

[0034] It should be noted that while the solutions described in this disclosure are particularly suitable for real-time analysis of video data, they are equally applicable to the analysis of video data that has already been fully acquired.

[0035] The tear film detection system based on video data disclosed herein can be used to identify detection segments and / or detect tear film. The tear film detection system disclosed herein may also be referred to as a detection system or a tear film detection system, etc.

[0036] Examples of this disclosure will now be described in detail with reference to the accompanying drawings. Figure 1 This is an exemplary schematic diagram illustrating the detection environment 1 involved in the examples of this disclosure. For example... Figure 1 As shown, the testing environment 1 may include testing equipment 100 and testing system 200.

[0037] The detection device 100 can be used to acquire video data of the tear film 2 of a subject. For example, the detection device 100 can be a slit-lamp microscope. The detection device 100 can transmit the acquired video data to the detection system 200. In some examples, an observer 3 (e.g., a doctor) can use the detection device 100 to observe the tear film 2 of the subject to adjust the imaging field of view and / or identify the tear film state of the tear film 2.

[0038] The detection system 200 can receive video data and identify detection segments within the video data and / or perform tear film detection based on the video data. For example, during the examination of a subject's tear film 2 using the detection device 100, the detection device 100 can transmit the acquired video data to the detection system 200 in real time. The detection system 200 can analyze the real-time video data and respond in real time to the detection results of each detection segment until it is confirmed that the subject has completed the tear film examination. Furthermore, the detection system 200 can be implemented using a computing device. The computing device can be any device with computing capabilities. For example, the computing device can be a cloud server, personal computer, mainframe, or distributed computing system, etc.

[0039] Figure 2 This is an exemplary block diagram illustrating the detection system 200 as described in this disclosure.

[0040] refer to Figure 2The detection system 200 may include an acquisition module 210, a state analysis module 230, a detection segment analysis module 250, and a result analysis module 270. The acquisition module 210 can be configured to determine a tear film data queue based on video data of the tear film 2. The state analysis module 230 can be configured to determine the eye state of a detection frame based on changes in the corneal region between multiple consecutive detection frames. The detection segment analysis module 250 can be configured to acquire multiple detection segments corresponding to the video data based on the eye state of the detection frames. The result analysis module 270 can be configured to determine the target detection result based on multiple detection segments.

[0041] refer to Figure 2 In this embodiment, the acquisition module 210 can be configured to determine a tear film data queue based on video data of the tear film 2. The video data can correspond to multiple detection frames that change over time (i.e., multiple frames of tear film data). In some examples, the acquisition module 210 can be configured to use multiple detection frames that change over time corresponding to the video data as the tear film data queue. For example, for video data lasting 1-2 minutes, at 60 frames per second, 3600-7200 detection frames can be obtained as the tear film data queue. Furthermore, the detection frames can be images or data streams.

[0042] In some examples, video data may include at least one of video files and video buffer data (also referred to as video streams). The video file may be fully acquired data. This allows for the analysis of non-real-time video data. In some examples, the video file may be stored in a storage unit (e.g., a storage server), from which the detection system 200 can read and analyze the video file. Alternatively, the video buffer data may be data acquired in real-time. This allows for online analysis of real-time video data. In some examples, the video buffer data may be data generated in real-time during a tear film examination of the subject, which the detection system 200 can receive and analyze in real-time.

[0043] In some examples, the video data can be acquired after staining the tear film 2 with a contrast agent. In this case, the contrast of the image in the video data can be improved, which is beneficial for identifying the tear film state (e.g., tear film breakup morphology). In some examples, the contrast agent can be sodium fluorescein. For example, sodium fluorescein can be used to stain the tear film 2, and when the tear film 2 breaks up, dry spots will appear on the tear film surface. This makes it easy to observe and identify the tear film state. In other examples, video data can also be acquired non-invasively.

[0044] In some examples, the acquisition module 210 can be configured to decode video data (e.g., video files and / or video buffer data) into multiple detection frames using video decoding technology. This allows for the acquisition of multiple detection frames corresponding to the video data over time. In some examples, the video data may originate from a subject. For example, the video data may be obtained by the detection device 100 from a subject during a tear film examination.

[0045] In some examples, detection frames entering the tear film data queue can wait for the detection segment analysis module 250 to acquire them for detection segment division. Specifically, the detection segment analysis module 250 can determine the eye state of the detection frame through the state analysis module 230, and then segment the video data based on the eye state of the detection frame to obtain multiple detection segments.

[0046] In some examples, the tear film data queue can be a buffered data queue. The buffered data queue can be stored in memory. In this case, a buffered detection frame-based approach can improve data reading speed and enable rapid response, thereby further supporting real-time analysis.

[0047] Additionally, the buffered data queue can be flushed. For example, the buffered data queue can be flushed to load video files in segments and / or receive real-time video buffered data.

[0048] Continue to refer to Figure 2 In this embodiment, the state analysis module 230 can be configured to determine the eye state of a detection frame based on changes in the corneal region between multiple consecutive detection frames. Multiple consecutive detection frames can form a frame dataset.

[0049] In some examples, multiple consecutive detection frames can be three consecutive detection frames (i.e., three adjacent detection frames) in the tear film data queue. That is, the frame dataset can consist of three consecutive detection frames from the tear film data queue. In other words, the frame dataset can sequentially include the preceding frame, the middle frame, and the following frame.

[0050] Specifically, the state analysis module 230 can be configured to receive the detection frame to be analyzed, obtain the frame dataset corresponding to the detection frame to be analyzed, and determine the eye state of the detection frame to be analyzed based on the changes in the shape and size of the corneal region between detection frames in the frame dataset. The frame dataset can consist of three consecutive detection frames from the tear film data queue, with the detection frame to be analyzed located in the middle of these three consecutive detection frames. That is, the eye state of a detection frame can be determined by the changes in the shape and size of the corneal region between the preceding frame, the frame's own data (i.e., the middle frame), and the following frame. In this case, determining the eye state of the middle frame by analyzing the changes in the shape and size of the corneal region of three consecutive detection frames takes into account that the tear film 2 is a dynamic structure and the dynamic changes of the cornea. Compared to the existing technology that classifies a single detection frame, which ignores the dynamic changes, this method can more accurately determine the eye state of the detection frame. Furthermore, using changes in the corneal region to determine the eye state has better interpretability than the existing method that uses machine learning models for eye state prediction. In addition, it has a more stable ability to adapt to various extreme situations (such as blinking).

[0051] Furthermore, the shape and size of the corneal region can be represented using relative size. That is, the eye state of the detection frame to be analyzed can be determined based on the changes in the relative size of the corneal region between detection frames in the frame dataset. In this case, the negative impact of possible offsets between multiple detection frames (i.e., misalignment between multiple detection frames) on the changes can be reduced, thereby enabling compatibility with various complex video data acquisition environments.

[0052] Additionally, the relative size of the corneal region can be relative to the size of a reference object. In some examples, the relative size can be the ratio of the size of the corneal region in the detection frame to the size of the reference object. The reference object can be an object of relatively fixed size across multiple detection frames. In some examples, the reference object can be at least one of the entire image corresponding to the detection frame (i.e., the entire image corresponding to the detection frame) and the region of interest (ROI) in the detection frame. For example, the relative size can be the ratio of the size of the corneal region in the detection frame to the size of the entire image. Furthermore, the ROI can include, but is not limited to, stained regions. In other examples, the shape and size of the corneal region can also be represented by an absolute size.

[0053] In some examples, the shape and size of the corneal region can be the area of ​​the corneal region. Accordingly, relative size can be relative area, and absolute size can be absolute area.

[0054] Figure 3 This is an exemplary flowchart illustrating the determination of eye state for three consecutive detection frames as described in this disclosure.

[0055] In some examples, eye states may include an open state, an eye-opening process state, and an eye-closing process state. Therefore, examples of this disclosure also provide an exemplary method for determining eye states for three consecutive detection frames. A state analysis module 230 can be configured to implement this method for determining eye states. It should be noted that, unless contradictory, the shape and size of the corneal region in this method for determining eye states can be replaced by either a relative size or an absolute size of the corneal region without limitation. Reference Figure 3 For a frame dataset consisting of a previous frame, a middle frame, and a next frame, methods for determining the eye state of the middle frame (i.e., the detection frame to be analyzed) can include:

[0056] Step S102: The eye state of the intermediate frame can be determined based on the changes. Specifically, if the changes are that the shape and size of the corneal region in the previous frame are larger than a preset size, the shape and size of the corneal region in the intermediate frame are larger than a preset size, and the shape and size of the corneal region in the next frame are also larger than a preset size, then the eye state of the intermediate frame can be open. In some examples, if the changes do not meet the conditions for an open state, it can be further determined whether other eye state conditions are met or the eye state of the intermediate frame can be set to another state.

[0057] Step S104: The eye state of the intermediate frame can be determined based on the changes. Specifically, if the changes are that the shape and size of the corneal region in the previous frame are smaller than a preset size, the shape and size of the corneal region in the intermediate frame are larger than a preset size, and the shape and size of the corneal region in the subsequent frame are larger than the shape and size of the corneal region in the intermediate frame, then the eye state of the intermediate frame can be the eye-opening process state. In some examples, if the changes do not meet the conditions for the eye-opening process state, it can be further determined whether other eye state conditions are met or the eye state of the intermediate frame can be set to another state.

[0058] Step S106: The eye state of the intermediate frame can be determined based on the changes. Specifically, if the changes are that the shape and size of the corneal region in the previous frame are larger than a preset size, the shape and size of the corneal region in the intermediate frame are smaller than the shape and size of the corneal region in the previous frame, and the shape and size of the corneal region in the subsequent frame are smaller than the preset size, then the eye state of the intermediate frame can be a closed-eye state. In some examples, if the changes do not meet the conditions for a closed-eye state, it can be further determined whether other eye state conditions are met or the eye state of the intermediate frame can be set to another state.

[0059] In some examples, the eye state may also include a closed state. Therefore, the method for determining the eye state of an intermediate frame may further include determining whether the eye state of the intermediate frame is closed based on changes in the surrounding environment. Specifically, if the changes are that the shape and size of the corneal region in the previous frame is smaller than a preset size, the shape and size of the corneal region in the intermediate frame is smaller than a preset size, and the shape and size of the corneal region in the subsequent frame is smaller than a preset size, then the eye state of the intermediate frame can be a closed state.

[0060] It should be noted that the steps in the method for determining the eye state of intermediate frames described above are not subject to any particular limitation, and the order of the steps can be adjusted or corresponding steps can be deleted according to the actual situation. In some examples, the eye state may also include the instantaneous blink state.

[0061] Furthermore, the aforementioned preset size can be a fixed value (e.g., an empirical value). For example, if the relative size is the ratio of the size of the corneal region of the detection frame to the size of the reference object, then the preset size can be a pre-set ratio determination threshold (also known as a proportion determination threshold). Those skilled in the art can experiment and adjust the scheme according to the examples in this disclosure to determine a suitable preset size.

[0062] In some examples, the state analysis module 230 can be configured to obtain the corneal region of the detection frame through a segmentation method. Preferably, the corneal region of the detection frame can be obtained through a deep learning-based corneal region segmentation model. The corneal region segmentation model can be pre-trained. In this case, the deep learning-based segmentation method can learn as much as possible the high-level semantic feature information of the corneal region in the video data through the annotation information of the training data, and can be compatible with video data under different acquisition conditions (e.g., the video is affected by different objective conditions such as lighting and staining, or subjective factors such as eye movement, focus, shooting distance and angle, and eyelash occlusion), thus enabling more accurate segmentation of the corneal region.

[0063] Return to reference Figure 2In this embodiment, the detection segment analysis module 250 can be configured to acquire multiple detection segments corresponding to the video data based on the eye state of the detection frame. In some examples, the eye state of the detection frame can be obtained by the state analysis module 230. That is, the detection segments that can be used for analysis can be automatically segmented using the eye state of the detection frame obtained by the state analysis module 230. In this case, based on the more accurate eye state of the detection frame, the segmentation accuracy of tear film detection can be higher, reducing the dependence on further processing of the segmented detection segments to obtain better detection segments, thereby supporting real-time analysis. For example, the solution in patent document (CN111062443A) directly identifies the opening and closing state without considering the impact of the dynamic changes of the cornea on the classification accuracy, thus affecting the segmentation accuracy of the detection segments. It is necessary to obtain the detection segments and then extract the longest open eye segment, which cannot support real-time analysis.

[0064] In some examples, the detection segment analysis module 250 can be configured to read detection frames from the tear film data queue and obtain the eye state of the detection frames through the state analysis module 230 (for example, the detection frame to be analyzed can be used as input to the state analysis module 230 to obtain the eye state of the detection frame).

[0065] In some examples, the detection segment analysis module 250 can be configured to read detection frames in the tear film data queue sequentially in chronological order. In some examples, the detection segment analysis module 250 can be configured to read the eye state of the detection frames obtained by the state analysis module 230 while reading the detection frames in the tear film data queue. In other examples, the detection segment analysis module 250 can be configured to read the detection frames in the tear film data queue and read the eye state of the detection frames already obtained by the state analysis module 230.

[0066] In some examples, the detection segment analysis module 250 can be configured to delay reading detection frames from the tear film data queue by one frame (i.e., it can start reading from the second detection frame), so that the state analysis module 230 can determine the eye state of the detection frame based on the changes in the corneal region between the preceding frame, the data of the detection frame itself, and the following frame. In other examples, the detection segment analysis module 250 can also be configured not to delay reading the tear film data queue (i.e., it can start reading from the first detection frame), and set the eye state of the first detection frame in the tear film data queue to any state that does not affect the segment division. It should be noted that those skilled in the art can choose appropriate means to implement this according to the actual situation, and this disclosure does not impose any particular limitations.

[0067] In this embodiment, each detection segment can correspond to a detection frame within a specific frame range in the tear film data queue. The frame range can be represented as the period from the start frame to the end frame. That is, each detection segment can have a start frame and an end frame. Furthermore, the time corresponding to the start frame can be shorter than the time corresponding to the end frame.

[0068] In some examples, the start and end frames of each detection segment can be determined from the detection frames in the tear film data queue where the eye is in the open state and the detection frames where the eye is in the closed state, respectively. That is, the start frame of a detection segment can be determined from the detection frames where the eye is in the open state, and the end frame of a detection segment can be determined from the detection frames where the eye is in the closed state. Specifically, the detection segment analysis module 250 can be configured to, upon identifying a detection frame in the tear film data queue where the eye is in the open state, use that detection frame as the start frame of a detection segment, continue identifying the eye states of the detection frames in the tear film data queue to obtain a detection frame where the eye is in the closed state as the end frame of that detection segment, and then continue to identify the start frame of another new detection segment. Thus, a detection segment can be determined.

[0069] In some examples, the detection segment analysis module 250 can also be configured to determine the detection results of each detection segment based on detection frames where the eyes are open. That is, the detection segment analysis module 250 can be configured to identify detection frames where the eyes are open and determine the detection result of the detection segment to which that frame belongs. In some examples, the detection segment analysis module 250 can also be configured to store the detection results of a detection segment after determining them. This facilitates easy retrieval.

[0070] In some examples, the detection segment analysis module 250 can be configured to detect whether a corneal tear has occurred in a detection frame where the eye is open. If a corneal tear has occurred in the detection frame, the detection result of the detection segment to which the detection frame belongs is determined based on the detection frame. In some examples, whether a corneal tear has occurred in the detection frame can be determined by whether a dry spot is present in the detection frame. In some examples, the detection frame used to determine the detection result of each detection segment can be the detection frame in the detection segment where a corneal tear occurs for the first time and the eye is open.

[0071] Figure 4A This is a schematic diagram showing a related image of a detection frame in an open and unbroken state, as described in the examples of this disclosure. Figure 4B This is a schematic diagram showing a related image of a detection frame that is in an open state and has been broken, as described in the example of this disclosure. Figure 4C This is a schematic diagram showing a related image of a detection frame that illustrates the closing process state involved in the example of this disclosure and where the breakage state cannot be identified.

[0072] Additionally, as an example of a related image for a detection frame that is open and unbroken, Figure 4A The image shows the corneal segmentation result P11, the open state map P12, and the corneal region map P13 of frame image P10. This serves as an example of a related image for a detection frame that is both open and ruptured. Figure 4B The image shows the corneal segmentation result P21, the open state map P22, and the corneal region map P23 of frame image P20. These images serve as examples of relevant images for detection frames in the closure process state where the rupture state cannot be identified. Figure 4C The results of corneal segmentation (P31), the closure process state (P32), and the corneal region (P33) of frame image P30 are shown.

[0073] In some examples, the detection segment analysis module 250 can be configured to detect whether the corneal region of the detection frame has ruptured using a machine learning classification method. Preferably, the corneal region rupture of the detection frame can be detected using a deep learning-based tear film rupture detection model. The tear film rupture detection model can be pre-trained. In this case, the deep learning-based classification method can learn as much as possible the high-level semantic feature information of the corneal region in the video data through the annotation information of the training data, and can be compatible with video data under different acquisition conditions, thus enabling more accurate classification.

[0074] In some examples, the test results may include at least one of tear film break-up time and tear film break-up morphology. Tear break-up time (TBUT) can be the time from blinking and keeping the eyes open until the first dry spot appears on the tear film 2.

[0075] Regarding tear film breakup time, in some examples, the detection segment analysis module 250 can be configured to calculate the tear film breakup time of a detection segment based on the position of the detection frame in which the corneal breakup occurs within its respective detection segment. In some examples, the tear film breakup time of a detection segment can be calculated based on the frame number of the detection frame in which the corneal breakup occurs and the frame number of the starting frame of the detection segment.

[0076] Regarding tear film breakup morphology, in some examples, the detection segment analysis module 250 can be configured to acquire the tear film breakup morphology of the detection segment based on the detection frame in which the corneal region breaks up. In some examples, the tear film breakup morphology may include at least one of the following: an image corresponding to the detection frame, descriptive information about the tear film state obtained based on the image, and the shape of the tear film 2 obtained based on the image.

[0077] Figure 5 This is an exemplary flowchart illustrating the detection segment recognition involved in the examples of this disclosure.

[0078] Furthermore, this disclosure also provides an exemplary method for detecting segment identification. This method uses a tear film data queue as a buffer data queue and determines the eye state of the current frame in the following order: eye-opening process state, eye-closing process state, and then open state. It should be noted that, unless there is a contradiction, the order of eye state determination can be adjusted according to the actual situation without restriction. Additionally, the buffer data queue can be replaced with a tear film data queue without restriction.

[0079] refer to Figure 5 An exemplary method for identifying this detection segment may include:

[0080] Step S202: Read a detection frame from the buffer data queue in chronological order and use it as the current frame.

[0081] Step S204: Determine the eye state of the current frame. Specifically, the eye state of the current frame can be determined based on the changes in the shape and size of the corneal region between the previous frame, the current frame, and the next frame. See the description of the exemplary method for determining the eye state above for details.

[0082] Step S206: Determine whether the eye state of the current frame is in the process of opening the eyes. In some examples, if the eye state of the current frame is in the process of opening the eyes, then step S208 can be executed. In some examples, if the eye state of the current frame is not in the process of opening the eyes, then step S210 can be executed.

[0083] Step S208: Use the current frame as the starting frame of the detection segment. That is, if the eye state of the current frame is in the open state, it indicates that the starting frame of a detection segment has been identified. In other words, we can wait for the starting frame to be identified before starting to identify a new detection segment. In addition, after executing step S208, step S214 can be executed.

[0084] Step S210: Determine whether the eye state of the current frame is in the closed-eye process state. In some examples, if the eye state of the current frame is in the closed-eye process state, step S212 can be executed. In some examples, if the eye state of the current frame is not in the closed-eye process state, step S214 can be executed or the eye state of the current frame can be determined (e.g., step S213 can be executed).

[0085] Step S212: The current frame is taken as the end frame of the detection segment with the determined start frame, thus determining a detection segment (i.e., completing the recognition of a detection segment). That is, after confirming the recognition of the start frame of a detection segment, detection frames in the eye-closed state can be identified as the end frames of that detection segment to complete its recognition. In some examples, the state of the detection segment can be set to "complete" to indicate that the recognition of the detection segment is complete. Furthermore, after executing step S212, step S214 can be executed.

[0086] Step S214: Continue reading the next detection frame from the buffer data queue as the current frame, and restart execution from step S204 until the buffer data queue is completely read. Specifically, the current frame can be updated to the next detection frame in the buffer data queue, and judgments can continue based on the current frame until the buffer data queue is completely read.

[0087] In some examples, continue to refer to Figure 5 The exemplary method for identifying the detection segment described above may further include step S213. In step S213, if the eye state of the current frame is open, it is detected whether the corneal region of the current frame is ruptured. If the corneal region of the current frame is ruptured, the detection result of the corresponding detection segment is determined based on the current frame. In some examples, the detection result may also be recorded in step S213. For details, please refer to the above description of determining the detection results of each detection segment.

[0088] In some examples, in the exemplary method of the detection segment recognition described above, for the current frame in which the eye is in a closed state, step S214 can continue to be executed.

[0089] The exemplary method for detection segment identification disclosed herein is based on the eye state determined by the dynamic changes of the cornea to obtain a more accurate detection segment in real time, and the more accurate detection segment enables real-time analysis.

[0090] In some examples, the detection segment analysis module 250 can be configured to automatically split the detection segments and display the detection results for each segment. In this case, the user can independently determine whether to use the detection results of the corresponding detection segment. For example, the tear film breakup time of a momentary blink is usually less used to assess tear film status, and the detection segment of a momentary blink is often very short and easy to identify; the user can choose to discard the result of that segment.

[0091] Return to reference Figure 2 In this embodiment, the result analysis module 270 can be configured to determine the target detection result based on multiple detection segments. Specifically, the result analysis module 270 can be configured to determine the target detection result based on multiple detection results corresponding to multiple detection segments.

[0092] In some examples, the target detection result may include at least one of target tear film breakup time and target tear film breakup morphology. The target tear film breakup time may be determined by multiple tear film breakup times corresponding to multiple detection segments. The target tear film breakup morphology may be determined by multiple tear film breakup morphologies corresponding to multiple detection segments.

[0093] In some examples, the result analysis module 270 can be configured to fuse multiple tear film breakup times corresponding to multiple detection segments to obtain a target tear film breakup time. In some examples, the fusion method can include, but is not limited to, at least one of random selection, maximum value selection, minimum value selection, median value selection, and average value selection. Preferably, the fusion method can be average value selection. That is, the result analysis module 270 can be configured to average the multiple tear film breakup times corresponding to multiple detection segments to obtain the target tear film breakup time.

[0094] In some examples, the result analysis module 270 can be configured to comprehensively analyze multiple tear film breakup patterns corresponding to multiple detection segments to obtain a target tear film breakup pattern. In some examples, the target tear film breakup pattern may include multiple tear film breakup patterns corresponding to multiple detection segments. In some examples, the result analysis module 270 can be configured to filter or fuse images of detection frames where corneal breakup occurs corresponding to multiple detection segments to obtain a target image. This results in a higher quality image. In some examples, the result analysis module 270 can also be configured to obtain descriptive information about the tear film state and / or the shape of the tear film 2 based on the target image. This results in more accurate tear film breakup information.

[0095] In some examples, the results analysis module 270 can be configured to display multiple detection results and / or target detection results corresponding to multiple detection segments. In this case, it facilitates the analysis of results by users (e.g., doctors).

[0096] In some examples, the results analysis module 270 can also be configured to filter out detection segments of detection frames where no corneal rupture occurs before determining the target detection result. In some examples, the user can use the results analysis module 270 to filter detection segments used to determine the target detection result.

[0097] It should be noted that if the detection system 200 is used to identify multiple detection segments, the result analysis module 270 may not be necessary.

[0098] Figure 6 This is an exemplary flowchart illustrating a training method for a corneal region segmentation model as described in this disclosure.

[0099] In some examples, the detection system 200 may also include the pre-trained corneal region segmentation model described above. As mentioned above, the corneal region segmentation model can be used to acquire the corneal regions of the detection frames. The corneal region segmentation model can be a lightweight semantic segmentation model. For example, the semantic segmentation model may include, but is not limited to, U-Net, Deeplab, STDC, or BiSeNet. To this end, the examples of this disclosure also provide a method for training the corneal region segmentation model.

[0100] refer to Figure 6 Training methods for corneal region segmentation models can include:

[0101] Step S302: Construct the first training sample. The first training sample may include multiple examination images, each of which may have annotation information. In some examples, a predetermined number of examination videos for tear film stability can be collected, and multiple examination images can be obtained based on the examination videos. For example, the examination videos can be decoded to obtain multiple examination images. In some examples, multiple examination images of an examination video can be stored in chronological order (also known as frame sequence order). This facilitates subsequent annotation of multiple examination images of the same examination video.

[0102] In some examples, the annotation information for each examination image can be a corneal contour mask. In other examples, point coordinates can be marked along the corneal edge of each examination image, and the corresponding corneal contour mask can be generated based on these coordinates. This allows for the acquisition of annotation information for each examination image. In some examples, for multiple examination images from a single examination video, annotation can be performed sequentially in chronological order. This improves the convenience and efficiency of annotation.

[0103] Step S304: Train the corneal region segmentation model based on the first training samples. In some examples, the first training samples can be divided into a first training set and a first validation set. The corneal region segmentation model is trained using the first training set and validated using the first validation set to adjust the corneal region segmentation model, thereby obtaining the optimal corneal region segmentation model.

[0104] Figure 7 This is an exemplary flowchart illustrating a training method for a tear film breakage detection model as described in this disclosure.

[0105] In some examples, the detection system 200 may also include the pre-trained tear film breakage detection model described above. As mentioned above, the tear film breakage detection model can be used to detect whether a tear has occurred in the corneal region of a detection frame. The tear film breakage detection model can be a lightweight classification network model. For example, the classification network model may include, but is not limited to, ResNet, ShuffleNet, EfficientNet, or MobileNet. To this end, the examples of this disclosure also provide a method for training the tear film breakage detection model. Reference Figure 7 Training methods for tear film breakage detection models can include:

[0106] Step S402: Construct a second training sample. The second training sample may include multiple examination images. In some examples, a predetermined number of examination videos for tear film stability can be collected, and multiple examination images can be obtained based on the examination videos. For example, the examination videos can be decoded to obtain multiple examination images. In some examples, multiple examination images of a single examination video can be stored in chronological order. This facilitates the identification of the examination image where the first tear film rupture occurs during subsequent annotation.

[0107] Step S404: Annotate the inspection images in the second training sample to determine the annotation labels for each inspection image. This allows each inspection image to have an annotation label. In some examples, the scheme described in the detection segment analysis module 250 can be used to obtain multiple detection segments corresponding to the inspection video, and inspection images belonging to the same detection segment can be stored in the same storage unit. In this case, it is beneficial for annotators to quickly determine the annotation targets and improve the convenience of annotation. Specifically, multiple inspection images can be saved in separate storage units based on the divided detection segments, and annotation can be performed on a unit-by-unit basis. In some examples, the storage unit for storing detection segments can be a folder.

[0108] In some examples, when annotating multiple examination images corresponding to a detection segment, annotators can label the examination image where tear film rupture is first observed. In some examples, the annotation labels for examination images corresponding to a detection segment can be determined based on the already labeled examination images. In some examples, multiple annotators can label the images separately, and then an expert annotator can review and determine the final labels. This reduces the impact of subjective factors on the accuracy of labeling.

[0109] In some examples, the labels can include both "unbroken" and "broken." Specifically, multiple inspection images corresponding to a detection segment can be traversed. After obtaining the labeled inspection images, those images are labeled as broken, images before them are labeled as unbroken, and images after them are labeled as broken. In this way, the labeling information of multiple inspection images corresponding to a detection segment can be obtained.

[0110] In some examples, the labeling can also include a closed label. Specifically, inspection images with closed eyes can be added to the second training samples, and the label for these inspection images can be set to closed. In this case, the class ratio of the training samples can be balanced. In addition, the model can distinguish inspection images with closed eyes, thus adapting to situations where the video data itself contains closed eyes in the application scenario.

[0111] In some examples, three-digit numbers can be used to represent labels. It should be noted that this does not imply a limitation of this disclosure. For example, "000" can represent closure, "100" can represent unbroken, and "111" can represent broken.

[0112] Step S406: Train the tear film breakage detection model based on the second training samples. In some examples, the second training samples can be divided into a second training set and a second validation set. The tear film breakage detection model is trained using the second training set and validated using the second validation set to adjust the tear film breakage detection model and obtain the optimal tear film breakage detection model.

[0113] The tear film detection method based on video data disclosed herein is described in detail below with reference to the accompanying drawings. The tear film detection method can be implemented by the detection system 200 described above. The description of the detection system 200 also applies to the tear film detection method. The tear film detection method illustrated in this disclosure may also be referred to as a detection method or a tear film detection method, etc. Figure 8 This is an exemplary flowchart illustrating a tear film detection method based on video data as described in this disclosure.

[0114] In some examples, such as Figure 8 As shown, the detection method may include determining a tear film data queue based on video data for the tear film 2 (step S502), obtaining multiple detection segments corresponding to the video data based on the eye state of the detection frame (step S504), and determining the target detection result based on the multiple detection segments (step S506).

[0115] In this embodiment, in step S502, a tear film data queue can be determined based on video data for the tear film 2. The video data can correspond to multiple detection frames that change over time. In some examples, multiple detection frames that change over time corresponding to the video data can be used as the tear film data queue. For details, please refer to the relevant description of the acquisition module 210.

[0116] In this embodiment, in step S504, multiple detection segments corresponding to the video data can be obtained based on the eye state of the detection frame. In some examples, in step S504, detection frames in the tear film data queue can be read, the eye state of the detection frames can be obtained, and multiple detection segments corresponding to the video data can be obtained based on the eye state of the detection frames. In some examples, the eye state can include an open state, an open-eye process state, and a closed-eye process state. In some examples, the start frame and end frame of each detection segment can be determined by the detection frames in the tear film data queue with the eye state being in the open-eye process state and the detection frames with the eye state being in the closed-eye process state, respectively. In some examples, in step S504, the detection result of each detection segment can also be determined based on the detection frames with the eye state being in the open state. For details, please refer to the relevant description of the detection segment analysis module 250.

[0117] In some examples, the eye state of a detection frame can be determined based on changes in the corneal region between multiple consecutive detection frames in the tear film data queue. Multiple consecutive detection frames can form a frame dataset. In some examples, multiple consecutive detection frames can be three consecutive detection frames from the tear film data queue. Specifically, in analyzing the eye state of a detection frame, the frame dataset corresponding to the detection frame to be analyzed can be obtained. The eye state of the detection frame to be analyzed is determined based on changes in the shape and size of the corneal region between detection frames in the frame dataset. The frame dataset can consist of three consecutive detection frames from the tear film data queue, and the detection frame to be analyzed is located in the middle of these three consecutive detection frames. Furthermore, the shape and size of the corneal region can be represented using relative sizes. See the relevant description in the state analysis module 230 for details.

[0118] In this embodiment, in step S506, the target detection result can be determined based on multiple detection segments. Specifically, the target detection result can be determined based on multiple detection results corresponding to multiple detection segments. See the relevant description in the result analysis module 270 for details. In some examples, step S506 may not be mandatory.

[0119] This disclosure also relates to an electronic device that may include at least one processing circuit. The at least one processing circuit is configured to perform one or more steps of the methods described above.

[0120] This disclosure also relates to a computer-readable storage medium that may store at least one instruction that, when executed by a processor, implements one or more steps of the methods described above.

[0121] The detection system 200 and detection method disclosed herein accurately determine the eye state of the tear film 2 in the detection frames of video data based on the dynamic changes of the cornea. Multiple detection segments are then determined based on the eye state of the detection frames, and these segments are analyzed to obtain the target detection result. In this case, the eye state determined based on the dynamic changes of the cornea enables high segmentation accuracy in tear film detection, and the more accurate segmentation supports real-time analysis and provides interpretability. Furthermore, the shape and size of the corneal region are represented using relative sizes, which reduces the negative impact of possible offsets between multiple detection frames on the changes, thereby enabling compatibility with various complex video data acquisition environments.

[0122] While the present disclosure has been specifically described above in conjunction with the accompanying drawings and examples, it is to be understood that the foregoing description does not limit the present disclosure in any way. Those skilled in the art can make modifications and variations to the present disclosure as needed without departing from its essential spirit and scope, and all such modifications and variations shall fall within the scope of the present disclosure.

Claims

1. A tear film detection system based on video data, characterized in that, It includes an acquisition module, a state analysis module, a detection segment analysis module, and a result analysis module; the acquisition module is configured to acquire multiple detection frames that change over time corresponding to the video data for the tear film and use them as a tear film data queue, the video data includes video buffer data, the video buffer data is data generated in real time by the subject during the tear film examination, and the tear film data queue is a buffer data queue; The state analysis module is configured to receive the detection frame to be analyzed, acquire three consecutive detection frames in the tear film data queue that are the middle of the detection frame to be analyzed, the three detection frames being the previous frame, the middle frame, and the next frame, with the middle frame being the detection frame to be analyzed. Based on the changes in the shape and size of the corneal region in the three detection frames, the eye state of the detection frame to be analyzed is determined for real-time acquisition of detection segments. The eye state includes an open state, an open-eye process state, and a closed-eye process state, and the shape and size of the corneal region are represented by relative size. The detection segment analysis module is configured to read the detection frames in the tear film data queue, acquire multiple detection segments corresponding to the video data in real-time based on the eye state of the detection frames obtained by the state analysis module, and determine the detection result of each detection segment based on the detection frames where the eye state is open. In the process, when a detection frame in the tear film data queue where the eye state is in the open-eye process state is identified, this detection frame is used as the starting frame of the detection segment. The process continues to identify detection frames in the tear film data queue where the eye state is in the closed-eye process state and uses these as the ending frame of the detection segment. Then, the process continues to identify the starting frame of another new detection segment. The result analysis module is configured to determine a target detection result based on multiple detection results corresponding to the multiple detection segments. The detection result includes at least one of tear film breakup time and tear film breakup morphology, and the target detection result includes at least one of target tear film breakup time and target tear film breakup morphology. The detection segment analysis module is further configured to determine the detection result of the detection segment to which the detection frame belongs when the detection frame is detected in the open-eye state and tear film breakup occurs within the detection segment. The state analysis module is configured to: If the change is that the shape and size of the corneal region in the previous frame is greater than the preset size, the shape and size of the corneal region in the intermediate frame is greater than the preset size, and the shape and size of the corneal region in the next frame is greater than the preset size, then the eye state in the intermediate frame is open. If the change is such that the shape and size of the corneal region in the previous frame is smaller than the preset size, the shape and size of the corneal region in the intermediate frame is larger than the preset size, and the shape and size of the corneal region in the next frame is larger than the shape and size of the corneal region in the intermediate frame, then the eye state in the intermediate frame is the open-eye process state. If the change is such that the shape and size of the corneal region in the previous frame is greater than the preset size, the shape and size of the corneal region in the intermediate frame is smaller than the shape and size of the corneal region in the previous frame, and the shape and size of the corneal region in the next frame is smaller than the preset size, then the eye state in the intermediate frame is the closed-eye state.

2. The system according to claim 1, characterized in that: The video data was collected after staining the tear film with a contrast agent.

3. The system according to claim 1, characterized in that: The relative size is the ratio of the size of the corneal region of the detection frame to the size of the reference object, wherein the reference object is at least one of the whole image and the region of interest corresponding to the detection frame.

4. The system according to claim 1, characterized in that: The detection segment analysis module is configured as follows: A detection frame is read from the buffer data queue in chronological order and used as the current frame; The eye state of the current frame is determined based on the changes in the shape and size of the corneal region between the previous frame, the current frame, and the next frame. If the eye state of the current frame is in the open eye process state, then the current frame is taken as the starting frame of the detection segment; If the eye state of the current frame is in the closed-eye process state, then the current frame is taken as the end frame of the detection segment whose start frame has been determined, thereby determining a detection segment; and The current frame is updated to the next detection frame in the buffer data queue, and the judgment is continued based on the current frame until the buffer data queue is completely read.

5. The system according to claim 1, characterized in that: If the eye state of a detection frame is open, it is determined whether the corneal region of the detection frame has ruptured. If the corneal region of the detection frame has ruptured, the tear film breakup time corresponding to the detection segment is calculated based on the position of the detection frame in the detection segment, and / or the tear film breakup morphology corresponding to the detection segment is obtained based on the detection frame.

6. The system according to claim 1, characterized in that: The result analysis module is configured to calculate the average of multiple tear film breakup times corresponding to the multiple detection segments to determine the target tear film breakup time.

7. The system according to claim 1, characterized in that: It also includes a pre-trained deep learning-based corneal region segmentation model, which is used to obtain the corneal region of the detection frame; and / or It also includes a pre-trained deep learning-based tear film breakage detection model, which is used to detect whether the corneal region of the detection frame has broken.

8. A tear film detection method based on video data, characterized in that, include: Multiple detection frames that change over time corresponding to the video data for the tear film are acquired and used as a tear film data queue. The video data includes video buffer data, which is data generated in real time by the subject during the tear film examination. The tear film data queue is a buffer data queue. The system reads detection frames from the tear film data queue, acquires multiple detection segments corresponding to the video data in real time based on the eye state of the detection frames, and determines the detection results of each detection segment based on the detection frames where the eye state is open. The eye state includes open state, open-eye process state, and closed-eye process state. When a detection frame in the tear film data queue where the eye state is open is identified, this detection frame is used as the starting frame of the detection segment. The system continues to identify detection frames in the tear film data queue where the eye state is closed, using these as the ending frames of the detection segment. Then, it continues to identify the starting frame of another new detection segment. In analyzing the eye state of the detection frames, the system selects detection frames in the tear film data queue that are in the middle and continuous. The system uses three detection frames, which sequentially include a previous frame, a middle frame, and a next frame. The middle frame is the detection frame to be analyzed. The eye state of the detection frame to be analyzed is determined based on the changes in the shape and size of the corneal region in the three detection frames to obtain the detection segment in real time. The shape and size of the corneal region are represented by relative size. Furthermore, a target detection result is determined based on multiple detection results corresponding to the multiple detection segments. The detection result includes at least one of tear film breakup time and tear film breakup morphology, and the target detection result includes at least one of target tear film breakup time and target tear film breakup morphology. When the detection frame is detected to be in an open-eye state and tear film breakup has occurred in the detection segment, the detection result of the corresponding detection segment is determined based on the detection frame. If the change is that the shape and size of the corneal region in the previous frame is greater than the preset size, the shape and size of the corneal region in the intermediate frame is greater than the preset size, and the shape and size of the corneal region in the next frame is greater than the preset size, then the eye state in the intermediate frame is open. If the change is such that the shape and size of the corneal region in the previous frame is smaller than the preset size, the shape and size of the corneal region in the intermediate frame is larger than the preset size, and the shape and size of the corneal region in the next frame is larger than the shape and size of the corneal region in the intermediate frame, then the eye state in the intermediate frame is the open-eye process state. If the change is such that the shape and size of the corneal region in the previous frame is greater than the preset size, the shape and size of the corneal region in the intermediate frame is smaller than the shape and size of the corneal region in the previous frame, and the shape and size of the corneal region in the next frame is smaller than the preset size, then the eye state in the intermediate frame is the closed-eye state.

Images