Tear film state monitoring system based on ai analysis and intelligent glasses dry eye early warning method

Abstract

The application discloses an AI analysis-based tear film state monitoring system and an intelligent glasses dry eye early warning method, relates to the technical field of image processing, and comprises a pattern projection module, an ocular surface image acquisition module and a highlight suppression image generation module; the AI analysis-based tear film state monitoring system provided in the application collects eyelid edge region images by projecting a near-infrared laser speckle pattern, uses the near-infrared laser speckle pattern as a source texture, and generates a highlight suppression image by predicting a distortion field to guide the deformation of the source texture, thereby completing highlight repair; the feature information loss problem caused by the non-fixed position of the highlight reflected by the tear meniscus mirror is relieved, and stable imaging of the meibomian gland opening area under a dynamic ocular surface environment is realized.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an AI-based tear film state monitoring system and a method for early warning of dry eye in smart glasses. Background Technology

[0002] Dry eye syndrome is a common ocular surface disease, with evaporative dry eye being closely related to meibomian gland dysfunction. Therefore, non-invasive, dynamic monitoring of the meibomian gland openings at the margins of the upper and lower eyelids is crucial for the early warning and diagnosis of dry eye syndrome. Current research attempts to acquire ocular surface images using smart glasses with integrated cameras and analyze tear film status using image recognition technology.

[0003] However, in practical applications, the above method faces a problem: at the eyelid margin, the tear film naturally forms a meniscus, or tear river. When illuminated by the built-in light source of the smart glasses, this tear film meniscus produces strong, non-fixed specular highlights at specific angles. These highlights can easily completely obscure the meibomian gland openings below, preventing the camera from capturing effective opening feature information and rendering subsequent AI analysis ineffective.

[0004] To suppress specular interference, existing technologies mostly focus on optimizing light sources such as angle and polarization, or image processing algorithms. However, these methods struggle to adapt to changes in specular position caused by eye movement, resulting in insufficient robustness. Therefore, there is an urgent need for a solution that can effectively overcome specular reflection interference and achieve stable monitoring of meibomian gland openings in dynamic ocular surface environments. Summary of the Invention

[0005] To address the technical problems existing in the background art and to overcome the shortcomings of the prior art, this invention proposes an AI-based tear film state monitoring system and a dry eye early warning method for smart glasses.

[0006] The present invention proposes an AI-based tear film state monitoring system, comprising: Pattern projection module: When using smart glasses, after activating the monitoring function of the smart glasses, a unique and fixed, invisible near-infrared laser speckle pattern is projected onto the edge area of ​​the user's eyelids through the projector. Ocular surface image acquisition module: While the projector continuously projects a near-infrared laser speckle pattern, the built-in camera of the smart glasses exposes and acquires a single frame of the eyelid edge region image. Highlight suppression image generation module: Generates a predicted distortion field based on the eyelid edge region image and the near-infrared laser speckle pattern; generates a highlight suppression image based on the predicted distortion field.

[0007] Preferably, in the pattern projection module, the projector is integrated into the smart glasses, and the projector is a projection module, which includes a VCSEL laser and diffractive optical elements. Preferably, in the pattern projection module, when projecting near-infrared laser speckle patterns, near-infrared band projection is selected; Preferably, in the specular suppression image generation module, a predicted distortion field is generated based on the eyelid edge region image and the near-infrared laser speckle pattern, as follows: Based on the image of the eyelid edge region, the highlight area and the non-highlight area are divided on the image of the eyelid edge region; In the non-highlight region, the phase correlation method is first used to register the eyelid edge region image with the near-infrared laser speckle pattern. Then, the dense optical flow algorithm is used to obtain the first two-dimensional displacement vector of the eyelid edge region image relative to the near-infrared laser speckle pattern pixel by pixel. All the first two-dimensional displacement vectors are obtained to form the first displacement vector set, and the first displacement vector set is used as the first two-dimensional displacement field. In the first set of displacement vectors, all first two-dimensional displacement vectors on the boundary line between the highlight area and the non-highlight area are extracted to form the second set of displacement vectors. In the highlight region, the second two-dimensional displacement vector corresponding to each pixel in the highlight region is obtained by solving the Poisson equation for the second displacement vector set. All the second two-dimensional displacement vectors are obtained to form the third displacement vector set, and the third displacement vector set is used as the second two-dimensional displacement field. The first two-dimensional displacement field and the second two-dimensional displacement field are combined to form the predicted distortion field; Preferably, based on the eyelid edge region image, highlight areas and non-highlight areas are divided on the eyelid edge region image, as follows: The image of the eyelid edge region is converted into a grayscale image; a threshold segmentation method is applied to the grayscale image to generate a binary mask, which is used to divide the image of the eyelid edge region into a highlight region and a non-highlight region. Preferably, in the specular suppression image generation module, a specular suppression image is generated based on the estimated distortion field, as follows: A blank digital image with the exact same size and spatial coordinate system as the image of the eyelid edge region is used as the reconstructed image; Based on the predicted distortion field, image deformation technology is used to obtain the corresponding source coordinates of each pixel in the reconstructed image in the near-infrared laser speckle pattern; thus forming a coordinate mapping relationship between the reconstructed image and the near-infrared laser speckle pattern. Based on the coordinate mapping relationship between the reconstructed image and the near-infrared laser speckle pattern, the near-infrared laser speckle pattern is resampled to fill the reconstructed image, generating a speckle-suppressed image. Preferably, based on the coordinate mapping relationship between the reconstructed image and the near-infrared laser speckle pattern, a resampling operation is performed on the near-infrared laser speckle pattern to fill the reconstructed image, generating a speckle-suppressed image, as follows: For each pixel location in the reconstructed image, the source coordinates corresponding to each pixel location are obtained according to the coordinate mapping relationship between the reconstructed image and the near-infrared laser speckle pattern. The pixel brightness value at the source coordinates is obtained from the near-infrared laser speckle pattern using a bilinear interpolation algorithm. All pixels in the reconstructed image are traversed, and the pixel brightness value at the source coordinates is assigned to the corresponding pixel location in the reconstructed image to generate a specular suppression image.

[0008] Preferably, in S1, the projector is integrated into the smart glasses, and the projector is a projection module, which includes a VCSEL laser and diffractive optical elements. VCSEL lasers refer to vertical cavity surface-emitting lasers. As an explanation, the near-infrared laser speckle pattern projected by the projector is a static reference pattern. "Fixed" means that the near-infrared laser speckle pattern will not change. For clarity, the eyelid margin area refers to the meibomian gland openings and the surrounding ocular surface area at the edges of the user's upper and lower eyelids.

[0009] Preferably, in S1, the speckle size parameter of the near-infrared laser speckle pattern is 20-50 micrometers. This is because the diameter of a normal meibomian gland opening ranges from 50-150 micrometers, while the diameter of a blocked or atrophied opening will shrink to 20-50 micrometers. The purpose of the 20-50 micrometer design is to ensure that the physical contour of a single meibomian gland opening can produce significant geometric distortion for the multiple speckle particles covering it. In practical applications, the 20-50 micrometer design is significantly smaller than the diameter of a single eyelash (60-100 micrometers) to avoid hair occlusion and the formation of large-area shadows. At the same time, it avoids the manufacturing process bottleneck of speckles smaller than 20 micrometers, balancing detection effect and hardware feasibility, thereby providing a high-contrast, high-discrimination feature basis for subsequent image recognition algorithms. When projecting near-infrared laser speckle patterns, the near-infrared band should be selected to reduce signal differences caused by skin pigmentation. The contrast parameter of the near-infrared laser speckle pattern itself is greater than or equal to 0.8; As an explanation, the contrast parameter of the near-infrared laser speckle pattern describes the degree of brightness difference within the near-infrared laser speckle pattern itself.

[0010] The contrast ratio is calculated by first obtaining the peak brightness of the bright spot region and the valley brightness of the dark spot region in the near-infrared laser speckle pattern; then dividing the difference between the peak brightness of the bright spot region and the valley brightness of the dark spot region by the sum of the peak brightness of the bright spot region and the valley brightness of the dark spot region. The design with a contrast parameter of ≥0.8 aims to ensure clear distinction between light and dark areas in the speckle pattern, capturing the speckle distortion characteristics caused by the meibomian gland openings even with interference from tear specular reflection. This ≥0.8 contrast parameter design is based on the difference in near-infrared reflectance between tears and eyelid skin. Tears have a higher reflectivity in the near-infrared band than eyelid skin, which homogenizes the brightness of the projected speckle. If the speckle itself lacks contrast, the brightness difference will be further reduced after homogenization by tear reflection; however, by pre-calibrating the contrast parameter of the near-infrared laser speckle pattern to ≥80%, the homogenizing effect of tears can be offset.

[0011] Preferably, in S3, a predicted distortion field is generated based on the image of the eyelid margin region and the near-infrared laser speckle pattern, as follows: Based on the image of the eyelid edge region, the highlight area and the non-highlight area are divided on the image of the eyelid edge region; In the non-highlight region, the phase correlation method is first used to register the eyelid edge region image with the near-infrared laser speckle pattern. Then, the dense optical flow algorithm is used to obtain the first two-dimensional displacement vector of the eyelid edge region image relative to the near-infrared laser speckle pattern pixel by pixel. All the first two-dimensional displacement vectors are obtained to form the first displacement vector set, and the first displacement vector set is used as the first two-dimensional displacement field. In the first set of displacement vectors, all first two-dimensional displacement vectors on the boundary line between the highlight area and the non-highlight area are extracted to form the second set of displacement vectors. In the highlight region, the second two-dimensional displacement vector corresponding to each pixel in the highlight region is obtained by solving the Poisson equation for the second displacement vector set. All the second two-dimensional displacement vectors are obtained to form the third displacement vector set, and the third displacement vector set is used as the second two-dimensional displacement field. The first two-dimensional displacement field and the second two-dimensional displacement field are combined to form the predicted distortion field; A specular suppression image is generated based on the estimated distortion field.

[0012] Preferably, based on the eyelid edge region image, highlight areas and non-highlight areas are divided on the eyelid edge region image, as follows: The image of the eyelid edge region is converted into a grayscale image; a threshold segmentation method is used on the grayscale image to generate a binary mask, which is used to divide the eyelid edge region image into highlight regions and non-highlight regions.

[0013] Preferably, in S3, a specular suppression image is generated based on the estimated distortion field, as follows: A blank digital image with the exact same size and spatial coordinate system as the image of the eyelid edge region is used as the reconstructed image; Based on the predicted distortion field, image deformation technology is used to obtain the corresponding source coordinates of each pixel in the reconstructed image in the near-infrared laser speckle pattern; thus forming a coordinate mapping relationship between the reconstructed image and the near-infrared laser speckle pattern. Based on the coordinate mapping relationship between the reconstructed image and the near-infrared laser speckle pattern, the near-infrared laser speckle pattern is resampled to fill the reconstructed image, generating a specular suppression image.

[0014] Preferably, based on the coordinate mapping relationship between the reconstructed image and the near-infrared laser speckle pattern, a resampling operation is performed on the near-infrared laser speckle pattern to fill the reconstructed image, generating a speckle-suppressed image, as follows: For each pixel location in the reconstructed image, the source coordinates corresponding to each pixel location are obtained according to the coordinate mapping relationship between the reconstructed image and the near-infrared laser speckle pattern. The pixel brightness value at the source coordinates is obtained from the near-infrared laser speckle pattern using a bilinear interpolation algorithm. All pixels in the reconstructed image are traversed, and the pixel brightness value at the source coordinates is assigned to the corresponding pixel location in the reconstructed image to generate a specular suppression image.

[0015] The method for providing early warning of dry eye in smart glasses includes the following steps: S1. When using smart glasses, after activating the monitoring function of the smart glasses, a unique and fixed, invisible near-infrared laser speckle pattern is projected onto the edge area of ​​the user's eyelids through the projector. S2. While the projector continuously projects a near-infrared laser speckle pattern, the built-in camera of the smart glasses is used to expose and capture a single frame of the eyelid edge area image. S3. Generate a predicted distortion field based on the eyelid edge region image and the near-infrared laser speckle pattern; generate a speckle-suppressed image based on the predicted distortion field. S4. Based on the highlight suppression image, obtain the effective evaluation region from the highlight suppression image; S5. Based on the effective evaluation area, generate the opening density and average equivalent diameter; S6. Obtain historical normal meibomian gland opening data, which includes meibomian gland opening density and average equivalent diameter of meibomian gland openings. Based on historical normal meibomian gland opening data, an anomaly detection model is generated using the isolated forest algorithm. The anomaly detection model is used to detect whether the input opening density or average equivalent diameter is within the normal range. Input the opening density into the anomaly detection model. If the anomaly detection model outputs an opening density outside the normal range, a dry eye warning is triggered. Input the average equivalent diameter into the anomaly detection model. If the average equivalent diameter output by the anomaly detection model is outside the normal range, a dry eye warning is triggered. As an explanation, meibomian gland dysfunction is the main cause of evaporative dry eye syndrome, so this triggers a dry eye warning.

[0016] Preferably, in S4, the effective evaluation region is obtained from the highlight suppression image, as follows: The highlight-suppressed image was binarized using the Otsu method to obtain an initial binary mask. A morphological closing operation is performed on the initial binary mask to obtain a binary mask for the eyelid region. Image connected component analysis is performed on the binary mask of the eyelid region, and the connected component with the largest area is selected as the main eyelid region; Obtain the minimum bounding rectangle of the main eyelid region and use it as the effective evaluation region in the specular suppression image.

[0017] Preferably, in S5, based on the effective evaluation area, the opening density and average equivalent diameter are generated as follows: The effective evaluation region in the specular suppression image is filtered to obtain the filtered specular suppression image. When filtering a highlight-suppressed image, Laplacian filtering or high-pass filtering can be used. Binarization segmentation was performed on the filtered highlight-suppressed image to obtain candidate meibomian gland opening regions; Obtain all candidate meibomian gland opening regions and count the total number of candidate meibomian gland opening regions; Obtain the pixel area of ​​the effective evaluation region; The opening density is obtained by dividing the total number of candidate meibomian gland opening regions by the pixel area of ​​the effective evaluation region. Obtain the pixel area of ​​the candidate meibomian gland opening region, use the pixel area of ​​the candidate meibomian gland opening region as the area of ​​a circle, obtain the circle diameter of the circle area according to the circle area formula, and use the circle diameter of the circle area as the equivalent circle diameter of the candidate meibomian gland opening region. Obtain the equivalent circle diameter of all candidate meibomian gland opening regions, and take the arithmetic mean of the equivalent circle diameters of all candidate meibomian gland opening regions as the average equivalent diameter.

[0018] The AI-based tear film state monitoring system and smart glasses dry eye early warning method proposed in this invention have the following beneficial technical effects: 1. The AI-based tear film state monitoring system proposed in this application acquires images of the eyelid edge region while continuously projecting a near-infrared laser speckle pattern onto a projector. It utilizes a known, static near-infrared laser speckle pattern as the source texture and constructs a predicted distortion field to guide the deformation of this source texture. A blank digital image with the same size and spatial coordinate system as the eyelid edge region image is used as the reconstructed image. For each pixel position in the reconstructed image, the source coordinates corresponding to each pixel position are obtained according to the coordinate mapping relationship between the reconstructed image and the near-infrared laser speckle pattern. The pixel brightness value at the source coordinates is obtained from the near-infrared laser speckle pattern using a bilinear interpolation algorithm. All pixels in the reconstructed image are traversed, and the pixel brightness value at the source coordinates is assigned to the corresponding pixel position in the reconstructed image, generating a highlight-suppressed image. During this process, the highlight areas that were originally overexposed and lost information due to specular reflection are filled and replaced by new pixel brightness values ​​determined from the source texture through deformation mapping, thus completing highlight restoration. This application relies solely on a single frame of eyelid edge region image, fundamentally avoiding the multi-frame registration failure problem caused by micro-movements of the eyeball; it also alleviates the problem of feature information loss caused by the non-fixed position of the specular reflection highlight on the meniscus of the tear film, and realizes stable imaging of the meibomian gland opening region in a dynamic ocular surface environment.

[0019] 2. The intelligent glasses dry eye early warning method proposed in this application, based on a highlight-suppressed image, processes the highlight-suppressed image to obtain the opening density and average equivalent diameter, and introduces the step of generating an anomaly detection model using the isolated forest algorithm based on historical normal meibomian gland opening data. The isolated forest algorithm automatically learns the normal value range from historical normal data. Then, by inputting the opening density into the anomaly detection model, if the output of the anomaly detection model indicates that the opening density is outside the normal value range, a dry eye early warning is triggered; similarly, by inputting the average equivalent diameter into the anomaly detection model, if the output of the anomaly detection model indicates that the average equivalent diameter is outside the normal value range, a dry eye early warning is triggered. This application avoids reliance on subjectively preset thresholds, achieving objective and automatic early warning of the risk of evaporative dry eye caused by meibomian gland dysfunction, thus improving the reliability in home monitoring scenarios. Attached Figure Description

[0020] Figure 1 This is a block diagram illustrating the principle of the AI-based tear film state monitoring system of the present invention. Figure 2 This is a flowchart of the dry eye warning method for smart glasses according to the present invention. Detailed Implementation

[0021] Embodiments of the present invention are described in detail below. Examples of these embodiments are illustrated in the accompanying drawings, wherein the same or similar symbols denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0022] like Figure 1 The AI-based tear film state monitoring system shown includes: Pattern projection module: When using smart glasses, after activating the monitoring function of the smart glasses, a unique and fixed, invisible near-infrared laser speckle pattern is projected onto the edge area of ​​the user's eyelids through the projector. In an optional embodiment, in the pattern projection module, the projector is integrated on the smart glasses, and the projector is a projection module that includes a VCSEL laser and diffractive optical elements. In an optional embodiment, when projecting near-infrared laser speckle patterns in the pattern projection module, near-infrared band projection is selected; Ocular surface image acquisition module: While the projector continuously projects a near-infrared laser speckle pattern, the built-in camera of the smart glasses exposes and acquires a single frame of the eyelid edge region image. Highlight suppression image generation module: Generates a predicted distortion field based on the eyelid edge region image and the near-infrared laser speckle pattern; generates a highlight suppression image based on the predicted distortion field.

[0023] In an optional embodiment, the specular suppression image generation module generates a predicted distortion field based on the eyelid edge region image and the near-infrared laser speckle pattern, as follows: Based on the image of the eyelid edge region, the highlight area and the non-highlight area are divided on the image of the eyelid edge region; In the non-highlight region, the phase correlation method is first used to register the eyelid edge region image with the near-infrared laser speckle pattern. Then, the dense optical flow algorithm is used to obtain the first two-dimensional displacement vector of the eyelid edge region image relative to the near-infrared laser speckle pattern pixel by pixel. All the first two-dimensional displacement vectors are obtained to form the first displacement vector set, and the first displacement vector set is used as the first two-dimensional displacement field. In the first set of displacement vectors, all first two-dimensional displacement vectors on the boundary line between the highlight area and the non-highlight area are extracted to form the second set of displacement vectors. In the highlight region, the second two-dimensional displacement vector corresponding to each pixel in the highlight region is obtained by solving the Poisson equation for the second displacement vector set. All the second two-dimensional displacement vectors are obtained to form the third displacement vector set, and the third displacement vector set is used as the second two-dimensional displacement field. The first two-dimensional displacement field and the second two-dimensional displacement field are combined to form the predicted distortion field; In an optional embodiment, based on the eyelid edge region image, highlight areas and non-highlight areas are divided on the eyelid edge region image, as follows: The image of the eyelid edge region is converted into a grayscale image; a threshold segmentation method is applied to the grayscale image to generate a binary mask, which is used to divide the image of the eyelid edge region into a highlight region and a non-highlight region. In an optional embodiment, the specular suppression image generation module generates a specular suppression image based on the estimated distortion field, as follows: A blank digital image with the exact same size and spatial coordinate system as the image of the eyelid edge region is used as the reconstructed image; Based on the predicted distortion field, image deformation technology is used to obtain the corresponding source coordinates of each pixel in the reconstructed image in the near-infrared laser speckle pattern; thus forming a coordinate mapping relationship between the reconstructed image and the near-infrared laser speckle pattern. Based on the coordinate mapping relationship between the reconstructed image and the near-infrared laser speckle pattern, the near-infrared laser speckle pattern is resampled to fill the reconstructed image, generating a speckle-suppressed image. In an optional embodiment, based on the coordinate mapping relationship between the reconstructed image and the near-infrared laser speckle pattern, the near-infrared laser speckle pattern is resampled to fill the reconstructed image, generating a speckle-suppressed image, as follows: For each pixel location in the reconstructed image, the source coordinates corresponding to each pixel location are obtained according to the coordinate mapping relationship between the reconstructed image and the near-infrared laser speckle pattern. The pixel brightness value at the source coordinates is obtained from the near-infrared laser speckle pattern using a bilinear interpolation algorithm. All pixels in the reconstructed image are traversed, and the pixel brightness value at the source coordinates is assigned to the corresponding pixel location in the reconstructed image to generate a specular suppression image.

[0024] The AI-based tear film state monitoring system proposed in this application acquires images of the eyelid edge region while continuously projecting a near-infrared laser speckle pattern onto a projector. It utilizes a known, static near-infrared laser speckle pattern as the source texture and constructs a predicted distortion field to guide the deformation of this source texture. A blank digital image with the exact same size and spatial coordinate system as the eyelid edge region image is used as the reconstructed image. For each pixel position in the reconstructed image, the source coordinates corresponding to each pixel position are obtained according to the coordinate mapping relationship between the reconstructed image and the near-infrared laser speckle pattern. The pixel brightness value at the source coordinates is obtained from the near-infrared laser speckle pattern using a bilinear interpolation algorithm. All pixels in the reconstructed image are traversed, and the pixel brightness value at the source coordinates is assigned to the corresponding pixel position in the reconstructed image, generating a highlight-suppressed image. During this process, the highlight areas that were originally overexposed and lost information due to specular reflection are filled and replaced by new pixel brightness values ​​determined from the source texture through deformation mapping, thus completing highlight restoration. This application relies solely on a single frame of eyelid edge region image, fundamentally avoiding the multi-frame registration failure problem caused by micro-movements of the eyeball; it also alleviates the problem of feature information loss caused by the non-fixed position of the specular reflection highlight on the meniscus of the tear film, and realizes stable imaging of the meibomian gland opening region in a dynamic ocular surface environment.

[0025] like Figure 2 The dry eye warning method for smart glasses shown includes the following steps: S1. When using smart glasses, after activating the monitoring function of the smart glasses, a unique and fixed, invisible near-infrared laser speckle pattern is projected onto the edge area of ​​the user's eyelids through the projector. In an optional embodiment, in S1, the projector is integrated into the smart glasses. The projector is a projection module and includes a VCSEL laser and diffractive optical elements. VCSEL lasers refer to vertical cavity surface-emitting lasers. As an explanation, the near-infrared laser speckle pattern projected by the projector is a static reference pattern. "Fixed" means that the near-infrared laser speckle pattern will not change. For clarity, the eyelid margin area refers to the meibomian gland openings and the surrounding ocular surface area at the edges of the user's upper and lower eyelids; To ensure that it can be repeatedly implemented without excessive experimentation by those skilled in the art, the speckle size parameter of the near-infrared laser speckle pattern in S1 is 20-50 micrometers. This is because the diameter of a normal meibomian gland opening ranges from 50 to 150 micrometers, while the diameter of a blocked or atrophied opening will shrink to 20-50 micrometers. The purpose of the 20-50 micrometer design is to ensure that the physical contour of a single meibomian gland opening can produce significant geometric distortion for the multiple speckle particles covering it. From a practical application perspective, the 20-50 micrometer design is significantly smaller than the diameter of a single eyelash, which is 60-100 micrometers, to avoid hair occlusion and the formation of large-area shadows. At the same time, it avoids the manufacturing process bottleneck of speckles smaller than 20 micrometers, balancing detection effect and hardware feasibility, thereby providing a high-contrast, high-discrimination feature basis for subsequent image recognition algorithms. When projecting near-infrared laser speckle patterns, the near-infrared band should be selected to reduce signal differences caused by skin pigmentation. In an optional embodiment, when near-infrared band projection is selected, the wavelength is 850 nm; In an optional embodiment, when near-infrared band projection is selected, the wavelength is 940 nm; The contrast parameter of the near-infrared laser speckle pattern itself is greater than or equal to 0.8; As an explanation, the contrast parameter of the near-infrared laser speckle pattern describes the degree of brightness difference within the near-infrared laser speckle pattern itself.

[0026] The contrast ratio is calculated by first obtaining the peak brightness of the bright spot region and the valley brightness of the dark spot region in the near-infrared laser speckle pattern; then dividing the difference between the peak brightness of the bright spot region and the valley brightness of the dark spot region by the sum of the peak brightness of the bright spot region and the valley brightness of the dark spot region. The design with a contrast parameter of ≥0.8 aims to ensure clear distinction between light and dark areas in the speckle pattern, capturing the speckle distortion characteristics caused by the meibomian gland openings even with interference from tear specular reflection. This ≥0.8 contrast parameter design is based on the difference in near-infrared reflectance between tears and eyelid skin. Tears have a higher reflectivity in the near-infrared band than eyelid skin, which homogenizes the brightness of the projected speckle. If the speckle itself lacks contrast, the brightness difference will be further reduced after homogenization by tear reflection; however, by pre-calibrating the contrast parameter of the near-infrared laser speckle pattern to ≥80%, the homogenizing effect of tears can be offset.

[0027] S2. While the projector continuously projects a near-infrared laser speckle pattern, the built-in camera of the smart glasses is used to expose and capture a single frame of the eyelid edge area image. S3. Generate a predicted distortion field based on the image of the eyelid margin region and the near-infrared laser speckle pattern; In an optional embodiment, in S3, a predicted distortion field is generated based on the eyelid margin region image and the near-infrared laser speckle pattern, as follows: Based on the image of the eyelid edge region, the highlight area and the non-highlight area are divided on the image of the eyelid edge region; In an optional embodiment, based on the eyelid edge region image, highlight areas and non-highlight areas are divided on the eyelid edge region image, as follows: The image of the eyelid edge region is converted into a grayscale image; a threshold segmentation method is applied to the grayscale image to generate a binary mask, which is used to divide the image of the eyelid edge region into a highlight region and a non-highlight region. In the non-highlight region, the phase correlation method is first used to register the eyelid edge region image with the near-infrared laser speckle pattern. Then, the dense optical flow algorithm is used to obtain the first two-dimensional displacement vector of the eyelid edge region image relative to the near-infrared laser speckle pattern pixel by pixel. All the first two-dimensional displacement vectors are obtained to form the first displacement vector set, and the first displacement vector set is used as the first two-dimensional displacement field. In the first set of displacement vectors, all first two-dimensional displacement vectors on the boundary line between the highlight area and the non-highlight area are extracted to form the second set of displacement vectors. In the highlight region, the second two-dimensional displacement vector corresponding to each pixel in the highlight region is obtained by solving the Poisson equation for the second displacement vector set. All the second two-dimensional displacement vectors are obtained to form the third displacement vector set, and the third displacement vector set is used as the second two-dimensional displacement field. The first two-dimensional displacement field and the second two-dimensional displacement field are combined to form the predicted distortion field; Generate a specular-suppressed image based on the estimated distortion field; In an optional embodiment, in S3, a specular suppression image is generated based on the estimated distortion field, as follows: A blank digital image with the exact same size and spatial coordinate system as the image of the eyelid edge region is used as the reconstructed image; Based on the predicted distortion field, image deformation technology is used to obtain the corresponding source coordinates of each pixel in the reconstructed image in the near-infrared laser speckle pattern; thus forming a coordinate mapping relationship between the reconstructed image and the near-infrared laser speckle pattern. Based on the coordinate mapping relationship between the reconstructed image and the near-infrared laser speckle pattern, the near-infrared laser speckle pattern is resampled to fill the reconstructed image, generating a speckle-suppressed image. In an optional embodiment, based on the coordinate mapping relationship between the reconstructed image and the near-infrared laser speckle pattern, the near-infrared laser speckle pattern is resampled to fill the reconstructed image, generating a speckle-suppressed image, as follows: For each pixel location in the reconstructed image, the source coordinates corresponding to each pixel location are obtained according to the coordinate mapping relationship between the reconstructed image and the near-infrared laser speckle pattern. The pixel brightness value at the source coordinates is obtained from the near-infrared laser speckle pattern using a bilinear interpolation algorithm. All pixels in the reconstructed image are traversed, and the pixel brightness value at the source coordinates is assigned to the corresponding pixel location in the reconstructed image to generate a specular suppression image. This application utilizes a known, static near-infrared laser speckle pattern as the source texture, and guides the deformation of the source texture through a predicted distortion field. The predicted distortion field describes the geometric distortion of the three-dimensional shape of the eye surface to the original near-infrared laser speckle pattern in the absence of specular interference, thereby completing specular restoration within a single frame of an image of the eyelid edge region.

[0028] S4. Based on the highlight suppression image, obtain the effective evaluation region from the highlight suppression image; In an optional embodiment, in S4, the effective evaluation region is obtained from the highlight suppression image based on the highlight suppression image, as follows: The highlight-suppressed image was binarized using the Otsu method to obtain an initial binary mask. A morphological closing operation is performed on the initial binary mask to obtain a binary mask for the eyelid region. Image connected component analysis is performed on the binary mask of the eyelid region, and the connected component with the largest area is selected as the main eyelid region; Obtain the minimum bounding rectangle of the main eyelid region, and use the minimum bounding rectangle of the main eyelid region as the effective evaluation region in the specular suppression image; S5. Based on the effective evaluation area, generate the opening density and average equivalent diameter; In an optional embodiment, in S5, based on the effective evaluation area, the opening density and average equivalent diameter are generated as follows: The effective evaluation region in the specular suppression image is filtered to obtain the filtered specular suppression image. When filtering a highlight-suppressed image, Laplacian filtering or high-pass filtering can be used. Binarization segmentation was performed on the filtered highlight-suppressed image to obtain candidate meibomian gland opening regions; Obtain all candidate meibomian gland opening regions and count the total number of candidate meibomian gland opening regions; Obtain the pixel area of ​​the effective evaluation region; The opening density is obtained by dividing the total number of candidate meibomian gland opening regions by the pixel area of ​​the effective evaluation region. Obtain the pixel area of ​​the candidate meibomian gland opening region, use the pixel area of ​​the candidate meibomian gland opening region as the area of ​​a circle, obtain the circle diameter of the circle area according to the circle area formula, and use the circle diameter of the circle area as the equivalent circle diameter of the candidate meibomian gland opening region. Obtain the equivalent circle diameter of all candidate meibomian gland opening regions, and take the arithmetic mean of the equivalent circle diameters of all candidate meibomian gland opening regions as the average equivalent diameter. S6. Obtain historical normal meibomian gland opening data, which includes meibomian gland opening density and average equivalent diameter of meibomian gland openings. Based on historical normal meibomian gland opening data, an anomaly detection model is generated using the isolated forest algorithm. The anomaly detection model is used to detect whether the input opening density or average equivalent diameter is within the normal range. Input the opening density into the anomaly detection model. If the anomaly detection model outputs an opening density outside the normal range, a dry eye warning is triggered. Input the average equivalent diameter into the anomaly detection model. If the average equivalent diameter output by the anomaly detection model is outside the normal range, a dry eye warning is triggered. As an explanation, meibomian gland dysfunction is the main cause of evaporative dry eye syndrome, so this triggers a dry eye warning.

[0029] The proposed intelligent glasses dry eye early warning method, based on a highlight-suppressed image, processes the image to obtain the opening density and average equivalent diameter, and introduces an anomaly detection model using the isolated forest algorithm based on historical normal meibomian gland opening data. The isolated forest algorithm automatically learns the normal value range from historical normal data. Then, by inputting the opening density into the anomaly detection model, a dry eye warning is triggered if the model's output indicates that the opening density is outside the normal range; similarly, the average equivalent diameter is input into the anomaly detection model, and a dry eye warning is triggered if the model's output indicates that the average equivalent diameter is outside the normal range. This application avoids reliance on subjectively preset thresholds, achieving objective and automatic early warning of the risk of evaporative dry eye caused by meibomian gland dysfunction, thus improving the reliability and practicality in home monitoring scenarios.

[0030] As an explanation, this application is a targeted design for the meibomian gland opening situation, but this does not mean that the dry eye warning is only based on the meibomian gland opening situation. For example, it is also necessary to acquire and analyze the tear hemorrhage height, as the amount of tear fluid can be indirectly reflected by the tear hemorrhage height. Currently, a tear hemorrhage height of <0.2mm indicates that the tear fluid volume may be insufficient; it is also necessary to acquire the tear film breakup time; these are all existing mature technologies, so they are not explained.

[0031] Furthermore, any content not described in detail in this specification is existing technology known to those skilled in the art.

[0032] In the embodiments provided by this invention, it should be understood that the disclosed system or method can be implemented in other ways. For example, the embodiments of the invention described above are merely illustrative; for instance, the division of modules is only a logical functional division, and there may be other division methods in actual implementation.

[0033] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0034] Furthermore, the functional modules in the various embodiments of this invention can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated module can be implemented in hardware or in a combination of hardware and software functional modules.

[0035] For those skilled in the art, it is obvious that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the basic characteristics of the present invention.

[0036] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. An AI-based tear film state monitoring system, characterized in that, include: Pattern projection module: When using smart glasses, after activating the monitoring function of the smart glasses, a unique and fixed, invisible near-infrared laser speckle pattern is projected onto the edge area of ​​the user's eyelids through the projector. Ocular surface image acquisition module: While the projector continuously projects a near-infrared laser speckle pattern, the built-in camera of the smart glasses exposes and acquires a single frame of the eyelid edge region image. Highlight suppression image generation module: Generates a predicted distortion field based on the eyelid edge region image and the near-infrared laser speckle pattern; generates a highlight suppression image based on the predicted distortion field.

2. The AI-based tear film state monitoring system according to claim 1, characterized in that, In the pattern projection module, the projector is integrated into the smart glasses. The projector is a projection module that includes a VCSEL laser and diffractive optical elements.

3. The AI-based tear film state monitoring system according to claim 1 or 2, characterized in that, In the pattern projection module, near-infrared laser speckle pattern projection is performed using the near-infrared band.

4. The AI-based tear film state monitoring system according to claim 1, characterized in that, In the specular suppression image generation module, a predicted distortion field is generated based on the eyelid edge region image and the near-infrared laser speckle pattern, as follows: Based on the image of the eyelid edge region, the highlight area and the non-highlight area are divided on the image of the eyelid edge region; In the non-highlight region, the phase correlation method is first used to register the eyelid edge region image with the near-infrared laser speckle pattern. Then, the dense optical flow algorithm is used to obtain the first two-dimensional displacement vector of the eyelid edge region image relative to the near-infrared laser speckle pattern pixel by pixel. All the first two-dimensional displacement vectors are obtained to form the first displacement vector set, and the first displacement vector set is used as the first two-dimensional displacement field. In the first set of displacement vectors, all first two-dimensional displacement vectors on the boundary line between the highlight area and the non-highlight area are extracted to form the second set of displacement vectors. In the highlight region, the second two-dimensional displacement vector corresponding to each pixel in the highlight region is obtained by solving the Poisson equation for the second displacement vector set. All the second two-dimensional displacement vectors are obtained to form the third displacement vector set, and the third displacement vector set is used as the second two-dimensional displacement field. The first two-dimensional displacement field and the second two-dimensional displacement field are combined to form the predicted distortion field.

5. The AI-based tear film state monitoring system according to claim 4, characterized in that, Based on the image of the eyelid edge region, highlight areas and non-highlight areas are divided on the eyelid edge region image, as follows: The image of the eyelid edge region is converted into a grayscale image; a threshold segmentation method is used on the grayscale image to generate a binary mask, which is used to divide the eyelid edge region image into highlight regions and non-highlight regions.

6. The AI-based tear film state monitoring system according to claim 4 or 5, characterized in that, In the specular suppression image generation module, a specular suppression image is generated based on the estimated distortion field, as follows: A blank digital image with the exact same size and spatial coordinate system as the image of the eyelid edge region is used as the reconstructed image; Based on the predicted distortion field, image deformation technology is used to obtain the corresponding source coordinates of each pixel in the reconstructed image in the near-infrared laser speckle pattern; thus forming a coordinate mapping relationship between the reconstructed image and the near-infrared laser speckle pattern. Based on the coordinate mapping relationship between the reconstructed image and the near-infrared laser speckle pattern, the near-infrared laser speckle pattern is resampled to fill the reconstructed image, generating a specular suppression image.

7. The AI-based tear film state monitoring system according to claim 6, characterized in that, Based on the coordinate mapping relationship between the reconstructed image and the near-infrared laser speckle pattern, the near-infrared laser speckle pattern is resampled to fill the reconstructed image, generating a speckle-suppressed image, as follows: For each pixel location in the reconstructed image, the source coordinates corresponding to each pixel location are obtained according to the coordinate mapping relationship between the reconstructed image and the near-infrared laser speckle pattern. The pixel brightness value at the source coordinates is obtained from the near-infrared laser speckle pattern using a bilinear interpolation algorithm. All pixels in the reconstructed image are traversed, and the pixel brightness value at the source coordinates is assigned to the corresponding pixel location in the reconstructed image to generate a specular suppression image.

8. A method for early warning of dry eye in smart glasses, using the AI-based tear film state monitoring system according to any one of claims 1-7, characterized in that, Includes the following steps: S1. When using smart glasses, after activating the monitoring function of the smart glasses, a unique and fixed, invisible near-infrared laser speckle pattern is projected onto the edge area of ​​the user's eyelids through the projector. S2. While the projector continuously projects a near-infrared laser speckle pattern, the built-in camera of the smart glasses is used to expose and capture a single frame of the eyelid edge area image. S3. Generate a predicted distortion field based on the eyelid edge region image and the near-infrared laser speckle pattern; generate a speckle-suppressed image based on the predicted distortion field. S4. Based on the highlight suppression image, obtain the effective evaluation region from the highlight suppression image; S5. Based on the effective evaluation area, generate the opening density and average equivalent diameter; S6. Obtain historical normal meibomian gland opening data, including meibomian gland opening density and average equivalent diameter of meibomian gland openings. Based on the historical normal meibomian gland opening data, use the isolated forest algorithm to generate an anomaly detection model. The anomaly detection model is used to detect whether the input opening density or average equivalent diameter is within the normal range. Input the opening density into the anomaly detection model. If the anomaly detection model outputs that the opening density is outside the normal range, trigger a dry eye warning. Input the average equivalent diameter into the anomaly detection model. If the anomaly detection model outputs that the average equivalent diameter is outside the normal range, trigger a dry eye warning.

9. The method for early warning of dry eye in smart glasses according to claim 8, characterized in that, In S4, based on the highlight suppression image, the effective evaluation region is obtained from the highlight suppression image, as follows: The highlight-suppressed image was binarized using the Otsu method to obtain an initial binary mask. A morphological closing operation is performed on the initial binary mask to obtain a binary mask for the eyelid region. Image connected component analysis is performed on the binary mask of the eyelid region, and the connected component with the largest area is selected as the main eyelid region; Obtain the minimum bounding rectangle of the main eyelid region and use it as the effective evaluation region in the specular suppression image.

10. The method for early warning of dry eye in smart glasses according to claim 9, characterized in that, In S5, based on the effective evaluation area, the opening density and average equivalent diameter are generated as follows: The effective evaluation region in the specular suppression image is filtered to obtain the filtered specular suppression image. When filtering a highlight-suppressed image, Laplacian filtering or high-pass filtering can be used. Binarization segmentation was performed on the filtered highlight-suppressed image to obtain candidate meibomian gland opening regions; Obtain all candidate meibomian gland opening regions and count the total number of candidate meibomian gland opening regions; Obtain the pixel area of ​​the effective evaluation region; The opening density is obtained by dividing the total number of candidate meibomian gland opening regions by the pixel area of ​​the effective evaluation region. Obtain the pixel area of ​​the candidate meibomian gland opening region, use the pixel area of ​​the candidate meibomian gland opening region as the area of ​​a circle, obtain the circle diameter of the circle area according to the circle area formula, and use the circle diameter of the circle area as the equivalent circle diameter of the candidate meibomian gland opening region. Obtain the equivalent circle diameter of all candidate meibomian gland opening regions, and take the arithmetic mean of the equivalent circle diameters of all candidate meibomian gland opening regions as the average equivalent diameter.

Images