Method and system for diagnosing dry eye syndrome using rgb-to-spectrum conversion-based hyperspectral imaging

Abstract

The method for diagnosing dry eye syndrome using an RGB-spectrum conversion-based hyperspectral image according to the present invention comprises the steps of acquiring an image of a user's eye, generating a hyperspectral image by predicting wavelength-specific reflectance data of each pixel constituting the eye image, generating state data for each eye structure of the user based on the wavelength-specific reflectance data of the hyperspectral image, and inputting the state data for each eye structure into an artificial intelligence model to diagnose the user's dry eye syndrome, thereby enabling non-invasive and quantitative diagnosis of dry eye syndrome based on the hyperspectral image generated from the RGB eye image and artificial intelligence analysis.

Description

Technology Field

[0001] The present invention relates to a method and system for diagnosing dry eye disease using RGB-spectrum conversion-based hyperspectral images. More specifically, it relates to a technology for precisely diagnosing dry eye disease by generating hyperspectral images through an MLP model using RGB images as input values ​​and analyzing complex biomarkers for each ocular structure based thereon. Background Technology

[0002] Dry eye syndrome is one of the most common ophthalmic diseases worldwide, with a prevalence rate of approximately 60% in Korea. Recently, the incidence rate has been rapidly increasing due to external environmental factors such as an aging population, increased use of smart devices, fine dust, and climate change. In particular, patients who have undergone corneal resection or incision surgeries, such as LASIK or cataract surgery, often develop chronic dry eye syndrome as tear film instability and reduced secretion caused by corneal nerve damage persist over the long term. Since this condition can go beyond mere discomfort and lead to vision loss, chronic inflammation, and in severe cases, blindness due to corneal damage, early diagnosis and precise pathological analysis are essential.

[0003] However, current diagnostic technologies for dry eye disease are often limited to fragmentary classifications based on severity, and it is difficult to quantitatively analyze lesion indicators for multilayer structures such as the lipid layer, aqueous layer, mucin layer, cornea, and meibomian glands, or to comprehensively evaluate their interactions. Existing tests, such as the Schirmer test, TBUT (tear film breakup time), and meibography—representative diagnostic methods—are based on independent indicators. Consequently, there are limitations in image interpretation and indicator analysis due to subjective biases by the examiner, and it is difficult to reflect the subtle progression of the disease. Furthermore, since most diagnostic equipment relies on imports, issues such as limited accessibility to expensive devices and discrepancies in resolution and analysis standards exist.

[0004] Accordingly, there is a growing need for technology that can quantitatively analyze various biomarkers on the ocular surface using high-resolution hyperspectral imaging and enable integrated and objective diagnosis based on multi-wavelength spectral data predicted from RGB images.

[0005] In particular, there is a need for a new approach that can realize quantitative and highly reproducible diagnosis of dry eye disease in a low-cost and non-invasive manner by utilizing deep learning models to restore multi-wavelength reflectance information from RGB images and precisely extracting pathological indicators for each ocular structural layer. Prior art literature

[0006] U.S. Patent Application Publication US2022 / 0313077 The problem to be solved

[0007] In addressing the aforementioned problems, the objective of the present invention is to overcome the subjectivity and fragmentation of existing diagnostic methods and realize an integrated and objective diagnosis of dry eye disease by converting RGB images into hyperspectral images through MLP-based spectral transformation and multi-stage optimization to quantitatively derive biomarkers for each structural layer of the eye.

[0008] The problems that the present invention aims to solve are not limited to those mentioned above, and other problems not mentioned will be clearly understood by a person skilled in the art to which the present invention belongs from the description below. means of solving the problem

[0009] A method for diagnosing dry eye syndrome using an RGB-spectrum conversion-based hyperspectral image according to one embodiment may include the steps of: quantifying pixel values ​​of an RGB image of a user's eye and converting them into standardized color space data; inputting color space data into a Multi-Layer Perceptron (MLP) model and cross-validating the spectral values ​​of predicted RGB pixels to evaluate the accuracy of the MLP model; performing first-order optimization to improve the accuracy of the MLP model; applying statistical techniques to perform second-order optimization to correct additional errors in the MLP model; generating a hyperspectral image by predicting wavelength-specific reflectance data of each pixel from the RGB image using the MLP model; and diagnosing dry eye syndrome in a user based on values ​​calculated from the wavelength-specific reflectance data of the hyperspectral image to determine the condition indicators of the user's eye structure.

[0010] According to one embodiment, the step of converting to standardized color space data may include the step of correcting color deviation between devices by correcting lighting conditions and camera sensitivity characteristics at the time of shooting for each pixel value constituting an RGB image, the step of converting the corrected pixel value into a CIE XYZ color space, and the step of normalizing the converted CIE XYZ value to convert it into standardized color space data suitable for input to an MLP model.

[0011] According to one embodiment, the step of evaluating the accuracy of an MLP model may include the step of training an MLP model using standardized color space data, the step of predicting spectrum values ​​for each wavelength band of RGB pixels using the trained MLP model and performing cross-validation (Leave-one-out) by comparing the predicted spectrum values ​​with actual measured spectrum data, and the step of evaluating the prediction accuracy of the MLP model by calculating Root Mean Square Error (RMSE) and RGB expanded bias indicators from the cross-validation results.

[0012] According to one embodiment, the step of performing first-order optimization may be to perform a learning process of iteratively adjusting the weights and bias parameters of the MLP model to minimize the loss function defined by the RMSE and RGB extended bias calculated in the step of evaluating the accuracy of the MLP model.

[0013] According to one embodiment, the step of performing second-order optimization may include a step of calculating a residual by comparing the difference between a predicted spectrum value output from an MLP model and an actual measured spectrum value, a step of calculating a correction matrix based on the residual, and a step of repeatedly applying the calculated correction matrix to the prediction result of the MLP model to reduce the prediction error and recalculating the residual using the corrected result accordingly.

[0014] According to one embodiment, the step of generating a hyperspectral image may include predicting the reflectance value for each pixel of an RGB image by wavelength band and generating a three-dimensional data cube (x, y, λ) composed of position information (x, y) of each pixel and reflectance information according to multiple wavelength bands (λ) in the position information.

[0015] According to one embodiment, the method may further include the steps of: generating image data representing at least one state among a lipid layer, an aqueous layer, a mucin layer, and a cornea by visualizing reflectance values ​​of one or more wavelength ranges selected from a three-dimensional data cube; and estimating thickness, uniformity, moisture state, and degree of surface damage for at least one structure among the lipid layer, an aqueous layer, a mucin layer, and a cornea from the image data.

[0016] According to one embodiment, it may be a wavelength range in which the change in reflectance according to the optical properties of the lipid layer, aqueous layer, mucus layer, and cornea appears above a reference value, a wavelength range selected as a feature to distinguish the layers during the learning process of an artificial intelligence model, or a wavelength range dynamically set through a correction algorithm according to the user's eye condition, lighting conditions, or shooting environment.

[0017] According to one embodiment, the step of generating state data for each eye structure may include: analyzing the spatial distribution of the reflectance of the lipid layer by wavelength range and calculating the thickness and uniformity of the lipid layer based on local deviations in the spatial distribution; analyzing the change in reflectance obtained in a plurality of wavelength ranges corresponding to the aqueous layer and deriving the stability and local non-uniform distribution of the aqueous layer based on the pattern of the change in reflectance; and analyzing the reflectance and absorption rate of the mucus layer by wavelength range and predicting whether there is a change in viscosity or detachment of the mucus layer.

[0018] According to one embodiment, the step of generating state data by ocular structure may include analyzing the reflectance by wavelength collected from the surface of the cornea and estimating the moisture state and the degree of surface damage of the cornea based on changes in reflectance, and calculating an index for predicting the tear film break-up time (TBUT) from the reflectance change pattern accumulated in chronological order.

[0019] According to one embodiment, the step of generating state data by ocular structure may include analyzing the reflectance of the meibomian gland region by wavelength and determining at least one of the sebum secretion status, structural continuity, and closure status of the meibomian gland based on changes in reflectance.

[0020] According to one embodiment, the step of diagnosing a user's dry eye syndrome may include the step of calculating a dry eye risk score through a neural network model based on the interrelationships of a plurality of state indicators derived from the user's eye structure state data as input, and the step of determining a risk grade of the user's eye condition based on the calculated dry eye risk score.

[0021] According to one embodiment, the method may include the step of automatically generating a customized report including a description, recommended measures, and a status summary corresponding to the risk level of the user's eye condition, and the step of transmitting the customized report to at least one of the user's terminal, the medical staff terminal, and the hospital server.

[0022] According to one embodiment, the artificial intelligence model uses a multi-output regression model that takes time-series data of the user's eye structure status data as input and calculates at least one predicted value among tear film stability, interlayer interaction index, water evaporation rate, and ocular surface abnormality distribution area, wherein the multi-output regression model may be a multi-layer neural network-based model trained in a supervised learning manner using a plurality of pre-labeled eye image data and diagnostic result data corresponding to the image data as training data.

[0023] According to one embodiment, the step of acquiring an eye image of a user may include the step of sequentially acquiring a plurality of eye images before and after the user's eye blinking or over time to generate temporal image data.

[0024] According to one embodiment, the step of acquiring an image of a user's eyeball may include providing a guide interface to which preprocessing conditions are applied, including fixing the user's gaze, inducing eyelid opening and closing, and setting lighting conditions, and photographing the front of the user's eyeball using an RGB camera under predetermined distance and light intensity conditions while the user is induced to gaze at a fixation point displayed on the guide interface.

[0025] According to one embodiment, a computer program stored on a computer-readable recording medium can be provided for executing a dry eye disease diagnosis method using RGB-spectrum conversion-based hyperspectral images on a computer.

[0026] According to one embodiment, a dry eye diagnosis device using an RGB-spectrum conversion-based hyperspectral image can be provided, comprising: an image acquisition unit that quantifies pixel values ​​of an RGB image of a user's eye and converts them into standardized color space data; an MLP optimization unit that inputs color space data into a Multi-Layer Perceptron (MLP) model to cross-validate the spectral values ​​of predicted RGB pixels to evaluate the accuracy of the MLP model, performs first-order optimization to improve the accuracy of the MLP model, and performs second-order optimization by applying statistical techniques to correct additional errors in the MLP model; a hyperspectral conversion unit that generates a hyperspectral image by predicting wavelength-specific reflectance data of each pixel from the RGB image using the MLP model; and a diagnosis processing unit that diagnoses the user's dry eye based on values ​​calculated from the wavelength-specific reflectance data of the hyperspectral image to determine the user's eye structure status indicator. Effects of the invention

[0027] As described above, the method and device for diagnosing dry eye disease using RGB-spectrum conversion-based hyperspectral images according to the present invention can quantify RGB images at the pixel level and minimize color deviations depending on the imaging device and environment through CIE XYZ color space conversion and normalization. Through this, variability between devices or between imaging conditions is reduced, thereby enabling the securing of reliable and standardized input data not only in clinical environments but also in various imaging environments.

[0028] In addition, the present invention can simultaneously improve the precision of model prediction and generalization performance by using a Multi-Layer Perceptron (MLP) model to accurately estimate reflectance by wavelength from RGB images, evaluating the model's fit through cross-validation (Leave-one-out), and then iteratively performing first-order optimization to minimize RMSE and Bias, and second-order statistical optimization using Moore-Penrose pseudo-inverse and backslash operations.

[0029] Furthermore, wavelength-specific reflection information can be acquired in a low-cost, non-invasive manner without the need for conventional, complex, and expensive hyperspectral cameras. The resulting hyperspectral images can calculate quantitative indicators such as thickness, uniformity, moisture status, viscosity changes, and the degree of surface damage for major ocular structures like the lipid layer, aqueous layer, mucin layer, cornea, and meibomian glands, thereby supporting clinically meaningful pathological interpretation and monitoring.

[0030] Furthermore, by combining a standard RGB camera with software-based spectrum conversion technology, the present invention can be applied in existing image-based inspection environments without the need for separate hardware modifications. As a non-invasive and cost-effective diagnostic solution, it offers scalability to various environments, including not only clinical settings but also remote diagnostics and personal healthcare devices.

[0031] The effects of the present invention are not limited to those mentioned above, and other unmentioned effects will be clearly understood by a person skilled in the art to which the present invention pertains from the description below. Brief explanation of the drawing

[0032] FIG. 1 is a diagram showing the schematic configuration flow of a dry eye diagnosis system using an RGB-spectrum conversion-based hyperspectral image according to one embodiment. FIG. 2 is a flowchart illustrating a method for diagnosing dry eye syndrome using an RGB-spectrum conversion-based hyperspectral image according to one embodiment. FIG. 3 is a diagram illustrating a process of generating a hyperspectral image based on a user's eye image according to one embodiment and estimating the thickness by analyzing the state of the geological layer. FIG. 4 shows an MLP model structure diagram for generating hyperspectral images based on RGB-spectral conversion according to one embodiment. FIG. 5 is a diagram showing the process of separating the RGB channels of an eye image and extracting the reflectance by wavelength according to one embodiment. Figure 6 is a diagram showing the results of measuring the thickness of the lipid layer in the eyelid region for each subject according to one embodiment. Figure 7 shows an example of state analysis of the layered structure of the tear film based on ocular imaging according to one embodiment. FIG. 8 is a diagram showing the step-by-step visualization of the results of color correction of an eye image captured by a user terminal according to one embodiment and analysis of the lipid layer thickness measurement results. Figure 9 is a graph showing the results of comparing the lipid layer thickness by subject group according to one embodiment. FIG. 10 is a diagram showing the configuration of a dry eye diagnosis system using an RGB-spectrum conversion-based hyperspectral image according to one embodiment. Specific details for implementing the invention

[0033] In the present invention, the attached drawings may be illustrated with exaggerated expressions to distinguish it from the prior art, ensure clarity, and facilitate the understanding of the technology. Furthermore, the terms described below are defined considering their functions in the present invention; since these terms may vary depending on the intentions or conventions of the user or operator, their definitions should be based on the technical content throughout this specification. Meanwhile, the embodiments are merely exemplary details of the components presented in the claims of the present invention and do not limit the scope of the rights of the present invention; the scope of rights should be interpreted based on the technical concept throughout the specification of the present invention.

[0034] Throughout the specification, when a configuration is described as "including" a configuration, this means that, unless specifically stated otherwise, it does not exclude other configurations but may include additional configurations.

[0035] Furthermore, when it is said that one configuration is "connected," "connected," or "combined" with another configuration, this means that it is not only "directly connected," "directly connected," or "directly combined," but also that there may be cases where it is "connected with another configuration interposed," "connected with another configuration interposed," or "combined with another configuration interposed." On the other hand, when it is said that one configuration is "directly connected," "directly connected," or "directly combined" with another configuration, it should be understood that there is no other configuration in between.

[0036] In addition, when directional terms such as "front," "back," "up," "down," "left," "right," "first end," "other end," and "both ends" are used, they are used exemplarily in relation to the orientation of the disclosed drawings and should not be interpreted restrictively, and when terms such as "first" and "second" are used, they are terms used to distinguish each configuration and should not be interpreted restrictively.

[0037] In order to more clearly explain the features of the embodiments of the present invention, detailed descriptions of matters widely known to those skilled in the art to which the following embodiments pertain are omitted. Additionally, detailed descriptions of parts in the drawings that are unrelated to the description of the embodiments are omitted.

[0038] Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings.

[0039] In the following, the dry eye diagnosis system (100) using RGB-spectrum conversion-based hyperspectral images will be described as a dry eye diagnosis system (100).

[0040] FIG. 1 is a diagram showing the schematic configuration flow of a dry eye diagnosis system using an RGB-spectrum conversion-based hyperspectral image according to one embodiment.

[0041] According to one embodiment, the dry eye diagnosis system (100) performs image acquisition of the eye using a user's smartphone or a hospital diagnostic device, and at this time, precisely recognizes the position of the eye through an auto-tracking function and can acquire an image of the eye under optimal shooting conditions. Furthermore, the dry eye diagnosis system (100) applies an auto-tracking function during this process to stably maintain the user's gaze and enable the acquisition of an optimal image of the eye.

[0042] Furthermore, the dry eye diagnosis system (100) can provide advanced data-based learning and optimization techniques to estimate and restore hyperspectral images from general RGB images. Specifically, it can ensure the quality of input data by converting and quantifying each pixel unit data of an RGB image into a standardized color space, and learn the relationship of spectral transformation of RGB images through a non-linear mapping model based on an MLP (Multi-Layer Perceptron). In this process, it can not stop at simple learning, but can verify the generalization performance of the model using cross-validation (Leave-one-out), and gradually improve the prediction accuracy and stability of the model by repeatedly performing loss minimization based on RMSE and Bias (first-order optimization) and statistical correction based on Moore-Penrose pseudo-inverse matrix and backslash operation (second-order optimization).

[0043] According to one embodiment, a dry eye diagnosis system (100) can generate a hyperspectral image from an acquired RGB eye image and quantitatively analyze the condition of major structures of the eye, such as the lipid layer, aqueous layer, mucin layer, and cornea, based on reflectance information of each wavelength range.

[0044] According to one embodiment, a dry eye disease diagnosis system (100) performs an analysis of the condition of major structures of the eye through a multilayer neural network-based artificial intelligence model, predicts multiple condition indicators including the interaction between constituent layers of the tear film, the tear film breakup time (TBUT), and the degree of surface damage, and calculates a dry eye disease risk score by synthesizing these.

[0045] According to one embodiment, the dry eye diagnosis system (100) visualizes the eye diagnosis results and provides them to the user, generates a report including a condition description, risk level, recommended measures, etc. customized for the user, and can transmit the report to a smartphone, a hospital system, or a cloud-based server.

[0046] According to one embodiment, FIG. 1 is a schematic diagram briefly illustrating the representative features of a dry eye diagnosis system (100), and the actual implementation can be applied in various forms, such as a portable mobile device or a fixed device for hospitals.

[0047] FIG. 2 is a flowchart illustrating a method for diagnosing dry eye syndrome using an RGB-spectrum conversion-based hyperspectral image according to one embodiment.

[0048] According to one embodiment, the method may include the steps of: quantifying pixel values ​​of an RGB image of a user's eye and converting them into standardized color space data (S10); inputting color space data into a Multi-Layer Perceptron (MLP) model and cross-validating the spectral values ​​of the predicted RGB pixels to evaluate the accuracy of the MLP model (S20); performing first-order optimization to improve the accuracy of the MLP model (S30); applying statistical techniques to perform second-order optimization to correct additional errors in the MLP model (S40); generating a hyperspectral image by predicting wavelength-specific reflectance data of each pixel from the RGB image using the MLP model (S50); and diagnosing dry eye syndrome of the user based on values ​​calculated from the wavelength-specific reflectance data of the hyperspectral image to determine the condition indicator of the user's eye structure (S60).

[0049] According to one embodiment, the dry eye diagnosis system (100) can acquire an eye image in RGB format by taking a picture of the user's eye through an image acquisition unit of a smartphone or a hospital diagnostic device. According to one embodiment, the step of converting to standardized color space data (S10) may include the step of correcting color deviation between devices by correcting lighting conditions and camera sensitivity characteristics at the time of shooting for each pixel value constituting the RGB image, the step of converting the corrected pixel value into a CIE XYZ color space, and the step of normalizing the converted CIE XYZ value to convert it into standardized color space data suitable for input to an MLP model.

[0050] According to one embodiment, the step (S20) of evaluating the accuracy of the MLP model may include the step of training the MLP model using standardized color space data, the step of predicting the spectrum values ​​for each wavelength band of RGB pixels using the trained MLP model, the step of performing cross-validation (Leave-one-out) by comparing the predicted spectrum values ​​with actual measured spectrum data, and the step of evaluating the prediction accuracy of the MLP model by calculating the Root Mean Square Error (RMSE) and RGB expanded bias indicators from the cross-validation results.

[0051] According to one embodiment, the MLP model is a multilayer neural network structure composed of an input layer, one or more hidden layers, and an output layer. By linearly transforming input data through weights and biases and then hierarchically processing it by applying a nonlinear activation function, it can approximate complex nonlinear relationships between input and output that are difficult to capture with a simple linear model. In the present invention, it is used to learn the mapping between RGB color space data and spectral values ​​by wavelength range.

[0052] According to one embodiment, MLP model training is performed using a supervised learning method with input data and ground truths (actual spectrum values). The difference between the output predicted by the model and the actual value is calculated as a loss function, and weights and biases are iteratively adjusted using backpropagation and gradient descent to optimize the model and gradually reduce the prediction error.

[0053] Furthermore, the leave-one-out cross-validation method can be applied to verify the generalization performance of the trained MLP model. That is, one dataset is set aside for validation, and the remainder is used for training; this process is repeated to evaluate prediction performance on all data. The Root Mean Square Error (RMSE) calculated at this stage represents the magnitude of the mean squared error between the predicted and actual values, while RGB expanded bias reflects the systematic prediction error in specific wavelength ranges. Through these metrics, it is possible to quantitatively evaluate how accurately and unbiasedly the model can reconstruct spectrum values.

[0054] According to one embodiment, the step (S30) of performing first-order optimization may be to perform a learning process of iteratively adjusting the weights and bias parameters of the MLP model in order to minimize the loss function defined by the RMSE and RGB extended bias calculated in the step of evaluating the accuracy of the MLP model.

[0055] According to one embodiment, the difference between the spectrum value predicted by the MLP model from an RGB image and the actual measured spectrum value is calculated, and the error is expressed as RMSE and Bias indicators and defined as a loss function. In addition, to minimize the calculated loss function, the model is optimized by repeating the learning process while adjusting the weights and biases of the MLP model multiple times so that the prediction result becomes increasingly closer to the actual spectrum value.

[0056] According to one embodiment, the step (S40) of performing second-order optimization may include the step of calculating a residual by comparing the difference between the predicted spectrum value output from the MLP model and the actual measured spectrum value, the step of calculating a correction matrix based on the residual, and the step of repeatedly applying the calculated correction matrix to the prediction result of the MLP model to reduce the prediction error and recalculating the residual using the corrected result accordingly.

[0057] According to one embodiment, the residual is calculated as the difference between the predicted spectrum value output by the MLP model and the actual measured spectrum value for each pixel and wavelength range. In the second optimization step, these residuals are collected for the entire training data and statistically analyzed, and a correction matrix reflecting the distribution and correlation of the residuals is calculated. The calculated correction matrix is ​​applied by multiplying or adding to the output value of the MLP model to reduce the systematic error that the model repeatedly exhibits in a specific wavelength range or specific pixel area. By comparing the corrected prediction result with the actual measured value again to calculate a new residual and repeating this process, the prediction error is reduced stepwise until the residual converges to below the allowable error range.

[0058] According to one embodiment, the step of generating a hyperspectral image (S50) may include the step of predicting the reflectance value for each pixel of an RGB image by wavelength band and generating a three-dimensional data cube (x, y, λ) composed of position information (x, y) of each pixel and reflectance information according to multiple wavelength bands (λ) in the position information.

[0059] According to one embodiment, the hyperspectral image used in this specification is a multidimensional data cube of (x, y, λ) generated through color space normalization and wavelength-specific reflectance prediction from an RGB-based eye image acquired from a user, configured to quantitatively analyze the lipid layer, aqueous layer, mucus layer, and cornea without the need for actual expensive hyperspectral equipment. This serves as a means for extracting key diagnostic information in the low-cost, non-contact dry eye diagnosis system of the present invention.

[0060] According to one embodiment, the step of generating a hyperspectral image (S50) may include the step of acquiring a user eye image in RGB format, the step of performing color space normalization to correct color distortion of the acquired RGB image, and the step of predicting the reflectance value for each pixel of the color-corrected RGB image by wavelength band to generate a multidimensional data cube in the form of (x, y, λ).

[0061] For example, a dry eye diagnosis system (100) captures the front of the user's eye as a color image with a resolution of 1920X1080 using a 24-bit RGB camera. Since color distortion may occur depending on lighting and light intensity conditions, the system can normalize the color coordinates of each pixel using a color chart reference color space (CIE XYZ or sRGB reference). The normalized RGB values ​​are input into an artificial intelligence regression model or a pre-trained neural network to predict the reflectance spectrum for each pixel in the range of 400 to 700 nm (e.g., 10 nm intervals, a total of 31 bands). Through this process, a three-dimensional hyperspectral data cube having a structure of (x-coordinate, y-coordinate, wavelength λ) can be finally generated.

[0062] The multidimensional data cube in the present invention refers to a three-dimensional data structure that includes position information (x, y) of each pixel constituting an eye image and reflectance information according to multiple wavelength bands (λ) at the corresponding position, thereby enabling precise analysis of the optical characteristics of the eye surface structure, such as the lipid layer, aqueous layer, mucus layer, and cornea.

[0063] According to one embodiment, the multidimensional data cube may be a three-dimensional data cube composed of position information (x, y) of each pixel and reflectance information according to multiple wavelength bands (λ) in the position information.

[0064] For example, a multidimensional data cube can be configured as follows.

[0065] -Image resolution: 1920X1080 (x, y)

[0066] - Wavelength range: 400–700 nm (λ), 10 nm intervals → 31 bands

[0067] As a result, data in the form of a 3D tensor consisting of a total of 1,920 × 1,080 × 31 = approximately 60 million reflectance data points is generated, and this data can be used to analyze reflectance characteristics at specific wavelengths according to the structure of the eye.

[0068] According to one embodiment, the method may further include the steps of: generating image data representing at least one state among a lipid layer, an aqueous layer, a mucin layer, and a cornea by visualizing reflectance values ​​of one or more wavelength ranges selected from a multidimensional data cube; and estimating thickness, uniformity, moisture state, and degree of surface damage for at least one structure among the lipid layer, an aqueous layer, a mucin layer, and a cornea from the image data.

[0069] According to one embodiment, the lipid layer exhibits strong reflectance at a wavelength of 620 nm, and at 540 nm, the area with a high moisture content in the aqueous layer exhibits more absorption. Accordingly, the diagnostic system (100) can generate a 620 nm-based lipid layer color-corrected image and a 540 nm-based aqueous layer moisture state map using reflectance data at these wavelengths, and these images are each converted to RGB and visualized by the user, and the artificial intelligence model can estimate numerical data for each structure, such as lipid layer thickness (average 3.2 μm), aqueous layer uniformity index (85%), and the number of corneal surface damage areas (2 locations).

[0070] According to one embodiment, the wavelength range may be a wavelength range in which the change in reflectance according to the optical characteristics of the lipid layer, aqueous layer, mucus layer, and cornea appears above a reference value, a wavelength range selected as a feature to distinguish the layers during the learning process of an artificial intelligence model, or a wavelength range dynamically set through a correction algorithm according to the user's eye condition, lighting conditions, or shooting environment.

[0071] For example, the dry eye diagnosis system (100) can predefine wavelength ranges as follows through prior research or internal database analysis.

[0072] - Geological layer: 600~650nm

[0073] - Aqueous layer: 500~550nm

[0074] -Mucous layer: 430~470nm

[0075] In addition, the AI ​​model automatically extracts wavelengths that played an important role in distinguishing each layer during training, and when a specific user is shooting, it can correct the accuracy of reflectance prediction by dynamically adjusting wavelength selection weights in real time by considering the lighting spectrum and ambient light effects.

[0076] According to one embodiment, depending on the optical properties of the geological layer, aqueous layer, or mucilage layer, a wavelength range in which a distinct change in the reflectance or absorption rate of the layer is observed may be selected.

[0077] For example, the lipid layer exhibits a characteristic of rapidly increasing reflectance at 620 nm or higher, while the mucus layer exhibits a characteristic of strong absorption at around 420 nm, which is close to ultraviolet light. Accordingly, the diagnostic system (100) selectively utilizes a 630 nm wavelength range for lipid layer analysis and a 430 nm wavelength range for mucus layer analysis. This enables higher sensitivity and specificity than conventional single RGB value-based analysis.

[0078] According to one embodiment, the step of generating state data by ocular structure may include analyzing the reflectance by wavelength collected from the surface of the cornea and estimating the moisture state and the degree of surface damage of the cornea based on changes in reflectance, and calculating an index for predicting the tear film break-up time (TBUT) from the reflectance change pattern accumulated in chronological order.

[0079] According to one embodiment, the step of generating state data by ocular structure may include analyzing the reflectance of the meibomian gland region by wavelength and determining at least one of the sebum secretion status, structural continuity, and closure status of the meibomian gland based on changes in reflectance.

[0080] According to one embodiment, the step (S60) of diagnosing dry eye syndrome in a user may include the step of calculating a dry eye syndrome risk score through a neural network model based on the interrelationship of a plurality of state indicators derived from state data of the user's eye structure as input, and the step of determining a risk grade of the user's eye condition based on the calculated dry eye syndrome risk score.

[0081] According to one embodiment, the method may include the step of automatically generating a customized report including a description, recommended measures, and a status summary corresponding to the risk level of the user's eye condition, and the step of transmitting the customized report to at least one of the user's terminal, the medical staff terminal, and the hospital server.

[0082] According to one embodiment, the artificial intelligence model uses a multi-output regression model that takes time-series data of the user's eye structure status data as input and calculates at least one predicted value among tear film stability, interlayer interaction index, water evaporation rate, and ocular surface abnormality distribution area, wherein the multi-output regression model may be a multi-layer neural network-based model trained in a supervised learning manner using a plurality of pre-labeled eye image data and diagnostic result data corresponding to the image data as training data.

[0083] According to one embodiment, the step of acquiring an eye image of a user may include the step of sequentially acquiring a plurality of eye images before and after the user's eye blinking or over time to generate temporal image data.

[0084] According to one embodiment, the step of acquiring an image of a user's eyeball may include providing a guide interface to which preprocessing conditions are applied, including fixing the user's gaze, inducing eyelid opening and closing, and setting lighting conditions, and photographing the front of the user's eyeball using an RGB camera under predetermined distance and light intensity conditions while the user is induced to gaze at a fixation point displayed on the guide interface.

[0085] For example, the dry eye diagnosis system (100) continuously captures images of the eye for about 10 seconds at regular intervals (e.g., 1 second intervals) starting immediately after the user blinks their eye once. The images captured at this time are each assigned a time tag so that changes in the tear film over time can be observed. In particular, in the initial images, the tear film is uniformly distributed, but as time passes, changes in reflectance or changes in brightness due to water evaporation occur at specific locations. This temporal image data serves as a basis for quantitatively analyzing the tear film break-up time (TBUT).

[0086] For example, the dry eye diagnosis system (100) outputs a guide interface containing voice or text instructions to open the eyes wide along with a central gaze fixation point on the display of a smartphone or hospital equipment. While the user gazes at the point, the position and angle of the eyes are aligned according to an automatic tracking algorithm built into the device. Subsequently, the system automatically adjusts the lighting to a light intensity of 500 Lux at a distance of 30 cm and can capture the front of the eyeball in high resolution using an RGB camera while the eyelids are fully open. If the preprocessing conditions are not satisfied, a re-shooting instruction message is displayed, and data can be saved only when certain conditions are met. This process can contribute to increasing the accuracy of image analysis and reducing variability between users.

[0087] According to one embodiment, the step of generating state data for each ocular structure may include: analyzing the spatial distribution of reflectance by wavelength range of the lipid layer and calculating the thickness and uniformity of the lipid layer based on local deviations in the spatial distribution; analyzing the change in reflectance obtained in a plurality of wavelength ranges corresponding to the aqueous layer and deriving the stability and local non-uniform distribution of the aqueous layer based on the pattern of the change in reflectance; and analyzing the reflectance and absorption rate of the mucus layer by wavelength range and predicting whether there is a change in viscosity or detachment of the mucus layer.

[0088] According to one embodiment, a dry eye diagnosis system (100) can extract the reflectance of each pixel (x, y) in an image by considering that the reflectance at wavelengths of 590 nm, 610 nm, and 670 nm changes characteristically according to the interference color of the lipid layer, analyze the result using spatial filtering and local standard deviation calculation, and quantify the average thickness of the lipid layer (e.g., 60~80 nm) and the spatial variability of the thickness (uniformity index) from the result.

[0089] According to one embodiment, the dry eye diagnosis system (100) observes a decrease in reflectance or a change in refractive index in the wavelength range of 700 nm or higher depending on the moisture concentration of the aqueous layer. For example, based on a continuous reflectance curve of multiple wavelength ranges such as 740 nm and 780 nm, the slope, area, and location of extreme values ​​of the curve are analyzed, and from this, the presence of a localized dry area and the stability index of the aqueous layer (AQI: Aqueous Quality Index) can be calculated.

[0090] According to one embodiment, the dry eye diagnosis system (100) has a mucus layer that is sensitive to changes in absorption and diffusion characteristics in the 400-500 nm range, and scattering of the viscous component may be reduced when it is shed.

[0091] According to one embodiment, the dry eye diagnosis system (100) can estimate damage to the mucus layer when the reflectance around 450 nm drops sharply below a reference or when the spatial boundary is unclear. According to one embodiment, the dry eye diagnosis system (100) can calculate the mucus layer viscosity index using reflectance histogram analysis and texture-based GLCM indices for this analysis.

[0092] According to one embodiment, the step of generating state data for each ocular structure may include analyzing the reflectance by wavelength collected from the surface of the cornea and estimating the moisture state and the degree of surface damage of the cornea based on changes in reflectance, and calculating an index for predicting the tear film break-up time (TBUT) from the pattern of changes in reflectance accumulated in chronological order.

[0093] According to one embodiment, a dry eye diagnosis system (100) can detect areas of reduced wetness based on collected wavelength-specific reflectance data by utilizing the characteristic that the reflectance of the cornea changes sensitively according to moisture content in the infrared region (e.g., 850 to 1000 nm), and determine whether there is damage to the surface through a boundary detection algorithm (Canny, Sobel, etc.). According to one embodiment, the dry eye diagnosis system (100) can calculate a corneal moisture distribution map and a damage severity grade (Grade 1 to 3) from this.

[0094] According to one embodiment, the dry eye diagnosis system (100) can determine the point in time when the reflectance change pattern at a specific pixel decreases by more than a reference value based on an image sequence acquired for a certain period of time immediately after the user blinks their eyes as the point in time when TBUT occurs. According to one embodiment, the dry eye diagnosis system (100) can quantitatively extract a tear film duration indicator through the average TBUT, shortest / longest TBUT, deviation value, etc., within the front image.

[0095] According to one embodiment, a dry eye diagnosis system (100) can divide the area around the eyelid margin where the meibomian glands are located and analyze whether the reflectance of a specific wavelength range (e.g., 650-680 nm) is above or below that area. At this time, whether meibomian gland secretions are accumulated is determined by the size and continuity of the low-reflectivity area, and the dry eye diagnosis system (100) can evaluate whether there is blockage, insufficient sebum secretion, or discontinuity of the gland's continuity (distance between pixels) through this.

[0096] According to one embodiment, the step (S30) of generating state data for each ocular structure may include analyzing the reflectance of the meibomian gland region by wavelength and determining at least one of the sebum secretion state, structural continuity, and closure of the meibomian gland based on the change in reflectance.

[0097] According to one embodiment, the dry eye diagnosis system (100) can realize quantitative and highly reliable dry eye diagnosis by quantifying the thickness, concentration, stability, and degree of damage of each layer of the user's eye structure through the above analysis process and generating a set of high-resolution status indicators to be input into an artificial intelligence diagnosis model.

[0098] According to one embodiment, the step (S60) of diagnosing dry eye syndrome in a user may include the step of calculating a dry eye syndrome risk score through a neural network model based on the interrelationship of a plurality of state indicators derived from state data of the user's eye structure as input, and the step of determining a risk grade of the user's eye condition based on the calculated dry eye syndrome risk score.

[0099] According to one embodiment, a dry eye diagnosis system (100) can calculate multiple quantitative condition indicators such as lipid layer thickness (LT), uniformity (GI), aqueous layer stability (AQI), mucus layer viscosity (MVI), corneal moisture index (CWI), TBUT (Time-Based Tear Film Break-Up Time), and meibomian gland function (MGI). Since these condition indicators reflect biological responses based on different wavelength ranges, a multilayer perceptron (MLP) or transformer-based model may be used for the diagnosis of dry eye, which is a complex disease. The artificial intelligence model can calculate a risk score (DryEyeScore, 0 to 100) based on a non-linear relationship rather than a simple average by reflecting feature interaction. For example, the dry eye diagnosis system (100) can diagnose a high-risk group when the DryEyeScore is 78.

[0100] According to one embodiment, the dry eye diagnosis system (100) can classify DryEyeScore into four risk levels according to the following criteria, which are set as examples for illustrative purposes only and are not limited thereto.

[0101] 0~24: Normal

[0102] 25~49: Caution Level (Mild Risk)

[0103] 50~74: Moderate Risk

[0104] 75~100: Severe Risk

[0105] According to one embodiment, the method may include the step of automatically generating a customized report including a description, recommended measures, and a status summary corresponding to the risk level of the user's eye condition, and the step of transmitting the customized report to at least one of the user's terminal, the medical staff terminal, and the hospital server.

[0106] According to one embodiment, the report generated by the dry eye diagnosis system (100) may include analysis results such as the thickness and uniformity of the lipid layer, the stability of the aqueous layer, changes in the viscosity of the mucus layer, the moisture status and degree of damage of the cornea, and meibomian gland functionality. Along with a comprehensive diagnostic opinion on the user's eye condition, it may present recommended treatment methods or lifestyle improvement measures according to the risk level. Additionally, the report may include hyperspectral imaging-based visualization images and AI inference grounds for key indicators, so that both the patient and the medical staff can intuitively understand the diagnostic results.

[0107] According to one embodiment, the step of transmitting the generated customized report may include the step of automatically transmitting the report to at least one of a user's terminal, a medical staff terminal, or a hospital server. Here, the report may be provided in real time to a user's smartphone app or a web-based platform, and may be transmitted to medical staff in conjunction with an electronic medical record (EMR) system or a PACS (Picture Archiving and Communication System) within the hospital. Additionally, the report may be generated in a standard medical information format such as PDF or HL7 CDA to maintain interoperability between medical institutions and consistency of medical records. If necessary, the report may be automatically saved and transmitted through an encrypted secure communication channel.

[0108] According to one embodiment, the artificial intelligence model uses a multi-output regression model that takes time-series data of the user's eye structure status data as input and calculates at least one predicted value among tear film stability, interlayer interaction index, water evaporation rate, and ocular surface abnormality distribution area, wherein the multi-output regression model may be a multi-layer neural network-based model trained in a supervised learning manner using a plurality of pre-labeled eye image data and diagnostic result data corresponding to the image data as training data.

[0109] According to one embodiment, the artificial intelligence model may include a multi-output regression model that takes time-series data, in which the user's eye structure status data is sorted in chronological order, as input and calculates at least one predicted value among tear film stability, interlayer interaction index, water evaporation rate, and abnormal distribution area of ​​the eye surface.

[0110] More specifically, the above-mentioned multiple output regression model can be trained by refining indicators extracted from multiple eye hyperspectral images taken at regular time intervals after a user blinks, such as changes in the thickness of the lipid layer, changes in the reflectance of the aqueous layer, and changes in the continuity of the mucus layer, into time series data, converting this time series data into an input vector, and then utilizing a recurrent neural network structure of the LSTM (Long Short-Term Memory) family that can reflect temporal patterns.

[0111] This multi-output regression model can simultaneously calculate predicted values ​​for final tear film stability (TBUT or NIBUT, etc.), interlayer interaction indices (e.g., change in interface reflectance between the aqueous and lipid layers), water evaporation rate (based on the gradient of reflectance change per hour), and ocular surface abnormality distribution areas (e.g., spatial density of abnormally high reflectance points), based on numerical changes in the ocular structural state corresponding to each time interval.

[0112] In addition, the above artificial intelligence model can be implemented as a deep neural network (DNN) model trained using a supervised learning method, based on thousands of previously collected eye image data, with diagnostic indicators (e.g., TBUT score, meibomian gland function, etc.) labeled by ophthalmologists for each image as ground truth data.

[0113] For example, the input layer may be 4-dimensional data of (x, y, λ, t) or a vector of feature compressed data thereof, the intermediate hidden layer may include BatchNorm, Dropout, ReLU activation functions, etc., and the output layer may be structured to simultaneously calculate regression values ​​for multiple biomarkers to be predicted.

[0114] Through this structure, the artificial intelligence model of the present invention performs time-series-based multilayer analysis, thereby enabling predictions with much higher accuracy and consistency than existing single-frame-based judgment methods, and can contribute to the precise diagnosis of various types of dry eye syndrome.

[0115] FIG. 3 is a diagram illustrating a process of generating a hyperspectral image based on a user's eye image according to one embodiment and estimating the thickness by analyzing the state of the geological layer.

[0116] First, in the step of photographing the user's eyeball, the user photographs their eyeball using an RGB camera equipped on a mobile terminal or a dedicated shooting device. At this time, a pre-processing guide interface, such as a gaze fixation guide, lighting condition settings, and eyelid opening and closing timing guidance, may be provided to the user.

[0117] According to one embodiment, the result of capturing the user's eye is stored as an eye image (the second image from the top left of FIG. 3), and this image may include a color chart or a color reference target. The captured image is used as reference data for color distortion correction and can be used in a subsequent color normalization process.

[0118] Subsequently, in the color correction process, color space correction is performed based on the color chart included in the original eye image in RGB format, thereby minimizing color distortion caused by the shooting environment or lighting conditions. As a result, a color-corrected eye image (upper right image in Fig. 3) can be generated.

[0119] According to one embodiment, a color-corrected eye image is used as an input to predict the reflectance value for each wavelength band for each pixel, and in the hyperspectral image generation step, the reflectance value for each specific wavelength band (λ) for each pixel location (x, y) is predicted in a multidimensional space to generate a multidimensional data cube (hyperspectral data cube) having a structure of (x, y, λ). This cube is illustrated in the center of FIG. 3.

[0120] According to one embodiment, the generated multidimensional data cube is utilized to visualize the spatial distribution of reflectance changes for a specific wavelength range and serves as a basis for extracting reflectance characteristics of ocular constituent layers such as the lipid layer, aqueous layer, mucus layer, and cornea.

[0121] Among these, in an embodiment for geological layer analysis, a wavelength range in which the optical response of the geological layer is distinct is selected from a multidimensional data cube, and reflectance information corresponding to this wavelength range is spatially visualized to generate a color-corrected geological layer image (bottom right image of Fig. 3). The image visually highlights the location and shape of specific structures and serves as the basis for analysis.

[0122] According to one embodiment, a dry eye diagnosis system (100) can estimate local thickness changes, uniformity, and surface waveforms of a lipid layer by analyzing the reflectance distribution at the pixel level included in the lipid layer image. In this process, calculations based on the reflectance gradient or reflectance amplitude difference at each pixel are performed, and as a result, a 3D thickness estimation result of the lipid layer (bottom-left 3D graph of FIG. 3) is generated.

[0123] In this way, the dry eye diagnosis system (100) according to the present invention performs color correction and hyperspectral analysis based on a single RGB image captured from a user, and can quantitatively diagnose the condition of the lipid layer, which is one of the constituent layers of the eye.

[0124] FIG. 4 shows an MLP model structure diagram for generating hyperspectral images based on RGB-spectral conversion according to one embodiment.

[0125] According to one embodiment, the model approximates the complex nonlinear mapping between RGB inputs and wavelength-specific spectra through the structure shown in FIG. 4, and enables high-quality hyperspectral image restoration by reducing prediction errors through cross-validation and first / second-order optimization.

[0126] According to one embodiment, the MLP model may be composed of an input unit, a hidden unit, and an output unit. The input unit provides pixel-by-pixel quantified color space data (R, G, B) extracted from an RGB image as input.

[0127] According to one embodiment, input data sequentially passes through dense hidden layers, and in each layer, non-linear characteristics are reflected by passing through a ReLU activation function (ReLuLayer), and learning stability and convergence speed are increased through Batch Normalization in each layer. According to one embodiment, the hidden layers are composed of four dense layers including, for example, 10, 20, 30, and 40 nodes, thereby extracting progressively higher-dimensional characteristics and learning the relationships necessary for spectrum prediction.

[0128] According to one embodiment, the output of the hidden layer passes through a Fully Connected Layer (n=32), performs a non-linear transformation in a Sigmoid Layer, and finally outputs continuous spectrum values ​​for each wavelength band in a Regression Layer. The output unit is composed of the predicted reflectances (SPECTRUM) for multiple wavelength bands and can provide a result of converting RGB inputs into hyperspectral information.

[0129] FIG. 5 is a diagram showing the process of separating the RGB channels of an eye image and extracting the reflectance by wavelength according to one embodiment.

[0130] According to one embodiment, a dry eye diagnosis system (100) can generate a spectrum signature by separating each color channel (R, G, B) based on a user's eye image, and then predicting the wavelength-specific reflectance curve of the corresponding location from the separated channels.

[0131] According to one embodiment, the image on the left is a full image of a user's eyeball captured by an RGB camera, and the three images in the middle represent the separation of the R (Red), G (Green), and B (Blue) channels from the original image, respectively, illustrating the process of extracting color components to analyze reflection characteristics at the pixel level. The graph on the right is a spectral signature representing the predicted wavelength-dependent reflectance at a pixel of a specific location, where the horizontal axis represents wavelength (450–900 nm) and the vertical axis represents reflectance.

[0132] These spectral signatures can be used to distinguish the optical properties of each ocular component layer (e.g., lipid layer, aqueous layer, etc.) and to extract biomarkers related to dry eye disease.

[0133] According to one embodiment, a dry eye diagnosis system (100) can predict the thickness of the lipid layer on the surface of the eye for three different patients (Patient I to III) and visualize it in three-dimensional space.

[0134] According to one embodiment, (a) to (c) are eye images of patients I, II, and III, respectively, with a region of interest (X, Y) where hyperspectral-based analysis is performed. Here, (d) and (f) are graphs visualizing the lipid layer thickness predicted in the region of interest of (a) and (c) in 3D coordinates (X, Y, Thickness), where each axis represents pixel coordinates (X, Y) and the predicted lipid layer thickness (nm).

[0135] According to one embodiment, it can be seen through (d) that Patient I exhibits an overall uniform and thin lipid layer distribution, through (e) that Patient II exhibits localized thick areas and an overall uneven distribution, and through (f) that Patient III exhibits an overall thick and uneven lipid layer distribution with some abnormally protruding areas. This visualization can be usefully utilized to quantitatively compare the lipid layer status of patients with dry eye syndrome and to visually assess the degree of the lesion.

[0136] Figure 7 shows an example of state analysis of the layered structure of the tear film based on ocular imaging according to one embodiment.

[0137] According to one embodiment, a dry eye diagnosis system (100) can perform major diagnoses related to dry eye by utilizing the three-layer structure of the tear film. For example, thickness and uniformity can be analyzed based on hyperspectral by utilizing the reflective characteristics of the surface where the lipid layer is located in an actual eye image. In addition, the stability and non-uniformity of moisture distribution in the area where the aqueous layer exists can be analyzed through the pattern of reflectance change by wavelength. Furthermore, the dry eye diagnosis system (100) can utilize the continuity, area, and occlusion of the structure by analyzing the sebum secretion function of the meibomian glands located beneath the mucus layer as a target for analysis, and utilize this for the diagnosis of dry eye.

[0138] FIG. 8 is a diagram showing the step-by-step visualization of the results of color correction of an eye image captured by a user terminal according to one embodiment and analysis of the lipid layer thickness measurement results.

[0139] According to one embodiment, (a) of FIG. 8 represents the color patch input of the original eye image according to the CIE RGB color space, and (b) is the result after a color correction algorithm is applied to the input image of (a), which is an image corrected to be closer to the original reference color. (c) is a graph visualizing the correction accuracy for each RGB channel before and after color correction, where an ideal correction result indicates accurate correction the closer it is to the diagonal line. (d) shows a comparison of Root Mean Square Error (RMSE) for various smartphone models with and without color correction. It shows that the error rate is significantly lower when color correction is applied compared to when it is not applied.

[0140] In addition, (e), (f), and (g) are the results of spectral analysis of lipid layer thickness measured on different smartphone devices, respectively, visualizing the difference in precision before and after correction. (e) and (f) show a relatively non-uniform thickness distribution and contain noise, whereas (g) is the result after color correction and algorithm application, revealing a flat and clear lipid layer thickness pattern.

[0141] According to one embodiment, (h) is a graph that numerically summarizes the analysis results of (e) to (g), and shows that the corrected state has lower error and higher reproducibility than the uncorrected state by comparing the variance values ​​of the lipid layer thickness measured for various smartphone models (A51, S21, iPhone 8, 11, 12).

[0142] FIG. 9 is a diagram illustrating the criteria for extracting valid image data according to the user's eye opening / closing state according to one embodiment.

[0143] According to one embodiment, the vertical axis (B) represents an eyelid brightness value, and B opened represents the brightness value when the eyelids are fully open, and B closedrepresents the brightness value when the eyelid is completely closed. Threshold is a brightness baseline that determines the eyelid state by dividing the entire range into upper and lower 1 / 3 and 2 / 3. According to one embodiment, the horizontal axis (Time) represents the flow of time, and it can be seen that a change in state from Opened → Closed → Opened occurs depending on the movement of the eyelid.

[0144] According to one embodiment, the dry eye diagnosis system (100) determines ROI data only in the section where the eye is fully open, and processes the transition section where the eye is closed and the completely closed state as discarded sections, thereby preventing image distortion due to eye blinking and improving diagnostic accuracy.

[0145] Referring to the preceding Fig. 6, the results of measuring the lipid layer thickness of the eyelid region for each subject according to one embodiment can be seen.

[0146] According to one embodiment, an eyelid image captured for each subject (Patient I, II, III), a segmented region of interest (ROI), and a graph showing the distribution and thickness of the lipid layer calculated therefrom are sequentially displayed, wherein the first row is the actual captured eyelid image. The second row is the segmented ROI (Region of Interest) in the image, showing the area to which lipid layer analysis was applied. The third row is a planar distribution map visualizing the relative lipid layer thickness at each location through color mapping of the segmented ROI. The fourth row is a 3D topographic map generated based on the same data, allowing for a more intuitive verification of the quantitative difference in lipid layer thickness.

[0147] Through this analysis, differences in the thickness distribution and morphology of lipid layers among subjects can be compared, and it can be utilized for the diagnosis of dry eye disease or the evaluation of meibomian gland dysfunction.

[0148] Figure 9 is a graph showing the results of comparing the lipid layer thickness by subject group according to one embodiment.

[0149] According to one embodiment, the dry eye diagnosis system (100) is a scatter plot graph comparing the lipid layer thickness (nm) of three groups of subjects, wherein the X-axis represents the number of each subject or measurement repetition within each subject group, and the Y-axis represents the lipid layer thickness in nanometers (nm).

[0150] As can be seen in the graph, most patients with hyposecretory MGD have a lipid layer thickness of 30–50 nm or less, patients with hypersecretory MGD show a distribution of 50–100 nm, and normal control groups tend to maintain a thick and consistent lipid layer of 100 nm or more.

[0151] This diagram visually shows that the lipid layer thickness is clearly distinguished according to the MGD type, which is one of the main causes of dry eye disease. Therefore, the analysis performed by the dry eye disease diagnostic system (100) can be significantly utilized for diagnosing the type of dry eye disease and determining the suitability for treatment.

[0152] FIG. 10 is a diagram showing the configuration of a dry eye diagnosis system using an RGB-spectrum conversion-based hyperspectral image according to one embodiment.

[0153] According to one embodiment, a dry eye diagnosis device using an RGB-spectrum conversion-based hyperspectrum image may be provided, comprising: an image acquisition unit (110) that quantifies pixel values ​​of an RGB image of a user's eye and converts them into standardized color space data; an MLP optimization unit (120) that inputs color space data into a Multi-Layer Perceptron (MLP) model to cross-validate the spectral values ​​of predicted RGB pixels to evaluate the accuracy of the MLP model, performs first-order optimization to improve the accuracy of the MLP model, and performs second-order optimization by applying statistical techniques to correct additional errors in the MLP model; a hyperspectrum conversion unit (130) that generates a hyperspectrum image by predicting wavelength-specific reflectance data of each pixel from the RGB image using the MLP model; and a diagnosis processing unit (150) that diagnoses the user's dry eye based on values ​​calculated from the wavelength-specific reflectance data of the hyperspectrum image to determine the user's eye structure status indicator.

[0154] According to one embodiment, the dry eye diagnosis system (100) may further include a communication unit, a memory, a processor, and an input / output device.

[0155] According to one embodiment, the communication unit is configured to perform data transmission and reception between the dry eye diagnosis system (100) and an external device, and may include, for example, Bluetooth, Wi-Fi, 5G communication modules, and through this, can be linked with a user terminal, a medical institution server, a cloud platform, etc.

[0156] According to one embodiment, the memory is configured to store image data, spectrum data, status information, and diagnostic results, and may include, for example, flash memory, SSD (Solid State Drive), DRAM, etc., and is utilized to temporarily or permanently store diagnostic algorithms or user information.

[0157] According to one embodiment, the processor is a central computing unit that performs control and computation of the entire dry eye diagnosis system (100), performs various computations including image processing, spectrum conversion, and execution of a diagnostic algorithm, and may include computational resources such as an ARM-based chipset, a GPU, and an NPU (Neural Processing Unit).

[0158] According to one embodiment, the MLP optimization unit (120) may evaluate the accuracy of the MLP model by inputting color space data into the MLP (Multi-Layer Perceptron) model and cross-validating the spectral values ​​of the predicted RGB pixels, perform first-order optimization to improve the accuracy of the MLP model, and perform second-order optimization by applying statistical techniques to correct additional errors in the MLP model.

[0159] According to one embodiment, the MLP optimization unit (120) can perform the function of correcting lighting conditions and camera response characteristics during shooting, converting pixel-unit RGB data into a standardized color space (such as CIE XYZ), and providing it in a format suitable for input to the MLP model. In addition, the prediction accuracy of the MLP model can be evaluated by separating some data from the training dataset for verification and comparing the spectrum values ​​predicted by the model with the actual measured spectrum values ​​using a leave-one-out method.

[0160] According to one embodiment, the MLP optimization unit (120) can perform a learning process of iteratively adjusting the weights and bias parameters of the MLP model to minimize the loss function defined by the Root Mean Square Error (RMSE) and RGB expanded bias calculated from the cross-validation result, and can perform iterative statistical optimization by calculating the residual between the predicted spectrum value output from the MLP model and the actual measurement value, calculating a correction matrix by applying the Moore-Penrose pseudo-inverse matrix and backslash (\) operation, applying this to the model output, and then recalculating the residual to gradually reduce the prediction error. Furthermore, the final performance of the optimized MLP model can be quantitatively evaluated using indicators such as RMSE, PSNR (Peak Signal-to-Noise Ratio), and SSIM (Structural Similarity Index) to verify the model's generalization ability and spectrum reconstruction quality.

[0161] According to one embodiment, the input / output device is configured to provide an interface with a user or to connect with an external device, and may include, for example, a touch display, a speaker, a button, a USB port, an HDMI terminal, etc. This configuration is used to receive a diagnostic request or to provide results to the user visually or audibly.

[0162] According to one embodiment, when a user's command input or a diagnostic request from an external terminal is received through a communication unit, the processor recognizes the request and initializes the diagnostic process. Subsequently, the image acquisition unit (110) photographs the eyeball of the target user to acquire a raw image, and the acquired image is converted into a multi-channel spectral image by a hyperspectral conversion unit (130).

[0163] According to one embodiment, the converted spectrum data is preprocessed by a state data generation unit (140) and state information such as the lipid layer thickness, reflectance, and spectral response of the tear film is calculated. The state data thus generated is transmitted to a diagnostic processing unit (150), and the processor performs analysis using a built-in diagnostic algorithm and then generates a final diagnostic result.

[0164] According to one embodiment, the generated diagnostic result is stored in memory and can be transmitted to an external user terminal, a cloud server, or a hospital electronic medical record (EMR) system through a communication unit. Additionally, the diagnostic result can be displayed on a screen or provided to the user as voice guidance through an input / output device.

[0165] According to one embodiment, the dry eye diagnosis system (100) can be linked with various external devices. For example, the diagnosis results can be stored in the EMR in real time through linkage with a hospital server, and the diagnosis results can be provided directly to the user by connecting with a user's mobile application. In addition, the system can be linked with a cloud server to perform distributed processing of the diagnosis algorithm or utilized for large-scale data-based learning. If necessary, it is also possible to print a diagnosis report through linkage with a printer using USB or LAN.

[0166] According to one embodiment, a computer program stored on a computer-readable recording medium can be provided for executing a dry eye disease diagnosis method using RGB-spectrum conversion-based hyperspectral images on a computer.

[0167] In addition, according to one embodiment, the method according to the various embodiments disclosed herein may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or distributed online (e.g., download or upload) through an application store (e.g., Play Store™) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created in a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0168] As described above, the present invention has been explained with reference to the embodiments illustrated in the drawings, but this is merely illustrative, and it should be understood that various modifications and equivalent alternative embodiments are possible based on the ordinary knowledge of the art to which the art belongs. Accordingly, the true technical scope of protection of the present invention is defined by the claims described below and should be determined based on the specific details of the invention described above. Explanation of the symbols

[0169] 100: Dry Eye Syndrome Diagnostic System 110: Image acquisition unit 120: MLP Optimizer 130: Hyperspectral converter 140: State data generation section 150: Diagnostic processing unit

Claims

Claim 1 A method for diagnosing dry eye syndrome using RGB-spectrum conversion-based hyperspectrum images, performed by a dry eye syndrome diagnosis device using RGB-spectrum conversion-based hyperspectrum images, comprising: a step of quantifying pixel values ​​of an RGB image of a user's eye and converting them into standardized color space data; a step of inputting the color space data into a Multi-Layer Perceptron (MLP) model and cross-validating the spectral values ​​of the predicted RGB pixels to evaluate the accuracy of the MLP model; a step of performing first-order optimization to improve the accuracy of the MLP model; a step of applying statistical techniques to perform second-order optimization to correct additional errors in the MLP model; and a step of generating a hyperspectrum image by predicting wavelength-specific reflectance data of each pixel from the RGB image using the MLP model. and a step of diagnosing dry eye syndrome of the user based on values ​​calculated from reflection data by wavelength band of the hyperspectral image for the user's eye structure; wherein the step of converting to standardized color space data includes: a step of correcting color deviation between devices by correcting lighting conditions and camera sensitivity characteristics at the time of shooting for each pixel value constituting the RGB image; a step of converting the corrected pixel value to the CIE XYZ color space; and a step of normalizing the converted CIE XYZ value to convert it into standardized color space data suitable for input to the MLP model; and the step of evaluating the accuracy of the MLP model includes: a step of training the MLP model using the standardized color space data; a step of predicting spectrum values ​​by wavelength band of the RGB pixels using the trained MLP model, and performing cross-validation (Leave-one-out) by comparing the predicted spectrum values ​​with actual measured spectrum data.A method for diagnosing dry eye syndrome using RGB-spectrum-transformed hyperspectral images, comprising: a step of evaluating the prediction accuracy of the MLP model by calculating RMSE (Root Mean Square Error) and RGB expanded bias indices from the cross-validation results; wherein the step of performing first-order optimization comprises a learning process of iteratively adjusting the weights and bias parameters of the MLP model to minimize a loss function defined by the RMSE and RGB expanded bias calculated in the step of evaluating the accuracy of the MLP model; and the step of performing second-order optimization comprises: a step of calculating a residual by comparing the difference between the predicted spectrum value output from the MLP model and the actual measured spectrum value; a step of calculating a correction matrix based on the residual; and a step of repeatedly applying the calculated correction matrix to the prediction result of the MLP model to reduce the prediction error and recalculating the residual using the corrected result accordingly. Claim 2 delete Claim 3 delete Claim 4 delete Claim 5 delete Claim 6 A method for diagnosing dry eye syndrome using an RGB-spectrum conversion-based hyperspectrum image, wherein, in claim 1, the step of generating the hyperspectrum image comprises: predicting the reflectance value for each pixel of the RGB image by wavelength band and generating a three-dimensional data cube (x, y, λ) composed of position information (x, y) of each pixel and reflectance information according to multiple wavelength bands (λ) at the position information. Claim 7 A method for diagnosing dry eye disease using RGB-spectrum conversion-based hyperspectral imaging, characterized in that, in claim 6, the method further comprises the steps of: generating image data representing at least one state among a lipid layer, an aqueous layer, a mucin layer, and a cornea by visualizing the reflectance values ​​of one or more wavelength ranges selected from the 3D data cube; and estimating the thickness, uniformity, moisture state, and degree of surface damage for at least one structure among the lipid layer, the aqueous layer, the mucin layer, and the cornea from the image data. Claim 8 A method for diagnosing dry eye disease using RGB-spectrum conversion-based hyperspectral images, wherein, in claim 7, the wavelength range is a wavelength range in which the change in reflectance according to the optical properties of the lipid layer, the aqueous layer, the mucus layer, and the cornea appears above a reference value, or a wavelength range selected as a feature to distinguish the layers during the learning process of the MLP model, or a wavelength range dynamically set through a correction algorithm according to the user's eye condition, lighting conditions, or shooting environment. Claim 9 A method for diagnosing dry eye disease using RGB-spectrum conversion-based hyperspectral imaging, wherein, in claim 1, the step of generating state data for each ocular structure comprises: a step of analyzing the spatial distribution of reflectance by wavelength band of a lipid layer and calculating the thickness and uniformity of the lipid layer based on local deviations of the spatial distribution; a step of analyzing the change in reflectance obtained in a plurality of wavelength bands corresponding to an aqueous layer and deriving the stability and local non-uniform distribution of the aqueous layer based on the pattern of the change in reflectance; and a step of analyzing the reflectance and absorption rate of a mucus layer by wavelength band and predicting whether there is a change in viscosity or shedding of the mucus layer. Claim 10 A method for diagnosing dry eye disease using RGB-spectrum conversion-based hyperspectral imaging, wherein, in claim 1, the step of generating state data for each ocular structure comprises: a step of analyzing the reflectance by wavelength collected from the surface of the cornea and estimating the moisture state and the degree of surface damage of the cornea based on the change in the reflectance; and a step of calculating an index for predicting the tear break-up time (TBUT) from the reflectance change pattern accumulated in chronological order. Claim 11 A method for diagnosing dry eye disease using RGB-spectrum conversion-based hyperspectral imaging, characterized in that, in claim 1, the step of generating state data for each ocular structure comprises: analyzing the reflectance of the meibomian gland region by wavelength band and determining at least one of the sebum secretion state, structural continuity, and closure of the meibomian gland based on the change in the reflectance. Claim 12 A method for diagnosing dry eye syndrome using RGB-spectrum conversion-based hyperspectral imaging, wherein, in claim 1, the step of diagnosing dry eye syndrome of the user comprises: a step of calculating a dry eye syndrome risk score through a neural network model based on the interrelationship of a plurality of state indicators derived from state data of the user's eye structure as input; and a step of determining a risk grade of the user's eye condition based on the calculated dry eye syndrome risk score. Claim 13 A method for diagnosing dry eye disease using RGB-spectrum conversion-based hyperspectral imaging, characterized in that, in claim 12, the method comprises: a step of automatically generating a customized report including an explanation, recommended measures, and a status summary corresponding to the risk grade of the user's eye condition; and a step of transmitting the customized report to at least one of the user's terminal, a medical staff terminal, and a hospital server. Claim 14 A method for diagnosing dry eye disease using RGB-spectrum-transformed hyperspectral images, wherein, in claim 1, the MLP model uses a multiple output regression model that takes time-series data of the user's eye structure-specific state data as input and calculates at least one predicted value among tear film stability, interlayer interaction index, water evaporation rate, and ocular surface abnormality distribution area, wherein the multiple output regression model is a multilayer neural network-based model trained in a supervised learning manner using a plurality of pre-labeled eye image data and diagnostic result data corresponding to the image data as training data. Claim 15 A method for diagnosing dry eye syndrome using RGB-spectrum conversion-based hyperspectral images, characterized in that, in claim 1, the step of acquiring an eye image of the user comprises the step of sequentially acquiring a plurality of eye images before and after the user’s blinking or over time to generate temporal image data. Claim 16 A method for diagnosing dry eye syndrome using an RGB-spectrum-transformed hyperspectral image, wherein, in claim 1, the step of acquiring an image of the user’s eye comprises: a step of providing a guide interface to which preprocessing conditions are applied, including fixation of the user’s gaze, induction of eyelid opening and closing, and setting of lighting conditions; and a step of photographing the front of the user’s eye using an RGB camera under predetermined distance and light intensity conditions while the user is induced to gaze at a fixation point displayed on the guide interface. Claim 17 A computer program stored on a computer-readable recording medium to execute the method according to paragraph 1 on a computer. Claim 18 delete

Images