Method for detecting ocular hyperemia and device therefor

Abstract

The present disclosure relates to a technique for detecting whether an eye is congested, and provides an eye congestion detection device and method. The device includes an image acquisition unit that acquires image information, a feature point acquisition unit that acquires eye region feature points extracted from an eye region image of the image information, an eye detail region extraction unit that extracts an eye detail region image by segmenting the eye detail region image using the eye region feature points, a conjunctival region image generation unit that generates a conjunctival region image using the eye detail region image and an iris region detected from the eye region image or the eye detail region image, and a determination unit that inputs the conjunctival region image to a congestion determination algorithm learned in advance to derive a classification result value regarding whether the eye is congested.

Description

Technical Field

[0001] This disclosure relates to a technique for detecting whether the eye is congested. Background Technology

[0002] Bloodshot eyes are a symptom of many diseases, and can also occur in normal circumstances due to various reasons. While the most accurate way to determine if there is bloodshot eyes is to visit an ophthalmologist with medical equipment, this requires a hospital visit, which incurs time and financial costs.

[0003] To prevent such inconvenience, existing technology provides a similar technique that uses a special camera developed specifically for medical purposes to perform imaging, thereby determining whether there is bleeding in a specific area of ​​the eye, and subsequently identifying conditions such as diabetic macular edema. However, even in this case, a special camera developed specifically for medical use is required, which limits its cost and the widespread availability of such equipment.

[0004] When using a regular camera to determine if there is bloodshot eyes, the determination is made probabilistically by calculating the probability values ​​of pixels in the image. In particular, using pixel-based probability values ​​reduces the accuracy of eye region detection, and there is a problem that noise from areas other than the eye can significantly affect the judgment results.

[0005] To address these issues, a technology is designed to provide a way to easily and accurately determine whether the eyes are congested using an RGB camera. Summary of the Invention

[0006] (a) Technical problems to be solved This disclosure aims to provide a technique for detecting whether the eye is congested.

[0007] (II) Technical Solution In one aspect, this embodiment provides an eye congestion detection device, comprising: an image acquisition unit for acquiring image information; a feature point acquisition unit for acquiring eye region feature points of an eye region image extracted from the image information; an eye detail region extraction unit for segmenting and extracting an eye detail region image using the eye region feature points; a conjunctival region image generation unit for detecting the iris region from the eye region image or the eye detail region image, and generating a conjunctival region image using the eye detail region image and the iris region; and a judgment unit for inputting the conjunctival region image into a pre-learned congestion judgment algorithm and deriving a classification result value regarding whether congestion exists.

[0008] In another aspect, this embodiment provides a method for detecting eye congestion, used to detect whether the eye is congested, including: an image acquisition step, acquiring image information; a feature point acquisition step, acquiring eye region feature points of the eye region image extracted from the image information; an eye detail region extraction step, segmenting and extracting the eye detail region image using the eye region feature points; a conjunctival region image generation step, detecting the iris region from the eye region image or the eye detail region image, and generating a conjunctival region image using the eye detail region image and the iris region; and a judgment step, inputting the conjunctival region image into a pre-learned congestion judgment algorithm, and deriving a classification result value regarding whether there is congestion.

[0009] (III) Beneficial Effects According to this disclosure, a technique for detecting whether the eye is congested can be provided. Attached Figure Description

[0010] Figure 1 This is a diagram illustrating the configuration of an eye redness detection device according to one embodiment.

[0011] Figure 2 This is a diagram illustrating the operation of processing image information according to one embodiment.

[0012] Figure 3 This is a diagram illustrating the operation of segmenting an image of a region of eye detail according to one embodiment.

[0013] Figure 4 This is a diagram illustrating the operation of detecting the iris region according to one embodiment.

[0014] Figure 5 This is a diagram illustrating the operation of detecting the iris region according to another embodiment.

[0015] Figure 6 This is a diagram illustrating the operation of generating a conjunctival region image according to one embodiment.

[0016] Figure 7 This is a diagram illustrating a dataset used to illustrate a congestion determination algorithm according to one embodiment.

[0017] Figure 8 This is a flowchart illustrating a method for detecting ocular congestion according to one embodiment. Detailed Implementation

[0018] Hereinafter, some embodiments of the present disclosure will be described in detail with reference to the exemplary accompanying drawings. When affixing reference numerals to components in the various drawings, the same components will be represented by the same symbol whenever possible, even if they are labeled in different drawings. Furthermore, when describing these embodiments, detailed descriptions of related well-known structures or functions may be omitted if it is determined that such detailed descriptions would obscure the essence of the technical concept. In this specification, the use of terms such as "comprising," "having," and "consisting of," unless "only" is used, may include additional parts. When components are described in a singular, a plural form may be included unless specifically stated otherwise.

[0019] Furthermore, in describing the components of this disclosure, terms such as first, second, A, B, (a), and (b) may be used. These terms are merely used to distinguish the component from other components and do not limit the nature, order, sequence, or number of the components.

[0020] In descriptions of the positional relationships of components, when two or more components are described as "connected," "combined," or "accessed," it should be understood that the two or more components can be directly "connected," "combined," or "accessed," but it can also be that two or more components further "intervene" with other components to "connect," "combine," or "access." Here, other components can also be included among one or more of the two or more components that are mutually "connected," "combined," or "accessed."

[0021] In descriptions of the temporal flow of components, operating methods, or manufacturing methods, for example, if phrases such as "after," "following," "after," or "before" are used to describe the temporal or flow sequence, discontinuous situations may also be included unless "immediately following" or "directly" is used.

[0022] On the other hand, when referring to a numerical value or its corresponding information (e.g., grade) for a component, even without further explicit documentation, the numerical value or its corresponding information can be interpreted as including a range of errors caused by various factors (e.g., process factors, internal or external shocks, noise, etc.).

[0023] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings.

[0024] Bloodshot eyes can be a factor in identifying a variety of health problems. While there are many medical causes of bloodshot eyes, it can also lead to dangerous situations due to health abnormalities and decreased eye function.

[0025] In particular, industrial sites present various environmental hazards such as dust and sand, increasing the likelihood of workers experiencing eye congestion. Furthermore, areas prone to eye congestion, such as those involving working at heights or in hazardous areas, are also widely affected.

[0026] Therefore, it is necessary to perform a simple eye check on industrial workers before they begin work to ensure their eyes are clear and free of blood. This helps to provide a safe working environment and prevent accidents.

[0027] However, under current circumstances, a medical camera can be used to directly photograph the cornea, or images taken with a regular camera can be used to determine whether there is congestion based on pixel probability values.

[0028] These existing technologies present challenges in terms of ease of installation and operation in industrial settings due to the need for expensive equipment or the equipment's sensitivity to external environments. Furthermore, when using pixel color or probability values ​​in an image to determine congestion, the limitations of ordinary cameras make it difficult to accurately extract only the corneal portion for assessment, leading to reduced reliability.

[0029] Therefore, this disclosure aims to provide a technology that can quickly and accurately determine whether the eyes are congested using images easily captured by RGB cameras such as mobile phone cameras in industrial settings.

[0030] For example, this embodiment provides a technology where a user can take a picture using a mobile phone camera or a regular camera installed in an industrial setting. The eye redness detection device then quickly extracts only the corneal region image from this image information and uses a preset artificial intelligence algorithm to determine a classification value regarding whether redness has occurred. This provides highly usable and accurate judgment results in various industrial settings.

[0031] The apparatus and method according to this disclosure will now be described in detail with reference to the accompanying drawings. The term "eye" can refer to the eye itself, and "iris" can refer to the portion including the pupil. Furthermore, "feature point" can refer to a landmark, and the algorithms described below are illustrative examples; various algorithms performing the same purpose and function can be applied to this disclosure. In addition, programs capable of performing the functions of this embodiment are also included in this disclosure, and it should be interpreted that the recording medium containing the program is also included in this disclosure.

[0032] Figure 1 This is a diagram illustrating the configuration of an eye redness detection device according to one embodiment.

[0033] Reference Figure 1The eye congestion detection device 100 may include: an image acquisition unit 110 for acquiring image information; a feature point acquisition unit 120 for acquiring eye region feature points of an eye region image extracted from the image information; an eye detail region extraction unit 130 for segmenting and extracting an eye detail region image using the eye region feature points; a conjunctival region image generation unit 140 for detecting the iris region from the eye region image or the eye detail region image, and generating a conjunctival region image using the eye detail region image and the iris region; and a judgment unit 150 for inputting the conjunctival region image into a pre-learned congestion judgment algorithm and deriving a classification result value for whether there is congestion.

[0034] For example, the image acquisition unit 110 can receive captured image information from a sensor such as a camera. Here, "camera" refers to a common camera such as an RGB camera as described above. Alternatively, the image acquisition unit 110 can also acquire image information by capturing frames in a video.

[0035] The feature point acquisition unit 120 can acquire an eye region image, including the eyes, from the acquired image information. For example, the eye region image can be configured to include eye region feature points, excluding other region feature points contained in the facial image information. As an example, the image information may include background and facial image information, as well as other body images such as the neck. After acquiring the image information, the feature point acquisition unit 120 can use an artificial intelligence algorithm to extract facial image information from the image information. For this purpose, a marker extractor can be used to extract facial image information from the image information. The marker extractor performs the function of extracting feature points related to the facial contours of a person from the image; the markers will be referred to as feature points below.

[0036] After extracting facial image information, facial image information including the eye area can be identified from the facial image information. As described above, the facial image information can be set to include only feature points assigned to the eye region. Therefore, the feature point acquisition unit 120 can acquire eye region feature points from the image information.

[0037] The eye detail region extraction unit 130 can extract eye detail region images using eye region feature points and a preset interpolation algorithm. The eye region image includes not only feature points of the eye region but also images of the periocular skin and eyebrows. Therefore, to accurately determine whether there is congestion, the eye detail region extraction unit 130 needs to extract only the eye detail region image from the eye region image.

[0038] Therefore, the eye detail region extraction unit 130 can utilize eye region feature points. The eye detail region extraction unit 130 can use the acquired eye region feature points and image processing algorithms to determine the outer contour line of the eye detail region. After determining the outer contour line, the eye detail region extraction unit 130 can extract only the portion within the outer contour line as the eye detail region image.

[0039] As an example, the preset interpolation algorithm can be set to predict the outer contour coordinates of the eye detail region based on the two-dimensional coordinates of the feature points of the eye region, thereby extracting the eye detail region image. For example, after confirming the two-dimensional coordinates of N feature points of the eye region, the interpolation algorithm can use these two-dimensional coordinates to predict the coordinates between the feature points of the eye region. For example, the interpolation algorithm can use various algorithms such as forward / backward interpolation, linear interpolation, bilinear interpolation, and polynomial interpolation. Thus, the eye detail region extraction unit 130 can obtain the outer contour coordinates connecting the feature points of the eye region and extract the outer contour line of the eye detail region accordingly.

[0040] The eye detail region extraction unit 130 can extract the image based on the outer contour line after extraction, thereby generating an image only for the eye detail region within the outer contour line.

[0041] The conjunctival region image generation unit 140 can generate a conjunctival region image by separating only the conjunctival region, which serves as a reference for whether or not there is congestion. For this purpose, the conjunctival region image generation unit 140 needs to detect the iris region from the eye detail region.

[0042] As an example, the conjunctival region image generation unit 140 can calculate the sum of gray level values ​​of pixels on a circle with arbitrary center coordinates and radius values ​​on an eye region image or an eye detail region image, and set the circle with the largest change in the sum of gray level values ​​as the radius value changes as the boundary of the iris region. For example, the conjunctival region image generation unit 140 can set any coordinate within the eye region image or the eye detail region image as the center coordinate, and also set the radius value as an arbitrary value, and then calculate the sum of gray level values ​​of pixels on a circle with each arbitrary center coordinate and arbitrary radius value.

[0043] The conjunctival region image generation unit 140 can continuously calculate the sum of grayscale values ​​while changing the center coordinates and radius values. The conjunctival region image generation unit 140 can set the circumference of the circle where the calculated sum of grayscale values ​​changes the most as the boundary of the iris region, and save the center coordinates and radius values ​​at this time. For example, during the generation of an eye region image or an eye detail region image, the conjunctival region image generation unit 140 can determine the center coordinates using an experimental range for the eye center coordinates. That is, in the eye region image or eye detail region image, the center point of the iris (i.e., the center point of the eye) is formed in the central part of the image. Therefore, for fast processing, the conjunctival region image generation unit 140 preferentially sets the center coordinates arbitrarily within a preset coordinate range. Similarly, the radius value can also be set to a range of arbitrary values ​​so that selection is preferentially made within a predetermined range based on image scale, etc. Through this operation, the conjunctival region image generation unit 140 can detect the iris region more quickly.

[0044] As another example, the conjunctival region image generation unit 140 can use a circular detection algorithm to determine a predetermined number of candidate iris regions, and use a DeepEye model to determine the pupil center point. It then filters the candidate iris regions to identify the iris region corresponding to the pupil center point and determines it as the final iris region. Therefore, even when multiple candidate iris regions, such as eyelashes, are generated by the circular detection algorithm, a more accurate iris region can be detected. The circular detection algorithm can use the aforementioned algorithm that detects iris regions by changing pixel variations while altering the center coordinates and radius values.

[0045] As another example, the conjunctival region image generation unit 140 can apply a convex hull algorithm to an eye region image or an eye detail region image to detect the iris region. For example, the conjunctival region image generation unit 140 can use an eye detail region image and employ a convex hull algorithm that connects the outermost parts of multiple data values ​​to form a polygon to detect the iris region. Since the shape of the iris may not be circular due to features such as the eyelid, the iris region detection using the center coordinates and radius values ​​described above may produce errors depending on the situation. In particular, when using an eye detail region image instead of an eye region image to detect the iris region, the likelihood of the iris not being circular is higher. Therefore, the conjunctival region image generation unit 140 can use image processing algorithms such as convex hulls to more accurately obtain the shape of the iris region. For example, the conjunctival region image generation unit 140 can connect the outermost parts of the given data values ​​(containing all given data values) to form a polygon on a two-dimensional plane of the eye detail region image, and detect the interior of this polygon as the iris region.

[0046] In addition to the above operations, the conjunctival region image generation unit 140 may also use an image processing algorithm or artificial intelligence algorithm capable of detecting the iris region to detect the iris region.

[0047] After detecting the iris region, the conjunctival region image generation unit 140 can generate a conjunctival region image from the remaining area in the eye detail region image excluding the iris region. That is, the conjunctival region image generation unit 140 can mask the iris region in an eye detail region image that only contains the conjunctiva and iris regions to generate a conjunctival region image that retains only the conjunctiva region.

[0048] As an example, the conjunctival region image generation unit 140 can perform bitwise operations on the iris region and eye detail region images to generate a conjunctival region image. For instance, the conjunctival region image generation unit 140 can perform bitwise operations on the detected iris region and eye detail region images to generate a conjunctival region image that only presents the conjunctival region. That is, the conjunctival region image generation unit 140 can use image bitwise operations to set the overlapping area of ​​the eye detail region image and the iris region to 0, thereby generating a conjunctival region image that only retains the conjunctival region.

[0049] As another example, the conjunctival region image generation unit 140 can perform zero-masking on the iris region in the image of the eye detail region to generate a conjunctival region image that retains only the conjunctival region.

[0050] Through the above operations, the eye congestion detection device 100 can preprocess image information to generate an image of the conjunctival region. This enables a highly accurate determination of whether or not there is eye congestion.

[0051] The eye congestion detection device 100 may include a judgment unit 150, which inputs a conjunctival region image into a pre-learned congestion judgment algorithm and derives a classification result value regarding whether congestion exists. The eye congestion detection device 100 may input a conjunctival region image into a pre-learned congestion judgment algorithm that has been learned to determine whether congestion exists using the conjunctival region image.

[0052] For example, a pre-learned congestion detection algorithm is an artificial intelligence algorithm that learns from the entire dataset containing image data of congested conjunctival regions and image data of normal conjunctival regions.

[0053] As an example, the pre-learned congestion detection algorithm is an algorithm learned using a CNN-based artificial intelligence algorithm. To improve the accuracy of congestion detection, conjunctival region images can be extracted from the aforementioned image information to construct a dataset for the congestion detection algorithm. For example, after acquiring the image information, the congestion detection algorithm learner extracts detailed eye region images using feature points from the eye region images and performs a preprocessing process to extract conjunctival region images using the iris region and detailed eye region images.

[0054] Next, after extracting images of the conjunctival region, the learner uses the entire dataset containing both congested and normal conjunctival region image data to learn a congestion detection algorithm.

[0055] Furthermore, the entire dataset is divided into a learning dataset used during the learning process and a validation dataset used during the learning process to calculate accuracy by changing at least one of the epochs, hyperparameters, and hidden layers. The validation dataset, although used during the learning process, is not used for learning itself, but rather for validating the learning using the learning dataset. Because it is used during the learning process, it consists of a dataset different from the regular test set.

[0056] During the learning process using the training dataset, increasing the number of epochs and calculating the accuracy on the validation dataset allows us to identify the optimal epochs that prevent overfitting. Similarly, while learning using the training dataset, changing the number of hyperparameters or hidden layers and calculating the accuracy on the validation dataset allows us to determine the optimal parameters and number of layers.

[0057] In addition, to verify the rationality of the dataset during the learning process, the image information in RGB form before preprocessing can be applied to algorithms such as Grad-CAM, which are interpretable artificial intelligence, to confirm the rationality of using the preprocessed conjunctival region image.

[0058] The judgment unit 150 inputs a conjunctival region image as input to a pre-learned congestion judgment algorithm that has been learned through the above process, and obtains a classification value on whether or not there is congestion as output, so as to determine whether the eye in the image is congested.

[0059] Through the above operations, even images captured using ordinary RGB camera sensors can be accurately determined to determine whether blood vessels are congested. Furthermore, by appropriately combining artificial intelligence algorithms and image processing algorithms, the increased processing time that might occur when using only artificial intelligence algorithms can be prevented.

[0060] The following description, with reference to the accompanying drawings, provides a more detailed and varied explanation of the operation for determining blood congestion using the aforementioned eye congestion detection device.

[0061] Figure 2 This is a diagram illustrating the operation of processing image information according to one embodiment.

[0062] Reference Figure 2 The image acquisition unit can receive image information 200 captured by an RGB camera. Image information 200 may include various objects, including the face of the subject being photographed. For example, image information 200 may include background information. Background information may change in various ways when used in scenarios such as industrial sites, and because it is not captured in a fixed location and under fixed conditions like a medical camera, the possibility of errors is higher.

[0063] Therefore, a preprocessing procedure is needed to extract the desired portion of the image from the image information 200. For this purpose, the feature point acquisition unit can acquire eye region images 220 and 225 from the image information 200. The eye region images can be configured to include feature points from the eye region and exclude feature points from other regions.

[0064] For example, the feature point acquisition unit can detect the facial region 210 after acquiring image information 200. As an example, the facial region 210 can be constructed using a deep learning-based algorithm and detected using a convolutional neural network (CNN). Various algorithms for detecting the facial region 210 have been disclosed and are not limited to any particular algorithm, as long as they can detect the facial region 210.

[0065] However, in this embodiment, it is crucial to obtain feature points for the eye regions 220 and 225 from the face region 210. Therefore, the feature point acquisition unit acquires feature points for the eye regions 220 and 225. For example, cascade-based CNNs or autoencoder-based learning can be used to find facial landmarks. Cascade-based CNNs are ensemble learning methods that combine multiple hierarchical structures or classifiers, while autoencoder-based learning is used to find hidden patterns in the data. As an example, APIs such as Mediapipe, Openpose, MMPose, and OpenFace can be used. Furthermore, feature points for the eye regions 220 and 225 can also be extracted using various image processing techniques, without limitation.

[0066] The facial region 210 can be detected by extracting feature points from the image information 200. Feature points, which can be called markers, can correspond to the coordinate values ​​of parts of the image that are features, such as color and boundaries.

[0067] On the other hand, after acquiring feature points for eye regions 220 and 225, the feature point acquisition unit can separate the two eye regions 220 and 225 respectively, so that the operation of the aforementioned eye congestion detection device can be performed. Eye regions 220 and 225 can be divided into left and right sides, and for each image, image extraction of eye detail areas, generation of conjunctival area images, and export of classification result values ​​can be performed. However, in order to be set as belonging to the same user, the distinguished left and right eye regions 220 and 225 can be mapped to the same flag or recognition value. Thus, it is possible to distinguish whether there is only unilateral eye congestion, and matching can be performed on both the user's left and right eye regions 220 and 225 for determination.

[0068] Figure 3 This is a diagram illustrating the operation of segmenting an image of a region of eye detail according to one embodiment.

[0069] Reference Figure 3 The eye detail region extraction unit can segment and extract the eye detail region image using eye region feature points. The eye region image 300 may contain information from feature points 301 to 307. Feature points 301 to 307 are acquired by the feature point acquisition unit. However, using only the information from feature points 301 to 307 may not be sufficient to extract the eye detail region. This is because the shape of the eye detail region is presented in a curved form, and feature points 301 to 307 are only acquired for a subset of points.

[0070] Therefore, the eye detail region extraction unit can use the coordinate values ​​of feature points 301 to 307 and a preset interpolation algorithm to extract the eye detail region image.

[0071] For example, the preset interpolation algorithm can be set to predict the outer contour coordinates of the eye detail region based on the two-dimensional coordinates of the eye region feature points 301 to 307. For example, although the eye region feature points 301 to 307 are represented as two-dimensional points with coordinate values ​​around the eye region, in order to extract the curved eye detail region, an interpolation algorithm can be used to segment the eye detail region.

[0072] As an example, an interpolation algorithm could be nearest neighbor interpolation, which uses the pixel values ​​of the nearest feature points to determine the coordinates of the outer contour of the eye detail region. While this is faster, the resolution for the boundary lines may be reduced. As another example, an interpolation algorithm could be bilinear interpolation, which uses the pixel values ​​and distance ratios of the four nearest feature points.

[0073] As another example, the interpolation algorithm could be a bicubic interpolation algorithm that uses the pixel values ​​of the 16 nearest neighbor feature points as a weighted product of distance. This can be used when the number of feature points is sufficiently guaranteed. As yet another example, the interpolation algorithm could be a B-spline interpolation algorithm that uses the pixel values ​​of the 16 nearest neighbor feature points as a weighted product of distance and is used to produce a smooth curve shape.

[0074] Furthermore, the interpolation algorithm is unrestricted, as long as it is an algorithm that finds new data values ​​based on given data values. For example, various algorithms such as forward / backward interpolation, linear interpolation, bilinear interpolation, and polynomial interpolation can be used.

[0075] Furthermore, interpolation algorithms can be selectively used based on the number of feature points in the eye region. For example, the eye detail extraction unit can select an interpolation algorithm based on the number of feature points in the eye region and the minimum number of feature points required by each interpolation algorithm. If there are two or more interpolation algorithms with the same minimum number of feature points, a specific interpolation algorithm can be selected based on the priority among preset interpolation algorithms.

[0076] Therefore, it is possible to quickly extract detailed images of the eye region in various image environments.

[0077] The eye detail region extraction unit can use an interpolation algorithm to predict the coordinates of the outer contour lines connecting feature points for extraction. Therefore, the region within the outer contour lines can be extracted as the eye detail region using lines connecting the coordinates of each outer contour line. Thus, the eye region image 300 can be segmented into an eye detail region image containing only the conjunctiva and iris.

[0078] On the other hand, the conjunctival region image generation unit needs to detect the iris region in order to generate images that target only the conjunctival region.

[0079] As an example, the conjunctival region image generation unit can calculate the sum of gray values ​​of pixels on a circle with arbitrary center coordinates and radius values ​​on an eye region image or an eye detail region image, and set the circle with the largest change in the sum of gray values ​​as the radius value changes as the boundary of the iris region.

[0080] The iris region is circular in shape within the eye. Furthermore, the iris region exhibits a more significant color variation compared to the conjunctiva region. Therefore, utilizing these characteristics, the conjunctiva region image generation unit can detect the iris region.

[0081] For example, the conjunctival region image generation unit arbitrarily sets center coordinates within a preset range of center coordinate allocation area in the eye region image. After setting the center coordinates, the conjunctival region image generation unit sets an arbitrary radius value within the preset range for that center coordinate to calculate the sum of gray values ​​of the pixels on that circumference. While changing the radius value by a predetermined offset within the preset range, the conjunctival region image generation unit calculates the sum of gray values ​​of the pixels on the generated circumference. The conjunctival region image generation unit can save the corresponding center coordinates and radius value when the change in gray values ​​exceeds a predetermined standard, and determine the interior of the circle as the iris region. Alternatively, the conjunctival region image generation unit can extract an iris region using only a center coordinate and radius value that changes for the maximum sum of gray values.

[0082] Figure 4 This is a diagram illustrating the operation of detecting the iris region according to one embodiment.

[0083] Reference Figure 4 The conjunctival region image generation unit can calculate candidate iris regions 400, 410, and 420 with the aforementioned center coordinates and radius values. If three candidates are calculated, the conjunctival region image generation unit can use circle number 400, which has the largest rate of change, as a reference and determine the area inside it as the iris region. Figure 4 The image shows a scenario where the iris region is calculated based on an image of the eye region, resulting in multiple iris region candidates due to factors such as the eyelids. In contrast, when analyzing based on detailed areas of the eye, the number of iris region candidates can be limited.

[0084] However, when using only the maximum rate of change as a benchmark, there is still a possibility that the iris region cannot be precisely distinguished. Therefore, this disclosure can also be applied to other embodiments described below.

[0085] For example, iris region candidates can be selected when the sum of grayscale values ​​measured while changing the aforementioned center coordinates and radius values ​​exceeds a predetermined standard. Subsequently, to detect accurate iris region candidates, the iris region can be corrected based on the center coordinates for final extraction.

[0086] As an example, pupil center information detected using a deep learning model can be used to determine the iris region. For instance, new boundary points can be extracted based on the inaccurate boundaries of the detected iris region and corrected to an elliptical shape, thereby allowing for the extraction of a more accurate iris region.

[0087] Refer again Figure 4 After extracting three iris region candidates (400, 410, and 420), the conjunctival region image generation unit uses the DeepEye algorithm to calculate the pupil center information. The DeepEye algorithm estimates a mask image representing the probability that each pixel around the pupil belongs to the pupil region, and calculates the pupil center coordinates through post-processing operations. After calculating the pupil center coordinates, the iris region candidate 400, which is closest to the pupil center coordinates among the three iris region candidates (400, 410, and 420), is detected as the iris region.

[0088] The DeepEye model uses convolutional (Conv.) layers and strided conv. layers, along with residuals, to form two convolutional layers. The image size is reduced through the strided conv. layers. In the Atrous Spatial Pyramid Pooling (ASPP) layer, atrous convolution / dilated convolution, 1×1 convolution, and max-pooling are performed simultaneously on the input image, and all results are combined. Then, the number of channels is reduced through 1×1 convolution, and finally, bilinear up-sampling is used to make the output size the same as the input.

[0089] Figure 5 This is a diagram illustrating the operation of detecting the iris region according to another embodiment.

[0090] Reference Figure 5The conjunctival region image generation unit can use the convex hull algorithm to detect the pupil region based on the outer contour of the pupil. For example, the conjunctival region image generation unit can use the convex hull algorithm, which connects the outermost parts of the given data values ​​on a two-dimensional plane and is shown in polygon shape 500, to detect the iris region.

[0091] The convex hull algorithm is defined as generating a convex polygon containing all points from a subset of points existing in a two-dimensional coordinate plane. As mentioned above, the convex hull algorithm refers to the algorithm for obtaining the points that constitute the convex hull given the coordinates of each point in the two-dimensional coordinate plane. For this purpose, there are several different algorithms, and the outermost contour can be calculated using the Graham scan algorithm.

[0092] The convex hull algorithm extracts the coordinates of pixels with a preset grayscale value or higher from image information. These coordinates are then set on a two-dimensional plane. A reference point is selected within this coordinate system. Typically, the reference point is the one with the smallest y-coordinate. Using this reference point, other points are aligned counter-clockwise. Lines are then connected counter-clockwise between the coordinates, while simultaneously checking if the Counter Clockwise Wise (CCW) is 0 or higher. If it is less than 0, the connection between that coordinate and the previous coordinate is removed, and the same operation is repeated for other coordinates selected from the previous set. In other words, only lines with a CCW of 0 or higher are set, thus determining the outermost contour line.

[0093] As described above, the conjunctival region image generation unit can detect the iris region in various ways, such as embodiments using center coordinates and radius values, embodiments using center coordinates and radius values ​​and finding the pupil center point to detect the final iris region, or embodiments applying the convex hull algorithm, etc.

[0094] Figure 6 This is a diagram illustrating the operation of generating a conjunctival region image according to one embodiment.

[0095] Reference Figure 6 After detecting the iris region, the conjunctival region image generation unit can mask the iris region 610 in the eye detail region image 600 to generate an image containing only the conjunctival region 620.

[0096] For example, the conjunctival region image generation unit can perform bitwise operations on the iris region 610 and the eye detail region image 600 to generate a conjunctival region image 620. The conjunctival region image generation unit can provide data to set the overlapping area of ​​the iris region 610 and the eye detail region image 600 to 0, thereby retaining only the conjunctival portion, which is the white part of the eye. Alternatively, the conjunctival region image generation unit can map the iris region 610 onto the eye detail region image 600, thereby generating the conjunctival region image 620 by zero-masking the iris region 610.

[0097] Therefore, the remaining areas except for the conjunctival region image 620 are masked as black, thus enabling accurate identification of the conjunctival region.

[0098] The conjunctival region image generated through these operations can be used as input to a congestion detection algorithm, deriving the result value for determining whether congestion has occurred. By accurately masking the conjunctival region image and processing it with black, the accuracy of determining whether congestion has occurred in the white areas of the conjunctiva can be improved.

[0099] Furthermore, the aforementioned operation of extracting conjunctival region image information from image information can also be applied to the image information preprocessing operations used in the congestion detection algorithm. That is, to perform a more accurate learning process, preprocessing operations are performed to extract conjunctival region images from the image information. The preprocessed conjunctival region images are used as the entire dataset for the congestion detection algorithm.

[0100] For example, the congestion detection algorithm can be an artificial intelligence algorithm based on CNN. Exemplarily, it can be generated based on AlexNet, ResNet, GoogLeNet, VGG, etc.

[0101] Figure 7 This is a diagram illustrating a dataset used to illustrate a congestion determination algorithm according to one embodiment.

[0102] Reference Figure 7 During the learning process of the congestion detection algorithm, the entire dataset can be divided into a training dataset and a validation dataset. A test dataset can also be set up within a portion of the dataset. The entire dataset can include images of congested conjunctival regions and images of normal conjunctival regions.

[0103] The training dataset is used to train the congestion judgment model. The learner uses the training dataset to train different congestion judgment models for different epochs. Here, different congestion judgment models refer to slight changes to the hidden layers or hyperparameters.

[0104] A validation dataset is a dataset used to validate a model that has completed its learning process. A test dataset is a dataset used to evaluate the performance of a model that has completed both learning and validation.

[0105] For example, the entire dataset can be divided into the learning dataset and the test dataset in an 8:2 ratio, or the learning dataset, the test dataset, and the validation dataset can be divided in a 6:2:2 ratio.

[0106] The validation and test datasets share the characteristic that they are not used to train the model. However, although the validation dataset is not used for training, it still participates in the model's learning process.

[0107] For example, the first congestion-determining model learns by varying the epoch using a training dataset. Validation is then performed for each epoch using a validation dataset. In the case of artificial intelligence models, learning over too many epochs can lead to overfitting and increased error. To prevent this, accuracy is assessed using the validation dataset for each epoch. When the validation dataset shows the lowest error rate (highest accuracy), epoch progression for that model is stopped, thus halting further learning. A congestion-determining model learned in this way can be applied as a congestion-determining algorithm.

[0108] Similarly, for the second to Nth congestion judgment models with some variant hyperparameters and hidden layers, accuracy was measured using a validation dataset while increasing the number of epochs.

[0109] Through the above operations, the epoch, hyperparameter, and hidden layer of the model with the highest accuracy obtained by verifying the accuracy measurement results of the dataset are used to determine the congestion judgment model.

[0110] The eye redness detection device described above preprocesses images captured by common camera devices such as RGB cameras, thereby improving the accuracy of determining whether or not eye redness exists. Furthermore, during image preprocessing, by selecting only the desired portion of the image and applying a mask, a fast processing speed is provided.

[0111] Hereinafter, with reference to the accompanying drawings, the method for detecting ocular hyperemia by performing the operations of the above embodiments will be briefly described again. The method described below can perform some or all of the above operations.

[0112] Figure 8 This is a flowchart illustrating a method for detecting ocular congestion according to one embodiment.

[0113] Reference Figure 8 The method for detecting whether the eyes are congested may include an image acquisition step S810 for acquiring image information.

[0114] For example, in the image acquisition step, image information can be received from a sensor such as a camera. Here, "camera" refers to a common camera such as an RGB camera mentioned above. Alternatively, the image acquisition step can also acquire image information by capturing frames from a video.

[0115] The eye congestion detection method may include a feature point acquisition step S820, which acquires eye region feature points of the eye region image extracted from image information.

[0116] In the feature point acquisition step, an image of the eye region, including the eyes, can be extracted from the acquired image information. For example, the eye region image can be configured to include eye region feature points, excluding feature points from other regions contained in the facial image information. As an example, the image information may include background and facial image information, as well as other body images such as the neck. In the feature point acquisition step, after acquiring the image information, artificial intelligence algorithms can be used to extract facial image information from the image information. For this purpose, a marker extractor can be used to extract facial image information from the image information. The marker extractor performs the function of extracting feature points related to the facial contours of a person from the image.

[0117] After extracting facial image information, facial image information containing the eye area can be identified from the facial image information. As mentioned above, the facial image information can be set to contain only feature points assigned to the eye region. Therefore, in the feature point acquisition step, feature points of the eye region can be obtained from the image information.

[0118] The eye congestion detection method may include an eye detail region extraction step S830, which uses eye region feature points to segment and extract the eye detail region image.

[0119] An eye region image may contain feature points of the eye area, as well as images of the periocular skin and eyebrows. Therefore, to accurately determine whether there is congestion, the eye detail region extraction step needs to extract only the eye detail region image from the eye region image.

[0120] Therefore, in the eye detail region extraction step, eye region feature points can be utilized. In this step, the acquired eye region feature points and image processing algorithms can be used to determine the outer contour of the eye detail region. After determining the outer contour, in the eye detail region extraction step, only the portion within the outer contour can be extracted as the eye detail region image.

[0121] As an example, a preset interpolation algorithm can be set to predict the outer contour coordinates of the eye detail region based on the two-dimensional coordinates of the feature points in the eye region, thereby extracting the eye detail region image. For example, after confirming the two-dimensional coordinates of N feature points in the eye region, the interpolation algorithm can use these two-dimensional coordinates to predict the coordinates between the feature points in the eye region. For example, the interpolation algorithm can use various algorithms such as forward / backward interpolation, linear interpolation, bilinear interpolation, and polynomial interpolation. Thus, in the eye detail region extraction step, the outer contour coordinates connecting the feature points in the eye region can be obtained, and the outer contour line of the eye detail region can be extracted accordingly.

[0122] In the step of extracting eye detail areas, after extracting the outer contour line, the image can be extracted based on this, thereby generating an image that only targets the eye detail areas within the outer contour line.

[0123] The method for detecting eye congestion may include a conjunctival region image generation step S840, which detects the iris region from an eye region image or an eye detail region image, and generates a conjunctival region image using the eye detail region image and the iris region.

[0124] In the conjunctival region image generation step, only the conjunctival region used as a reference for whether or not there is congestion can be separated to generate a conjunctival region image. Therefore, in this step, the iris region needs to be detected from the eye detail region.

[0125] As an example, in the conjunctival region image generation step, the sum of grayscale values ​​of pixels on a circle defined by arbitrary center coordinates and radius values ​​can be calculated on the eye region image or eye detail region image. The circle whose sum of grayscale values ​​changes most with the radius value is then set as the boundary of the iris region. For instance, in the conjunctival region image generation step, arbitrary coordinates within the eye region image or eye detail region image can be set as center coordinates, and the radius value can also be set to arbitrary values. Then, the sum of grayscale values ​​of pixels on the circle for each arbitrary center coordinate and arbitrary radius value can be calculated.

[0126] In the conjunctival region image generation step, the sum of these grayscale values ​​can be continuously calculated while changing the center coordinates and radius values. In this step, the circle with the largest change in the calculated sum of grayscale values ​​can be set as the boundary of the iris region, and the center coordinates and radius values ​​at this point can be saved. For example, in the conjunctival region image generation step, the center coordinates can be determined using an experimental range for the eye center coordinates during the generation of the eye region image or eye detail region image. That is, in the eye region image or eye detail region image, the center point of the iris (i.e., the center point of the eye) is formed in the central part of the image. Therefore, for fast processing, the center coordinates are arbitrarily set within a preset coordinate range in the conjunctival region image generation step. Similarly, the radius value can also be set to a range of arbitrary values ​​to prioritize selection within a predetermined range based on image scale, etc. Through this operation, the iris region can be detected more quickly in the conjunctival region image generation step.

[0127] As another example, in the conjunctival region image generation step, a circular detection algorithm can be used to determine a predetermined number of candidate iris regions, and the DeepEye model can be used to determine the pupil center point. This allows for the selection of the iris region corresponding to the pupil center point from the candidate iris regions, which is then determined as the final iris region. Therefore, even when multiple candidate iris regions, such as eyelashes, are generated using the circular detection algorithm, a more accurate iris region can be detected. The circular detection algorithm can utilize the aforementioned algorithm that uses pixel changes while altering the center coordinates and radius values ​​for detection.

[0128] As another example, in the conjunctival region image generation step, a convex hull algorithm can be applied to either the eye region image or the eye detail region image to detect the iris region. For instance, in the conjunctival region image generation step, the iris region can be detected using a convex hull algorithm that connects the outermost portions of multiple data values ​​to form a polygon shape, utilizing the eye detail region image. Since the shape of the iris may not be circular due to factors like the eyelids, the iris region detection described above using center coordinates and radius values ​​may introduce errors depending on the situation. In particular, when using an eye detail region image instead of an eye region image to detect the iris region, the likelihood of the iris not being circular is higher. Therefore, in the conjunctival region image generation step, image processing algorithms such as convex hulls can be used to more accurately obtain the shape of the iris region. For example, in the conjunctival region image generation step, on the two-dimensional plane of the eye detail region image, the outermost portions of the given data values ​​(containing all given data values) can be connected to form a polygon shape, and the interior of this polygon can be detected as the iris region.

[0129] In addition to the above operations, in the conjunctival region image generation step, image processing algorithms or artificial intelligence algorithms capable of detecting the iris region can also be used to detect the iris region.

[0130] After detecting the iris region, in the conjunctival region image generation step, the remaining areas in the eye detail region image other than the iris region can be generated as a conjunctival region image. That is, in the conjunctival region image generation step, the iris region can be masked in the eye detail region image containing only the conjunctiva and iris regions to generate a conjunctival region image that retains only the conjunctiva region.

[0131] As an example, in the conjunctival region image generation step, bitwise operations can be performed on the iris region and eye detail region images to generate a conjunctival region image. For instance, in the conjunctival region image generation step, bitwise operations can be performed on the detected iris region and eye detail region images to generate a conjunctival region image that only presents the conjunctival region. That is, in the conjunctival region image generation step, image bitwise operations can be used to set the overlapping area of ​​the eye detail region image and the iris region to 0, thereby generating a conjunctival region image that only retains the conjunctival region.

[0132] As another example, in the conjunctival region image generation step, the iris region can be zero-masked in the image of the eye detail region to generate a conjunctival region image that retains only the conjunctival region.

[0133] The eye congestion detection method may include a judgment step S850, in which the conjunctival region image is input into a pre-learned congestion judgment algorithm, and a classification result value for whether or not there is congestion is derived.

[0134] For example, a pre-learned congestion detection algorithm is an artificial intelligence algorithm that learns from the entire dataset containing image data of congested conjunctival regions and image data of normal conjunctival regions.

[0135] As an example, the pre-learned congestion detection algorithm is an algorithm learned using a CNN-based artificial intelligence algorithm. To improve the accuracy of congestion detection, conjunctival region images can be extracted from the aforementioned image information to construct a dataset for the congestion detection algorithm. For example, after acquiring the image information, the congestion detection algorithm learner extracts detailed eye region images using feature points from the eye region images and performs a preprocessing process to extract conjunctival region images using the iris region and detailed eye region images.

[0136] Next, after extracting images of the conjunctival region, the learner uses the entire dataset containing both congested and normal conjunctival region image data to learn a congestion detection algorithm.

[0137] Furthermore, the entire dataset is divided into a learning dataset used during the learning process and a validation dataset used during the learning process to calculate accuracy by changing at least one of the epochs, hyperparameters, and hidden layers. The validation dataset, although used during the learning process, is not used for learning itself, but rather for validating the learning using the learning dataset. Because it is used during the learning process, it consists of a dataset different from the regular test set.

[0138] During the learning process using the training dataset, increasing the number of epochs and calculating the accuracy on the validation dataset allows us to identify the optimal epochs that prevent overfitting. Similarly, while learning using the training dataset, changing the number of hyperparameters or hidden layers and calculating the accuracy on the validation dataset allows us to determine the optimal parameters and number of layers.

[0139] In addition, to verify the rationality of the dataset during the learning process, the image information in RGB form before preprocessing can be applied to algorithms such as Grad-CAM, which are interpretable artificial intelligence, to confirm the rationality of using the preprocessed conjunctival region image.

[0140] In the judgment step, the input conjunctival region image is used as the input to the pre-learned congestion judgment algorithm that has been learned through the above process, and the classification value of whether or not there is congestion is obtained as the output to determine whether the eye in the image is congested.

[0141] In each of the above steps, a reference can be executed. Figures 1 to 7 The description may include some or all of the operations, and the execution step may involve performing two or more steps simultaneously or changing the order of some steps. Furthermore, specific steps may be combined into one step, or separated into two or more steps.

[0142] The above description is merely an illustrative example of the technical concept of this disclosure. For those skilled in the art, various modifications and variations can be made without departing from the essential characteristics of this technical concept. Furthermore, these embodiments are not intended to limit the technical concept of this disclosure, but rather to illustrate it; therefore, the scope of this technical concept is not limited by these embodiments. The scope of protection of this disclosure must be interpreted by the following claims, and all technical concepts within the equivalent scope must be interpreted as included within the scope of this disclosure.

Claims

1. An eye redness detection device, comprising: The image acquisition unit acquires image information; The feature point acquisition unit acquires eye region feature points of the eye region image extracted from the image information; The eye detail region extraction unit uses the eye region feature points to segment and extract the eye detail region image; The conjunctival region image generation unit detects the iris region from the eye region image or the eye detail region image, and generates a conjunctival region image using the eye detail region image and the iris region; as well as The judgment unit inputs the conjunctival region image into a pre-learned congestion judgment algorithm and derives a classification result value regarding whether there is congestion.

2. The eye congestion detection device according to claim 1, wherein, The eye region image is configured to include the eye region feature points and exclude other region feature points contained in the facial image information.

3. The eye congestion detection device according to claim 1, wherein, The eye detail region extraction unit uses the eye region feature points and a preset interpolation algorithm to extract the eye detail region image.

4. The eye congestion detection device according to claim 3, wherein, The preset interpolation algorithm is set to predict the outer contour coordinates of the eye detail region based on the two-dimensional coordinates of the feature points of the eye region, thereby extracting the image of the eye detail region.

5. The eye congestion detection device according to claim 1, wherein, The conjunctival region image generation unit calculates the sum of gray values ​​of pixels on a circle with arbitrary center coordinates and radius values ​​on the eye region image or the eye detail region image, and sets the circle with the largest change in the sum of gray values ​​as the radius value changes as the boundary of the iris region.

6. The eye congestion detection device according to claim 1, wherein, The conjunctival region image generation unit applies a convex hull algorithm to the eye region image or the eye detail region image to detect the iris region.

7. The eye congestion detection device according to claim 1, wherein, The conjunctival region image generation unit generates the remaining area in the eye detail region image, excluding the iris region, as the conjunctival region image.

8. The eye congestion detection device according to claim 7, wherein, The conjunctival region image generation unit performs bitwise operations on the images of the iris region and the eye detail region to generate the conjunctival region image.

9. The eye congestion detection device according to claim 1, wherein, The pre-learned congestion detection algorithm is an artificial intelligence algorithm that learns from the entire dataset containing image data of congested conjunctival regions and image data of normal conjunctival regions. The entire dataset is divided into a learning dataset used during the learning process and a validation dataset used during the learning process to calculate accuracy by changing at least one of the rounds, hyperparameters, and hidden layers.

10. A method for detecting eye redness, used to detect whether the eye is red, comprising: Image acquisition steps: acquiring image information; The feature point acquisition step involves acquiring eye region feature points extracted from the image information. The step of extracting eye detail regions involves segmenting and extracting the eye detail region image using the feature points of the eye region. The conjunctival region image generation step involves detecting the iris region from the eye region image or the eye detail region image, and generating a conjunctival region image using the eye detail region image and the iris region. as well as The judgment step involves inputting the conjunctival region image into a pre-learned congestion judgment algorithm to derive a classification result value regarding whether or not there is congestion.

11. The method for detecting ocular congestion according to claim 10, wherein, The eye region image is configured to include the eye region feature points and exclude other region feature points contained in the facial image information.

12. The method for detecting ocular congestion according to claim 10, wherein, In the step of extracting the detailed eye area The image of the detailed area of ​​the eye is extracted using the feature points of the eye region and a preset interpolation algorithm.

13. The method for detecting ocular congestion according to claim 12, wherein, The preset interpolation algorithm is set to predict the outer contour coordinates of the eye detail region based on the two-dimensional coordinates of the feature points of the eye region, thereby extracting the image of the eye detail region.

14. The method for detecting ocular congestion according to claim 10, wherein, In the conjunctival region image generation step, On the image of the eye region or the image of the eye detail region, the sum of the gray values ​​of the pixels on a circle with arbitrary center coordinates and radius values ​​is calculated, and the circle whose sum of gray values ​​changes the most with the change of the radius value is set as the boundary of the iris region.

15. The method for detecting ocular congestion according to claim 10, wherein, In the conjunctival region image generation step, The convex hull algorithm is applied to the image of the eye region or the image of the eye detail region to detect the iris region.

16. The method for detecting ocular congestion according to claim 10, wherein, In the conjunctival region image generation step, Bitwise operations are performed on the images of the iris region and the detailed eye region to generate the image of the conjunctival region.

17. The method for detecting ocular congestion according to claim 10, wherein, The pre-learned congestion detection algorithm is an artificial intelligence algorithm that learns from the entire dataset containing image data of congested conjunctival regions and image data of normal conjunctival regions. The entire dataset is divided into a learning dataset used during the learning process and a validation dataset used during the learning process to calculate accuracy by changing at least one of the rounds, hyperparameters, and hidden layers.

Images