Method for calculating age of eye area

Abstract

The present invention provides a method for calculating the age of the eye area, the method comprising the steps in which: (a) an eye area image extraction unit (310) extracts an eye area image from a face image including the eye area of an evaluation target; (b) a calculation unit (320) calculates wrinkle information, pigmentation information, and elasticity information from the eye area image using a preset method; and (c) an eye area age calculation unit (330) calculates the age of the eye area by inputting at least one of the wrinkle information, pigmentation information, or elasticity information into a pre-generated linear regression model.

Description

How to calculate eye age

[0001] The present invention relates to a method for calculating eye age, and more specifically, to a method for calculating the eye age of a subject using wrinkle information, pigmentation information, and elasticity information.

[0002]

[0003] According to recent research, the appearance of the area around or the edges of the eyes is a significant factor in determining first impressions. Consequently, there is a growing demand and interest in cosmetics that can slow down eye aging. Furthermore, the skin around the eyes reflects a person's health status, as evidenced by the fact that pigmentation around the eyes is used as an auxiliary criterion for diagnosing atopic dermatitis.

[0004] Meanwhile, the method for assessing the degree of aging in the skin around the eyes needs to differ from that used for other facial skin. First, this is due to the anatomical characteristics of the eye area. The area around the eyes is surrounded by the orbicularis oculi muscle, which is responsible for blinking and facial expressions. The repetitive movement of this muscle causes repeated undulations to form in the skin around the eyes. Consequently, the skin around the eyes is more prone to wrinkle formation than other facial skin. Second, this is due to the physiological characteristics of the skin around the eyes. The skin around the eyes is thinner than other facial skin and has fewer sweat and sebaceous glands, resulting in a reduced ability to retain moisture. Consequently, the skin's barrier function is weakened, making it more vulnerable to aging than other facial skin. Therefore, considering both the anatomical and physiological characteristics of the eye area, the assessment of the degree of aging in this area needs to be distinguished from the assessment of aging in other facial skin.

[0005] Meanwhile, according to research, morphological changes around the eyes associated with aging include sagging of the eyelids and the corners of the eyes, as well as a decrease in the distance between the canthus. In other words, as people age, the eyelids and corners of the eyes tend to sag, and the distance between the eyes tends to decrease. Additionally, crow's feet (wrinkles at the corners of the eyes) form less in younger people and progress more rapidly compared to under-eye wrinkles.

[0006] Therefore, although there is a need for technology capable of assessing the degree of eye aging by considering factors such as eyelid and corner sagging, the distance between the canthuses, and wrinkles, there is a shortage of technology to meet this demand.

[0007]

[0008] (Patent Document 1) Korean Published Patent Document No. 10-2023-0046210 (April 5, 2023)

[0009]

[0010] The present invention aims to provide a method for calculating eye age that can calculate the eye age of a subject using wrinkle information, pigmentation information, and elasticity information.

[0011]

[0012] An embodiment of the present invention for solving the above-mentioned problem provides a method for calculating eye age, comprising: (a) a step in which an eye area image extraction unit (310) extracts an eye area image from a face image including the eye area of ​​a subject to evaluation; (b) a step in which a calculation unit (320) calculates wrinkle information, pigmentation information, and elasticity information from the eye area image using a preset method; and (c) a step in which an eye age calculation unit (330) inputs at least one of the wrinkle information, pigmentation information, and elasticity information into a preset linear regression model to calculate the eye age.

[0013]

[0014] The above wrinkle information may include at least one of the following: wrinkle information next to the eye (Wb), wrinkle information below the eye (Wu), and wrinkle information at the upper inner corner of the eye (Wa).

[0015]

[0016] The above pigmentation information may include at least one of the pigmentation information next to the eye (Pb) and pigmentation information under the eye (Pu).

[0017]

[0018] The above elasticity information may include at least one of the eye sagging information (Md) and margin information (Mw).

[0019]

[0020] The above eyelid sagging information can be calculated through the following mathematical formula 1.

[0021] [Mathematical Formula 1]

[0022] Md = H1 - H2

[0023] The above margin information can be calculated through the following mathematical formula 2.

[0024] [Mathematical Formula 2]

[0025] Mw = (L2+ L3) / L1

[0026] The above step (b) may include: (b1) a step in which a measuring unit (321) extracts wrinkles next to the eye, wrinkles under the eye, pigmentation next to the eye, and pigmentation under the eye, and calculates wrinkle information next to the eye (Wb), wrinkle information under the eye (Wu), pigmentation information next to the eye (Pb), and pigmentation information under the eye (Pu) from the eye area image; (b2) a step in which a wrinkle extraction unit (322) extracts wrinkles on the inner upper eye from the eye area image and calculates wrinkle information on the inner upper eye (Wa); and (b3) a step in which an elasticity calculation unit (323) calculates eye sagging information (Md) and margin information (Mw) from the eye area image.

[0027]

[0028] The method for the measurement unit (321) of step (b1) to calculate the eye side wrinkle information (Wb) and the eye under wrinkle information (Wu) from the eye area image comprises: (b11) the measurement unit (321) extracting the eye side wrinkle image and the eye under wrinkle image from the eye area image; (b12) the measurement unit (321) converting the eye side wrinkle image and the eye under wrinkle image into grayscale; (b13) the measurement unit (321) extracting the eye side wrinkle and the eye under wrinkle from the grayscale-converted image through a hybrid Hessian filter; (b14) the measurement unit (321) highlighting the extracted eye side wrinkle and eye under wrinkle and removing noise through a Fast Fourier Transform; and (b15) the measurement unit (321) may include a step of calculating eye side wrinkle information (Wb) and eye lower wrinkle information (Wu) based on the emphasized eye side wrinkle and eye lower wrinkle.

[0029]

[0030] The method by which the measurement unit (321) of step (b1) calculates the pigmentation information next to the eye (Pb) and the pigmentation information under the eye (Pu) from the eye area image comprises: (b16) the measurement unit (321) extracting the image next to the eye and the image under the eye from the eye area image; (b17) the measurement unit (321) converting the image next to the eye and the image under the eye into grayscale; (b18) the measurement unit (321) selecting an area darker than the surrounding skin color through a local thresholding algorithm in the extracted image; (b19) the measurement unit (321) excluding an area among the selected area that is below a preset threshold; and (b20) the measurement unit (321) may include a step of calculating pigmentation information next to the eye (Pu) and pigmentation information under the eye (Pu) by scoring the remaining area excluding the area excluded in step (b19) from the image extracted in step (b18) according to a preset standard.

[0031]

[0032] The above step (b2) may include: (b21) the wrinkle extraction unit (322) extracting an image of the inner upper corner of the eye from the eye area image; (b22) the wrinkle extraction unit (322) converting the image of the inner upper corner of the eye into grayscale; (b23) the wrinkle extraction unit (322) extracting the inner upper corner wrinkle from the image converted into grayscale through a hybrid hessian filter; (b24) the wrinkle extraction unit (322) highlighting the extracted inner upper corner wrinkle and removing noise through a Fast Fourier Transform; and (b25) the wrinkle extraction unit (322) calculating inner upper corner wrinkle information (Wa) based on the highlighted inner upper corner wrinkle.

[0033]

[0034] The above step (b3) may include: (b31) a step in which the elasticity calculation unit (323) calculates the eye sagging information (Md) using the difference in y-coordinates between the inner eye feature point and the outer eye feature point among the face feature points; and (b32) a step in which the elasticity calculation unit (323) calculates margin information (Mw) by dividing the sum of the left and right distances to the face contour line that is horizontal to the outer eye feature point by the width of the entire face up to the face contour line that is horizontal to the eye.

[0035]

[0036] The linear regression model generated in step (c) above may be a plurality of linear regression models generated by the model generation unit (331) learning at least one or a combination of pre-stored wrinkle information, pigmentation information, and elasticity information, and pre-stored actual age information received from a database.

[0037]

[0038] The above step (c) may include: (c1) a step in which at least one type among wrinkles, pigmentation, and elasticity is input into an input unit (120); (c2) a step in which a model determination unit (332) determines a linear regression model that matches the type input into the input unit (120) among the plurality of linear regression models generated; and (c3) a step in which an eye age calculation unit (330) inputs at least one information among wrinkle information, pigmentation information, and elasticity information calculated by the calculation unit (320) of step (b) into the linear regression model determined in step (c2) to calculate the eye age.

[0039]

[0040] The eye area image of step (a) above can be extracted by extracting facial feature points and using eye area and eyebrow feature point information.

[0041]

[0042] Another embodiment of the present invention for solving the above-mentioned problem provides a computer-readable recording medium having a program recorded thereon for performing the above-mentioned method on a computer.

[0043]

[0044] Another embodiment of the present invention for solving the above-mentioned problem provides a computer program stored in a computer-readable storage medium to execute the above-mentioned method.

[0045]

[0046] A method for calculating eye age according to one embodiment of the present invention has the effect of quantitatively evaluating the degree of eye aging by calculating eye age using wrinkle information, pigmentation information, and elasticity information.

[0047] A method for calculating eye age according to one embodiment of the present invention calculates eye age by determining a linear regression model that matches the input type and the purpose of use when a user inputs at least one type among wrinkles, pigmentation, and elasticity into an input section. Through this, by determining a model suitable for the product used by the user and calculating eye age, the effect of product usage can be assessed.

[0048] The eye age calculation method according to one embodiment of the present invention has the effect of informing the user of their current eye aging status by having a display unit compare and display the eye age with the actual age.

[0049]

[0050] FIG. 1 is a diagram showing the eye age calculation process of an eye age calculation system according to one embodiment of the present invention.

[0051] Figure 2 is a diagram illustrating an exemplary case in which a model generation unit generates multiple linear regression models.

[0052] Figure 3 (a) is an example photo showing the location where information on wrinkles next to the eye, wrinkles below the eye, pigmentation next to the eye, and pigmentation below the eye is measured, (b) is an example photo showing the location where information on the inner upper corner of the eye is measured, and (c) is an example photo showing the method of measuring elasticity information.

[0053] Figure 4 is a diagram showing the correlation between wrinkle information, pigmentation information, elasticity information, and eye age.

[0054] FIG. 5 is a flowchart of a method for calculating eye age according to an embodiment of the present invention.

[0055] FIG. 6 is a flowchart of a method for calculating eye age according to another embodiment of the present invention.

[0056] Figure 7 is a photograph showing facial feature points as an example.

[0057] FIG. 8 is a diagram showing the results of a verification experiment to verify the correlation between the predicted age predicted through a plurality of linear regression models according to the present invention and the actual age.

[0058] Figure 9 is a diagram showing the results of a verification experiment to verify the association between the age of the eyes predicted according to the method of the present invention and the history of past diseases.

[0059]

[0060] In some cases, to avoid obscuring the concept of the present invention, known structures and devices may be omitted or illustrated in the form of a block diagram focusing on the core functions of each structure and device.

[0061] Throughout the specification, when a part is described as "comprising" or "including" a component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components. Furthermore, terms such as "...part," "...unit," and "module" as used in the specification refer to a unit that processes at least one function or operation, and this may be implemented in hardware, software, or a combination of hardware and software. Additionally, "one (a or an)," "one," "the," and similar related terms may be used in the context describing the invention (particularly in the context of the following claims) to include both singular and plural forms, unless otherwise indicated in the specification or clearly contradicted by the context.

[0062] In describing the embodiments of the present invention, specific descriptions of known functions or configurations will be omitted if it is determined that such detailed descriptions could unnecessarily obscure the essence of the invention. Furthermore, the terms described below are defined in consideration of their functions in the embodiments of the present invention, and these definitions may vary depending on the intentions or conventions of the user or operator. Therefore, such definitions should be based on the content throughout this specification.

[0063]

[0064] In the following, “wrinkle information” refers to information in which wrinkles are quantified and stored according to their degree, and “pigmentation information” refers to information in which pigmentation is quantified and stored according to its degree. The quantification method may utilize known methods for measuring wrinkle and pigmentation values, or methods for calculating scores based on criteria pre-set by the user. However, the method is not limited to these and may be implemented by modifying it in various ways.

[0065]

[0066] In the following, “elasticity information” refers to information that quantifies and stores morphological changes in the eye area. Morphological changes in the eye area may include sagging of the eyelid and the eye area or a decrease in the distance between the canthus. In the following, “eye sagging information” refers to the downward sagging of the eyelid and the eye area, and “margin information” refers to the increase in the distance between the outer corner of the eye and the facial boundary line resulting from the decrease in the distance between the canthus and the sagging of the eye area, which causes the outer corner of the eye to shift toward the central part of the face.

[0067]

[0068] FIG. 1 is a diagram showing the eye age calculation process of an eye age calculation system according to an embodiment of the present invention, FIG. 2 is a diagram exemplarily showing a model generation unit generating a plurality of linear regression models, FIG. 3 (a) is an example photograph showing the location where eye side wrinkle information, eye under wrinkle information, eye side pigmentation information, and eye under pigmentation information are measured, (b) is an example photograph showing the location where inner upper corner information is measured, (c) is an example photograph showing a method of measuring elasticity information, and FIG. 4 is a diagram showing the correlation between wrinkle information, pigmentation information, elasticity information and eye age.

[0069]

[0070] Hereinafter, an eye age calculation system according to an embodiment of the present invention will be described in detail with reference to FIGS. 1 to 4.

[0071] Referring to FIG. 1, an eye age calculation system (1) according to one embodiment of the present invention includes a user terminal (100), a skin analysis unit (200), and a server (300).

[0072] The user terminal (100) may include an image acquisition unit (110), an input unit (120), a communication unit (130), and a display unit (140).

[0073] The image acquisition unit (110) can acquire a face image including the eye area of ​​the subject to evaluation by directly photographing the face of the subject to evaluation or receiving an image from an external device.

[0074] Various information to be used for information processing of the configuration of the user terminal (100) or server (300) is input into the input unit (120). As described below, at least one type of age, wrinkles, pigmentation, and elasticity of the subject to evaluation may be input into the input unit (120).

[0075] The communication unit (130) is configured for communication with an external device or server (300). Various previously disclosed methods may be used for communication.

[0076] The display unit (140) visualizes information received from the server (300). The display unit (140) can visualize the eye age calculated by the server (300). In another embodiment, a screen comparing the user's actual age and eye age can be displayed. If the eye age is older than the actual age, customized cosmetics for preventing eye aging can be recommended to the user.

[0077]

[0078] The skin analysis unit (200) measures wrinkle information (Wb, Wu) next to and below the eyes and pigmentation information (Pb, Pu) next to and below the eyes from a face image generated by photographing the face of the subject to evaluation. In one embodiment, the skin analysis unit (200) may include Janus 3 (PIE), a known device for measuring wrinkle information and pigmentation information. Since Janus 3 uses a light source of normal light and polarized light, it can generate a normal light image by photographing the face with normal light and a polarized image by photographing the face with polarized light. Wrinkle information can be extracted from the normal light image, and pigmentation information can be extracted from the polarized image.

[0079]

[0080] The server (300) includes an eye area image extraction unit (310), a calculation unit (320), an eye area age calculation unit (330), and a communication unit (340).

[0081] The eye area image extraction unit (310) extracts an eye area image from a face image received from an image acquisition unit (110) or a skin analysis unit (200). Face feature points can be extracted from the face image, and an eye area image can be extracted through eye area, eyebrow, and face contour feature point information. Specifically, face feature points can be extracted from the face image through OpenCV after processing the face image through OpenCV and then using a data mining tool (dlib). Face feature points refer to specific points on the face shown in FIG. 7. The eye area image can be a minimum rectangle or a rectangle including some margins that includes all points 1, 2, 16, 27, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, and 48 in FIG. 7, and can include coordinate information for each feature point.

[0082] The calculation unit (320) is configured to calculate basic information for estimating eye age from an eye image and may include a measurement unit (321), a wrinkle extraction unit (322), and an elasticity calculation unit (323).

[0083] The measuring unit (321) measures wrinkle information and pigmentation information next to and under the eyes from an image of the eye area. Specifically, it measures wrinkle information next to the eyes (Wb), wrinkle information under the eyes (Wu), pigmentation information next to the eyes (Pb), and pigmentation information under the eyes (Pu). The positions next to and under the eyes are the same as the positions shown in FIG. 3 (a). Meanwhile, FIG. 3 (a) is illustrated as if wrinkle information is measured based on the right eye and pigmentation information is measured based on the left eye, but this is merely an example, and wrinkle information and pigmentation information can be measured on either the left or right eye, and the average of the left and right measurements can be used to improve accuracy.

[0084] To measure information on wrinkles next to and below the eyes, the measurement unit (321) can convert an image of the eye area into grayscale and extract wrinkles next to and below the eyes from the converted image. Subsequently, the extracted wrinkles next to and below the eyes can be highlighted and noise removed through a Fast Fourier Transform. The highlighted wrinkles next to and below the eyes can be scored into wrinkle values ​​(range of 0 to 100, with higher values ​​indicating greater aging) according to preset criteria, and then stored as wrinkle information next to the eyes (Wb) and wrinkle information below the eyes (Wu).

[0085] To measure pigmentation information next to and below the eyes, the measurement unit (321) can extract the area next to and below the eyes, respectively, from an image of the eye area. Afterward, the extracted image is converted to grayscale, and then a region darker than the surrounding skin color can be selected through a local thresholding algorithm. Afterward, areas below the threshold can be excluded to exclude spots or hair. Afterward, the pigmentation is scored according to a preset standard (range of 0 to 100, with a higher value indicating greater aging) and can be stored as pigmentation next to the eyes (Pb) and pigmentation below the eyes (Pu), respectively.

[0086] The wrinkle extraction unit (322) measures the inner upper corner wrinkle information (Wa) from the eye area image. The inner upper corner position is a pentagon shown in FIG. 3 (b), and each point uses the coordinates of feature points 28, 43, 44, 23, and 24 relative to the right eye, and points 23 and 24 are calculated as coordinates moved downward by 1 / 3 of the y-axis height of the pentagon formed by the five points in order to move them below the eyebrow position. The left area is calculated in the same way as the right eye using the coordinates of five symmetrical points 28, 40, 39, 21, and 22.

[0087] The elasticity calculation unit (323) calculates elasticity information from the eye area image. The elasticity information may include eye sagging information and margin information.

[0088] Eyelid sagging information (Md) can be calculated using the following [Equation 1].

[0089]

[0090] Here, Md is the eyelid ptosis information, H1 is the height of the medial canthus (y-coordinates 40 and 43 in Fig. 7), and H2 is the height of the lateral canthus (y-coordinates 37 and 46 in Fig. 7). That is, the eyelid ptosis information (Md) represents the difference between the height of the medial canthus and the height of the lateral canthus.

[0091] Specifically, the height of the inner canthus (H1) is calculated as the difference between the y-coordinate of feature point 40 and the y-coordinate of feature point 43 included in the eye corner image, and the height of the outer canthus (H2) is calculated as the difference between the y-coordinate of feature point 37 and the y-coordinate of feature point 46. Next, the elasticity calculation unit (323) can store the difference between the height of the inner canthus (H1) and the height of the outer canthus (H2) as eye sagging information (Md). It can be replaced with the average of the eye sagging information calculated in the same way for the opposite eye.

[0092] Margin information (Mw) can be calculated using the following [Equation 2].

[0093]

[0094] Here, Mw is margin information, L2 is the horizontal distance (absolute value of the difference in x-coordinates) from the outer corner of the right eye (point 46 in Fig. 7) to the right edge of the face (center point of points 17 and 16 in Fig. 7), L3 is the horizontal distance (absolute value of the difference in x-coordinates) from the outer corner of the left eye (point 37 in Fig. 7) to the left edge of the face (center point of points 1 and 2 in Fig. 7), and L1 is the horizontal distance from the left edge of the face (center point of points 1 and 2 in Fig. 7) to the right edge of the face (center point of points 17 and 16 in Fig. 7) (see Fig. 3 (c)).

[0095] Specifically, the horizontal distance (L2) from the outer corner of the right eye to the right edge of the face is the absolute value of the difference in x-coordinates from feature point 46 to the center point of feature points 17 and 16, and the horizontal distance (L3) from the outer corner of the left eye to the left edge of the face is calculated as the absolute value of the difference in x-coordinates from feature point 37 to the center point of points 1 and 2. In addition, the horizontal distance (L1) from the left edge of the face to the right edge is calculated as the horizontal distance from the center point of feature points 1 and 2 to the center point of feature points 17 and 16.

[0096]

[0097] Referring to Figure 4, it can be seen that wrinkle information (Wb, Wu, Wa), pigmentation information (Pb, Pu), and elasticity information (Md, Mw) have a high correlation with actual age.

[0098]

[0099] The eye age calculation unit (330) is configured to calculate the eye age by inputting wrinkle information, pigmentation information, and elasticity information calculated by the calculation unit (320) into a linear regression model, and includes a model generation unit (331) and a model determination unit (332).

[0100] The model generation unit (331) generates a linear regression model for calculating eye age. The linear regression model can calculate eye age using [Equation 4] by using a model learned using actual age through [Equation 3] below.

[0101]

[0102]

[0103] Here, y represents actual age, β0 represents the intercept, ε represents the error, and x i represents the i-th independent variable, and β i represents the coefficient of the i-th independent variable and the estimated The value is used as eye age. The independent variables are lateral eye wrinkle information (Wb), lower eyelid wrinkle information (Wu), upper inner corner eye wrinkle information (Wa), lateral eye pigmentation information (Pb), lower eyelid pigmentation information (Pu), eyelid sagging information (Md), and margin information (Mw).

[0104] The model generation unit (331) can generate multiple linear regression models by learning at least one of the previously stored wrinkle information, pigmentation information, and elasticity information, or a combination thereof, and the actual age information stored in the database (a). For example, referring to FIG. 2, the training data combinations can be configured in nine ways. Each combination may consist only of wrinkle information, pigmentation information, and elasticity information (combinations 1, 2, and 3), a combination of two of these types of information (combinations 4, 5, and 6), all types of information (combination 7), only wrinkle information next to the eyes (Wb) and wrinkle information under the eyes (Wu) among the wrinkle information (combination 8), or wrinkle information next to the eyes (Wb), wrinkle information under the eyes (Wu), and pigmentation information (Pb, Pu) (combination 9). Each of these is trained together with the actual age information received from the database (a) to generate multiple linear regression models. For example, the first model is generated using combination 1, and the second model is generated using combination 2. This process is repeated to generate a total of nine linear regression models. That is, for each model, the intercept (β0), error (ε), and coefficients of the independent variable (β) in Equation 3 i Can perform operations on ).

[0105]

[0106] The model determination unit (332) determines a linear regression model that matches the type entered into the input unit (120) among a plurality of linear regression models. For example, if wrinkles and pigmentation are entered as the types entered into the input unit (120), the model determination unit (332) may select the first model, the second model, the fourth model, the eighth model, and the ninth model. The model determination unit (332) may determine one of the first model, the second model, the fourth model, the eighth model, or the ninth model that has good performance (judging that performance is higher the lower the mean squared error value, or judging that performance is higher the higher the correlation coefficient with actual age), or one of the above models may be determined by the user's selection. Next, the information calculated by the calculation unit (320) (at least one of Wb, Wu, Wa, Pb, Pu, Md, Mw) can be input into the model determined by the model determination unit (332) to calculate the eye age. That is, since the intercept of Equation 3 and the coefficient of the independent variable are calculated by the model generation unit (331), the information calculated by the calculation unit (320) can be input as an independent variable to calculate the eye age.

[0107] By determining a linear regression model that matches the type entered into the input unit (120) by the model determination unit (332), a model suitable for the product used by the user is selected to calculate the age of the eyes, and based on this, the effect of using the product can be determined. For example, if the product used by the user is a cosmetic for improving wrinkles around the eyes, pigmentation and elasticity are not factors to consider when determining the effect of using the product. Therefore, when the user inputs wrinkles into the input unit (120), the model determination unit (332) determines a first model that has learned only wrinkle information and calculates the age of the eyes. If the age of the eyes is calculated to be lower than before using the product, it can be determined that the product has an effect of improving wrinkles around the eyes.

[0108]

[0109] The communication unit (340) transmits the eye age calculated by the eye age calculation unit (330) to the display unit (140).

[0110]

[0111] FIG. 5 is a flowchart of a method for calculating eye age according to one embodiment of the present invention, and FIG. 6 is a flowchart of a method for calculating eye age according to another embodiment of the present invention.

[0112] Below, a method for calculating eye age is explained with reference to FIGS. 5 and FIGS. 6.

[0113] First, the image acquisition unit (110) captures the face of the subject to evaluation and acquires a face image. The face image must include the eye area. The image acquisition unit (110) transmits the face image to the eye area image extraction unit (310) through the communication unit (130).

[0114] In another embodiment, the skin analysis unit (200) can photograph the face of the subject being evaluated and directly measure wrinkle information (Wb, Wu) and pigmentation information (Pb, Pu) next to and below the eyes of the subject being evaluated. This has the difference that the information (Wb, Wu, Pb, Pu) can be measured immediately without the measurement unit (321) described later. Meanwhile, since the skin analysis unit (200) acquires face image data, the acquired face image data can be transmitted to the eye area image extraction unit (310).

[0115] Next, the eye area image extraction unit (310) extracts an eye area image from the face image (S110). Face feature points are extracted through a data mining tool (dlib), and an eye area image is extracted through eye area and eyebrow feature point information.

[0116] Next, the calculation unit (320) calculates wrinkle information, pigmentation information, and elasticity information from the eye area image (S120).

[0117] Specifically, the measuring unit (321) calculates the wrinkle information next to the eye (Wb), the wrinkle information under the eye (Wu), the pigmentation information next to the eye (Pb), and the pigmentation information under the eye (Pu).

[0118] To compute information on wrinkles next to and below the eyes (Wb, Wu), the eye area image is converted to grayscale. Next, wrinkles next to and below the eyes are extracted from the converted image using a hybrid Hessian filter. Then, the extracted wrinkles next to and below the eyes are enhanced and noise is removed using the Fast Fourier Transform. Based on the enhanced wrinkle information, information on wrinkles next to the eyes (Wb) and wrinkles below the eyes (Wu) is computed.

[0119] To calculate pigmentation information (Pb, Pu) for the side and under the eyes, the measurement unit (321) converts the eye area image into grayscale and extracts the areas next to and under the eyes. Next, an area darker than the surrounding skin color is selected using a local thresholding algorithm. Next, to exclude spots or hair, areas below a preset threshold are excluded. Next, the measurement unit (321) calculates pigmentation information (Pu) for the side of the eyes and pigmentation information (Pu) for the under-eyes by scoring the remaining area excluding the excluded area from the extracted image according to a preset pigmentation judgment criterion.

[0120]

[0121] The wrinkle extraction unit (322) extracts wrinkle information (Wa) on the inner upper corner of the eye. Specifically, the inner eye region image is converted to grayscale. Next, the inner upper corner wrinkles are extracted from the converted image through a hybrid hessian filter. Next, the extracted inner upper corner wrinkles are emphasized and noise is removed through a Fast Fourier Transform. Based on the emphasized wrinkle information, the inner upper corner wrinkle information (Wa) is calculated.

[0122] As described above, the elasticity calculation unit (323) calculates eye sagging information (Md) and margin information (Mw). The eye sagging information (Md) is calculated using the above mathematical formula 1. The margin information (Mw) is calculated through the above mathematical formula 2.

[0123] Next, the eye age calculation unit (330) calculates the eye age by inputting at least one of the wrinkle information, pigmentation information, and elasticity information calculated by the calculation unit (320) into a linear regression model (S130). Specifically, the eye age calculation unit (330) can determine at least one of the following information to be input into the linear regression model: eye side wrinkle information (Wb), eye under-eye wrinkle information (Wu), eye side pigmentation information (Pb), eye under-eye pigmentation information (Pu), eye sagging information (Md), and margin information (Mw).

[0124] Next, the eye age calculation unit (330) can calculate the eye age by inputting at least one of the wrinkle information, pigmentation information, and elasticity information calculated by the calculation unit (320) into a determined linear regression model (S140).

[0125]

[0126] As described above, information was input into a linear regression model to calculate eye age; below, a method for determining the linear regression model to which the above input information will be input is described.

[0127] First, at least one type among wrinkles, pigmentation, and elasticity is input into the input section (120) of the user terminal. For example, only one of wrinkles, pigmentation, and elasticity may be input into the input section (120), two types such as wrinkles + pigmentation, wrinkles + elasticity may be input, or all types such as wrinkles + pigmentation + elasticity may be input.

[0128] Afterwards, the model determination unit (332) determines a linear regression model that matches the type input to the input unit (120) among the multiple linear regression models that have already been generated.

[0129] As described above, a plurality of linear regression models can be generated for each combination by the model generation unit (331) learning at least one or a combination of pre-stored wrinkle information (Wb, Wu, Wa), pigmentation information (Pb, Pu), and elasticity information (Md, Mw) and actual age information stored in the database (a) (see FIG. 2).

[0130] The model determination unit (332) determines a linear regression model that matches the input type among a plurality of models. For example, if the input type is wrinkles and pigmentation, the model determination unit (332) selects a first model, a second model, a fourth model, an eighth model, and a ninth model, each of which has at least one of wrinkle information and pigmentation information as an input variable. The model determination unit (332) may determine any one of the above models with good performance as the target model. In another embodiment, the user may directly select any one of the above models.

[0131]

[0132] Next, the eye age can be calculated by inputting at least one of the wrinkle information, pigmentation information, and elasticity information calculated by the calculation unit (320) into the linear regression model determined by the model determination unit (332).

[0133]

[0134] Next, the calculated eye age can be transmitted to a user terminal (100) and displayed through a display unit (140).

[0135]

[0136] With reference to FIGS. 8 and FIGS. 9, the results of the verification experiment of the present invention will be explained.

[0137] Referring to FIG. 8, for a verification experiment, the inventors generated a total of nine linear regression models for predicting actual age based on the combination of wrinkle information (Wa, Wb, Wu), pigmentation information (Pb, Pu), and elasticity information (Md, Mw) and the above Equation 3. Models 1 to 3 in FIG. 8 were generated using one of the information on wrinkles, pigmentation, and elasticity as an independent variable, models 4 to 6 were generated using two of these information as independent variables, model 7 was generated using all of the information as independent variables, and models 8 and 9 were generated by removing the wrinkle information (Wa) on the inner upper eye from models 1 and 5, respectively. Therefore, models 3, 8, and 9 were generated based solely on indicators obtained from the Janus 3 system.

[0138] Training data for model generation was constructed by collecting frontal face images from 2,000 subjects and measuring wrinkle information, pigment spot information, and elasticity information from them, and was evaluated using 10-fold cross-validation. The results were presented as correlation coefficients (see 'Performance' in Fig. 8). Most models, excluding models 2 and 8, showed high performance of 0.7 or higher. Through this, it was confirmed that the linear regression model according to the present invention has excellent prediction accuracy with respect to actual age.

[0139]

[0140] In addition, referring to Fig. 9, the inventors demonstrated the association between the age around the eyes calculated through the system and method according to the present invention and the health status of the subjects being evaluated. The inventors surveyed a total of 515 subjects, consisting of 103 individuals in each age group from their 20s to their 60s, to determine whether they had a history of past diseases. Past diseases were defined as heart disease, hypertension, hyperlipidemia, fatty liver, kidney disease, diabetes, and cancer. Furthermore, the inventors demonstrated the association between the age around the eyes calculated through the present invention and the presence or absence of past diseases in the subjects through the Beta-coefficient. Here, a positive Beta-coefficient indicates that the higher the age around the eyes, the higher the probability of having experienced the corresponding disease. Referring to Fig. 9, the Beta-coefficients for heart disease, hypertension, hyperlipidemia, fatty liver, heart disease, diabetes, and cancer were positive, confirming that there is an association between the age around the eyes and a history of past diseases.

[0141]

[0142] For the time being, the present specification has been described with reference to embodiments illustrated in the drawings so that those skilled in the art can easily understand and reproduce the present invention; however, this is merely illustrative, and those skilled in the art will understand that various modifications and equivalent alternative embodiments are possible from the embodiments of the present invention. Accordingly, the scope of protection of the present invention should be determined by the claims.

[0143]

[0144] (Explanation of symbols)

[0145] 1 : Eye Age Calculation System

[0146] 100 : User terminal

[0147] 110 : Image acquisition unit

[0148] 120 : Input section

[0149] 130 : Communications Department

[0150] 140 : Display section

[0151] 200 : Skin Analysis Department

[0152] 300 : Server

[0153] 310 : Eye area image extraction unit

[0154] 320 : Operation unit

[0155] 321 : Measurement section

[0156] 322 : Wrinkle extraction part

[0157] 323 : Elasticity Calculation Unit

[0158] 330 : Eye age calculation unit

[0159] 331 : Model Creation Section

[0160] 332 : Model Determination Unit

[0161] 340 : Communications Department

Claims

1. (a) A step in which an eye area image extraction unit (310) extracts an eye area image from a face image including the eye area of ​​a subject to evaluation; (b) a step in which the calculation unit (320) calculates wrinkle information, pigmentation information, and elasticity information from the eye area image in a preset manner; and (c) a step in which an eye age calculation unit (330) inputs at least one of the wrinkle information, pigmentation information, and elasticity information into a pre-generated linear regression model to calculate the eye age; comprising, Method for calculating eye age.

2. In Paragraph 1, The above wrinkle information includes at least one of the following: wrinkle information next to the eye (Wb), wrinkle information below the eye (Wu), and wrinkle information at the inner upper corner of the eye (Wa). Method for calculating eye age.

3. In Paragraph 2, The above pigmentation information includes at least one of the pigmentation information next to the eye (Pb) and pigmentation information under the eye (Pu). Method for calculating eye age.

4. In Paragraph 3, The above elasticity information includes at least one of the eye sagging information (Md) and margin information (Mw). Method for calculating eye age.

5. In Paragraph 4, The above eyelid sagging information is calculated through the following mathematical formula 1, [Mathematical Formula 1] Md = H1 - H2 Here, Md is eyelid ptosis information, H1 is the height of the inner corner of the eye, and H2 is the height of the outer corner of the eye, Method for calculating eye age.

6. In Paragraph 4, The above margin information is calculated through the following mathematical formula 2, [Mathematical Formula 2] Mw = (L2+ L3) / L1 Here, Mw is margin information, L2 is the horizontal distance from the outer corner of the right eye to the right edge of the face, L3 is the horizontal distance from the outer corner of the left eye to the left edge of the face, and L1 is the horizontal distance from the left edge of the face to the right edge of the face, Method for calculating eye age.

7. In Paragraph 4, The above step (b) is, (b1) A step in which a measuring unit (321) extracts wrinkles next to the eye, wrinkles under the eye, pigmentation next to the eye and pigmentation under the eye from the eye area image and calculates wrinkle information next to the eye (Wb), wrinkle information under the eye (Wu), pigmentation information next to the eye (Pb) and pigmentation information under the eye (Pu); (b2) A wrinkle extraction unit (322) extracts an inner upper eye wrinkle from the eye area image and calculates inner upper eye wrinkle information (Wa); and (b3) A step in which the elasticity calculation unit (323) calculates eye sagging information (Md) and margin information (Mw) from the eye area image; comprising, Method for calculating eye age.

8. In Paragraph 7, The method by which the measuring unit (321) of step (b1) above calculates the eye side wrinkle information (Wb) and the eye under wrinkle information (Wu) from the eye area image is: (b11) A step in which the measuring unit (321) extracts an image of wrinkles next to the eye and an image of wrinkles under the eye from the eye area image; (b12) A step in which the measuring unit (321) converts the image of wrinkles next to the eye and the image of wrinkles under the eye into grayscale; (b13) A step in which the measurement unit (321) extracts wrinkles next to the eyes and wrinkles under the eyes from the grayscale image converted through a hybrid Hessian filter; (b14) A step in which the measurement unit (321) highlights the extracted wrinkles next to and under the eyes and removes noise through a Fast Fourier Transform; and (b15) A step in which the measuring unit (321) calculates eye side wrinkle information (Wb) and eye lower wrinkle information (Wu) based on the emphasized eye side wrinkle and eye lower wrinkle; comprising Method for calculating eye age.

9. In Paragraph 7, The method by which the measuring unit (321) of step (b1) above calculates the pigmentation information next to the eye (Pb) and the pigmentation information under the eye (Pu) from the eye area image is: (b16) A step in which the measuring unit (321) extracts an image next to the eye and an image below the eye from the eye area image; (b17) A step in which the measuring unit (321) converts the image next to the eye and the image below the eye into grayscale; (b18) A step in which the measurement unit (321) selects an area darker than the surrounding skin color in the extracted image through a local thresholding algorithm; (b19) A step in which the measurement unit (321) excludes an area among the selected areas that is below a preset threshold value; and (b20) A step in which the measurement unit (321) calculates pigmentation information next to the eye (Pu) and pigmentation information under the eye (Pu) by scoring the remaining area excluding the area excluded in step (b19) from the image extracted in step (b18) according to a preset standard; Method for calculating eye age.

10. In Paragraph 7, The above (b2) step is, (b21) A step in which the wrinkle extraction unit (322) extracts an image of the inner upper corner of the eye from the eye area image; (b22) A step in which the wrinkle extraction unit (322) converts the upper inner corner image of the eye into grayscale; (b23) A step in which the wrinkle extraction unit (322) extracts the inner upper corner wrinkles from an image converted to grayscale through a hybrid Hessian filter; (b24) A step in which the wrinkle extraction unit (322) emphasizes the extracted inner upper corner wrinkles of the eye and removes noise through a Fast Fourier Transform; and (b25) A step in which the wrinkle extraction unit (322) calculates inner upper eye wrinkle information (Wa) based on the emphasized inner upper eye corner wrinkle; comprising, Method for calculating eye age.

11. In Paragraph 7, The above (b3) step is, (b31) A step in which the elasticity calculation unit (323) calculates the eye sagging information (Md) using the difference in y-coordinates between the inner eye feature point and the outer eye feature point among the face feature points; and (b32) A step in which the elasticity calculation unit (323) calculates margin information (Mw) by dividing the sum of the left and right distances to the face contour line that is horizontal to the outer eye feature point by the width of the entire face up to the face contour line that is horizontal to the eye; Method for calculating eye age.

12. In Paragraph 1, The linear regression model generated in step (c) above is a plurality of linear regression models generated by the model generation unit (331) learning at least one or a combination of pre-stored wrinkle information, pigmentation information, and elasticity information, and pre-stored actual age information received from a database. Method for calculating eye age.

13. In Paragraph 12, The above step (c) is, (c1) A step in which at least one type of wrinkles, pigmentation, and elasticity is input into the input section (120); (c2) A step in which the model determination unit (332) determines a linear regression model that matches the type input to the input unit (120) among the plurality of linear regression models that have already been generated; and (c3) A step in which an eye age calculation unit (330) calculates the eye age by inputting at least one of the wrinkle information, pigmentation information, and elasticity information calculated by the calculation unit (320) of step (b) into a linear regression model determined in step (c2); comprising Method for calculating eye age.

14. In Paragraph 1, The eye area image of step (a) above is extracted by extracting facial feature points and using eye area and eyebrow feature point information. Method for calculating eye age.

15. A computer-readable recording medium having a program recorded thereon for performing a method according to any one of paragraphs 1 through 14 on a computer.

16. A computer program stored on a computer-readable storage medium to execute a method according to any one of paragraphs 1 through 14.

Images