Device and method for determining cosmetic skin attributes

Abstract

The present invention discloses a method for determining a cosmetic skin property of a person. The method comprises the following steps: a) obtaining at least one color channel image including at least a portion of the person's skin; b) analyzing at least one color channel image using entropy statistics to obtain an entropy value; and c) determining a cosmetic skin property of at least a portion of the person's skin based on the entropy value.

Description

Technical Field

[0001] The present invention relates to an apparatus and method for determining cosmetic skin attributes. Background Art

[0002] Skin imaging methods have been widely used to study different phenotypes of skin aging. Numerous image analysis techniques and algorithms have been proposed in the literature to characterize aging skin, with particular attention to aging phenotypes such as wrinkles, spots and sagging. The appearance of phenotypes associated with skin aging is known to be a continuous process over time. For example, uneven pigmentation may first lead to visually imperceptible spots, which may eventually become visible over time. Therefore, younger consumers (e.g., less than 30 years old) generally do not have classic visible skin aging-related phenotypes, and therefore have the impression that there is no urgent need to prevent, delay and / or alleviate such visually imperceptible aging phenotypes, and it is too late when there is such a need.

[0003] U.S. Publication No. 2010 / 0284610A1 ("the '610 Publication") relates to a skin color evaluation method for evaluating skin color from an input image including a face region. The '610 Publication describes dividing the face region of an image into predetermined regions based on first feature points formed by at least 25 regions set in advance and second feature points set by using the first feature points. The '610 Publication also describes performing skin color distribution evaluation by: using L*, a*, b*, C*, and L*a*b* color system to calculate the skin color distribution. ab * and h ab , tristimulus values ​​X, Y, Z of the XYZ color system and at least one of RGB, hue H, lightness V, chroma C, melanin amount, and hemoglobin amount are used to generate skin color distribution based on an average value, and then an evaluation is performed based on the measured results with respect to the divided areas, and the measured results or the evaluation results are displayed on a screen. However, the '610 publication focuses on visible skin color distribution, and it cannot describe skin aging-related phenotypes, and therefore does not describe any method for evaluating skin aging-related phenotypes that are not visually perceptible.

[0004] Therefore, there is a need for a method for determining a person's cosmetic skin attributes so that proactive skin care treatments can be implemented at an early stage. Summary of the invention

[0005] The present invention relates to a method for determining the cosmetic skin attributes of a person, the method comprising the following steps:

[0006] a) obtaining at least one color channel image including at least a portion of the person's skin;

[0007] b) analyzing the at least one color channel image using entropy statistics to obtain an entropy value; and

[0008] c) determining the cosmetic skin property of the at least a portion of the skin of the person based on the entropy value. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 is a diagram illustrating an exemplary system for determining cosmetic skin attributes via a network in accordance with the present invention;

[0010] Figure 2 is a diagram showing an alternative exemplary system for determining cosmetic skin attributes in accordance with the present invention;

[0011] Figure 3 is a block diagram illustrating components of an exemplary apparatus for determining cosmetic skin attributes according to the present invention;

[0012] Figure 4 is a block diagram illustrating a method for determining cosmetic skin attributes according to the present invention;

[0013] Figure 5 is a flow chart illustrating a method for determining a method for determining cosmetic skin attributes according to the present invention;

[0014] FIG. 6A to FIG. 6C is a series of process flow charts illustrating a method for determining cosmetic skin attributes according to the present invention;

[0015] Figure 7 is a flow chart illustrating a method for determining cosmetic skin attributes according to the present invention.

[0016] Fig. 8A and Figure 8B is a process flow diagram illustrating a method of visualizing an entropy value of at least one cosmetic skin property of at least a portion of a face of a subject in a digital image in accordance with the present invention;

[0017] Fig. 9 is a flow chart illustrating a method of visualizing an entropy value of at least one cosmetic skin property of at least a portion of a face of a subject in a digital image according to the present invention;

[0018] FIG. 10A to FIG. 10C is a series of process flow charts showing details of the steps of obtaining a first digital image in the method of determining cosmetic skin attributes according to the present invention;

[0019] Fig.11 is a flow chart showing the steps of obtaining a first digital image;

[0020] Fig.12 is a diagram showing a step of defining a plurality of tiles in a method for determining a cosmetic skin attribute according to the present invention;

[0021] Fig.13 is a flow chart showing the steps of defining a plurality of tiles;

[0022] FIG. 14A to FIG. 14C is a process flow diagram illustrating a process of displaying a plurality of tiles according to the present invention;

[0023] Fig.15 is a flow chart illustrating a process of displaying a plurality of tiles according to the present invention;

[0024] FIG. 16A to FIG. 16D is a process flow chart illustrating a method of visualizing an entropy value of at least one cosmetic skin attribute according to the present invention;

[0025] Fig.17 is a flow chart illustrating a method of visualizing an entropy value of at least one cosmetic skin attribute according to the present invention;

[0026] Fig.18 is a flow chart illustrating an alternative method of visualizing an entropy value of at least one cosmetic skin property according to the present invention;

[0027] FIG. 19A to FIG. 19E are screenshots each showing an exemplary user interface for visualizing an entropy value of at least one cosmetic skin property according to the present invention;

[0028] Fig. 20 is a screenshot showing an exemplary user interface including a heat map as an example of an image description for visualizing at least one cosmetic skin attribute according to the present invention;

[0029] Fig.21 is a screenshot showing an alternative variation of a graphical depiction for visualizing at least one cosmetic skin attribute in the user interface of FIG. 19 ; and

[0030] Fig. 22 is a screenshot illustrating an exemplary user interface for visualizing at least one cosmetic skin attribute in accordance with the present invention. DETAILED DESCRIPTION

[0031] It is well known that when the skin is subjected to stress (caused by UV factors, aging factors, mental factors, environmental factors), the skin will be damaged at various levels, including DNA level, cellular level and tissue level. Such damage to the skin can lead to skin defects. The presence of these skin defects significantly affects the optical properties of the skin, such as described in the following examples:

[0032] If the stratum corneum is dry (winter / air conditions), light will be reflected more from the surface (surface reflection), thus enhancing the skin microtexture, which leads to lines on the skin

[0033] If the dermis is damaged (UV), less light will be scattered at the dermis (also called dermal scattering), and light will penetrate the skin (less subsurface reflection). When there is less dermal scattering, and the skin therefore appears darker.

[0034] If the skin is exposed to long-term UV, the skin produces more melanin. Melanin absorbs light, reducing subsurface reflections, and therefore the skin appears darker.

[0035] The above skin imperfections appear as signals of poor skin quality that are not visually perceptible to the consumer's eye. The consumer may perceive these imperfections as impurities on a subconscious level, but due to lack of conscious knowledge, cannot take action to improve the imperfections. If the skin is subjected to chronic stress and / or is left untreated, these visually imperceptible imperfections can eventually lead to visible and perceptible phenotypes (pigmented spots, wrinkles, sagging).

[0036] The present invention relates to a method, device and system for determining at least one cosmetic skin attribute of a subject, and a graphical user interface. The method comprises the following steps: (a) obtaining at least one color channel image including at least a portion of a person's skin; (b) analyzing the at least one color channel image using entropy statistics to obtain an entropy value; and (c) determining the cosmetic skin attribute of the at least a portion of the person's skin based on the entropy value.

[0037] It has surprisingly been found that entropy values ​​obtained by analyzing a color channel image comprising at least a portion of a person's skin can be used to determine visually perceptible cosmetic skin properties as well as visually non-perceptible cosmetic skin properties.

[0038] As explained below, if a person's cosmetic skin attribute can be determined based on an entropy value obtained by analyzing a color channel image including at least a portion of the person's skin using entropy statistical data, useful information (e.g., the condition of the cosmetic skin attribute) can be inferred so that the person can seek proactive skin care treatment to improve the condition of the cosmetic skin attribute. Specifically, as described below, a color channel image having at least a portion of the person's skin can have an area of ​​interest with a lower entropy value on the at least a portion of the skin relative to other areas of interest. The lower entropy value in the area of ​​interest shows less intensity variation, which indicates a better cosmetic skin attribute condition. Correspondingly, the higher entropy value in other areas of interest shows more intensity variation, which indicates a poorer cosmetic skin attribute condition. Another advantage is that proactive skin care treatment can be targeted to a specific area of ​​interest.

[0039] Before describing the present invention in detail, the following terms are defined, and undefined terms should be given their ordinary meanings as understood by those skilled in the relevant art.

[0040] As used herein, "entropy" refers to the Shannon entropy (E) of a discrete random distribution (p(x)) and is defined by the following equation:

[0041]

[0042] Where p(x) is the distribution of gray levels

[0043] E(p) represents the amount of information in the digital image or the amount of information in the color channel image after the digital image is converted into a color channel image in a color system. The "entropy statistics" used herein refers to a statistical method that uses entropy as a descriptive statistic for analyzing a digital image or a color channel image. In a non-limiting example where the digital image is an RGB image, the entropy (entropy value) of each R (red) channel, G (green) channel, and B (blue) channel can be calculated separately. The entropy value of the image can be calculated by calculating the entropy value of the pixel value in a 2-dimensional area centered at (i, j) at each pixel position (i, j). The 2-dimensional area can be part of the color channel image. A programming software package such as Python can be used to calculate the entropy value.

[0044] As used herein, "cosmetic skin attributes" include all skin attributes that provide a visual / aesthetic effect on an area of ​​the human body or affect the appearance and / or feel of the skin. Some non-limiting examples of cosmetic skin attributes may include skin purity, skin age, skin topography, skin tone, skin pigmentation, skin pores, skin inflammation, skin hydration, skin sebum levels, acne, moles, skin luminosity, skin radiance, skin dullness, uneven tone, or skin barrier. The skilled person will appreciate that the above cosmetic skin attributes are standard terms and that corresponding definitions of cosmetic skin attributes may be found in the following published references, i.e., "Handbook of cosmetic science and technology, 3rd edition, edited by Andre O. Barel, Marc Paye, Howard I. Maiback, CRC Press, 2009 (Handbook of cosmetic science and technology, 3rd edition, edited by Andre O. Barel, Marc Paye, Howard I. Maiback, CRC Press, 2009). rded., editors Andre O. Barel, Marc Paye, Howard I. Maiback, CRC Press, 2009), "Cosmetic Science and Technology-Theoretical Principles and Applications, editors Kazutami Sakamoto Robert Y. Lochhead, Howard I. Maibach, Yuji Yamashita, Elsavier, 2017", "Cosmetic Dermatology: Products and Procedures, Editor(s): Zoe Diana Draelos, Blackwell Publishing Ltd, 2010". Cosmetic skin attributes do not include skin attributes related to medical conditions or underlying medical conditions.

[0045] As used herein, "imperceptible cosmetic skin attributes" refer to cosmetic skin attributes that cannot be perceived by or are not perceptible to a perceiver, i.e., a person, user, or human subject. Perception is derived from the word "Perception", which refers to the organization, identification, and interpretation of sensory information in order to represent and understand the presented information or environment. All perceptions involve signals passing through the nervous system, which in turn are caused by physical or chemical stimulation of the sensory system. For example, vision involves light shining on the retina of the eye, smell is mediated by odor molecules, and hearing involves pressure waves. Perception is not only the passive reception of these signals, but is also shaped by the learning, memory, expectations, and attention of the recipient. Perception can be divided into two processes, namely, process (1), which involves processing sensory inputs, which transforms these low-level information into higher-level information (e.g., extracting shapes for object recognition), and process (2), which involves processing associated with human concepts and expectations (or knowledge), restorative and selective mechanisms that affect perception (such as attention). For example, a perceiver may see an object in process (1), but not have the knowledge to perceive and recognize what the object represents / means in process (2), and may therefore consider the object to be visually imperceptible.

[0046] As used herein, "visually imperceptible cosmetic skin attributes" include all cosmetic skin attributes that cannot be detected by the unassisted eye or cosmetic skin attributes that can be visually detected by a consumer but the consumer cannot understand the cosmetic skin attribute and are therefore considered to be imperceptible cosmetic skin attributes. Some non-limiting examples of visually imperceptible cosmetic skin attributes that cannot be visually detected by the unassisted eye include cosmetic skin inflammation, skin sebum levels, or any basic cosmetic skin attribute.

[0047] As used herein, "unassisted" means without assistance from diagnostic equipment.

[0048] As used herein, a "tile" includes a unit, such as a pixel, that forms part of a digital image, and thus a "tile" forms the entire digital image.

[0049] As used herein, "digital image data" includes image data obtained from an image acquisition device, including (but not limited to) a digital camera, a photo scanner, a computer-readable storage medium capable of storing digital images, and any electronic device including picture-taking capabilities. Digital image data may also include a color channel image converted from an RGB image to a color channel image in a color system.

[0050] As used herein, "unidimensional indicia" includes all electronic visual representations, including (but not limited to) graphic symbols, numerical values, color codes, lighting techniques, and combinations thereof.

[0051] As used herein, "skin attribute index" refers to a score that can be calculated based on a mathematical formula or a model or lookup table (information array) derived from statistical methods and data. The skin attribute index can be generated as a probability value indicating the condition of a cosmetic skin attribute relative to at least a portion of the skin of a person of a defined population, preferably, the skin attribute index is generated as a function of an entropy value defined by F (entropy value), wherein the function is determined by a model established on a training data set, wherein the training data set includes: (i) a plurality of color channel images of a defined population, wherein each of the plurality of color channel images includes facial skin of a person in the defined population, wherein the facial skin includes cosmetic skin attributes; (ii) an associated category definition based on cosmetic skin attributes.

[0052] As used herein, "L*a*b*" refers to a universally recognized color space specified by the International Commission on Illumination ("CIE"). The three coordinates represent (i) the lightness of a color (i.e., L*=0 produces black and L*=100 indicates diffuse white); (ii) the position of a color between magenta and green (i.e., negative a* values ​​indicate green, while positive a* values ​​indicate magenta); and (iii) the position of a color between yellow and blue (i.e., negative b* values ​​indicate blue and positive b* values ​​indicate yellow).

[0053] As used herein, "chromophore mapping" refers to a generally recognized chromophore space used for melanin and hemoglobin mapping and determining melanin or hemoglobin concentrations, which can be used as an indicator of overall skin color. The average melanin or hemoglobin can be calculated from the chromophore mapping data. In addition, the uniformity of skin tone can be determined by the melanin or hemoglobin uniformity (e.g., standard deviation), which can also be calculated from the chromophore mapping data.

[0054] As used herein, "skin purity" refers to the appearance of the absence of skin imperfections in at least a portion of a person's skin. The skin imperfections include cosmetic skin attributes that affect irregular or non-uniform spectral properties consisting of surface reflectance of skin topography and / or sub-surface reflectance of skin chromophores such as melanin, hemoglobin and / or keratinocyte and fibroblast-directed cellular metabolites, and include (but are not limited to) skin luminescence, skin tone, and the like.

[0055] "Skin age" as used herein refers to apparent age, which refers to the age of a person's skin visually estimated or perceived based on physical appearance, preferably based on the person's face, preferably based on at least a portion of the person's face, more preferably based on at least one region of interest (ROI) of at least a portion of the person's face compared to a standard age skin appearance, more preferably, the at least one ROI is selected from the group consisting of: the skin area around the eyes ("eye area"), the skin area around the cheeks ("cheek area"), the skin area around the mouth ("oral area"), and combinations thereof.

[0056] As used herein, "skin tone" generally refers to the overall appearance of underlying skin tone or color uniformity. Skin tone is typically characterized over a large area of ​​skin. This area may be more than 100 mm2, but larger areas are contemplated, such as the entire facial skin or other human skin surfaces (e.g., arms, legs, back, hands, neck).

[0057] As used herein, "skin wrinkles" generally refer to folds, ridges or creases in the skin and include, but are not limited to, fine lines, ultrafine lines, fine lines, ultrafine lines, wrinkles, lines. Skin wrinkles can be measured, for example, in terms of density and / or length.

[0058] As used herein, "skin luminosity" generally refers to the amount of light reflected by the skin, and may be referred to as skin sheen.

[0059] As used herein, "skin texture" generally refers to the topography or roughness of the skin's surface.

[0060] As used herein, "skin tone" generally refers to the firmness or elasticity of the skin.

[0061] As used herein, "skin sebum levels" generally refers to the amount of sebum, which is an oily or waxy substance secreted by the sebaceous glands in the skin.

[0062] As used herein, "skin spots" generally refer to discoloration or uneven pigmentation of the skin (eg, hyperpigmentation, mottling). Skin spots can be assessed, for example, based on the density, size, and / or extent of the discoloration.

[0063] As used herein, "skin care product" refers to a product that contains skin care actives and regulates and / or improves the condition of the skin.

[0064] As used herein, "digital image" refers to a digital image formed by pixels in an imaging system (including but not limited to standard RGB, etc.) and images obtained under different lighting conditions and / or modes. Non-limiting examples of digital images include color images (RGB), monochrome images, videos, multispectral images, hyperspectral images, etc. Non-limiting light conditions include white light, blue light, UV light, IR light, light at a specific wavelength, such as a light source emitting light of 100nm to 1000nm, 300nm to 700nm, 400nm to 700nm, or different combinations of the above upper and lower limits or any integer combination of the above listed ranges. The digital image can be obtained from an image acquisition device, which includes but is not limited to a digital camera, a photo scanner, a computer-readable storage medium capable of storing a digital image, and any electronic device including a picture taking capability.

[0065] In the following description, the method described is a method for determining cosmetic skin attributes. Accordingly, the device described is a device for determining cosmetic skin attributes. The device may also be configured to generate entropy statistics of digital image data of at least a portion of a subject's face for display on a display, and the graphical user interface described is a graphical user interface for displaying entropy statistics of digital image data of at least a portion of the subject's face. The system described is an entropy-based system for determining cosmetic skin attributes. In an exemplary embodiment, the system is a stand-alone imaging system (at a retail cosmetics counter) for analyzing and recommending beauty and skin care products. Figure 2 However, it is expected that the system and method can be configured to be used anywhere through an electronic portable device including an image acquisition unit / device and a display, such as Figure 1 As shown in , the electronic portable device is connected to an apparatus for generating a graphical user interface for visualizing entropy values ​​of cosmetic skin attributes for display on a display via a network.

[0066] Figure 1 1 is a schematic diagram showing a system 10 for visualizing beauty skin attributes according to the present invention. The system 10 may include a network 100, which may be embodied as a wide area network (such as a mobile phone network, a public switched telephone network, a satellite network, the Internet, etc.), a local area network (such as wireless fidelity, Wi-Max, ZigBee, etc.), or a wireless network. TM 、Bluetooth TMThe portable electronic device 12 and the device 14 for generating a graphical user interface for visualizing the beauty skin attributes for display on a display are coupled to the network 100. The device 14 is remotely located and connected to the portable electronic device through the network 100. The portable electronic device 12 can be a mobile phone, a tablet computer, a laptop computer, a personal digital assistant and / or other computing devices configured to capture, store and / or transmit digital images such as digital photos. Therefore, the portable electronic device 12 may include an input device 12a for receiving user input, an image acquisition device 18 for obtaining an image, such as a digital camera, and an output device 12b for displaying the image. The portable electronic device 12 may also be configured to communicate with other computing devices via the network 100. The portable electronic device 12 may also include an image processing device (not shown), which is coupled to the image acquisition device 18 to analyze the obtained at least one color channel image using entropy statistics to obtain an entropy value, and determine the beauty skin attributes of at least a portion of the skin of the person based on the entropy value. The image processing device preferably includes a processor with computer executable instructions. The portable electronic device 12 may also include a display generating unit (not shown), such as an electronic LED / LCD display, for generating a display for displaying content data describing the determined cosmetic skin attributes.

[0067] The device 14 may include a non-transitory computer-readable storage medium 14a (hereinafter referred to as "storage medium"), which stores image acquisition logic 144a, image analysis logic 144a, and graphical user interface (hereinafter referred to as "GUI") logic 144c. The storage medium 14a may include random access memory (such as SRAM, DRAM, etc.), read-only memory (ROM), registers, and / or other forms of computing storage hardware. The image acquisition logic 144a, image analysis logic 144b, and GUI logic 144c define computer executable instructions. The processor 14b is coupled to the storage medium 14a, wherein the processor 14b is configured to implement the following with respect to Figure 4 The block diagram and Figure 5 A method 90 for determining a cosmetic skin attribute of a subject according to the present invention is described in a flowchart. The cosmetic skin attribute may be a cosmetic skin attribute that is not visually perceptible, wherein the cosmetic skin attribute that is not visually perceptible is a cosmetic skin attribute that cannot be detected by unassisted eyes, or a cosmetic skin attribute that can be visually detected by a consumer but the consumer cannot understand the cosmetic skin attribute. The advantage of determining a cosmetic skin attribute that is not visually perceptible is that it enables the consumer to make an informed decision and take proactive actions to improve the condition of the cosmetic skin attribute that is not visually perceptible.

[0068] Determine the method

[0069] refer to Figure 4 and Figure 5 When the processor 14b is activated, the processor 14b causes at least one color channel image 60L including at least a portion of the skin of a person to be obtained in step 91, for example by converting the digital image into a color channel image in a color system, which will be referred to below. Fig. 6A , Figure 6B , Figure 6C and Figure 7 The at least one color channel image 60L is analyzed using entropy statistics in step 92 to obtain an analysis output 80, wherein the analysis output 80 includes an entropy value. In step 93, a cosmetic skin property of at least a portion of the skin of the person is determined based on the entropy value. The at least one color channel image can be an image in a color system selected from the group consisting of: an L*a*b* color system, an RGB color system, an HSL / HSV color system, and a CMYK color system.

[0070] Table 1 below lists each entropy value, wherein the corresponding color channel image and the corresponding cosmetic skin attribute are to be determined based on the entropy value. The color channel image described in Table 1 is an image selected from the group consisting of the following items in the L*a*b* color system: an L channel image, an a channel image, a b channel image, a c channel image, and a combination thereof.

[0071] Table 1

[0072]

[0073] Determining the cosmetic skin attribute may include generating a skin attribute index as a probability value indicating the condition of the cosmetic skin attribute relative to at least a portion of the skin of a person of a defined population. Specifically, in a visual perception study, consumers may be asked to rank digital images (e.g., photos) of a defined population for cosmetic skin attributes based on a predetermined scale. The ranked digital images may be stored as a database for analysis according to method 90 to determine the entropy value with the highest correlation to the cosmetic skin attribute.

[0074] Alternatively, the skin attribute index may be generated according to an entropy value defined by a function F (entropy value), wherein the function is determined by a model established on a training data set. The training data set may include: (i) a plurality of color channel images of a defined population, wherein each of the plurality of color channel images includes facial skin of a person in the defined population, wherein the facial skin includes a cosmetic skin attribute; (ii) an associated category definition based on the cosmetic skin attribute. Techniques for constructing a training data set are known to those skilled in the art of image processing methods and will not be described further.

[0075] The model can be a regression model or a classification model, preferably a linear regression model, more preferably a machine learning linear regression model, and most preferably a machine learning support vector regression (SVR) model. The SVR model is a specific example of a support vector machine (SVM) model. The machine learning model can also be a support vector classification model or a random forest regression model.

[0076] The use of the SVR model achieves the following advantages: accuracy, reproducibility and speed of execution of the method when implemented as a native application on a portable electronic device. Specifically, the weights of the SVR model allow the native application to have a smaller hardware footprint, and thus the method according to the present invention can be easily deployed in a portable electronic device such as a mobile phone with a mobile phone operating system (OS), including (but not limited to) iOS for Apple™ phones or Android OS for Android phones.

[0077] The classification model may be used to classify consumers into a plurality of groups, each group having a different degree of condition of the same cosmetic skin attribute, preferably into two groups, more preferably into three groups, so as to define the associated category definition based on the numerical value of the skin attribute index. For example, the method may display a heat map configured to classify the skin into a high level of cosmetic skin attribute condition or a low level of cosmetic skin attribute condition based on a threshold value assigned to each of the groups.

[0078] At least one color channel image is an image in the L*a*b* color system selected from the group consisting of: an L color channel image, an a channel image, a b channel image, and a c channel image from the RGB color system, and combinations thereof; wherein the entropy value is selected from the group consisting of: an L entropy value, an a entropy value, a b entropy value, a c entropy value, and combinations thereof; and wherein the function has the following formula:

[0079] Skin property index = A + B × (L entropy value) + C × (a entropy value) + D × (b entropy value) + E × (c entropy),

[0080] Wherein A, B, C, D and E are constants; wherein at least one of B, C, D and E is not zero.

[0081] It should be appreciated that constants A, B, C, D, and E may vary based on the size and content of the training data set, and may be any values ​​generated by the model based on the training data set.

[0082] Specifically, each of the above entropy values ​​can be used alone or in combination with another entropy value in the entropy value. For example, using a single entropy value can result in faster calculation speeds, which enables the use of small devices with very basic hardware, thereby resulting in more efficient and cost-effective products.

[0083] At least one color channel image may be an L channel image; wherein the entropy value is an L entropy value; wherein C, D, and E each have a value of 0; and wherein the generated skin property index indicates skin purity, skin tone, or skin luminosity.

[0084] It is well known that when the skin is subjected to stress (caused by UV factors, aging factors, mental factors, environmental factors), the skin will be damaged at various levels, including DNA level, cellular level and tissue level. Such damage to the skin can lead to skin defects. The presence of these skin defects significantly affects the optical properties of the skin, such as described in the following examples:

[0085] If the stratum corneum is dry (winter / air conditions), light will be reflected more from the surface (surface reflection), thus enhancing the skin microtexture, which leads to lines on the skin

[0086] If the dermis is damaged (UV), less light will be scattered at the dermis (also called dermal scattering), and light will penetrate the skin (less subsurface reflection). When there is less dermal scattering, and therefore the skin appears darker.

[0087] If the skin is exposed to long-term UV, the skin produces more melanin. Melanin absorbs light, reducing subsurface reflections, and therefore the skin appears darker.

[0088] The above skin imperfections appear as signals of skin quality that are not perceptible to the consumer's eye. The consumer may perceive these imperfections as impurities but fail to take action to improve them due to lack of knowledge. If the skin is subjected to chronic stress and / or is left untreated, these imperceptible impurities can eventually lead to visible and perceptible phenotypes (pigmented spots, wrinkles, sagging).

[0089] It has been surprisingly found that the L entropy value of the L color channel image has the highest correlation with skin purity.

[0090] The technical effect of selecting the L channel image as at least one color channel image for the analysis step to obtain the L entropy value according to the method according to the present invention and determining the skin purity based on the L entropy value is because the L entropy value has the highest correlation (r=0.89) with the skin purity relative to other entropy values ​​based on the analysis color channel image. The following is data generated based on the correlation with the results from the visual perception study, which used a statistical analysis using the Pearson correlation coefficient (r). The correlation results are shown below in Table 2 below.

[0091] Table 2

[0092] Entropy Pearson correlation coefficient (r) with the results of visual perception studies L entropy value 0.89 a Entropy value 0.55 b Entropy value 0.7 c Entropy value 0.76

[0093] A higher Pearson correlation coefficient (r) means that the entropy value is a factor that has a greater impact on the condition of the cosmetic skin attribute studied in the visual perception study. Specifically, the visual perception study was conducted based on a predetermined number of panelists = 302, and the age of the panelists = 20-50. The panelists were asked to rank photos of people on a scale of 1 to 5 for skin purity (as an example of a cosmetic skin attribute). Based on the visual perception study results and the above correlation results, it has been found that the L channel entropy value of the filtered image (through a frequency filter) has the highest correlation with the skin purity attribute. Therefore, using the L entropy value of the L channel to determine the skin purity of at least a portion of a person's skin in a digital image can be used to transform skin purity from a visually imperceptible cosmetic skin attribute to a cosmetic skin attribute that can be explained to consumers in a consumer-relevant manner.

[0094] At least one color channel image may be an a-channel image; wherein the entropy value is an a-entropy value; wherein B, D, and E each have a value of 0; and wherein the generated skin property index indicates skin inflammation.

[0095] At least one color channel image may be a b-channel image; wherein the entropy value is a b-entropy value; wherein B, C, E each have a value of 0; and wherein the generated skin property index indicates skin pigmentation or skin dullness.

[0096] At least one color channel image can be a c-channel image; wherein the entropy value is a c-entropy value; wherein B, C, and D each have a value of 0; and wherein the generated skin property index indicates skin morphology, which is preferably selected from the group consisting of: pores, fine lines, wrinkles, sagging, skin elasticity, and combinations thereof.

[0097] Get color channel image

[0098] As referenced below Figure 5 A. Figure 5 B. Figure 5 C and FIG. 6 , color channel images 60a, 60b, 60c can be obtained from the digital image 51. Figure 1 , a digital image can be acquired from a portable electronic device 12 using a network 100, and the digital image can be transmitted to an apparatus 14 for use in a method 200 for determining a cosmetic skin attribute according to the present invention. An input device 12a can be coupled to or integrated with the portable electronic device 12 to receive a user input for starting a processor 14b. The portable electronic device 12 may include an output device 12b for displaying a plurality of tiles, each tile having a uniquely assigned single degree label. The input device 12a may include, but is not limited to, a mouse, a touch screen display, etc.

[0099] refer to Figure 1 , Figure 5 A and FIG. 6, when the processor 14b is started, the processor 14b causes, in step 202, for example, via the image acquisition logic 144a, to obtain a digital image 51 of at least a portion of the subject's face. The obtained digital image 51 may be an RGB XP digital image or an RGB glossy digital image. The digital image 51 in the RGB system is converted from the RGB image to digital image data, such as a color channel image in a different color system. The processor 14b further causes, in step 204, for example, via the image analysis logic 144b, to extract at least one color channel image 60a, 60b, 60c from the obtained digital image 51. The at least one color channel may be a color channel image obtained by further processing the RGB color channel based on an equation of (for example) 0.1R+0.2G+0.7B. The at least one color channel image 60a, 60b, 60c may be selected from any one of the color channels 60a, 60b, 60c in the color system. In step 206, the extracted at least one color channel image 60a, 60b, 60c is filtered using a frequency filter. The filtered at least one color channel image 61a, 61b, 61c is analyzed using entropy statistics in step 208 to determine cosmetic skin attributes of the person. The use of a frequency filter in step 206 removes noise from the extracted at least one color channel image 60a, which increases the sensitivity of the analysis in step 208, thereby resulting in a higher accuracy in the analysis output from step 208 relative to analyzing the unfiltered color channel images. However, analyzing the unfiltered color channel images can be advantageous in reducing the use of computing hardware, such as reducing hardware footprint, data storage space, or processing power in situations where only very small and basic hardware is available for implementing the method according to the present invention.

[0100] Optionally, method 200 may further include applying image correction factors to the filtered color channels for optimizing parameters in the filtered color channels before analyzing the filtered color channels. The parameters may include illumination correction, blur correction, rotation correction, etc.

[0101] The technical effect of determining at least one skin attribute using the method 90, 200 according to the present invention is that this provides a fast and accurate analysis of cosmetic skin attributes. Table 3 below describes the age correlation with commonly used imaging endpoints, and the correlation of the entropy statistic has a better correlation (0.81) relative to other imaging endpoints. The imaging endpoint can also be described as an imaging method for analyzing skin characteristics.

[0102] Table 3

[0103] Endpoints Relevance Texture area fraction 0.43 Spot area fraction 0.55 Wrinkle area fraction 0.57 L -0.52 a 0.39 b 0.54 LStandard Deviation 0.02 aStandard Deviation 0.28 bStandard deviation 0.35 Haralick Contrast 0.65 Entropy statistics 0.81

[0104] Methods 90, 200 may be performed in less than 5 seconds, preferably in 0.01 seconds to 5 seconds, more preferably in 0.5 seconds to 1 second, or different combinations of the upper and lower limits or any integer combination in the ranges listed above. When methods 90, 200 may be performed in less than 1 second, method 200 may be implemented in commercially available hardware such as a portable handheld electronic device, including but not limited to a mobile phone, which is commercially advantageous due to its scalability to a wider consumer network.

[0105] The color system may be an L*a*b color system, and the at least one color channel image may be corresponding to Figure 5 The texture channel shown in B is a red color channel 60a, a yellow color channel 60b, or a blue color channel 60c. Figure 5 C shows filtered color channel images 61a, 61b, 61c, each of which is analyzed to obtain an entropy value describing the analyzed color channel image. The entropy value of the filtered red color channel 61a may be defined as a-entropy, the entropy value of the filtered yellow color channel 61b may be defined as b-entropy, and the entropy value of the filtered blue color channel 61c may be defined as c-entropy.

[0106] The color system may be the chromophore mapping space described above, and the at least one color channel image may be a hemoglobin channel image or a melanin channel image.

[0107] The frequency filter may be a fast Fourier transform filter, a wavelet transform filter, or a Gaussian difference (DoG) filter. More preferably, the frequency filter is a DoG filter. The DoG filter has a Gaussian filter 1 and a Gaussian filter 2. Gaussian filter 1 may include the following standard deviations: 1 to 200, 5 to 50, 10 to 20, different combinations of the above upper and lower limits, or any combination of integers in the ranges listed above. Gaussian filter 2 may include the following standard deviations: 1 to 200, 5 to 100, 20 to 60, different combinations of the above upper and lower limits, or any combination of integers in the ranges listed above. Non-limiting examples of combinations of Gaussian filter 1 and Gaussian filter 2 of the DoG filter are described in Table 4 below.

[0108] Table 4

[0109] Gaussian filter 1 standard deviation Gaussian filter 2 standard deviation 1 to 200 1 to 200 5 to 50 5 to 100 10 to 20 20 to 60

[0110] Output device 12b may include, but is not limited to, a touch screen display, a non-touch screen display, a printer, a projector for projecting a plurality of tiles, each tile having a uniquely assigned single degree label on a display surface such as a mirror, as described below with respect to Figure 2 described.

[0111] Figure 2 is a perspective view of system 10 configured as a stand-alone imaging system located at a retail cosmetics counter for visualizing at least one cosmetic skin attribute and recommending cosmetic and skin care products based on the visualized at least one cosmetic skin attribute. Figure 3 yes Figure 2 A block diagram of system 10 is shown. Figure 2 and Figure 3 , the system 10 includes a Figure 1 The housing 11 of the device 14 is connected to an image acquisition device 18 for acquiring a digital image of a subject in order to visualize at least one cosmetic skin attribute. Figure 2 , the system 10 may include a mirror 16, and an image acquisition device 18 may be mounted behind the mirror 16 within the housing 11 so that the image acquisition device 18 can be hidden from view. The image acquisition device 18 may be a digital camera, an analog camera connected to a digitizing circuit, a scanner, a video camera, etc. The system 10 may include a light 30, such as an LED light, arranged around the housing 11 to form an LED lighting system for assisting in generating a digital image of the subject. The system 10 has an input device 112a for receiving user input. The system 10 may also include an output device 112b, such as a projector, which is configured to receive the facial image 30 and project the facial image for display on the mirror 16. Figure 2The projector is not shown in the drawings because it can be a peripheral component that is separate from the housing 11 but coupled to the device 14 to form the system 10. The system 10 may also include a second output device 112c such as one or more speakers, which is optionally coupled to an amplifier for generating audio guidance output to supplement and / or enhance the overall consumer experience.

[0112] Preferably, obtaining at least one color channel may include obtaining at least two color channels, more preferably, obtaining three color channels. Specifically, the red color channel, the yellow color channel, and the blue color channel may be described as follows. When the red color channel is in the L*a*b* color system, the a entropy is the entropy value of the filtered red color channel. When the yellow color channel is in the L*a*b* color system, the b entropy is the entropy value of the filtered yellow color channel. When the blue color channel corresponds to the texture channel, the c entropy is the entropy value of the blue color channel.

[0113] The method 200 may further include the step of comparing at least one cosmetic attribute to a predefined data set to assign an index. The index may be described as a skin attribute index of the analyzed visually imperceptible consumer skin attribute, wherein the unique assignment of a unitary label to each tile is based on the assigned index. The plurality of tiles, each having a uniquely assigned index, may be displayed in another step after the comparing step.

[0114] Methods for Visualizing Entropy Statistics

[0115] The present invention also relates to a method of visualizing entropy statistics or entropy values ​​of at least one cosmetic skin attribute of at least a portion of a face of a subject in a digital image. The method is provided with reference to a series of process flow charts as an illustration of how to visualize entropy values. Fig. 8A and Figure 8B to describe, and Fig. 9 is a flow chart of a method 300 of visualizing entropy statistics or entropy values ​​of at least one cosmetic skin property of at least a portion of a face of a subject in a digital image.

[0116] exist Fig. 8A5 shows a digital image 51 of at least a portion of a face. The digital image 51 includes an area of ​​at least a portion of the face 1 defined by a boundary line 52, and includes a plurality of tiles 54 across the digital image 51, each of the plurality of tiles 54 having at least one cosmetic skin attribute analyzed using entropy statistics. An outer perimeter 53 surrounds the boundary line 52 surrounding the first digital image 51. The first digital image 51 is formed by a total number of pixels, for example, the first digital image 51 may have a number of pixels defining a total image size of the first digital image 51. For example, if the tile size is set to 40 by 40 pixels to 70 by 70 pixels, accordingly, the number of tiles 54 forming the plurality of tiles 54 across the first digital image 51 will be obtained by dividing the total image size by the specified tile size. It should be understood that the size of the tile 54 may be defined by the number of pixels on the horizontal side (tile width, W) and the number of pixels on the vertical side (tile height, H). Each tile may include a tile size of no greater than 100 by 100 pixels, 1 by 1 pixel to 100 by 100 pixels, 2 by 2 pixels to 100 by 100 pixels, 5 by 5 pixels to 90 pixels by 90 pixels, 40 by 40 pixels to 70 by 70 pixels, or a combination of different upper and lower limits or any integers within the ranges listed above. The technical effect of having a tile size within the above ranges is that this enables shorter processing time for analyzing image data, and therefore enables display of at least one cosmetic skin attribute visualization in a shorter amount of time.

[0117] refer to Fig. 9 The method 300 includes receiving a digital image 51 in step 302, and uniquely assigning a unitary label 40 to each of the defined plurality of tiles 54 based on at least one cosmetic skin attribute being analyzed in step 304. At least some of the plurality of tiles each having the uniquely assigned unitary label are displayed in step 306 to visualize the entropy value of the at least one cosmetic skin attribute, such as Figure 8B as shown in .

[0118] To explain the manner in which the system 10 and methods 90, 200, 300 determine and visualize at least one cosmetic skin attribute according to the present invention, it is helpful to understand how a digital image of a subject's face is obtained in step 202, how a color channel image is extracted from the obtained digital image in step 204, how the extracted at least one color channel image is filtered in step 206, how a single degree label is uniquely assigned to each tile in step 304, and how the tile is displayed in step 306. Therefore, steps 202, 204, 206, 208 of method 200 according to the present invention and steps 302, 304, and 306 of method 300 are described below as separate processes for performing each step. Each process may also be described as a subroutine, i.e., a series of program instructions for performing a corresponding step according to methods 200, 300 according to the present invention.

[0119] Get digital image

[0120] Reference is made to a series of process flow charts illustrating how to obtain digital image data from a digital image. Fig. 10A , Fig. 10B and Fig. 10C to describe step 202 of obtaining a digital image according to method 200 of the present invention, and Fig.11 is a flow chart of a process 400 of obtaining digital image data corresponding to step 202 .

[0121] exist Fig. 10A FIG. 5 shows an input image 50a of a face 1. The user may select the image 50a of a face 1 in the following manner. Fig.11 An input image 50a is captured using camera 18 in step 402 of process 400 shown in FIG. Fig. 10B Step 404 is shown: cropping the input image 50a to obtain edited image data 50b including at least a portion of the face. The input image 50a may be cropped by: identifying anchor features 1a of the face, including (but not limited to) facial features such as eyes, nose, nostrils, corners of the mouth, etc.; and cropping accordingly. Fig. 10B The eyes are depicted as anchor features 1a in FIG. 4 , but it should be understood that this is merely an example and any prominent or detectable facial feature may be an anchor feature. The edited image data 50b may be the first digital image 51 obtained in step 404. Alternatively, as Fig. 10CAs shown in , the edited image data 50b may be further processed by cropping to remove one or more unwanted portions of the input image 50a, thereby obtaining a first digital image data 51 including at least a portion of the face 1 defined by the boundary line 52 in step 408. The obtained first digital image 51 may include at least one region of interest (ROI) 2 of at least a portion of the face 1 defined by the boundary line 52. The ROI 2 may be the entire portion of the face 1, preferably at least a portion of the face, and more preferably one or more skin regions defining at least a portion of the face 1. FIG. 14A to FIG. 14C and Fig.15 A flowchart is provided to describe the details of how to define the skin area.

[0122] Optionally, process 400 may include step 406, wherein ROI 2 may be selected from the group consisting of: a skin area around the eyes ("eye area 2a"), a skin area around the cheeks ("cheek area 2b"), a skin area around the mouth ("mouth area 2c"), and combinations thereof, preferably, ROI 2 is a portion of at least a portion of the subject's face 1, more preferably, the obtained first digital image data defines the left side or right side of the face 1. ROI 2 may include an area of ​​at least 5%, 10% to 100%, 25% to 90% of the obtained first digital image.

[0123] Limited tiles

[0124] Fig.12 is a picture showing a plurality of tiles 54 on the first digital image data 51 . Fig.13 is a flow chart illustrating a process 500 for defining a plurality of tiles 54 on first digital image data 51. Fig.12 , the first digital image 51 includes at least a portion of the face 1 defined by the boundary line 52, as described above with reference to Fig. 10C Described. Reference Fig.13 , the process 500 includes defining an outer perimeter 53 that surrounds a boundary line 52 around the first digital image obtained (step 502). The first digital image 51 obtained is formed by a total number of pixels, for example, the first digital image 51 obtained may have a number of pixels determined at step 404 or step 406, depending on the image size after cropping the input image 50a. Therefore, the total image size based on the first digital image 51 obtained may be defined in step 504. For example, if the tile size is set to 40 by 40 pixels 70 by 70 pixels, accordingly, the number of tiles 54 that form the plurality of tiles 54 across the first digital image 51 obtained in step 506 will be obtained by dividing the total image size by the specified tile size.

[0125] show

[0126] The method according to the present invention may further comprise the step of generating an image description corresponding to the generated skin attribute index described above for visualizing the cosmetic skin condition. The image description may comprise a heat map (such as Figure 8B , Fig.21 ), the total score (such as in Fig.19D The fourth area 194, Fig. 22 The total score may be calculated based on the skin attribute index generated as described above.

[0127] FIG. 14A to FIG. 14C is a process flow diagram showing details of the process of displaying a plurality of tiles in step 306 of method 300 according to the present invention. Fig.15 is a flow chart illustrating a process 600 of displaying multiple tiles. Fig.14A is a picture showing a second digital image 60 inserted onto the first digital image data 51. The second digital image 60 includes at least a portion of a subject's face having a plurality of tiles 54 displayed, each tile having a single degree label 40 uniquely assigned. Fig. 14B Three zones are shown displayed on the first digital image obtained based on a plurality of tiles 54 each having a uniquely assigned unitary label: a first zone 110, a second zone 120, and a third zone 130. Each zone 110, 120, 130 identifies the zone described above with reference to Fig. 9 A to Fig. 9 C and the corresponding region of interest (ROI) 2 on the face 1 of the subject depicted in FIG. 10 . Fig. 14C and Fig. 14B The difference is that the boundary line 52 and the outer perimeter 53 are shown in Fig. 14B The second digital image data 60 is not shown in Fig. 14C The first zone 110 may include a first zone line having a first zone color 110a, the second zone 120 may include a second zone line having a second zone color 120a, and the third zone 130 may include a third zone line having a third zone color 130a. Based on the analyzed image data of the tiles 54 in each zone, the color of each zone line may be different to better visually distinguish the tiles that visualize the cosmetic skin attributes that may be in a normal, beautiful, or fragile condition relative to other zones of the subject, such as in Fig.19D As shown in an exemplary user interface of .

[0128] Fig.156 is a flow chart showing a process 600 of displaying a plurality of tiles in step 306 of method 300 according to the present invention. Process 600 may begin at step 602, where a processor reads analyzed image data of each tile 54 and uniquely assigns a single degree tag to each tile 54 of the plurality of tiles based on at least one visual cosmetic skin attribute analyzed of the tile 54 (step 604). When the single degree tag is illumination, the analyzed image data of each tile in the tile may be converted in step 606 to reflect the brightness of the corresponding degree of illumination at each tile. In an exemplary example, relative to the degree of illumination at each tile in the tiles in the tiles in the partition 120, the partition 110 may have a lower degree of illumination at each tile in the partition 110. In addition, in step 608, the partitions may be defined such that the first partition 110 may correspond to the eye partition, the second partition 120 may correspond to the cheek partition, and the third partition 130 may correspond to the mouth partition. The average index of the tiles of each partition may be calculated to generate a diagnosis of the skin condition related to the displayed cosmetic skin attribute according to the corresponding partition, so as to assign a product recommendation item to the partition to process the displayed cosmetic skin attribute in the partition. Specifically, the method 300 may also include displaying at least one product recommendation item to process the displayed cosmetic skin attribute.

[0129] FIG. 16A to FIG. 16D is a process flow diagram illustrating a method of visualizing at least one cosmetic skin attribute according to the present invention. Fig.17 is a flow chart illustrating a method 700 of visualizing at least one cosmetic skin attribute according to the present invention. Fig.16A It is shown in Fig.17 In step 702 of method 700 a color picture of a first digital image of at least a portion of a face of a subject is displayed. Fig. 16B is a second digital image showing at least a portion of the subject's face and a color picture of a plurality of tiles each having a uniquely assigned unitary label, wherein in step 704, the second digital image is inserted on the first digital image. Optionally, the Fig. 16C The first digital image is converted to grayscale as shown in , to provide better contrast between the plurality of tiles each having a uniquely assigned single degree label and the first digital image. In step 706, three partitions are displayed on the second digital image based on the plurality of tiles each having a uniquely assigned single degree label.

[0130] Fig.18 It is shown as Fig.17 A flowchart of a variation of the method 700 for visualizing at least one cosmetic skin attribute is shown in FIG. Fig.17After step 706 of method 700, at least one product recommendation item is displayed in step 708. In step 710, the user is prompted to select to end method 700, and if the user selects yes, method 700 ends at step 712. If the user selects no, the steps of method 200 of FIG. 6 are executed, and method 700 returns to step 708.

[0131] Human Machine User Interface

[0132] The present invention also relates to a human-machine user interface (hereinafter referred to as "user interface") for providing product recommendations to process at least one cosmetic skin attribute. The user interface may be a graphical user interface on a portable electronic device including a touch screen display / display having an input device and an image acquisition device. The user interface may include a first area of ​​the touch screen display displaying a first digital image of at least a portion of a subject's face obtained from the image acquisition device and a second digital image inserted on the first digital image, the second digital image having at least a portion of the subject's face and a plurality of tiles of the display each having a uniquely assigned mono-degree mark. The user interface may also include a second area of ​​the touch screen display different from the first area, the second area displaying a selectable icon for receiving user input, wherein if the user activates the selectable icon, an image of at least one product recommendation item for processing the displayed cosmetic skin attribute is displayed on the touch screen display.

[0133] FIG. 19A to FIG. 19E are screenshots, each showing an exemplary user interface that cooperates with each other for visualizing cosmetic skin attributes according to the present invention. FIG. 19A to FIG. 19E The user interfaces described above are provided in a sequential manner in response to the aforementioned user interfaces, but it should be understood that the user interfaces may be programmed in a number of ways. FIG. 19A to FIG. 19E to define an overall user interface for visualizing at least one cosmetic skin attribute according to the method according to the invention as described above. FIG. 19A to FIG. 19E All user interfaces of define exemplary user interfaces for visualizing cosmetic skin attributes according to the present invention.

[0134] Fig.19A A user interface 160 is depicted for receiving a first user input, preferably the age of a user. The user interface 160 may include a first area 162 for receiving the first user input. The first area 162 may include one or more user input features 164 for receiving the first user input. The user input features 164 may be, for example, Fig.19A. The user interface 160 may further include a second area 166 including corresponding instructions to the user for providing the first user input. The second area 166 may be disposed above the first area 162 to provide a more user-friendly interface. The user interface 160 may be part of a startup option for starting the method 200 according to the present invention.

[0135] Fig.19B A user interface 170 is depicted for receiving a second user input, preferably a cosmetic skin attribute that is brought to the user's attention. The cosmetic skin attribute may be described as a skin problem of the user. Fig.19A The user interface 170 may include a first area 172 for receiving a second user input. The first area 172 may include one or more user input features 174 for receiving the second user input. The user input feature 174 may be a selectable input icon such as a corresponding to a predetermined skin problem. The first area 172 may also include an explanation area 173 corresponding to the one or more input features 174, wherein the explanation area 173 includes a brief description of the cosmetic skin attribute or skin problem. The user interface 170 may also include a second area 176, which includes corresponding instructions for the user to provide user input. The second area 176 may be arranged above the first area 172 to provide a more user-friendly interface.

[0136] Fig.19C A user interface 180 is depicted for obtaining an input image from a user. The user interface 180 may include a first area 182 having a user interface for aligning an anchor feature (such as an eye) to Fig.11 The process 400 described in the foregoing obtains the first digital image data. Fig.19B One or more user input features 174 of the present invention select a second user input and provide a user interface 180 .

[0137] Fig.19D A user interface 190 is depicted for displaying at least one cosmetic skin attribute. Fig.19C After obtaining the user input image in the user interface 180, the user interface 190 is provided. Fig.19DAs shown in , at least a portion of a person's skin is facial skin; wherein the facial skin includes at least one region of interest (ROI), and the at least one region of interest is preferably selected from the group consisting of: cheek region / partition, eye region / partition, forehead region / partition, nose region / partition, and combinations thereof; and wherein the image description visualizes the need for improvement in at least one ROI or the difference in cosmetic skin properties between a first ROI and a second ROI.

[0138] The user interface 190 may include a first area 191 that displays a plurality of tiles each having a uniquely assigned unitary label to visualize at least one cosmetic skin attribute according to the method of the present invention. The first area 191 may display a plurality of tiles each having a uniquely assigned unitary label to visualize at least one cosmetic skin attribute according to the method of the present invention. Fig.18 D, but the difference is that the lines defining the plurality of tiles may be bent and / or set as an invisible layer. The first area 191 may include a first zone 110 corresponding to an eye zone of at least a portion of the user's face, a second zone 120 corresponding to a cheek zone of at least a portion of the user's face, and a third zone 130 corresponding to a mouth zone of at least a portion of the user's face. Fig.19D As shown in , the partition result can be displayed in the third area 193, whereby the partition result includes an index generated for each partition based on a relative comparison of the indexes of the partitions within at least a portion of the user's face. In an exemplary embodiment, based on the partition result, the first partition 110 can be described as a normal / beautiful / fragile partition, the second partition 120 can be described as a normal / beautiful / fragile partition, and the third partition 130 can be described as a normal / beautiful / fragile partition. Preferably, each partition can have a different description based on the relative difference in the partition result. The user interface 190 also includes a second area 192 for receiving a third user input. The second area 192 may include one or more user input features 1921 for receiving the third user input. The user input feature 1921 may be a selectable input icon such as for continuing to the next step of the method according to the present invention. Optionally, the user interface 190 may include a fourth area 194 for displaying the user's skin age based on at least one cosmetic skin attribute analyzed for each of a plurality of tiles of the first digital image data obtained based on at least a portion of the user's face.

[0139] Fig.19E A user interface 800 is depicted that includes a first area 801 for displaying product recommendation items 210. Fig.19DThe user interface 800 is provided by selecting a user input feature 1921 of the user interface 190 of the present invention. Optionally, the user interface 800 may include a second area 802 for providing details of the product recommendation item 210. Preferably, the user interface 800 may include a third area 803 for receiving a fourth user input, such as a request for assistance from a product advisor for inquiring about and / or purchasing the product recommendation item 210. The third area 803 may include one or more user input features 2031 for receiving the fourth user input. The user input feature 2031 may be a selectable input icon such as for continuing to the next step of the method according to the present invention.

[0140] Fig. 20 A partial view of an exemplary user interface 900 including an image description 901 overlaid on a digital image 51 for visualizing at least one cosmetic skin attribute according to the present invention is depicted. The image description 902 includes a heat map generated based on the entropy values ​​output from the method 90 described above. The heat map includes a first heat map segment 906 based on a low entropy value corresponding to a better cosmetic skin attribute condition. The heat map 904 also includes a second heat map segment 908 based on a high entropy value corresponding to a poor cosmetic skin attribute condition. The first heat map segment 906 is formed by a first plurality of tiles that are visually different from a second plurality of tiles in the second heat map segment 908. For example, the first plurality of tiles are converted to display a color different from the color of the second plurality of tiles. The undisplayed heat map segment 910 (hereinafter referred to as "non-displayed heat map segment 910") corresponds to an entropy value between a high entropy value and a low entropy value. The heat map segment can be configured as follows to display entropy information related to the cosmetic skin attribute condition and the skin attribute index, as summarized in Table 5 below.

[0141] Table 5

[0142]

[0143] Fig.21 An alternative variation of an image depiction 920 for visualizing at least one cosmetic skin attribute in the user interface 900 of FIG. 19 is depicted. The image depiction 920 differs from the image depiction 902 of FIG. 19 in that the image depiction 920 includes a displayed region of interest (ROI) 922 that is transformed to display a color to indicate a poor cosmetic skin attribute condition relative to other non-displayed regions of interest (ROIs) 924 corresponding to better cosmetic skin attribute conditions. Displaying only a single heatmap segment (see FIG. 19 ) Fig. 20 ) or ROI is that it does not overload the consumer viewing the user interface with too much visual information.

[0144] Fig. 2219 is a screenshot showing an exemplary user interface 930 for visualizing at least one cosmetic skin attribute according to the present invention, wherein the at least one cosmetic skin attribute is skin purity. The user interface 930 is different from the user interface 902 of FIG. 19 in that the user interface 930 includes alternative text 932 that describes the cosmetic skin attribute and a total score 934 based on the generated skin attribute index. The user interface 930 may also include a meter 936 and a meter mark 938 for indicating the total score along the meter 936 on a scale of 0 to 100. The meter 936 is a different way to visualize the total score 934 and may be optional. The color of the meter 936 may be configured to show a color gradient representing the first heat map segment 904 and the second heat map segment 906.

[0145] The method for determining a cosmetic skin condition according to the present invention described above may further include the following steps: tracking cosmetic skin attributes over a predetermined period of time. Fig.21 The user interface 930 shown in FIG. 1 may include a first selectable icon 940 that, when selected, causes the processor to receive instructions and perform steps by the processor to generate a calendar or timetable to produce a cosmetic skin attribute diary to track improvements in cosmetic skin attributes. For example, when the consumer uses it on Day 1, the date and facial analysis are recorded and saved in memory. Subsequently, whenever the consumer uses the method according to the present invention in the future (after a predetermined period, 1 week, 1 month, 6 months), the consumer's facial skin is analyzed again, and the consumer can compare how his / her facial skin looks at that time after a predetermined period relative to Day 1. The method according to the present invention may be configured as a downloadable software application that is stored as a native application on a portable electronic device; or a web application that can be accessed through a login account unique to the consumer, so that the consumer can perform self-skin analysis based on the method according to the present invention and view and / or monitor improvements (reduction of ROIs with poor cosmetic skin attribute conditions) over a period of time.

[0146] The user interface 930 may further comprise a second selectable icon 942 which, when selected, enables the method for determining cosmetic skin attributes according to the invention to be repeated. For example, the method 90 described above may be repeated.

[0147] Representative embodiments of the present disclosure described above may be described as set forth in the following paragraphs:

[0148] A. A method for determining a person's visually imperceptible cosmetic skin attribute, the method comprising the following steps:

[0149] a) obtaining a digital image of a person, wherein the digital image comprises a color channel;

[0150] b) extracting at least one color channel from the acquired digital image to provide an extracted color channel image;

[0151] c) filtering the extracted color channel image; and

[0152] d) Using entropy statistics to analyze the filtered color channel images to determine human visually imperceptible cosmetic skin attributes.

[0153] B. The method according to paragraph A, wherein in step (b), a frequency filter is used to filter the color channel image, preferably, the frequency filter is selected from the group consisting of: a Fourier transform filter, a wavelet transform, a Gaussian difference (DoG) filter, more preferably, the frequency filter is a DoG filter, more preferably, the DoG filter comprises a Gaussian filter 1 having a standard deviation of 1 to 200, preferably 5 to 50, more preferably 10 to 20, and a Gaussian filter 2 having a standard deviation of 1 to 200, preferably 5 to 100, more preferably 20 to 60.

[0154] C. A method according to paragraph A or B, wherein the visually imperceptible cosmetic skin attribute is selected from the group consisting of: skin morphology, skin elasticity, skin tone, skin pigmentation, skin texture, skin pores, cosmetic skin inflammation, skin hydration, skin sebum level, acne, moles, skin luminosity, skin shine, skin dullness, uneven tone or skin barrier.

[0155] D. A method according to any of paragraphs A to C, wherein at least one color channel is one of the following: a red color channel, a yellow color channel, and a blue color channel, preferably at least two color channels, more preferably three color channels; wherein,

[0156] The red color channel is in L*a*b* color space, where a entropy is the entropy value of the filtered red color channel;

[0157] The yellow color channel is in the L*a*b* color system, where b-entropy is the entropy value of the filtered yellow color channel;

[0158] The blue color channel corresponds to the texture channel, where c entropy is the entropy value of the blue color channel.

[0159] E. A method according to paragraph D, wherein at least one color channel image is a red color channel, preferably, wherein the visually imperceptible attribute cosmetic skin attribute is skin inflammation, and the skin attribute index associated with skin inflammation is based on the alpha entropy of the red channel.

[0160] F. A method according to paragraph D, wherein at least one color channel is a yellow color channel, preferably, wherein the visually imperceptible cosmetic skin attribute is skin pigmentation / dullness, and the skin attribute index associated with skin pigmentation / dullness is based on the b-entropy of the yellow color channel.

[0161] G. The method of paragraph D, wherein at least one color channel is a texture channel,

[0162] Preferably, the visually imperceptible cosmetic skin attribute is skin topography, more preferably the skin topography is selected from the group consisting of: pores, fine lines, wrinkles, sagging, skin elasticity, and the skin attribute index associated with skin topography is based on c-entropy of the blue color channel.

[0163] H. A method according to any of paragraphs A to D, wherein at least one color channel is a melanin channel or a hemoglobin channel, preferably at least two color channels; wherein the hemoglobin channel is in the chromophore mapping space and has an entropy value;

[0164] The melanin channel is in the chromophore mapping space and has an entropy value;

[0165] I. A method according to paragraph H, wherein at least one color channel image is a hemoglobin channel, preferably, wherein the visually imperceptible attribute cosmetic skin attribute is skin inflammation, and the skin attribute index associated with skin inflammation is based on the entropy of the hemoglobin channel.

[0166] J. A method according to paragraph H, wherein at least one color channel is a melanin channel, preferably, wherein the visually imperceptible cosmetic skin attribute is skin pigmentation / dullness, and the skin attribute index associated with skin pigmentation / dullness is based on the entropy of the melanin channel.

[0167] K. A method as described in any of paragraphs A to J, further comprising applying image correction factors to the filtered color channel images before analyzing the image data.

[0168] L. The method according to any of paragraphs A to K, wherein the method is performed in less than 5 seconds, preferably in 0.01 seconds to 5 seconds, and more preferably in 0.5 seconds to 1 second.

[0169] M. A method according to A to L, wherein the acquired image data includes at least one region of interest (ROI) of at least a portion of the subject's face, preferably one to three ROIs, more preferably three ROIs on the acquired image data, wherein each ROI is defined by a partition including a partition line displayed on the acquired image data.

[0170] N. The method according to paragraph M, wherein the obtained image data includes three ROIs, and the method further comprises the following steps:

[0171] a) scoring a skin property index of a determined visually imperceptible cosmetic skin property of a person in a first subarea of ​​one of the three ROIs relative to a second pair of subareas of another of the three ROIs; and

[0172] b) Outputting the skin property index to a display to visualize the entropy statistics of the visually imperceptible cosmetic skin properties of a person.

[0173] O. The method according to any one of paragraphs A to N, further comprising the following steps:

[0174] a) scoring a skin property index of a determined visually imperceptible cosmetic skin property of a person relative to the visually imperceptible cosmetic skin properties of a defined group; and

[0175] b) Outputting the skin property index to a display to visualize the entropy statistics of the visually imperceptible cosmetic skin properties of a person.

[0176] P. The method according to paragraph O, wherein the average age of the person and the defined population is 18 to 40 years old, preferably 20 to 40 years old, more preferably 25 to 35 years old, and more preferably 28 to 32 years old.

[0177] Q. According to the method described in any of paragraphs A to P, the method also includes displaying at least one product recommendation item.

[0178] R. An apparatus for determining a visually imperceptible cosmetic skin attribute of a person, the apparatus comprising:

[0179] a non-transitory computer-readable storage medium configured to store a digital image of a person and computer-executable instructions, wherein the digital image includes a color channel; and

[0180] A processor is coupled to the storage medium, the processor being configured to perform the following operations based on computer executable instructions:

[0181] a) obtaining a digital image;

[0182] b) extracting at least one color channel from the acquired digital image to provide a color channel image, wherein the color channel image includes cosmetic skin attributes,

[0183] c) filtering the color channel image; and

[0184] d) Using entropy statistics to analyze the filtered color channel images to determine visually imperceptible cosmetic skin attributes.

[0185] S. The apparatus of paragraph R, wherein the processor is configured to perform the following operations based on the computer executable instructions:

[0186] scoring a skin property index of a visually imperceptible cosmetic skin property based on an entropy value of the filtered color channel image relative to a predetermined entropy value, wherein the predetermined entropy value is associated with a population; and

[0187] Output skin property index.

[0188] T. A system for determining visually imperceptible cosmetic skin attributes, wherein the system comprises an apparatus according to paragraph R or S and a display for displaying a facial image of a person including a skin attribute index, preferably displaying the skin attribute index as one of the following: a number, a graphic symbol, a scale, or a combination thereof.

[0189] U. A method for determining a person's cosmetic skin attributes, the method comprising the following steps:

[0190] a) receiving user input, wherein the user input includes cosmetic skin attributes;

[0191] b) obtaining a digital image of a person, wherein the digital image includes a color channel;

[0192] c) extracting at least one color channel from the acquired digital image to provide an extracted color channel image;

[0193] d) filtering the extracted color channel image;

[0194] e) Using entropy statistics to analyze the filtered color channel images to determine the person's cosmetic skin attributes.

[0195] V. A method for determining a person's cosmetic skin attributes, the method comprising the following steps:

[0196] a) using facial recognition to detect at least a portion of a person's face;

[0197] b) obtaining a digital image of a person, wherein the digital image includes a color channel;

[0198] c) extracting at least one color channel from the acquired digital image to provide an extracted color channel image;

[0199] d) filtering the extracted color channel image; and

[0200] e) Using entropy statistics to analyze the filtered color channel images to determine the person's cosmetic skin attributes.

[0201] W. A method of visualizing an entropy value of at least one cosmetic skin property of at least a portion of a face of a subject in a digital image, the method comprising:

[0202] a) receiving digital image data of at least a portion of a face of a subject, wherein the digital image data comprises a plurality of patches, each patch of the plurality of patches having at least one cosmetic skin property analyzed using entropy statistics;

[0203] b) uniquely assigning a single degree label to each of the plurality of tiles based on the at least one cosmetic skin attribute being analyzed; and

[0204] c) displaying at least some of the plurality of tiles, each having a uniquely assigned unitary label, to visualize the entropy value of at least one cosmetic skin property.

[0205] X. A graphical user interface for determining at least one visually imperceptible cosmetic skin attribute, the graphical user interface being on a portable electronic device including a touch screen display / display having an input device and an image acquisition device, the graphical user interface comprising:

[0206] a first area of ​​the display, the first area displaying content data describing beauty skin attributes; and

[0207] A second area of ​​the display, which is different from the first area, displays a selectable icon for receiving user input, wherein when the user activates the selectable icon, a method according to any of paragraphs A to W is initiated by a processor in digital communication with the device.

[0208] Unless expressly excluded or otherwise limited, each document cited herein, including any cross-referenced or related patent or patent application and any patent application or patent to which this application claims priority or the benefit of, is hereby incorporated by reference in its entirety. The citation of any document is not an admission that the document is prior art with respect to any invention disclosed or claimed herein, or that the document alone or in any combination with any other reference teaches, suggests or discloses any such invention. In addition, to the extent that any meaning or definition of a term in this document conflicts with any meaning or definition of the same term in a document incorporated by reference, the meaning or definition assigned to the term in this document shall prevail.

[0209] Although specific embodiments of the present invention have been illustrated and described, it will be apparent to those skilled in the art that various other changes and modifications may be made without departing from the spirit and scope of the present invention. It is therefore intended that all such changes and modifications within the scope of the present invention be covered in the appended claims.

Claims

1. A method for determining a person's cosmetic skin attributes, the method comprising the following steps: a) obtaining at least one color channel image comprising at least a portion of the skin of said person; b) analyzing the at least one color channel image using entropy statistics to obtain an entropy value; as well as c) determining the cosmetic skin property of the at least a portion of the skin of the person based on the entropy value, Wherein the at least one color channel image is an image in the L*a*b* color system selected from the group consisting of: an L color channel image, an a channel image, a b channel image, and a c channel image from the RGB color system and combinations thereof; wherein the entropy value is selected from the group consisting of: an L entropy value, an a entropy value, a b entropy value, a c entropy value and combinations thereof; and wherein the function of the entropy value has the following formula: Skin property index = A + B × (L entropy value) + C × (a entropy value) + D × (b entropy value) + E × (c entropy), Wherein A, B, C, D and E are constants; wherein at least one of B, C, D and E is not zero.

2. The method according to claim 1, wherein the at least one color channel image is an image in a color system selected from the group consisting of: an L*a*b* color system, an RGB color system, an HSL / HSV color system, and a CMYK color system.

3. The method of claim 1, wherein a skin property index is generated as a probability value indicative of a condition of the cosmetic skin property of the at least a portion of the person's skin relative to a defined population.

4. The method of claim 3, wherein the skin property index is generated as a function of the entropy value defined by F(entropy value), wherein the function is determined by a model built on a training data set, wherein the training data set comprises: (i) a plurality of color channel images of the defined population, wherein each of the plurality of color channel images includes facial skin of a person in the defined population, wherein the facial skin includes the cosmetic skin attribute; and (ii) associated category definitions based on the cosmetic skin attribute. The method according to claim 3 , wherein the model is a regression model or a classification model. The method of claim 5 , wherein the regression model comprises a linear regression model.

7. The method of claim 6, wherein the linear regression model comprises a machine learning linear regression model.

8. The method of claim 7, wherein the machine learning linear regression model comprises a machine learning support vector regression model or a random forest regression model.

9. A method according to claim 3, wherein the at least one color channel image is an L channel image; wherein the entropy value is an L entropy value; wherein C, D, and E each have a value of 0; and wherein the generated skin property index indicates skin purity, skin tone, or skin luminosity.

10. The method of claim 3, wherein the at least one color channel image is an a channel image; wherein the entropy value is an a entropy value; wherein B, D, and E each have a value of 0; And wherein the generated skin property index is indicative of skin inflammation.

11. The method of claim 3, wherein the at least one color channel image is a b-channel image; wherein the entropy value is a b-entropy value; wherein B, C, and E each have a value of 0; And wherein the generated skin attribute index indicates skin pigmentation or skin dullness.

12. A method according to claim 3, wherein the at least one color channel image is a c-channel image; wherein the entropy value is a c-entropy value; wherein B, C, and D each have a value of 0; and wherein the generated skin attribute index indicates skin morphology, and the skin morphology is selected from the group consisting of: pores, fine lines, wrinkles, sagging, skin elasticity, and combinations thereof.

13. The method according to claim 3, further comprising step (d): generating an image description corresponding to the generated skin attribute index.

14. The method of claim 13, wherein the at least a portion of the person's skin is facial skin; wherein the facial skin comprises at least one region of interest (ROI), the at least one region of interest being selected from the group consisting of: a cheek region, an eye region, a forehead region, a nose region, and combinations thereof; and wherein the image description visualizes a need for improvement in the at least one ROI or a difference in the cosmetic skin attribute between a first ROI and a second ROI.

15. The method of claim 1, wherein the cosmetic skin attribute is selected from the group consisting of skin purity, skin age, skin topography, skin tone, skin pigmentation, skin pores, skin inflammation, skin hydration, skin sebum level, acne, moles, skin luminosity, skin shine, skin dullness, and skin barrier.

16. The method of claim 1, wherein the cosmetic skin attribute is a visually imperceptible cosmetic skin attribute, wherein the visually imperceptible cosmetic skin attribute is a cosmetic skin attribute that is not detectable by unassisted eyes, or a cosmetic skin attribute that is visually detectable by a consumer but is not understandable by the consumer.

17. The method of claim 1, wherein prior to analysis, the at least one color channel image is filtered by using a frequency filter.

18. The method of claim 17, wherein the frequency filter is selected from the group consisting of: a Fourier transform filter, a wavelet transform, and a Difference of Gaussian (DoG) filter. The method of claim 18 , wherein the frequency filter is a DoG filter.

20. The method of claim 19, wherein the DoG filter comprises a first Gaussian filter having a standard deviation of 1 to 200.

21. The method of claim 19, wherein the DoG filter comprises a first Gaussian filter having a standard deviation of 5 to 50.

22. The method of claim 19, wherein the DoG filter comprises a first Gaussian filter having a standard deviation of 10 to 20.

23. The method of any one of claims 20-22, wherein the DoG filter comprises a second Gaussian filter having a standard deviation of 1 to 200.

24. The method of any one of claims 20-22, wherein the DoG filter comprises a second Gaussian filter having a standard deviation of 5 to 100.

25. The method of any one of claims 20-22, wherein the DoG filter comprises a second Gaussian filter having a standard deviation of 20 to 60.

26. An apparatus for determining cosmetic skin attributes of a person, the apparatus comprising: an image acquisition device for acquiring at least one color channel image including at least a portion of the skin of the person, wherein the image acquisition device comprises a non-transitory computer-readable storage medium configured to store the acquired at least one color channel image; an image processing device coupled to the image acquisition device for analyzing the acquired at least one color channel image using entropy statistics to obtain an entropy value, and determining the cosmetic skin property of the at least a portion of the skin of the person based on the entropy value, wherein the image processing device comprises a processor having computer executable instructions; a display generation unit for generating a display for displaying content data describing the determined beauty skin attribute, Wherein the at least one color channel image is an image in the L*a*b* color system selected from the group consisting of: an L color channel image, an a channel image, a b channel image, and a c channel image from the RGB color system and combinations thereof; wherein the entropy value is selected from the group consisting of: an L entropy value, an a entropy value, a b entropy value, a c entropy value and combinations thereof; and wherein the function of the entropy value has the following formula: Skin property index = A + B × (L entropy value) + C × (a entropy value) + D × (b entropy value) + E × (c entropy), Wherein A, B, C, D and E are constants; wherein at least one of B, C, D and E is not zero.

27. A graphical user interface for determining cosmetic skin attributes of a person, the graphical user interface being on a portable electronic device comprising a touch screen display / display having an input device and an image acquisition device, the image acquisition device being configured to acquire at least one color channel image comprising at least a portion of the person's skin; The graphical user interface comprises: a first area of ​​the display, the first area displaying content data describing cosmetic skin properties of the at least a portion of the person's skin; and a second area of ​​the display, the second area being different from the first area, the second area displaying selectable icons for receiving user input; and a network interface digitally coupling the device to an entropy analysis unit, the entropy analysis unit being configured to perform the following operations based on computer executable instructions stored in the entropy analysis unit: analyzing the at least one color channel image using entropy statistics to obtain an entropy value; and determining the cosmetic skin property of the at least a portion of the person's skin based on the entropy value if the user activates the selectable icon, wherein the at least one color channel image is an image in the L*a*b* color system selected from the group consisting of: an L color channel image, an a channel image, a b channel image, and a c channel image from the RGB color system and combinations thereof; wherein the entropy value is selected from the group consisting of: L entropy value, a entropy value, b entropy value, c entropy value, and combinations thereof; and wherein the function of the entropy value has the following formula: Skin property index = A + B × (L entropy value) + C × (a entropy value) + D × (b entropy value) + E × (c entropy), Wherein A, B, C, D and E are constants; wherein at least one of B, C, D and E is not zero.

Images