Method and system for determining cosmetic skin attributes based on impairment values

Abstract

The present invention relates to a method and system for determining a person's cosmetic skin attributes based on impairment values, which exhibits an improved match to human skin tone or skin condition, and / or an improved match to the perception of human skin tone or skin condition.

Description

[Technical Field]

[0001] The present invention relates to a method and system for determining a person's cosmetic skin attributes based on impairment values, which exhibits an improved match to human skin tone or skin condition, and / or an improved match to the perception of human skin tone or skin condition. [Background technology]

[0002] To provide information on consumers' skin attributes, various digital skin assessment tools have been developed to meet consumer needs.

[0003] For example, U.S. Patent Application Publication No. 2020184642(A1) (11100639B2) relates to a method for skin examination, more specifically to a method for skin examination based on RBX color space conversion. This U.S. patent publication discloses a method for detecting skin condition, particularly the degree of skin redness, more specifically by the intensity of skin redness. In paragraph 0058 of this U.S. publication, "All individuals were the same with respect to the average red intensity value. However, as shown in Figure 5, the group was ranked from high to low in terms of severe rosacea, moderate rosacea, mild rosacea, and normal according to the difference between the average red intensity value and the average green intensity value, i.e., the RG value."

[0004] Another example could be PCT application publication WO2019144247(A1), which relates to a system and method for evaluating and monitoring facial acne from digital photographic images.

[0005] One or more examples include U.S. Patent Publication 2010 / 0284610(A1) ('610) for a method of evaluating skin tone from an input image including a facial region. Publication '610 describes dividing a facial region of an image into predetermined regions according to first feature points formed in at least 25 predetermined regions and second feature points set using the first feature points. Publication '610 further describes L * a * b *L of the color system * , a * , b * , C ab * , and h ab , based on the average value using at least one of the tristimulus values X, Y, Z of the XYZ color system, and the values of RGB, hue H, lightness V, chroma C, melanin amount, and hemoglobin amount, generate a skin color tone distribution, and then perform an evaluation based on the measurement results for the divided regions, and perform a skin color tone distribution evaluation by displaying the measurement results or evaluation results on the screen.

[0006] However, the present inventors have found that the measurement results of a person from such a method may not match the skin state of the person and / or may not match the perception of the skin state of the person. A person who receives such measurement results cannot easily accept the recommendation of the following skin care products to improve their skin state.

Prior Art Documents

Patent Documents

[0007]

Patent Document 1

Patent Document 2

Patent Document 3

Summary of the Invention

Problems to be Solved by the Invention

[0008] Therefore, there is still a need for a method for determining the cosmetic skin attributes of a person that shows an improved match to the person's skin state and / or an improved match to the person's perception of their own skin state.

Means for Solving the Problems

[0009] The present invention relates to a method for determining the beauty skin attributes of a person, comprising: a) obtaining at least one color image comprising at least a part of the skin of a person; b) analyzing the at least one color image to obtain a defect value for a specific color; c) determining the beauty skin attributes of at least a part of the skin of the person based on the defect value.

[0010] The present invention also relates to a system for determining the beauty skin attributes of a person, the apparatus comprising: an image acquisition unit for obtaining at least one color image including at least a part of the skin of a person, preferably, the image acquisition device comprising a non-temporary computer-readable storage medium configured to store the at least one color image obtained; an image processing unit coupled to the image acquisition unit, analyzing the at least one obtained image to obtain a defect value, and determining the beauty skin attributes of at least a part of the skin of the person based on the defect value; a display generation unit coupled to the image acquisition unit and configured to generate a display for displaying content data describing the determined beauty skin attributes.

[0011] The present invention provides a method and system for determining a person's cosmetic skin attributes based on impairment values, which demonstrates an improved match to a person's skin tone or skin condition, and / or an improved match to a person's perception of a person's skin tone or skin condition. The inventors have surprisingly found that by using impairment values, the method of the present invention can provide an improved match to a person's skin tone or skin condition, and / or an improved match to a person's perception of a person's skin tone or skin condition, particularly an improved match to a person's perception of a group consisting of stressed skin, healthy skin, inflamed skin, hidden aging skin, and mixtures thereof. In particular, the method of the present invention can provide improved results for the early detection of skin defects, especially skin aging, i.e., early detection of hidden aging skin, compared to known digital tools for skin assessment. Furthermore, the method of the present invention can provide a convenient method for evaluating accumulated stress (stressed skin) and inflammatory symptoms (inflamed skin) through image analysis, which has previously only been measured through biological assays, and stressed skin and / or inflamed skin may be signals of hidden aging skin.

[0012] Cosmetic skin attributes may include imperceptible cosmetic skin attributes, such as those that are not visually perceptible, those that are difficult to define clearly (e.g., stressed skin, healthy skin, hidden aging skin), those that cannot be detected by the naked eye, and / or those that consumers can visually detect but do not understand. The advantage of determining visually imperceptible cosmetic skin attributes is that it enables consumers to make informed decisions and take proactive actions to improve the condition of those imperceptible cosmetic skin attributes. [Brief explanation of the drawing]

[0013] It should be understood that both the general description above and the detailed description below are intended to describe various embodiments and provide an overview or framework for understanding the essence and features of the subject matter of the claimed invention. The accompanying drawings are included to provide a further understanding of the various embodiments and are incorporated herein and constitute part of this specification. The drawings illustrate the various embodiments described herein and, together with the descriptions, serve to illustrate the principles and operation of the subject matter of the claimed invention. [Figure 1] This figure shows an exemplary system for determining cosmetic skin attributes via a network according to the present invention. [Figure 2] This figure illustrates an alternative exemplary system for determining cosmetic skin attributes according to the present invention, and in particular, is a perspective view of the system of Figure 1 configured as an exemplary standalone imaging system. [Figure 3] This is a block diagram showing the components of an exemplary system for determining cosmetic skin attributes according to the present invention. [Figure 4] This is a flowchart showing a method for determining cosmetic skin attributes according to the present invention. [Figure 5] This is a flowchart illustrating an exemplary method for determining cosmetic skin attributes according to the present invention. [Figure 6] This shows several impairment values ​​obtained for three different skin samples compared to conventional color analysis (a* mean). [Figure 7A] This is a series of process flow diagrams illustrating the details of the step of acquiring a first digital image in the method for determining cosmetic skin attributes according to the present invention. [Figure 7B] This is a series of process flow diagrams illustrating the details of the step of acquiring a first digital image in the method for determining cosmetic skin attributes according to the present invention. [Figure 7C] This is a series of process flow diagrams illustrating the details of the step of acquiring a first digital image in the method for determining cosmetic skin attributes according to the present invention. [Figure 8] This flowchart exemplifies the process of acquiring the first digital image. [Figure 9] This photograph exemplifies the process of defining multiple tiles in the method for determining cosmetic skin attributes according to the present invention. [Figure 10] This flowchart exemplifies the process of defining multiple tiles. [Figure 11] This figure exemplifies the display of multiple tiles according to the present invention, and in particular exemplifies a second digital image inserted on a first digital image. [Figure 12] This flowchart shows an exemplary process for displaying multiple tiles according to the present invention. [Figure 13A] This is a diagram illustrating the first digital image. [Figure 13B] This diagram exemplifies the display of multiple tiles, and in particular, exemplifies the display of a second digital image inserted on a first digital image. [Figure 14] This is a flowchart illustrating an exemplary method for visualizing at least one cosmetic skin attribute according to the present invention. [Figure 15] This is a screenshot showing an exemplary user interface for visualizing at least one cosmetic skin attribute according to the present invention. [Modes for carrying out the invention]

[0014] Before describing the present invention in detail, the following terms are defined, and any undefined terms should be given the ordinary meaning as understood by those skilled in the art.

[0015] As used herein, “cosmetic skin attributes” include all skin attributes that provide a visual / aesthetic effect to an area of ​​the human body or affect the appearance and / or feel of the skin. Some non-exclusive examples of cosmetic skin attributes include skin purity, skin age, skin topography, skin tone, skin pigmentation, skin pores, skin inflammation, skin hydration, skin sebum levels, acne, moles, skin luster, skin shine, skin dullness, uneven skin tone, or skin barrier. Those skilled in the art will understand that the above-mentioned cosmetic skin attributes are standard terms, and that corresponding definitions of cosmetic skin attributes can be found in the following published references: "Handbook of cosmetic science and technology, 3rd edition, editors Andre O. Barel, Marc Paye, Howard I. Maiback, CRC Press, 2009," "Cosmetic Science and Technology - Theoretical Principles and Applications, editors Kazutami Sakamoto, Robert Y. Lochhead, Howard I. Maiback, Yuji Yamashita, Elsavier, 2017," and "Cosmetic Dermatology: Products and Procedures, Editor(s): Zoe Diana Draelos, Blackwell Publishing Ltd, 2010." Cosmetic skin attributes do not include skin attributes related to a medical condition or an underlying medical condition. Cosmetic skin attributes are preferably selected from the group consisting of stressed skin, healthy skin, inflamed skin, hidden aging skin, skin age, skin topography, skin tone, skin pigmentation, skin pores, skin hydration, skin sebum level, acne, moles, skin luster, skin shine, skin dullness, and skin barrier, prediction of future cosmetic skin attributes, and combinations thereof.Cosmetic skin attributes are more preferably selected from the group consisting of stressed skin, healthy skin, inflamed skin, hidden aging skin, skin age, skin topography, skin tone, skin pigmentation, skin hydration, sebum level, moles, skin luster, skin shine, skin dullness, and skin barrier, prediction of future cosmetic skin attributes, and combinations thereof. Cosmetic skin attributes are more preferably selected from the group consisting of stressed skin, healthy skin, inflamed skin, hidden aging skin, and combinations thereof.

[0016] As used herein, "tile" includes units such as pixels, which form part of a digital image and therefore the "tile" forms the whole of the digital image.

[0017] As used herein, “digital image data” includes, but is not limited to, image acquisition devices including, digital cameras, photographic scanners, computer-readable storage media capable of storing digital images, and any electronic devices including photographic capabilities. Digital image data may also include color channel images obtained by converting an RGB image to a color channel image in a color system.

[0018] As used herein, “a single degree of indicium” includes, but is not limited to, all electronic visual representations, including graphic symbols, numbers, color codes, lighting techniques, and combinations thereof.

[0019] When used in this specification, "L * a * b * " refers to the generally accepted color space defined by the International Commission on Illumination (CIE). The three coordinates are (i) the brightness of the color (i.e., L * =0 produces black, L *(i) =100 indicates diffuse white), (ii) the position of the color between magenta and green (i.e., negative a * The value is shown in green, and positive a * (iii) the value of which indicates magenta, and the position of the color between yellow and blue (i.e., negative b * The value is shown in blue, and positive b * The value indicates yellow.

[0020] As used herein, “skin age” means apparent age, which refers to the visually estimated or perceived skin age of a person based on physical appearance, preferably a person’s face, preferably at least a portion of a person’s face, more preferably at least one ROI of a person’s face, where the at least one ROI is selected from the group consisting of the skin area around the eyes (“eye area”), the skin area around the cheeks (“cheek area”), the skin area around the mouth (“mouth area”), and combinations thereof, more preferably the skin area around the cheeks (“cheek area”).

[0021] As used herein, “skin tone” generally refers to the overall appearance of the underlying skin color or uniformity of color. Skin tone is generally characterized over a broad area of ​​skin. The area is 100 mm². 2 It may be larger, but a larger area is envisioned, such as the entire surface of the facial skin or other body skin (e.g., arms, legs, back, hands, neck).

[0022] As used herein, “skin wrinkles” generally refers to folds, raised areas, or creases within the skin, and includes, but is not limited to, fine wrinkles, ultrafine wrinkles, fine wrinkles, ultrafine wrinkles, wrinkles, and creases. Skin wrinkles may be measured, for example, with respect to density and / or length.

[0023] As used herein, "skin sheen" generally refers to the amount of light reflected by the skin and is sometimes called skin gloss.

[0024] As used herein, “skin texture” generally refers to the topography or roughness of the skin surface.

[0025] As used herein, "skin firmness" generally refers to the firmness or elasticity of the skin.

[0026] As used herein, “skin sebum level” generally refers to the amount of sebum, which is an oily or waxy substance secreted by the sebaceous glands in the skin.

[0027] As used herein, “skin spots” generally refers to discoloration or uneven pigmentation of the skin (e.g., hyperpigmentation, plaques). Skin spots may be evaluated, for example, in terms of density, size, and / or degree of discoloration.

[0028] As used herein, “skincare product” means a product that contains skincare active ingredients and modulates and / or improves the condition of the skin.

[0029] As used herein, “digital image” refers to a digital image formed by pixels in an imaging system, including but not limited to standard RGB, and under different lighting conditions and / or modes. Non-limiting examples of digital images include color images (RGB), monochrome images, video, multispectral images, and hyperspectral images. Non-limiting lighting conditions include light sources emitting white light, blue light, UV light, IR light, light of specific wavelengths, e.g., 100-1000 nm, 300-700 nm, 400-700 nm, or different combinations of the above upper and lower limits, or any integer combination within the ranges listed above. Digital images may be acquired by image acquisition devices, including but not limited to digital cameras, photographic scanners, computer-readable storage media capable of storing digital images, and any electronic devices including photographic capabilities.

[0030] In the following description, the systems, methods, and apparatus described are systems, methods, and apparatus for determining cosmetic skin attributes based on specific color impairment values ​​of human skin.

[0031] In an exemplary embodiment, the system is a standalone imaging system (shown in Figure 2) located at a retail cosmetics counter for the purpose of analyzing and / or recommending cosmetic and skincare products based on cosmetic skin attributes determined based on fault values. However, it is conceivable that the system and method may be configured for use at any location via an electronic portable device comprising an image acquisition unit and a display, as shown, for example, in Figure 1, the electronic portable device being connected to a device for generating a display on the display and a graphical user interface for visualizing cosmetic skin characteristics via a network.

[0032] system Figure 1 is a schematic diagram showing a system 10 according to the present invention for determining cosmetic skin attributes based on fault values, and optionally for visualizing fault values ​​and / or cosmetic skin attributes. System 10 may include a network 100 which can be embodied as a wide area network (such as a mobile telephone network, public switched telephone network, satellite network, or the Internet), a local area network (such as Wireless Fidelity, Wi-Max, ZigBee®, or Bluetooth®), and / or other forms of network functionality. Network 100 is coupled with a portable electronic device 12, a device 14 for generating a display on a screen, and a graphical user interface for visualizing cosmetic skin attributes. The device 14 is located remotely and is connected to the portable electronic device via network 100.

[0033] The portable electronic device 12 may be a mobile phone, tablet, laptop, personal digital assistant, and / or other computing device configured to capture, store, and / or transfer digital images such as digital photographs. Therefore, the portable electronic device 12 may include an input device 12a for receiving user input, an image acquisition device 18 such as a digital camera for acquiring images, and an output device 12b for displaying images. The portable electronic device 12 may also be configured to communicate with other computing devices via a network 100. The portable electronic device 12 may further include an image processing device (not shown) coupled to the image acquisition device 18 for analyzing at least one obtained color image to acquire fault values ​​and determining cosmetic skin attributes of at least a portion of a person's skin based on the fault values. The image processing device preferably includes a processor having computer executable instructions. The portable electronic device 12 may further include a display generation unit (not shown, such as an electronic LED / LCD display) for generating a display to display content data describing the determined cosmetic skin attributes.

[0034] The device 14 may include a non-temporary computer-readable storage medium 14a (hereinafter referred to as the "storage medium") that stores image acquisition logic 144a, image analysis logic 144a, and graphical user interface (hereinafter referred to as the "GUI") logic 144c. The storage medium 14a may include random access memory (SRAM, DRAM, etc.), read-only memory (ROM), registers, and / or other forms of computing storage hardware. The image acquisition logic 144a, image analysis logic 144b, and GUI logic 144c define computer-executable instructions. The processor 14b is coupled to the storage medium 14a and is configured to implement a method 90 for determining cosmetic skin attributes according to the present invention, as described later with respect to the flowchart in Figure 4, based on the computer-executable instructions.

[0035] method Referring to Figure 4, once the processor is started, in step 91 the processor obtains at least one color channel image, preferably one color channel image comprising at least a portion of human skin, by, for example, converting a digital image to a color channel image in a color system described later with reference to Figure 5. The at least one color channel image is analyzed in step 92 to obtain fault values. In step 93, the cosmetic skin attributes of at least a portion of human skin are determined based on the fault values.

[0036] Preferably, the fault value is based on a color gradient.

[0037] Alternatively, and / or simultaneously, the fault value is preferably based on at least one of the following: the total length of the edge of a particular color, the ratio of the longest radius to the shortest radius, both radii measured from the same center of the particular color; the difference between tiles of a particular color; and a mixture thereof. More preferably, the fault value is based on at least one of the following: the total length of the edge of a particular color, the ratio of the longest radius to the shortest radius, both radii measured from the same center of the particular color; even more preferably, the fault value is based on the ratio of the longest radius to the shortest radius, both radii measured from the same center of the particular color.

[0038] At least one color channel image is L * a * b * The image may be within a color system selected from the group consisting of a color system, an RGB color system, an HSL / HSV color system, and a CMYK color system.

[0039] Referring to Figure 1, the network 100 may be used to acquire a digital image from the portable electronic device 12 and transmit the digital image to the apparatus 14 used in the method 200 according to the present invention. An input device 12a may be coupled to or integrated with the portable electronic device 12 to receive user input to start the processor 14b. The portable electronic device 12 may include an output device 12b for displaying a plurality of tiles, each having a single degree of indicium uniquely assigned. The input device 12a may include, but is not limited to, a mouse, a touchscreen display, etc. The output device 12b may include, but is not limited to, a touchscreen display, a non-touchscreen display, a printer, or a projector for projecting the face image map 30 onto a display surface, such as a mirror, as described below with respect to Figure 2.

[0040] Figure 2 is a perspective view of system 10 configured as an exemplary standalone imaging system, located at a retail cosmetics counter for the purpose of visualizing color gradients and / or at least one cosmetic skin attribute, and may also be for the purpose of recommending cosmetics and skincare products based on the visualized color gradients and / or at least one cosmetic skin attribute. Figure 3 is a block diagram of the exemplary system 10 of Figure 2. Referring to Figures 2 and 3, system 10 includes a housing 11 for the apparatus 14 of Figure 1, connected to an image acquisition device 18 for acquiring a digital image of a target for visualizing at least one cosmetic skin attribute. Referring to Figure 2, system 10 may also include a mirror 16, and the image acquisition device 18 may be mounted behind the mirror 16 in the housing 11 so that the image acquisition device 18 can be hidden from view. The image acquisition device 18 may be a digital camera, an analog camera connected to a digitization circuit, a scanner, a video camera, etc. System 10 may include lighting 30, such as LED lighting, arranged around the housing 11, to form an LED lighting system to assist in the generation of a digital image of a target. System 10 has an input device 112a for receiving user input. System 10 may further include an output device 112b, such as a projector, configured to receive and project a face map 30 for display on the mirror 16. The projector is separate from the housing 11 but may be a peripheral component coupled to the device 14 to form System 10 and is therefore not shown in Figure 2. System 10 may further include a second output device 112c, such as one or more speakers optionally coupled to an amplifier for generating audio guidance output, to complement and / or enhance the overall consumer experience.

[0041] To explain how System 10 and Method 90 function to determine and visualize at least one cosmetic skin attribute according to the present invention, it is useful to understand the details of the units / steps included in the System and Method. Therefore, each step is described as a separate process for performing each step. Each process can also be described as a subroutine, i.e., a sequence of program instructions that perform the corresponding step according to Method 90 according to the present invention.

[0042] Acquisition of digital images The process 202 for acquiring a digital image by method 200 according to the present invention is described with reference to a series of process flow diagrams showing how digital image data is acquired from the digital image, with reference to Figures 7A, 7B, and 7C. Figure 8 is a flowchart of the process 400 for acquiring digital image data corresponding to process 202.

[0043] An input image 50a of face 1 is shown in Figure 7A. The input image 50a may be captured by the user using the camera 18 in step 402 of process 400, for example, as shown in Figure 8. Figure 7B shows step 404, in which the input image 50a is cropped to obtain edited image data 50b that includes at least a portion of the face. The input image 50a may be cropped by identifying an anchor feature 1a of the face, which includes, but is not limited to, facial features such as the eyes, nose, nostrils, and corners of the mouth, and cropping accordingly. As shown in Figure 7B, the eyes are shown as an anchor feature 1a, but this is just an example, and it will be understood that any prominent or detectable facial feature can be an anchor feature. The edited image data 50b may be a first digital image 51 obtained in step 404. Alternatively, as shown in Figure 7C, the edited image data 50b may be further processed by trimming to remove one or more unnecessary portions of the input image 50a, thereby obtaining a first digital image data 51 that includes at least a portion of face 1 defined by the boundary line 52 in step 408. Preferably, the first digital image is a cross-polarized image. The obtained first digital image 51 may include at least one region of interest (ROI) 2 of at least a portion of face 1 defined by the boundary line 52. The ROI 2 may be one or more skin regions that define the entire face 1, preferably at least a portion of the face, more preferably at least a portion of face 1. Details of the definition of skin regions will be described later with reference to the flowcharts in Figures 7 and 10.

[0044] Optionally, process 400 may select ROI2 from the skin area around the cheek ("cheek area 2b"), preferably ROI2 is a portion of at least a portion of the face 1 of the subject, and more preferably, step 406 may define the left or right side of the face 1 obtained as the first digital image data. ROI2 may include at least 5%, 10% to 100%, or 25% to 90% of the obtained first digital image data.

[0045] Definition of a tile In some embodiments, image data, preferably ROI, is divided into tiles having a defined tile size. Figure 9 is a photograph showing a plurality of tiles 54 on a first digital image data 51. Figure 10 is a flowchart of the process 500 for defining the plurality of tiles 54 on the first digital image data 51. Referring to Figure 9, the first digital image 51 includes at least a portion of a face 1 defined by a boundary line 52, as described above with reference to Figure 7C. Referring to Figure 10, the process 500 includes defining a perimeter 53 that encloses the boundary line 52 surrounding the acquired first digital image (step 502). The acquired first digital image 51 is formed by a total number of pixels, and for example, the acquired first digital image 51 may have a number of pixels determined in step 404 or step 406, depending on the cropped image size of the input image 50a. Thus, the overall image size based on the acquired first digital image 51 may be defined in step 504. For example, if the tile size is set to 40 x 40 pixels to 70 x 70 pixels, in step 506, the number of tiles 54 that form multiple tiles 54 across the acquired first digital image 51 is obtained by dividing the total image size by the specific tile size. Alternatively, or simultaneously, the area of ​​one tile may form about 1% to about 20%, preferably about 1% to about 10%, and more preferably about 1% to about 5% of the ROI area.

[0046] Acquisition of a color channel image Color channel images can be obtained from digital images, as described below with reference to Figure 5.

[0047] Referring to Figure 5, when processor 14b is started, in step 202, processor 14b acquires a digital image 51 of at least a portion of the target face, for example, via image acquisition logic. The acquired digital image 51 may be an RGB cross-polarized digital image or an RGB glossy digital image. The digital image 51 of the RGB system is converted from an RGB image to digital image data such as a color channel image in a different color system. Processor 14b further causes the acquired digital image 51 to be extracted, for example, via image analysis logic 144b, in step 204. The at least one color channel image may be selected from any one of the color channels of the color system. In step 206, the extracted at least one color channel image is filtered using a frequency filter. The filtered at least one color channel image is analyzed in step 208 to obtain fault values ​​and determine the person's cosmetic skin attributes. By using a frequency filter in step 206, noise is removed from at least one extracted color channel image, which increases the sensitivity of the analysis in step 208, thereby resulting in higher accuracy in the analysis output from step 208 compared to analyzing an unfiltered color channel image. However, analyzing an unfiltered color channel image may be advantageous in reducing the use of computing hardware, such as reducing the hardware footprint, data storage space, or processing power, when minimal and basic hardware is available to implement the method according to the present invention.

[0048] Analyzing image data may include analyzing at least two color channels, particularly the red and yellow color channels.

[0049] Method 200 may further include a step of comparing at least one beauty attribute with a predefined dataset and assigning an index. The index may be displayed in a further step following the comparison step.

[0050] Image data analysis Figure 5 is a flowchart of the process 200 for analyzing image data according to the present invention. The process 200 may include step 204 of extracting at least one color channel from an acquired first digital image to obtain a fault value and providing an extracted color channel image for analysis to determine cosmetic skin attributes based on the fault value.

[0051] In the following description, at least one color channel image is selected from the group consisting of an L color channel image, an a-channel image, an a-b-channel image, and combinations thereof, preferably an a-channel image, an a-b-channel image, and combinations thereof, more preferably an a-channel image (red). * a * b * This is an image of a color system. However, it will be understood that at least one color channel may also be a chromophore system, and at least one color channel may be a melanin channel or a hemoglobin channel. The color system may also be an HSL / HSV color system and a CMYK color system.

[0052] The extracted color channels may be filtered, and the filtered color channels may be analyzed for impairment values ​​or cosmetic skin attributes based on impairment values. It will be understood that the filtered color channels may also be analyzed using other descriptive statistics, including but not limited to standard deviation and mean. The technical effect of using impairment values ​​according to the present invention is that they have a higher correlation with a person's skin tone or skin condition, and / or the perception of a person's skin tone or skin condition.

[0053] Preferably, the fault value is based on the color gradient. Alternatively and / or simultaneously, the fault value is preferably based on the total length of the edge of a particular color, the ratio of the longest radius to the shortest radius, where both radii are measured from the same center of the particular color, the ratio, the number of centers of the particular color, the length of the radius from one center of the particular color, the difference between tiles of the particular color, and their mixture, more preferably based on the total length of the edge of the particular color. Figure 6 shows a conventional color analysis as a comparative example (a * The above preferred impairment values ​​obtained for each of the three different skin samples are shown compared to the mean.

[0054] Preferably, the first digital image, more specifically the color channel image, is filtered by using a smoothing filter, preferably a Gaussian filter and / or a frequency filter, more preferably a Gaussian difference (DoG) filter among frequency filters, to help remove noise that occurs during the image acquisition process. In particular, frequency filters are useful for separately evaluating the spatial patterns of color and topographic features. Optionally, method 200 may further include applying image correction coefficients to the filtered color channels before analyzing the filtered color channels.

[0055] In step 208, the cosmetic skin attributes of at least a portion of the human skin are determined based on the impairment value.

[0056] Preferably, the fault value is based on the color gradient, 1) The step of selecting a region of interest (ROI) on at least one color channel image, 2) The process of defining multiple tiles across ROI, 3) A step of calculating the average intensity value of a specific color for each tile. 4) A step of calculating the gradient of the average intensity value between adjacent tiles using the following formula, i,j This is the average strength value of the tile at position (i,j) calculated in step (3) above, and I i+1,jThis is the average intensity value of the tile at position (i+1,j) calculated in step (3) above, and I i,j+1 This is the average intensity value of the tile at position (i,j+1) calculated in step (3) above.

[0057]

number

[0058]

number

[0059] Table 1 below shows the fault values ​​based on each color gradient with corresponding color channel images, and the preferred corresponding cosmetic skin attributes that should be determined based on the fault values ​​based on each color gradient. The color channel images listed in Table 1 are selected from the group consisting of L channel images, a-channel images, a-b-channel images, and combinations thereof. * a * b * This is an image of a color system.

[0060] [Table 1]

[0061] Preferably, the color channel image is an alpha channel image, the impairment value is determined based on the alpha gradient, and the preferred cosmetic skin attributes to be determined are selected from the group consisting of stressed skin, healthy skin, inflamed skin, and hidden aging skin.

[0062] The inventors have found that the a-slope also indicates vascular status; for example, a lower a-slope indicates a normal vascular status. A moderate a-slope indicates more vasodilation (temporary), which is a signal of transient inflammation, while a higher a-slope indicates more vasodilation (temporary) and angiogenesis (chronic), which are signals of chronic inflammation.

[0063] Preferably, the cosmetic skin attribute is generated as a value that indicates the state of cosmetic skin attributes of at least some of a person's skin for a defined group of people.

[0064] Specifically, in a visual perception test, consumers may be asked to rank digital images (e.g., photographs) of a defined group of people with specific cosmetic skin attributes based on a given scale. The ranked digital images may be stored as a database for analysis.

[0065] Preferably, the cosmetic skin attribute is generated as a function of the fault value of at least one color channel image defined by F (fault value), the function being determined by a model established for the training dataset, the training dataset comprising (i) a plurality of color channel images of a defined population of people, each of which comprises the facial skin of a person within the defined population of people, and (ii) association class definitions based on the cosmetic skin attribute. More preferably, the cosmetic skin attribute is generated as a function of the fault value in combination with the base skin color in a tile defined by F (fault value, base skin color).

[0066] Preferably, the target age and the average age of the defined population of people may be independently 18 to 60 years, preferably 20 to 40 years, more preferably 25 to 35 years, and even more preferably 28 to 32 years.

[0067] The techniques for constructing the training dataset are known to those skilled in the art in the field of image processing methods and will not be further explained.

[0068] The model is either a regression model or a classification model. Preferably, the model is a classification model, more preferably a machine learning classification model, and most preferably a random forest classification model or a gradient boosting classification model.

[0069] The use of machine learning models enables advantages in terms of accuracy, repeatability, and performance speed when implemented as a native application on mobile electronic devices. Specifically, the weight of the SVR model allows the native application to have a smaller hardware footprint, and as a result, the methods according to the present invention can be easily deployed on mobile electronic devices such as mobile phone operating systems (OS) including, but not limited to, iOS for Apple® phones or Android OS for Android phones.

[0070] The classification model can be used to categorize consumers into multiple groups, each group having different degrees of the same cosmetic skin attribute, preferably two groups, and thus define the relevant class definitions based on visual grading or any other numerical value of the cosmetic skin attribute. For example, the method may display heatmaps configured to classify areas of skin into high-level cosmetic skin attribute states or low-level cosmetic skin attribute states based on thresholds assigned to each group.

[0071] The following data was generated based on correlations with results from visual perception tests using statistical analysis with the Pearson correlation coefficient (r). The correlation results are shown in Table 2 below.

[0072] [Table 2]

[0073] A higher Pearson correlation coefficient (r) indicates that the gradient value is a more contributing factor to the condition of the cosmetic skin attribute being studied in the visual perception study. Specifically, the visual perception test is conducted based on a predetermined number of panelists = 577, with panelists aged 20-50. Panelists are asked to rate each cosmetic attribute (as an example of cosmetic skin attributes), such as stressed skin, on a scale of 1-6.

[0074] Based on the results of visual perception tests and the correlation results described above, it has been found that the impairment values ​​of filtered images (by frequency filtering) have a higher correlation with the above-mentioned cosmetic skin attributes. Therefore, the use of impairment values ​​to determine the cosmetic skin attributes of at least some of the human skin in digital images can be used to transform cosmetic skin attributes from visually imperceptible to explainable in a way that is relevant to consumers.

[0075] display The method according to the present invention may further include a step of displaying a defect value and / or cosmetic skin attributes based on the defect value. The method according to the present invention may further include a step of generating an image description corresponding to the generated defect value described above in order to visualize the cosmetic skin condition. The image description may include a heat map (such as the one shown in Figure 13B), an aggregate score (such as feature 934 in Figure 15), and a combination thereof. The aggregate score can be calculated based on the generated defect value described above.

[0076] Figure 11 is a photograph showing a second digital image 60 interposed on a first digital image data 51. The second digital image 60 includes at least a portion of a subject's face displaying a plurality of tiles 54, each having a single degree 40 of an indicium uniquely assigned to it.

[0077] Figure 12 is a flowchart of process 600, which displays a plurality of tiles in step 306 of method 300 according to the present invention. Process 600 may begin with step 602, in which a processor reads the analyzed image data of each tile 54 and assigns a unique single degree of indicium to each tile 54 of the plurality of tiles based on the analyzed at least one visually aesthetic skin attribute of the tile 54. When the single degree of indicium is illumination, the analyzed image data of each tile may be transformed in step 606 to reflect the corresponding degree of brightness of illumination at each tile. In an exemplary example, a tile with a higher illumination level has a higher impairment value using a color gradient value, but has a worse condition in at least one of the aesthetic skin attributes, compared to a tile with a lower illumination level but has a better condition in at least one of the aesthetic skin attributes. Specifically, method 300 may further include displaying at least one product recommendation item for treating the displayed aesthetic skin attributes.

[0078] Figure 14 is a flowchart of method 700 for visualizing at least one cosmetic skin attribute according to the present invention. Figure 13A is a color photograph showing a first digital image of at least a portion of the face of a subject displayed in step 702 of method 700 in Figure 14. Figure 13B is a color image showing a second digital image of at least a portion of the face of a subject, and a plurality of tiles, each having a single degree of indicium uniquely assigned, the second digital image being interposed on the first digital image in step 704. Figure 13B is an example of visualization of inflamed skin and / or hidden aging skin based on a distress value using a color gradient according to the present invention, which may also be a prediction of skin pigmentation such as spot location / melanin location. In Figure 13B, whiter tiles correspond to worse skin condition, and darker tiles correspond to better skin condition. This visualization of cosmetic skin attributes according to the present invention is a *It provides an improved match to human skin tone or skin condition, and / or an improved match to human perception of human skin tone or skin condition, compared to other visualizations such as averages and spots.

[0079] Human-machine user interface The present invention also relates to a human-machine user interface (hereinafter, "user interface") for providing product recommendations for treating at least one cosmetic skin attribute. The user interface may be a graphical user interface on a portable electronic device including a display having a touchscreen display / input device and an image acquisition device. The user interface may include a first area of ​​the touchscreen display that displays a first digital image of at least a portion of a subject's face acquired from the image acquisition device and a second digital image interposed on the first digital image, wherein the second digital image has at least a portion of the subject's face, and each of the displayed tiles has a single degree of a uniquely assigned indicium. The user interface may further include a second area of ​​the touchscreen display distinct from the first area, the second area displaying selectable icons for receiving user input, and images of at least one product recommendation item for treating the displayed cosmetic skin attribute are displayed on the touchscreen display when the user activates a selectable icon.

[0080] Figure 15 is a screenshot illustrating an exemplary user interface 930 for visualizing at least one cosmetic skin attribute according to the present invention, wherein the at least one cosmetic skin attribute is "hidden aging". The user interface 930 includes alternative text 932 describing the cosmetic skin attribute and an aggregate score 934 based on a defect value. The user interface 930 may further include a meter 936 and a meter marker 938 for representing the aggregate score on a scale of 0 to 100 along the meter 936. The meter 936 is a different method of visualizing the aggregate score 934 and may be optional.

[0081] The method for determining cosmetic skin condition according to the present invention described above may further include the step of tracking cosmetic skin attributes over a predetermined period of time, for example, by generating a calendar or schedule for creating a cosmetic skin attribute diary to track improvements in cosmetic skin attributes. For example, when a consumer uses the method on day 1, the date and facial analysis are recorded and stored in memory. Subsequently, whenever the consumer uses the method according to the present invention in the future (after a predetermined period, one week, one month, six months), the consumer's facial skin is analyzed again, and the consumer can compare how their facial skin looks after the predetermined period to day 1. The method according to the present invention may be configured as a downloadable software application stored as a native application on a mobile electronic device or a web application that can be accessed via a consumer-specific login account, so that the consumer can perform self-skin analysis based on the method according to the present invention and see and / or monitor improvements over a period of time (reduction in ROI with worse cosmetic skin attribute conditions).

[0082] The user interface 930 may further include a second selectable icon 942 that, when selected, allows the method for determining cosmetic skin attributes according to the present invention to be repeated. For example, the above method 90 may be repeated.

[0083] combination Typical embodiments of the present disclosure described above may be described as follows in the following paragraphs. 1. A method for determining the cosmetic skin attributes of a person, the method being: a) A step of obtaining at least one color image comprising at least a portion of human skin, b) A step of analyzing at least one color image to obtain a specific color failure value, c) A method comprising the step of determining the cosmetic skin attributes of at least a portion of a person's skin based on a disability value. 2. The failure value is, The total length of the edge of a specific color, The ratio of the longest radius to the shortest radius, where both radii are measured from the same center of a particular color. Differences in specific colored tiles, and a method according to any of the features described above, based on at least one of these mixtures. 3. The failure value is, The total length of the edge of a specific color, The ratio of the longest radius to the shortest radius, where both radii are measured from the same center of a particular color. and a method according to any of the features described above, based on at least one of these mixtures. 4. The method described above, in which the fault value is based on the ratio of the longest radius to the shortest radius, and both radii are measured from the same center of a particular color. 5. The method described in any of the above characteristics, wherein the fault value of a specific color is the fault value of red. 6. The method according to any of the features described above, wherein at least one color channel image is at least one color channel image. 7. At least one color channel image is L * a * b * The method according to any of the above-described features, wherein the image is of a color system selected from the group consisting of a color system, an RGB color system, an HSL / HSV color system, and a CMYK color system. 8. At least one color channel image is L * a * b * A method of any of the aforementioned features, which is an image channel of a color system. 9. Step (b) is, 1) A step of selecting a region of interest (ROI) on at least one color channel image, 2) The process of defining multiple tiles across ROI, 3) A step of calculating the average intensity value of a specific color for each tile, 4) A step of calculating the gradient of the average intensity value between adjacent tiles using the following formula, i,j This is the average strength value of the tile at position (i,j) calculated in step (3) above, and I i+1,j This is the average intensity value of the tile at position (i+1,j) calculated in step (3) above, and I i,j+1 This is the average intensity value of the tile at position (i,j+1) calculated in step (3) above,

[0084]

number

[0085]

number

[0086] All documents referenced herein, including any patents or patent applications that are cross-referenced or related, and any patent applications or patents for which this application claims priority or benefit thereof, are incorporated herein by reference in their entirety unless expressly excluded or otherwise limited. No reference to any document shall be deemed prior art to any invention disclosed or claimed herein, nor shall it be deemed to teach, suggest or disclose any such invention, either alone or in combination with any other reference(s). Furthermore, to the extent that any meaning or definition of a term in this document conflicts with any meaning or definition of the same term in any document incorporated by reference, the meaning or definition given to that term in this document shall prevail.

[0087] While specific embodiments of the present invention have been illustrated and described, it will be apparent to those skilled in the art that various other changes and modifications can be made without departing from the spirit and scope of the invention. Therefore, it is intended that all such changes and modifications within the scope of the invention be covered in the appended claims.

Claims

1. A method for determining a person's cosmetic skin attributes, wherein the method is a) A step of acquiring at least one color image comprising at least a portion of the person's skin, b) A step of analyzing the at least one color image to obtain a fault value for a specific color, c) A method comprising the step of determining the cosmetic skin attributes of at least a portion of the person's skin based on the impairment value.

2. The aforementioned fault value is, The total length of the edge of the aforementioned specific color, The ratio of the longest radius to the shortest radius, where both radii are measured from the same center of the specified color, The differences in the aforementioned specific color tiles, and mixtures thereof, Based on at least one of the following, Preferably, the fault value is The total length of the edge of the aforementioned specific color, The ratio of the longest radius to the shortest radius, where both radii are measured from the same center of the specified color, and mixtures thereof, Based on at least one of the following, The method according to claim 1, more preferably, the fault value is based on the ratio of the longest radius to the shortest radius, and both radii are measured from the same center of the particular color.

3. The method according to any one of claims 1 to 2, wherein the fault value of the specific color is the fault value of red.

4. At least one color image is at least one color channel image, preferably the at least one color channel image is L ＊ a ＊ b ＊ The image is of a color system selected from the group consisting of a color system, an RGB color system, an HSL / HSV color system, and a CMYK color system, and more preferably, the at least one color channel image is L ＊ a ＊ b ＊ The method according to any one of claims 1 to 3, wherein the image channel is a color system.

5. The above step (b) is, 1) A step of selecting a region of interest (ROI) on the at least one color channel image, 2) A step of defining multiple tiles across the ROI, 3) A step of calculating the average intensity value of the specific color for each tile, 4) A step of calculating a gradient of the average intensity value between adjacent tiles by the following equation, where I ｉ，ｊ is the average intensity value of the tile at the position (i, j) calculated in the step (3), and I ｉ＋１，ｊ is the average intensity value of the tile at the position (i + 1, j) calculated in the step (3), and I ｉ，ｊ＋１ is the average intensity value of the tile at the position (i, j + 1) calculated in the step (3), and [Math 1] 5) A step of calculating the fault value using the following formula, S ＲＯＩ This is the total number of tiles in the ROI, [Math 2] Executed by The method according to any one of claims 1 to 4.

6. The method according to claim 5, wherein step (4) is performed for all tiles within the ROI.

7. The method according to any one of claims 1 to 6, wherein, prior to step (b), the at least one color channel image is filtered by using a smoothing filter and / or a frequency filter.

8. The method according to any one of claims 1 to 7, wherein the cosmetic skin attributes are selected from the group comprising skin age, skin topography, skin tone, skin pigmentation, skin pores, skin inflammation, skin hydration, skin sebum level, acne, moles, skin luster, skin shine, skin dullness, and skin barrier, prediction of future cosmetic skin attributes, and mixtures thereof.

9. The aforementioned cosmetic skin attributes are generated as values ​​indicating the state of the aforementioned cosmetic skin attributes of at least a portion of the skin of a defined population of people, and are generated as a function of the fault value of at least one image defined by F (fault value). The function is determined by a model established based on a training dataset, the training dataset comprising: (i) a plurality of images of the defined population of people, each of which includes the facial skin of a person within the defined population of people; and (ii) association class definitions based on the cosmetic skin attributes. Preferably, the cosmetic skin attribute is generated as a function of the impairment value in combination with the base skin color in the tile defined by F (impairment value, base skin color), Preferably, the model is a regression model or a classification model, and the model is preferably a classification model, more preferably a machine learning classification model, most preferably a machine learning random forest classification model or a gradient boosting classification model, according to any one of claims 1 to 8.

10. The method according to any one of claims 1 to 9, wherein the at least one color channel image is an a-image and the fault value is an a-fault value.

11. A system for determining a person's cosmetic skin attributes, wherein the apparatus is An image acquisition unit for acquiring at least one image including at least a portion of the skin of the person, preferably the image acquisition device comprises a non-temporary computer-readable storage medium configured to store the acquired at least one color image, An image processing unit coupled to the image acquisition unit analyzes the acquired at least one image to obtain a defect value and determines the cosmetic skin attributes of at least a portion of the person's skin based on the defect value, A system comprising: an image acquisition unit coupled to a display generation unit for generating a display to show content data describing the determined cosmetic skin attributes.

12. The system according to claim 11, wherein the image processing device comprises a processor having computer executable instructions.

Images