Skin condition estimation device

Abstract

To provide a device that can estimate the state of skin by easier procedures.SOLUTION: The estimation device includes an image acquisition unit and an estimation unit. The image estimation unit acquires an image of a sample of a horny layer collected from the surface of skin taken by a microscope. The estimation unit derives output from a machine learning model by inputting the acquired image into a learned machine learning model, and estimates at least one of an index showing the state of the skin and advice on a daily care suitable for the skin on the basis of the output.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The present invention relates to a skin condition estimation device. [Background technology]

[0002] Conventionally, methods for objectively evaluating skin conditions have been developed. Such methods are applied to cosmetic sales, interviews at dermatologists or aesthetic salons, and the like.

[0003] Patent Document 1 discloses a method for evaluating skin condition by measuring components in a keratin sample collected from a subject. The evaluation results of the skin condition can be used by, for example, a beauty professional to provide advice on skin care suitable for the subject. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 10-216106 DISCLOSURE OF THEINVENTION [Problem to be solved by the invention]

[0005] In the method disclosed in Patent Document 1, the components contained in the collected keratin are directly measured. Therefore, it is necessary to extract and isolate the components contained in the keratin, which is very time-consuming, so there is still a need for a method and device that can more easily evaluate the skin condition. The present invention has been made to solve such problems, and its purpose is to provide a device, method, program, etc. that can estimate the skin condition with a more simple procedure. [Means for solving the problem]

[0006] The skin condition estimation device according to the present invention includes an image acquisition unit and an estimation unit. The image acquisition unit acquires an image of a stratum corneum sample taken from the skin surface using a microscope. The estimation unit inputs the acquired image into a trained machine learning model to derive an output from the machine learning model, and estimates at least one of an index representing the skin condition and advice on skin care habits suitable for the skin based on the output.

[0007] In addition, the output derived from the machine learning model may include at least one of data representing an index representing a skin condition and data representing advice on skin care habits suitable for the skin. The data may be numerical data, text data, image data, etc.

[0008] In the estimation device, the skin condition is estimated based on an image of a stratum corneum sample taken from the surface of the skin, captured by a microscope. Therefore, it is not necessary to directly measure a specific component contained in the collected stratum corneum sample. Therefore, at least one of an index representing the skin condition of the person who provided the stratum corneum sample and advice on skin care habits suitable for the skin is estimated without going through a time-consuming process.

[0009] In the estimation device, the image may include an image captured by a polarizing microscope.

[0010] In the estimation device, the index representing the skin condition may be a unit ceramide amount representing an amount of ceramide per keratinocyte contained in the stratum corneum sample.

[0011] In the estimation device, the estimation unit may estimate at least one of an index representing the skin condition and advice on skin care habits suitable for the skin using a machine learning model trained using learning data, which is a dataset of images of a stratum corneum sample taken by a microscope and ground truth data, and the ground truth data may be at least one of an index representing the skin condition based on values ​​measured from the stratum corneum sample, a dataset of ceramide amounts and protein amounts measured from the stratum corneum sample, and advice on skin care habits suitable for the skin.

[0012] In addition, the index representing the skin condition based on the value measured from the stratum corneum sample included in the correct answer data may be data representing the index, and the advice on skin care habits suitable for the skin may be data representing the advice. The data may be numerical data, text data, image data, etc.

[0013] The estimation device may further include a result output unit that outputs a result estimated by the estimation unit.

[0014] The method for estimating a skin condition according to the present invention includes the following (1) and (2). (1) Obtaining images of stratum corneum samples taken from the surface of the skin using a microscope. (2) Inputting the acquired image into a trained machine learning model to derive an output from the machine learning model, and estimating at least one of an indicator of skin condition and advice regarding skin care habits suitable for the skin based on the output.

[0015] In the estimation method, the image may include an image captured by a polarizing microscope.

[0016] In the estimation method, the index representing the skin condition may be a unit ceramide amount representing an amount of ceramide per keratinocyte contained in the stratum corneum sample.

[0017] The skin condition estimating program according to the present invention causes a computer to execute the following (1) and (2). (1) Obtaining images of stratum corneum samples taken from the surface of the skin using a microscope. (2) Inputting the acquired image into a trained machine learning model to derive an output from the machine learning model, and estimating at least one of an indicator of skin condition and advice regarding skin care habits suitable for the skin based on the output.

[0018] In addition, the output derived from the machine learning model may include at least one of data representing an index representing a skin condition and data representing advice on skin care habits suitable for the skin. The data may be numerical data, text data, image data, etc.

[0019] In the estimation program, the image may include an image captured by a polarizing microscope.

[0020] In the estimation program, the index representing the skin condition may be a unit ceramide amount representing an amount of ceramide per keratinocyte contained in the stratum corneum sample.

[0021] The learning device according to the present invention includes a memory unit and a learning unit. The memory unit stores learning data, which is a data set of an image of a stratum corneum sample taken from the surface of the skin captured by a microscope and ground truth data. The learning unit uses the learning data to train a machine learning model. The ground truth data is at least one of an index representing a skin condition based on a value measured from the stratum corneum sample, a data set of ceramide amounts and protein amounts measured from the stratum corneum sample, and advice on skin care habits suitable for the skin.

[0022] In the learning device, the image may include an image captured by a polarizing microscope.

[0023] In the learning device, the index representing the skin condition may be a unit ceramide amount representing the amount of ceramide per keratinocyte contained in the stratum corneum sample.

[0024] The method for generating a trained model according to the present invention includes the following steps (1) and (2). (1) A step of preparing learning data, which is a data set of images taken under a microscope of a stratum corneum sample taken from the surface of the skin and ground truth data. (2) A step of performing machine learning using the learning data so that when an image of a stratum corneum sample taken from the surface of the skin taken with a microscope is input, data corresponding to the ground truth data is output. In addition, the correct data may be at least one of an index representing the condition of the skin based on values ​​measured from the stratum corneum sample, a dataset of the amounts of ceramide and protein measured from the stratum corneum sample, and advice regarding skin care habits suitable for the skin.

[0025] In addition, the index representing the skin condition based on the value measured from the stratum corneum sample included in the correct answer data may be data representing the index, and the advice on skin care habits suitable for the skin may be data representing the advice. The data may be numerical data, text data, image data, etc.

[0026] In the method for generating a trained model, the image may include an image captured by a polarizing microscope.

[0027] In the method for generating a trained model, the index representing the skin condition may be a unit ceramide amount representing the amount of ceramide per keratinocyte contained in a stratum corneum sample. Effect of the Invention

[0028] According to the present invention, the skin condition of a subject is estimated based on a microscopic image of a stratum corneum sample taken from the subject, making it possible to easily estimate the skin condition and provide advice on skin care habits suitable for the subject. [Brief description of the drawings]

[0029] [Figure 1] FIG. 2 is a block diagram showing the electrical configuration of the estimation device according to the embodiment of the present invention. [Diagram 2] FIG. 1 is a diagram for explaining the principle of the present invention. [Figure 3A] FIG. 1 is a diagram for explaining the principle of the present invention. [Figure 3B] FIG. 1 is a diagram for explaining the principle of the present invention. [Figure 4] FIG. 2 is a block diagram showing the electrical configuration of a learning device according to an embodiment of the present invention. [Diagram 5] 13 is a flowchart showing the procedure of an estimation process. [Figure 6] 3A to 3C are diagrams illustrating the operation of an estimation device according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0030] <1. Overview of the estimation device> Hereinafter, a skin condition estimation device, an estimation method, an estimation program, and a method for generating a machine-learned model according to one embodiment of the present invention will be described with reference to the drawings.

[0031] The estimation device 1 shown in FIG. 1 is one embodiment of the estimation device according to the present invention. The hardware of the estimation device 1 is a general-purpose computer. An estimation program 20 provided from a computer-readable recording medium such as a CD-ROM or a USB memory or via a network or the like is installed in the estimation device 1. The estimation program 20 according to this embodiment is application software that estimates an index representing a skin condition and advice regarding a skin care habit suitable for the skin from an image of a stratum corneum sample. The estimation program 20 causes the estimation device 1 to execute steps included in the operation described below.

[0032] The estimation device 1 includes a storage unit 2, a control unit 3, a display unit 6, an input unit 4, and a communication unit 5. These units 2 to 5 are connected to each other via a communication line 8 such as a bus line or a cable, and can communicate with each other as needed. The communication unit 5 communicates with an external device such as a microscope. The display unit 6 is composed of a liquid crystal display or the like, and displays information to a user, which will be described later. The input unit 4 is composed of a mouse, a keyboard, a touch panel, and receives input from a user to the estimation device 1. The storage unit 2 is a non-volatile storage area composed of a hard disk, a flash memory, or the like. The control unit 3 is composed of a central processing unit (CPU), a read only memory (ROM), a random access memory (RAM), and the like, and may include a graphics processing unit (GPU), a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), and other chips dedicated to AI. The user of the estimation device 1 may be a person who accepts consultations regarding the skin condition of a subject who provides a stratum corneum sample from the skin surface, and recommends products and treatments suitable for the subject.

[0033] A skin condition estimation program 20 is stored in the memory unit 2. The estimation program 20 of this embodiment derives an index representing the skin condition from the estimation model 22 by inputting an image of a stratum corneum sample to a trained machine learning model (estimation model 22) trained in the learning device 100 described below. Also, a database 21 is saved in the memory unit 2, which stores advice on skin care habits in association with an index representing the estimated skin condition. The estimation program 20 of this embodiment reads out advice on appropriate skin care habits from the database 21 based on the derived index representing the skin condition, and causes the display unit 6 to display it.

[0034] The control unit 3 reads and executes the estimation program 20 stored in the memory unit 2, thereby virtually operating as an image acquisition unit 30, an estimation unit 31, and a result output unit 32. The image acquisition unit 30 reads an image of a stratum corneum sample acquired using an external microscope. The estimation unit 31 estimates the subject's skin condition using the estimation model 22. The result output unit 32 causes the display unit 6 to display an index representing the skin condition, and in addition to or instead of this, advice regarding skin care habits. Detailed operations of each unit will be described later.

[0035] <2. Principle of skin condition estimation> The principle of estimating a skin condition according to an embodiment of the present invention will be described below. The estimation device 1 estimates a skin condition based on this principle.

[0036] In the stratum corneum of the skin, keratinocytes overlap to form multiple layers. The space between each keratinocyte is filled with intercellular lipids. One of the intercellular lipids is ceramide, which retains water to keep the skin moist and also acts as a barrier against external stimuli. In general, the more ceramide there is in the stratum corneum, the higher the skin's moisturizing power and barrier function.

[0037] The stratum corneum can be easily collected by, for example, attaching a stratum corneum sampling tool having an adhesive layer to the skin surface and peeling it off, that is, by tape stripping. The present inventor observed a stratum corneum sample attached to a stratum corneum sampling tool under a microscope, and found that information about the state of the skin appears as image features in an image captured by the microscope. Figure 2 is a diagram illustrating this. The present inventor discovered that the CIA value, which is the sum of the luminance values ​​of an image (polarized microscope image) of a stratum corneum sample captured by a polarizing microscope for each pixel, correlates with the actual amount of ceramide contained in the stratum corneum sample. Based on the results of the experiments conducted by the present inventor, a graph plotting the CIA value and the actual amount of ceramide is shown in Figure 2. The present inventor also discovered that the DIA value, which is the sum of the luminance values ​​of an image (bright field image) of a stratum corneum sample captured by a bright field microscope for each pixel, correlates with the actual amount of protein contained in the stratum corneum sample. Based on the results of the experiments conducted by the present inventor, a graph plotting the DIA value and the actual amount of protein is shown in Figure 2. This protein is the main component that constitutes keratinocytes. The actual amount of ceramide was measured by subjecting a sample solution prepared containing a stratum corneum specimen to a known HPLC (high performance liquid chromatography). The actual amount of protein was measured by subjecting a sample solution prepared containing a stratum corneum specimen to a known protein assay BCA kit.

[0038] Hereinafter, the amount of ceramide contained in the stratum corneum sample may be referred to as the absolute ceramide amount, and the amount of protein contained in the stratum corneum sample may be referred to as the absolute protein amount. The value obtained by dividing the absolute ceramide amount by the absolute protein amount is referred to as the unit ceramide amount. It can be said that the unit ceramide amount is an amount indicating the amount of ceramide per keratinocyte. As shown in FIG. 2, the value obtained by dividing the CIA value by the DIA value (CIA / DIA) correlates with the amount of ceramide obtained by dividing the actual amount of ceramide contained in the stratum corneum sample by the actual amount of protein (unit ceramide amount based on the actual amount).

[0039] As described above, the condition of skin can be evaluated by the amount of ceramide present in stratum corneum.However, because the ease of peeling off stratum corneum by tape stripping varies from person to person, the absolute amount of ceramide may not necessarily reflect the condition of skin properly.Therefore, it is preferable to evaluate the condition of skin by using the amount of unit ceramide.The more the amount of unit ceramide, the higher the skin condition of moisturizing power and barrier function, and the less the amount of unit ceramide, the lower the skin condition of moisturizing power and barrier function.

[0040] The inventor further applied the above findings and found that there is a relationship between the feature amount of the polarizing microscope image extracted by machine learning and the actual amount of ceramide and the unit amount of ceramide. FIG. 3A is a diagram explaining this. FIG. 3A shows a graph in which the estimated amount of ceramide estimated from the polarizing microscope image using the machine learning model constructed by the inventor is plotted against the actual amount of ceramide measured by the above method. As can be seen from this graph, the actual amount of ceramide and the estimated amount of ceramide have a correlation coefficient of about 0.81. Also, FIG. 3A shows a graph in which the estimated unit amount of ceramide estimated from the polarizing microscope image using a similar machine learning model is plotted against the unit amount of ceramide calculated based on the actual amount. As can be seen from this graph, the unit amount of ceramide based on the actual amount and the estimated unit amount of ceramide have a correlation coefficient of about 0.65. In other words, the unit amount of ceramide can be estimated by the image processing method based on the polarizing microscope image and the bright field image, but when machine learning is used, it can be estimated with good accuracy based only on the polarizing microscope image.

[0041] The inventor further found that there is a relationship between the feature amount of the bright-field image extracted by the same machine learning as above and the actual amount of protein. FIG. 3B shows a graph in which the estimated protein amount estimated from the bright-field image using the same machine learning model as FIG. 3A is plotted against the actual amount of protein measured by the above method. As can be seen from this graph, the actual amount of protein and the estimated amount of protein have a correlation coefficient of about 0.88. In this way, the amount of protein can be estimated with good accuracy based on the bright-field image. Therefore, in the same machine learning model, it is also possible to output the absolute amount of ceramide from the input of the polarizing microscope image and output the absolute amount of protein from the input of the bright-field image, and estimate the unit amount of ceramide based on these outputs.

[0042] Based on these findings, the inventors have constructed a machine learning model that, when a polarized light microscope image is input, outputs values ​​corresponding to an estimated absolute ceramide amount and an estimated unit ceramide amount. The estimation device 1 uses a trained machine learning model (estimation model 22) constructed based on the above findings to estimate an index representing a skin condition and advice regarding skin care habits based on a microscope image of a stratum corneum sample (see FIG. 6). The following describes the learning of the estimation model by the learning device 100 according to one embodiment of the present invention. <3. Configuration of the learning device>

[0043] FIG. 4 is a block diagram showing the electrical configuration of the learning device 100. The learning device 100 is a general-purpose computer in terms of hardware, and may be the same device as the estimation device 1. The learning device 100 includes a memory unit 200, a control unit 300, a display unit 600, an input unit 400, and a communication unit 500. These units 200 to 500 are connected to each other via a communication line 800 such as a bus line or a cable, and can communicate with each other as appropriate. The display unit 600 is composed of a liquid crystal display or the like. The input unit 400 is composed of a mouse, a keyboard, and a touch panel, and accepts input from a user to the learning device 100. The memory unit 200 is a non-volatile storage area composed of a hard disk, a flash memory, or the like. The control unit 300 is composed of a CPU, a ROM, a RAM, and the like, and may include a GPU, an FPGA, an ASIC, and other AI-dedicated chips.

[0044] A learning program 210 provided from a computer-readable recording medium such as a CD-ROM or a USB memory or via a network or the like is installed in the storage unit 200. The learning program 210 has a function of constructing a machine learning model and training the constructed machine learning model.

[0045] The memory unit 200 stores learning data 220 prepared for learning. The learning data 220 is a large number of data sets of images of stratum corneum samples captured by a microscope and correct answer data. The learning data 220 of this embodiment is a large number of data sets in which data of polarizing microscope images of stratum corneum samples are associated with absolute ceramide amounts and unit ceramide amounts measured from the same stratum corneum samples as correct answer data. The unit ceramide amount data is calculated in advance based on the absolute ceramide amount measured directly from the stratum corneum samples and the absolute protein amount measured directly from the stratum corneum samples. The polarizing microscope image data included in the learning data 220 of this embodiment all have the same number of pixels and are converted to 8-bit grayscale.

[0046] While there is no particular limitation on the method of collecting a stratum corneum sample for creating the training data 220, in this embodiment, a stratum corneum sample collected by tape stripping from the skin surface of the subject's arm is used. Although there is no particular limitation on the stratum corneum collecting tool used for tape stripping, it is preferable to use one that has low birefringence and excellent light transmission.

[0047] The polarizing microscope image is an image of the stratum corneum attached to the stratum corneum collector captured by a polarizing microscope. The magnification of the polarizing microscope is 100 times in this embodiment. However, the magnification is not limited to this and may be changed as appropriate as long as it is possible to extract features of the polarizing microscope image related to the skin condition.

[0048] The method of measuring the absolute ceramide amount and the absolute protein amount is not particularly limited, and known methods can be used. In the measurement method according to the present embodiment, first, a sample solution is prepared from a sample containing the collected stratum corneum using hexane, and ceramide is extracted from the precipitate obtained by centrifuging the sample solution. Next, the extracted ceramide is enzymatically decomposed to obtain a sample solution in which the components derived from ceramide are fluorescently modified. The obtained sample solution is subjected to HPLC, and the measured ceramide amount is regarded as the absolute ceramide amount. In addition, the sample solution after measuring the ceramide amount is used, and the protein amount measured by a known protein assay BCA kit is regarded as the absolute protein amount.

[0049] When the control unit 300 detects that the user has performed a predetermined operation via the input unit 400, it starts the learning program 210. The learning unit 310 constructs a machine learning model by a predetermined operation by the user. The type of the machine learning model is not particularly limited, but in this embodiment, it is a neural network as shown in FIG. 3. The number of intermediate layers of the neural network is not particularly limited and can be set by the user as appropriate. When the neural network is constructed, the learning unit 310 reads the learning data 220 by a predetermined operation by the user, and performs learning of the neural network by using the data of the polarizing microscope image as input and the absolute ceramide amount and the unit ceramide amount (i.e., data corresponding to the correct answer data) as output. The number of learning epochs is not particularly limited and can be set as appropriate. In addition, learning may be repeated until the error due to verification meets a predetermined standard, rather than a predetermined number of times. The learning unit result output unit 320 displays the learning result for each time on the display unit 600, and stores the parameters at the end of learning in the learning result storage area 230 of the storage unit 200. The machine learning model to which the learned parameters are applied is stored in the estimation device 1 as the estimation model 22, and is applied to the processing of the estimation device 1. In other words, the estimation model 22 is an inference program incorporating the learned parameters, and causes a computer to output or estimate a unit ceramide amount and an absolute ceramide amount by inputting a polarized microscope image. The estimation model 22 itself may be stored as an inference program in a non-volatile storage medium independent of the storage unit of the estimation device 1. Hereinafter, the estimation model 22 may be referred to as a neural network 22.

[0050] <4. Operation of the estimation device> The operation of the estimation device 1 to which the neural network 22 trained by the learning device 100 is applied will be described below with reference to Figs. 5 and 6. The estimation device 1 estimates a unit ceramide amount from the feature amount of an image of a stratum corneum sample, and outputs advice on skin care habits tailored to the subject based on the estimated unit ceramide amount. The estimation device 1 is connected to an external microscope (not shown) so as to be able to communicate with it via wired or wireless communication. When the estimation program 20 of the estimation device 1 is started and preparations are made to read an image captured by the microscope, the process reaches the start state shown in the flowchart of Fig. 5.

[0051] First, an image to be read by the estimation device 1 is prepared. Specifically, a user collects a stratum corneum sample from the skin surface of the subject's arm by tape stripping. The stratum corneum sampler used for collection is preferably the same as that used when creating the learning data 220. Next, the user uses a microscope to capture an image of the stratum corneum sample attached to the stratum corneum sampler. The magnification of the microscope can be, for example, 100 times. In this embodiment, the microscope used at this time is a polarizing microscope.

[0052] In step S1, the control unit 3 receives an instruction from a user and causes the image acquisition unit 30 to read an image of a stratum corneum sample. The image acquisition unit 30 reads the image and temporarily stores the image data in a RAM or the like.

[0053] In step S2, the image acquisition unit 30 converts the temporarily stored image into an 8-bit grayscale image in the same format and size as the image data included in the learning data 220. In this embodiment, the image acquisition unit 30 converts the temporarily stored image into an 8-bit grayscale image.

[0054] When the image conversion is completed, the process proceeds to step S3. In step S3, the estimation unit 31 estimates an index representing the skin condition from an index representing the characteristics of the loaded image using the neural network 22 trained by the learning device 100. In this embodiment, the index representing the skin condition is the unit ceramide amount. More specifically, when the image loaded by the estimation unit 31 is input to the neural network 22, values ​​corresponding to the estimated ceramide amount and the unit ceramide amount are output from the neural network 22. Hereinafter, the unit ceramide amount at this time may be referred to as the estimated unit ceramide amount. Herein, the unit ceramide amount may be the value of the unit ceramide amount itself, or may represent a predetermined category classified according to the value of the unit ceramide amount. In this embodiment, the output value of the neural network 22 represents the value of the ceramide amount and the unit ceramide amount.

[0055] In step S4 of the present embodiment, the estimation unit 31 classifies the estimated unit ceramide amount into a category according to its value based on the estimated unit ceramide amount. The categories are, for example, ceramide levels classified in stages from the largest unit ceramide amount. More specifically, the categories may be ceramide levels 1 to 5 classified in five stages from the largest unit ceramide amount. The categories may also represent the state related to moisture amount, such as "moisturized," "normal," and "dry," classified in stages from the largest unit ceramide amount. The categories may also represent the state related to the barrier function of the skin, such as "high," "normal," and "low," classified in stages from the largest unit ceramide amount. In addition, when learning the estimation model, learning may be performed by including data representing these categories in the correct answer data, and data representing the category into which the subject's skin condition is classified may be included in the output derived from the neural network 22.

[0056] In the next step S5, the result output unit 32 reads out the advice on skin care habits associated with the index representing the skin condition, i.e., the unit ceramide amount, from the database 21, and displays it on the display unit 6 together with the estimated unit ceramide amount. The display mode of the index representing the skin condition is not particularly limited. For example, it can be displayed in the form of a numerical value representing the estimated unit ceramide amount, a character string representing a category into which the value of the estimated unit ceramide amount is classified, an icon representing a category into which the value of the estimated unit ceramide amount is classified, and a graphic indicating the location of the category to which the value of the estimated unit ceramide amount corresponds in a scale representing the entire category. In the example shown in FIG. 6, a character string such as "ceramide level: 4" is displayed as a character string representing the category into which the value of the estimated unit ceramide amount is classified. In addition, the advice on skin care habits can include information specifying a product for further improving or maintaining the skin condition of the subject according to the skin condition. The products include cosmetics, quasi-drugs, dietary supplements, foods, and medicines. In addition, the advice on skin care habits may include content related to lifestyle habits that may affect the skin condition, such as face washing methods, diet, and sleep time, and an introduction of products for improving these lifestyle habits. 6, an external view of the product is displayed together with a character string identifying the product, such as "The recommended product is XX." Note that when learning the estimation model, learning may be performed by including data representing advice on skin care habits in the ground truth data, and the data representing advice on skin care habits estimated to be suitable for the subject's skin may be included in the output derived from the neural network 22.

[0057] The user can present the subject with the indicators representing the skin condition and advice on skin care habits displayed by the estimation device 1. The subject can recognize his / her own skin condition and obtain advice on skin care habits suitable for the skin condition without being bound by the subject's own or the user's subjective opinion. The user can also provide further explanations regarding the subject's skin condition, recommended products, and skin care habits, and answer questions from the subject.

[0058] <5. Application form> The estimation device 1 according to one embodiment of the present invention can be used, for example, by a beauty professional engaged in beauty-related services in an esthetic salon, a cosmetic store, a clinic, etc., when selling a skin care product to a customer. In other words, the user of the embodiment can be a beauty professional, and the subject can be a customer receiving a beauty-related service.

[0059] <6. Variations> Although one embodiment of the present invention has been described above, the present invention is not limited to the above embodiment, and various modifications are possible without departing from the spirit of the present invention. The following modified examples can be combined as appropriate.

[0060] (1) The output of the estimation model 22 of the estimation device 1 that uses a polarizing microscope image as input may be only the estimated unit ceramide amount, or may be only advice regarding skin care habits. Also, the output of the estimation model 22 may be the estimated unit ceramide amount and advice regarding skin care habits, or may be the estimated unit ceramide amount, the estimated ceramide amount, and advice regarding skin care habits. The correct answer data of the learning data 220 may be the same type of data as the output of the estimation model 22. For example, when the output of the estimation model 22 is a unit ceramide amount, the correct answer data may be the unit ceramide amount. Also, when the output of the estimation model 22 is advice regarding skin care habits, the correct answer data may be advice regarding skin care habits. Similarly, when the types of outputs of the estimation model 22 are multiple, the correct answer data may be the multiple types of data.

[0061] (2) The result output unit 32 may display only either the index representing the skin condition or the advice on skin care habits on the display unit 6. In cases such as when only the index representing the skin condition is displayed on the display unit 6, the configuration of the database 21 in the estimation device 1 may be omitted.

[0062] (3) The subject who collects the stratum corneum sample, the subject who images the stratum corneum sample under a microscope, and the subject who operates the estimation device 1 may each be different subjects. Furthermore, the subject who collects the stratum corneum sample may not be the user of the estimation device 1, but the subject himself. In this case, a remote subject can collect his or her own stratum corneum sample and send the collected stratum corneum sample to the user who uses the estimation device 1. The user can have the estimation device 1 estimate the subject's skin condition from the sent stratum corneum sample. The user can provide at least one of the index representing the skin condition output by the estimation device 1 and advice on skin care habits to the subject in the form of a document, electronic data, audio data, etc.

[0063] (4) The result output unit 32 may output at least one of the index representing the skin condition and the advice on skin care habits onto a medium such as paper using a printer. In this case, the display unit 6 may be omitted from the estimation device 1.

[0064] (5) In the above embodiment, an example was shown in which the data of the polarizing microscope image was used as the image data input to the neural network 22 and the image data included in the learning data 220, but the data of the microscope image is not limited to this. As long as the image data has a feature related to at least one of the absolute protein amount and the absolute ceramide amount contained in the stratum corneum sample, image data from various types of microscopes can be used. For example, as described above, the feature amount of the bright field image is related to the absolute protein amount. Therefore, the estimation model 22 may be configured to receive the input of the bright field image data and the polarizing microscope image data, and to output the absolute protein amount and the absolute ceramide amount. In this case, the output may further include at least one of the estimated unit ceramide amount and advice on skin care habits. Alternatively, the estimation unit 31 may estimate at least one of an index representing the skin condition and advice on skin care habits based on the absolute protein amount and absolute ceramide amount output from the estimation model 22. Also, the estimation model 22 may be configured to receive the input of the bright field image data and the polarizing microscope image data, and to output the output of the index representing the skin condition and advice on skin care habits.

[0065] (6) Furthermore, the data set constituting the training data 220 may be appropriately changed according to the input and output of the estimation model 22. For example, the training data 220 may be a data set in which bright-field image data and polarizing microscope image data are combined with absolute protein amount and absolute ceramide amount data as correct answer data. Furthermore, the correct answer data may include at least one of an estimated unit ceramide amount and advice on skin care habits. Furthermore, the training data 220 may be a data set in which bright-field image data and polarizing microscope image data are combined with at least one of an index representing a skin condition and advice on skin care habits as correct answer data.

[0066] (7) The estimation device 1 may store the image of the stratum corneum sample acquired by the image acquisition unit 30 in the memory unit 2. At this time, data for identifying the subject, data on the date the image was captured, the estimated skin condition, etc. may be stored in association with the image. In such a configuration, for example, a user can compare the skin condition of the same subject over time, and verify the effectiveness of advice regarding skin care habits practiced by the subject.

[0067] (8) The learning device 100 and the estimation device 1 use a neural network as a machine learning model to estimate the skin condition, but the machine learning model is not limited to this and any other known model such as a support vector machine, a regression model, or a decision tree model may be used.

[0068] (9) The estimation device 1 may not include the estimation unit 31, and the function of the estimation unit 31 may be provided in another hardware. For example, the estimation model 22 held in a server providing a cloud service may estimate the unit amount of ceramide.

[0069] (10) The learning device 100 may not include the learning unit 310, and the functions of the learning unit 310 may be provided in other hardware. For example, a program stored in a server that provides a cloud service may construct and learn a machine learning model.

[0070] <7.Features> (1) According to the embodiment, the estimation device 1 estimates the skin condition of a subject from an image of a stratum corneum sample taken from the subject. In this manner, the skin condition can be estimated without directly analyzing the components of the stratum corneum sample, which is convenient and easy to use for both the user and the subject, and significantly reduces the time burden.

[0071] (2) According to the estimation device 1 of the embodiment, the skin condition is estimated based on a large amount of data and is not subjective to the user or subject. This reduces the effort required of the user to evaluate the skin condition through a medical interview or component analysis of the stratum corneum. This allows the user to focus on activities that focus more on proposing skin care habits, including product recommendations, to improve or maintain the skin condition of the subject. [Explanation of symbols]

[0072] 1 Estimation device 2 Storage section 3. Control Unit 4 Input section 5. Communications Department 100 Learning Device

Claims

1. an image acquisition unit that acquires an image of a stratum corneum sample taken from the surface of the skin using a microscope; an estimation unit that inputs the acquired image into a trained machine learning model, derives an output from the machine learning model that is a unit ceramide amount that represents the amount of ceramide per keratinocyte contained in the stratum corneum sample, and estimates advice regarding skin care habits suitable for the skin based on the output; Equipped with A skin condition estimation device.

2. The image includes an image captured by a polarizing microscope. The estimation device according to claim 1 .

3. the estimation unit estimates advice regarding skin care habits suitable for the skin using a machine learning model trained using learning data, which is a data set of an image obtained by photographing a stratum corneum sample taken from the surface of the skin with a microscope and ground truth data; The correct data is the unit ceramide amount based on a value measured from a stratum corneum sample. The estimation device according to claim 1 or 2.

4. a result output unit that outputs the result estimated by the estimation unit Further comprising: The estimation device according to any one of claims 1 to 3.

5. Obtaining an image of a stratum corneum sample taken from the surface of the skin using a microscope; inputting the acquired image into a trained machine learning model to derive an output from the machine learning model, the output being a unit ceramide amount representing the amount of ceramide per keratinocyte contained in the stratum corneum sample, and estimating advice regarding skin care habits suitable for the skin based on the output; Equipped with A method for estimating skin condition.

6. The image includes an image captured by a polarizing microscope. The estimation method according to claim 5.

7. Obtaining an image of a stratum corneum sample taken from the surface of the skin using a microscope; inputting the acquired image into a trained machine learning model to derive an output from the machine learning model, the output being a unit ceramide amount representing the amount of ceramide per keratinocyte contained in the stratum corneum sample, and estimating advice regarding skin care habits suitable for the skin based on the output; to cause a computer to execute A program to estimate skin condition.

8. The image includes an image captured by a polarizing microscope. The estimation program according to claim 7.

9. A memory unit that stores learning data, which is a data set of an image of a stratum corneum sample taken from the surface of the skin captured by a microscope and ground truth data; a learning unit that uses the learning data to learn a machine learning model so that when the image is input to the machine learning model, data corresponding to the correct answer data is output; Equipped with The correct data is a unit amount of ceramide representing the amount of ceramide per keratinocyte contained in the stratum corneum sample. Learning device.

10. The image includes an image captured by a polarizing microscope. The learning device according to claim 9.

11. A step of preparing learning data which is a data set of an image of a stratum corneum sample taken from the surface of the skin captured by a microscope and ground truth data; performing machine learning using the learning data such that, when an image of a stratum corneum sample taken from the surface of the skin captured by a microscope is input, data corresponding to the ground truth data is output; Equipped with The correct data is a unit amount of ceramide representing the amount of ceramide per keratinocyte contained in the stratum corneum sample. How to generate a trained model.

12. The image includes an image captured by a polarizing microscope. A method for generating a trained model according to claim 11.

Images