Training method, evaluation method, information processing system, and program

Abstract

This training method comprises: acquiring a facial part image group including a plurality of facial part images indicating a plurality of evaluation parts in the faces of a plurality of users, and skin evaluation values of a plurality of skin evaluation items for each of the plurality of evaluation parts; generating a database in which the facial part image group, the plurality of skin evaluation items for each of the evaluation parts, and each piece of data of the skin evaluation values are associated with each other; associating an integrated label, which is a superordinate concept item of the skin evaluation items, with data of the skin evaluation values in which the evaluation parts are different and the skin evaluation items are common or similar in the database; calculating a normalized skin evaluation value obtained by normalizing the skin evaluation value for each integrated label; and executing training of a deep learning model so as to predict the normalized skin evaluation value by using, for each integrated label, the facial part image group as an input.

Description

Learning Method, Evaluation Method, Information Processing System, and Program

[0001] The present invention relates to a learning method, an evaluation method, an information processing system, and a program for evaluating a user's skin condition.

[0002] Conventionally, there has been a technique of extracting a partial image from a user's face image and evaluating the skin condition based on the partial image using a learned model.

[0003] For example, in Patent Document 1 below, one or more processors capable of using a learned discrimination model learned based on a plurality of teacher data including a plurality of combinations of correct answer information indicating an attribute or state common to the photographed skin of a teacher skin image and a group of teacher skin patch images of a predetermined image size extracted from the teacher skin image acquire an evaluation skin image, acquire a group of skin patch images of a predetermined image size from the acquired evaluation skin image, perform pixel value normalization on each of the acquired skin patch images, and input each of the normalized skin patch images into the discrimination model, respectively, to obtain index values of skin attributes or skin conditions for each of the skin patch images.

[0004] Also, in Patent Document 2 below, an image including skin is input into a database or a storage unit, one or a plurality of standardized skin units (such as the outer corner of the eye region, the under-eye region, the nose region, the mouth region, and the cheek region, etc.) are identified from the image including skin, one or a plurality of features (such as the number of freckles, the number of pores, the amount of stratum corneum moisture, etc.) within the skin unit are analyzed using an AI model, and the result of the analysis is displayed.

[0005] Japanese Patent Application Laid-Open No. 2022-078936, Japanese Patent Application Laid-Open No. 2023-166041

[0006] However, in the techniques of Patent Documents 1 and 2 above, since images of different multiple parts extracted from a face image are used as teacher data for learning, it is necessary to prepare a large number of learning samples for each part, and if the number of the learning samples is not sufficient, the prediction accuracy will also be low.

[0007] The object of the present invention is to provide a learning method, an evaluation method, an information processing system, and a program that can generate a learning model capable of efficiently and accurately evaluating skin on multiple different parts of the face.

[0008] A learning method according to one embodiment of the present invention includes: acquiring a group of facial part images consisting of multiple facial part images showing multiple evaluation areas within the faces of multiple users, and skin evaluation values ​​for multiple skin evaluation items for each of the multiple evaluation areas; generating a database that links the data of the facial part image group, the multiple skin evaluation items for each evaluation area, and the skin evaluation values; linking integrated labels, which are higher-level conceptual items for the skin evaluation items, to the data of the skin evaluation values ​​in the database for which the evaluation areas are different but the skin evaluation items are common or similar; calculating a normalized skin evaluation value by normalizing the skin evaluation value for each integrated label; and training a deep learning model to predict the normalized skin evaluation value for each integrated label, using the facial part image group as input.

[0009] Another embodiment of the present invention is a skin evaluation method using a deep learning model trained by the learning method described above, comprising: acquiring an image of the facial parts of a user to be evaluated; acquiring a conversion formula for calculating the original skin evaluation value from the normalized skin evaluation value trained by the deep learning model corresponding to the integrated label of the skin evaluation item to be evaluated; inputting the image of the facial parts to be evaluated into the deep learning model corresponding to the integrated label of the skin evaluation item to be evaluated; acquiring the normalized skin evaluation prediction value for each evaluation area from the deep learning model; acquiring the skin evaluation prediction value for each evaluation area using the normalized skin evaluation prediction value and the conversion formula; and transmitting information regarding the acquired skin evaluation prediction value to the user's user terminal.

[0010] An information processing system according to another embodiment of the present invention is an information processing system that performs skin evaluation processing using a skin evaluation deep learning model learned by the learning method described above, and comprises a control unit. The control unit acquires an image of the user's facial parts to be evaluated, acquires a conversion formula for calculating the original skin evaluation value from the normalized skin evaluation value learned by the deep learning model corresponding to the integrated label of the skin evaluation item to be evaluated, inputs the image of the facial parts to be evaluated into the deep learning model corresponding to the integrated label of the skin evaluation item to be evaluated, acquires the normalized skin evaluation prediction value for each evaluation area from the deep learning model, acquires a skin evaluation prediction value for each evaluation area using the normalized skin evaluation prediction value and the conversion formula, and transmits information regarding the acquired skin evaluation prediction value to the user's user terminal.

[0011] A program according to yet another embodiment of the present invention causes an information processing device to perform the following steps: acquire a group of facial part images consisting of multiple facial part images showing multiple evaluation areas within the faces of multiple users, and skin evaluation values ​​for multiple skin evaluation items for each of the multiple evaluation areas; generate a database linking the data of the group of facial part images, the multiple skin evaluation items for each of the evaluation areas, and the skin evaluation values; link an integrated label, which is a higher-level conceptual item for the skin evaluation item, to the data of the skin evaluation values ​​in the database for which the evaluation areas are different but the skin evaluation items are common or similar; calculate a normalized skin evaluation value by normalizing the skin evaluation value for each integrated label; and, for each integrated label, use the group of facial part images as input and train a deep learning model to predict the normalized skin evaluation value.

[0012] According to one embodiment of the present invention, a learning method can be generated that efficiently and accurately evaluates skin on multiple different parts of the face. However, this effect is not limited to the present invention.

[0013] This is a diagram showing the configuration of a skin evaluation information provision system according to one embodiment of the present invention. This is a diagram showing the hardware configuration of a skin evaluation information provision server according to one embodiment of the present invention. This is a diagram showing the configuration of a database owned by a skin evaluation information provision server according to one embodiment of the present invention. This is a flowchart showing the flow of the deep learning model training process by a skin evaluation information provision server according to one embodiment of the present invention. This is a diagram conceptually illustrating the deep learning model training process by a skin evaluation information provision server according to one embodiment of the present invention. This is a flowchart showing the flow of the skin evaluation information provision process by a skin evaluation information provision server according to one embodiment of the present invention.

[0014] Embodiments of the present invention will be described below with reference to the drawings.

[0015] [System Configuration] As shown in Figure 1, this system includes a skin evaluation information server 100 on the Internet 50 and a plurality of user terminals 200.

[0016] The skin evaluation information provision server 100 is a server (information processing device) that performs a service providing users with skin evaluation information. The skin evaluation information provision server 100 is connected to multiple user terminals 200 via the Internet 50.

[0017] The skin evaluation information server 100 transmits the user's skin evaluation information to the user terminal 200 as an evaluation result, based on the user's facial image received from the user terminal 200 along with the skin evaluation request.

[0018] The skin evaluation information server 100 uses a deep learning model 10 to estimate skin evaluation values ​​from skin part images extracted from the user's face image in order to acquire the above skin evaluation information.

[0019] The deep learning model 10 is trained to predict skin evaluation values ​​from face part images, using multiple face part images (group of face part images) that show multiple evaluation areas in face images provided by multiple users, and skin evaluation values ​​for multiple skin evaluation items for each of the multiple evaluation areas, as training data. As will be described in detail later, the deep learning model 10 is trained and generated for each evaluation item in the user's face. The deep learning model 10 may be generated by a training process via the skin evaluation information provision server 100, or it may be generated by another information processing device.

[0020] User terminals 200 (200A, 200B, 200C...) are terminals used by users, such as smartphones, mobile phones, tablet PCs (Personal Computers), notebook PCs, and desktop PCs. User terminals 200 access the skin evaluation information provision server 100, receive web pages containing the skin evaluation information, and display them on the screen using a browser or similar. Alternatively, an application compatible with the skin evaluation information provision service may be installed on the user terminal 200, and the user terminal 200 may access the skin evaluation information provision server 100 and display the skin evaluation information using this application.

[0021] [Hardware configuration of the skin evaluation information server] As shown in Figure 2, the skin evaluation information server 100 includes a CPU (Central Processing Unit) 11, ROM (Read Only Memory) 12, RAM (Random Access Memory) 13, an input / output interface 15, and a bus 14 that connects these to each other.

[0022] The CPU 11 accesses RAM 13 and other memory as needed, performing various calculations and comprehensively controlling each block of the skin evaluation information provision server 100. Multiple CPUs 11 may be provided depending on the processing. ROM 12 is a non-volatile memory in which the OS, programs, and firmware such as various parameters to be executed by the CPU 11 are permanently stored. RAM 13 is used as a working area for the CPU 11 and temporarily holds the OS, various running applications, and various data being processed.

[0023] The input / output interface 15 is connected to a display unit 16, an operation reception unit 17, a storage unit 18, a communication unit 19, and the like.

[0024] The display unit 16 is a display device that uses, for example, an LCD (Liquid Crystal Display), an OLED (Organic ElectroLuminescence Display), or a CRT (Cathode Ray Tube).

[0025] The operation reception unit 17 is, for example, a pointing device such as a mouse, a keyboard, a touch panel, or other input device. If the operation reception unit 17 is a touch panel, the touch panel may be integrated with the display unit 16.

[0026] The storage unit 18 is a non-volatile memory such as an HDD (Hard Disk Drive), flash memory (SSD; Solid State Drive), or other solid-state memory. The OS, various applications, and various data are stored in this storage unit 18.

[0027] As will be described later, in this embodiment in particular, the storage unit 18 has a user information database, a learning-related information database, and a skin evaluation information database, in addition to programs such as applications necessary for the skin evaluation information provision process described later.

[0028] The communication unit 19 consists of various modules for wireless communication, such as a NIC (Network Interface Card) for Ethernet or a wireless LAN, and is responsible for communication processing with the user terminal 200.

[0029] Although not shown in the diagram, the basic hardware configuration of the user terminal 200 is substantially the same as that of the skin evaluation information provision server 100.

[0030] [Database configuration of the skin evaluation information server]

[0031] As shown in Figure 3, the skin evaluation information providing server 100 has a user information database 31, a learning-related information database 32, and a skin evaluation information database 33 in its storage unit 18. Note that these databases may be stored in a storage device or server externally connected to the skin evaluation information providing server 100, rather than in the storage unit 18.

[0032] The user information database 31 stores attribute information for each user of the cosmetics information service provided by the skin evaluation information server 100. This attribute information includes general information such as name, user ID for identification, age (age group), occupation, address (residential area), gender, and email address, as well as information about the user's skin, such as their perceived or understood skin type, cosmetic preferences, and any skin problems they may have. The user ID functions as account information for using this skin evaluation information service.

[0033] The learning-related information database 32 stores training data and related information necessary for training the deep learning model, such as a group of face images, a group of face part images extracted from the face image data, evaluation areas within the face to be evaluated, skin evaluation items for each evaluation area, skin evaluation values ​​for each evaluation area's skin evaluation item, an integrated label attached to each skin evaluation value, and normalized skin evaluation values ​​obtained by normalizing the skin evaluation value for each integrated label.

[0034] Evaluation items include visual evaluation values ​​such as sagging, wrinkles, and pore visibility, as well as measurement evaluation values ​​measured by measuring instruments, such as stratum corneum moisture content, transepidermal water loss, sebum content, and skin surface temperature.

[0035] The above visual evaluation values ​​are assessed using a multi-level rating system based on different visual evaluation scales defined for each evaluation area, such as 0-7 or 1-5. The rating values ​​are integers, but the average of multiple evaluators may be calculated, in which case the value will be a non-negative real number.

[0036] The above stratum corneum moisture content is measured using a stratum corneum moisture content measuring device such as SKICON® or Corneometer®, and is expressed as a real number greater than or equal to 0 (approximately 0 to 300, depending on the device specifications). Transepidermal water loss is measured using a transepidermal water loss measuring device such as Tewameter or Vapometer®, and is expressed as a real number greater than or equal to 0 (approximately 0 to 100, depending on the device specifications). Sebum amount is measured using a sebum secretion measuring device such as Sebumeter, and is expressed as an integer greater than or equal to 0 (approximately 0 to 300, depending on the device specifications). Skin surface temperature is measured using a radiation thermometer, thermometer, etc., and is expressed as a real number greater than or equal to 0 (approximately 0 to 36, depending on the device specifications and measurement site).

[0037] The evaluation sites are set for each of the above evaluation items, and the evaluation items for the above measurement values ​​may also differ depending on the measuring instrument. For example, for stratum corneum moisture content and transepidermal water loss, the evaluation sites are the cheeks, mouth area, and outer corners of the eyes. For sebum content, the evaluation sites are the forehead, sides of the nose, bridge of the nose, temples, cheeks, and chin. For skin surface temperature, the evaluation sites are the forehead, outer corners of the eyes, under the eyes, cheeks, mouth area, and chin.

[0038] Furthermore, the visual evaluation criteria mentioned above include, for example, under the eyes (with eyes closed) and corners of the mouth for sagging skin, and the forehead, outer corners of the eyes, under the eyes (with eyes closed), and nasolabial folds for wrinkles.

[0039] The integrated label attached to each skin evaluation value is a label that indicates a higher-level concept item for the skin evaluation item, and is attached to data of skin evaluation values ​​that are evaluated in different areas but share common or similar skin evaluation items. For example, wrinkles under the eyes, crow's feet, and nasolabial folds are in different areas but all have the common characteristic of being wrinkles, so the integrated label "wrinkles" is attached to these skin evaluation values ​​as a higher-level concept. In addition, the integrated labels "sagging" are attached to sagging around the mouth and sagging under the eyes, "skin surface temperature" is attached to skin surface temperature of the cheeks, skin surface temperature of the forehead, and skin surface temperature of the crow's feet, and "stratum corneum moisture content" is attached to stratum corneum moisture content of the cheeks, stratum corneum moisture content of the crow's feet, and stratum corneum moisture content of the mouth.

[0040] Normalized skin evaluation values ​​are data obtained by normalizing the skin evaluation values ​​for different evaluation areas for each of the above integrated labels. Possible normalization methods include scaling the maximum and minimum values ​​so that the distribution of evaluation values ​​falls within the range of 0 to 1, or standardizing the data so that the mean is 0 and the variance is 1.

[0041] In addition to normalization at the integrated label level as described above, skin evaluation values ​​for each evaluation area may also be normalized separately for each evaluation area. For example, if wrinkle grades are evaluated using different numerical values ​​or ranges, such as 8 levels for the outer corners of the eyes and 4 levels for the under-eye area, it is preferable to normalize for each evaluation area. On the other hand, if measurements are taken for multiple evaluation areas using a consistent standard, such as skin temperature, it may not be necessary to consider differences between areas.

[0042] The skin evaluation information database 33 stores each user's skin evaluation information, obtained using the deep learning model 10, in association with the user ID. The skin score information is the skin evaluation value for each evaluation area before normalization, calculated using a conversion formula from the normalized skin evaluation value, or the corresponding evaluation information.

[0043] These databases are referenced and used as needed in the training process of the deep learning model 10 by the skin evaluation information provision server 100, which will be described later, and in the skin evaluation information provision process using the said deep learning model 10.

[0044] [Operation of Skin Evaluation Information Providing Server] Next, the operation of the skin evaluation information providing server 100 configured as described above will be described. This operation is executed by the cooperation of the hardware such as the CPU 11 and the communication unit 19 of the skin evaluation information providing server 100 and the software stored in the storage unit 18. In the following description, for convenience, the CPU 11 is taken as the operating entity.

[0045] (Learning Process by Skin Evaluation Information Providing Server) FIG. 4 is a flowchart showing the flow of the learning process of the deep learning model 10 by the skin evaluation information providing server 100.

[0046] As shown in the figure, first, the CPU 11 creates teacher data consisting of a large number of face image samples (face image group), skin evaluation items for each part, evaluation parts, and skin evaluation values (step 41). As described above, the skin evaluation items for each part are, for example, wrinkles under the eyes, wrinkles at the outer corners of the eyes, moisture on the cheeks, moisture around the mouth, etc. This teacher data is stored in the learning-related information database 32.

[0047] Subsequently, the CPU 11 adds integrated label information to the teacher data (skin evaluation items for each part) (step 42). As described above, the integrated label is, for example, a label such as "wrinkle" for wrinkles under the eyes and wrinkles at the outer corners of the eyes, and "moisture" for moisture on the cheeks and moisture around the mouth.

[0048] Subsequently, the CPU 11 normalizes the skin evaluation values for each of the above integrated labels as described above (step 43).

[0049] Subsequently, the CPU 11 recognizes the facial feature points from each face image in the face image group included in the teacher data (step 44).

[0050] Subsequently, the CPU 11 cuts out a small area (face part) including the above evaluation part from each face image based on the recognized feature points, and creates a face part image group (step 45).

[0051] Subsequently, the CPU 11 associates the evaluation parts in the teacher data and the other data corresponding thereto with the part image group (step 46).

[0052] Then, for each of the integrated labels, the CPU 11 takes the group of facial part images as input and trains the deep learning model 10 to predict the normalized skin evaluation value (step 47).

[0053] The CPU 11 performs the above processing for all the integrated labels generated, and generates a deep learning model 10 for each integrated label.

[0054] Figure 5 is a conceptual diagram illustrating the training process of the deep learning model.

[0055] For example, when learning evaluation values ​​for each type of wrinkle using images of facial features such as wrinkles under the eyes, crow's feet, and nasolabial folds as training data, the general approach is to perform deep learning separately for each type of wrinkle, since each wrinkle has different characteristics and evaluation criteria (grades), in order to keep the input data as consistent as possible.

[0056] However, in this embodiment, as shown in the figure, the skin evaluation information providing server 100 integrates the above evaluation values, which differ in evaluation area and evaluation criteria, into a higher-level label (integrated wrinkle label) and performs training processing on the deep learning model.

[0057] The inventors have found that deep learning processing using such integrated labels improves prediction accuracy compared to deep learning performed for each evaluation part as described above. This reduces the total training time and the number of models required compared to the learning method for each evaluation part, and consequently, it also has the advantage of reducing the system cost when implemented as a system and improving computation speed.

[0058] The improved prediction accuracy in this embodiment may be due to the deep learning model 10 learning common features (for example, the "shadow on a line" feature in the case of wrinkles) even though the evaluation areas are different. It may also be because integrating each evaluation area with an integrated label increases the relative number of training data per integrated label.

[0059] (Skin evaluation information processing by the skin evaluation information provision server) Figure 6 is a flowchart showing the flow of the skin analysis information provision process.

[0060] As shown in the figure, first the CPU 11 receives a facial image along with a skin evaluation request from the user terminal 200 (step 61). The skin evaluation request may be sent, for example, via a skin evaluation application installed on the user terminal 200.

[0061] Next, the CPU 11 performs face recognition on the face image and identifies the position of the face and its various parts (step 62).

[0062] Next, the CPU 11 obtains learning-related information from the learning-related information database 32 for the skin evaluation item to be evaluated, including the deep learning model 10 corresponding to the skin evaluation item (integrated label), the integrated label learned by the deep learning model 10, all evaluation areas, evaluation items for each evaluation area, and a conversion formula for calculating the original skin evaluation value from the normalized skin evaluation value learned by the deep learning model 10 (step 63).

[0063] Next, the CPU 11 extracts an evaluation facial part image from the facial image, based on all the evaluation areas included in the learning-related information, in which the position of the part corresponding to the evaluation area to be evaluated is identified (step 64).

[0064] Next, the CPU 11 inputs the extracted facial part images for evaluation into the deep learning model 10 and obtains normalized skin prediction values ​​for each evaluation area (step 65).

[0065] Next, the CPU 11 calculates the unnormalized skin prediction value for each evaluation area from the normalized prediction value using the conversion formula included in the learning-related information (step 66).

[0066] The CPU 11 then transmits information indicating the predicted skin value for the evaluation area to the user terminal (step 67). This information may be the predicted skin value itself, an evaluation comment corresponding to the predicted skin value, or an improvement comment based on the evaluation. The predicted skin value is stored in the skin evaluation information database 33.

[0067] As described above, according to this embodiment, the skin evaluation information providing server 100 can generate a deep learning model 10 that efficiently and accurately evaluates skin by assigning integrated labels to common evaluation items for multiple different parts of the face, and can use this deep learning model 10 to predict highly accurate skin evaluation values.

[0068] [Modifications] Although embodiments of the present invention have been described above, the present invention is not limited to the embodiments described above, and various modifications can be made without departing from the spirit of the present invention.

[0069] In the above-described embodiment, the skin evaluation information providing server 100, in the training process of the deep learning model 10, recognized the facial feature points of the facial image group, extracted the facial parts corresponding to the evaluation areas based on the feature points, generated the facial part image group, and associated the facial part image group with each data in the learning-related information database 32. However, these processes do not have to be performed by the skin evaluation information providing server 100; the skin evaluation information providing server 100 may receive the facial part image group generated by another device performing the above processes from the other device and use it for subsequent processing.

[0070] In the above-described embodiment, the skin evaluation information providing server 100, in the skin evaluation information providing process using the deep learning model 10, acquires a face image from the user terminal 200, identifies the position of the face and each part of the face from the face image, and extracts an evaluation face part image corresponding to the evaluation area from the face image based on the data in the learning-related information database 32 and the position of each identified part. However, these processes do not have to be performed by the skin evaluation information providing server 100, and the skin evaluation information providing server 100 may receive an evaluation face part image generated by another device that performs the above processes from the other device and use it for subsequent processing. For example, the application on the user terminal 200 may have a function to extract an evaluation face part image from a face image, and the evaluation face part image extraction process may be performed on the application prior to the skin evaluation request, and this (in place of the face image) may be sent to the skin evaluation information providing server 100 together with the skin evaluation request.

[0071] In the embodiment described above, only one skin evaluation information providing server 100 is shown, but the processing performed by the skin evaluation information providing server 100 may be distributed and executed across multiple servers. For example, the training process of the deep learning model 10 and the skin evaluation information provision process using the deep learning model 10 may be executed on separate servers.

[0072] Of the inventions described in the claims of this application, the invention described as "information processing method" is one in which each step is performed automatically by at least one device such as a computer through information processing by software, and not by a human using a computer or other device. In other words, the "information processing method" is an information processing method using computer software, and not a method in which a human operates a computer as a calculating tool.

[0073] 10... Deep-trained model 11... CPU 18... Memory unit 19... Communication unit 31... User information database 32... Learning-related information database 33... Skin evaluation information database 100... Skin evaluation information server 200... User terminal

Claims

1. A learning method comprising: acquiring a group of facial part images consisting of multiple facial part images showing multiple evaluation areas within the faces of multiple users, and skin evaluation values ​​for multiple skin evaluation items for each of the multiple evaluation areas; generating a database that links the data of the facial part image group, the multiple skin evaluation items for each evaluation area, and the skin evaluation values; linking integrated labels, which are higher-level conceptual items for the skin evaluation items, to the data of the skin evaluation values ​​in the database for which the evaluation areas are different but the skin evaluation items are common or similar; calculating normalized skin evaluation values ​​by normalizing the skin evaluation values ​​for each integrated label; and training a deep learning model to predict the normalized skin evaluation values ​​for each integrated label, using the facial part image group as input.

2. The learning method according to claim 1, which involves recognizing facial feature points of a group of facial images consisting of multiple facial images of multiple users, generating a group of facial part images by extracting facial parts corresponding to the evaluation area based on the feature points, and associating the group of facial part images with each data in the database.

3. The learning method according to claim 1 or 2, wherein the learning method obtains one or more visual evaluation values ​​from wrinkles, sagging, and pore visibility as the skin evaluation values, and the visual evaluation values ​​are based on different visual evaluation scales defined for each evaluation area.

4. The learning method according to claim 1 or 2, wherein one or more of the following are obtained as skin evaluation values, measured for different evaluation sites using the same measuring instrument: stratum corneum water content, transepidermal water loss, sebum content, and skin surface temperature.

5. A skin evaluation method using a deep learning model trained by the learning method described in claim 1 or 2, comprising: acquiring an image of the facial parts of a user to be evaluated; acquiring a conversion formula for calculating the original skin evaluation value from the normalized skin evaluation value learned by the deep learning model corresponding to the integrated label of the skin evaluation item to be evaluated; inputting the image of the facial parts to be evaluated into the deep learning model corresponding to the integrated label of the skin evaluation item to be evaluated; acquiring the normalized skin evaluation prediction value for each evaluation area from the deep learning model; acquiring the skin evaluation prediction value for each evaluation area using the normalized skin evaluation prediction value and the conversion formula; and transmitting information regarding the acquired skin evaluation prediction value to the user's user terminal.

6. The evaluation method according to claim 5, comprising: obtaining a face image from the user terminal, including at least the entire face of the user; identifying the positions of the face and each part of the face from the face image; obtaining the integrated label learned by the deep learning model, all of the evaluation areas, and the skin evaluation items corresponding to all of the evaluation areas; and obtaining an evaluation face part image by cutting out the evaluation face part image corresponding to the evaluation area from the face image based on all of the evaluation areas and the positions of each of the identified parts.

7. An information processing system that performs skin evaluation processing using a skin evaluation deep learning model learned by the learning method described in claim 1 or 2, comprising a control unit that: acquires an image of the facial parts of a user to be evaluated; acquires a conversion formula for calculating the original skin evaluation value from the normalized skin evaluation value learned by the deep learning model corresponding to the integrated label of the skin evaluation item to be evaluated; inputs the image of the facial parts to be evaluated into the deep learning model corresponding to the integrated label of the skin evaluation item to be evaluated; acquires the normalized skin evaluation prediction value for each evaluation area from the deep learning model; acquires a skin evaluation prediction value for each evaluation area using the normalized skin evaluation prediction value and the conversion formula; and transmits information regarding the acquired skin evaluation prediction value to the user's user terminal.

8. A program that causes an information processing device to execute the following steps: acquire a group of facial part images consisting of multiple facial part images showing multiple evaluation areas within the faces of multiple users, and skin evaluation values ​​for multiple skin evaluation items for each of the multiple evaluation areas; generate a database linking the data of the group of facial part images, the multiple skin evaluation items for each evaluation area, and the skin evaluation values; link an integrated label, which is a higher-level conceptual item for the skin evaluation item, to the data of the skin evaluation values ​​in the database for which the evaluation areas are different but the skin evaluation items are common or similar; calculate a normalized skin evaluation value by normalizing the skin evaluation value for each integrated label; and for each integrated label, use the group of facial part images as input and train a deep learning model to predict the normalized skin evaluation value.

Images