Learning methods, evaluation methods, information processing systems, and programs
By integrating facial part images and skin evaluation values with integrated labels and normalizing them, the method enhances skin evaluation accuracy and efficiency across multiple facial areas.
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- KAO CORP
- Filing Date
- 2024-11-06
- Publication Date
- 2026-05-19
AI Technical Summary
Existing skin evaluation technologies require extensive training data from multiple facial parts, leading to low prediction accuracy if insufficient samples are used.
A learning method that integrates facial part images and skin evaluation values into a database with integrated labels, normalizes these values, and trains a deep learning model to predict skin evaluation values across multiple facial areas.
This approach enables efficient and accurate skin evaluation across multiple facial parts by reducing training time and system costs while improving prediction accuracy.
Smart Images
Figure 2026082133000001_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a learning method, an evaluation method, an information processing system, and a program for evaluating the skin condition of a user.
Background Art
[0002] Conventionally, there has been a technique of extracting a partial image from a user's face image and evaluating the skin condition using a learned model based on the partial image.
[0003] For example, in Patent Document 1 below, one or more processors capable of using a learned discrimination model learned based on a plurality of teacher data including a plurality of combinations of correct answer information indicating an attribute or state common to the subject skin of the teacher skin image and a group of teacher skin patch images of a predetermined image size extracted from the teacher skin image acquire an evaluation skin image, acquire a group of skin patch images of a predetermined image size from the acquired evaluation skin image, perform pixel value normalization on each of the acquired skin patch images, and input each of the normalized skin patch images into the discrimination model, respectively, to obtain index values of skin attributes or skin conditions for each of the skin patch images.
[0004] Also, in Patent Document 2 below, an image including skin is input into a database or a storage unit, one or a plurality of standardized skin units (such as the outer corner of the eye area, the area under the eyes, the nose area, the mouth area, and the cheek area, etc.) are identified from the image including skin, one or a plurality of features (such as the number of freckles, the number of pores, the amount of horny layer moisture, etc.) within the skin unit are analyzed using an AI model, and the result of the analysis is displayed.
Prior Art Documents
Patent Documents
[0005]
Patent Document 1
Patent Document 2
[0006] However, the technologies described in Patent Documents 1 and 2 above require training using images of multiple different body parts extracted from a facial image as training data. This necessitates preparing many training samples for each body part, and if the number of training samples is insufficient, the prediction accuracy will also be low.
[0007] The object of the present invention is to provide a learning method, an evaluation method, an information processing system, and a program that can generate a learning model capable of efficiently and accurately evaluating skin on multiple different parts of the face. [Means for solving the problem]
[0008] A learning method according to one embodiment of the present invention is: A group of facial part images consisting of multiple facial part images showing multiple evaluation areas within the faces of multiple users, and skin evaluation values for multiple skin evaluation items for each of the multiple evaluation areas are obtained. A database is generated by linking the aforementioned facial part image group, the multiple skin evaluation items for each evaluation area, and the skin evaluation value data. In the aforementioned database, data of skin evaluation values that differ in the evaluation area but have common or similar skin evaluation items are linked to an integrated label, which is a higher-level conceptual item for the skin evaluation item. For each of the aforementioned integrated labels, a normalized skin evaluation value is calculated by normalizing the skin evaluation value. This includes, for each of the integrated labels, taking the set of facial feature images as input and training a deep learning model to predict the normalized skin evaluation value.
[0009] Another embodiment of the present invention is a skin evaluation method using a deep learning model trained by the above-mentioned learning method, Obtain the facial feature images for evaluation of the user being evaluated. A conversion formula is obtained to calculate the original skin evaluation value from the normalized skin evaluation value learned by the deep learning model corresponding to the integrated label of the skin evaluation item to be evaluated. The evaluation facial part images are input into the deep learning model corresponding to the integrated label of the skin evaluation item to be evaluated, and the normalized skin evaluation prediction values for each evaluation area are obtained from the deep learning model. Using the normalized skin evaluation prediction value and the conversion formula, obtain the skin evaluation prediction value for each evaluation area. This includes transmitting the information regarding the acquired skin evaluation prediction values to the user's user terminal.
[0010] Another embodiment of the present invention is an information processing system that performs skin evaluation processing using a skin evaluation deep learning model learned by the learning method described above, and comprises a control unit. The control unit acquires evaluation images of the user being evaluated, A conversion formula is obtained to calculate the original skin evaluation value from the normalized skin evaluation value learned by the deep learning model corresponding to the integrated label of the skin evaluation item to be evaluated. The evaluation facial part images are input into the deep learning model corresponding to the integrated label of the skin evaluation item to be evaluated, and the normalized skin evaluation prediction values for each evaluation area are obtained from the deep learning model. Using the normalized skin evaluation prediction value and the conversion formula, obtain the skin evaluation prediction value for each evaluation area. The information regarding the acquired skin evaluation prediction value is transmitted to the user's terminal.
[0011] A program according to yet another embodiment of the present invention is provided for an information processing device. A step of obtaining a group of facial part images consisting of multiple facial part images showing multiple evaluation areas within the faces of multiple users, and skin evaluation values for multiple skin evaluation items for each of the multiple evaluation areas, The steps include generating a database that links the aforementioned facial part image group, the multiple skin evaluation items for each evaluation area, and the skin evaluation value data, A step of associating an integrated label, which is a superordinate concept item of the skin evaluation item, with data of the skin evaluation values in the database, where the evaluation sites are different and the skin evaluation items are common or similar; A step of calculating a normalized skin evaluation value obtained by normalizing the skin evaluation value for each integrated label; A step of executing learning of a deep learning model so as to input the face part image group for each integrated label and predict the normalized skin evaluation value.
Advantages of the Invention
[0012] According to the learning method according to one embodiment of the present invention, it is possible to generate a learning model that efficiently and accurately performs skin evaluation on a plurality of different parts of the face. However, this effect does not limit the present invention.
Brief Description of the Drawings
[0013] [Figure 1] The figure which showed the structure of the skin evaluation information provision system which concerns on one Embodiment of this invention. [Figure 2] The figure which showed the hardware constitutions of the skin evaluation information provision server which concerns on one Embodiment of this invention. [Figure 3] The figure which showed the structure of the database which the skin evaluation information provision server which concerns on one Embodiment of this invention has. [Figure 4] The flowchart which showed the flow of the learning process of the deep learning model by the skin evaluation information provision server which concerns on one Embodiment of this invention. [Figure 5] The figure which showed conceptually the learning process of the deep learning model by the skin evaluation information provision server which concerns on one Embodiment of this invention. [Figure 6] The flowchart which showed the flow of the skin evaluation information provision process by the skin evaluation information provision server which concerns on one Embodiment of this invention.
Embodiments for Carrying Out the Invention
[0014] Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[0015] [System Configuration] As shown in Figure 1, this system includes a skin evaluation information server 100 on the Internet 50 and multiple user terminals 200.
[0016] The skin evaluation information provision server 100 is a server (information processing device) that performs a service providing users with skin evaluation information. The skin evaluation information provision server 100 is connected to multiple user terminals 200 via the Internet 50.
[0017] The skin evaluation information server 100 receives the user's facial image along with the skin evaluation request from the user terminal 200, and based on this image, it sends the user's skin evaluation information to the user terminal 200 as an evaluation result.
[0018] The skin evaluation information server 100 uses a deep learning model 10 to estimate skin evaluation values from skin part images extracted from the user's face image in order to acquire the above skin evaluation information.
[0019] The deep learning model 10 is trained to predict skin evaluation values from face part images, using multiple face part images (group of face part images) representing multiple evaluation areas in face images provided by multiple users, and skin evaluation values for multiple skin evaluation items for each of the multiple evaluation areas, as training data. As will be described in detail later, the deep learning model 10 is trained and generated for each evaluation item in the user's face. The deep learning model 10 may be generated by a training process via the skin evaluation information provision server 100, or it may be generated by another information processing device.
[0020] User terminals 200 (200A, 200B, 200C, etc.) are terminals used by users, such as smartphones, mobile phones, tablet PCs (Personal Computers), notebook PCs, and desktop PCs. User terminals 200 access the skin evaluation information provision server 100, receive web pages containing the above-mentioned skin evaluation information, and display them on the screen using a browser or similar. Alternatively, user terminals 200 may have an application installed that supports the above-mentioned skin evaluation information provision service, and user terminals 200 may access the skin evaluation information provision server 100 and display the skin evaluation information using this application.
[0021] [Hardware configuration of the skin evaluation information server] As shown in Figure 2, the skin evaluation information server 100 includes a CPU (Central Processing Unit) 11, ROM (Read Only Memory) 12, RAM (Random Access Memory) 13, an input / output interface 15, and a bus 14 connecting these to each other.
[0022] The CPU 11 accesses RAM 13 and other memory as needed, performing various calculations and comprehensively controlling each block of the skin evaluation information server 100. Multiple CPUs 11 may be provided depending on the processing. ROM 12 is a non-volatile memory in which the OS, programs, and firmware such as various parameters to be executed by the CPU 11 are permanently stored. RAM 13 is used as a working area for the CPU 11 and temporarily holds the OS, various running applications, and various data being processed.
[0023] The input / output interface 15 is connected to a display unit 16, an operation reception unit 17, a storage unit 18, a communication unit 19, and the like.
[0024] The display unit 16 is a display device that uses, for example, an LCD (Liquid Crystal Display), an OLED (Organic ElectroLuminescence Display), or a CRT (Cathode Ray Tube).
[0025] The operation reception unit 17 is, for example, a pointing device such as a mouse, a keyboard, a touch panel, or other input device. If the operation reception unit 17 is a touch panel, the touch panel may be integrated with the display unit 16.
[0026] The storage unit 18 is a non-volatile memory such as an HDD (Hard Disk Drive), flash memory (SSD; Solid State Drive), or other solid-state memory. The OS, various applications, and various data are stored in this storage unit 18.
[0027] As will be described later, in this embodiment in particular, the storage unit 18 has a user information database, a learning-related information database, and a skin evaluation information database, in addition to programs such as applications necessary for the skin evaluation information provision process described later.
[0028] The communication unit 19 consists of various modules for wireless communication, such as a NIC (Network Interface Card) for Ethernet or a wireless LAN, and is responsible for communication processing with the user terminal 200.
[0029] Although not shown in the diagram, the basic hardware configuration of the user terminal 200 is substantially the same as that of the skin evaluation information provision server 100.
[0030] [Database configuration of the skin evaluation information server]
[0031] As shown in Figure 3, the skin evaluation information providing server 100 has a user information database 31, a learning-related information database 32, and a skin evaluation information database 33 in its storage unit 18. Note that these databases may be stored in a storage device or server externally connected to the skin evaluation information providing server 100, rather than in the storage unit 18.
[0032] The user information database 31 stores attribute information for each user of the cosmetics information service provided by the skin evaluation information server 100. This attribute information includes general information such as name, user ID for identification, age (age group), occupation, address (residential area), gender, and email address, as well as information about the user's skin, such as their perceived or understood skin type, cosmetic preferences, and any skin problems they may have. The user ID functions as account information for using this skin evaluation information service.
[0033] The learning-related information database 32 stores training data and related information necessary for training the deep learning model, such as a group of face images, a group of face part images extracted from the face image data, evaluation areas within the face to be evaluated, skin evaluation items for each evaluation area, skin evaluation values for each skin evaluation item, an integrated label attached to each skin evaluation value, and normalized skin evaluation values obtained by normalizing the skin evaluation values for each integrated label.
[0034] Evaluation items include visual evaluation values such as sagging, wrinkles, and pore visibility, as well as measurement evaluation values measured by measuring instruments, such as stratum corneum moisture content, transepidermal water loss, sebum content, and skin surface temperature.
[0035] The above visual evaluation values are assessed using a multi-level rating system based on different visual evaluation scales defined for each evaluation area, such as 0-7 or 1-5. The rating values are integers, but the average of multiple evaluators may be calculated, in which case the value will be a non-negative real number.
[0036] The above stratum corneum moisture content is measured using a stratum corneum moisture content measuring device such as SKICON® or Corneometer®, and is expressed as a real number greater than or equal to 0 (approximately 0 to 300, depending on the device specifications). Transepidermal water loss is measured using a transepidermal water loss measuring device such as Tewameter or Vapometer®, and is expressed as a real number greater than or equal to 0 (approximately 0 to 100, depending on the device specifications). Sebum amount is measured using a sebum secretion measuring device such as Sebumeter, and is expressed as an integer greater than or equal to 0 (approximately 0 to 300, depending on the device specifications). Skin surface temperature is measured using a radiation thermometer, thermometer, etc., and is expressed as a real number greater than or equal to 0 (approximately 0 to 36, depending on the device specifications and measurement site).
[0037] The evaluation sites are set for each of the above evaluation items, and the evaluation items for the above measurement values may also differ depending on the measuring instrument. For example, for stratum corneum moisture content and transepidermal water loss, the evaluation sites are the cheeks, mouth area, and outer corners of the eyes. For sebum content, the evaluation sites are the forehead, sides of the nose, bridge of the nose, temples, cheeks, and chin. For skin surface temperature, the evaluation sites are the forehead, outer corners of the eyes, under the eyes, cheeks, mouth area, and chin.
[0038] Furthermore, the visual evaluation criteria mentioned above include, for example, under the eyes (with eyes closed) and corners of the mouth for sagging skin, and the forehead, outer corners of the eyes, under the eyes (with eyes closed), and nasolabial folds for wrinkles.
[0039] The integrated label attached to each skin evaluation value is a label that indicates a higher-level concept item for the skin evaluation item, and is attached to data of skin evaluation values that are evaluated in different areas but share common or similar skin evaluation items. For example, wrinkles under the eyes, crow's feet, and nasolabial folds are in different areas but all have the common characteristic of being wrinkles, so the integrated label "wrinkles" is attached to these skin evaluation values as a higher-level concept. In addition, the integrated labels "sagging" are attached to sagging around the mouth and sagging under the eyes, "skin surface temperature" is attached to skin surface temperature of the cheeks, skin surface temperature of the forehead, and skin surface temperature of the crow's feet, and "stratum corneum moisture content" is attached to stratum corneum moisture content of the cheeks, stratum corneum moisture content of the crow's feet, and stratum corneum moisture content of the mouth.
[0040] Normalized skin evaluation values are data obtained by normalizing the skin evaluation values for different evaluation areas for each of the above integrated labels. Possible methods of normalization include scaling the maximum and minimum values so that the distribution of evaluation values falls within the range of 0 to 1, or standardizing the data so that the mean is 0 and the variance is 1.
[0041] In addition to normalization at the integrated label level as described above, skin evaluation values for each evaluation area may also be normalized separately for each evaluation area. For example, if wrinkle grades are evaluated using different numerical values or ranges, such as 8 levels for the outer corners of the eyes and 4 levels for the under-eye area, it is preferable to normalize for each evaluation area. On the other hand, if measurements are taken for multiple evaluation areas using a consistent standard, such as skin temperature, it may not be necessary to consider differences between areas.
[0042] The skin evaluation information database 33 stores each user's skin evaluation information, obtained using the deep learning model 10, in association with the user ID mentioned above. The skin score information is the skin evaluation value for each evaluation area before normalization, calculated using a conversion formula from the normalized skin evaluation value, or the corresponding evaluation information.
[0043] These databases are referenced and used as needed in the training process of the deep learning model 10 by the skin evaluation information provision server 100, which will be described later, and in the skin evaluation information provision process using the said deep learning model 10.
[0044] [Operation of the skin evaluation information server] Next, the operation of the skin evaluation information server 100 configured as described above will be explained. This operation is performed through the cooperation of the hardware of the skin evaluation information server 100, such as the CPU 11 and communication unit 19, and the software stored in the storage unit 18. For convenience, in the following explanation, the CPU 11 will be considered the main operator.
[0045] (Learning process by the skin evaluation information server) Figure 4 is a flowchart showing the learning process of the deep learning model 10 by the skin evaluation information server 100.
[0046] As shown in the figure, the CPU 11 first creates training data consisting of a large number of facial image samples (group of facial images), skin evaluation items for each part, evaluation areas, and skin evaluation values (step 41). As mentioned above, the skin evaluation items for each part include, for example, wrinkles under the eyes, wrinkles at the corners of the eyes, moisture in the cheeks, moisture around the mouth, etc. This training data is stored in the learning-related information database 32.
[0047] Next, CPU11 adds integrated label information to the training data (skin evaluation items for each area) (step 42). As mentioned above, integrated labels are labels such as "wrinkles" for wrinkles under the eyes and wrinkles at the corners of the eyes, and "moisture" for moisture on the cheeks and moisture around the mouth.
[0048] Next, CPU 11 normalizes the skin evaluation value for each of the integrated labels as described above (step 43).
[0049] Next, CPU 11 recognizes facial feature points from each face image in the group of face images included in the training data (step 44).
[0050] Next, the CPU 11 extracts small regions (face parts) containing the evaluated areas from each face image based on the recognized feature points, and creates a set of face part images (step 45).
[0051] Next, the CPU 11 associates the evaluation region in the training data with other corresponding data and the group of part images (step 46).
[0052] Then, for each of the integrated labels mentioned above, the CPU 11 takes the group of facial part images as input and trains the deep learning model 10 to predict the normalized skin evaluation value (step 47).
[0053] The CPU 11 performs the above processing for all the integrated labels generated, and generates a deep learning model 10 for each integrated label.
[0054] Figure 5 is a conceptual diagram illustrating the training process of the deep learning model.
[0055] For example, when learning evaluation values for each type of wrinkle using images of facial features such as wrinkles under the eyes, crow's feet, and nasolabial folds as training data, the general approach is to perform deep learning separately for each type of wrinkle, since each wrinkle has different characteristics and evaluation criteria (grades), in order to keep the input data as consistent as possible.
[0056] However, in this embodiment, as shown in the figure, the skin evaluation information providing server 100 integrates the above evaluation values, which differ in evaluation area and evaluation criteria, into a higher-level label (integrated wrinkle label) and performs training processing on the deep learning model.
[0057] The inventors have found that deep learning processing using such integrated labels improves prediction accuracy compared to deep learning performed for each evaluation part as described above. This reduces the total training time and the number of models required compared to the learning method for each evaluation part, and consequently, it also has the advantage of reducing the system cost when implemented as a system and improving computation speed.
[0058] The improved prediction accuracy in this embodiment may be due to the deep learning model 10 learning common features (for example, the "shadow on a line" feature in the case of wrinkles) even though the evaluation areas are different. It may also be because integrating each evaluation area with an integrated label increases the relative number of training data per integrated label.
[0059] (Skin evaluation information processing by the skin evaluation information provision server) Figure 6 is a flowchart showing the process of providing the skin analysis information.
[0060] As shown in the figure, first the CPU 11 receives a facial image along with a skin evaluation request from the user terminal 200 (step 61). The skin evaluation request may be sent, for example, via a skin evaluation application installed on the user terminal 200.
[0061] Next, the CPU 11 performs face recognition on the above face image and identifies the position of the face and each part of the face (step 62).
[0062] Next, the CPU 11 obtains learning-related information from the learning-related information database 32 for the skin evaluation item to be evaluated, including the deep learning model 10 corresponding to the skin evaluation item (integrated label), the integrated label learned by the deep learning model 10, all evaluation areas, evaluation items for each evaluation area, and a conversion formula for calculating the original skin evaluation value from the normalized skin evaluation value learned by the deep learning model 10 (step 63).
[0063] Next, the CPU 11 extracts an evaluation facial part image from the facial image, based on all the evaluation areas included in the learning-related information above, in which the position of the part corresponding to the evaluation area to be evaluated is identified (step 64).
[0064] Next, the CPU 11 inputs the extracted facial part images for evaluation into the deep learning model 10 and obtains normalized skin prediction values for each evaluation area (step 65).
[0065] Next, CPU 11 calculates the unnormalized skin prediction value for each evaluation area from the normalized prediction value using the conversion formula included in the learning-related information (step 66).
[0066] The CPU 11 then transmits information indicating the predicted skin value for the evaluation area to the user terminal (step 67). This information may be the predicted skin value itself, an evaluation comment corresponding to the predicted skin value, or an improvement comment based on the evaluation. The predicted skin value is stored in the skin evaluation information database 33.
[0067] As described above, according to this embodiment, the skin evaluation information providing server 100 can generate a deep learning model 10 that efficiently and accurately evaluates skin by assigning integrated labels to common evaluation items for multiple different parts of the face, and can use this deep learning model 10 to predict highly accurate skin evaluation values.
[0068] [Differentiation] Although embodiments of the present invention have been described above, the present invention is not limited to the embodiments described above, and various modifications can be made without departing from the spirit of the present invention.
[0069] In the embodiment described above, the skin evaluation information providing server 100, in the training process of the deep learning model 10, recognized the facial feature points of the facial image group, extracted the facial parts corresponding to the evaluation areas based on the feature points, generated the facial part image group, and associated the facial part image group with each data in the learning-related information database 32. However, these processes do not have to be performed by the skin evaluation information providing server 100; the skin evaluation information providing server 100 may receive the facial part image group generated by another device performing the above processes from the other device and use it for subsequent processing.
[0070] In the above-described embodiment, the skin evaluation information providing server 100, in the skin evaluation information providing process using the deep learning model 10, acquired a face image from the user terminal 200, identified the position of the face and each part of the face from the face image, and extracted evaluation face part images corresponding to the evaluation areas from the face image based on the data in the learning-related information database 32 and the identified positions of each part. However, these processes do not have to be performed by the skin evaluation information providing server 100, and the skin evaluation information providing server 100 may receive evaluation face part images generated by another device that has performed the above processes from the other device and use them for subsequent processing. For example, the application on the user terminal 200 may be equipped with a function to extract evaluation face part images from a face image, and the evaluation face part image extraction process may be performed on the application prior to the skin evaluation request, and this may be sent to the skin evaluation information providing server 100 together with the skin evaluation request (in place of the face image).
[0071] In the embodiment described above, only one skin evaluation information providing server 100 is shown, but the processing performed by the skin evaluation information providing server 100 may be distributed and executed across multiple servers. For example, the training process of the deep learning model 10 and the skin evaluation information provision process using the deep learning model 10 may be executed on separate servers.
[0072] Of the inventions described in the claims of this application, the invention described as "information processing method" is one in which each step is performed automatically by at least one device such as a computer through information processing by software, and not by a human using a computer or other device. In other words, the "information processing method" is an information processing method using computer software, and not a method in which a human operates a computer as a calculating tool. [Explanation of symbols]
[0073] 10…Deep-trained models 11…CPU 18...Storage section 19… Communications Department 31…User Information Database 32…Learning-related information database 33…Skin evaluation information database 100...Skin evaluation information server 200... User terminal
Claims
1. A group of facial part images consisting of multiple facial part images showing multiple evaluation areas within the faces of multiple users, and skin evaluation values for multiple skin evaluation items for each of the multiple evaluation areas are obtained. A database is generated by linking the aforementioned facial part image group, the multiple skin evaluation items for each evaluation area, and the skin evaluation value data. In the aforementioned database, data of skin evaluation values that differ in the evaluation area but have common or similar skin evaluation items are linked to an integrated label, which is a higher-level conceptual item for the skin evaluation item. For each of the aforementioned integrated labels, a normalized skin evaluation value is calculated by normalizing the skin evaluation value. For each of the aforementioned integrated labels, a deep learning model is trained to predict the normalized skin evaluation value, using the set of facial part images as input. Learning methods.
2. The system recognizes the facial feature points of the facial image group consisting of multiple facial images of multiple users, and generates the facial part image group by extracting the facial parts corresponding to the evaluation area based on the feature points. The aforementioned group of facial part images are associated with each data in the aforementioned database. The learning method according to claim 1.
3. As the aforementioned skin evaluation value, one or more visual evaluation values from wrinkles, sagging, and pore visibility are obtained. The aforementioned visual evaluation values are based on different visual evaluation scales defined for each evaluation area. The learning method according to claim 1 or 2.
4. As the skin evaluation value, one or more of the following are obtained, measured using the same measuring instrument for different evaluation areas: stratum corneum moisture content, transepidermal water loss, sebum content, and skin surface temperature. The learning method according to claim 1 or 2.
5. A skin evaluation method using a deep learning model trained by the learning method described in claim 1 or 2, Obtain the facial feature images for evaluation of the user being evaluated. A conversion formula is obtained to calculate the original skin evaluation value from the normalized skin evaluation value learned by the deep learning model corresponding to the integrated label of the skin evaluation item to be evaluated. The evaluation facial part images are input into the deep learning model corresponding to the integrated label of the skin evaluation item to be evaluated, and the normalized skin evaluation prediction values for each evaluation area are obtained from the deep learning model. Using the normalized skin evaluation prediction value and the conversion formula, obtain the skin evaluation prediction value for each evaluation area. The information regarding the acquired skin evaluation prediction value is transmitted to the user's terminal. Evaluation method.
6. From the user terminal, a facial image including at least the entire face of the user is obtained. From the aforementioned facial image, the positions of the face and each part of the face are identified. The deep learning model acquires the integrated label, all of the evaluation areas, and the skin evaluation items corresponding to all of the evaluation areas. Based on the positions of all the evaluation areas and the identified parts, the evaluation facial part images corresponding to the evaluation areas are extracted from the face image to obtain the evaluation facial part images. The evaluation method according to claim 5.
7. An information processing system that performs skin evaluation processing using a skin evaluation deep learning model trained by the learning method described in claim 1 or 2, Obtain the facial feature images for evaluation of the user being evaluated. A conversion formula is obtained to calculate the original skin evaluation value from the normalized skin evaluation value learned by the deep learning model corresponding to the integrated label of the skin evaluation item to be evaluated. The evaluation facial part images are input into the deep learning model corresponding to the integrated label of the skin evaluation item to be evaluated, and the normalized skin evaluation prediction values for each evaluation area are obtained from the deep learning model. Using the normalized skin evaluation prediction value and the conversion formula, obtain the skin evaluation prediction value for each evaluation area. The information regarding the acquired skin evaluation prediction value is transmitted to the user's terminal. control unit An information processing system equipped with the following features.
8. In an information processing device, A step of obtaining a group of facial part images consisting of multiple facial part images showing multiple evaluation areas within the faces of multiple users, and skin evaluation values for multiple skin evaluation items for each of the multiple evaluation areas, The steps include generating a database that links the aforementioned facial part image group, the multiple skin evaluation items for each evaluation area, and the skin evaluation value data, The steps include: linking integrated labels, which are higher-level conceptual items for the skin evaluation items, to data in the database of skin evaluation values that differ in the evaluation area but have common or similar skin evaluation items; The steps include: calculating a normalized skin evaluation value by normalizing the skin evaluation value for each of the integrated labels; For each of the aforementioned integrated labels, the steps include: training a deep learning model to predict the normalized skin evaluation value using the set of facial part images as input; A program that executes the command.