Information processing device, information processing method, and program

Abstract

To enable the recommendation of a different skin diagnosis from the one used to obtain the skin evaluation information if there are any problems with the skin evaluation information included in the user's skin diagnosis results, or with the changes in that information over time. [Solution] The management server 100 of the skin diagnosis management system includes an acquisition unit 1121 that acquires a first skin diagnosis result, which is the result of a first skin diagnosis including one or more skin evaluation pieces of information, based on an image captured of the user's skin; a storage unit 120 that stores the first skin diagnosis result for each user, associated with the time information of when the skin diagnosis was performed; and a notification unit 1122 that, if the first evaluation piece among the one or more evaluation pieces of information included in the stored first skin diagnosis result satisfies predetermined conditions for at least one of the evaluation piece and its temporal changes, notifies the user of second skin diagnosis information that can diagnose the first skin index indicated by the first evaluation piece.

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, and a program.

Background Art

[0002] Conventionally, a technique for evaluating a user's skin from an image of the user's face captured by a camera included in a terminal device such as a smartphone has been known. For example, Patent Document 1 describes a color unevenness evaluation device that acquires a skin image of an evaluation target person and determines a color unevenness evaluation value through predetermined analysis on the acquired skin image.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in Patent Document 1, there is a problem that the quality of the skin image varies depending on the external environment or the like when acquiring the skin image, and thus the skin evaluation result may be affected. When such an influence is considered, it is preferable to propose a skin diagnosis method capable of obtaining a skin image in a situation less affected by the external environment.

[0005] One object of the present invention is to provide an information processing apparatus, an information processing method, and a program that enable a recommendation of a different skin diagnosis from the skin diagnosis that obtained the skin evaluation information when there is some problem or the like in the skin evaluation information included in the skin diagnosis result implemented by the user or its temporal change.

Means for Solving the Problems

[0006] An information processing device according to one aspect of the present invention includes: an acquisition unit that acquires a first skin diagnosis result, which is a diagnosis result of a first skin diagnosis including one or more skin evaluation pieces of information, based on an image captured of a user's skin; a storage unit that stores the first skin diagnosis result for each user in association with the time information of when the skin diagnosis was performed; and a notification unit that, if the first evaluation piece among the one or more evaluation pieces of information included in the stored first skin diagnosis result satisfies a predetermined condition with respect to at least one of the evaluation piece and its temporal change, notifies the user of second skin diagnosis information that can diagnose a first skin index indicated by the first evaluation piece. [Effects of the Invention]

[0007] According to the present invention, if there is a problem with the skin evaluation information included in the skin diagnosis results performed by the user, or with its changes over time, it becomes possible to recommend a different skin diagnosis from the one used to obtain the skin evaluation information. [Brief explanation of the drawing]

[0008] [Figure 1] This figure shows an example of the overall configuration of a skin diagnostic management system according to one embodiment of the present invention. [Figure 2] This is a schematic diagram showing an example of the hardware configuration and functional blocks of a management server in one embodiment of the present invention. [Figure 3] This is a schematic diagram showing an example of the hardware configuration and functional blocks of a user terminal in one embodiment of the present invention. [Figure 4] This is a schematic diagram showing an example of the hardware configuration and functional blocks of a store terminal in one embodiment of the present invention. [Figure 5] This figure shows an example of the structure of user registration information. [Figure 6] This figure shows an example of the structure of input data for skin diagnosis. [Figure 7] This figure shows an example of the structure of skin diagnosis result data. [Figure 8] This figure shows an example of the structure of information for a skin diagnosis facility. [Figure 9]This figure shows an example of skin evaluation value measurement. [Figure 10] This figure shows an example of skin evaluation value measurement. [Figure 11] This sequence diagram illustrates the skin diagnosis result determination processing sequence in the skin diagnosis management system 1 according to one embodiment of the present invention. [Figure 12] This sequence diagram illustrates a modified example of the skin diagnosis result determination processing sequence in the skin diagnosis management system 1 according to one embodiment of the present invention. [Figure 13] This flowchart shows an example of the recommended store extraction process flow on the management server. [Figure 14] This diagram illustrates a policy for determining the measurement range based on a specific area of ​​a facial image. [Figure 15] This flowchart shows an example of the recommended store extraction process flow on the management server. [Figure 16] This figure shows an example of a screen displaying skin evaluation values. [Figure 17] This figure shows an example of a display screen for a skin diagnosis facility, such as a store, that is presented to the user. [Modes for carrying out the invention]

[0009] The present invention will be described below with reference to the accompanying drawings, in accordance with its embodiments.

[0010] <Configuration of the skin diagnosis management system> First, with reference to Figure 1, the configuration of the skin diagnosis management system as an information processing system according to this embodiment will be described. Figure 1 is a diagram showing an example of the overall configuration of the skin diagnosis management system 1 in this embodiment. The skin diagnosis management system 1 in this embodiment includes a management server 100 that mainly performs data processing for managing skin diagnoses, a user terminal 200 which is a mobile terminal owned by the user and has a program used for skin diagnosis processing (hereinafter also referred to as the "skin diagnosis app") implemented on it, and a store terminal 300 which includes a skin diagnosis device installed in various facilities including stores such as cosmetics stores and department stores.

[0011] The management server 100, the user terminal 200, and the store terminal 300 are interconnected and communicable via a communication network NW. The communication network NW may be an appropriate wired or wireless communication line such as a LAN, a WAN, a dedicated line, a mobile communication network, or the Internet.

[0012] <Configuration example of management server 100> Next, a configuration example and functions of the management server 100 will be described with reference to FIG. 2. FIG. 2 is a schematic diagram showing a hardware configuration example and functional blocks of the management server 100 in the present embodiment. The management server 100 has a configuration as a general computer that can communicate with external devices such as the user terminal 200 and the store terminal 300 via the communication network NW.

[0013] The management server 100 includes a processing unit 110, a storage unit 120, an input / output unit 130, and a communication unit 140.

[0014] The processing unit 110 is a processor that performs various operations and controls necessary for the operation of the management server 100, and realizes the functions of the management server 100 by executing various programs.

[0015] The storage unit 120 stores various programs executed by the processor constituting the processing unit 110, data used during program execution, and also functions as a work area temporarily used for performing various processes. The storage unit 120 is configured, for example, as various memory chips including a non-volatile memory and a volatile memory. In the present embodiment, various data such as user face image data, skin diagnosis result data, etc. acquired from the user terminal 200 and the store terminal 300, user registration information, login information, skin diagnosis reservation status at the store for each user, and information indicating the usage status of the store are stored in the data storage unit 121 provided in the storage unit 120.

[0016] The input / output unit 130 includes an input unit that accepts external operations such as a keyboard, mouse, tablet, and touch panel, an output unit consisting of a display panel such as an LCD, and a speaker and printer.

[0017] The communication unit 140 controls the communication between the management server 100 and external devices such as user terminals 200 and store terminals 300. The communication unit 140 is composed of various wired or wireless communication modules, such as a network interface card (NIC).

[0018] Next, the functions implemented by the processing unit 110 of the management server 100 will be described. As illustrated in Figure 2, the processing unit 110 of this embodiment includes a reservation management unit 111, a skin diagnosis management unit 112, a user management unit 113, and an input / output control unit 114. The functional units provided in the processing unit 110 are not limited to these and can be appropriately modified and added according to the functions that the management server 100 should have.

[0019] The reservation management unit 111 receives skin diagnosis reservations from user terminals 200 and generates and updates information indicating the availability of reservations, such as the store and the date and time of the appointment, based on the information related to the skin diagnosis reservation and the reservation information for each user recorded at each store terminal 300. The reservation management unit 111 also has the function of generating and transmitting screen data to display reservation-related information on the display screen of the user terminal 200.

[0020] The skin diagnosis management unit 112 performs data processing to provide users with necessary information to ensure that their skin diagnosis is performed more accurately, based on facial image data and skin diagnosis result data of each user acquired from the user terminal 200 and the store terminal 300. This includes, as needed, recommendations for skin diagnosis at the store terminal 300 or advice on taking facial images at the user terminal 200. More specifically, the skin diagnosis management unit 112 comprises an acquisition unit 1121, a notification unit 1122, and a setting unit 1123. The acquisition unit 1121 has the function of acquiring the user's skin diagnosis results from the user terminal 200 and the store terminal 300, including evaluation information on various skin indicators that are indices for evaluating the condition of the skin, such as blemishes and transparency. Skin evaluation information is information that shows the evaluation result of the skin regarding a certain skin indicator, obtained as a result of performing a skin diagnosis. The evaluation information may include skin evaluation values ​​as numerical data obtained for skin indicators such as blemish index and transparency. In addition to evaluation values, the evaluation information may also include qualitative evaluations such as "good, normal, need improvement." The notification unit 1122 has the function of determining whether it is necessary to perform a different skin diagnosis on the store terminal 300 than the skin diagnosis that can be performed on the user terminal 200, based on the evaluation information, which is the skin diagnosis result acquired by the acquisition unit 1121, and whether or not there are any changes over time, and notifying the user terminal 200. The setting unit 1123 has the function of setting the measurement range for a specific part of the user's face when performing the different skin diagnosis, from the management server 100 or the user terminal 200.

[0021] The user management unit 113 stores user information, including registration information and login information for each user, entered from the user terminal 200, for use in managing each user's reservations and skin diagnosis data, and performs management such as updating.

[0022] The input / output control unit 114 has the function of controlling the exchange of input / output data between the processing unit 110, the storage unit 120, the input / output unit 130, and the communication unit 140. For example, the input / output control unit 114 has the function of passing information such as skin diagnosis reservation information, physical and mental measurement data for skin diagnosis, and skin diagnosis result data from the communication unit 140 to the reservation management unit 111 and the skin diagnosis management unit 112, respectively.

[0023] <Example configuration of user terminal 200> Next, an example configuration of the user terminal 200 and its functions will be described with reference to Figure 3. Figure 3 is a schematic diagram showing an example hardware configuration and functional blocks of the user terminal 200 in this embodiment. The user terminal 200 has a configuration as a general computer, capable of communicating with external devices such as the management server 100 and store terminals 300 via a communication network NW. Specifically, the user terminal 200 can be configured as a mobile communication terminal such as a smartphone.

[0024] The user terminal 200 includes a processing unit 210, a storage unit 220, an input / output unit 230, a communication unit 240, an imaging unit 250, and a GNSS unit 260.

[0025] The processing unit 210 is a processor that performs various calculations and control processes necessary for the operation of the user terminal 200, and realizes the functions of the user terminal 200 by executing various programs.

[0026] The storage unit 220 stores various programs, including a skin diagnosis application executed by the processor constituting the processing unit 210, and data used during program execution, and also functions as a work area temporarily used for various processing tasks. The storage unit 220 is configured, for example, as various memory chips equipped with non-volatile memory and volatile memory. In this embodiment, the data storage unit 221 provided in the storage unit 220 stores physical and mental measurement data for skin diagnosis, facial image data, skin diagnosis result data, user registration information for using the skin diagnosis management system 1, and skin diagnosis reservation information at stores.

[0027] The input / output unit 230 includes an input unit that accepts user operations, such as buttons, a touch panel, and a microphone; a display panel consisting of an LCD or the like; and an output unit such as a speaker.

[0028] The communication unit 240 controls the communication between the user terminal 200 and external devices such as the management server 100 and the store terminal 300. The communication unit 240 is composed of various wired or wireless communication modules, such as a network interface card (NIC) and an NFC communication module such as BLE.

[0029] The imaging unit 250 is equipped with a camera that enables the user's facial image to be captured by the skin diagnosis application on the user terminal 200.

[0030] The GNSS unit 260 has the function of acquiring time information and location information from satellite signals received from positioning satellites such as GPS satellites. For example, the GNSS unit 260 can derive the position of its own device from satellite signals received from GPS satellites and provide it to the management server 100 as user location information.

[0031] Next, the functions implemented by the processing unit 210 of the user terminal 200 will be described. As illustrated in Figure 3, the processing unit 210 of this embodiment includes a skin diagnosis unit 211, a user management unit 212, a reservation unit 213, and an input / output control unit 214. The functional units provided in the processing unit 210 are not limited to these and can be appropriately modified and added according to the functions that the user terminal 200 should have.

[0032] The skin diagnosis unit 211 is implemented as a function of the skin diagnosis application installed on the user terminal 200. The skin diagnosis unit 211 performs skin diagnosis processing using skin diagnosis data such as facial image data entered by the user and outputs skin diagnosis result data. Alternatively, the skin diagnosis processing by the skin diagnosis unit 211 may involve transferring the skin diagnosis data such as facial image data entered by the user to an external device on the network, such as a management server 100, performing skin diagnosis processing on that external device, and then transferring the obtained skin diagnosis result data to the user terminal 200.

[0033] The user management unit 212 stores user information, including registration information and login information of users who use the skin diagnosis app, for use in managing user reservations and skin diagnosis data, and performs management such as updating.

[0034] The reservation unit 213 is implemented as a function of the skin diagnosis application installed on the user terminal 200. The reservation unit 213 has functions such as sending reservation request information entered by the user from the skin diagnosis application to the management server 100, displaying the reservation status based on the reservation status display data received from the management server 100, and sending information about the store and time slot desired by the user to the management server 100.

[0035] The input / output control unit 214 has the function of controlling the exchange of input / output data between the processing unit 210 and the storage unit 220, input / output unit 230, communication unit 240, imaging unit 250, and GNSS unit 260. For example, the input / output control unit 214 has the function of performing processes such as passing user information from the communication unit 240 and reservation status information received from the management server 100 to the skin diagnosis unit 211 and reservation unit 213, respectively.

[0036] <Example configuration of store terminal 300> Next, an example configuration of the store terminal 300 and its functions will be described with reference to Figure 4. Figure 4 is a schematic diagram showing an example hardware configuration and functional blocks of the store terminal 300 in this embodiment. The store terminal 300 has a configuration as a general computer, capable of communicating with external devices such as a management server 100 and a user terminal 200 via a communication network NW. Specifically, the store terminal 300 can be suitably configured to provide a facial image acquisition environment unique to a store, and to include a camera for facial image acquisition, skin measurement equipment such as a microscope that can observe the condition of the skin more precisely than a normal camera, and a large display screen suitable for counseling based on skin diagnosis results. The store terminal 300 may be realized as a facility capable of performing skin diagnosis based on self-taken facial image acquisition, such as a photo booth installed in the city. In short, the store terminal 300 can be realized in various forms as a facility equipped with an imaging device that provides a stable facial image acquisition environment and can obtain high-quality facial image data. In other words, in this specification, the store terminal 300 includes not only skin diagnostic devices installed in stores such as cosmetics stores and department stores, but also various types of facilities as described above without any particular limitations. The store terminal 300 is preferably equipped not only with a facial image capturing device, but also with a microscope that can magnify and observe the skin surface more precisely, a temperature sensor that can detect the temperature of the skin surface, and means for performing measurements etc. by being in close proximity to or in contact with the skin of a part of the face, such as an elastic meter for mechanically measuring skin firmness, and a moisture meter for checking skin moisture.

[0037] The store terminal 300 includes a processing unit 310, a storage unit 320, an input / output unit 330, a communication unit 340, and a skin measurement unit 350.

[0038] The processing unit 310 is a processor that performs various calculations and control processes necessary for the operation of the store terminal 300, and realizes the functions of the store terminal 300 by executing various programs.

[0039] The storage unit 320 stores various programs executed by the processor constituting the processing unit 310, as well as data used during program execution, and also functions as a work area temporarily used for various processing tasks. The storage unit 320 is configured, for example, as various memory chips equipped with non-volatile memory and volatile memory. In this embodiment, the data storage unit 321 provided in the storage unit 320 stores facial image data for skin diagnosis, skin diagnosis result data, skin diagnosis reservation information at the store, and the like.

[0040] The input / output unit 330 includes an input unit that accepts operations from store staff or users, such as buttons and touch panels, a display panel consisting of an LCD or the like, and an output unit such as a speaker and printer.

[0041] The communication unit 340 controls the communication between the store terminal 300 and external devices such as the management server 100 and user terminals 200. The communication unit 340 is composed of various wired or wireless communication modules, such as a network interface card (NIC) and an NFC communication module such as BLE.

[0042] The skin measurement unit 350 is equipped with a camera that enables the store terminal 300 to capture images of the user's face. The camera installed in this skin measurement unit 350 can be one that has superior performance in terms of resolution, color reproduction, etc., compared to the camera installed in a smartphone or the like, which serves as the user terminal 200. In addition, the skin measurement unit 350 can also be equipped with various measurement means such as the aforementioned microscope, temperature sensor, elasticity meter, and moisture meter.

[0043] Next, the functions realized by the processing unit 310 of the store terminal 300 will be described. As illustrated in Figure 4, the processing unit 310 of this embodiment includes a skin diagnosis unit 311, a reservation processing unit 312, and an input / output control unit 313. The functional units provided in the processing unit 310 are not limited to these and can be appropriately changed or added depending on the functions that the store terminal 300 should have.

[0044] The skin diagnosis unit 311 is implemented as a function of the skin diagnosis software installed on the store terminal 300. The skin diagnosis unit 311 has the function of performing skin diagnosis processing using skin diagnosis data such as facial image data acquired from the user at the store and outputting skin diagnosis result data. The skin diagnosis unit 311 also implements functions that work in conjunction with the skin diagnosis application running on the user terminal 200, such as displaying a specific area on the display screen of the skin diagnosis device when performing a precise diagnosis of a specific area of ​​the face, as described later.

[0045] The reservation processing unit 312 is implemented as a function of the skin diagnosis software. The reservation processing unit 312 has functions such as receiving and managing new reservation information from the management server 100, and when a user who has made a reservation arrives, it retrieves the reservation information and works in cooperation with the skin diagnosis unit 311 to have the skin diagnosis unit 311 execute the reserved skin diagnosis process.

[0046] The input / output control unit 313 has the function of controlling the exchange of input / output data between the processing unit 310 and the storage unit 320, input / output unit 330, communication unit 340, and skin measurement unit 350. For example, the input / output control unit 313 has the function of executing processes such as passing reservation information received from the management server 100 via the communication unit 340 to the skin diagnosis unit 311 and the reservation processing unit 312, respectively.

[0047] <Example of data used in the skin diagnosis management system 1 of this embodiment> Next, an example of the data structure used in the skin diagnosis management system 1 of this embodiment will be described.

[0048] [User Registration Information] Figure 5 shows an example of the user registration information structure. User registration information is data generated when a user registers to use the skin diagnosis management system 1 of this embodiment. For example, it consists of data that associates a user ID, which is a unique identification code for each user, with the user's name, date of birth, and gender. The data items of the user registration information are examples and are not limited to these items.

[0049] [Input data for skin diagnosis] Figure 6 shows an example of the configuration of input data for skin diagnosis. The input data for skin diagnosis is skin diagnosis input data such as facial image data entered by the user using a skin diagnosis application installed on the user terminal 200 or on the store terminal 300. The input data for skin diagnosis is stored in the data storage unit 121 of the management server 100, and can be referenced or reused as existing data when a user undergoes a skin diagnosis. In the example in Figure 6, the input data for skin diagnosis is stored in the data storage unit 121 of the management server 100, associated with the user ID, the date and time the data was recorded, the recording method, and physical and mental data measured or entered by the user, such as height, weight, BMI, body fat percentage, heart rate, blood pressure, and stress level, as well as facial image data (image file) captured on the user terminal 200 or store terminal 300. The data items of the input data for skin diagnosis are examples and are not limited to these items.

[0050] [Skin diagnosis results data] Figure 7 shows an example of the structure of skin diagnosis result data. Skin diagnosis result data is data that shows the results of a skin diagnosis performed by the skin diagnosis application on the user terminal 200 or the store terminal 300. In the example in Figure 7, the skin diagnosis result data is associated with the user ID, the diagnosis date and time information related to the skin diagnosis result, and the diagnosis result data for skin diagnosis items (skin indicators) such as moisture, barrier strength, brightness, spots, firmness, and transparency, and is stored in the data storage unit 121 of the management server 100. The items included in the skin diagnosis result data are examples and are not limited to the example in Figure 7.

[0051] [Skin Diagnosis Facility Information] Figure 8 shows an example of the structure of skin diagnosis facility information, which is information about various types of facilities where users of the skin diagnosis management system 1 can receive skin diagnosis. The skin diagnosis facility information in Figure 8 is structured with the facility name, location, applicable skin indicators, measurement methods, and user visit records all linked together. The example in Figure 8 is information about "Store A," and the address of the location, the type of skin indicator, the measurement methods, and the user visit record are recorded as shown in the example. The location may be shown as location coordinate data. The user visit record may include the ID of the user who visited the facility. The skin diagnosis facility information is used as basic data when the notification unit 1122 of the management server 100 recommends a store or other facility suitable for the user.

[0052] <How skin diagnosis results are used> Here, we will explain how the skin diagnosis results are used in the skin diagnosis management system 1 of this embodiment. Figure 9 schematically shows an example of skin evaluation value measurement. The upper histogram shows the blemish index measured on the right side of the face, and the lower histogram shows the blemish index measured on the left side of the face, with the horizontal axis showing the measurement date. In this case, compared to the temporal change of the upper histogram, it can be seen that the values ​​in the lower histogram have decreased significantly over a period of about one week. In this case, it is unlikely that the blemishes on the left side of the face would worsen so rapidly compared to the right side, so it can be inferred that there is some cause in the facial image acquisition environment at the user terminal 200. In such a case, the notification unit 1122 of the management server 100 that acquired the skin diagnosis result data can advise the user to conduct a skin diagnosis at a store or other facility, or to retake facial image data at the user terminal 200. Here, the conditions for determining that there is some problem with facial image acquisition, etc., can be assumed as predetermined conditions. When viewed over relatively long periods such as 6 months or 1 year, evaluation information for actual skin indicators may change based on improvements in skin condition, etc. Therefore, if a change exceeding a predetermined threshold, for example 20%, is observed in the skin evaluation value, which indicates evaluation information, over a relatively short period such as 1 week or 1 month, it can be determined that the predetermined conditions have been met. When such conditions are met, services such as suggesting skincare more suitable for the user can be provided by guiding the user to a skin diagnosis at the aforementioned store or other facility. Note that the above-mentioned period and threshold values ​​are examples and may be set as appropriate. For the period, for example, a longer or shorter period may be set regardless of the above example, and different periods may be set for each skin indicator. For the threshold, for example, the user may specify the threshold themselves, or it may be set based on the past evaluation values ​​of a specific user's skin indicator, for example, blemishes. More specifically, for example, it may be set based on the rate of change of evaluation values ​​for past skin indicators; that is, if the trend of past rate of change is within 10% on average compared to the previous one, the threshold may be set to 20%. In addition, the threshold may be set based on external environmental factors such as temperature and humidity.Furthermore, the system may determine whether a predetermined condition is met not only based on the evaluation value of a single skin indicator, but also based on the evaluation values ​​of multiple skin indicators. For example, the system may be configured to determine that a predetermined condition is met if a change exceeding a threshold occurs in more than a predetermined number of skin indicators.

[0053] Figure 10 shows an example of transparency index measurement on the user terminal 200. For example, as shown in the upper line graph, if the transparency index of the skin diagnosis results improves sharply from Thursday to Friday, it is unlikely that the transparency of the user's skin improved all at once. Therefore, as with the case of the blemish index mentioned above, it is possible to infer that there is some cause in the facial image acquisition environment on the user terminal 200. Also, as shown in the lower line graph, even if there is no large change over time, if the transparency index value remains consistently below the good zone, it can be inferred that the skincare based on the skin diagnosis results is not working effectively. In this case as well, this system recommends a skin diagnosis at a store or other facility in order to provide the user with skincare that is expected to be more effective. In other words, not only when the evaluation information for a certain skin index fluctuates beyond a certain range over time, but also when the change over time is relatively small, for example, if the evaluation information value remains 20% or more below the target value for a month, the system can determine that the predetermined conditions have been met and guide the user to a store or other facility. The above-mentioned period and threshold values ​​are examples only, and as explained with the example of blemishes as a skin indicator in Figure 9, they may be set as appropriate.

[0054] <Example of data processing performed on the management server based on online skin diagnosis results> The data processing performed in the skin diagnosis management system 1 having the configuration exemplified above will be explained with reference to Figure 11. Figure 11 is a sequence diagram showing an example of data processing based on online skin diagnosis results, mainly performed on the management server 100 of the skin diagnosis management system 1. As a prerequisite, it is assumed that the management server 100 is operational and the skin diagnosis application is running on the user terminal 200.

[0055] In step S1, the user management unit 113 of the management server 100 receives user registration information or a login request from the user terminal 200. User registration information is received when the user attempts to use the skin diagnosis management system 1 for the first time. In step S2, the acquisition unit 1121 in the skin diagnosis management unit 112 of the management server 100 receives skin image data captured by the user terminal 200 using the user terminal 200, such as a smartphone. This skin image data is, for example, image data of a face image captured by a camera on the user terminal 200. In this embodiment, the following explanation will assume the use of face image data.

[0056] In step S3, the acquisition unit 1121 of the management server 100 performs skin diagnosis processing using the received skin image data. Skin diagnosis processing is a process that calculates and outputs evaluation information for predetermined skin indicators from the current face image data, based on the correlation between, for example, face image data acquired and analyzed for the same user in the past and evaluation information (evaluation values ​​if the evaluation information is numerical data) of skin indicators as a diagnosis result, or the learning results thereof. The skin diagnosis results also include time information such as the date and time the skin diagnosis was performed. The obtained skin indicator evaluation information is stored in the data storage unit 121 of the management server 100. Alternatively, the skin diagnosis processing may be performed by a skin diagnosis application running on the user terminal 200, and the skin diagnosis result data may be sent to the acquisition unit 1121 of the management server 100 along with the face image data.

[0057] In step S4, the notification unit 1122 in the skin diagnosis management unit 112 of the management server 100 compares the skin diagnosis result data received from the user terminal 200 with past skin diagnosis result data of the same user stored in the data storage unit 121 of the storage unit 120 to determine whether there are any skin indicators that meet predetermined conditions. These predetermined conditions include, for example, the following conditions. However, other conditions are not excluded. • The skin index evaluation value for a specific area of ​​the face has changed significantly over time, exceeding a predetermined threshold compared to the previous skin diagnosis. For example, the blemish index has fluctuated by more than 20% in the past month. • In the most recent N skin diagnosis results, the difference between the lowest and highest skin index evaluation values ​​exceeds a predetermined threshold. The difference between the evaluation values ​​of skin indicators that are correlated with each other exceeds a predetermined threshold compared to the previous skin diagnosis. Here, two or more skin indicators that are correlated with each other are stored in the storage unit 120 of the management server 100 in association with each other. For example, if a correlation is found between the skin indicators blemishes and wrinkles, the skin indicators blemishes and wrinkles are stored in association with each other.

[0058] If it is determined in step S4 that the predetermined conditions are not met for any of the skin indicators (step S4, No), this processing flow ends. If it is determined that there is a skin indicator for which the predetermined conditions are met (step S4, Yes), the notification unit 1122 of the management server 100 selects a recommended store suitable for measuring the skin indicator for which the predetermined conditions are met from the skin diagnosis facility information in the data storage unit 121 (step S5), notifies the user terminal 200, and ends the process (step S6).

[0059] With this type of data processing, if a problem is suspected with the skin diagnosis results based on facial image data captured by the user terminal 200, the user can be guided to a store or other facility equipped with a facial image capture environment, thereby providing a more appropriate skin diagnosis and skincare based on that diagnosis.

[0060] Next, we will describe a modified version of the data processing in the management server 100 described above. Figure 12 is a sequence diagram showing an example of the data processing flow related to the modified version. The basic processing flow is the same as in the example in Figure 11, so for Figure 12, we will mainly describe the processing that differs from Figure 11.

[0061] The process from user registration / login in Step 1 to the determination of whether the predetermined conditions are met in the management server 100 in Step S4 is the same as the example processing flow in Figure 11. If it is determined in Step S4 that the predetermined conditions are not met for any of the skin indicators (Step S4, No), this processing flow ends.

[0062] On the other hand, if it is determined that there is a skin indicator that meets the predetermined conditions (step S4, Yes), in step S7, the notification unit 1122 of the management server 100 sends a notification to the user terminal 200 suggesting that the face image be recaptured. This suggestion to recapture is made because it is presumed that the reason the predetermined conditions were not met was due to a problem with the face image capture environment at the user terminal 200. If the user chooses to recapture in step S8 (step S8, Yes), the process proceeds to step S2. If the user chooses not to recapture (step S8, No), this is received by the notification unit 1122 of the management server 100 in step S9. In step S10, the skin diagnosis results from step S3 are stored in the data storage unit 121, and in steps S5 and S6, similar to Figure 11, stores suitable for the met predetermined conditions are extracted from the skin diagnosis facility information and notified to the user terminal 200, ending this processing flow.

[0063] This type of data processing provides an opportunity to capture a proper facial image through re-capture if there is a problem with capturing a facial image on the user terminal 200. This benefits the user by allowing them to obtain accurate skin diagnosis results without having to visit a store.

[0064] <Example of data processing flow for recommended store extraction process> Next, we will explain the recommended store extraction process executed in the notification unit 1122 of the management server 100. Figure 13 shows a flowchart illustrating the data processing flow of the recommended store extraction process. When a predetermined condition is met for one skin indicator, it is sufficient to extract and present to the user stores that can provide diagnosis for that skin indicator. However, when predetermined conditions are met for multiple skin indicators, a method of presenting recommended stores that takes user convenience into consideration can be considered. Here, we illustrate such a method in the form of a data processing flow.

[0065] In step S20, the notification unit 1122 of the management server 100 determines whether predetermined conditions are met for the evaluation values ​​of multiple skin indicators in the skin diagnosis result. If it is determined that the predetermined conditions are not met for multiple skin indicators (step S20, No), this process ends. If it is determined that the predetermined conditions are met for multiple skin indicators (step S20, Yes), the notification unit 1122 prioritizes extracting stores from the skin diagnosis facility information that have many skin indicators for which the predetermined conditions are met and for which diagnosis is possible (step S21). That is, if the predetermined conditions are met for two skin indicators, the notification unit 1122 first prioritizes searching for and extracting stores that can handle those two skin indicators.

[0066] In step S22, the user's selection regarding the priority of store extraction, which is input from the user terminal 200, is examined. If the user prioritizes stores they have visited before (step S22, visited before), the notification unit 1122 extracts stores from the skin diagnosis facility information for which the user has a record of visiting and terminates the process. If the user prioritizes stores close to their current location, as indicated by the location of the user terminal 200 (step 22, close), the notification unit 1122 extracts stores from the skin diagnosis facility information that are within a predetermined distance from the location information indicating the user's current location and terminates the process.

[0067] This data processing allows the system to recommend stores that closely match the user's selection criteria. The priority order for extracting recommended stores is not limited to the data processing flow example in Figure 13. Various modifications are possible, such as allowing the user to set the priority order in the skin diagnosis app on the user terminal 200 beforehand, or pre-setting a default priority order. Furthermore, even if only one skin indicator satisfies the predetermined conditions, the system may prioritize the extraction of recommended stores based on the user's visit records or other factors. In addition, if the number of skin diagnosis information items that can be displayed on the user terminal 200 is set, the notification unit 1122 may, for the user's convenience, prioritize displaying information on stores that can perform skin diagnosis for multiple skin indicators.

[0068] <Example of data processing based on skin diagnosis results for a specific area> Next, we will explain examples of store recommendations and other data processing based on skin diagnosis results for specific areas. Figure 14 schematically shows how users are guided to stores based on skin diagnosis results for specific areas. In the illustrated example, we assume a situation where a user performs a skin diagnosis on a facial image using the skin diagnosis app on user terminal 200, and obtains a skin diagnosis result for a specific area, namely, decreased transparency in the left cheek. In this case, the user is recommended to have a skin diagnosis at a store that can precisely diagnose the left cheek using a microscope, for example. At this time, any matters that need to be noticed by the user regarding the skin diagnosis of a specific area can be displayed on the screen of user terminal 200. For example, a message such as "You need to remove your makeup to measure the area" may be displayed.

[0069] The user can specify the area they wish to have measured with a microscope by touching the facial image displayed on the touch panel display of the user terminal 200. The skin diagnosis app on the user terminal 200 is configured to allow the user to make a reservation for a store diagnosis after specifying the measurement area. The facial image data, including the measurement area specified by the user on the user terminal 200, is sent to the store terminal 300 at the reserved store. When the user visits the store to receive a skin diagnosis, they can confirm the measurement area they specified on the display screen of the store terminal 300.

[0070] The following describes the extraction process for stores that perform skin diagnosis of specific areas, executed by the notification unit 1122 and the setting unit 1123 of the management server 100, with reference to Figure 15. Figure 15 is a flowchart showing an example of the data processing flow. When processing begins, the notification unit 1122 determines in step 30 whether predetermined conditions are met for a specific area of ​​the user's face. Here, the predetermined conditions are the same as the determination conditions in step S4 of the processing flow illustrated in Figure 11. If it is determined that the predetermined conditions are not met for a specific area of ​​the user's face (step 30, No), this process ends. If it is determined that the predetermined conditions are met for a specific area of ​​the user's face (step 30, Yes), the notification unit 1122 extracts stores equipped with measurement means capable of handling that specific area, such as a microscope, from the skin diagnosis facility information and notifies the user (step S31).

[0071] In step S32, the notification unit 1122 sends a notification as display screen data prompting the user to specify a measurement range for a specific area. Next, the notification unit 1122 determines whether the face image data acquired from the user terminal 200 contains the measurement range specified by the user (step S33). If it is determined that there is a user-specified measurement range (step S33, Yes), the setting unit 1123 of the management server 100 determines the user-specified range as the measurement range at the store (step S34). If it is determined that there is no user-specified measurement range (step S33, No), the setting unit 1123 of the management server 100 determines the measurement range based on the skin evaluation value for the specific area of ​​the face image acquired from the user terminal 200 (step S35). In step S36, the setting unit 1123 of the management server 100 transmits the face image data with the determined measurement range superimposed to the store terminal 300 of the reserved store, based on the reservation made by the user terminal 200 for the stores extracted in step S31, in a displayable format. This allows the store terminal 300 to display a facial image, including the measurement range, on its display screen.

[0072] According to this data processing, if a problem is found in a specific area of ​​the user's skin, it is possible to extract and present stores or other facilities that can perform a skin diagnosis based on more precise measurements such as a microscope, as a different skin diagnosis from a regular skin diagnosis at a store or other facility, i.e., a second skin diagnosis based on the first skin diagnosis performed on the user terminal 200. This allows for the provision of skincare that is more suitable for the user.

[0073] <Example of screen display for skin diagnosis results and recommended facilities> Next, we will explain examples of screen displays for skin diagnosis results and recommended facilities on the user terminal 200 and the store terminal 300. Figure 16 shows an example of skin diagnosis result display, where the black and white columns illustrate the spot index obtained as a diagnosis result from the user terminal 200, and the gray column illustrates the spot index from the store terminal 300. By displaying the diagnosis results from the user terminal 200 and the store terminal 300 in a distinguishable manner in this way, the user or store counselor can compare the two to analyze problems with the imaging environment at the user terminal 200, changes over time for specific skin indicators, etc.

[0074] Figure 17 shows an example of a display screen when showing stores and other facilities extracted based on the user's skin diagnosis results. In the example in Figure 17, it is assumed that predetermined conditions, such as large temporal fluctuations, are met for three skin indicators: blemishes, transparency, and firmness. In Figure 17, stores A, which can diagnose all three skin indicators, store C, which can diagnose blemishes and transparency, and store D, which can diagnose only firmness, are extracted. The extracted stores are displayed in order of the number of skin indicators they can diagnose, starting with store A, which can diagnose all skin indicators. This allows the user to easily find a store that is convenient for them, where they can receive diagnoses for as many skin indicators as possible at once, and effectively guides the user to skin diagnosis at stores and other facilities. If the predetermined conditions are met for two skin indicators, the system may be designed to distinguish between facilities that can diagnose both skin indicators and facilities that can diagnose only one skin indicator.

[0075] As described above, the management server 100 of the skin diagnosis management system 1 according to this embodiment includes: an acquisition unit 1121 that acquires a first skin diagnosis result, which is a diagnosis result of a first skin diagnosis including one or more skin evaluation pieces of information, based on an image captured of the user's skin; a storage unit 120 that stores the first skin diagnosis result for each user in association with the time information of when the skin diagnosis was performed; and a notification unit 1122 that notifies the user of second skin diagnosis information that can diagnose the first skin index indicated by the first evaluation piece of information, if the first evaluation piece of information among the one or more evaluation pieces of information included in the stored first skin diagnosis result satisfies predetermined conditions for at least one of the evaluation piece of information and its temporal change.

[0076] In this way, if there is a problem with the skin evaluation information included in the skin diagnosis results performed by the user, or with its changes over time, it becomes possible to recommend a different skin diagnosis from the one that obtained the aforementioned skin evaluation information.

[0077] The evaluation information includes skin evaluation values, and the notification unit 1122 may notify the second skin diagnosis information if the difference between the first skin evaluation value and the first skin evaluation value acquired in the past is greater than a preset threshold, and may not notify the second skin diagnosis information if the difference is less than or equal to the threshold.

[0078] In this way, if the evaluation information for a certain skin indicator fluctuates beyond a predetermined range, the user can be notified of the information from a second skin diagnosis conducted at a store or other facility, thereby guiding them to a different skin diagnosis than the first one.

[0079] The notification unit 1122 may, if the predetermined conditions are met, notify the user of a suggestion to perform the first skin diagnosis again, and if the user accepts the suggestion, execute a process to guide the user to perform the first skin diagnosis again, and if the user does not accept the suggestion, execute a process to provide information regarding the second skin diagnosis.

[0080] In this way, if a predetermined change or other issue is observed in the skin diagnosis results performed through the skin diagnosis app on the user terminal 200, the user can be offered the option of re-diagnosing using the skin diagnosis app or being guided to a different skin diagnosis than the first one.

[0081] When the number of displayable skin diagnostic information items is set, the notification unit 1122 can, when the first evaluation information and the second evaluation information (which is other evaluation information) satisfy the predetermined conditions, display the second skin diagnostic information, which can measure the corresponding first and second skin indicators, to the user terminal with priority over other skin diagnostic information.

[0082] In this way, if certain conditions such as fluctuations are met in the evaluation information of two or more skin indicators, the user will be presented with skin diagnostic information that can diagnose those skin indicators, thereby improving user convenience.

[0083] The notification unit 1122 may, when the first evaluation information and the second evaluation information, which is other evaluation information, satisfy the predetermined conditions, display the skin diagnosis information that can be measured for the first skin index and the second skin index with priority over the skin diagnosis information for which only one of them can be measured.

[0084] This approach presents users with skin diagnostic information that allows them to evaluate both the first and second evaluation pieces of data simultaneously, thus increasing user convenience.

[0085] The notification unit 1122 can, when the first evaluation information and the second evaluation information, which is other evaluation information, satisfy the predetermined conditions, display the measurable skin diagnosis information for the first skin index and the second skin index in a way that allows them to be distinguished from the skin diagnosis information for which only one of them can be measured.

[0086] This approach enhances user convenience by clearly distinguishing between skin diagnostic information that can diagnose both the first and second skin indicators simultaneously, and skin diagnostic information that can diagnose only one of the skin indicators.

[0087] If the notification unit 1122 performs the notified second skin diagnosis, it may display the second skin diagnosis result on a screen capable of displaying the first skin diagnosis result performed on the user terminal in a manner that allows it to be identified as the first skin diagnosis result.

[0088] In this way, users can compare the results of the first skin diagnosis with the results of the second skin diagnosis, which is different from the first skin diagnosis.

[0089] The first skin diagnosis measures the evaluation information for each part of the user's face, and if the first evaluation information for a predetermined part of the user's face is determined to satisfy the predetermined conditions, the notification unit may notify the user of second skin diagnosis information, which enables a different skin diagnosis for the predetermined part than the first skin diagnosis.

[0090] In this way, users can specify a particular area of ​​their face and be guided to a different skin diagnosis method than the first one.

[0091] The management server 100 may also include a setting unit 1123 that sets a skin diagnosis with the predetermined area as the measurement range when a proposal for a skin diagnosis different from the first skin diagnosis is accepted based on the notified skin diagnosis information.

[0092] This allows the measurement points to be set automatically.

[0093] If a skin diagnosis proposal different from the first skin diagnosis is accepted based on the notified skin diagnosis information, the setting unit 1123 may cause the user terminal 200 to display a screen prompting the user to specify the measurement range.

[0094] In this way, users can set their desired measurement range using the user terminal 200, improving user convenience.

[0095] The screen prompting the user to specify the measurement range is a screen that displays the evaluation information associated with each part of the user's face image, and the setting unit 1123 may set the range specified by the user for the face image as the measurement range.

[0096] This approach allows users to specify the measurement range based on evaluation information of skin indicators for each body part, thereby improving user convenience and satisfaction.

[0097] If a skin diagnosis is scheduled based on the notified skin diagnosis information, the notification unit 1122 can control the information, including the set measurement range, so that it can be displayed at a predetermined facility associated with the skin diagnosis information.

[0098] In this way, the set measurement range can be displayed during the skin diagnosis that the user has booked, and the user can check the area within the measurement range on a screen installed at the designated facility.

[0099] This embodiment includes an information processing method executed by the management server 100 as the information processing device, and a program for causing a computer to execute the information processing method, and each can achieve the same effects as the information processing device.

[0100] The series of processes described above can be executed by hardware or by software. In other words, the functional configurations in Figures 4 to 6 are merely examples and are not limited to one or the other. That is, it is sufficient to have the functionality to execute the series of processes described above as a whole, and the functional blocks used to realize this functionality are not limited to the examples in Figures 4 to 6.

[0101] For example, the reservation management unit 111 and the skin diagnosis management unit 112 in the processing unit 110 may have their functions performed by other units. Also, at least a portion of the information processing in the management server 100 may be performed at the user terminal 200 and the store terminal 300.

[0102] Furthermore, a single functional block may consist of hardware alone, software alone, or a combination thereof. The functional configuration in this embodiment is realized by a processor that performs arithmetic processing. Processors that can be used in this embodiment include single-processor, multi-processor, and multi-core processors, as well as combinations of these various processing units with processing circuits such as ASICs (Application Specific Integrated Circuits) and FPGAs (Field-Programmable Gate Arrays).

[0103] In this specification, the step of describing a program executed by the processing unit 110 of the management server 100 includes not only processes that are performed chronologically in that order, but also processes that are not necessarily performed chronologically, but are executed in parallel or individually.

[0104] Although several embodiments of the present invention have been described above, these embodiments are merely illustrative and do not limit the technical scope of the present invention. The present invention can take on a variety of other embodiments, and it is also possible to combine the above embodiments with their modified configurations. Furthermore, various modifications such as omissions and substitutions can be made without departing from the spirit of the present invention. These embodiments and their variations are included in the scope and spirit of the invention as described herein, and are also included in the scope of the invention and its equivalents as described in the claims. [Explanation of Symbols]

[0105] 1. Skin Diagnosis Management System 100 Management Servers 111 Reservation Management Department 112 Skin Diagnosis Management Department 1121 Acquisition Department 1122 Notification Department 1123 Settings Section 121 Data Storage Unit 200 user terminals 211 Skin Diagnosis Department 213 Reservation Department 300 store terminals 311 Skin Diagnosis Department 312 Reservation Processing Unit

Claims

1. An acquisition unit acquires a first skin diagnosis result, which is a diagnosis result from a first skin diagnosis that includes one or more skin evaluation pieces of information, based on an image of the user's skin, A storage unit that stores the first skin diagnosis result for each user, associated with the time information when the skin diagnosis was performed, A notification unit notifies a second skin diagnosis information capable of diagnosing the first skin index indicated by the first evaluation information if, among the one or more evaluation pieces of stored first skin diagnosis results, the first evaluation piece satisfies a predetermined condition with respect to at least one of the evaluation piece and its temporal changes, An information processing device equipped with the following features.

2. The aforementioned evaluation information includes skin evaluation values. The information processing device according to claim 1, wherein the notification unit notifies the information of the second skin diagnosis if the difference between the first skin evaluation value and the first skin evaluation value obtained in the past is greater than a preset threshold, and does not notify the information of the second skin diagnosis if the difference is less than or equal to the threshold.

3. The information processing apparatus according to claim 1, wherein the notification unit notifies the user of a proposal to perform the first skin diagnosis again if the predetermined conditions are met, performs a process to guide the user to perform the first skin diagnosis again if the user accepts the proposal, and performs a process to provide information regarding the second skin diagnosis if the user does not accept the proposal.

4. The information processing apparatus according to claim 1, wherein, when the number of displayable skin diagnostic information items is set, the notification unit displays the second skin diagnostic information, which allows the measurement of the first skin index and the second skin index, corresponding to the first and second skin index, to the user terminal with priority over other skin diagnostic information, when the first evaluation information and the second evaluation information, which is other evaluation information, satisfy the predetermined conditions.

5. The information processing apparatus according to claim 1, wherein the notification unit, when the first evaluation information and the second evaluation information, which is other evaluation information, satisfy the predetermined conditions, displays measurable skin diagnostic information for the first skin index and the second skin index with priority over skin diagnostic information for which only one of them can be measured.

6. The information processing apparatus according to claim 1, wherein, when the notification unit satisfies the predetermined conditions, it displays the measurable skin diagnosis information for the first skin index and the second skin index in a way that distinguishes it from the skin diagnosis information for which only one of them can be measured.

7. The information processing device according to any one of claims 4 to 6, wherein, when the notification unit performs the notified second skin diagnosis, it displays the second skin diagnosis result on a screen capable of displaying the first skin diagnosis result diagnosed on the user terminal in a manner that allows it to be identified as the first skin diagnosis result.

8. The information processing device according to claim 1, wherein the first skin diagnosis measures the evaluation information for each part of the user's face, and if the first evaluation information for a predetermined part of the user's face is determined to satisfy the predetermined conditions, the notification unit notifies the predetermined part of skin diagnosis information as second skin diagnosis information, which enables a skin diagnosis different from the first skin diagnosis.

9. The information processing device according to claim 8, further comprising a setting unit that, when a proposal for a skin diagnosis different from the first skin diagnosis is accepted based on the notified skin diagnosis information, sets a skin diagnosis with the predetermined area as the measurement range.

10. The information processing device according to claim 8, wherein, based on the notified skin diagnosis information, if a proposal for a skin diagnosis different from the first skin diagnosis is accepted, the setting unit causes the user terminal to display a screen prompting the user to specify a measurement range.

11. The information processing apparatus according to claim 10, wherein the screen prompting the user to specify the measurement range is a screen that displays the evaluation information associated with each part of the user's facial image, and the setting unit sets the range specified by the user for the facial image as the measurement range.

12. The information processing device according to claim 8, wherein, when a skin diagnosis is booked based on the notified skin diagnosis information, the notification unit controls the information, including the set measurement range, to be displayable at a predetermined facility associated with the skin diagnosis information.

13. Computers A step of obtaining a first skin diagnosis result, which is a diagnosis result from a first skin diagnosis that includes one or more skin evaluation pieces of information, based on an image of the user's skin, A step of storing the first skin diagnosis result for each user, associated with the time information when the skin diagnosis was performed, If, among the one or more evaluation pieces of information included in the stored first skin diagnosis result, the first evaluation piece satisfies a predetermined condition for at least one of the evaluation piece and its temporal changes, the second skin diagnosis information capable of diagnosing the first skin index indicated by the first evaluation piece is notified. An information processing method that performs the following.

14. On the computer, A step of obtaining a first skin diagnosis result, which is a diagnosis result from a first skin diagnosis that includes one or more skin evaluation pieces of information, based on an image of the user's skin, A step of storing the first skin diagnosis result for each user, associated with the time information when the skin diagnosis was performed, If, among the one or more evaluation pieces of information included in the stored first skin diagnosis result, the first evaluation piece satisfies a predetermined condition for at least one of the evaluation piece and its temporal changes, the second skin diagnosis information capable of diagnosing the first skin index indicated by the first evaluation piece is notified. A program that executes the command.

Images