Beauty and health information providing system, beauty and health information providing method, and beauty and health information providing program

Abstract

An object is to provide a technique for easily providing beauty and health information related to the scalp and / or hair. 【Solution means】 A beauty and health information providing system that provides beauty and health information related to the scalp and / or hair to a user, a storage unit that stores a diagnostic model that has learned using, as input data, inquiry information regarding a user's response to an inquiry and an imaging image of the scalp and / or hair, and, as output data, a step-by-step evaluation of the scalp and / or hair; a reception unit that receives inquiry information regarding a user's response to an inquiry and an imaging image of the scalp and / or hair; a diagnosis unit that determines a diagnosis result, which is a step-by-step evaluation of the scalp and / or hair, based on the inquiry information and the imaging image by using the diagnostic model; is provided.

Description

Technical Field

[0001] The present invention relates to a beauty and health information providing system, a beauty and health information providing method, and a beauty and health information providing program.

Background Art

[0002] Currently, by using a terminal such as a smartphone to provide information about themselves, users can easily obtain advice on their health from experts and the like.

[0003] Patent Document 1 discloses a skin condition determination system that outputs the result of skin diagnosis by providing the content of a medical interview conducted using a mark sheet or the like and image data of sebum.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, in the above-described determination system, images of cutin and sebum obtained by a sampling device using an adhesive substance such as a cutin checker or a sebum checker were used. However, for hair and scalp, such a sampling device using an adhesive substance cannot be used, and an expensive diagnostic instrument is required to diagnose hair and scalp. As a result, it was not possible to easily provide a diagnostic result regarding hair and scalp.

[0006] In view of the above problems, an object of the present invention is to provide a technique for easily providing beauty and health information regarding the scalp and / or hair.

Means for Solving the Problems

[0007] In order to solve the above problems, the present invention provides a beauty and health information providing system that provides beauty and health information regarding the scalp and / or hair to a user, comprising: a storage unit that stores a diagnostic model that has learned using, as input data, questionnaire information regarding answers to a user's questionnaire and captured images of the scalp and / or hair, and using, as output data, a stepwise evaluation of the scalp and / or hair; a reception unit that receives questionnaire information regarding answers to a user's questionnaire and captured images of the scalp and / or hair; a diagnostic unit that determines a diagnostic result, which is a stepwise evaluation of the scalp and / or hair, based on the questionnaire result and the captured image using the diagnostic model; and

[0008] With such a configuration, it is possible to easily provide a questionnaire result that is a stepwise evaluation based on simple data including an answer to a questionnaire and image data.

[0009] In a preferred form of the present invention, the beauty and health information providing system further comprises a learning unit, wherein the learning unit learns the diagnostic model using, as input data, questionnaire information regarding answers to a user's questionnaire and captured images of the scalp and / or hair, and using, as output data, a stepwise evaluation of the scalp and / or hair.

[0010] With such a configuration, it is possible to learn a diagnostic model capable of performing a stepwise evaluation.

[0011] In a preferred form of the present invention, the storage unit stores history information that is a past diagnostic result of a user in the storage unit, and the diagnostic unit further determines the diagnostic result based on the history information.

[0012] With such a configuration, it is possible to provide a diagnostic result that takes into account past diagnostic results.

[0013] In a preferred embodiment of the present invention, the beauty and health information providing system further includes a display processing unit, The diagnosis result includes a stepwise evaluation regarding a plurality of evaluation items for the scalp and / or hair, Based on the diagnosis result, the display processing unit displays a diagnosis result display screen that displays a stepwise evaluation related to a plurality of evaluation items for the scalp and / or hair.

[0014] With such a configuration, it is possible to display a stepwise evaluation regarding a plurality of evaluation items and easily provide information regarding the diagnosis result to the user.

[0015] In a preferred embodiment of the present invention, the diagnosis unit stores, in a storage unit, history information that is the past diagnosis result of the user, in association with the diagnosis date, The diagnosis result display screen includes a progress display diagram showing the progress of the evaluation result for the scalp and / or hair based on a plurality of the diagnosis results and the diagnosis dates.

[0016] With such a configuration, it is possible to provide the progress of the evaluation in a form that is intuitive and easy for the user to understand based on a plurality of diagnosis results such as past diagnosis results.

[0017] In a preferred embodiment of the present invention, the beauty and health information providing system further includes an advice providing unit, The advice unit outputs advice regarding the beauty of the scalp and / or hair based on the interview information.

[0018] With such a configuration, it is possible to provide advice such as a care method based on the diagnosis result to the user.

[0019] In a preferred embodiment of the present invention, the beauty and health information providing system further includes a care information providing unit, The storage unit further stores a plurality of care information regarding the care provided to the user, The care information providing unit provides care information about care suitable for the user based on the diagnosis result and the plurality of care information.

[0020] With such a configuration, it is possible to provide information regarding care such as products suitable for the user based on the diagnosis result.

Advantages of the Invention

[0021] The present invention can provide a novel technology for providing beauty and health information regarding the scalp and / or hair.

Brief Description of the Drawings

[0022]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Modes for Carrying Out the Invention

[0023] Hereinafter, the beauty and health information providing system, beauty and health information providing method, and beauty and health information providing program of the present invention will be described with reference to the drawings. Note that the following embodiments are examples of the present invention, and the present invention is not limited to the following embodiments, and various configurations can be adopted.

[0024] In this embodiment, the configuration, operation, etc. of the beauty and health information providing system will be described. However, a method, device, computer program having the same configuration, and a program storage medium storing the program can also achieve the same effects. A series of processes according to this embodiment described below are provided as a computer-executable program, and can be provided via a non-transitory computer-readable recording medium such as a CD-ROM or a flexible disk, or further via a communication line.

[0025] <System Configuration> FIG. 1 shows a system configuration diagram of the beauty and health information providing system 0. As shown in FIG. 1, the beauty and health information providing system 0 includes a beauty and health information providing device 1 and a user terminal 2, and each component is communicably connected via a communication network NW. The communication network NW is an arbitrary network using an arbitrary communication protocol such as an IP (Internet Protocol) network. Further, in this embodiment, the beauty and health information providing system 0 may further include a provider terminal 3 which is a terminal device of a provider that provides information related to beauty to users such as experts.

[0026] The user terminal 2 is a terminal device used by a user who receives the provision of beauty and health information regarding the scalp and / or hair using the beauty and health information providing system. In FIG. 1, only one user terminal 2 is shown, but in this embodiment, there are a plurality of user terminals 2. In this embodiment, the user terminal 2 receives the input of necessary information such as information about the scalp and / or hair from the user including interrogation information and captured images, and transmits it to the beauty and health information providing device 1.

[0027] In this embodiment, the user inputs answers to a medical interview and captures images of the scalp and / or hair via the user terminal 2 owned by the user, and transmits information about the user's scalp and / or hair, such as the content of the medical interview and the captured images, to the beauty and health information providing apparatus 1, thereby receiving the provision of beauty and health information such as a diagnosis result regarding the scalp and / or hair. Further, in this embodiment, the user can further receive, from the beauty and health information providing apparatus 1, advice according to the user's concerns and diagnosis results as beauty and health information, and product information about products suitable for the user. The beauty and health information provided by the beauty and health information providing system 0 is information related to beauty and / or health, and in this embodiment, it is beauty-related information such as a diagnosis result, advice information, and care information.

[0028] <Hardware Configuration> FIG. 2(a) shows a hardware configuration diagram of the beauty and health information providing apparatus 1. The beauty and health information providing apparatus 1 includes, as a hardware configuration, a processing unit 201, a storage unit 202, and a communication unit 203. In this embodiment, the beauty and health information providing apparatus 1 can use a computer device such as a server or a personal computer. Note that the beauty and health information providing apparatus 1 may be configured by a plurality of computer devices, and is not limited to the configuration shown in FIG. 2(a) as long as it can realize the functional components (11 - 17) described later as a whole.

[0029] The processing unit 101 is composed of one or more processors such as a CPU, and controls the overall processing in the beauty and health information providing apparatus 1 by executing a beauty and health information providing program, an OS (Operating System), and other applications. The storage unit 102 is composed of one or more memories such as an HDD (Hard Disk Drive), an SSD (Solid State Drive), a flash memory, and a RAM (Random Access Memory), and stores a beauty and health information providing program and various data. The communication unit 103 controls communication with the communication network NW and realizes data communication with the user terminal 2 and the like. The processing unit 101 functions as the beauty and health information providing apparatus 1 by executing the beauty and health information providing program, and executes the beauty and health information providing method.

[0030] In the present embodiment, the storage unit 102 of the beauty and health information providing apparatus 1 stores various necessary information such as data used as teacher data including inquiry information, captured images, and evaluation information, models used in the processing of the system including an evaluation model, and user information.

[0031] FIG. 2(b) shows a hardware configuration diagram of the user terminal 2. The user terminal 2 includes a processing unit 201, a storage unit 202, a communication unit 203, an input unit 204, and an output unit 205 as a hardware configuration. In the present embodiment, the user terminal 2 can use a terminal device such as a smartphone, a personal computer, or a tablet terminal. Further, the terminal device used as the provider terminal 3 that provides beauty and health information to the user may have the same hardware configuration as the user terminal 2 shown in FIG. 2(b).

[0032] The processing unit 201 is composed of one or more processors such as a CPU, and controls the overall processing in the user terminal 2 by executing an OS and other applications. The storage unit 202 is an HDD, SSD, flash memory, RAM, etc., and stores a browser application and various data. The communication unit 203 controls communication with the communication network NW and realizes data communication with the beauty and health information providing device 1 and the like. The input unit 204 is an input interface that receives operation requests and the like from an operator, and is composed of a touch panel, a mouse, a keyboard, and the like. The output unit 205 is composed of a display or the like that displays the processing result by the processing unit 201.

[0033] The beauty and health information providing device 1 stores a diagnostic model in the storage unit 102, and by inputting inquiry information regarding the user's answers to the inquiry and imaging images regarding the scalp and / or hair into this diagnostic model, determines a diagnostic result which is a step-by-step evaluation regarding the scalp and / or hair.

[0034] <Diagnostic model> The beauty and health information providing device 1 stores the diagnostic model learned in the learning unit 11 in the storage unit 102, and the diagnostic unit 12 determines the diagnostic result using this diagnostic model. Hereinafter, the diagnostic model in this embodiment will be described.

[0035] The diagnostic model is a trained model that uses, as input data, information about the user's hair, including the questionnaire information regarding the answers to the questionnaire for the user and the captured images of the user's scalp and / or hair, and performs learning with, as output data, a step-by-step evaluation regarding the scalp and / or hair determined by an expert based on the answers to the questionnaire and the captured images. In the present embodiment, the diagnostic model performs learning of the evaluation performed by the expert based on the captured image regarding the scalp and / or hair, the questionnaire information regarding the content of the questionnaire by the user, and the evaluation information regarding the step-by-step evaluation performed by the expert who has confirmed the captured image and the content of the questionnaire by the user. Thus, by performing learning based on the information about the user confirmed by the expert, including the captured image and the questionnaire information, and the evaluation performed by the expert, the diagnostic model in the present application can determine a step-by-step diagnostic result similar to that provided by the expert, rather than a diagnostic result with precise numerical values, based on the questionnaire information and the captured image. Further, in the present embodiment, when the questionnaire information and the captured image are input, the diagnostic model determines a diagnostic result that is a step-by-step evaluation such as a five-step evaluation regarding a plurality of evaluation items such as hair thickness, hair moisture content, and hair quantity.

[0036] Hereinafter, an example of data used for learning the diagnostic model and determining the diagnostic result using the diagnostic model will be shown with reference to FIG. 3. The data configuration shown below is an example in the present embodiment and is not limited to the data configuration shown in the figure. Also, the questionnaire information, the captured image, and the diagnostic result / evaluation information regarding the same diagnosis are stored in the storage unit 102 in association with each other using identification information such as an ID.

[0037] In FIG. 3(a), an example of inquiry information is shown. The inquiry information is information regarding the user's answers to inquiries input on the user terminal 2. In the present embodiment, the inquiry information is the user's answers to questions about the user's scalp and / or hair, such as the user's concerns about the scalp and hair, the status of countermeasures against the user's concerns, and the time when the concerns about the user's scalp and / or hair started. Since it accepts inputs other than options, it is acquired as text data. Also, the inquiry information may be any information as long as it is information that can be used for the diagnosis of the scalp and / or hair. For example, information about the user himself / herself such as the user's age and gender, information about care products used by the user for scalp and / or hair care, information about the state of the user's scalp and / or hair considered by the user himself / herself, and the like may be used. Also, the inquiry information may include information about the questions asked to the user and information about the answers given by the user in association with each other.

[0038] In FIG. 3(b), an example of an imaging image of the scalp and / or hair used for processing related to the learning of the diagnosis model or the determination of the diagnosis result is shown. In the present embodiment, the imaging image is an image of the user's hair and scalp imaged on the user terminal 2, and is an image obtained by photographing the user's head from a plurality of specified angles. The imaging image may be a plurality of image data, or may be an image obtained by combining images taken from a plurality of angles.

[0039] In FIG. 3(c), an example of the data configuration of the diagnosis result and the evaluation information is shown. In the present embodiment, the diagnosis result and the evaluation information show the same data configuration.

[0040] The diagnosis result is the result of the diagnosis of the user's scalp and / or hair determined using the diagnosis model based on the inquiry information and the imaging image, and is a step-by-step evaluation (such as a 5-step evaluation) for evaluation items such as hair thickness, hair moisture content, hair quantity, scalp firmness, and scalp color. In the present embodiment, the diagnosis result includes a step-by-step evaluation for a plurality of evaluation items.

[0041] The evaluation information is information about the step-by-step evaluation regarding multiple evaluation items performed by experts such as hair diagnosticians and doctors who have confirmed imaging images related to the scalp and / or hair and responses to the user's medical history inquiries, and is used as teacher data for training the diagnostic model. The evaluation information is information about the step-by-step evaluation (such as a 5-step evaluation) regarding evaluation items such as hair thickness, hair moisture content, hair volume, scalp firmness, and scalp color, and includes the step-by-step evaluation in each evaluation item. In the present embodiment, the diagnostic model determines a step-by-step evaluation as a diagnostic result regarding evaluation items similar to those in the evaluation information used as teacher data.

[0042] In addition, the diagnostic model may determine the diagnostic result based on the user's past diagnostic results in addition to the imaging image and the medical history result. Also, in the present embodiment, the diagnostic model is a common model for any user and concern, but may be a model suitable for each user, such as the user's age, gender, and the user's concern, selected based on the user information and / or the medical history information.

[0043] The diagnostic model in this embodiment is a multimodal model configured by combining models capable of processing natural language and image data that receive, as input data, data generated by preprocessing an imaging image, which is image data, and interview information, which is text data, in a data processing unit 13. Further, the diagnostic model may be a model configured by using a plurality of different models such that when an imaging image and interview information are input into different models that perform image processing and natural language processing, respectively, each model determines a diagnostic result based on feature amounts extracted from the imaging image and the interview information. In this embodiment, the diagnostic model is a model configured by a neural network such as a convolutional neural network. Further, when the reception unit 16 receives, as interview information, one answer selected from a plurality of options in an interview, the diagnostic model may be a model that cannot perform natural language processing and performs supervised learning based on the answer selected from the options acquired as interview information and the evaluation by an expert, and determines a diagnostic result (a value of a stepwise evaluation).

[0044] Further, the diagnostic model in this embodiment is a multi-input multi-output model that receives, as input, a plurality of data including a vector of feature amounts of an image acquired as an imaging image and a vector obtained by vectorizing a sentence acquired as interview information, and outputs evaluations for a plurality of evaluation items as a diagnostic result. As described above, the diagnostic model in this embodiment converts an imaging image, which is image data, and interview information, which is text data, into vectors so that they can be handled by one model, and then performs processing related to determination of a diagnostic result, thereby improving the processing speed as compared with the case of generating a plurality of models corresponding to individual inputs and determining a diagnostic result. Further, the diagnostic model is a supervised model that performs learning by attaching, as a label, the evaluation of an expert acquired as evaluation information to the vector of feature amounts of the image acquired as an imaging image and the vector of the sentence acquired as interview information.

[0045] Also, in this embodiment, the diagnostic model is configured by combining a convolutional neural network for determining a diagnostic result and a large language model for generating advice, and also functions as an advice-providing model. Further, the diagnostic model and the advice-providing model may be configured as separate models respectively.

[0046] <Functional Configuration> The beauty and health information providing device 1 is a device for providing beauty and health information such as a diagnostic result regarding the scalp and / or hair, and includes a learning unit 11, a diagnostic unit 12, a data processing unit 13, an advice providing unit 14, a care information providing unit 15, a reception unit 16, and a display processing unit 17. Hereinafter, the functional configuration of the beauty and health information providing device 1 will be described.

[0047] The learning unit 11 performs learning of the diagnostic model using the interrogation information regarding the user's answer to the interrogation and the captured image regarding the scalp and / or hair as input data, and the stage-by-stage evaluation regarding the scalp and / or hair as output data. In FIG. 1, the learning unit 11 is provided in the beauty and health information providing device 1, but may be provided in another server device used as a learning device. Further, the learning unit 11 may perform a process of adjusting the learned model, such as performing fine-tuning using the additionally acquired data. Note that the learned diagnostic model learned by the learning unit 11 is stored in the storage unit 102 of the beauty and health information providing device 1 and is used for the process regarding the determination of the diagnostic result in the diagnostic unit 12 described later. Also, in this embodiment, the learning unit 11 performs learning using the processed data (processed data) for which preprocessing for conversion into an easy-to-handle form has been performed by the data processing unit 13, but when processing of the un-preprocessed data (raw data) is not required, learning may be performed using the raw data.

[0048] The reception unit 16 receives information regarding the user's scalp and / or hair, including the inquiry information and the captured images, from the user terminal 2, and performs a process of passing it on to the diagnosis unit 12 and the like. The reception unit 16 may store the received information including the inquiry information in the storage unit 102 for use in the processes in the beauty and health information providing apparatus 1 such as the determination of the diagnosis result. Further, the reception unit 16 may receive the input of beauty and health information provided to the user and information (evaluation information) regarding the step-by-step evaluation of the scalp and / or hair used as teacher data, which is input in the provider terminal 3 operated by a provider such as an expert.

[0049] The data processing unit 13 performs, as preprocessing, a process of converting various data such as teacher data including the inquiry information, the captured images, and the evaluation information, and data for diagnosis including the inquiry information and the captured images received from the user terminal 2 into a form that is easy to handle in the beauty and health information providing system 0. When the raw data obtained is in a format that is easy to handle in the beauty and health information providing system 0, the data processing unit 13 may not perform the preprocessing. The raw data without preprocessing is called raw data, and the processed data after preprocessing is called processed data.

[0050] Regarding the captured images, the data processing unit 13 may perform, as preprocessing, processes such as converting a color image into a black-and-white image, and combining a plurality of image data captured from a plurality of angles into one image data by joining them left and right as shown in the example of FIG. 3. Further, regarding the inquiry information obtained as text data, the data processing unit 13 may perform, as preprocessing, processes such as vectorizing the text and tokenizing the text. Further, regarding other image data and text data, the data processing unit 13 may perform, as preprocessing, processes similar to the processes for the captured images and the inquiry information.

[0051] The diagnosis unit 12 determines a diagnosis result, which is a step-by-step evaluation regarding the scalp and / or hair, using a diagnosis model based on the interview information and captured images received by the reception unit 16 from the user terminal 2. In the present embodiment, the diagnosis result determined by the diagnosis unit 12 includes a step-by-step evaluation related to a plurality of evaluation items such as hair thickness and hair moisture content, which are the same as the items in the evaluation information used for learning as teacher data.

[0052] Also, in the present embodiment, for the purpose of using the information regarding the progress of the diagnosis result for providing it to the user, the diagnosis unit 12 stores the determined diagnosis result in the storage unit 102 in association with the user information of the user who is the subject of the diagnosis and information regarding the date and time of the diagnosis such as the diagnosis date. The storage unit 102 stores the diagnosis results in such past diagnoses by the user as history information. Further, it may store the interview information and captured images used for the determination of the diagnosis result in association with the diagnosis result. Furthermore, when the user has had a past diagnosis, in addition to the captured images and interview information, the diagnosis unit 12 determines a diagnosis result with reference to the previous diagnosis results of the user based on the past diagnosis results (history information) of the user stored in association with the user information.

[0053] The advice providing unit 14 outputs advice based on the diagnosis result by the diagnosis unit 12. In the present embodiment, as a process of outputting advice, the advice providing unit 14 performs a process of generating advice using an advice providing model which is a language model trained based on text data such as advice provided by experts and exchanges in chats between the user and experts. In the present embodiment, the advice providing model is a model that can generate advice regarding the scalp and / or hair of the user, which is trained using the inquiry information and the imaging image, and information about advice (advice history) provided by experts in the past after checking the inquiry information and the imaging image. The advice providing model in the present embodiment is a large language model capable of generating sentences. When a vector of an image acquired as an imaging image and a vector obtained by vectorizing a sentence obtained as an inquiry result are input, similar to the diagnosis model, it is a model that outputs advice. Further, it may also accept a past diagnosis result (diagnosis history) as an input. Further, the advice providing unit 14 may generate a prompt for input to the advice providing model based on the inquiry information, the imaging image, and the diagnosis result, and output the generated prompt and the advice generated using the advice providing model.

[0054] Further, the advice providing model may be a model that determines which advice to provide based on the interview information, imaging images, and diagnosis results. At this time, the advice providing unit 14 determines a classification (advice classification) regarding the advice to be provided based on the interview information, imaging images, and diagnosis results, and outputs advice corresponding to the determined classification with reference to the advice information stored in the storage unit 102. The advice information stored in the storage unit 102 is information regarding advice set for each classification (advice classification) determined by the advice providing model, and includes information regarding the classification by the advice providing model and the content of the advice corresponding to the classification (for example, when a classification regarding premature gray hair is determined, advice such as "It is advisable to take care like 〇〇 for premature gray hair." corresponding to the classification regarding premature gray hair). The advice providing unit 14, for example, when the user's concern is the hair fineness and the evaluation of the hair fineness item in the diagnosis result is 1 out of 5 levels of evaluation (1 being the thinnest and 5 being the thickest evaluation), outputs advice such as "In the case of thin hair, it is advisable to take care like 〇〇." as advice corresponding to the classification determined by the advice providing model with reference to the advice information.

[0055] In addition, based on the care information about a plurality of cares stored in the storage unit 102 and the diagnosis result, the care information providing unit 15 determines a product suitable for the user, such as a product suitable for the user's hair quality, etc., or care suitable for the user, and provides care information about the determined care. The care information stored in the storage unit 102 includes product information about products used for scalp and / or hair care, such as hair care products and scalp care products, ingredient information about beauty ingredients that the user should take, etc., training information about training that the user should perform, recipe information about the diet that the user should take, and consultation destination information about experts (such as dietitians, beauticians, doctors, etc.) who should consult about hair. It is information about care provided to the user. In the present embodiment, for example, when a result such as thin hair is obtained as the diagnosis result, the care information providing unit 15 provides information about care for people with thin hair, etc., determines care suitable for the user based on the diagnosis result, and provides care information about the determined care. At this time, the care information providing unit 15 may classify the user based on the diagnosis result and provide care information for each classification based on the classification, or provide information about care related to the item that obtained the evaluation furthest from the average among the user's diagnosis results (for example, when other items are about average, but only the evaluation result for the item of hair thickness is very far from the average, provide care information about care for dealing with thin hair, etc.), or also provide care information about the item specified by the user (when the item related to hair thickness is selected, provide information about care according to the hair thickness of the user obtained from the diagnosis result, etc.).

[0056] The following shows an example of providing product information about products suitable for a user when providing care information. In this embodiment, the product information includes a product name, a product image, a product ID, and a provided product classification. In this embodiment, the care information providing unit 15 determines a provided product classification, which is a classification of products to be provided to the user based on the diagnosis result, and provides product information about products belonging to the determined classification based on the provided product classification and the product information. Further, the care information providing unit 15 may determine products to provide information about, such as products suitable for the user's concerns, based on other information used for diagnosis, such as inquiry information and imaging images. At this time, the care information providing unit 15 classifies products for which information is to be provided based on any one or a plurality of the diagnosis result, inquiry information, and imaging images, and provides product information suitable for the user by referring to the product information about products belonging to the determined provided product classification. Also, when a plurality of products are registered for the determined provided product classification, the care information providing unit 15 may determine a product to provide product information based on information about the user, such as age and gender, from among the products registered in the provided product classification, or may provide information about a plurality of products corresponding to the determined provided product classification.

[0057] Furthermore, the care information providing unit 15 may, based on the range of evaluation related to a predetermined evaluation item included in the product information and the evaluation of the predetermined evaluation item included in the diagnosis result, for example, when the evaluation of hair thickness is from 1 to 2 (where 1 is the thinnest and 5 is the thickest evaluation), provide product information for people with thin hair, and provide product information about specific products when the evaluation in any one or a plurality of evaluation items is within a predetermined range based on the diagnosis result.

[0058] The display processing unit 17 performs display processing on a screen including a diagnosis result display screen based on various necessary information including the diagnosis result. Hereinafter, the screen displayed by the display processing unit 17 will be described with reference to the drawings. In the present embodiment, the beauty and health information providing system 0 provides beauty and health information to the user by displaying, on the user terminal 2, the diagnosis result, the advice provided by the advice providing unit 14, and the beauty and health information including the care information.

[0059] In FIG. 4, an example of the screen display of the information input screen displayed on the user terminal 2 when the user inputs the information necessary for performing a diagnosis including the inquiry information and the captured image is shown. As shown in the figure, the information input screen W1 is a screen for receiving the input of various information necessary for performing a diagnosis on the scalp and / or hair including the inquiry information and the captured image, which is displayed when the reception unit 16 receives the user's selection to start the diagnosis. In the present embodiment, the display processing unit 17 displays questions on the information input screen W1 based on the question information stored in the storage unit 102, and the reception unit 16 receives the answers to the displayed questions as the inquiry information. At this time, the display processing unit 17 may determine the questions to be displayed based on the characteristics of the user such as age and gender based on the user information, and the user's answers to other questions such as the content of the user's concerns, and perform the display processing of the questions. At this time, the reception unit 16 may receive inquiry information that varies depending on the user's gender, age, etc. (for example, whether the user is a woman and whether she is pregnant or after childbirth). The reception unit 16 receives the inquiry information and the captured image input on the information input screen W1.

[0060] In FIG. 5, an example of the screen display of a diagnostic result display screen for displaying a diagnostic result is shown. The diagnostic result display screen W2 shown in the figure is a screen that is display-processed based on the diagnostic result and displayed on the user terminal 2 when the diagnostic result is determined by the diagnostic unit 12, and includes an evaluation display unit W21 and a progress display diagram W22. Further, the diagnostic result display screen W2 shown in FIG. 5 includes a plurality of evaluation display units W21 for each evaluation item. Further, the diagnostic result display screen W2 may include an advice display unit for displaying the advice provided by the advice providing unit 14 and a product information display unit for displaying the product information provided by the care information providing unit 15.

[0061] The evaluation display unit W21 is a screen display that displays a step-by-step evaluation (a five-step evaluation in FIG. 5) for any one of the evaluation items included in the diagnostic result. In the example shown in FIG. 5, for the sake of intuitive understanding, the evaluation display unit W21 performs display in the form of a graph such that the thinly colored bar graph is colored with another color such as a dark color according to the evaluation, but the step-by-step evaluation may be displayed as a numerical value. Further, the evaluation display unit W21 further includes a reference display that is a screen display indicating the reference evaluation for a predetermined evaluation item, and can indicate the reference evaluation for a predetermined evaluation item (when the evaluation item is the hair thickness, the thickness of the fourth step out of the five steps is the reference evaluation, etc.). At this time, the display processing unit 17 performs display of the reference for each evaluation item based on the reference information about the reference evaluation for each evaluation item stored in the storage unit 102. Further, the reference of the evaluation may be the average of the evaluations for each evaluation item of the user or the value of the reference evaluation for each evaluation item set for each age and gender.

[0062] The progress display diagram W22 is a screen display that shows the transition of evaluations included in the user's diagnosis result. It is a screen display that shows the transition of evaluations in the result of the current diagnosis and the past diagnosis results (history information) of the user stored in association with the diagnosis result of the current execution and the user information, and is subjected to display processing based on the past diagnosis results (history information) of the user. In this embodiment, it is a graph generated for each evaluation item (in the example shown in FIG. 5, a graph of the progress regarding hair thickness), but it may also be a graph that summarizes the transitions of evaluations regarding a plurality of evaluation items into one. Further, in this embodiment, the progress display diagram W22 is shown by a line graph and a bar graph, but any graph format may be used as long as it can display the transition of evaluations. The progress display diagram W22 shown in FIG. 5 is a graph in which the vertical axis represents the evaluation and the horizontal axis represents the number of diagnoses, and shows the transition of evaluations in the executed diagnosis, but the horizontal axis may be time, and it may be a diagram that shows the transition of evaluations over time.

[0063] Also, on the diagnosis result display screen W2, by pressing a date display such as a calendar, date input is performed, and the display processing unit 17 may perform display processing of a screen showing the result of the diagnosis on the specified date based on the history information. At this time, the display processing unit 17 may perform display processing of a history display screen showing the evaluations in past diagnoses, having the same screen configuration as the diagnosis result display screen as shown in FIG. 5, based on the history information.

[0064] <Flow of processing> Hereinafter, the flow of processing of the beauty and health information providing apparatus 1 in this embodiment will be described.

[0065] <Learning process> Hereinafter, the flow of processing regarding the learning of the diagnosis model will be described. The data processing unit 13 acquires the interrogation information, the captured image, and the diagnosis information, which are the raw data before processing stored in the storage unit 102, and performs preprocessing for converting them into a form that is easy to handle by the diagnosis model. When the learning unit 11 acquires the interrogation information, the captured image, and the diagnosis information, which are the processed data preprocessed by the data processing unit 13, it performs learning of the diagnosis model using those data as teacher data.

[0066] <Diagnostic process> With reference to FIG. 6, the process for diagnosing the scalp and / or hair will be described. When the user selects to start the diagnosis and the reception unit 16 receives a request to display an information input screen from the user terminal 2, the display processing unit 17 performs display processing of the information input screen (S101). The user terminal 2 receives input of inquiry information and captured images by the user and transmits them to the beauty and health information providing apparatus 1 (S201). The reception unit 16 receives the inquiry information and the captured images from the user terminal 2 (S102). At this time, the beauty and health information providing apparatus 1 determines whether the received captured image is an image at an appropriate angle, etc., and is an image suitable for processing such as diagnosis in the diagnosis unit 12. As a result of the determination, if it is not an appropriate image, the display processing unit 17 may perform display processing of an instruction to display a warning message or to transmit a captured image again, and the reception unit 16 may receive the input again. The data processing unit 13 performs pre-processing for converting the raw data such as the inquiry information and the captured images received by the reception unit 16 into a form that is easy to handle in the system (S103). When the diagnosis unit 12 receives the inquiry information and the captured images that have been processed by the data processing unit 13, it uses a diagnosis model to determine a diagnosis result based on the inquiry information and the captured images (S104). The diagnosis unit 12 determines whether a diagnosis result associated with the user information of the user who performs the diagnosis is stored in the storage unit 102. If the user has been diagnosed in the past and there is a past diagnosis result (history information) associated with the user information, in addition to the inquiry information and the captured images, the diagnosis result is further determined based on the history information, which is the history of the past diagnosis results of the user. The display processing unit 17 performs display processing of a diagnosis result display screen that displays a step-by-step evaluation related to a plurality of evaluation items based on the diagnosis result by the diagnosis unit 12 (S105).

[0067] Also, after the processing by the diagnosis unit 12 as shown in FIG. 6, either or both of the processing of providing product information by the care information providing unit 15 and the processing of providing advice by the advice providing unit 14 may be performed.

[0068] In this embodiment, the diagnosis unit 12 determines, based on the interview information and the imaging images, a staged evaluation regarding the scalp and / or hair as a diagnosis result, and provides it as the diagnosis result. For example, the diagnosis result regarding happiness, mental state, etc. based on the interview result may be further provided as the diagnosis result. Further, the diagnosis unit 12 may further receive data such as the analysis result of a sample of the user's scalp, hair, etc., and use it to determine the diagnosis result. The diagnosis result may include evaluations regarding items such as hormonal balance and stress levels.

[0069] <Embodiment 2> In Embodiment 2, an example of when the beauty and health information providing system 0 functions as a system for providing a marketplace where products for maintaining beauty and health can be purchased will be described. Also, in the following example, the care information provided to the user is product information about products used for beauty and health care.

[0070] In this embodiment, the reception unit 16 receives the registration of product information and stores it in the storage unit 102. The product information is information about the product provided to the user, including the product name, product price, product category, product image, and product description, and is uniquely identified and managed by identification information such as a product ID. In this embodiment, the product information is registered by the administrator of the beauty and health information providing system 0, but it may also be registered by the manufacturer that produced the product, etc. The display processing unit 17 performs display processing of a product purchase screen for explaining the product and accepting the purchase of the product based on the product information. At this time, the beauty and health information providing device 1 receives the purchase of the product from the user and performs processing for purchase such as settlement processing. Also, the purchase history of the user's products in the marketplace and information about the products registered by the user as favorites (favorite registration information) are stored in the storage unit 102 in association with the user information using identification information such as the user ID as the user's behavior history in the marketplace. This user's behavior history may be used, for example, in processing related to the provision of information about suitable products by the care information providing unit 15 described later, such as being used in collaborative filtering.

[0071] In addition, the reception unit 16 further receives and stores in the storage unit 102 the registration of the provided attribute information, which is information about the attributes of the user who is provided with the product information about the product. The provided attribute information is information regarding the attributes of the user who receives the product information about the product, such as for which users it should be provided, and is registered by the administrator of the beauty and health information providing system 0, the manufacturer of the product, and the like. In the present embodiment, the provided attribute information includes information such as answers to questions in the questionnaire (for example, the trouble being gray hair), evaluation values included in the diagnosis result (for example, the evaluation of the hair moisture content being 4 or more), the user's environment (for example, the amount of ultraviolet rays in the user's living area, weather, season, etc.), and components necessary for the user. In the present embodiment, the provided attribute information is information regarding an item corresponding to any information about the user including user information, diagnosis results, and questionnaire information, but may also include information regarding the user's behavior history, such as not having purchased products of the same category (for example, shampoo, etc.) for one month or more. The product information and the provided attribute information are stored in association with each other by unique identification information such as a product ID. Further, the storage unit 102 may store, as the provided attribute information, coordinates, vectors, feature amounts, etc. in the feature space generated based on information regarding the attributes of the user to whom the product should be provided, such as answers to questions in the questionnaire received by the reception unit 16, evaluation values included in the diagnosis result, and the user's environment.

[0072] The care information providing unit 15 determines a product that provides product information based on information about the user, such as a diagnosis result, inquiry information, and user information, and condition information. The care information providing unit 15 refers to the product information related to the product determined to provide information, which is stored in the storage unit 102, and provides the product information suitable for the user. The condition information is information regarding the conditions for providing information on any product. In this embodiment, it is generated based on the product information and the providing attribute information regarding the attributes of the user who receives the product information. Based on the condition information generated based on the product information and the providing attribute information, for example, for a user evaluated as having a low moisture content as a diagnosis result, the care information providing unit 15 determines, as the product for which information should be provided, a product (for example, shampoo and treatment) that is set to be provided to a user with a low moisture content in the providing attribute information. At this time, the care information providing unit 15 may determine a product that satisfies all the conditions based on a plurality of conditions included in the condition information, or may determine a product that satisfies a certain number or more of some of the conditions as the product for which information is provided. Also, for each product, conditions regarding different items may be set, such as a condition regarding age being set for a certain product and a condition regarding gender being set for a certain product.

[0073] In this embodiment, the care information providing unit 15 determines, as the product for which information is provided, a product having a certain degree or more of relevance to the user based on information about the user including a diagnosis result, inquiry information, and user information, and condition information. At this time, the care information providing unit 15 may determine the product having the closest relevance as the product for which information is provided, or may determine a plurality of products having a certain degree or more of relevance as the products for which information is provided. Also, the care information providing unit 15 may determine the components necessary for the user from the user's concerns and the evaluation in the diagnosis based on the inquiry information and the diagnosis result, and determine, as the product for which information is provided, a product including the product having the determined components.

[0074] In addition, the care information providing unit 15 may generate a graph regarding the relationship of products based on the product information and the provided attribute information, and use it as condition information for providing the product information. The graph mentioned here is not a diagram such as a pie chart or a bar graph, but a data structure consisting of the relationship between nodes, and it is a data structure capable of expressing the relationship between nodes (such as people and things). In the present embodiment, the care information providing unit 15 expresses the relationship of products on the feature space as a graph (for example, coordinates on a feature space such as (X, Y, Z)) based on the product information and the provided attribute information, and determines the product with the closest relationship to the user as the product for providing information. At this time, the care information providing unit 15 may obtain the relevance by referring to the distance between products as a vector. Further, the storage unit 102 stores the graph information (for example, coordinates on a feature space such as (X, Y, Z)) generated based on the product information and the provided attribute information in association with the product information as condition information, and the care information providing unit 15 may determine the product for providing information using the condition information. At this time, the care information providing unit 15 searches for a feature space indicating the relationship between products based on information about the user such as user information, diagnosis results, and inquiry information, and determines the product with coordinates closest to or within a certain range from the coordinates indicating the features of the user as the product for providing product information. In addition, the care information providing unit 15 may determine the product for providing information using a model that handles graph data such as a graph neural network.

[0075] The display processing unit 17 may display the information of the product determined to be provided by the care information providing unit 15 on the diagnosis result display screen W2 as shown in FIG. 5, or may display the information of the product provided as a recommended product on the market screen for purchasing the product. In addition, the reception unit 16 may receive information regarding the living area of the user, such as the address and region (for example, prefecture name and Kanto), for providing the product information, and store it in the storage unit 102 as user information. At this time, the care information providing unit 15 accesses an external weather forecast system or the like, obtains information regarding the environment of the user's living area, such as the weather and temperature of the user's living area, based on information regarding the user's living area, such as the user's address, and uses it for the process of determining the product for providing information.

[0076] In addition, the care information providing unit 15 determines the relevance between users based on the user's behavior history in the marketplace such as the purchase history of the user, and based on the relevance between users, determines the products that users with close relevance have purchased or liked as the products for which information is to be provided, and may provide product information.

[0077] In this embodiment, the beauty and health information providing system 0 provides information on products suitable for the user every month. When there is a shortage of data used for providing product information, such as when the user has not performed a diagnosis this month, the display processing unit 17 may display a screen that displays a message prompting the user to perform a diagnosis, or may display a screen for accepting input of the missing information.

[0078] The reception unit 16 further receives input of answers (questionnaire results) to questionnaires such as impressions of using the product from users who have purchased the product or users who have received samples. The questionnaire results received by the reception unit 16 are stored in the storage unit 102 in association with the product by identification information such as the product ID. A questionnaire may be displayed during the interview, and the reception unit 16 may receive an answer to the questionnaire simultaneously with the interview information. At this time, the display processing unit 17 may display a screen such as the information input screen W1 shown in FIG. 4 for displaying the questions of the interview and the questionnaire and receiving answers to the questions, and may receive input of answers (interview information, questionnaire results) to the questions.

[0079] In this embodiment, the provision attribute information regarding the conditions for providing product information is optimized based on the obtained questionnaire results. The care information providing unit 15 may include a totaling unit that totals the questionnaire results obtained for each product. At this time, the totaling unit totals the content of the questionnaire and the attributes of the users who answered the questionnaire (diagnosis results, user information, user attributes based on the interview results). The display processing unit 17 may support the optimal setting of the provision attribute information by the user by displaying the totaling results of the questionnaire results by the totaling unit.

[0080] Further, the beauty and health information providing apparatus 1 may include an optimization processing unit for optimizing the provided attribute information. At this time, regarding the optimization of the provided attribute information, the optimization processing unit is based on information (user information, diagnosis results, questionnaire information, etc.) about users who actually purchased the product or users who liked the product, and calculates the average of users who purchased the product or users who answered that they liked the product in a questionnaire, etc., to model the users who receive the information of the product (for example, model the user as coordinates in a graph represented by (X, Y, Z), etc.), and may perform the process of using the model as the provided attribute information of the product as a process related to the optimization of the provided attribute information.

Explanation of Signs

[0081] 0 Beauty and health information providing system 1 Beauty and health information providing apparatus 2 User terminal NW Communication network 11 Learning unit 12 Diagnosis unit 13 Data processing unit 14 Advice providing unit 15 Care information providing unit 16 Reception unit 17 Display processing unit

Claims

1. A beauty and health information providing system that provides beauty and health information regarding the scalp and / or hair to a user, a diagnosis model that learns using, as input data, inquiry information regarding the user's response to an inquiry and an imaging image of the scalp and / or hair, and using, as output data, a step-by-step evaluation of the scalp and / or hair, and a storage unit that stores history information that is the user's past diagnosis results, a reception unit that receives inquiry information regarding the user's response to an inquiry and an imaging image of the scalp and / or hair, a diagnosis unit that determines a diagnosis result, which is a step-by-step evaluation of the scalp and / or hair, based on the inquiry information, the imaging image, and the history information by using the diagnosis model, comprising: a beauty and health information providing system.

2. The beauty and health information providing system further comprises a learning unit, wherein the learning unit learns the diagnosis model by using, as input data, inquiry information regarding the user's response to an inquiry and an imaging image of the scalp and / or hair, and using, as output data, a step-by-step evaluation of the scalp and / or hair. The beauty and health information providing system according to claim 1.

3. The beauty and health information providing system further comprises a display processing unit, wherein the diagnosis result includes a step-by-step evaluation regarding a plurality of evaluation items for the scalp and / or hair, and the display processing unit performs display processing of a diagnosis result display screen that displays a step-by-step evaluation regarding a plurality of evaluation items for the scalp and / or hair based on the diagnosis result. The beauty and health information providing system according to claim 1.

4. The diagnosis unit stores the history information, which is the user's past diagnosis result, in the storage unit in association with the diagnosis date, and the diagnosis result display screen includes a progress display diagram showing the progress of the evaluation of the scalp and / or hair based on the plurality of diagnosis results and the diagnosis date. The beauty and health information providing system according to claim 3.

5. The beauty and health information providing system further comprises an advice providing unit, wherein the advice providing unit outputs advice regarding the beauty of the scalp and / or hair based on the inquiry information. The beauty and health information providing system according to claim 1.

6. The beauty and health information providing system further comprises a care information providing unit, wherein the storage unit further stores a plurality of care information regarding the care provided to the user. The care information providing unit provides care information about care suitable for the user based on the diagnosis result and the plurality of care information. The beauty and health information providing system according to claim 1.

7. A beauty and health information providing method executed by a computer that stores in a storage unit inquiry information regarding a user's answer to an inquiry, an imaging image of the scalp and / or hair as input data, a diagnostic model that has learned a stepwise evaluation of the scalp and / or hair as output data, and history information that is the user's past diagnosis result, A beauty and health information providing method for providing information regarding beauty of the skin and hair to a user. A reception step of receiving inquiry information regarding a user's answer to an inquiry and an imaging image of the scalp and / or hair. A diagnosis step of determining a diagnosis result, which is a stepwise evaluation of the scalp and / or hair, based on the inquiry information, the imaging image, and the history information using the diagnostic model. A beauty and health information providing method comprising the above.

8. A beauty and health information providing program for providing information regarding beauty of the skin and hair to a user. A reception unit that receives inquiry information regarding a user's answer to an inquiry and an imaging image of the scalp and / or hair. A diagnosis unit that determines a diagnosis result, which is a stepwise evaluation of the scalp and / or hair, based on the inquiry information, the imaging image, and history information that is the user's past diagnosis result using a diagnostic model. As a computer having a storage unit that stores the inquiry information regarding the user's answer to the inquiry, the imaging image of the scalp and / or hair as input data, the diagnostic model that has learned a stepwise evaluation of the scalp and / or hair as output data, and the history information, is caused to function. A beauty and health information providing program.

Images