Beauty health information provision system, beauty health information provision method and beauty health information provision program

Abstract

To provide a technology capable of easily providing beauty health information related to scalp and / or hair.SOLUTION: A beauty and health information provision system for providing a user with beauty and health information related to scalp and / or hair, comprises: a storage unit that stores a diagnostic model trained using questionnaire information related to user's responses to a medical questionnaire and captured images of the user's scalp and / or hair as input data and a gradual evaluation on the scalp and / or hair as output data; a reception unit that receives the questionnaire information related to the user's responses to the medical questionnaire and the captured images of the scalp and / or hair; and a diagnosis unit that uses the diagnostic model to determine a diagnosis result, which is a step-by-step evaluation on the scalp and / or hair, based on the questionnaire information and the captured images.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a beauty and health information providing system, a beauty and health information providing method, and a beauty and health information providing program. [Background technology]

[0002] Currently, users can easily receive health advice from experts by providing information about themselves using devices such as smartphones.

[0003] Patent Document 1 discloses a skin condition determination system that outputs the results of a skin diagnosis by providing the details of a medical interview conducted using a mark sheet or the like and image data of sebum. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2016-077649 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the above-mentioned determination system uses images of keratin and sebum obtained by a collection device that uses adhesive substances, such as a keratin checker or sebum checker, but such collection devices that use adhesive substances cannot be used for hair or scalp, and expensive diagnostic equipment is required to perform a diagnosis on hair or scalp, making it difficult to provide diagnostic results on hair or scalp.

[0006] In view of the above-mentioned problems, an object of the present invention is to provide a technology that easily provides beauty and health information related to the scalp and / or hair. [Means for solving the problem]

[0007] In order to solve the above problems, the present invention provides: A beauty and health information providing system that provides a user with beauty and health information related to the scalp and / or hair, A storage unit that stores a diagnostic model that has been trained using questionnaire information regarding the user's answers to the questionnaire and captured images of the scalp and / or hair as input data and a graded evaluation of the scalp and / or hair as output data; a reception unit that receives medical interview information regarding the user's answers to the medical interview and captured images of the scalp and / or hair; A diagnostic unit that uses the diagnostic model to determine a diagnostic result that is a graded evaluation of the scalp and / or hair based on the interview result and the captured image; Equipped with.

[0008] With this configuration, it is possible to easily provide the medical interview result, which is a graded evaluation, based on simple data including answers to the medical interview and image data.

[0009] In a preferred embodiment of the present invention, the beauty and health information providing system further comprises a learning unit, The learning unit learns the diagnostic model using medical interview information regarding the user's answers to medical interviews and captured images of the scalp and / or hair as input data and a graded evaluation of the scalp and / or hair as output data.

[0010] With this configuration, it is possible to learn a diagnostic model that can perform step-by-step evaluation.

[0011] In a preferred embodiment of the present invention, the storage unit stores history information, which is a past diagnosis result of the user, in the storage unit, The diagnosis unit further determines the diagnosis result based on the history information.

[0012] With this configuration, it is possible to provide a diagnosis result that takes into account past diagnosis results.

[0013] In a preferred embodiment of the present invention, the beauty and health information providing system further comprises a display processing unit, The diagnostic result includes a graded evaluation of a plurality of evaluation items for the scalp and / or hair, The display processing unit processes and displays a diagnostic result display screen that displays a graded evaluation relating to multiple evaluation items for the scalp and / or hair based on the diagnostic result.

[0014] With this configuration, it is possible to display graded evaluations for a plurality of evaluation items and easily provide the user with information on the diagnosis results.

[0015] In a preferred embodiment of the present invention, the diagnosis unit stores history information, which is a past diagnosis result of the user, in the storage unit in association with the diagnosis date, The diagnostic result display screen includes a progress chart showing the progress of the evaluation results for the scalp and / or hair based on the multiple diagnostic results and diagnostic dates.

[0016] With this configuration, the progress of the evaluation can be provided in a form that is intuitively easy for the user to understand, based on a plurality of diagnostic results such as past diagnostic results.

[0017] In a preferred embodiment of the present invention, the beauty and health information providing system further comprises an advice providing unit, The advice unit outputs advice regarding scalp and / or hair care based on the medical interview information.

[0018] With this configuration, it is possible to provide the user with advice on care methods and the like based on the diagnosis results.

[0019] In a preferred embodiment of the present invention, the beauty and health information providing system further comprises a care information providing unit, The storage unit further stores a plurality of pieces of care information regarding care provided to the user; The care information providing unit provides care information about care suitable for the user based on the diagnosis result and the plurality of pieces of care information.

[0020] With this configuration, it is possible to provide information on care such as products suitable for the user based on the diagnosis results. [Effects of the Invention]

[0021] The present invention can provide a novel technology for providing beauty and health information related to the scalp and / or hair. [Brief explanation of the drawings]

[0022] [Figure 1] FIG. 1 is a block diagram showing the overall configuration of the system. [Figure 2] Hardware configuration diagram of the system. [Figure 3] 1A and 1B are diagrams showing examples of data configurations and captured images. [Figure 4] FIG. 10 is a diagram showing an example of a screen display. [Figure 5] FIG. 10 is a diagram showing an example of a screen display. [Figure 6] 10 is a flowchart showing a processing flow. DETAILED DESCRIPTION OF THE INVENTION

[0023] The beauty and health information providing system, beauty and health information providing method, and beauty and health information providing program of the present invention will be described below with reference to the accompanying drawings. Note that the following embodiments are merely examples of the present invention, and the present invention is not limited to the following embodiments, and various configurations can be adopted.

[0024] In this embodiment, the configuration, operation, etc. of a beauty and health information providing system will be described, but similarly configured methods, devices, computer programs, and program storage media storing such programs can also achieve similar effects. The series of processes according to this embodiment described below is provided as a computer-executable program, and can be provided via a non-transitory computer-readable recording medium such as a CD-ROM or a flexible disk, or even via a communication line.

[0025] <System configuration> FIG. 1 shows a system configuration diagram of a beauty and health information provision system 0. As shown in FIG. 1, the beauty and health information provision system 0 includes a beauty and health information provision device 1 and a user terminal 2, and each component is communicatively connected via a communication network NW. The communication network NW is any network that uses any communication protocol, such as an IP (Internet Protocol) network. In this embodiment, the beauty and health information provision system 0 may further include a provider terminal 3, which is a terminal device of a provider that provides beauty-related information to users, such as experts.

[0026] The user terminal 2 is a terminal device used by a user who receives beauty and health information related to the scalp and / or hair using the beauty and health information providing system. Although only one user terminal 2 is shown in FIG. 1 , in this embodiment, there are multiple user terminals 2. In this embodiment, the user terminal 2 accepts input of necessary information, such as information about the scalp and / or hair from the user, including medical interview information and captured images, and transmits the information to the beauty and health information providing device 1.

[0027] In this embodiment, the user inputs answers to a medical questionnaire and takes images of the scalp and / or hair via the user's own user terminal 2, and transmits information about the user's scalp and / or hair, such as the content of the medical questionnaire and the captured images, to the beauty and health information providing device 1, thereby receiving beauty and health information, such as diagnostic results related to the scalp and / or hair. Furthermore, in this embodiment, the user can receive, as beauty and health information, advice based on the user's concerns and diagnostic results, as well as product information about products suitable for the user, from the beauty and health information providing device 1. The beauty and health information provided by the beauty and health information providing system 0 is information related to beauty and / or health, and in this embodiment, it is information related to beauty, such as diagnostic results, advice information, and care information.

[0028] <Hardware configuration> FIG. 2(a) shows a hardware configuration diagram of the beauty and health information providing device 1. The beauty and health information providing device 1 includes a processing unit 201, a storage unit 202, and a communication unit 203 as its hardware configuration. In this embodiment, the beauty and health information providing device 1 can be a computer device such as a server or a personal computer. Note that the beauty and health information providing device 1 may be configured with multiple computers, and is not limited to the configuration shown in FIG. 2(a) as long as it can realize the functional components (11-17) described below as a whole.

[0029] The processing unit 101 is composed of one or more processors such as a CPU, and controls the overall processing of the beauty and health information providing device 1 by executing a beauty and health information providing program, an OS (Operating System), and other applications. The storage unit 102 is composed of one or more memories such as an HDD (Hard Disk Drive), SSD (Solid State Drive), flash memory, RAM (Random Access Memory), etc., and stores the beauty and health information providing program and various data. The communication unit 103 controls communication with the communication network NW, and realizes data communication with the user terminal 2, etc. The processing unit 101 executes the beauty and health information providing program, causing the computer to function as the beauty and health information providing device 1 and execute the beauty and health information providing method.

[0030] In this embodiment, the memory unit 102 of the beauty and health information providing device 1 stores various necessary information such as data used as training data including medical interview information, captured images, and evaluation information, models used in the processing of the system including evaluation models, and user information.

[0031] FIG. 2(b) shows a hardware configuration diagram of the user terminal 2. The user terminal 2 includes a processing unit 201, a storage unit 202, a communication unit 203, an input unit 204, and an output unit 205 as its hardware configuration. In this embodiment, the user terminal 2 may be a terminal device such as a smartphone, a personal computer, or a tablet terminal. Furthermore, a terminal device used as a provider terminal 3 that provides beauty and health information to users may have the same hardware configuration as the user terminal 2 shown in FIG. 2(b).

[0032] The processing unit 201 is composed of one or more processors such as a CPU, and controls the overall processing in the user terminal 2 by executing an OS and other applications. The storage unit 202 is an HDD, SSD, flash memory, RAM, etc., and stores a browser application and various data. The communication unit 203 controls communication with the communication network NW and realizes data communication with the beauty and health information providing device 1, etc. The input unit 204 is an input interface that accepts operation requests from the operator, etc., and is composed of a touch panel, mouse, keyboard, etc. The output unit 205 is composed of a display, etc. that displays the processing results by the processing unit 201.

[0033] The beauty and health information providing device 1 stores a diagnostic model in the memory unit 102, and by inputting medical interview information regarding the user's answers to the medical interview and captured images of the scalp and / or hair into this diagnostic model, determines a diagnostic result that is a graded evaluation of the scalp and / or hair.

[0034] <Diagnostic Model> The beauty and health information providing device 1 stores the diagnostic model learned by the learning unit 11 in the storage unit 102, and the diagnostic unit 12 determines the diagnosis result using this diagnostic model. The diagnostic model in this embodiment will be described below.

[0035] The diagnostic model is a trained model that uses as input data information about the user's hair, including questionnaire information about the user's answers to the questionnaire and captured images of the user's scalp and / or hair, and as output data a graded evaluation of the scalp and / or hair determined by an expert on scalp and hair based on the answers to the questionnaire and the captured images. In this embodiment, the diagnostic model learns the evaluation made by the expert based on the captured images about the scalp and / or hair, questionnaire information about the content of the user's questionnaire, and evaluation information about the graded evaluation made by the expert who confirmed the captured images and the content of the user's questionnaire. In this way, by learning based on the information about the user confirmed by the expert, including the captured images and questionnaire information, and the evaluation made by the expert, the diagnostic model in this application can determine a graded diagnostic result similar to that provided by the expert, rather than a strict numerical diagnostic result, based on the questionnaire information and the captured images. In addition, in this embodiment, when the medical interview information and captured images are input, the diagnostic model determines a diagnostic result that is a graded evaluation, such as a five-point evaluation, for multiple evaluation items such as hair thinness, hair moisture content, and hair volume.

[0036] An example of data used for learning a diagnostic model and determining a diagnostic result using the diagnostic model will be shown below with reference to Fig. 3. The data configuration shown below is an example in this embodiment and is not limited to the data configuration shown in the figure. In addition, interview information, captured images, and diagnostic result / evaluation information related to the same diagnosis are stored in the storage unit 102 in association with each other using identification information such as an ID.

[0037] FIG. 3(a) shows an example of medical interview information. The medical interview information is information about the user's answers to the medical interviews entered into the user terminal 2. In this embodiment, the medical interview information is the user's answers to questions about the user's scalp and / or hair, such as concerns about the user's scalp and hair, the user's response to the concerns, and when the user started to be concerned about the concerns. Since input other than options is accepted, the medical interview information is acquired as text data. The medical interview information may be any information that can be used to diagnose the scalp and / or hair. For example, the medical interview information may include information about the user himself / herself, such as the user's age and gender, information about care products used by the user for scalp and / or hair care, and information about the user's own thoughts on the condition of the user's scalp and / or hair. The medical interview information may also include information about the questions asked to the user and information about the user's answers, in association with each other.

[0038] 3(b) shows an example of a captured image of the scalp and / or hair used in processing related to learning a diagnostic model or determining a diagnostic result. In this embodiment, the captured image is an image of the user's hair or scalp captured by the user terminal 2, and is an image obtained by capturing images of the user's head from multiple specified angles. The captured image may be a set of multiple pieces of image data, or may be an image obtained by combining images captured from multiple angles.

[0039] 3(c) shows an example of the data structure of the diagnostic result and the evaluation information. In this embodiment, the diagnostic result and the evaluation information have the same data structure.

[0040] The diagnosis result is a diagnosis result about the user's scalp and / or hair determined using a diagnostic model based on the medical interview information and the captured image, and is a graded evaluation (e.g., a five-point evaluation) of evaluation items such as hair thinness, hair moisture content, hair volume, scalp stiffness, scalp color, etc. In this embodiment, the diagnosis result includes graded evaluations of multiple evaluation items.

[0041] The evaluation information is used as training data for training the diagnostic model and is information about graded evaluations of multiple evaluation items performed by experts such as hair diagnosticians and doctors who have reviewed captured images of the scalp and / or hair and responses to user interviews. The evaluation information is information about graded evaluations (e.g., a five-point scale) of evaluation items such as hair thinness, hair moisture content, hair volume, scalp stiffness, and scalp color, and includes graded evaluations for each evaluation item. In this embodiment, the diagnostic model determines graded evaluations as diagnostic results for evaluation items similar to the items in the evaluation information used as training data.

[0042] Furthermore, the diagnostic model may determine the diagnostic result based on the captured image and the medical interview result as well as the user's past diagnostic results. In this embodiment, the diagnostic model is a common model for any user or concern, but it may also be a model suitable for each user, such as the user's age, gender, or concern, selected based on user information and / or medical interview information.

[0043] The diagnostic model in this embodiment is a multimodal model constructed by combining models capable of processing natural language and image data, such as accepting as input data data generated by preprocessing a captured image (image data) and medical interview information (text data) in the data processing unit 13. Alternatively, the diagnostic model may be constructed by using multiple different models, such as a model that performs image processing and a model that performs natural language processing when the captured image and medical interview information are input, respectively, and each model determines a diagnosis based on features extracted from the captured image and medical interview information. In this embodiment, the diagnostic model is a model constructed using a neural network such as a convolutional neural network. Alternatively, the diagnostic model may be a model that cannot process natural language, such as a model that performs supervised learning based on the answer selected from multiple options acquired as medical interview information and expert evaluations to determine a diagnosis (scaled evaluation value) when the reception unit 16 receives one answer selected from multiple options in a medical interview as medical interview information.

[0044] Furthermore, the diagnostic model in this embodiment is a multi-input, multi-output model that accepts as input multiple data sets, including a vector of feature quantities of an image acquired as a captured image and a vector obtained by vectorizing a sentence acquired as medical interview information, and outputs evaluations for multiple evaluation items as a diagnostic result. In this way, the diagnostic model in this embodiment converts the captured image (image data) and the medical interview information (text data) into vectors so that they can be handled by a single model, and then performs processing related to determining a diagnostic result, thereby improving processing speed compared to generating multiple models corresponding to individual inputs and determining a diagnostic result. Furthermore, the diagnostic model is a supervised model that is trained by labeling the feature quantity vectors of the image acquired as a captured image and the sentence vectors acquired as medical interview information with expert evaluations acquired as evaluation information.

[0045] In this embodiment, the diagnostic model is configured by combining a convolutional neural network for determining a diagnostic result and a large-scale language model for generating advice, and also functions as an advice-providing model. Alternatively, the diagnostic model and the advice-providing model may be configured as separate models.

[0046] <Functional configuration> The beauty and health information providing device 1 is a device for providing beauty and health information such as diagnostic results related to the scalp and / or hair, and includes a learning unit 11, a diagnosis unit 12, a data processing unit 13, an advice providing unit 14, a care information providing unit 15, a reception unit 16, and a display processing unit 17. The functional configuration of the beauty and health information providing device 1 will be described below.

[0047] The learning unit 11 learns a diagnostic model using medical interview information related to the user's answers to the medical interview and captured images of the scalp and / or hair as input data and a graded evaluation of the scalp and / or hair as output data. In FIG. 1, the learning unit 11 is provided in the beauty and health information providing device 1, but it may also be provided in another server device used as a learning device. The learning unit 11 may also perform processing to adjust the trained model, such as fine-tuning, using additionally acquired data. The trained diagnostic model trained by the learning unit 11 is stored in the storage unit 102 of the beauty and health information providing device 1 and is used in processing related to determining a diagnosis result in the diagnosis unit 12, which will be described later. In this embodiment, the learning unit 11 performs training using processed data (processed data) that has been preprocessed by the data processing unit 13 to convert it into a form that is easy to handle. However, if unprocessed data (raw data) does not require processing, the learning may also be performed using raw data.

[0048] The reception unit 16 receives information about the user's scalp and / or hair, including medical interview information and captured images, from the user terminal 2, and performs processing to transfer the information to the diagnosis unit 12, etc. The reception unit 16 may store the received information, including the medical interview information, in the storage unit 102 for use in processing in the beauty and health information providing device 1, such as determining a diagnosis result. The reception unit 16 may also receive input of beauty and health information to be provided to the user, which is input into the provider terminal 3 operated by a provider such as an expert, or information (evaluation information) about a graded evaluation of the scalp and / or hair to be used as training data.

[0049] The data processing unit 13 performs preprocessing to convert various data, such as training data including medical interview information, captured images, and evaluation information, and data used for diagnosis including medical interview information and captured images received from the user terminal 2, into a form that is easy to handle in the beauty and health information provision system 0. Data that has not undergone preprocessing and is left as is is called raw data, and processed data that has undergone preprocessing is called processed data, but the data processing unit 13 does not need to perform preprocessing when the acquired raw data is in a format that is easy to handle in the beauty and health information provision system 0.

[0050] The data processing unit 13 may perform preprocessing on the captured image, such as converting a color image into a black-and-white image, or combining multiple pieces of image data captured from multiple angles into a single piece of image data by combining them side-by-side as in the example shown in Fig. 3. Furthermore, the data processing unit 13 may perform preprocessing on the medical interview information acquired as text data, such as vectorizing the text or tokenizing the text. Furthermore, the data processing unit 13 may perform preprocessing on other image data and text data similar to the processing on the captured image and medical interview information.

[0051] The diagnosis unit 12 determines a diagnosis result, which is a graded evaluation of the scalp and / or hair, using a diagnostic model based on the medical interview information and captured images received by the reception unit 16 from the user terminal 2. In this embodiment, the diagnosis result determined by the diagnosis unit 12 includes a graded evaluation of multiple evaluation items, such as hair thinness and hair moisture content, similar to the items in the evaluation information used for learning as training data.

[0052] In this embodiment, the diagnostic unit 12 stores the determined diagnostic result in the storage unit 102 in association with user information of the user who was diagnosed and information on the date and time of the diagnosis, such as the date of diagnosis, in order to use the result in a process of providing the user with information on the progress of the diagnostic result. The storage unit 102 stores the diagnostic result of such a past diagnosis performed by the user as history information, and may further store the medical interview information and captured images used in determining the diagnostic result in association with the diagnostic result. Furthermore, if the user has previously been diagnosed, the diagnostic unit 12 determines the diagnostic result by referring to the user's previous diagnostic results based on the user's past diagnostic results (history information) stored in association with the user information, in addition to the captured images and medical interview information.

[0053] The advice providing unit 14 outputs advice based on the diagnosis result by the diagnosing unit 12. In this embodiment, the advice providing unit 14 performs a process of generating advice using an advice providing model, which is a language model trained based on text data such as advice provided by experts and chat conversations between the user and the expert, as a process of outputting advice. In this embodiment, the advice providing model is a model that can generate advice regarding the scalp and / or hair for a user, and is trained using medical interview information, captured images, and information (advice history) about advice previously provided by experts after reviewing the medical interview information and captured images. The advice providing model in this embodiment is a large-scale language model capable of generating sentences, and, like the diagnostic model, outputs advice when vectors of images acquired as captured images and vectorized vectors of sentences acquired as medical interview results are input. However, it may also accept past diagnostic results (diagnosis history) as input. In addition, the advice providing unit 14 may generate a prompt to be input to the advice providing model based on the medical interview information, captured images, and diagnostic results, and output advice generated using the generated prompt and the advice providing model.

[0054] The advice providing model may also be a model that determines which advice to provide based on the medical interview information, the captured image, and the diagnosis results. In this case, the advice providing unit 14 determines a classification (advice classification) for the advice to be provided based on the medical interview information, the captured image, and the diagnosis results, and outputs advice corresponding to the determined classification by referring to the advice information stored in the storage unit 102. The advice information stored in the storage unit 102 is information about advice set for each classification (advice classification) determined by the advice providing model, and includes information about the classification by the advice providing model and the content of the advice corresponding to the classification (for example, when a classification related to premature graying is determined, advice corresponding to the classification related to premature graying such as "It is recommended to take care of premature graying like this"). For example, when the user's concern is thin hair and the evaluation of the thinness item in the diagnosis results is 1 on a 5-point scale (1 being the thinnest and 5 being the thickest), the advice providing unit 14 refers to the advice information and outputs advice such as "If your hair is thin, it is recommended to take care of it like this" as advice corresponding to the classification determined by the advice providing model.

[0055] Furthermore, the care information providing unit 15 determines care suitable for the user, such as products suitable for the user that correspond to the user's hair type, based on the care information for multiple types of care stored in the storage unit 102 and the diagnosis results, and provides care information about the determined care. The care information stored in the storage unit 102 is information about care provided to the user, including product information about products used for scalp and / or hair care, such as hair care products and scalp care products, ingredient information about beauty ingredients that the user should take, training information about training that the user should do, recipe information about meals that the user should eat, and consultation information about experts (e.g., nutritionists, beauticians, doctors, etc.) who should be consulted about hair. In this embodiment, the care information providing unit 15 determines care suitable for the user based on the diagnosis results, and provides care information about the determined care, for example, if the diagnosis result indicates that the user has thin hair, it provides information about care for people with thin hair. At this time, the care information providing unit 15 may classify the user based on the diagnosis results and provide care information for each classification based on that classification, or may provide information about care related to the item in the user's diagnosis results that was rated the furthest from the average (for example, if the other items were about average but the evaluation result for the hair thinning item was very far from the average, care information about care to address hair thinning may be provided), or may provide care information about an item specified by the user (for example, if an item related to hair thickness is selected, information about care according to the user's hair thickness obtained from the diagnosis results may be provided).

[0056] An example of providing care information, in which product information on products suitable for a user is provided, is described below. In this embodiment, the product information includes a product name, a product image, a product ID, and a provided product classification. In this embodiment, the care information providing unit 15 determines a provided product classification, which is a classification of products for which information is to be provided to the user, based on the diagnosis result, and provides product information on products belonging to the determined classification based on the provided product classification and the product information. The care information providing unit 15 may also determine products for which information is to be provided, such as products suitable for the user's concerns, based on other information used for diagnosis, such as medical interview information and captured images. In this case, the care information providing unit 15 classifies the products for which information is to be provided based on one or more of the diagnosis result, the medical interview information, and the captured images, and provides product information suitable for the user based on the determined provided product classification by referring to product information on products belonging to that classification. In addition, when multiple products are registered for the determined provided product classification, the care information providing unit 15 may determine the product for which to provide product information from among the products registered in that provided product classification based on information about the user such as age and gender, or may provide information about multiple products corresponding to the determined provided product classification.

[0057] Furthermore, the care information providing unit 15 may provide product information about a specific product when the evaluation for one or more evaluation items falls within a predetermined range based on the diagnostic results, for example, by providing information about a product for people with thin hair when the evaluation for the hair thinness is 1 to 2 (1 being the thinnest and 5 being the thickest) based on the evaluation range for the specified evaluation items included in the product information and the evaluation for the specified evaluation items included in the diagnostic results.

[0058] The display processing unit 17 displays screens including a diagnosis result display screen based on various necessary information including the diagnosis result. The screens displayed by the display processing unit 17 will be described below with reference to the drawings. In this embodiment, the display processing unit 17 displays beauty and health information including the diagnosis result, advice provided by the advice providing unit 14, and care information on the user terminal 2, so that the beauty and health information providing system 0 provides beauty and health information to the user.

[0059] FIG. 4 shows an example of an information input screen displayed on the user terminal 2 when the user inputs information necessary for diagnosis, including medical interview information and captured images. The information input screen W1 shown in the figure is a screen for accepting input of various information necessary for diagnosis of the scalp and / or hair, including medical interview information and captured images, which is displayed when the reception unit 16 accepts the user's selection to start diagnosis. In this embodiment, the display processing unit 17 displays questions on the information input screen W1 based on question information stored in the storage unit 102, and the reception unit 16 accepts answers to the displayed questions as medical interview information. At this time, the display processing unit 17 may determine questions to be displayed based on user characteristics, such as age and gender, based on the user information, and the user's answers to other questions, such as the user's concerns, and perform display processing of the questions. At this time, the reception unit 16 may accept medical interview information that varies depending on the user's gender, age, etc. (e.g., whether the user is pregnant or has given birth if she is a woman). The reception unit 16 accepts the medical interview information and captured images entered on the information input screen W1.

[0060] FIG. 5 shows an example of a diagnostic result display screen that displays diagnostic results. The diagnostic result display screen W2 shown in the figure is a screen that, when a diagnostic result is determined by the diagnostic unit 12, is displayed based on the diagnostic result and displayed on the user terminal 2, and includes an evaluation display section W21 and a progress chart W22. The diagnostic result display screen W2 shown in FIG. 5 also includes multiple evaluation display sections W21 for each evaluation item. The diagnostic result display screen W2 may also include an advice display section that displays advice provided by the advice providing unit 14 and a product information display section that displays product information provided by the care information providing unit 15.

[0061] The evaluation display unit W21 is a screen display that displays a graded evaluation (a five-point evaluation in FIG. 5 ) for any evaluation item included in the diagnosis result. In the example shown in FIG. 5 , the evaluation display unit W21 displays the evaluation in a graphical format, in which the lightly colored bar graph is colored in a darker color or other color depending on the evaluation, for easier intuitive understanding. However, the graded evaluation may also be displayed as a numerical value. The evaluation display unit W21 further includes a standard display, which is a screen display that shows a standard evaluation for a predetermined evaluation item. For example, if the evaluation item is hair thinness, the standard evaluation for the predetermined evaluation item is the fourth level of thinness out of five levels. In this case, the display processing unit 17 displays the standard for each evaluation item based on standard information for the standard evaluation for each evaluation item stored in the storage unit 102. The evaluation standard may be the average evaluation for each evaluation item by the user, or a standard evaluation value for each evaluation item set for each age or gender.

[0062] The progress display diagram W22 is a screen display that displays the transition of evaluations included in the user's diagnostic results. The screen display is processed for display based on the diagnostic results of the currently performed diagnosis and the user's past diagnostic results (history information) stored in association with the user information, and shows the transition of evaluations in the results of the current diagnosis and the results of previous diagnoses up to the previous diagnosis. In this embodiment, a graph is generated for each evaluation item (in the example shown in FIG. 5, a graph showing the transition of hair thinness), but it may also be a graph that combines the transition of evaluations for multiple evaluation items. In this embodiment, the progress display diagram W22 is shown as a line graph and a bar graph, but it may be any type of graph that can display the transition of evaluations. The progress display diagram W22 shown in FIG. 5 is a graph showing the transition of evaluations in the performed diagnoses, with the vertical axis representing evaluation and the horizontal axis representing the number of diagnoses, but it may also be a graph showing the transition of evaluations over time, with the horizontal axis representing time.

[0063] Furthermore, on the diagnosis result display screen W2, a date may be input by pressing a date display such as a calendar, and the display processing unit 17 may display a screen showing the diagnosis results for the specified date based on the history information. At this time, the display processing unit 17 may display a history display screen showing evaluations of past diagnoses, having a screen configuration similar to the diagnosis result display screen shown in Fig. 5, based on the history information.

[0064] <Processing flow> The flow of processing performed by the beauty and health information providing device 1 in this embodiment will be described below.

[0065] <Learning process> The flow of processing related to learning the diagnostic model will be explained below. The data processing unit 13 acquires the medical interview information, captured images, and diagnostic information, which are raw data before processing stored in the storage unit 102, and performs preprocessing to convert them into a form that is easy to handle by the diagnostic model. When the learning unit 11 acquires the medical interview information, captured images, and diagnostic information, which are processed data that have been preprocessed by the data processing unit 13, the learning unit 11 uses these data as training data to learn the diagnostic model.

[0066] <Diagnosis processing> The process of diagnosing the scalp and / or hair will be described with reference to FIG. 6. When the user selects to start a diagnosis and the receiving unit 16 receives a display request for an information input screen from the user terminal 2, the display processing unit 17 performs display processing for the information input screen (S101). The user terminal 2 receives input of medical interview information and captured images from the user and transmits them to the beauty and health information providing device 1 (S201). The receiving unit 16 receives the medical interview information and captured images from the user terminal 2 (S102). At this time, the beauty and health information providing device 1 determines whether the received captured images are suitable for processing, such as diagnosis, in the diagnosing unit 12, for example, by determining whether the images are captured at an appropriate angle. If the determination results in an inappropriate image, the display processing unit 17 may display a warning message or an instruction to send the captured images again, and the receiving unit 16 may receive the input again. The data processing unit 13 performs preprocessing to convert the raw data, such as the medical interview information and captured images, received by the receiving unit 16 into a format that is easy to handle in the system (S103). When the diagnostic unit 12 receives the medical interview information and captured images processed by the data processing unit 13, it uses a diagnostic model to determine a diagnostic result based on the medical interview information and captured images (S104). The diagnostic unit 12 determines whether a diagnostic result linked to the user information of the user to be diagnosed is stored in the storage unit 102. If the user has previously been diagnosed and there is a past diagnostic result (history information) linked to the user information, the diagnostic result is determined based on the history information, which is a history of the user's past diagnostic results, in addition to the medical interview information and captured images. The display processing unit 17 displays a diagnostic result display screen that displays a graded evaluation for multiple evaluation items based on the diagnostic result by the diagnostic unit 12 (S105).

[0067] Furthermore, after the processing by the diagnosis unit 12 as shown in FIG. 6, either or both of the processing of providing product information by the care information providing unit 15 and the processing of providing advice by the advice providing unit 14 may be performed.

[0068] In this embodiment, the diagnosis unit 12 determines a graded evaluation of the scalp and / or hair as a diagnostic result based on the medical interview information and the captured image, and provides the result as the diagnostic result. However, for example, the diagnosis result based on the medical interview result regarding happiness level, mental state, etc. may also be provided as a diagnostic result. Furthermore, the diagnosis unit 12 may further receive data such as analysis results of samples of the user's scalp, hair, etc., and use the data to determine the diagnostic result. The diagnostic result may include evaluations regarding items such as hormone balance and stress level.

[0069] <Embodiment 2> In the second embodiment, an example will be described in which the beauty and health information providing system 0 functions as a system for providing a marketplace where products for maintaining beauty and health can be purchased. In the following example, the care information provided to the user is product information about products used for beauty and health care.

[0070] In this embodiment, the reception unit 16 receives registration of product information and stores it in the storage unit 102. The product information is information about a product provided to a user, including product name, product price, product category, product image, and product description, and is uniquely identified and managed by identification information such as a product ID. In this embodiment, the product information is registered by an administrator of the beauty and health information provision system 0, but may also be registered by the manufacturer of the product. The display processing unit 17 explains the product based on the product information and displays a product purchase screen to accept the purchase of the product. At this time, the beauty and health information provision device 1 accepts the purchase of the product from the user and performs purchase processing such as payment processing. In addition, information about the user's product purchase history in the marketplace and products registered as favorites by the user (favorite registration information) is associated with user information using identification information such as a user ID as the user's behavior history in the marketplace and stored in the storage unit 102. This user's behavior history may be used, for example, for collaborative filtering, or for other processes related to the provision of information about suitable products by the care information provision unit 15, which will be described later.

[0071] The receiving unit 16 also receives registration of provided attribute information, which is information about the attributes of users who will receive product information about the product, and stores the information in the storage unit 102. The provided attribute information is information about the attributes of users who will receive product information about the product, such as the type of users to whom the product information should be provided, and is registered by the administrator of the beauty and health information providing system 0, the manufacturer of the product, etc. In this embodiment, the provided attribute information includes information about answers to questions in the medical interview (e.g., the user is concerned about gray hair), evaluation values ​​included in the diagnosis results (e.g., a hair moisture rating of 4 or higher), the user's environment (e.g., the amount of UV rays in the user's living area, weather, season, etc.), ingredients necessary for the user, etc. In this embodiment, the provided attribute information is information about items corresponding to any information about the user, including user information, diagnosis results, and medical interview information, but may also include information about the user's behavioral history, such as not purchasing a product from the same category (e.g., shampoo) for more than one month. The product information and the provided attribute information are stored in association with each other using unique identification information such as a product ID. In addition, the memory unit 102 may store, as provided attribute information, coordinates, vectors, feature quantities, etc. in a feature space generated based on information regarding the attributes of the user to whom the product should be provided, such as answers to questions in the medical interview received by the reception unit 16, evaluation values ​​included in the diagnosis results, and the user's environment.

[0072] The care information providing unit 15 determines products for which product information is to be provided based on information about the user, such as the diagnosis result, medical interview information, and user information, and on condition information. The care information providing unit 15 references product information about the products for which information is to be provided, stored in the storage unit 102, and provides product information suitable for the user. The condition information is information regarding conditions for which product information is to be provided, and in this embodiment, is generated based on the product information and provided attribute information regarding the attributes of the user receiving the product information. For example, for a user who is evaluated as having low moisture content as a diagnosis result, the care information providing unit 15 determines, as products for which information is to be provided, products (e.g., shampoo and treatment) set to be provided to users with low moisture content in the provided attribute information, based on the condition information generated based on the product information and the provided attribute information. At this time, the care information providing unit 15 may determine products that satisfy all conditions based on multiple conditions included in the condition information, or may determine products that satisfy a certain level of some of the conditions as products for which information is to be provided. Furthermore, conditions relating to different items may be set for each product, such as a condition relating to age being set for one product and a condition relating to gender being set for another product.

[0073] In this embodiment, the care information providing unit 15 determines, as products for which information is to be provided, products that have a certain level of relevance to the user based on information about the user, including diagnostic results, medical interview information, and user information, and condition information. At this time, the care information providing unit 15 may determine, as products for which information is to be provided, a product that has the closest relevance to the user, or may determine, as products for which information is to be provided, multiple products that have a certain level of relevance or higher. Furthermore, the care information providing unit 15 may determine ingredients necessary for the user based on the user's concerns and diagnostic evaluations based on the medical interview information and diagnostic results, and determine, as products for which information is to be provided, products that include the determined ingredients.

[0074] The care information providing unit 15 may also generate a graph relating to the relationships between products based on the product information and the provided attribute information and use the graph as condition information to provide the product information. The graph referred to here is not a diagram such as a pie chart or a bar graph, but a data structure consisting of nodes and the relationships between the nodes, capable of expressing the relationships between nodes (such as people and things). In this embodiment, the care information providing unit 15 expresses the relationships between products in a feature space based on the product information and the provided attribute information as a graph (e.g., coordinates in a feature space such as (X, Y, Z)), and determines the product having the closest relationship with the user as the product for which information is to be provided. In this case, the care information providing unit 15 may determine the relevance by referencing the distance between products as a vector. The storage unit 102 may also store graph information (e.g., coordinates in a feature space such as (X, Y, Z)) generated based on the product information and the provided attribute information in association with the product information as condition information, and the care information providing unit 15 may use the condition information to determine the product for which information is to be provided. At this time, the care information providing unit 15 searches a feature space indicating the relationship between products based on information about the user, such as user information, diagnosis results, and medical interview information, and determines a product whose coordinates are closest to or within a certain range from the coordinates indicating the user's characteristics as the product for which product information is to be provided. The care information providing unit 15 may also determine a product for which information is to be provided using a model that handles graph data, such as a graph neural network.

[0075] The display processing unit 17 may display information about products determined to be provided by the care information providing unit 15 on a diagnosis result display screen W2 as shown in Fig. 5, or may display information about products to be provided as recommended products on a market screen for purchasing products. Furthermore, the reception unit 16 may receive information about the user's living area, such as an address or region (e.g., prefecture name or Kanto region), to provide product information, and store the information as user information in the storage unit 102. At this time, the care information providing unit 15 accesses an external weather forecast system or the like, and acquires information about the environment of the user's living area, such as the weather and temperature in the user's living area, based on information about the user's living area, such as the user's address, and uses the information in the process of determining products for which information is to be provided.

[0076] In addition, the care information providing unit 15 may determine the relevance between users based on the user's behavioral history in the marketplace, such as the user's purchase history, and based on the relevance between users, determine products that users who are closely related have purchased or liked as products for which information will be provided, and provide the product information.

[0077] In this embodiment, the beauty and health information providing system 0 provides information on suitable products to the user every month. If there is a lack of data to be used to provide product information, such as if the user has not performed a diagnosis for this month, the display processing unit 17 may display a screen that displays a message urging the user to perform a diagnosis, or may display a screen for receiving input of the missing information.

[0078] The reception unit 16 further receives input of responses (survey results) to a questionnaire such as impressions on using the product from users who have purchased the product or received a sample product. The survey results received by the reception unit 16 are linked to the product using identification information such as a product ID and stored in the storage unit 102. The survey may be displayed during the medical interview, and the reception unit 16 may receive responses to the survey at the same time as the medical interview information. At this time, the display processing unit 17 may display a screen for displaying the medical interview questions and the survey questions and for receiving responses to the questions, such as the information input screen W1 shown in FIG. 4, and may receive input of responses to the questions (medical interview information, survey results).

[0079] In this embodiment, the attribute information to be provided regarding the conditions for providing product information is optimized based on the acquired survey results. The care information providing unit 15 may include a counting unit that tallys up the survey results acquired for each product. In this case, the counting unit tallys up the contents of the survey and the attributes of the users who answered the survey (diagnosis results, user information, and user attributes based on medical interview results). The display processing unit 17 may support the user in optimally setting the attribute information to be provided by displaying the results of the survey tallied by the counting unit.

[0080] The beauty and health information providing device 1 may also include an optimization processing unit for optimizing the provided attribute information. In this case, the optimization processing unit may perform the process of optimizing the provided attribute information by taking an average of the users who purchased the product or who responded that they liked the product in a questionnaire based on information about the users who actually purchased the product or who liked the product (user information, diagnosis results, medical interview information, etc.), modeling the users who will receive the information about the product (for example, modeling the users as coordinates on a graph expressed by (X, Y, Z), etc.), and using the model as the provided attribute information of the product, as the process of optimizing the provided attribute information. [Explanation of symbols]

[0081] 0 Beauty and Health Information System 1 Beauty and health information provision device 2. User terminal NW communication network 11 Learning Department 12 Diagnostic Department 13 Data Processing Unit 14 Advice Department 15. Care Information Department 16 Reception Department 17 Display processing section

Claims

1. A beauty and health information providing system that provides a user with beauty and health information related to the scalp and / or hair, A storage unit that stores a diagnostic model that has been trained using questionnaire information regarding the user's answers to the questionnaire and captured images of the scalp and / or hair as input data and a graded evaluation of the scalp and / or hair as output data; a reception unit that receives medical interview information regarding the user's answers to the medical interview and captured images of the scalp and / or hair; A diagnostic unit that uses the diagnostic model to determine a diagnostic result that is a graded evaluation of the scalp and / or hair based on the interview information and the captured image; Equipped with Beauty and health information system.

2. The beauty and health information providing system further includes a learning unit, The learning unit learns the diagnostic model using questionnaire information regarding the user's answers to the questionnaire and captured images of the scalp and / or hair as input data and a graded evaluation of the scalp and / or hair as output data. The beauty and health information providing system according to claim 1.

3. The storage unit stores history information that is a past diagnosis result of the user, The diagnosis unit further determines the diagnosis result based on the history information. The beauty and health information providing system according to claim 1.

4. The beauty and health information providing system further includes a display processing unit, The diagnostic result includes a graded evaluation of a plurality of evaluation items for the scalp and / or hair, The display processing unit displays a diagnostic result display screen that displays a graded evaluation of a plurality of scalp and / or hair evaluation items based on the diagnostic result, The beauty and health information providing system according to claim 1.

5. the diagnosis unit stores history information, which is the user's past diagnosis results, in the storage unit in association with the diagnosis date; The diagnostic result display screen includes a progress chart showing the progress of the evaluation of the scalp and / or hair based on the plurality of diagnostic results and diagnostic dates, The beauty and health information providing system according to claim 4.

6. The beauty and health information providing system further includes an advice providing unit, The advice providing unit outputs advice regarding scalp and / or hair beauty based on the medical interview information. The beauty and health information providing system according to claim 1.

7. The beauty and health information providing system further includes a care information providing unit, The storage unit further stores a plurality of pieces of care information regarding care provided to the user; the care information providing unit provides care information about care suitable for the user based on the diagnosis result and the plurality of pieces of care information. The beauty and health information providing system according to claim 1.

8. A beauty and health information providing method executed by a computer that stores in a storage unit a diagnostic model learned using questionnaire information related to a user's answers to a questionnaire and captured images of the scalp and / or hair as input data and a graded evaluation of the scalp and / or hair as output data, A beauty and health information providing method for providing a user with information on skin and hair beauty, comprising: a receiving step of receiving medical interview information regarding the user's answers to the medical interview and captured images of the scalp and / or hair; A diagnostic process of determining a diagnostic result, which is a graded evaluation of the scalp and / or hair, based on the interview information and the captured image using the diagnostic model; A method for providing beauty and health information, comprising:

9. A beauty and health information providing program that provides a user with information about skin and hair beauty, a reception unit that receives medical interview information regarding the user's answers to the medical interview and captured images of the scalp and / or hair; A diagnostic unit that determines a diagnostic result, which is a graded evaluation of the scalp and / or hair, based on the interview information and the captured image using a diagnostic model; The method functions as a computer having a storage unit that stores the diagnostic model learned using questionnaire information regarding the user's answers to the questionnaire and captured images of the scalp and / or hair as input data and a graded evaluation of the scalp and / or hair as output data. Beauty and health information program.

Images