Information providing device, information providing method, and information providing program

Abstract

To provide an information providing device capable of providing a user with useful information, an information providing method, and an information providing program.SOLUTION: In an information processing system in which an information providing device 10, a user terminal, and a makeup information database are communicably connected, the information providing device comprises: an acquisition unit 13a which acquires mode information on a hair mode capable of being provided by a user; a calculation unit 13b which calculates a congeniality degree indicating a congeniality degree of combinations of hair modes and makeup on the basis of the acquired mode information; a generation unit 13c which generates provision information provided to the user who has selected a hair mode on the basis of the calculated congeniality degree; and a learning unit 13e which learns a machine learning model so as to output congeniality degrees of combinations of hair modes and makeup made by object persons having hair of the mode. The information providing device calculates a congeniality degree using the machine learning model.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to an information providing device, an information providing method, and an information providing program. [Background technology]

[0002] 2. Description of the Related Art Conventionally, there are known techniques for assisting in coordinating outfits, such as a technique for generating outfits based on rules regarding fashion combinations. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2013-235528 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the above-mentioned conventional techniques have room for improvement in terms of providing useful information to customers (referred to as "users"). For example, the conventional techniques have difficulty in suggesting makeup products to users that match their available hairstyles.

[0005] The present application has been made in view of the above, and aims to provide an information providing device, an information providing method, and an information providing program that can provide useful information to users. [Means for solving the problem]

[0006] In order to solve the above-mentioned problems and achieve the object, the information providing device of the present invention is characterized by comprising an acquisition unit that acquires aspect information regarding hair aspects that can be provided by a user, a calculation unit that calculates a compatibility degree that indicates the degree of compatibility between the aspect and makeup based on the aspect information, and a generation unit that generates information to be provided to the user based on the compatibility degree.

[0007] In addition, the information providing method of the present invention is an information providing method executed by an information providing device, and is characterized by including an acquisition step of acquiring hair style information related to hair styles that can be provided by a user, a calculation step of calculating a compatibility degree indicating the degree of compatibility between the style and makeup based on the hair style information, and a generation step of generating information to be provided to the user based on the compatibility degree.

[0008] In addition, the information provision program of the present invention is characterized in that it causes a computer to execute an acquisition procedure for acquiring hair style information relating to hair styles that can be provided by a user, a calculation procedure for calculating a compatibility degree indicating the degree of compatibility between the style and makeup based on the hair style information, and a generation procedure for generating provision information to be provided to the user based on the compatibility degree. [Effects of the Invention]

[0009] The present invention can provide useful information to users. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of an information providing system according to an embodiment. [Figure 2] FIG. 2 is a block diagram illustrating an example of the configuration of the information providing device according to the embodiment. [Figure 3] FIG. 3 is a diagram illustrating an example of a mode information storage unit according to the embodiment. [Figure 4] FIG. 4 is a diagram illustrating an example of a makeup information storage unit according to the embodiment. [Figure 5] FIG. 5 is a diagram illustrating an example of a compatibility information storage unit according to the embodiment. [Figure 6] FIG. 6 is a diagram illustrating an example of a provided information storage unit according to the embodiment. [Figure 7] FIG. 7 is a diagram illustrating a specific example of the information providing process according to the embodiment. [Figure 8]FIG. 8 is a flowchart showing an example of the flow of the information providing process according to the embodiment. [Figure 9] FIG. 9 is a diagram illustrating an example of a hardware configuration. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, an information providing device, an information providing method, and an information providing program according to the present application (hereinafter, referred to as an embodiment) will be described in detail with reference to the drawings. Note that the information providing device, the information providing method, and the information providing program according to the present application are not limited to these embodiments. Furthermore, the same components in the following embodiments will be denoted by the same reference numerals, and duplicated descriptions will be omitted.

[0012] [Embodiment] The configuration of the information providing system 100 according to the embodiment, the configuration of the information providing device 10, a specific example of the information providing process, and the flow of the information providing process will be described below, and finally the effects of this embodiment will be described.

[0013] 1. Configuration of Information Providing System 100 The processing of an information provision system (referred to as "this system" where appropriate) 100 according to this embodiment will be described using Fig. 1. Fig. 1 is a diagram showing an example of the configuration of the information provision system 100 according to this embodiment. Below, the example of the configuration of this system 100, the processing of this system 100, the set matching technology applied in this system 100, and the effects of this system 100 will be described in that order.

[0014] (1-1. Configuration example of information provision system 100) 1 includes an information providing device 10, a user terminal 20, and a cosmetic information database 30. The information providing device 10, the user terminal 20, and the cosmetic information database 30 are communicably connected via a predetermined communication network (not shown) via wired or wireless communication. Note that the system 100 may include multiple information providing devices 10, multiple user terminals 20, or multiple cosmetic information databases 30.

[0015] (1-1-1. Information provision device 10) The information providing device 10 is a device that transmits and receives data between the user terminal 20 and the cosmetic information database 30, and is realized, for example, by a server device or a cloud system. The example in FIG. 1 shows a case where the information providing device 10 is realized by a server device.

[0016] (1-1-2. User terminal 20) The user terminal 20 is a device (computer) used by a user U to browse web pages related to cosmetics (or "makeup" as appropriate) and to do things like online shopping for cosmetics (or "cosmetics" as appropriate) on the web. The user terminal 20 accepts operations by the user U. The user terminal 20 may be realized, for example, by a smartphone, a tablet terminal, a notebook PC (Personal Computer), a desktop PC, a mobile phone, or a PDA (Personal Digital Assistant). The example in FIG. 1 shows a case where the user terminal 20 is realized by a smartphone.

[0017] Hereinafter, the user terminal 20 may be referred to as the user U. That is, the user U may be read as the user terminal 20. The user U may also be used to represent multiple customers.

[0018] (1-1-3. Cosmetic Information Database 30) The makeup information database 30 is a storage device that stores makeup information related to makeup, which will be described later.

[0019] (1-2. Processing of Information Providing System 100) The following describes steps S1 to S6 as processing of the information providing system 100. Note that the following steps S1 to S6 may be executed in a different order. Also, some of the following steps S1 to S6 may be omitted.

[0020] (1-2-1. Processing of step S1) In the present system 100, first, the information providing device 10 acquires, via the user terminal 20 of the user U, who may be a stylist at the beauty salon, hairstyle information related to hairstyles available to the user U at the beauty salon (step S1). Here, the hairstyle information refers to information related to the hairstyle in general, such as the hairstyle and hair color. The hairstyle information may also be information related to the hairstyle of a wig or toupee, and is not particularly limited. In the example of FIG. 1, the information providing device 10 acquires the hairstyle information from the user terminal 20, but it may also acquire the hairstyle information from a terminal device database (not shown). That is, it is also possible to acquire hairstyle information associated with the hairstyle included in an image selected by the user U from among images showing multiple hair styles displayed on the user terminal 20.

[0021] At this time, the information providing device 10 may acquire user information of the user U from the user terminal 20. Here, the user information is information including user attributes such as the user U's gender, age, generation, occupation, annual income, place of residence, marital status, whether the user U has children, videos of the user U, categories of interest to the user U, and behavioral history such as the user U's search history and browsing history on websites, purchase history in internet shopping, and location information of the user U. Furthermore, the user information may be information including screen information of the user terminal 20, biometric information of the user U, and the like, and is not particularly limited.

[0022] (1-2-2. Processing of step S2) Second, in the present system 100, the information providing device 10 refers to the makeup information database 30 and acquires makeup information (step S2). Here, the makeup information refers to information related to makeup, such as cosmetics such as foundation, lipstick, and eyeshadow, tools and implements used for makeup, and makeup procedures, but is not limited thereto. In the example of FIG. 1, the information providing device 10 acquires the makeup information from the makeup information database 30, but it may also acquire the makeup information from a terminal device or database (not shown).

[0023] (1-2-3. Processing of step S3) Third, in the system 100, the information providing device 10 calculates a compatibility level indicating the degree of compatibility between the hair style and makeup that the user U can provide, based on the acquired hairstyle information and makeup information (step S3). For example, the information providing device 10 uses a machine learning model to calculate the compatibility level, which takes a value between 0 and 1 depending on the degree of compatibility between the combination of hair style and makeup. In this case, when the information providing device 10 receives hairstyle information and makeup information related to the hair style that the user U can provide, the information providing device 10 calculates the compatibility level using a machine learning model such as a DNN (Deep Neural Network) that has been trained to output the compatibility level for each combination of the hair style and makeup. Alternatively, the information providing device 10 may calculate the compatibility level based on rules.

[0024] The above-mentioned step S3 will be described using a specific example. For example, the information providing device 10 can calculate the compatibility degree using a machine learning model trained for each attribute of the subject to whom the user U applies makeup, and a machine learning model trained for each situation of the subject, as shown below.

[0025] The information providing device 10 can calculate the compatibility level using a machine learning model trained for each attribute, including the subject's hobbies and preferences. For example, the information providing device 10 generates multiple machine learning models for a subject who likes "XX (e.g., celebrity, era, type of drama (action, etc.))" and generates correct answer data for each machine learning model. Specifically, the machine learning model is trained using images that have been "liked" by a subject who likes "XX" or images that match "XX" as correct answer data. As a result, the information providing device 10 can select a machine learning model that matches the subject's attributes specified by the user U and suggest makeup that is deemed to have a high compatibility level by the selected machine learning model.

[0026] Furthermore, the information providing device 10 can calculate the compatibility level using a machine learning model trained for each situation of the subject. For example, for each corporate recruitment situation, such as new graduate, job change, and mid-career recruitment, the information providing device 10 generates a machine learning model for each of new graduate, job change, and mid-career recruitment, using images of subjects who passed document screening during their job hunting as correct answer data. Therefore, for example, if the user U selects "job change," the information providing device 10 can suggest makeup that is appropriate for the document screening at the time of job change.

[0027] (1-2-4. Processing of step S4) Fourth, in the system 100, information to be provided to the user U is generated based on the calculated compatibility, the acquired appearance information, and the user information of the user U (step S4). Here, the information to be provided is information about makeup identified from the makeup indicated by the makeup information, such as information about makeup that matches the hairstyle that the user U can wear, but is not limited to such information. At this time, the information providing device 10 identifies makeup indicated by the makeup information that has a compatibility level equal to or greater than a predetermined threshold, and generates information to be provided. For example, the information providing device 10 identifies an image of makeup that has a compatibility level of "0.7" or greater with the hairstyle that the user U can wear, and generates information about the image and the cosmetics required for the makeup indicated in the image as information to be provided.

[0028] The above-mentioned step S4 will be described using a specific example. For example, the information providing device 10 can generate information to be provided based on the user information of the user U as shown below.

[0029] The information providing device 10 can generate information to be provided based on user attributes indicated in the user information. For example, the information providing device 10 can preferentially suggest makeup candidates that are similar to the makeup characteristics of the user U's preferred makeup registered in advance. Furthermore, the information providing device 10 can preferentially suggest images of makeup that are relatively similar to the makeup characteristics of a sample image provided by the user U to a hair and makeup booking site or the like. Therefore, the information providing device 10 can suggest makeup that is highly similar from among the candidates identified from the cosmetic information based on the user attributes, etc.

[0030] Furthermore, the information providing device 10 can generate information to be provided based on the behavioral history indicated by the user information. For example, if the information providing device 10 can estimate the cosmetics that the user U has purchased based on the behavioral history such as the purchase history of the user U, the information providing device 10 can suggest makeup based on the cosmetics.

[0031] (1-2-5. Processing of step S5) Fifth, in the present system 100, the information providing device 10 transmits the generated provision information to the user terminal 20 of the user U (step S5). For example, the information providing device 10 transmits to the user terminal 20 of the user U provision information including an image of makeup that has a compatibility rating of 0.7 or higher with the hair and makeup that the user U can provide, and information on the cosmetics and makeup tools required for the makeup.

[0032] (1-2-6. Processing of step S6) In the present system 100, sixth, the information providing device 10 learns the calculated compatibility degree (step S6). For example, the information providing device 10 uses a machine learning model to learn using backpropagation or the like so that when information indicating a good compatibility between a hair style that the user U can provide and makeup is input, the information providing device 10 outputs a numerical value approaching "1" as the compatibility degree of the makeup for that hair style. Furthermore, when information indicating a bad compatibility between a hair style that the user U can provide and makeup is input, the information providing device 10 uses a machine learning model to learn using backpropagation or the like so that the information providing device 10 outputs a numerical value approaching "0" as the compatibility degree of the makeup for that hair style.

[0033] (1-3. Set matching technology applied in the information provision system 100) Here, the set matching technology applied in the information provision system 100 described above will be described in detail. A technology called set matching is known as an example of a technology for calculating a score indicating the degree of harmony between sets (hereinafter referred to as a "matching score" as appropriate) based on a set of information indicating feature quantities such as images. Set matching is a technology that uses, for example, deep learning, and estimates that the higher the matching score, the higher the compatibility between the sets. Therefore, set matching can quantitatively evaluate the compatibility between the sets. The calculation of a matching score using set matching is disclosed, for example, in "Exchangeable Deep Neural Networks for Set-to-Set Matching and Learning" by Y. Saito, T. Nakamura, H. Hachiya, and K. Fukumizu. Note that set matching is just an example, and embodiments are not limited to those using this technology.

[0034] The information providing device 10 calculates a matching score (compatibility) for each combination of hair style and makeup by applying a technique for calculating matching scores between sets, such as set matching. Specifically, the information providing device 10 calculates the compatibility between makeup and each hair style that the user U can provide. At this time, the information providing device 10 can estimate that the higher the compatibility, the better the compatibility between the hair style that the user U can provide and the makeup. Therefore, the information providing device 10 identifies the makeup that has been calculated to have the highest compatibility for a particular hair style. Furthermore, the information providing device 10 estimates that the identified makeup for a particular hair style is the makeup that has the highest compatibility among the makeups included in the makeup information. This allows the information providing device 10 to quantitatively determine the compatibility between hair styles and makeup, rather than relying on the user U's subjective judgment of appearance, etc.

[0035] The information providing device 10 also trains the machine learning model as follows. For example, the information providing device 10 identifies images posted on an SNS that have been favorably rated by viewers of the SNS. Next, the information providing device 10 identifies the hair style (hairstyle, hair color) and makeup in the identified image. At this time, the information providing device 10 may use various classification models or obtain classification results through crowdsourcing, etc. The information providing device 10 uses the above classification results as correct answer data. The information providing device 10 may also use classification results from images posted by user U as correct answer data. The information providing device 10 then trains the machine learning model so that it outputs a high score when a hair style and makeup that are correct answer data are input, and outputs a low score when a hair style and makeup that are not included in the correct answer data are input. At this time, the information providing device 10 may generate such a machine learning model for each attribute of the subject to whom user U applies makeup, or for each situation of the subject. Furthermore, the information providing device 10 may reflect the degree of compatibility taking into account multiple perspectives by employing manipulation of the output values ​​of multiple models as the output of the machine learning model.

[0036] (1-4. Effects of the information provision system 100) As described above, in the information providing system 100, the information providing device 10 acquires hairstyle information regarding the hair styles that the user U can provide, calculates a compatibility score indicating the degree of compatibility between the hair styles that the user U can provide and makeup based on the hairstyle information, and generates information to be provided to the user U based on the compatibility score. The information providing device 10 calculates the compatibility score using a machine learning model that has learned the degree of compatibility between the hair style and the makeup applied by a subject with that hair style. Therefore, the system 100 can estimate makeup that matches the hair and makeup that the user U can provide or desires to provide, and suggest cosmetics to the user U. In other words, the system 100 enables hairdressers and other users who make hair and makeup suggestions on hair and makeup booking sites and the like to provide useful information to the user U.

[0037] Furthermore, in the present system 100, the information providing device 10 further acquires user information about the user U who provides the hairstyle, identifies makeup based on the acquired user information, and generates information to be provided. As a result, the present system 100 can suggest to the user U hairstyles and hair and makeup that the user U can provide and that have a high affinity with hairstyles that are highly rated on social media and hair and makeup that are favored by celebrities and other famous people. In other words, the present system 100 can provide more specific and useful information to the user U for whom the system is proposing hair and makeup.

[0038] Furthermore, in the present system 100, the information providing device 10 calculates the compatibility level using a machine learning model that has learned the compatibility level between a hair style and makeup applied to a subject with that hair style. The information providing device 10 calculates the compatibility level using a machine learning model that has learned the compatibility level using video images including the hair style and makeup, a machine learning model that has learned the compatibility level for each attribute of the subject receiving the makeup, or a machine learning model that has learned the compatibility level for each situation of the subject. Therefore, the present system 100 generates various machine learning models trained using corrective data collected from social media, television, movies, etc., and can suggest more accurate makeup to the user U who will be applying the makeup. In other words, the present system 100 can provide more specific and useful information with high accuracy to the user U to whom the system 100 is making hair and makeup suggestions.

[0039] 2. Configuration of Information Providing Device 10 The configuration of the information providing device 10 according to the embodiment will be described with reference to Fig. 2. Fig. 2 is a block diagram showing an example of the configuration of the information providing device 10 according to the embodiment. As shown in Fig. 2, the information providing device 10 includes a communication unit 11, a storage unit 12, and a control unit 13. The information providing device 10 may also include an input unit (e.g., a keyboard, a mouse, etc.) that accepts various operations from an administrator of the information providing device 10, and a display unit (e.g., a liquid crystal display, etc.) that displays various information.

[0040] (2-1. Communications Department 11) The communication unit 11 is realized by, for example, a network interface card (NIC), etc. The communication unit 11 is connected to a predetermined communication network by wire or wirelessly, and transmits and receives information to and from various devices.

[0041] (2-2. Storage section 12) The storage unit 12 is realized by, for example, a semiconductor memory element such as a random access memory (RAM) or a flash memory, or a storage device such as a hard disk or an optical disk. As shown in Fig. 2, the storage unit 12 according to the embodiment has a behavior information storage unit 12a, a cosmetic information storage unit 12b, a compatibility information storage unit 12c, and a provided information storage unit 12d. The storage unit 12 stores various types of information referenced when the control unit 13 operates and various types of information acquired when the control unit 13 operates.

[0042] (2-2-1. Mode information storage unit 12a) The aspect information storage unit 12a stores various information (aspect information) relating to the hair aspect that the user U can provide. Here, an example of the information stored in the aspect information storage unit 12a will be described with reference to FIG. 3. FIG. 3 is a diagram showing an example of the aspect information storage unit 12a according to the embodiment. In the example of FIG. 3, the aspect information storage unit 12a has items such as "aspect ID" and "aspect information."

[0043] The "hair style ID" indicates identification information for identifying the hair style. The "hair style information" is information about the hair style, such as the hairstyle and hair color, or information about the hair style using a wig or toupee.

[0044] That is, Figure 3 shows an example in which the aspect information of the hair aspect identified by the aspect ID "HID#1" is "Aspect Information #1", and the aspect information of the hair aspect identified by the aspect ID "HID#2" is "Aspect Information #2".

[0045] (2-2-2. Cosmetic information storage unit 12b) The makeup information storage unit 12b stores various types of makeup-related information (makeup information). An example of the information stored in the makeup information storage unit 12b will be described below with reference to FIG. 4. FIG. 4 is a diagram showing an example of the makeup information storage unit 12b according to the embodiment. In the example of FIG. 4, the makeup information storage unit 12b has items such as "makeup ID" and "makeup information."

[0046] "Makeup ID" indicates identification information for identifying makeup. "Makeup information" refers to information about makeup, such as cosmetics such as foundation, lipstick, and eye shadow, the tools and implements used for makeup, and the makeup procedure.

[0047] That is, FIG. 4 shows an example in which the makeup information of the makeup identified by the makeup ID "DID#1" is "makeup information #1," and the makeup information of the makeup identified by the makeup ID "DID#2" is "makeup information #2."

[0048] (2-2-3. Compatibility Information Storage Unit 12c) The compatibility information storage unit 12c stores the compatibility (compatibility information) of each makeup application with respect to a hair style calculated by the calculation unit 13b of the control unit 13. An example of information stored in the compatibility information storage unit 12c will be described below with reference to FIG. 5. FIG. 5 is a diagram showing an example of the compatibility information storage unit 12c according to an embodiment. In the example of FIG. 5, the compatibility information storage unit 12c has items such as "style ID," "makeup ID," and "compatibility."

[0049] "Style ID" indicates identification information for identifying the style of hair. "Makeup ID" indicates identification information for identifying the makeup. "Compatibility" is a numerical value indicating the degree of compatibility of each makeup with the hair style.

[0050] That is, Figure 5 shows an example in which the hair style identified by the style ID "HID#1" has makeup information "makeup information #1" and a compatibility level "compatibility level #1," and the hair style identified by the style ID "HID#2" has makeup information "makeup information #2" and a compatibility level "compatibility level #2."

[0051] (2-2-4. Provided information storage unit 12d) The provided information storage unit 12d stores the provided information generated by the generation unit 13c of the control unit 13. An example of information stored in the provided information storage unit 12d will now be described with reference to FIG. 6. FIG. 6 is a diagram illustrating an example of the provided information storage unit 12d according to the embodiment. In the example of FIG. 6, the provided information storage unit 12d has items such as "mode ID" and "provided information."

[0052] The "hair style ID" indicates identification information for identifying the hair style. The "provided information" is information about makeup that matches the hair style that the user U can provide, such as cosmetics, makeup tools, and makeup procedures to be provided to the user U.

[0053] That is, Figure 6 shows an example in which the provided information about the hair style identified by the style ID "HID#1" is "Provided Information #1", and the provided information about the hair style identified by the style ID "HID#2" is "Provided Information #2".

[0054] (2-3. Control unit 13) The control unit 13 is realized by, for example, a CPU (Central Processing Unit) or an MPU (Micro Processing Unit) executing various programs (corresponding to an example of an information processing program) stored in a storage device inside the information providing device 10 using RAM as a work area. The control unit 13 is also realized by, for example, an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0055] As shown in Fig. 2, the control unit 13 has an acquisition unit 13a, a calculation unit 13b, a generation unit 13c, a transmission unit 13d, and a learning unit 13e, and realizes or executes the functions and actions of the information processing described below. Note that the internal configuration of the control unit 13 is not limited to the configuration shown in Fig. 2, and may be other configurations as long as they perform the information processing described below. Furthermore, the connection relationship between the processing units included in the control unit 13 is not limited to the connection relationship shown in Fig. 2, and may be other connection relationships.

[0056] (2-3-1. Acquisition part 13a) The acquisition unit 13a acquires, from the user terminal 20, the appearance information relating to the hair appearance that the user U can provide. For example, the acquisition unit 13a acquires, as the appearance information, information relating to the hair appearance, such as the hairstyle and hair color, associated with the image selected by the user U.

[0057] The acquisition unit 13a also acquires makeup-related information. For example, the acquisition unit 13a acquires information related to makeup, such as cosmetics, makeup tools, and makeup application procedures, from the makeup information database 30. At this time, the acquisition unit 13a may acquire, from the makeup information database 30, makeup information associated by a service provider of a hair and makeup booking site. The acquisition unit 13a may also acquire makeup information including makeup that has been registered in advance as being highly compatible with each hair style.

[0058] The acquisition unit 13a also acquires user information about the user U who provides the hair style. For example, the acquisition unit 13a acquires, from the user terminal 20 of the user U, user attributes including categories of interest to the user U, and behavioral history of the user U, such as search history, browsing history, purchase history, and location information.

[0059] The acquiring unit 13a stores the acquired behavior information in the behavior information storage unit 12a. The acquiring unit 13a also stores the acquired makeup information in the makeup information storage unit 12b. Furthermore, the acquiring unit 13a can also store the acquired user information in the storage unit 12.

[0060] (2-3-2. Calculation unit 13b) The calculation unit 13b calculates a compatibility degree indicating the degree of compatibility between a combination of a hair style and makeup based on the hairstyle information. For example, the calculation method will be described. The calculation unit 13b calculates the compatibility degree using a machine learning model that has learned the degree of compatibility between a combination of a hair style and makeup applied by a subject having hair of that style. As a specific example, the calculation unit 13b calculates the compatibility degree using a machine learning model such as a DNN that has been trained to output the compatibility degree for each makeup item with respect to the hair style when hairstyle information and makeup information selected by the user U are input. Alternatively, the calculation unit 13b may calculate the compatibility degree for each makeup item with respect to the hair style selected by the user U on a rule-based basis.

[0061] Regarding the calculated numerical values, the calculation unit 13b calculates the compatibility degree so that the degree of compatibility for each makeup is a numerical value between 0 and 1. The calculation unit 13b may also calculate the compatibility degree so that the degree of compatibility for each makeup is a numerical value between 0 and 100%, and the range and unit of the calculated numerical values ​​are not particularly limited.

[0062] To explain this using a specific example, if the compatibility between hairstyle H and cosmetic M1 is "0.9," the compatibility between hairstyle H and cosmetic M2 is "0.8," and the compatibility between hairstyle H and cosmetic M3 is "0.7," calculation unit 13b estimates that cosmetic M1, which has the highest compatibility, is highly compatible with hairstyle H. This allows calculation unit 13b to appropriately optimize the combination of clothing items and hair styles.

[0063] Furthermore, the learning data of the machine learning model used by the calculation unit 13b will be described. The calculation unit 13b calculates the compatibility degree using a machine learning model that has learned the degree of compatibility from videos including hair styles and makeup. As a specific example, the calculation unit 13b calculates the compatibility degree using a machine learning model that has learned the degree of compatibility from videos, still images, etc. that simultaneously show hair styles and applied makeup collected from websites, SNS, television, movies, etc.

[0064] Furthermore, the calculation unit 13b calculates the compatibility degree using a machine learning model that has learned the degree of compatibility for each attribute of the subject to be made up. As a specific example, the calculation unit 13b calculates the compatibility degree using a plurality of machine learning models generated for each user attribute including the subject's age, favorite entertainers, types of dramas, etc.

[0065] Furthermore, the calculation unit 13b calculates the compatibility degree using a machine learning model that has learned the degree of compatibility for each situation of the subject to be made up. As a specific example, the information providing device 10 calculates the compatibility degree for each situation, such as new graduate, job change, mid-career recruitment, etc., using a machine learning model that has learned images of subjects who passed document screening during their job hunting activities.

[0066] The calculation unit 13b acquires hairstyle information related to the hairstyle selected by the user U from the hairstyle information storage unit 12a. The calculation unit 13b also acquires makeup information indicating makeup from the makeup information storage unit 12b. Meanwhile, the calculation unit 13b stores the calculated compatibility degree in the compatibility degree information storage unit 12c.

[0067] (2-3-3. Generation unit 13c) The generation unit 13c generates information to be provided to the user U based on the calculated compatibility. For example, the generation unit 13c identifies cosmetics indicated in the acquired cosmetic information that have a compatibility level equal to or higher than a predetermined threshold, and generates information to be provided. Explaining this using a specific example, the generation unit 13c identifies cosmetics M1, M2, M3, ..., which are cosmetics necessary for makeup for hairstyle H, which is a hair style that the user U can provide, that have a compatibility level with hairstyle H of "0.7" or higher, and generates information to be provided that includes images of the cosmetics M1, M2, M3, etc.

[0068] Furthermore, the generation unit 13c identifies makeup and generates information to be provided based on the acquired user information of the user U. For example, the generation unit 13c identifies makeup that is registered as a preference of the user U from among makeup that has a compatibility degree equal to or greater than a predetermined threshold, and generates information to be provided that includes the makeup.

[0069] The generating unit 13c acquires the compatibility information from the compatibility information storage unit 12c, and stores the generated provided information in the provided information storage unit 12d.

[0070] (2-3-4. Transmitting unit 13d) The transmitting unit 13d transmits the information to be provided generated by the generating unit 13c to the user U. For example, the transmitting unit 13d transmits to the user terminal 20 of the user U an image including makeup whose compatibility with a hair style that the user U can provide is equal to or exceeds a predetermined threshold. At this time, the transmitting unit 13d may transmit information about cosmetics required for the makeup and makeup steps along with the image including the makeup.

[0071] The transmitting unit 13d acquires the provided information from the provided information storage unit 12d. The transmitting unit 13d may also transmit the provided information to a server, a business operator terminal, or a database (not shown).

[0072] (2-3-5. Learning section 13e) The learning unit 13e learns the machine learning model so as to output the compatibility of each makeup with the hair style when the appearance information and makeup selected by the user U are input. At this time, the learning unit 13e may learn the machine learning model by backpropagation or the like.

[0073] For example, when information indicating that a combination of makeup and a hair style that user U can provide is compatible is input to the machine learning model for each user U, the attributes of user U, and the situation of user U, the learning unit 13e trains the machine learning model so that the compatibility of the makeup with the hair style is output as a number approaching "1."

[0074] On the other hand, the learning unit 13e trains the machine learning model so that when information indicating that the combination of makeup and the hair style that user U can provide is incompatible is input to the machine learning model for each user U, the attributes of user U, and the situation of user U, the degree of compatibility of the makeup with the hair style is output as a number approaching "0".

[0075] [3. Specific examples of information provision processing] Next, a specific example of the information provision process according to the embodiment will be described. Below, the learning process of the machine learning model and the process of generating the information to be provided will be described, and then a specific example of the information to be provided that is transmitted to the user terminal 20 of the user U will be described.

[0076] (3-1. Learning process of machine learning model) The learning process of the machine learning model according to the embodiment will be described. By executing this process, the information providing device 10 can generate a trained machine learning model to be used in the process of generating information to be provided, which will be described later. Below, the supervised data collection process, image data extraction process, and image data learning process will be described in this order.

[0077] (3-1-1. Correct data collection process) First, the information providing device 10 collects correct data from SNS. Here, the collected correct data is, for example, an image in which a hairstyle and makeup are photographed at the same time, and the evaluation satisfies a predetermined condition. More specifically, the collected correct data is an image posted by a predetermined person, an image with more than a predetermined number of "likes," an image with more than a predetermined number of comments or favorable content, etc. Similarly, the information providing device 10 can also collect correct data from moving images such as television commercials and movies.

[0078] (3-1-2. Image data extraction process) Next, the information providing device 10 extracts hair and makeup parts (e.g., face, hands, and feet) from the collected correct answer data image. At this time, the information providing device 10 may extract makeup parts such as eyebrows, eyes, nose, mouth, and nails.

[0079] (3-1-3. Image data learning processing) Then, when a combination of an image of a hair part and an image of each makeup part extracted from the same image (i.e., a combination that is likely to be well-rated) is input, the information providing device 10 trains the machine learning model to output information indicating a high compatibility (e.g., a score of 1). Furthermore, when a combination of an image of a hair part and an image of a makeup part extracted from a different image (i.e., a combination that is not known to be well-rated) is input, the information providing device 10 trains the machine learning model to output information indicating a low compatibility (e.g., a score of 0 or a random number equal to or less than a predetermined value (0.7)). Through the above-described processing, the information providing device 10 generates a machine learning model that has learned the above-described set matching.

[0080] (3-2. Generation process of provided information) The process of generating information to be provided according to the embodiment will be described below. By executing this process, the information providing device 10 can provide, for example, makeup posted by each stylist, and when a user U selects makeup, provide a reservation service for the makeup salon to which the selected stylist belongs (stylist reservation service). Below, the process of acquiring appearance information, the process of generating candidate information, the process of identifying appearance, and the process of transmitting information to be provided will be described in this order.

[0081] (3-2-1. Status information acquisition process) First, the information providing device 10 is a server that provides information on, for example, makeup. At this time, the information providing device 10 acquires images of hair and makeup registered or selected by the user U. That is, through the above-described processing, the information providing device 10 acquires images (style information) of hair styles that the user U can provide.

[0082] (3-2-2. Candidate information generation process) Next, the information providing device 10 generates an image by cutting out the hair portion from the acquired image, and generates candidate information by combining each makeup portion with the image cut out from the hair portion. At this time, the information providing device 10 may accept the makeup image (makeup information) from each user U, or may acquire it from the makeup information database 30 that collects images of makeup portions.

[0083] (3-2-3. Mode Identification Processing) Next, the information providing device 10 inputs each of the generated candidate information into a machine learning model, and identifies makeup, etc. included in the candidate information whose output score exceeds a predetermined threshold. That is, through the above-described process, the information providing device 10 identifies makeup that is highly compatible with the hair style that the user U can wear.

[0084] (3-2-4. Provision information transmission process) The information providing device 10 then generates information to be provided by adding a link to a cosmetic purchasing site or the like to the image of the identified makeup, etc., and transmits one or more pieces of generated information to the user terminal 20. As a result, the information providing device 10 can provide the user U with information such as, "Here is the makeup that goes well with this hairstyle. Would you like to purchase the cosmetics?" The information providing device 10 can also provide the above information to the user U via a server of a hair and makeup reservation site where the user U provides information on beauty salons, etc. In this case, the information providing device 10 can also transmit the information to be provided directly to the user terminal 20 of the user U.

[0085] (3-3. Specific examples of information provided) A specific example of the information to be provided according to the embodiment will be described with reference to Fig. 7. Fig. 7 is a diagram showing a specific example of the information providing process according to the embodiment. Below, a specific example of the information to be provided that is transmitted to and displayed on the user terminal 20 of the user U will be described.

[0086] (3-3-1. Display screen W1) First, we will explain the display screen W1, which shows hair styles, hair colors, and other hair features that can be provided by the user U, as information to be displayed on the user terminal 20 of the user U. The display screen W1 displays an image H1 of a hairstyle H, which is a hair style that the user U uploaded to the hair and makeup booking site, and a screen H2 that shows the treatment fee, treatment details, etc. for that hair style.

[0087] (3-3-2. Display screen W2) Next, we will explain display screen W2, which contains information about makeup and other items that user U can provide for hair styles, such as hairstyles and hair colors, as information to be displayed on user terminal 20 of user U. On display screen W2, screens M1, M2, and M3 are displayed as "cosmetics that go well with this hair" and contain information about cosmetics X, Y, and Z. At this time, user U can click on screens M1, M2, and M3 containing information about cosmetics X, Y, and Z to view details about each cosmetic and images of makeup using the cosmetics. User U can also click on screens M1, M2, and M3 containing information about cosmetics X, Y, and Z to transition to a purchasing site for each cosmetic.

[0088] [4. Information provision process flow] The procedure of information processing of the information providing device 10 according to the embodiment will be described with reference to Fig. 8. Fig. 8 is a flowchart showing an example of the flow of the information providing process according to the embodiment. Note that the following steps S101 to S106 may be executed in a different order. Also, some of the following steps S101 to S106 may be omitted.

[0089] (4-1. Behavioral information collection process) First, the acquisition unit 13a of the information providing device 10 executes a behavior information acquisition process (step S101). For example, the acquisition unit 13a acquires behavior information relating to the hair behavior that the user U can provide from the user terminal 20.

[0090] (4-2. Cosmetic information collection and processing) Second, the acquisition unit 13a of the information providing device 10 executes a makeup information acquisition process (step S102). For example, the acquisition unit 13a refers to the makeup information database 30 and acquires makeup information related to makeup.

[0091] (4-3. Compatibility calculation process) Third, the calculation unit 13b of the information providing device 10 executes a compatibility calculation process (step S103). For example, the calculation unit 13b calculates the compatibility of each makeup with the hair style that the user U can provide, based on the acquired style information and makeup information.

[0092] (4-4. Provision information generation process) Fourth, the generation unit 13c of the information providing device 10 executes a provision information generation process (step S104). For example, the generation unit 13c generates provision information to be provided to the user U based on the calculated compatibility degree and user information.

[0093] (4-5. Provision information transmission process) Fifth, the transmitting unit 13d of the information providing device 10 executes a provided information transmitting process (step S105). For example, the transmitting unit 13d transmits the generated provided information to the user terminal 20 of the user U.

[0094] (4-6. Compatibility learning process) Sixth, the learning unit 13e of the information providing device 10 executes a compatibility learning process (step S106). For example, the learning unit 13e learns the calculated compatibility of each makeup with the hair style that the user U can provide.

[0095] 5. Effects of the embodiment Finally, the effects of the embodiment will be described below: Effects 1 to 7 corresponding to the processing according to the embodiment will be described below.

[0096] (5-1. Effect 1) In the process according to the embodiment described above, aspect information about the hair style that the user U can provide is acquired, and a compatibility score indicating the degree of compatibility between the hair style and the makeup is calculated based on the acquired product information. Therefore, this process can provide useful information to the user U.

[0097] (5-2. Effect 2) In the process according to the embodiment described above, the compatibility degree is calculated using a machine learning model that has learned the compatibility degree between a combination of a hair style and makeup applied by a subject having that hair style. Therefore, this process can more effectively provide useful information to the user U.

[0098] (5-3. Effect 3) In the process according to the embodiment described above, the compatibility degree is calculated using a machine learning model that has learned the compatibility degree from video images including hair styles and makeup. Therefore, this process can provide useful information to the user U more effectively and with high accuracy.

[0099] (5-4. Effect 4) In the process according to the embodiment described above, the compatibility degree is calculated using a machine learning model that has learned the compatibility degree for each attribute of the subject to be made up. Therefore, in this process, useful information can be provided to the user U with high accuracy for each attribute of the subject.

[0100] (5-5. Effect 5) In the process according to the embodiment described above, the compatibility degree is calculated using a machine learning model that has learned the compatibility degree for each situation of the subject to be made up. Therefore, in this process, useful information can be provided to the user U with high accuracy for each situation of the subject.

[0101] (5-6. Effect 6) In the process according to the present embodiment described above, makeup information related to makeup is further acquired, and makeup indicated by the makeup information that has a compatibility level equal to or greater than a predetermined threshold is identified and information to be provided is generated. Therefore, by using the makeup information, this process can more effectively provide useful information to the user U.

[0102] (5-7. Effect 7) In the process according to the present embodiment described above, user information about the user U who provides the hair style is further acquired, makeup is identified based on the user information, and information to be provided is generated. Therefore, by using the user information, this process can provide useful information to the user U more effectively.

[0103] [Hardware configuration] The information providing device 10 according to the above-described embodiment is realized by a computer 1000 having a configuration as shown in Fig. 9, for example. The information providing device 10 will be described below as an example. Fig. 9 is a diagram showing an example of a hardware configuration. The computer 1000 is connected to an output device 1010 and an input device 1020, and has a configuration in which a calculation device 1030, a primary storage device 1040, a secondary storage device 1050, an output I / F (Interface) 1060, an input I / F 1070, and a network I / F 1080 are connected via a bus 1090.

[0104] The arithmetic device 1030 operates based on programs stored in the primary storage device 1040 and the secondary storage device 1050, programs read from the input device 1020, and the like, and executes various processes. The arithmetic device 1030 is realized by, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or the like.

[0105] The primary storage device 1040 is a memory device such as RAM (Random Access Memory) that temporarily stores data used by the arithmetic device 1030 for various calculations. The secondary storage device 1050 is a storage device in which data used by the arithmetic device 1030 for various calculations and various databases are registered, and is realized by a ROM (Read Only Memory), an HDD (Hard Disk Drive), an SSD (Solid State Drive), a flash memory, or the like. The secondary storage device 1050 may be an internal storage device or an external storage device. The secondary storage device 1050 may also be a removable storage medium such as a USB memory or an SD (Secure Digital) memory card. The secondary storage device 1050 may also be cloud storage (online storage), a NAS (Network Attached Storage), a file server, or the like.

[0106] The output I / F 1060 is an interface for transmitting information to be output to an output device 1010 that outputs various types of information, such as a display, a projector, a printer, etc., and is realized by a connector conforming to a standard such as USB (Universal Serial Bus), DVI (Digital Visual Interface), or HDMI (High Definition Multimedia Interface), etc. The input I / F 1070 is an interface for receiving information from various input devices 1020, such as a mouse, a keyboard, a keypad, a button, a scanner, etc., and is realized by a USB, etc.

[0107] Furthermore, the output I / F 1060 and the input I / F 1070 may be wirelessly connected to the output device 1010 and the input device 1020, respectively. That is, the output device 1010 and the input device 1020 may be wireless devices.

[0108] The output device 1010 and the input device 1020 may be integrated into one device, such as a touch panel. In this case, the output I / F 1060 and the input I / F 1070 may also be integrated into one device as an input / output I / F.

[0109] The input device 1020 may be a device that reads information from, for example, an optical recording medium such as a CD (Compact Disc), a DVD (Digital Versatile Disc), or a PD (Phase Change Rewritable Disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory.

[0110] The network I / F 1080 receives data from other devices via the network N and sends it to the arithmetic device 1030, and also transmits data generated by the arithmetic device 1030 to other devices via the network N.

[0111] The arithmetic unit 1030 controls the output device 1010 and the input device 1020 via the output I / F 1060 and the input I / F 1070. For example, the arithmetic unit 1030 loads a program from the input device 1020 or the secondary storage device 1050 onto the primary storage device 1040 and executes the loaded program.

[0112] For example, when the computer 1000 functions as the information providing device 10, the arithmetic unit 1030 of the computer 1000 executes a program loaded onto the primary storage device 1040 to realize the functions of the control unit 130. The arithmetic unit 1030 of the computer 1000 may also load a program acquired from another device via the network I / F 1080 onto the primary storage device 1040 and execute the loaded program. The arithmetic unit 1030 of the computer 1000 may also cooperate with the other device via the network I / F 1080 to call and use the functions and data of a program from another program of the other device.

[0113] 〔others〕 Although the embodiments of the present application have been described above, the present invention is not limited to the contents of these embodiments. Furthermore, the above-described components include those that can be easily imagined by a person skilled in the art, those that are substantially the same, and those that are within the so-called equivalent range. Furthermore, the above-described components can be combined as appropriate. Furthermore, various omissions, substitutions, or modifications of the components can be made without departing from the spirit of the above-described embodiments.

[0114] Furthermore, among the processes described in the above embodiments, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically using a known method. In addition, the information including the processing procedures, specific names, various data, and parameters shown in the above documents and drawings can be changed as desired unless otherwise specified. For example, the various information shown in each drawing is not limited to the information shown in the drawings.

[0115] Furthermore, the components of each device shown in the figure are conceptual functional components and do not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure, and all or part of them can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc.

[0116] For example, the information providing device 10 described above may be realized by multiple server computers, and depending on the function, the configuration can be flexibly changed, such as by calling an external platform using an API (Application Programming Interface) or network computing.

[0117] Furthermore, the above-described embodiments and modifications can be combined as appropriate within the scope of not causing any contradiction in the processing content.

[0118] Furthermore, the above-mentioned "section, module, unit" can be read as "means" or "circuit," etc. For example, a control section can be read as control means or a control circuit. [Explanation of symbols]

[0119] 10 Information provision device 11 Communications Department 12 Storage section 12a Behavior information storage unit 12b Cosmetic information storage unit 12c Compatibility information storage section 12d Provided information storage unit 13 Control Unit 13a Acquisition part 13b Calculation part 13c generator 13d Transmitter 13e Learning Department 20 User terminal 30 Cosmetic Information Database 100 Information Provision System

Claims

1. an acquisition unit that acquires hair style information relating to hair styles that can be provided by a user; a calculation unit that calculates a compatibility level indicating a degree of compatibility between the appearance and makeup based on the appearance information; a generation unit that generates information to be provided to the user based on the compatibility degree; Equipped with The acquisition unit further acquires preferences of the user as user information related to the user who provides the aspect; the generation unit identifies the makeup similar to the preference of the user and generates the provided information. An information providing device characterized by:

2. the calculation unit calculates the compatibility degree using a machine learning model that has learned the compatibility degree between a combination of a hair style and makeup applied by a subject having the hair style; 2. The information providing device according to claim 1.

3. the calculation unit calculates the degree of compatibility using the machine learning model that has learned the degree of compatibility from video images including hair styles and makeup.

3. The information providing device according to claim 2.

4. the calculation unit calculates the degree of compatibility using the machine learning model that has learned the degree of compatibility for each attribute of the subject to be made up.

4. The information providing device according to claim 2 or 3.

5. the calculation unit calculates the degree of compatibility using the machine learning model that has learned the degree of compatibility for each situation of the subject to be made up.

5. The information providing device according to claim 2, wherein the information providing device is a device for providing information to a user.

6. The acquisition unit further acquires makeup information related to makeup applied to the subject, the generation unit identifies makeup indicated by the makeup information that has a compatibility level equal to or higher than a predetermined threshold, and generates the information to be provided.

6. The information providing device according to claim 1, wherein the information providing device is a device for providing information to a user.

7. An information providing method executed by an information providing device, comprising: an acquisition step of acquiring hair style information relating to hair styles that can be provided by a user; a calculation step of calculating a compatibility degree indicating a degree of compatibility between the appearance and makeup based on the appearance information; a generating step of generating information to be provided to the user based on the compatibility degree; Including, The acquiring step further acquires preferences of the user as user information related to the user who provides the aspect; the generating step identifies the makeup similar to the preference of the user and generates the provided information.

1. An information providing method comprising:

8. an acquisition step of acquiring aspect information relating to hair aspects that can be provided by a user; a calculation step of calculating a compatibility degree indicating a degree of compatibility between the appearance and makeup based on the appearance information; a generation step of generating information to be provided to the user based on the compatibility degree; on the computer, The obtaining step further obtains a preference of the user as user information related to the user who provides the aspect; the generation step includes identifying the makeup similar to the user's preference and generating the provided information. An information providing program characterized by:

Images