Information processing device, information processing method, and information processing program

Abstract

Provide appropriate reasons for suggesting coordination (items). [Solution] The information processing device according to the present application has a first calculation unit, a second calculation unit, and a generation unit. The first calculation unit calculates a first score for proposal destination information and proposal target information. The second calculation unit calculates a second score with some information missing from at least one of the proposal destination information and the proposal target information. The generation unit generates a reason for proposing the proposal target based on information where a comparison result between the first score and the second score satisfies a predetermined condition.

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing method, and an information processing program. [Background technology]

[0002] A technique called set matching has been known in the past. For example, a technique is known in which outfit suggestions are made using a model trained to output a score indicating the compatibility of outfits. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 7360412 [Patent Document 2] Patent No. 7407882 Summary of the Invention [Problem to be solved by the invention]

[0004] However, when suggesting coordination (items) with conventional techniques, there is room for further improvement in providing reasons for the suggestion, such as "why it looks good" or "why it is stylish."

[0005] The present application has been made in view of the above, and aims to provide appropriate reasons for suggesting coordination (items). [Means for solving the problem]

[0006] The information processing device according to the present application is characterized by having a first calculation unit that calculates a first score between the proposal destination information and the proposal target information, a second calculation unit that calculates a second score with some information missing from at least one of the proposal destination information and the proposal target information, and a generation unit that generates a reason for proposing the proposal target based on information in which the comparison result between the first score and the second score satisfies a predetermined condition. [Effects of the Invention]

[0007] According to one aspect of the embodiment, it is possible to provide an appropriate reason for suggesting a coordination (item). [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of an information processing system according to an embodiment. [Figure 2] FIG. 2 is an explanatory diagram regarding calculation of a score indicating whether an outfit suits an individual. [Figure 3] FIG. 3 is a diagram illustrating an example of information processing according to the embodiment. [Figure 4] FIG. 4 is a diagram illustrating an example of a prompt according to the embodiment. [Figure 5] FIG. 5 is a diagram illustrating an example of a model learning process according to the embodiment. [Figure 6] FIG. 6 is a diagram illustrating an example of a selection process of coordinate information according to the embodiment. [Figure 7] FIG. 7 is a diagram illustrating an example of the configuration of a user terminal according to the embodiment. [Figure 8] FIG. 8 is a diagram illustrating an example of the configuration of an information processing device according to the embodiment. [Figure 9] FIG. 9 is a diagram illustrating an example of a user information storage unit according to the embodiment. [Figure 10] FIG. 10 is a diagram illustrating an example of an item information storage unit according to the embodiment. [Figure 11] FIG. 11 is a diagram illustrating an example of a model storage unit according to the embodiment. [Figure 12] FIG. 12 is a flowchart illustrating an example of information processing according to the embodiment. [Figure 13] FIG. 13 is a hardware configuration diagram illustrating an example of a computer that realizes the functions of the information processing device. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, an information processing device, an information processing method, and an information processing program according to the present application (hereinafter referred to as "embodiments") will be described in detail with reference to the drawings. Note that the information processing device, the information processing method, and the information processing program according to the present application are not limited to these embodiments. Furthermore, the same components in the following embodiments will be denoted by the same reference numerals, and duplicated descriptions will be omitted.

[0010] (Embodiment) [1. Information Processing System Configuration] An information processing system 1 shown in Fig. 1 will be described. As shown in Fig. 1, the information processing system 1 includes a user terminal 10 and an information processing device 100. The user terminal 10 and the information processing device 100 are connected to each other via a predetermined communication network (network N) so as to be able to communicate with each other via wired or wireless communication. Fig. 1 is a diagram showing an example of the configuration of the information processing system 1 according to an embodiment.

[0011] The user terminal 10 is an information processing device used by a user to whom a proposal is to be made. For example, the user of the user terminal 10 is a user to whom a coordinate (item) that is the subject of a counterproposal proposed by the information processing device 100 is to be proposed. For example, the user of the user terminal 10 is a user of a predetermined service (such as a predetermined e-commerce service or a predetermined SNS (Social Networking Service)) provided by the information processing device 100.

[0012] The user terminal 10 may be any device that can implement the information processing in the embodiment. The user terminal 10 may also be a device such as a smartphone, a tablet terminal, a notebook PC, a desktop PC, a mobile phone, or a PDA. Figure 3 shows a case where the user terminal 10 is a smartphone.

[0013] The user terminal 10 is, for example, a smart device such as a smartphone or tablet, and is a portable terminal device capable of communicating with any server device via a wireless communication network such as 4G to 5G (Generations) or LTE (Long Term Evolution). The user terminal 10 may have a screen such as a liquid crystal display with a touch panel function, and may accept various operations on displayed data such as content, such as tapping, sliding, and scrolling, performed by a user's finger or stylus. In FIG. 3, the user terminal 10 is used by a user U1.

[0014] The information processing device 100 is an information processing device that aims to provide appropriate reasons for suggesting a coordination. For example, when suggesting a coordination, the information processing device 100 is an information processing device that aims to also provide reasons for the suggestion, such as "why it looks good on you" or "why it's stylish." The information processing device 100 is realized, for example, by a server device or a cloud system that suggests coordination on a predetermined service. For example, the information processing device 100 is realized by a server device or a cloud system that provides (or manages) such a predetermined service.

[0015] [2. An example of information processing] A technique for suggesting outfits using a trained model that outputs a score indicating the compatibility of outfits is known. This technique, called set matching, is gaining widespread attention in fields such as fashion.

[0016] By performing set matching on two axes of multiple item group information, it becomes possible to calculate a score indicating whether an outfit is fashionable. For example, by using coordination images posted on SNS or the like as training data and learning coordination information indicating the characteristics of the coordination (multiple item group information indicating the characteristics of each item that makes up the coordination), it becomes possible to calculate a score indicating whether the outfit is fashionable. Specifically, the model is trained to calculate a higher score the closer the coordination is to the coordination images posted on SNS or the like. Also, specifically, the model is trained to calculate a higher score the closer the coordination is to coordination images posted on SNS or the like that have been evaluated as fashionable.

[0017] Furthermore, by performing set matching on three axes, namely, two axes of multiple item group information and user information, it becomes possible to calculate a score indicating whether an outfit will be stylish when worn by a user. For example, by using coordinated outfit images posted on SNS or the like as training data and learning user information indicating user characteristics (gender, age, face, skin color, height, body type, etc.) and coordination information indicating coordination characteristics, it becomes possible to calculate a score indicating whether an outfit will be stylish when worn by a user. Specifically, the model is trained to calculate a higher score the closer the user and coordination are to the coordinated outfit in the coordinated outfit image posted on SNS or the like. Furthermore, the model is trained to calculate a higher score the closer the user and coordination are to the coordinated outfit in a coordinated outfit image posted on SNS or the like that has been evaluated as stylish.

[0018] Furthermore, by performing set matching on three axes, namely, two axes of multiple item group information and user information, it becomes possible to calculate a score indicating whether an outfit suits a user. For example, by using coordinated outfit images posted on SNS or the like as training data and learning user information indicating the user's characteristics and coordination information indicating the characteristics of the coordination, it becomes possible to calculate a score indicating whether an outfit suits a user. Specifically, the model is trained to calculate a higher score the closer the user and the coordination are to the coordinated outfit in the coordinated outfit image posted on SNS or the like. Furthermore, the model is trained to calculate a higher score the closer the user and the coordination are to the coordinated outfit in the coordinated outfit image posted on SNS or the like that is evaluated as looking good.

[0019] In the following embodiment, a model that has been trained to output a score indicating whether an outfit suits a user is used to suggest an outfit. When suggesting an outfit, the model also provides a reason for suggesting the outfit, such as "why it suits a user."

[0020] FIG. 2 is an explanatory diagram regarding the calculation of a score indicating whether an outfit matches well. In FIG. 2, set matching is performed along three axes: a group of closet items owned by user U1, a group of gallery items, and user information (e.g., physical information) of user U1. Here, a gallery item is an item from a group of items that is a candidate for coordination with a group of closet items, and is, for example, an item that can be purchased on a predetermined service. For example, it is an item selected from items that can be traded on a predetermined e-commerce service. Note that instead of set matching between a group of closet items and a group of gallery items, set matching may be performed between groups of closet items, or between groups of gallery items.

[0021] 2 shows that for user U1, the degree to which the coordination of closet item group G1 and gallery item group G2 suits them is 90% (score 90), the degree to which the coordination of closet item group G1 and gallery item group G3 suits them is 70%, and the degree to which the coordination of closet item group G1 and gallery item group G4 suits them is 30%. Because these results take into account the physical information of user U1, it is possible that the degree of suitability results will be different for other users with completely different physical information.

[0022] There is also a well-known technology called LIME (Local Interpretable Model-agnostic Explanations), which enables local interpretation by approximating locally with a linear model. This technology makes it possible to determine which information (features, etc.) is important for information processing. For example, by preparing a large number of pairs of scores and missing patterns when some of the input data is missing, the relationship between the scores and missing patterns can be obtained, making it possible to determine which information is important.

[0023] In the following embodiment, the missing portion is changed and the process of obtaining the score output is repeated. For example, among the portions of the body information such as the waist circumference, chest circumference, arms, and neck, the body circumference portion is first missing and processed, then the chest circumference portion is missing and processed, then the arms portion is missing and processed, and finally the neck portion is missing and processed. In this case, for example, if the input data is an image, the missing portion is made in pixel or superpixel units. Also, for example, if the input data is table data, the missing portion is made in cell units. Note that the user information is not limited to body information, and may be gender information or age information, and the information processing device 100 may make the gender missing and / or the age missing and process.

[0024] In the following embodiment, LIME is applied to the set of input data and output data created in this way, and L1 regularization is performed by approximating with a linear model, and the importance of the input data is estimated.

[0025] Hereinafter, information processing according to the embodiment will be described with reference to FIG. 3. FIG. 3 is a diagram illustrating an example of information processing according to the embodiment. The information processing device 100 acquires an image of an outfit posted on a predetermined service as learning data (step S101). This image includes a user (wearer) and multiple items of the outfit. The information processing device 100 may, for example, acquire user information and outfit information linked to the image, or may acquire user information and outfit information via image analysis technology or the like, or may acquire user information linked to the image and outfit information via image analysis technology or the like, or may acquire user information and outfit information linked to the image via image analysis technology or the like.

[0026] The information processing device 100 then uses the training data to generate a model that, when input with user information and coordinate information, outputs a score indicating whether the coordinate suits the user, and trains the model to output a higher score as the user evaluates the coordinate as looking good (step S102). Note that the model is not limited to a set matching model, and a multimodal large-scale language model that can handle images may be used. In this case, the information processing device 100 may generate a model that, when input with a coordinate image including user information and coordinate information, outputs a score indicating whether the coordinate suits the user, and trains the model to output a higher score as the user evaluates the coordinate as looking good. The information processing device 100 may also generate a model that, when input with user information and a coordinate image including coordinate information, outputs a score indicating whether the coordinate suits the user, and trains the model to output a higher score as the user evaluates the coordinate as looking good, or may generate a model that, when input with a coordinate image including user information and coordinate information, outputs a score indicating whether the coordinate suits the user, and trains the model to output a higher score as the user evaluates the coordinate as looking good.

[0027] The information processing device 100 inputs the user information of user U1 and the coordination information of the target coordination into the trained model (which may be a set matching model or a multimodal large-scale language model) and calculates a score indicating whether the target coordination suits user U1 (step S103). At this time, when the information processing device 100 uses a multimodal large-scale language model, the information processing device 100 calculates a score indicating whether the target coordination suits user U1 using a coordination image including the user information of user U1 and the coordination information of the target coordination as one piece of input information. Hereinafter, the score calculated in this manner will be referred to as the "first score" as appropriate. Whether a set matching model or a multimodal large-scale language model is used, or whether any trained model is used, the score calculated in this manner will be referred to as the "first score." Hereinafter, the model at this time will be referred to as the "model M1" as appropriate. Then, for example, the information processing device 100 proposes a coordination that has the highest first score. Furthermore, for example, the information processing device 100 proposes a coordination whose first score is equal to or greater than a predetermined score.

[0028] Furthermore, the information processing device 100 deletes a portion of at least one of the user information of the user U1 and the coordinate information of the target coordinate, inputs the deleted information into the model M1, and calculates the score again (step S104). Hereinafter, the score calculated in this deleted state will be referred to as the "second score" as appropriate. Note that, when the model M1 is a multimodal large-scale language model, the information processing device 100 may delete a portion (in pixel or superpixel units) of the coordinate image including at least one of the user information and the coordinate information, input the deleted coordinate image into at least the model M1, and calculate the score again.

[0029] The information processing device 100 changes the missing information (missing portions, etc.) and repeats the process of step S104, thereby obtaining a plurality of pairs of second scores and missing patterns.

[0030] Here, the missing information according to the embodiment will be described. When part of the user information is missing, for example, information on parts such as the arms and legs (body information, etc.) is missing. For example, if user U1 wants a coordination suggestion that takes waist circumference into consideration, it can be assumed that there will be no significant change in the score even if information on the arms and legs is missing. Furthermore, when part of the coordination information is missing, for example, information on the item, part of the item (for example, the collar), color, shape, size, and appearance (also known as the way of wearing, for example, whether the sleeves are rolled up) is missing.

[0031] In the embodiment, the user information may be any information relating to the user, and the missing information may be any information relating to the user. Similarly, the coordinate information may be any information relating to a coordinate, and the missing information may be any information relating to a coordinate.

[0032] Then, the information processing device 100 estimates which information (missing information) is important based on the fluctuation of the score according to the loss pattern (step S105). For example, the information processing device 100 estimates that information whose score fluctuates greatly (for example, decreases) is important.

[0033] For example, by applying LIME, the information processing device 100 identifies information for which the comparison result between the first score and the second score satisfies a predetermined condition, and estimates that the information is important.

[0034] Then, the information processing device 100 causes a generation AI such as a Generative Pre-trained Transformer (GPT) to generate a reason for proposal that takes into account the type of information (step S106). For example, the information processing device 100 generates the reason for proposal by requesting (instructing) the generation AI to generate a reason for proposal that takes into account the type of information. Note that the generation AI may be a multimodal large-scale language model, which is model M1.

[0035] Then, the information processing device 100 provides the reason for suggestion generated by the generation AI to the user U1 (step S107). For example, the information processing device 100 provides the reason for suggesting the coordination together with the coordination suggestion.

[0036] Here, an example of a prompt to instruct the generation AI is given. For example, a prompt such as that shown in Fig. 4 can be given. The information processing device 100 provides such a prompt to the generation AI, thereby requesting the generation AI to generate a reason for proposal.

[0037] The prompt shown in FIG. 4 requests that a reason for suggesting the coordination to be provided to user U1 be generated. The prompt also requests that the reason for suggestion be generated taking into consideration the type of information estimated to be an important factor in why the coordination suits user U1. The prompt also requests that a reason for suggestion be generated assuming that the type of information estimated to be an important factor is "XX", the user information is "XX", and the coordination information is "XX". Note that the information processing device 100 may provide a coordination image including the user information and the coordination information (which may be a virtual try-on image that recreates the appearance of the user wearing the coordination, generated based on the user information and the coordination information) and request that a reason for suggestion be generated. Note that the information processing device 100 may provide the generation AI with the user information and the coordination information and request that the generation AI generate a virtual try-on image that recreates the appearance of the user wearing the coordination.

[0038] (Variations in information processing) In the above embodiment, when the information processing device 100 receives further requests from the user U1 (such as concerns about body shape, the impression the user wants to give others, or a desire to try complex fashion), the information processing device 100 proposes outfits that take the requests into consideration and provides reasons for the proposal. Specifically, the information processing device 100 performs set matching that is expanded to take the requests into consideration. In this case, for example, the information processing device 100 may receive from the user U1 a concern such as being overweight and being concerned about her waistline, or a request from the user U1 to find outfits that will not make her waistline less noticeable even if she is overweight, or a request to hide her problem areas. The information processing device 100 then performs set matching that is expanded to take such concerns and requests into consideration. Below, the processing when a request is received from the user U1 will be described as an example; however, similar processing may be performed when a concern is received. For example, when a concern is received, the information processing device 100 may identify a request corresponding to the concern and perform expanded set matching to take the identified request into consideration.

[0039] The information processing device 100, when receiving input of user information, coordination information, and desired information, generates a model that outputs a score indicating whether a coordination suits the user, and trains the model to output a higher score when the coordination satisfies the user's desires and is evaluated as looking good when selected. Specifically, the information processing device 100 trains the model to calculate a higher score the closer the coordination image posted on a social networking site or the like is to the user (the user's concerns, for example, the parts the user is hiding) and the coordination (the impression the user wants to give to others, for example, looking mature, and how much effort the user put into it, for example, a complexity rating of 5 stars). Furthermore, specifically, the information processing device 100 trains the model to calculate a higher score the closer the coordination image posted on a social networking site or the like is to the user (the user's concerns, for example, the parts the user is hiding) and the coordination (the impression the user wants to give to others, for example, looking mature, and how much effort the user put into it, for example, a complexity rating of 5 stars) that is evaluated as looking good.

[0040] Hereinafter, the model in this case will be referred to as "model M2" as appropriate. Note that the information processing device 100 may generate model M2 by additional learning on model M1. Specifically, the information processing device 100 may generate model M2 by additional learning on model M1 so that it can propose outfits that take requests into consideration and present the reasons for the proposals. Model M2 is not limited to a set matching model, and may also be a multimodal large-scale language model that can handle images.

[0041] FIG. 5 is a diagram illustrating an example of a model learning process according to an embodiment. In the example illustrated in FIG. 5, various pieces of information, such as user information and outfit information, are converted into features, and a match is determined for each combination of features (i.e., whether the combination of each user information and outfit information matches) through set matching or the like using each feature. In FIG. 5, examples of user information include multiple types of user information, such as information about the user, information indicating the user's preferences, such as "I want to be," information (e.g., images) indicating the user's appearance, and a pie chart showing the classification and distribution of the user's outfits. Each of these types of user information is individually converted into feature quantities UF1 to UF5. For example, a person image 1 that the user desires to become is converted into feature quantity UF2, another person image 2 that the user desires to become is converted into feature quantity UF3, the user's appearance is converted into feature quantity UF4, and the outfit classification pie chart (preferences) is converted into feature quantity UF5. Note that this conversion into features can be achieved using any vector conversion technology that converts arbitrary information into vectors that are similar enough to be similar.

[0042] Similarly, various pieces of coordinate information are converted into individual features in Fig. 5. For example, in the example shown in Fig. 5, the classification pie chart of the coordinates is converted into feature CF1, the coordinate image is converted into feature CF2, the image consulting classification of item 1 is converted into feature CF3, the image consulting classification of another item 2 is converted into feature CF4, and the way item 1 is worn is converted into feature CF5.

[0043] 5, the information processing device 100 inputs various feature quantities corresponding to the user information and various pieces of coordinate information into a model M2 or the like to calculate a first score. At this time, the feature quantities may be converted in any manner, and the conversion method may not be particularly limited. Furthermore, the learning data includes, for example, user information, coordinate information of a matching reference, and coordinate information of a non-matching reference.

[0044] 5, the information processing device 100 may filter input information to be input to the model M2, etc. For example, the information processing device 100 may perform filtering based on any of the various types of information in the user information and any of the various types of information in the coordination information. For example, the information processing device 100 may perform filtering based on a classification pie chart (preferences) of coordination and a classification pie chart of coordination.

[0045] Here, various pieces of coordinate information may be selected based on user information. FIG. 6 is a diagram illustrating an example of a selection process for coordinate information according to an embodiment. In FIG. 6, the information processing device 100 generates various pieces of user information using a generation AI. For example, the information processing device 100 uses a prompt such as, "Based on the user information, please generate the user's input information, a person image 1 that the user desires to become, another person image 2 that the user desires to become, and the user's appearance." to cause the generation AI to generate various pieces of user information. At this time, for example, the information processing device 100 may use a prompt such as, "Please weight various pieces of information along with various pieces of user information." to cause the generation AI to weight the various pieces of user information.

[0046] Then, for example, the information processing device 100 converts the various pieces of user information generated by the generation AI into various feature quantities. At this time, for example, the information processing device 100 may convert into the various feature quantities taking into consideration the weighting of the various pieces of user information. As a result, feature quantities UF1 to UF4 are obtained.

[0047] Furthermore, for example, the information processing device 100 selects various pieces of coordinate information from the reference coordinate image by inputting various target feature quantities into a model that selects various pieces of coordinate information from various feature quantities. Hereinafter, the model in this case will be referred to as "model M3" as appropriate. Model M3 is a model that has learned, for example, the latest coordinate trend information. As a result, information such as "coordinate characteristics," "item 1 characteristics" (referring to the hat in FIG. 6), "item 2 characteristics" (referring to the shirt in FIG. 6), "item 3 characteristics" (referring to the pants in FIG. 6), "item 4 characteristics" (referring to the socks in FIG. 6), and "item 5 characteristics" (referring to the shoes in FIG. 6) can be obtained.

[0048] Then, for example, the information processing device 100 generates a reason for proposal using a generation AI from various pieces of user information and various pieces of coordinate information. For example, the information processing device 100 causes the generation AI to generate a reason for proposal using a prompt such as, "Please generate a reason for proposal from various pieces of user information and various pieces of coordinate information." At this time, for example, the information processing device 100 may cause the generation AI to generate a reason for proposal that takes into account the weighting of the various pieces of user information using a prompt such as, "Please generate a reason for proposal taking into account the weighting of the various pieces of user information." Furthermore, when a multimodal large-scale language model is used, the information processing device 100 may cause the generation AI to generate a reason for proposal using a prompt such as, "Please generate a reason for proposal from a coordinate image including various pieces of user information and various pieces of coordinate information."

[0049] When such an extended set matching model (i.e., model M2) is available, the information processing device 100 inputs the user information and desired information of user U1 and the coordinate information of the target coordinate to calculate the first score. Then, the information processing device 100 identifies the coordinate that results in the highest first score. Note that, when model M2 is a multimodal large-scale language model, the information processing device 100 may input a coordinate image that includes the user information and desired information of user U1 and the coordinate information of the target coordinate to calculate the first score.

[0050] Furthermore, the information processing device 100 may delete information of a type corresponding to the user U1's request from at least one of the user information and the coordinate information of the target coordinate, input the information after the deletion into the model M2, and calculate the second score. As in the above embodiment, by repeating this process while changing the missing information, multiple pairs of second scores and deletion patterns that take the request into consideration are obtained. Note that, when the model M2 is a multimodal large-scale language model, the information processing device 100 may delete a portion (in pixel or superpixel units) of the coordinate image that includes at least one of the user information and the coordinate information, input the coordinate image after the deletion into at least the model M2, and calculate the second score.

[0051] Then, the information processing device 100 identifies missing information that has a high second score, and estimates that the missing information is important.

[0052] Then, similar to the above embodiment, the information processing device 100 generates a proposal reason that takes into consideration the type of the information and presents the proposal reason. That is, the information processing device 100 explains the reason why the information satisfies the user's needs or suits the user based on the type of the information.

[0053] For example, when the information processing device 100 receives a concern from user U1, it may estimate important missing information by performing various deletions according to the concern. By performing such processing, the information processing device 100 can identify the type of coordination information corresponding to the concern and provide an explanation according to the identified type. For example, when various pieces of coordination information are deleted, the information processing device 100 can adopt the type of coordination information with the largest change in score as the type of coordination information corresponding to the concern. To give a more specific example, the information processing device 100 calculates each score by sequentially deleting other feature amounts while inputting the feature amount corresponding to the concern, "I'm worried about my shoulder line." Then, for example, when the score is lowest after deleting the feature amount of a top among the coordination information, the information processing device 100 can estimate that the top is an item that will solve the concern and provide an explanation such as, "This top will solve your shoulder concerns."

[0054] Furthermore, the information processing device 100 may, for example, select a proposed coordination and then generate a reason for proposal that is in line with the proposed coordination, or may first generate a reason for proposal and then select a proposed coordination. For example, the information processing device 100 may generate a reason for proposal based on the concerns and requests of user U1, and select a proposed coordination based on the generated reason for proposal.

[0055] [3. User terminal configuration] Next, the configuration of the user terminal 10 according to the embodiment will be described with reference to Fig. 7. Fig. 7 is a diagram showing an example of the configuration of the user terminal 10 according to the embodiment. As shown in Fig. 7, the user terminal 10 has a communication unit 11, an input unit 12, an output unit 13, and a control unit 14.

[0056] (Communications Department 11) The communication unit 11 is realized by, for example, a network interface card (NIC), etc. The communication unit 11 is connected to a predetermined network N by wire or wirelessly, and transmits and receives information to and from the information processing device 100, etc., via the predetermined network N.

[0057] (Input section 12) The input unit 12 accepts various operations from the user. For example, the input unit 12 may accept various operations from the user via a display screen using a touch panel function. Alternatively, the input unit 12 may accept various operations from buttons provided on the user terminal 10 or a keyboard or mouse connected to the user terminal 10.

[0058] (Output section 13) The output unit 13 is a display screen of a tablet terminal or the like realized by, for example, a liquid crystal display or an organic EL (Electro-Luminescence) display, and is a display device for displaying various information. For example, the output unit 13 displays information provided from the information processing device 100.

[0059] (Control unit 14) The control unit 14 is, for example, a controller, and is realized by a CPU (Central Processing Unit) or MPU (Micro Processing Unit) executing various programs stored in a storage device inside the user terminal 10 using RAM (Random Access Memory) as a work area. For example, these various programs include application programs installed in the user terminal 10. For example, these various programs include an application program that displays the reason for suggesting a coordination. The control unit 14 is also realized by an integrated circuit, such as an ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array).

[0060] As shown in FIG. 7, the control unit 14 has a receiving unit 141 and a transmitting unit 142, and realizes or executes the information processing operations described below.

[0061] (Receiving unit 141) The receiving unit 141 receives various information from another information processing device such as the information processing device 100. For example, the receiving unit 141 receives a reason for proposal generated by another information processing device such as the information processing device 100. Furthermore, for example, the receiving unit 141 receives coordination information of a proposed coordination selected by another information processing device such as the information processing device 100.

[0062] (Transmitter 142) The transmission unit 142 transmits various information to other information processing devices such as the information processing device 100. For example, the transmission unit 142 transmits user information. For example, the transmission unit 142 transmits user information such as user attributes, physical information (body type information), owned items (closet items, etc.), and purchase history. In addition, for example, the transmission unit 142 transmits request information received from the user.

[0063] 4. Configuration of Information Processing Device Next, the configuration of the information processing device 100 according to the embodiment will be described with reference to Fig. 8. Fig. 8 is a diagram showing an example of the configuration of the information processing device 100 according to the embodiment. As shown in Fig. 8, the information processing device 100 has a communication unit 110, a storage unit 120, and a control unit 130. Note that the information processing device 100 may also have an input unit (e.g., a keyboard or a mouse) that accepts various operations from an administrator of the information processing device 100, and a display unit (e.g., a liquid crystal display) that displays various information.

[0064] (Communication unit 110) The communication unit 110 is realized by, for example, a NIC etc. The communication unit 110 is connected to a network N by wire or wirelessly, and transmits and receives information to and from the user terminal 10 etc. via the network N.

[0065] (Storage unit 120) The storage unit 120 is realized by, for example, a semiconductor memory element such as a RAM or a flash memory, or a storage device such as a hard disk or an optical disk. As shown in FIG. 8 , the storage unit 120 has a user information storage unit 121, an item information storage unit 122, and a model storage unit 123.

[0066] The user information storage unit 121 stores user information. Fig. 9 shows an example of the user information storage unit 121 according to the embodiment. As shown in Fig. 9, the user information storage unit 121 has items such as "user ID" and "user information."

[0067] "User ID" indicates identification information for identifying a user. "User information" indicates user information. In the example shown in FIG. 9, conceptual information such as "User information #1" and "User information #2" is stored in "User information," but in reality, information indicating the user's gender and age, numerical information regarding their body or figure, information indicating items they own (such as closet items), and information indicating products they have purchased from a specific e-commerce service (such as gallery items) is stored.

[0068] The item information storage unit 122 stores item information that can be presented when proposing a coordination. For example, it stores item information that can be purchased on a predetermined service such as an e-commerce service. FIG. 10 shows an example of the item information storage unit 122 according to the embodiment. As shown in FIG. 10, the item information storage unit 122 has fields such as "item ID" and "item information."

[0069] "Item ID" indicates identification information for identifying an item (such as a product). "Item information" indicates item information (such as product information). In the example shown in FIG. 10, conceptual information such as "Item Information #1" and "Item Information #2" is stored in "Item Information," but in reality, information indicating the characteristics of the item, such as the item name and item image (which may include image information), or, in the case of an item from an e-commerce service, text information included in the product page (which may include information related to tags, ratings, etc.) may be stored. In addition, for example, a uniform resource locator (URL) where image information for the item image is located, or a file path name indicating the storage location, may also be stored.

[0070] The model storage unit 123 stores information about a model (such as model M1 or model M2) that outputs a score indicating whether an outfit suits an outfit. FIG. 11 shows an example of the model storage unit 123 according to the embodiment. As shown in FIG. 11, the model storage unit 123 has items such as "model ID" and "model information."

[0071] "Model ID" indicates identification information for identifying a model. "Model information" indicates information related to the model. In the example shown in FIG. 11, conceptual information such as "Model information #1" and "Model information #2" is stored in "Model information," but in reality, parameter information and learning data of the model are stored.

[0072] (control unit 130) The control unit 130 is a controller, and is realized by, for example, a CPU or an MPU executing various programs stored in a storage device inside the information processing device 100 using RAM as a work area. The control unit 130 is also realized by, for example, an integrated circuit such as an ASIC or an FPGA.

[0073] 8, the control unit 130 has an acquisition unit 131, a learning unit 132, a first calculation unit 133, a second calculation unit 134, an estimation unit 135, a generation unit 136, and a provision unit 137, and realizes or executes the information processing action described below. Note that the internal configuration of the control unit 130 is not limited to the configuration shown in FIG. 8, and may be any other configuration that performs the information processing described below.

[0074] (Acquisition part 131) The acquisition unit 131 acquires various pieces of information from an external information processing device. The acquisition unit 131 acquires various pieces of information from other information processing devices such as the user terminal 10.

[0075] The acquiring unit 131 acquires various pieces of information from the storage unit 120. The acquiring unit 131 also stores the acquired various pieces of information in the storage unit 120.

[0076] The acquisition unit 131 acquires, for example, images posted to a predetermined service such as a social networking service, etc. Furthermore, the acquisition unit 131 acquires, for example, user information and coordinate information linked to such images.

[0077] The acquisition unit 131 acquires, for example, request information regarding the coordination of the user to whom the suggestion is made. Also, the acquisition unit 131 acquires, for example, user information indicating the characteristics of the user to whom the suggestion is made and coordination information of the target coordination.

[0078] (Learning Section 132) The learning unit 132, for example, learns a model that outputs a score indicating whether an outfit suits a user. For example, the learning unit 132 trains the model to output a higher score as the outfit is evaluated as looking good by the user. Furthermore, for example, the learning unit 132 trains the model to output a higher score as the outfit satisfies the user's needs when selected and is evaluated as looking good.

[0079] (First calculation unit 133) The first calculation unit 133 calculates, for example, a first score between the user information and the coordinate information. For example, the first calculation unit 133 calculates the first score using a model learned by the learning unit 132. For example, the first calculation unit 133 calculates the first score by inputting the user information of the suggested item and the coordinate information of the target coordinate into the model.

[0080] (Second calculation unit 134) For example, the second calculation unit 134 calculates the second score between the user information and the coordination information with some information missing. For example, the second calculation unit 134 calculates the second score between the user information and the coordination information with some information missing from at least one of the user information and the coordination information. For example, the second calculation unit 134 calculates the second score with some information corresponding to the user's request missing.

[0081] (Estimation part 135) The estimation unit 135 estimates that, for example, information in which a comparison result between the first score calculated by the first calculation unit 133 and the second score calculated by the second calculation unit 134 satisfies a predetermined condition is an important factor as a reason why the outfit looks good. For example, the estimation unit 135 estimates that information corresponding to the second score that has a large variation from the first score is important.

[0082] (Generation unit 136) The generation unit 136 generates the reason for proposal based on, for example, the result of estimation by the estimation unit 135. For example, the generation unit 136 generates the reason for proposing coordination based on the type of information estimated by the estimation unit 135.

[0083] The generation unit 136 generates the reason for proposal by, for example, requesting the generation AI to generate a reason for proposal. For example, the generation unit 136 generates the reason for proposal of coordination by requesting the generation AI to generate a reason for proposal in consideration of the type of information estimated by the estimation unit 135.

[0084] (Providing Department 137) The providing unit 137 provides, for example, the reason for proposal generated by the generating unit 136. For example, the providing unit 137 provides the reason for proposal generated by the generating unit 136 to a user to whom the coordination is proposed. For example, the providing unit 137 proposes coordination and provides the reason for proposing the coordination.

[0085] [5. Information Processing Flow] Next, the procedure of information processing by the information processing system 1 according to the embodiment will be described with reference to Fig. 12. Fig. 12 is a flowchart showing the procedure of information processing by the information processing system 1 according to the embodiment.

[0086] 12, the information processing device 100 acquires user information of a user to whom a suggestion is made and coordinate information of a target coordinate (step S201). The information processing device 100 calculates a first score by inputting the acquired information into a predetermined model (step S202). The information processing device 100 calculates a second score by inputting information after partial information loss from the acquired information into the predetermined model (step S203).

[0087] The information processing device 100 estimates important factors as reasons why the outfit suits the user based on the calculated first score and second score (step S204). The information processing device 100 generates a suggestion reason that takes the estimated factors into consideration (step S205). The information processing device 100 provides the generated suggestion reason (step S206).

[0088] In the above embodiment, the information processing device 100 uses learning data to generate a model M1 that, when user information and coordination information are input, outputs a score indicating whether an outfit suits the user, and is trained to output a higher score the more the outfit is evaluated as looking good on the user. However, it may also be possible to generate a model that, when coordination information is input, outputs a score indicating whether the outfit is fashionable, and is trained to output a higher score the more the outfit is evaluated as fashionable, or to generate a model that, when user information and coordination information are input, outputs a score indicating whether the outfit is fashionable when worn by the user, and is trained to output a higher score the more the outfit is evaluated as fashionable when worn by the user.

[0089] Furthermore, the information processing device 100 may generate a model as model M2 that outputs a score indicating whether an outfit suits a user when user information, coordination information, and desired information are input, and train the model to output a higher score when the outfit satisfies the user's desires and is evaluated as looking good when selected. Alternatively, the information processing device 100 may generate a model that outputs a score indicating whether an outfit is fashionable when coordination information and desired information are input, and train the model to output a higher score when the outfit satisfies the user's desires and is evaluated as fashionable when selected, or may generate a model that outputs a score indicating whether an outfit is fashionable when worn by a user when user information, coordination information, and desired information are input, and train the model to output a higher score when the outfit satisfies the user's desires and is evaluated as fashionable when selected. The information processing device 100 may then calculate the first score and the second score, and provide a reason why the outfit is fashionable.

[0090] Furthermore, when the information processing device 100 receives a fashion concern, it may estimate important missing information by performing various deletions according to the fashion concern. The information processing device 100 may identify the type of coordination information corresponding to the fashion concern and provide an explanation according to the identified type. For example, when various pieces of coordination information are deleted, the information processing device 100 may adopt the type of coordination information with the largest change in the score indicating whether it is fashionable as the type of coordination information corresponding to the fashion concern. To give a more specific example, the information processing device 100 may continue inputting features corresponding to a specific fashion concern, and calculate scores indicating whether it is fashionable by sequentially deleting other features related to fashion. Then, for example, when the score indicating whether it is fashionable is lowest after deleting the feature of a top among the coordination information, the information processing device 100 may estimate that the top is an item that will solve the fashion concern and provide an explanation such as, "This top will help you solve your fashion concern."

[0091] In the above embodiment, a coordination, which is a combination of items, is proposed and the reason for the proposal is provided. However, an item (such as an individual item) may also be proposed and the reason for the proposal provided. In this case, coordination information may be interpreted as item information. For example, a reason for a look may be provided by evaluating whether an item looks good when worn by a user, or a reason for a look may be provided by evaluating whether an item looks fashionable when worn by a user. Furthermore, in the above embodiment, the coordination and the item may be collectively interpreted as a "suggestion target," and the user may be interpreted as a "suggestion recipient." Correspondingly, coordination information may be interpreted as "suggestion target information," user information as "suggestion recipient information," and the coordination image as a "suggestion target image."

[0092] [6. Effects] As described above, the information processing device 100 according to the embodiment includes a first calculation unit 133, a second calculation unit 134, and a generation unit 136. The first calculation unit 133 calculates a first score between the proposal destination information and the proposal target information. The second calculation unit 134 calculates a second score in a state where some information of at least one of the proposal destination information and the proposal target information is missing. The generation unit 136 generates a reason for proposing the proposal target based on information in which the comparison result between the first score and the second score satisfies a predetermined condition.

[0093] Thereby, the information processing device 100 according to the embodiment can provide a reason for suggesting an appropriate coordination (item) by taking into consideration, for example, which information is an important element as to why the coordination looks good on the user. Also, the information processing device 100 according to the embodiment can provide an appropriate reason for suggesting an appropriate coordination by taking into consideration, for example, which information of at least one of the user information and the coordination information is an important element as to why the coordination looks good on the user.

[0094] In addition, the first calculation unit 133 calculates the first score using a model that outputs a score indicating whether the proposed object indicated by the proposed target information suits the proposed recipient indicated by the proposed recipient information included in the input information, and that is trained to output a higher score the more the proposed object is evaluated as suiting the proposed recipient.

[0095] As a result, the information processing apparatus 100 according to the embodiment can calculate a more appropriate first score by using, for example, a learning model.

[0096] Furthermore, the second calculation unit 134 inputs the proposal destination information and proposal target information of the state into the model, and calculates the second score.

[0097] As a result, the information processing device 100 according to the embodiment can calculate a more appropriate second score by using, for example, a learning model.

[0098] Furthermore, the second calculation unit 134 calculates a second score by inputting a proposal target image including proposal destination information and proposal target information of the state to the model.

[0099] As a result, the information processing device 100 according to the embodiment can calculate a more appropriate second score by using, for example, a learning model.

[0100] Furthermore, the generating unit 136 generates a reason based on the type of information that satisfies a predetermined condition.

[0101] As a result, the information processing apparatus 100 according to the embodiment can provide an appropriate reason for suggestion that takes into consideration, for example, the type of information in which a large change has been obtained.

[0102] Furthermore, the generating unit 136 generates a reason by requesting the generating AI to generate a reason taking the type into consideration.

[0103] As a result, the information processing apparatus 100 according to the embodiment can provide the reason for proposal more quickly by, for example, causing the generation AI to generate the reason for proposal.

[0104] Furthermore, when the generation unit 136 receives a request from a proposal recipient, the generation unit 136 generates a reason for taking the request into consideration in order to propose a proposal target that takes the request into consideration.

[0105] As a result, the information processing apparatus 100 according to the embodiment can provide an appropriate reason for suggestion that takes into consideration, for example, the user's request.

[0106] Furthermore, the generation unit 136 generates a reason based on the first score and the second score calculated using a model that outputs a score indicating whether the proposed target indicated by the proposed target information suits the proposed destination indicated by the proposed destination information included in the input information, and that is trained to output a higher score when the proposed target is evaluated as meeting the needs when selected.

[0107] As a result, the information processing device 100 according to the embodiment can provide an accurate and appropriate reason for suggestion that takes into account the user's request, for example, by using a learning model.

[0108] Furthermore, the generation unit 136 generates a reason based on the second score calculated in a state where some information corresponding to the request is missing.

[0109] As a result, the information processing apparatus 100 according to the embodiment can provide an appropriate reason for suggestion that takes into consideration, for example, the user's request.

[0110] [7. Hardware Configuration] The user terminal 10 and information processing device 100 according to the above-described embodiments are realized, for example, by a computer 1000 configured as shown in Fig. 13. Fig. 13 is a hardware configuration diagram showing an example of a computer that realizes the functions of the user terminal 10 and the information processing device 100. The computer 1000 has a CPU 1100, a RAM 1200, a ROM 1300, an HDD 1400, a communication interface (I / F) 1500, an input / output interface (I / F) 1600, and a media interface (I / F) 1700.

[0111] The CPU 1100 operates and controls each unit based on programs stored in the ROM 1300 or the HDD 1400. The ROM 1300 stores a boot program executed by the CPU 1100 when the computer 1000 starts up, programs that depend on the hardware of the computer 1000, and the like.

[0112] The HDD 1400 stores programs executed by the CPU 1100, data used by these programs, etc. The communication interface 1500 acquires data from other devices via a predetermined communication network and sends it to the CPU 1100, and transmits data generated by the CPU 1100 to other devices via the predetermined communication network.

[0113] The CPU 1100 controls output devices such as a display and a printer, and input devices such as a keyboard and a mouse, via the input / output interface 1600. The CPU 1100 acquires data from the input devices via the input / output interface 1600. The CPU 1100 also outputs generated data to the output devices via the input / output interface 1600.

[0114] Media interface 1700 reads a program or data stored in recording medium 1800 and provides it to CPU 1100 via RAM 1200. CPU 1100 loads the program or data from recording medium 1800 onto RAM 1200 via media interface 1700 and executes the loaded program. Recording medium 1800 is, for example, an optical recording medium such as a DVD (Digital Versatile Disc) or a PD (Phase Change Rewritable Disc), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory.

[0115] For example, when the computer 1000 functions as the user terminal 10 and the information processing device 100 according to the embodiment, the CPU 1100 of the computer 1000 executes programs loaded onto the RAM 1200 to realize the functions of the control units 14 and 130. The CPU 1100 of the computer 1000 reads and executes these programs from the recording medium 1800, but as another example, the CPU 1100 may obtain these programs from another device via a predetermined communication network.

[0116] [8. Other] Furthermore, among the processes described in the above embodiments, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically using known methods. In addition, the information including the processing procedures, specific names, various data, and parameters shown in the above documents and drawings can be changed as desired unless otherwise specified. For example, the various information shown in each drawing is not limited to the information shown in the drawings.

[0117] Furthermore, the components of each device shown in the figure are conceptual functional components and do not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure, and all or part of them can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc.

[0118] Furthermore, the above-described embodiments can be combined as appropriate within the scope of not causing any contradiction in the processing content.

[0119] Although some of the embodiments of the present application have been described in detail above with reference to the drawings, these are merely examples, and the present invention can be implemented in other forms that include the embodiments described in the Disclosure of the Invention section and that have been modified and improved in various ways based on the knowledge of those skilled in the art.

[0120] Furthermore, the above-mentioned "section, module, unit" can be read as "means" or "circuit," etc. For example, an acquisition unit can be read as an acquisition means or an acquisition circuit. [Explanation of symbols]

[0121] 1. Information Processing Systems 10 User terminal 11 Communications Department 12 Input section 13 Output section 14 Control Unit 100 Information processing device 110 Communications Department 120 Storage section 121 User information storage unit 122 Item information storage unit 123 Model Memory Unit 130 Control Unit 131 Acquisition Department 132 Learning Department 133 First Calculation Section 134 Second Calculation Unit 135 Estimation Department 136 Generation part 137 Provision Department 141 Receiving unit 142 Transmitter N Network

Claims

1. a first calculation unit that calculates a first score between the proposal destination information and the proposal target information; a second calculation unit that calculates a second score in a state where a portion of information of at least one of the proposal destination information and the proposal target information is missing; a generation unit that generates a reason for proposing the proposed target based on information that a comparison result between the first score and the second score satisfies a predetermined condition; An information processing device comprising:

2. The first calculation unit The first score is calculated using a model that outputs a score indicating whether the proposed target indicated by the proposed target information suits the proposed recipient indicated by the proposed recipient information included in the input information, and that is trained to output a higher score as the proposed recipient is evaluated as suiting the proposed target.

2. The information processing apparatus according to claim 1, wherein:

3. The second calculation unit The second score is calculated by inputting the proposal destination information and proposal target information of the state into the model.

3. The information processing apparatus according to claim 2, wherein:

4. The second calculation unit The second score is calculated by inputting a proposal target image including proposal destination information and proposal target information of the state into the model.

4. The information processing apparatus according to claim 3,

5. The generation unit generating the reason based on the type of the information that satisfies the predetermined condition; 2. The information processing apparatus according to claim 1, wherein:

6. The generation unit The reason is generated by requesting a generation AI to generate the reason taking into account the type.

6. The information processing apparatus according to claim 5,

7. The generation unit When a request is received from a proposal recipient, the reason for taking the request into consideration is generated in order to propose a proposal target that takes the request into consideration.

2. The information processing apparatus according to claim 1, wherein:

8. The generation unit The reason is generated based on the first score and the second score calculated using a model that outputs a score indicating whether the proposed target indicated by the proposed target information suits the proposed destination indicated by the proposed destination information included in the input information, and that is trained to output a score that is higher when the proposed target is selected and is evaluated as meeting the desire.

8. The information processing apparatus according to claim 7,

9. The generation unit The reason is generated based on the second score calculated in a state where some information corresponding to the request is missing.

9. The information processing apparatus according to claim 8,

10. 1. A computer-implemented information processing method, comprising: a first calculation step of calculating a first score between the proposal destination information and the proposal target information; a second calculation step of calculating a second score in a state where a portion of information of at least one of the proposal destination information and the proposal target information is missing; a generating step of generating a reason for proposing the proposed target based on information indicating that a comparison result between the first score and the second score satisfies a predetermined condition; An information processing method comprising:

11. a first calculation step of calculating a first score between the proposal destination information and the proposal target information; a second calculation step of calculating a second score in a state where a portion of information of at least one of the proposal destination information and the proposal target information is missing; a generation step of generating a reason for proposing the proposed target based on information indicating that a comparison result between the first score and the second score satisfies a predetermined condition; An information processing program characterized by causing a computer to execute the above.

Images