Information processing apparatus, information processing method, and information processing program

Abstract

To allow a user to grasp physical characteristics that suit predetermined clothing.SOLUTION: An information processing apparatus includes: an estimation unit which estimates physical characteristics that suit predetermined clothing, on the basis of a distributed representation space in which an image of clothing and physical information indicating physical characteristics of a target person are projected, the distributed representation space being generated based on evaluations on the image of the clothing made from a plurality of evaluators, the evaluations indicating whether the clothing suits the person or not; and a providing unit which provides information on the physical characteristics estimated by the estimation unit, to a user.SELECTED DRAWING: Figure 15

Description

[Technical field]

[0001] The present invention relates to an information processing device, an information processing method, and an information processing program. [Background technology]

[0002] A technique is known in the art that generates distributed representations of input information and determines the relationship between information based on the results of comparing the generated distributed representations. As an example of such a technique, a technique is provided that estimates hidden feature vectors representing positions in a map space for each of a user and a product based on product information representing the characteristics of the product, word information representing the relationship between words, and learning data representing products that have become the target of an action according to the user's preferences. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2021 / 044460 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the above-mentioned techniques do not necessarily enable a user to grasp physical features that suit a given outfit.

[0005] For example, the above-mentioned technology merely estimates the latent feature vector so that the distance between the user's latent feature vector and the product's latent feature vector reflects the user's preferences for the product, and does not necessarily enable the user to grasp physical features that would suit a given piece of clothing.

[0006] The present application has been made in consideration of the above, and has an object to enable a user to grasp physical features that go well with a given piece of clothing. [Means for solving the problem]

[0007] The information processing device of the present application is characterized in that it has a distributed representation space generated based on evaluations of an image of clothing from a plurality of evaluators indicating whether the clothing looks good on the user, an estimation unit that estimates physical features that look good on a specific piece of clothing based on the distributed representation space onto which the image and physical information indicating the physical features of a subject are projected, and a provision unit that provides a user with information regarding the physical features estimated by the estimation unit. Effect of the Invention

[0008] According to one aspect of the embodiment, an effect is achieved in that a user can grasp physical features that suit a predetermined outfit. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram showing an example of the configuration of an information processing system 1 according to an embodiment. [Diagram 2] FIG. 2 is a diagram (1) showing an example of information processing according to the embodiment. [Diagram 3] FIG. 3 is a diagram illustrating an example of a distributed representation space according to the embodiment. [Figure 4] FIG. 4 is a diagram (2) showing an example of information processing according to the embodiment. [Diagram 5] FIG. 5 is a diagram showing an example of the configuration of the information processing device 10 according to the embodiment. [Figure 6] FIG. 6 is a diagram showing an example of the evaluator information database 31 according to the embodiment. [Figure 7] FIG. 7 is a diagram showing an example of the user information database 32 according to the embodiment. [Figure 8] FIG. 8 is a diagram (1) showing an example of a screen of the user terminal 100. [Figure 9] FIG. 9 is a diagram (2) showing an example of the screen of the user terminal 100. [Figure 10] FIG. 10 is a flowchart (1) showing an example of a procedure of information processing according to the embodiment. [Figure 11] FIG. 11 is a flowchart (2) showing an example of the procedure of information processing according to the embodiment. [Figure 12] FIG. 12 is a flowchart (3) showing an example of the procedure of information processing according to the embodiment. [Figure 13] FIG. 13 is a diagram showing an example of the configuration of an information processing system 1A according to the embodiment. [Figure 14] FIG. 14 is a diagram (3) showing an example of information processing according to the embodiment. [Figure 15] FIG. 15 is a diagram showing an example of the configuration of an information processing device 10A according to an embodiment. [Figure 16] FIG. 16 is a diagram showing an example of the evaluator information database 31A according to the embodiment. [Figure 17] FIG. 17 is a diagram showing an example of a user information database 32A according to the embodiment. [Figure 18] FIG. 18 is a flowchart (4) showing an example of the procedure of information processing according to the embodiment. [Figure 19] FIG. 19 is a flowchart (5) showing an example of the procedure of information processing according to the embodiment. [Figure 20] FIG. 20 is a hardware configuration diagram showing an example of a computer that realizes the functions of the information processing device 10A. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, the information processing device, the information processing method, and the information processing program according to the present application will be described in detail with reference to the drawings. Note that the information processing device, the information processing method, and the information processing program according to the present application are not limited to the embodiments. In addition, the same parts in the following embodiments are given the same reference numerals, and duplicated explanations are omitted.

[0011] (First embodiment) [1. Configuration of information processing system] First, an information processing system 1 according to an embodiment will be described. FIG. 1 is a diagram showing an example of the configuration of the information processing system 1 according to an embodiment. As shown in FIG. 1, the information processing system 1 includes an information processing device 10, a user terminal 100, and an evaluator terminal 200. The information processing device 10, the user terminal 100, and the evaluator terminal 200 are connected to each other via a predetermined communication network (network N) so as to be able to communicate with each other by wire or wirelessly. Note that the information processing system 1 shown in FIG. 1 may include a plurality of information processing devices 10, a plurality of user terminals 100, and a plurality of evaluator terminals 200.

[0012] The information processing device 10 is an information processing device that receives evaluations from a plurality of evaluators (annotators) of an image of a wearer wearing a combination of a plurality of clothes (also called fashion items, including footwear (also called shoes), headwear (caps, hats, etc.), and ornaments (also called accessories)), indicating whether the combination suits the wearer or not, generates a distributed representation space into which the image and evaluator information indicating the evaluators are projected based on the received evaluations, and realizes information processing using the generated distributed representation space, and is realized, for example, by a server device, a cloud system, etc.

[0013] For example, the information processing device 10 provides an electronic commerce service that provides (searches, sells, etc.) clothing. The information processing device 10 also provides a coordination service that accepts posts of content (images, videos, articles, etc.) showing clothing coordination from users and provides (searches, distributes, etc.) to other users.

[0014] The information processing device 10 may have a function as a web server that provides a website related to the service. The information processing device 10 may also be a device that distributes information to be displayed on applications related to various services installed in the user terminal 100 to the information processing device 10. The information processing device 10 may also be a device that distributes the application data itself.

[0015] Furthermore, the information processing device 10 may function as a distribution device that distributes control information to the user terminal 100. Here, the control information is described in, for example, a script language such as JavaScript (registered trademark) or a style sheet language such as CSS (Cascading Style Sheets). Note that the application itself distributed from the information processing device 10 may be regarded as the control information.

[0016] The user terminal 100 is an information processing device used by a user. The user terminal 100 is realized, for example, by a smartphone, a tablet terminal, a notebook PC (Personal Computer), a desktop PC, a mobile phone, a PDA (Personal Digital Assistant), or the like. The user terminal 100 displays information distributed by the information processing device 10 or a server device that provides a predetermined service, using a web browser or an application. In the example shown in FIG. 2, the user terminal 100 is a smartphone.

[0017] The evaluator terminal 200 is an information processing device used by an evaluator who evaluates an image of a wearer wearing a combination of multiple clothes. The evaluator terminal 200 is realized by, for example, a smartphone, a tablet terminal, a notebook PC, a desktop PC, a mobile phone, a PDA, or the like. The evaluator terminal 200 displays information distributed by the information processing device 10 or a server device that provides a predetermined service, or the like, using a web browser or an application. In the example shown in FIG. 2, the evaluator terminal 200 is a smartphone.

[0018] [2. An example of information processing] Next, an example of information processing implemented by the information processing device according to the present embodiment will be described with reference to FIGS.

[0019] [2-1. Regarding the first information processing] Below, a first information processing implemented by an information processing device etc. according to this embodiment will be described with reference to FIG. 2. FIG. 2 is a diagram (1) showing an example of information processing according to the embodiment. In the following description, it is assumed that the user terminal 100 is used by a user (user U1) identified by a user ID "UID#1". In addition, in the following description, the user terminal 100 may be regarded as the same as the user U1. In other words, in the following, the user U1 may also be read as the user terminal 100.

[0020] In the following description, the evaluator terminals 200-1 to 200-N (N is an arbitrary natural number) are described according to the evaluator using the evaluator terminal 200. For example, the evaluator terminal 200-1 is the evaluator terminal 200 used by the evaluator (evaluator A1) identified by the evaluator ID "AID#1". In the following description, the evaluator terminals 200-1 to 200-N are described as the evaluator terminals 200 when there is no particular distinction between them. In the following description, the evaluator terminal 200 may be regarded as the same as the evaluator. That is, in the following description, the evaluator may be read as the evaluator terminal 200.

[0021] First, the information processing device 10 receives an evaluator's evaluation of an image of a wearer wearing a combination (coordination) of multiple clothes from the evaluator terminal 200 (step Sa1). For example, the information processing device 10 receives an evaluation indicating whether the coordinated outfit of the wearer suits the wearer. As a specific example, the information processing device 10 presents a pair of images P1 and P2 to the evaluator, compares the two images, and receives an evaluation indicating which of the images the coordinated outfit suits the wearer and which of the images the coordinated outfit does not suit the wearer.

[0022] To give a more specific example, the information processing device 10 receives from evaluator A1 an evaluation that coordination #1 shown in image P1 suits the wearer #1 shown in image P1, and an evaluation that coordination #2 shown in image P2 does not suit the wearer #2 shown in image P2 (i.e., an evaluation that "it suits" for image P1 and an evaluation that "it does not suit" for image P2). In addition, if the outfit looks good in both examples, the evaluation that the outfit #1 shown in image P1 looks good on the wearer #1 shown in image P1 may be compared with the evaluation that the outfit #2 shown in image P2 looks good on the wearer #2 shown in image P2, and a judgment may be made as to which evaluation is relatively higher or lower. Conversely, if the outfit does not look good in either example, the evaluation that the outfit #1 shown in image P1 does not look good on the wearer #1 shown in image P1 may be compared with the evaluation that the outfit #2 shown in image P2 does not look good on the wearer #2 shown in image P2, and a judgment may be made as to which evaluation is relatively higher or lower. In the following examples, the case where the outfit is judged to look good relatively will be referred to simply as looking good, and the case where the outfit is judged to not look good relatively will be referred to simply as not looking good. Furthermore, the information processing device 10 receives from the evaluator A2 an evaluation that the outfit #1 shown in the image P1 does not suit the wearer #1 shown in the image P1, and an evaluation that the outfit #2 shown in the image P2 suits the wearer #2 shown in the image P2 (i.e., an evaluation of "does not suit" the image P1 and an evaluation of "suits" the image P2). Similarly, the information processing device 10 receives evaluations on the pair of images P1 and P2 from other evaluators. Furthermore, the information processing device 10 receives evaluations on other pairs, such as the pair of images P3 and P4, the pair of images P5 and P6, the pair of images P1 and P3, the pair of images P2 and P4, ... from the evaluators.

[0023] Next, the information processing device 10 generates a distributed representation space in which the image and the evaluator information indicating the evaluator are projected based on the evaluator's evaluation of the image (step Sa2). For example, the information processing device 10 generates the distributed representation space using a technique of VSE (Visual-Semantic Embedding). As a specific example, the information processing device 10 generates a distributed representation space F1 in which the image P1 is projected closer to the evaluator information D1 indicating the evaluator A1 who evaluated the image P1 as "suitable" than to the evaluator information D2 indicating the evaluator A2 who evaluated the image P1 as "unsuitable". In addition, the information processing device 10 generates a distributed representation space F1 in which the image P2 is projected closer to the evaluator information D2 indicating the evaluator A2 who evaluated the image P2 as "suitable" than to the evaluator information D1 indicating the evaluator A1 who evaluated the image P2 as "unsuitable".

[0024] The information processing device 10 may generate a distributed representation space in which various information related to the image and evaluator information indicating various information related to the evaluator are projected. Here, an example of the distributed representation space generated by the information processing device 10 will be described with reference to Fig. 3. Fig. 3 is a diagram showing an example of the distributed representation space according to the embodiment.

[0025] For example, it is estimated that the clothes (items) included in the coordination shown by the image contribute to whether or not the evaluator feels that the image suits the evaluator. Therefore, the information processing device 10 generates a distributed representation space F2 onto which an image of the clothes included in the coordination shown by the image is further projected. As a specific example, the information processing device 10 generates a distributed representation space F2 onto which an image of the clothes I1 included in the coordination shown by the image P1 is projected closer to the evaluator information D1 indicating the evaluator A1 who evaluated the image P1 as "suitable" than to the evaluator information D2 indicating the evaluator A2 who evaluated the image P1 as "unsuitable". In addition, the information processing device 10 generates a distributed representation space F2 onto which an image of the clothes I2 included in the coordination shown by the image P2 is projected closer to the evaluator information D2 indicating the evaluator A2 who evaluated the image P2 as "suitable" than to the evaluator information D1 indicating the evaluator A1 who evaluated the image P2 as "unsuitable".

[0026] Also, for example, the information processing device 10 generates a distributed representation space F3 in which information linked to the coordination shown by the image is further projected. As a specific example, the information processing device 10 generates a distributed representation space F3 in which a category indicating the purpose of the coordination, the situation in which it is worn, and the like is further projected. As a more specific example, the information processing device 10 generates a distributed representation space F3 in which a category T1 (casual) of the coordination shown by the image P1 is projected closer to the evaluator information D1 indicating the evaluator A1 who evaluated the image P1 as "suitable" than to the evaluator information D2 indicating the evaluator A2 who evaluated the image P1 as "unsuitable". Also, the information processing device 10 generates a distributed representation space F3 in which a category T2 (date) of the image P2 is projected closer to the evaluator information D2 indicating the evaluator A2 who evaluated the image P2 as "suitable" than to the evaluator information D1 indicating the evaluator A1 who evaluated the image P2 as "unsuitable". In addition, when the information processing device 10 generates a distributed representation space onto which an image of clothing included in the coordination shown by the image is further projected, the information processing device 10 may generate a distributed representation space onto which information linked to the clothing is further projected.

[0027] Also, for example, the information processing device 10 generates a distributed representation space F4 in which evaluator information indicating attribute information of the evaluators is projected. Here, it is assumed that the attribute information of evaluator A1 is "late 20s, living in Tokyo" and the attribute information of evaluator A2 is "early 20s, living in Osaka." In such a case, the information processing device 10 generates a distributed representation space F4 in which the image P1 is projected closer to the attribute information of evaluator A1 who evaluated the image P1 as "suitable" (for example, attribute information B1 (late 20s, living in Tokyo), attribute information B2 (late 20s), and attribute information B3 (living in Tokyo)) who evaluated the image P1 as "not suitable" than the attribute information of evaluator A2 who evaluated the image P1 as "not suitable." In addition, the information processing device 10 generates a distributed representation space F4 in which the image P2 is projected closer to the attribute information of the evaluator A2 who evaluated the image P2 as “suitable” (for example, attribute information B4 (early 20s, living in Osaka), attribute information B5 (living in Osaka), and attribute information B6 (early 20s)) than to the attribute information of the evaluator A1 who evaluated the image P2 as “not suitable”.

[0028] Also, for example, the information processing device 10 generates a distributed representation space F5 in which evaluator information indicating the evaluator's response to a questionnaire such as "Are you confident in fashion?" or "What is your favorite city?" is projected. Here, it is assumed that the evaluator A1 responds to the above questionnaire with "not confident in fashion" and "likes Shibuya", and the evaluator A2 responds with "confident in fashion" and "likes Harajuku". In such a case, the information processing device 10 generates a distributed representation space F5 in which the image P1 is projected closer to the response of the evaluator A1 who evaluated the image P1 as "suitable" (for example, response C1 (not confident in fashion, likes Shibuya), response C2 (not confident in fashion), and response C3 (likes Shibuya)) than the response of the evaluator A2 who evaluated the image P1 as "not suitable". In addition, the information processing device 10 generates a distributed representation space F5 in which image P2 is projected closer to the answers of evaluator A2 who evaluated image P2 as “suitable” (for example, answer C4 (confident in fashion, likes Harajuku), answer C5 (confident in fashion), and answer C6 (likes Harajuku)) than to the answers of evaluator A1 who evaluated image P2 as “not suitable”.

[0029] Note that the distributed representation space generated by the information processing device 10 is not limited to the above example. For example, the information processing device 10 may generate a distributed representation space in which an image, an image of clothing included in a coordination, a coordination category, attribute information of an evaluator, and an answer of an evaluator to a questionnaire are projected.

[0030] Furthermore, the information included in the distributed representation space generated by the information processing device 10 is not limited to the above example, and any information may be projected. For example, the information processing device 10 may generate a distributed representation space in which information indicating the source of the clothes (for example, a brand, a shop selling the clothes, etc.) is projected. As a specific example, the information processing device 10 generates a distributed representation space in which information indicating the source of the clothes included in the coordination shown by the image P1 is projected closer to the evaluator information D1 indicating the evaluator A1 who evaluated the image P1 as "suitable" than to the evaluator information D2 indicating the evaluator A2 who evaluated the image P1 as "unsuitable". Furthermore, the information processing device 10 generates a distributed representation space in which information indicating the source of the clothes included in the coordination shown by the image P2 is projected closer to the evaluator information D2 indicating the evaluator A2 who evaluated the image P2 as "suitable" than to the evaluator information D1 indicating the evaluator A1 who evaluated the image P2 as "unsuitable". Furthermore, for example, the information processing device 10 may generate a distributed representation space onto which the wearer's physical information (e.g., features related to skeleton and physique) is projected, and the method of non-patent document 1 (ViBE: Dressing for Diverse Body Shapes. Wei-Lin Hsiao, Kristen Grauman, Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020, pp. 11059-11069.) may be used when extracting physical features.

[0031] Furthermore, for example, the information processing device 10 may generate a distributed representation space projecting information indicating various attribute information of the evaluator (e.g., occupation, height, weight, place of residence, personal color, income, rent), answers to questionnaire items of the evaluator (e.g., favorite providers, amount of money spent on fashion, media that one looks at for fashion inspiration (magazines that one reads often, web services that one frequently uses, etc.), passions (hobbies, etc.), dreams for the future, people that one uses as inspiration for fashion (celebrities, influencers, etc.), desired image, items that one wants to incorporate, information on SNS of people one likes (admires), body shape concerns, fashion concerns, etc.), and information linked to the coordination shown in the image (e.g., item type, provider, usage scene, color, season, purpose of wearing, selling point, material, size, price range, etc.).

[0032] Returning to FIG. 2, the explanation will be continued. Next, in response to a request for providing a coordinated outfit from the user terminal 100 via an electronic commerce service or a coordinated outfit service, the information processing device 10 estimates an outfit that suits the user U1 (in other words, an outfit that is estimated to be highly rated by the user or that the user also estimates to look good on the user) based on the generated distributed representation space and information about the user U1 (step Sa3). Here, it is assumed that the attribute information of the user U1 is "late 20s." In such a case, the information processing device 10 estimates that the outfit shown by the image projected within a predetermined range from the attribute information B2 in the distributed representation space F4 is an outfit that suits the user U1.

[0033] The information processing device 10 may estimate a position when the information about user U1 is projected into the distributed representation space based on the information projected into the distributed representation space and the information about user U1, and may estimate an outfit that suits user U1 based on the estimated position. For example, if the attribute information of user U1 is "late 20s, living in Osaka", the information processing device 10 may estimate that the midpoint between attribute information B2 and B5 in the distributed representation space F4 is the position when the information about user U1 is projected in the distributed representation space F4, and may estimate that the outfit shown by the image projected within a predetermined range from that position is the outfit that suits user U1.

[0034] In the example of FIG. 2, it is assumed that the information processing device 10 estimates that the coordination #1 shown in the image P1, the coordination #3 shown in the image P3, and the coordination #4 shown in the image P4 are coordinations that suit the user U1. In such a case, the information processing device 10 provides information on the coordination #1, #3, and #4 to the user terminal 100 (step Sa4). For example, the information processing device 10 estimates that the closer the information on the coordination shown in the image is to the attribute information B2 in the distributed representation space F4, the higher the degree of suitability for the user U1, and provides the information with a higher priority. As a specific example, the information processing device 10 provides information on the coordination #1, #3, and #4 and information on the clothes included in the coordination #1, #3, and #4 in the electronic commerce service. In addition, the information processing device 10 provides information on the coordination #1, #3, and #4 and information on the clothes included in the coordination #1, #3, and #4 in the coordination service. At this time, the information processing device 10 may provide information such that the higher the priority of the information is displayed in a more prominent position or at the top of the ranking.

[0035] The information processing device 10 may provide information on coordination among the coordinations #1, #3, and #4 that corresponds to information on the user U1 (attribute information, physical information (height, weight, body type, bone structure, hairstyle, hair color, skin color, eye color, etc.)) and information related to the coordination (for example, the current season or current fashion). For example, if the user U1 is male, the information processing device 10 provides information on coordination for men among the coordinations #1, #3, and #4. For example, if the user U1 is male, the information processing device 10 may provide information on coordination for men among the coordinations #1, #3, and #4 that are worn by men or that include clothing for men. For example, if the current season is autumn, the information processing device 10 provides information on coordination for autumn or winter among the coordinations #1, #3, and #4.

[0036] Furthermore, the information processing device 10 may provide information on coordination according to conditions specified by the user U1. For example, when the user U1 specifies the category "casual" as a condition, the information processing device 10 provides, among the coordinations #1, #3, and #4, the coordination indicated by the image projected at a position closer to the category T1 in the distributed representation space (for example, the distributed representation space F3), with a higher priority.

[0037] The information processing device 10 may provide coordinated information using a model that has learned the clothing preferences of the user U1. For example, the information processing device 10 trains a model so that when information (e.g., an image) related to clothing is input, the higher the degree to which the user U1 likes that clothing, the higher the score that is output. The information processing device 10 then inputs information related to outfits #1, #3, and #4 into the model, and provides the information on outfits #1, #3, and #4 to the user U1 in descending order of the output scores.

[0038] Next, the information processing device 10 acquires from the user terminal 100 feedback indicating whether the user U1 was positive about the provided information (step Sa5). For example, the information processing device 10 acquires information (feedback) indicating whether the user U1 purchased clothes included in the coordinated outfits #1, #3, and #4 in the e-commerce service, or whether he / she applied for rental. The information processing device 10 also acquires information (feedback) indicating whether the user U1 registered the coordinated outfits #1, #3, and #4 as favorites in the coordinate service. The information indicating whether the user U1 was positive about the provided information is not limited to the above examples, and may be whether the user U1 viewed the coordinated outfits #1, #3, and #4 in the e-commerce service or the coordinate service, or whether the user U1 viewed clothes included in the coordinated outfits #1, #3, and #4 in the e-commerce service or the coordinate service.

[0039] Next, the information processing device 10 updates the distributed representation space based on the feedback from the user U1 (step Sa6). For example, if the user U1 is positive about the coordination #1 (for example, if the user U1 purchases clothes included in the coordination #1 in the electronic commerce service or registers the coordination #1 as a favorite in the coordination service), the information processing device 10 updates the attribute information B2 and the image P1 in the distributed representation space F4 to move them closer to each other by a predetermined distance. On the other hand, if the user U1 is negative about the coordination #3 (for example, if the user U1 does not view the information about the coordination #3 presented in the electronic commerce service or the coordination service), the information processing device 10 updates the attribute information B2 and the image P3 in the distributed representation space F4 to move them apart by a predetermined distance. Note that the feedback from the user U1 can also be regarded as an evaluation of whether the image showing the coordination looks good on the user, so evaluator information indicating the evaluator (user U1) who evaluated (gave feedback) the image may be further projected into the distributed representation space.

[0040] As described above, the information processing device 10 according to the embodiment generates a distributed representation space into which evaluations of whether an image showing an outfit looks good and evaluator information indicating the evaluator who evaluated the image are projected.

[0041] Here, in the past, whether or not a coordinated outfit "looks good" on a wearer differed greatly depending on each person's subjective opinion, and it was difficult to quantify. Therefore, according to the information processing device 10 according to the embodiment, by constructing a space that can measure the closeness between an image showing the coordinated outfit and the evaluator's personality, it is possible to quantify "looking good", and therefore it is possible to grasp whether the clothes look good on the wearer from the subjective opinion of each evaluator.

[0042] Furthermore, the information processing device 10 according to the embodiment uses the generated distributed representation space to provide information on outfits that are estimated to suit the user based on evaluations by evaluators similar to the user. This allows the information processing device 10 according to the embodiment to have the effect of allowing the user to grasp clothes that suit a specific target (such as the user himself / herself).

[0043] [2-2. Regarding the second information processing] Next, a second information processing realized by the information processing device according to the embodiment will be described with reference to Fig. 4. Fig. 4 is a diagram (2) showing an example of the information processing according to the embodiment.

[0044] First, the information processing device 10 receives an evaluation of an image of a wearer wearing a combination (coordination) of multiple clothes from the evaluator terminal 200 (step Sb1). For example, the information processing device 10 receives an evaluation indicating whether or not the coordination #1 shown in the image P1 suits the wearer #1 shown in the image P1.

[0045] Next, the information processing device 10 generates a convolutional neural network (CNN) for generating distributed representations of evaluators, images, etc., based on the evaluators' ratings of the images (step Sb2). For example, the information processing device 10 generates the CNN using learning data including images, the evaluators' ratings of the images, and evaluator information related to the evaluators. As a specific example, when an image is input, the information processing device 10 corrects the connection coefficients of the CNN so as to output a distributed representation similar to the distributed representation of the evaluator who evaluated the image as "suitable" and to output a distributed representation not similar to the distributed representation of the evaluator who evaluated the image as "unsuitable".

[0046] Here, when the information processing device 10 repeatedly performs CNN learning based on the evaluations by each evaluator, the CNN estimates the range on which each evaluator evaluated whether the image looked good or not, and generates distributed representations of each evaluator and image according to the estimation result. For example, when generating a distributed representation from an image, the CNN divides the image into multiple regions and generates a vector (hereinafter, may be referred to as a "region vector") indicating the characteristics of each divided region. Then, the CNN generates a distributed representation indicating the characteristics of the entire image by integrating the multiple region vectors. Note that the CNN repeatedly executes the same process for regions and features of various sizes to generate a final distributed representation. Also, the entire image may be regarded as one region. Also, a distributed representation may be generated based on a single region instead of multiple regions. That is, if necessary, a single feature vector considering the entire image may be calculated instead of generating a distributed representation based on a partial region on the image. Also, the definition of the region may be arbitrarily determined based on the application to be used. For example, a feature vector obtained by extracting features using a Fully CNN is considered to correspond to a region on one image in each dimension, and whether or not to use the region vector may be determined based on the content in the image space corresponding to the region. Specifically, a human body detector or the like may be used to identify multiple regions in which a person exists on an image, and the region vectors may be integrated by calculating an average pooling of these region vectors. As another example, a region may be identified by performing semantic region division on an image. Specifically, for example, multiple semantic regions can be obtained by identifying which part includes a top or bottom in a pixel in which a person exists captured on an image. A distributed representation indicating the characteristics of the entire image that is relatively robust to posture changes of the target subject may be calculated by calculating region vectors corresponding to these regions and concatenating (concatenating) the multiple region vectors in a unique order.In this case, if there is a semantic region that does not exist (for example, the shoe region does not exist when the subject's feet are not imaged), the relevant region vector may be substituted with any feature vector. For example, the average vector of the shoe region vectors in the entire training data may be used, or a feature vector filled with zeros may be used.

[0047] When such CNN learning is performed, the information processing device 10 fixes the position of each evaluator, and performs CNN learning so as to output a distributed representation similar to the distributed representation of the evaluator who evaluated each image as suitable (for example, a distributed representation based on evaluator information indicating attribute information, etc.) and output a distributed representation not similar to the distributed representation of the evaluator information of the evaluator who evaluated the image as unsuitable. When such learning is performed, the CNN adjusts the connection coefficients, etc., so as to generate a region vector relatively similar to the distributed representation of the evaluator information of the evaluator who evaluated the image as suitable and generate a region vector relatively not similar to the distributed representation of the evaluator information of the evaluator who evaluated the image as unsuitable. When such a learning process is repeated, the CNN generates a region vector for a certain region that is similar to the distributed representation of each evaluator who is likely to have evaluated the region as suitable and is not similar to the distributed representation of each evaluator who is likely to have evaluated the region as unsuitable.

[0048] Therefore, the information processing device 10 uses the region vectors generated by such a trained CNN to estimate the regions that each evaluator judged to look good, and provides information indicating the estimated regions, such as a heat map.

[0049] For example, when providing an image showing a coordinated outfit to a user U1 in an electronic commerce service or a coordinate coordination service, the information processing device 10 determines the degree to which each of the clothes shown in each region of the image suits the user U1 based on the generated CNN and information about the user U1 (step Sb3). In the example of FIG. 4, it is assumed that an image P1 is provided to the user U1. In such a case, the information processing device 10 identifies an evaluator whose distributed representation is similar to the distributed representation of the user U1 (for example, a distributed representation based on attribute information, etc.). Then, the information processing device 10 determines that the clothes shown in the region of the image P1 whose region vector is more similar to the distributed representation of the identified evaluator are more suitable for the user U1, and determines that the clothes shown in the region whose region vector is less similar to the distributed representation of the evaluator are less suitable for the user U1.

[0050] Note that what the information processing device 10 generates for use in the above-described processing is not limited to CNN, and anything may be generated as long as it is possible to acquire a distributed representation (embedded representation).

[0051] Next, the information processing device 10 outputs the image P1 to the user terminal 100 in a mode according to the degree determined in step Sb3 (step Sb4). For example, the information processing device 10 outputs an image showing the degree to which the clothes indicated by each region of the image P1 suit the user U1 in a heat map. As a specific example, if the region vector of the region indicating the bottoms of the image P1 is similar to the distributed representation of the evaluator specified in step Sb3 and the region vector of the region indicating the tops of the image P1 is not similar, the information processing device 10 outputs an image P1-1 in which the region indicating the bottoms of the image P1 is displayed in dark red and the region indicating the tops is displayed in dark blue. In addition, if the region vector of the region indicating the tops of the image P1 is similar to the distributed representation of the evaluator specified in step Sb3 and the region vector of the region indicating the bottoms of the image P1 is not similar, the information processing device 10 outputs an image P1-2 in which the region indicating the tops of the image P1 is displayed in dark red and the region indicating the bottoms is displayed in dark blue.

[0052] When outputting an image, the information processing device 10 may output the image together with information according to the degree to which the clothes in each region suit the user U1. For example, when outputting image P1-1, the information processing device 10 outputs image P1-1 together with information suggesting purchase of the clothes included in the region displayed in red, information suggesting avoidance of wearing the clothes included in the region displayed in blue, information on other coordinations that utilize the clothes included in the region displayed in red, etc.

[0053] As described above, the information processing device 10 according to the embodiment can visualize areas showing clothes that the evaluator has evaluated as looking good and clothes that the evaluator has evaluated as not looking good among the coordinated outfits, so that it is possible to grasp which clothes among a combination of multiple clothes look good on the wearer. Furthermore, the information processing device 10 according to the embodiment can allow the user to grasp which clothes among the coordinated outfits the evaluator with attributes similar to the user has evaluated as looking good, so that the user can obtain a guideline for clothes to wear.

[0054] [3. Other processing examples] The above-described process is merely an example, and the information processing device 10 may perform various processes using various information. In this regard, examples are given below.

[0055] [3-1. Predicting outfits based on user information] In the example of Fig. 2, the information processing device 10 may estimate an outfit that suits the user U1 based on the distributed representation space and the category selected by the user U1. For example, when receiving information indicating that the user U1 has selected the category "casual" together with a request for providing an outfit from the user terminal 100, the information processing device 10 estimates that an outfit shown by an image projected within a predetermined range from the attribute information B2 in the distributed representation space F4 and an image projected within a predetermined range from the category T1 in the distributed representation space F3 is an outfit that suits the user U1 in the category "casual", and provides information on the outfit that corresponds to information on the user U1 and information related to the outfit to the user terminal 100.

[0056] Furthermore, the information processing device 10 may estimate an outfit that suits the user U1 based on the distributed representation space and the user U1's answer to a questionnaire. For example, when the user U1 answers "I'm not confident in my fashion sense" to a questionnaire such as "Are you confident in your fashion sense?", the information processing device 10 estimates that an outfit shown by an image projected within a predetermined range from the answer C2 in the distributed representation space F5 is an outfit that suits the user U1, and provides the user terminal 100 with information about the outfit that corresponds to information about the user U1 and information related to the outfit.

[0057] The information processing device 10 may suggest a coordination based on information about the user U1. For example, when the user U1 answers "I'm not confident in fashion" to a questionnaire such as "Are you confident in fashion?", the information processing device 10 suggests a coordination shown by an image projected within a predetermined range from a predetermined position (e.g., a middle position) between the answers C2 and C5 in the distributed representation space F5, together with a comment such as "Why not try it to gain confidence in fashion?". That is, the information processing device 10 may suggest a coordination based on information about the future of the user U1.

[0058] [3-2. Estimating coordination based on information about the subject] In the example of FIG. 2, the information processing device 10 may estimate an outfit that the subject evaluates as looking good on the user U1 based on the distributed representation space and information on a subject having a predetermined relationship with the user U1 (for example, a person the user U1 will meet). For example, the information processing device 10 may estimate an outfit that the subject evaluates as looking good on the user U1 based on the distributed representation space and the attributes of the subject. To give a specific example, when the attribute information of the subject is "early 20s", the information processing device 10 estimates that the outfit shown by the image projected within a predetermined range from the attribute information B6 in the distributed representation space F4 is the outfit that the subject evaluates as looking good on, and provides the user terminal 100 with information on the outfit that corresponds to the information on the user U1 and information related to the outfit.

[0059] Furthermore, for example, the information processing device 10 may estimate an outfit that the subject evaluates as looking good on the basis of the distributed representation space and the subject's answer to a questionnaire. As a specific example, when the subject answers "I'm confident in fashion" to a questionnaire such as "Are you confident in fashion?", the information processing device 10 estimates that the outfit shown by the image projected within a predetermined range from the answer C5 in the distributed representation space F5 is the outfit that the subject evaluates as looking good on the subject, and provides the user terminal 100 with information on the outfit that corresponds to information on the user U1 and information related to the outfit.

[0060] Furthermore, the information processing device 10 may estimate a position when the information about the target person is projected into the distributed representation space based on the information projected into the distributed representation space and the information about the target person, and estimate an outfit that the target person evaluates as looking good on the user based on the estimated position. For example, if the attribute information of the target person is "early 20s, living in Tokyo", the information processing device 10 estimates that the midpoint between the attribute information B3 and B6 in the distributed representation space F4 is the position when the information about the target person is projected in the distributed representation space F4. Then, the information processing device 10 estimates that the outfit shown by the image projected within a predetermined range from the estimated position is the outfit that the target evaluates as looking good on the user, and provides information about the outfit that corresponds to the information about the user and information related to the outfit to the user terminal 100.

[0061] This allows, for example, an estimate of the outfit that the subject will evaluate as suiting the user, enabling the user to know what to wear in order to make a good impression on people they will meet.

[0062] In addition, when the subject provides information on a coordination that the user U1 evaluates as looking good on the user U1, the information processing device 10 may obtain feedback from the subject on the coordination. For example, the information processing device 10 obtains information (feedback) indicating whether the subject was positive or not about the user U1 wearing the coordination indicated by the provided information from the user terminal 100, a terminal device used by the subject, or the like. Then, when the subject is positive about the user U1 wearing the coordination indicated by the provided information, the information processing device 10 updates the information corresponding to the subject (for example, attribute information, answers to a questionnaire, etc.) and the image showing the coordination in the distributed representation space so as to move them closer to each other by a predetermined distance. On the other hand, when the subject is negative about the user U1 wearing the coordination indicated by the provided information, the information processing device 10 updates the information corresponding to the subject and the image showing the coordination in the distributed representation space so as to move them apart by a predetermined distance. In addition, since the feedback from the subject can also be regarded as an evaluation of whether the image showing the coordination looks good on the user U1, evaluator information indicating the evaluator (subject) who performed the evaluation (feedback) on the image may be further projected into the distributed representation space.

[0063] The information processing device 10 may estimate an outfit that the user evaluates as looking good on the user and that the target evaluates as looking good on the user, based on the distributed representation space, information about the user, and information about a target having a predetermined relationship with the user. For example, the information processing device 10 estimates that an outfit shown in an image projected from information about the user and from information about the target in the distributed representation space is an outfit that both the user and the target evaluate as looking good, and provides the user terminal 100 with information about the outfit, which corresponds to information about the user and information related to the outfit, among the outfits. The information processing device 10 also estimates that an outfit shown in an image projected from information indicating the user's response to a questionnaire in the distributed representation space and from information indicating the target's response to the questionnaire in the distributed representation space is an outfit that both the user and the target evaluate as looking good, and provides the user terminal 100 with information about the outfit, which corresponds to information about the user and information related to the outfit, among the outfits.

[0064] Furthermore, the information processing device 10 may estimate a position when information based on both the user and the target is projected into the distributed representation space based on the information projected into the distributed representation space, the information on the user, and the information on the target, and estimate an outfit that the user evaluates as looking good on the user and that the target evaluates as looking good on the user based on the estimated position. For example, when the attribute information of the user is "late 20s" and the attribute information of the target is "early 20s", the information processing device 10 estimates that the intermediate position between the attribute information B2 and B5 in the distributed representation space F4 is the position when information corresponding to both the user and the target is projected in the distributed representation space F4, and estimates that the outfit shown by the image projected within a predetermined range from the position is the outfit that both the user and the target evaluate as looking good. Then, the information processing device 10 provides the user terminal 100 with information on the outfit that corresponds to the information on the user and the information related to the outfit, among the estimated outfits.

[0065] [3-3. Estimating the best outfit for the subject] In the example of FIG. 2, the information processing device 10 may estimate an outfit that the target person evaluates as looking good on the user U1 based on the distributed representation space and information about a target person having a predetermined relationship with the user U1 (for example, a person to whom the user U1 gives a present). For example, the information processing device 10 may estimate an outfit that the target person evaluates as looking good on the target person based on the distributed representation space and the attributes of the target person. To give a specific example, when the attribute information of the target person is "early 20s", the information processing device 10 estimates that the outfit shown by the image projected within a predetermined range from the attribute information B6 in the distributed representation space F4 is an outfit that the target person evaluates as looking good on, and provides the user terminal 100 with information about the outfit (attribute information, physical information (height, weight, figure, bone structure, hairstyle, hair color, skin color, eye color, etc.)) and information related to the outfit (for example, if the target person is a woman, an outfit for women) corresponding to the outfit.

[0066] Furthermore, for example, the information processing device 10 may estimate an outfit that the subject evaluates as looking good on the subject based on the distributed representation space and the subject's answer to a questionnaire. As a specific example, when the subject answers "I'm confident in fashion" to a questionnaire such as "Are you confident in fashion?", the information processing device 10 estimates that the outfit shown by the image projected within a predetermined range from the answer C5 in the distributed representation space F5 is the outfit that the subject evaluates as looking good on the subject, and provides the user terminal 100 with information on the outfit that corresponds to information on the subject and information related to the outfit.

[0067] The information processing device 10 may also estimate a position when the information about the subject is projected into the distributed representation space based on the information projected into the distributed representation space and the information about the subject, and estimate an outfit that the subject evaluates as looking good on the subject based on the estimated position. For example, if the attribute information of the subject is "early 20s, living in Tokyo", the information processing device 10 estimates that the midpoint between the attribute information B3 and B6 in the distributed representation space F4 is the position when the information about the subject is projected in the distributed representation space F4. The information processing device 10 then estimates that the outfit shown by the image projected within a predetermined range from the estimated position is the outfit that the subject evaluates as looking good on the subject, and provides the user terminal 100 with information about the outfit that corresponds to the information about the subject and information related to the outfit.

[0068] In addition, when the subject provides information on a coordination that the subject evaluates as being suitable for the subject, the information processing device 10 may obtain feedback from the subject on the coordination. For example, the information processing device 10 obtains information (feedback) indicating whether the subject was positive or not when the user U1 presented the subject with clothes included in the coordination indicated by the provided information from the user terminal 100, a terminal device used by the subject, or the like. Then, when the user U1 presented the subject with clothes included in the coordination indicated by the information provided to the user U1, if the subject was positive, the information processing device 10 updates the information corresponding to the subject and the image showing the coordination in the distributed representation space so as to move them closer to each other by a predetermined distance. On the other hand, when the user U1 presented the subject with clothes included in the coordination indicated by the information provided to the user U1, if the subject was negative, the information processing device 10 updates the information corresponding to the subject and the image showing the coordination in the distributed representation space so as to move them apart by a predetermined distance. In addition, since feedback from the subject can also be interpreted as an evaluation of whether or not an image showing an outfit looks good on the subject, evaluator information indicating the evaluator (subject) who provided the evaluation (feedback) on the image may be further projected into the distributed representation space.

[0069] The information processing device 10 may estimate an outfit that the user evaluates as looking good on the target and that the target evaluates as looking good on the user, based on the distributed representation space, information about the user, and information about a target having a predetermined relationship with the user. For example, the information processing device 10 estimates that an outfit shown in an image projected from information about the user and from information about the target in the distributed representation space is an outfit that both the user and the target evaluate as looking good, and provides the user terminal 100 with information about the outfit corresponding to information about the target and information related to the outfit, among the outfits. The information processing device 10 also estimates that an outfit shown in an image projected from information indicating the user's response to a questionnaire in the distributed representation space and from information indicating the target's response to the questionnaire in the distributed representation space is an outfit that both the user and the target evaluate as looking good, and provides the user terminal 100 with information about the outfit corresponding to information about the target and information related to the outfit, among the outfits.

[0070] Furthermore, the information processing device 10 may estimate a position when information based on both the user and the target is projected into the distributed representation space based on the information projected into the distributed representation space, the information on the user, and the information on the target, and estimate an outfit that the user evaluates as looking good on the target and that the target evaluates as looking good on the user based on the estimated position. For example, when the attribute information of the user is "late 20s" and the attribute information of the target is "early 20s", the information processing device 10 estimates that the intermediate position between the attribute information B2 and B5 in the distributed representation space F4 is the position when information corresponding to both the user and the target is projected in the distributed representation space F4, and estimates that the outfit shown by the image projected within a predetermined range from the position is the outfit that both the user and the target evaluate as looking good. Then, the information processing device 10 provides the user terminal 100 with information on the outfit that corresponds to the information on the target and information related to the outfit, among the estimated outfits.

[0071] [3-4. Estimating outfit coordination based on images of the wearer and user] In the example of Fig. 2, the information processing device 10 may estimate an outfit that suits a specific target (user, target person, etc.) based on a distributed representation space onto which an image showing the wearer's face is projected and an image of the face of the target. For example, the information processing device 10 estimates that, among the outfits #1, #3, and #4 estimated based on the distributed representation space F4 and the attribute information of the user U1, an outfit worn by a wearer whose facial features are similar to those of the user U1 is an outfit that suits the user.

[0072] Furthermore, the information processing device 10 may estimate a coordination that suits a specific target based on a distributed representation space onto which an image showing the wearer's body type is projected and an image showing the body type of the target. For example, the information processing device 10 estimates that, among the coordinations #1, #3, and #4 estimated based on the distributed representation space F4 and the attribute information of the user U1, the coordination worn by a wearer whose body type (e.g., thin, thick) is similar to that of the user U1 is the coordination that suits the user.

[0073] [3-5. Output of images in a manner based on information about the subject] In the example of FIG. 4, the information processing device 10 may output an image showing a coordinated outfit in a manner based on the CNN and information on a subject having a predetermined relationship with the user U1 (e.g., a person the user U1 meets). For example, when providing the image P1 to the user U1, the information processing device 10 identifies an evaluator who is similar to the distributed representation of the subject (e.g., a distributed representation based on attribute information, etc.). Then, the information processing device 10 determines that the subject evaluates the clothes as being suitable for the user U1 to a higher degree as indicated by an area in which the area vector is more similar to the distributed representation of the identified evaluator, and determines that the subject evaluates the clothes as being suitable for the user U1 to a lower degree as indicated by an area in which the similarity to the distributed representation of the evaluator is less, and outputs the image P1 in a manner according to the degree determined.

[0074] When outputting image P1, information processing device 10 may output image P1 together with information according to the degree to which the subject evaluates that the clothes in each region suit user U1. For example, image P1-1 is output together with information suggesting the purchase of clothes indicated by regions which the subject evaluates highly as suiting user U1, information suggesting the avoidance of wearing clothes included in regions which the subject evaluates low, information on other coordinations which utilize clothes included in regions which the subject evaluates high, etc.

[0075] 4. Configuration of Information Processing Device Next, the configuration of the information processing device 10 will be described with reference to Fig. 5. Fig. 5 is a diagram showing an example of the configuration of the information processing device 10 according to an embodiment. As shown in Fig. 5, the information processing device 10 has a communication unit 20, a storage unit 30, and a control unit 40.

[0076] (Regarding communication unit 20) The communication unit 20 is realized by, for example, a network interface card (NIC) etc. The communication unit 20 is connected to the network N by wire or wirelessly, and transmits and receives information between the user terminal 100, the evaluator terminal 200, etc.

[0077] (Regarding the storage unit 30) The storage unit 30 is realized by, for example, a semiconductor memory element such as a random access memory (RAM) or a flash memory, or a storage device such as a hard disk or an optical disk. As shown in FIG. 3, the storage unit 30 has an evaluator information database 31 and a user information database 32.

[0078] (Regarding Evaluator Information Database 31) The evaluator information database 31 stores various types of information related to evaluators. An example of information stored in the evaluator information database 31 will now be described with reference to Fig. 6. Fig. 6 is a diagram showing an example of the evaluator information database 31 according to an embodiment. In the example of Fig. 6, the evaluator information database 31 has items such as "evaluator ID", "attribute information", "questionnaire information", and "evaluation information".

[0079] "Evaluator ID" indicates identification information for identifying the evaluator. "Attribute information" indicates attribute information of the evaluator. "Survey information" indicates the evaluator's response to a specified questionnaire. "Evaluation information" indicates information regarding the evaluation by the evaluator, and has items such as "Subject ID," "Image information," "Clothing information," "Wearer information," and "Evaluation."

[0080] "Subject ID" indicates identification information for identifying the subject (image) of evaluation. "Image information" indicates the image that is the subject of evaluation. "Clothing information" indicates information about the clothes included in the outfit shown in the image, and for example, identification information for identifying the clothes is stored. "Wearer information" indicates information about the wearer shown in the image, and for example, identification information for identifying the wearer, attribute information and physical information of the wearer are stored. "Evaluation" indicates the evaluator's evaluation of the image.

[0081] That is, Figure 6 shows an example in which the attribute information of an evaluator identified by an evaluator ID "AID#1" is "attribute information #1", the questionnaire information is "questionnaire information #1", the image information of the subject of evaluation identified by a subject ID "DID#1" is "image information #1", the clothing information is "clothing information #1", the wearer information is "wearer information #1", and the evaluation is "evaluation #1".

[0082] (Regarding User Information Database 32) The user information database 32 stores various types of information related to users. An example of information stored in the user information database 32 will now be described with reference to FIG. 7. FIG. 7 is a diagram showing an example of the user information database 32 according to an embodiment. In the example of FIG. 7, the user information database 32 has items such as "user ID," "attribute information," "survey information," "user image," and "feedback information."

[0083] "User ID" indicates identification information for identifying a user. "Attribute information" indicates attribute information of a user. "Survey information" indicates the user's response to a specified survey. "User image" indicates an image showing the user's face and body type. "Feedback information" indicates information regarding feedback from the user regarding information provided to the user.

[0084] That is, Figure 7 shows an example in which the attribute information of a user identified by user ID "UID#1" is "attribute information #11", the survey information is "survey information #11", the user image is "user image #1", and the feedback information is "feedback information #1".

[0085] (Regarding the control unit 40) The control unit 40 is a controller, and is realized, for example, by a CPU (Central Processing Unit) or an MPU (Micro Processing Unit) executing various programs stored in a storage device inside the information processing device 10 using a RAM as a working area. The control unit 40 is also a controller, and is realized, for example, by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). As shown in FIG. 5, the control unit 40 according to the embodiment has a receiving unit 41, a generating unit 42, an estimating unit 43, a providing unit 44, a determining unit 45, an output unit 46, an acquiring unit 47, and an updating unit 48, and realizes or executes the functions and actions of information processing described below.

[0086] (Regarding Reception Section 41) The receiving unit 41 receives, from a plurality of evaluators, evaluations of an image of a wearer wearing a combination of a plurality of clothes, indicating whether the combination suits the wearer. For example, in the example of Fig. 2, the receiving unit 41 presents a pair of images P1 and P2 to the evaluators, compares the two images, and receives evaluations indicating which of the images suits the wearer and which of the images does not suit the wearer, and stores the evaluations in the evaluator information database 31.

[0087] The receiving unit 41 may also receive evaluations from multiple evaluators regarding an image of a wearer wearing a combination of multiple clothes, indicating whether the combination suits the wearer. For example, in the example of Fig. 4, the receiving unit 41 receives evaluations indicating whether the outfit #1 shown in the image P1 suits the wearer #1 shown in the image P1.

[0088] (Regarding the generation unit 42) The generation unit 42 generates a distributed representation space onto which the image and the evaluator information indicating the evaluator are projected, based on the evaluation received by the reception unit 41. For example, in the example of Fig. 2, the generation unit 42 refers to the evaluator information database 31 and uses the VSE technology to generate a distributed representation space onto which the image and the evaluator information indicating the evaluator are projected.

[0089] The generation unit 42 may project the image and the evaluator information to a closer position in the distributed representation space as the evaluator evaluates the combination as being suitable for the wearer, and may project the image and the evaluator information to a farther position in the distributed representation space as the evaluator evaluates the combination as being unsuitable for the wearer. For example, in the example of FIG. 2, the generation unit 42 generates a distributed representation space F1 in which the image P1 is projected closer to the evaluator information D1 indicating the evaluator A1 who evaluated the image P1 as "suitable" than the evaluator information D2 indicating the evaluator A2 who evaluated the image P1 as "unsuitable". The generation unit 42 also generates a distributed representation space F1 in which the image P2 is projected closer to the evaluator information D2 indicating the evaluator A2 who evaluated the image P2 as "suitable" than the evaluator information D1 indicating the evaluator A1 who evaluated the image P2 as "unsuitable".

[0090] The generating unit 42 may generate a distributed representation space in which information indicating the clothes included in the combination is further projected based on the evaluation. For example, in the example of FIG. 2, the generating unit 42 generates a distributed representation space F2 in which an image of the clothes I1 included in the coordination shown by the image P1 is projected closer to the evaluator information D1 indicating the evaluator A1 who evaluated the image P1 as "suitable" than to the evaluator information D2 indicating the evaluator A2 who evaluated the image P1 as "unsuitable". The generating unit 42 also generates a distributed representation space F2 in which an image of the clothes I2 included in the coordination shown by the image P2 is projected closer to the evaluator information D2 indicating the evaluator A2 who evaluated the image P2 as "suitable" than to the evaluator information D1 indicating the evaluator A1 who evaluated the image P2 as "unsuitable".

[0091] Furthermore, the generating unit 42 may generate a distributed representation space in which information indicating a category to which the combination belongs is further projected based on the evaluation. For example, in the example of FIG. 2, the generating unit 42 generates a distributed representation space F3 in which the category T1 (casual) of the coordination indicated by the image P1 is projected closer to the evaluator information D1 indicating the evaluator A1 who evaluated the image P1 as "suitable" than to the evaluator information D2 indicating the evaluator A2 who evaluated the image P1 as "unsuitable". Furthermore, the generating unit 42 generates a distributed representation space F3 in which the category T2 (date) of the image P2 is projected closer to the evaluator information D2 indicating the evaluator A2 who evaluated the image P2 as "suitable" than to the evaluator information D1 indicating the evaluator A1 who evaluated the image P2 as "unsuitable".

[0092] The generating unit 42 may generate a distributed representation space in which information indicating a provider of the clothes included in the combination is further projected based on the evaluation. For example, in the example of FIG. 2, the generating unit 42 generates a distributed representation space in which information indicating a provider of the clothes included in the coordination shown by the image P1 is projected closer to the evaluator information D1 indicating the evaluator A1 who evaluated the image P1 as "suitable" than to the evaluator information D2 indicating the evaluator A2 who evaluated the image P1 as "unsuitable". The generating unit 42 also generates a distributed representation space in which information indicating a provider of the clothes included in the coordination shown by the image P2 is projected closer to the evaluator information D2 indicating the evaluator A2 who evaluated the image P2 as "suitable" than to the evaluator information D1 indicating the evaluator A1 who evaluated the image P2 as "unsuitable".

[0093] The generating unit 42 may generate a distributed representation space in which evaluator information indicating attributes of the evaluators is projected. For example, in the example of FIG. 2, the generating unit 42 generates a distributed representation space F4 in which the image P1 is projected closer to the attribute information of the evaluator A1 who evaluated the image P1 as "suitable" than to the attribute information of the evaluator A2 who evaluated the image P1 as "unsuitable". The generating unit 42 also generates a distributed representation space F4 in which the image P2 is projected closer to the attribute information of the evaluator A2 who evaluated the image P2 as "suitable" than to the attribute information of the evaluator A1 who evaluated the image P2 as "unsuitable".

[0094] The generating unit 42 may generate a distributed representation space in which evaluator information indicating the evaluator's response to a predetermined questionnaire is projected. For example, in the example of FIG. 2, the generating unit 42 generates a distributed representation space F5 in which the image P1 is projected closer to the response of the evaluator A1 who evaluated the image P1 as "suitable" than to the response of the evaluator A2 who evaluated the image P1 as "unsuitable". The generating unit 42 also generates a distributed representation space F5 in which the image P2 is projected closer to the response of the evaluator A2 who evaluated the image P2 as "suitable" than to the response of the evaluator A1 who evaluated the image P2 as "unsuitable".

[0095] Furthermore, the generation unit 42 may generate a convolutional neural network based on the evaluations received by the reception unit 41. For example, in the example of Fig. 4, the generation unit 42 generates a CNN using learning data including images, evaluations by evaluators on the images, and evaluator information on the evaluators.

[0096] (Regarding the estimation unit 43) The estimation unit 43 estimates a combination of multiple clothes that suits a user based on the distributed representation space and information about the user. For example, in the example of Fig. 2, in response to a request for a coordinated outfit from the user terminal 100 via an electronic commerce service or a coordinate service, the estimation unit 43 refers to the user information database 32 and estimates a coordinated outfit that suits the user U1 based on the generated distributed representation space and information about the user U1.

[0097] The estimation unit 43 may estimate a combination of multiple clothes that suits a predetermined target based on a distributed representation space in which an image of a wearer wearing a combination of multiple clothes is projected and evaluator information indicating the evaluators is generated based on evaluations of the image of the wearer wearing the combination of multiple clothes, the evaluations being based on evaluations from multiple evaluators indicating whether the combination suits the wearer. For example, in the example of Fig. 2, the estimation unit 43 estimates a coordination that suits the user U1 based on the distributed representation space generated in response to a request for coordination from the user terminal 100 via an electronic commerce service or a coordination service.

[0098] The estimation unit 43 may estimate a combination that suits a user based on the distributed representation space and information about the user. For example, in the example of Fig. 2, the estimation unit 43 estimates an outfit that suits the user U1 based on the generated distributed representation space and information about the user U1.

[0099] The estimation unit 43 may estimate a combination that suits the user based on at least one of the user's attributes, the category of clothing selected by the user, and the user's response to a predetermined questionnaire. For example, in the example of FIG. 2, when the attribute information of the user U1 is "late 20s", the estimation unit 43 estimates that the coordination shown by the image projected within a predetermined range from the attribute information B2 in the distributed representation space F4 is a coordination that suits the user U1, and provides the user terminal 100 with information on the coordination corresponding to the information on the user U1 and information related to the coordination among the coordination. When the estimation unit 43 receives information indicating that the user U1 has selected the category "casual", the estimation unit 43 estimates that the coordination shown by the image projected within a predetermined range from the attribute information B2 in the distributed representation space F4 and the image projected within a predetermined range from the category T1 in the distributed representation space F3 is a coordination that suits the user U1 in the category "casual", and provides the user terminal 100 with information on the coordination corresponding to the attribute information of the user U1 and information related to the coordination among the coordination. In addition, if user U1 answers "I'm not confident in my fashion sense" to a questionnaire such as "Are you confident in your fashion sense?", the estimation unit 43 estimates that the coordination shown in the image projected within a specified range from the answer C2 in the distributed representation space F5 is a coordination that suits user U1, and provides information about the coordination that corresponds to information about user U1 and information related to the coordination to the user terminal 100.

[0100] Furthermore, the estimation unit 43 may estimate a combination that the subject evaluates as looking good on the user based on the distributed representation space and information about the subject. For example, in the example of Fig. 2, the estimation unit 43 estimates an outfit that the subject evaluates as looking good on the user U1 based on the distributed representation space and information about the subject having a predetermined relationship with the user U1.

[0101] The estimation unit 43 may estimate a combination that the subject evaluates as looking good on the user based on at least one of the subject's attributes and the subject's response to a predetermined questionnaire. For example, in the example of FIG. 2, when the subject's attribute information is "early 20s", the estimation unit 43 estimates that the coordination shown by the image projected within a predetermined range from the attribute information B6 in the distributed representation space F4 is the coordination that the subject evaluates as looking good on the user, and provides the user terminal 100 with information on the coordination corresponding to the information on the user U1 and information related to the coordination among the coordination. When the subject answers "I'm confident in fashion" to a questionnaire such as "Are you confident in fashion?", the estimation unit 43 estimates that the coordination shown by the image projected within a predetermined range from the response C5 in the distributed representation space F5 is the coordination that the subject evaluates as looking good on the user, and provides the user terminal 100 with information on the coordination corresponding to the information on the user U1 and information related to the coordination among the coordination.

[0102] The estimation unit 43 may estimate a combination that suits a subject based on the distributed representation space and information about the subject. For example, in the example of Fig. 2, the estimation unit 43 estimates an outfit that the subject evaluates as suiting the subject based on the distributed representation space and information about the subject having a predetermined relationship with the user U1.

[0103] The estimation unit 43 may estimate a combination that suits the subject based on at least one of the subject's attributes and the subject's answer to a predetermined questionnaire. For example, in the example of FIG. 2, when the subject's attribute information is "early 20s", the estimation unit 43 estimates that the coordination shown by the image projected within a predetermined range from the attribute information B6 in the distributed representation space F4 is a coordination that the subject evaluates to suit the subject, and provides the user terminal 100 with information on the coordination corresponding to the information on the subject and information related to the coordination among the coordination. When the subject answers "I'm confident in fashion" to a questionnaire such as "Are you confident in fashion?", the estimation unit 43 estimates that the coordination shown by the image projected within a predetermined range from the answer C5 in the distributed representation space F5 is a coordination that the subject evaluates to suit the subject, and provides the user terminal 100 with information on the coordination corresponding to the information on the subject and information related to the coordination among the coordination.

[0104] The estimation unit 43 may estimate a combination that suits a specific target based on the distributed representation space onto which an image showing the wearer's face is projected and an image showing the face of the specific target. For example, in the example of Fig. 2, the estimation unit 43 estimates that, among the coordinations #1, #3, and #4 estimated based on the distributed representation space F4 and the attribute information of the user U1, the coordination worn by a wearer whose facial features are similar to those of the user U1 is the coordination that suits the user.

[0105] The estimation unit 43 may estimate a combination that suits a specific target based on a distributed representation space onto which an image showing the wearer's body type is projected and an image showing the body type of the specific target. For example, in the example of Fig. 2, the estimation unit 43 estimates that, among the coordinations #1, #3, and #4 estimated based on the distributed representation space F4 and the attribute information of the user U1, the coordination worn by a wearer whose body type is similar to that of the user U1 is the coordination that suits the user.

[0106] In addition, the estimation unit 43 may further estimate the degree to which the combination suits a predetermined target. For example, in the example of Fig. 4, the estimation unit 43 estimates that the closer the information on the coordination indicated by the image is to the attribute information B2 in the distributed representation space F4, the higher the degree to which the combination suits the user U1.

[0107] (About provider 44) The providing unit 44 provides the user with combination information regarding the combination estimated by the estimating unit 43. For example, in the example of Fig. 2, the providing unit 44 provides information on coordination #1, #3, and #4 and information on the clothes included in coordination #1, #3, and #4 in the electronic commerce service. The providing unit 44 also provides information on coordination #1, #3, and #4 in the coordination service.

[0108] Furthermore, the providing unit 44 may provide to the user information on the combination estimated by the estimating unit 43. For example, in the example of Fig. 2, the providing unit 44 provides information on coordinates #1, #3, and #4 and information on clothes included in coordinates #1, #3, and #4 in the e-commerce service. The providing unit 44 also provides information on coordinates #1, #3, and #4 in the coordination service.

[0109] The providing unit 44 may provide the combinations in a manner according to the degree of suitability. For example, in the example of Fig. 2, the providing unit 44 estimates that the closer the image in the distributed representation space F4 is to the attribute information B2, the more suitable the coordinated outfit information is for the user U1, and provides the coordinated outfit information with a higher priority.

[0110] Here, the manner in which information on clothing combinations is provided to the user terminal 100 will be described with reference to Figures 8 and 9. First, the manner in which information on clothing combinations is provided in an electronic commerce service will be described with reference to Figure 8. Figure 8 is a diagram (1) showing an example of a screen of the user terminal 100.

[0111] As shown in FIG. 8, the providing unit 44 provides a screen SC1 showing search results corresponding to the search query "T-shirt" or "cut and sew" input by the user U1 in the electronic commerce service. For example, the providing unit 44 provides a screen SC1 showing images of outfits including T-shirts or cut and sews in order of how well the outfits shown in the images look on the user U1 (in other words, a screen SC1 in which the order of how well the items look on the user U1 is specified in the area AR1 showing the pull-down menu). The providing unit 44 also displays information such as "the order in which the subject evaluates that the items look good on you (user U1)" or "the order in which the subject evaluates that the items look good on themselves" in a manner that allows the user to specify information to be displayed in the pull-down menu of the area AR1 (i.e., conditions for narrowing down the search results).

[0112] For example, when "in order of evaluation that the target person looks good on you" is specified in the area AR1, the providing unit 44 provides a screen SC1 that displays images showing coordinations corresponding to information about the user U1 and information related to the coordination, among images projected within a predetermined range from information about the target person in the distributed representation space, in order of the position projected closer to the information about the target person (in other words, in order of the degree to which the target person evaluates that the coordination looks good on the user U1). In this case, the information processing device 10 may receive information for identifying the target person in the electronic commerce service (in other words, a target person ID linked to information about the target person) and information about the target person from the user U1. Then, the providing unit 44 provides the screen SC1 based on the information received from the user U1.

[0113] Furthermore, when "order of evaluation that the subject person looks good on him / her" is specified in the area AR1, the providing unit 44 provides a screen SC1 that displays images showing coordinations corresponding to information about the subject and information related to the coordination, among images projected within a predetermined range from information about the subject in the distributed representation space, in the order of projection to a position closer to the information about the subject (in other words, in order of the degree to which the subject person evaluates that the coordination looks good on the user U1). In this case, the information processing device 10 may receive information for identifying the subject in the electronic commerce service and information about the subject from the user U1. The providing unit 44 then provides the screen SC1 based on the information received from the user U1.

[0114] Next, a mode of providing information on clothing combinations in a coordinated service will be described with reference to Fig. 9. Fig. 9 is a diagram (2) showing an example of a screen of the user terminal 100.

[0115] As shown in Fig. 9, the providing unit 44 provides a screen SC2 showing search results corresponding to a search query input by the user U1 in the coordination service. For example, the providing unit 44 provides a screen SC2 showing images of coordinations corresponding to the search query input by the user U1 in order of how well the coordinations shown in the images suit the user U1 (in other words, a screen SC2 in which display in order of suitability for the user U1 is specified in an area AR2 showing a pull-down menu). The providing unit 44 also displays information such as "the order in which the subject evaluates that the coordinations suit you (user U1)" or "the order in which the subject evaluates that the coordinations suit themselves" in a manner that allows the user to specify the information to be displayed in the pull-down menu of the area AR2.

[0116] For example, when "order of evaluation that the target person looks good on you" is specified in the area AR2, the providing unit 44 provides a screen SC2 that displays images showing coordination corresponding to information about the user U1 and information related to the coordination, among images projected within a predetermined range from information about the target person in the distributed representation space, in the order of projecting to a position closer to the information about the target person. In this case, the information processing device 10 may receive information for identifying the target person in the coordination service and information about the target person from the user U1. Then, the providing unit 44 provides the screen SC2 based on the information received from the user U1.

[0117] Furthermore, when "the order of evaluation of suitability by the subject" is specified in the area AR2, the providing unit 44 provides a screen SC2 that displays images showing coordination corresponding to information about the subject or information related to the coordination, among images projected within a predetermined range from information about the subject in the distributed representation space, in the order of projecting to a position closer to the information about the subject. Note that in such a case, the information processing device 10 may receive information for identifying the subject in the coordination service and information about the subject from the user U1. Then, the providing unit 44 provides the screen SC2 based on the information received from the user U1.

[0118] In addition, in Figures 8 and 9, the providing unit 44 may use the area vectors generated by the CNN to estimate the areas of the coordination shown in each image that each evaluator judges to look good (in other words, the key clothing in the coordination), and provide an image showing the estimated areas in a heat map.

[0119] In the above example, information on products and the like is displayed in order of suitability based on a specific search query, but the scope of the present invention is not limited to this case. For example, even if there is no search query related to a product category, the providing unit 44 may evaluate the suitability of a group of products including an arbitrary product category for the user, and provide the result to the user as a recommendation result.

[0120] In addition, even if it is not a search query, it is possible to combine various filters and other evaluation results with the evaluation result of the suitability and display it to the user. For example, in a recommendation system, it is assumed that candidates of items worth recommending to a user and the evaluation results of the recommendation degree for them are specified in advance. Then, the suitability degree of these recommended candidate items can be evaluated, and items ranked according to the recommendation degree and suitability degree can be displayed to the user. In this way, it is possible to provide information on items (products) and coordination to the user using the result of a composite combination of the suitability degree with other elements.

[0121] (Regarding the determination unit 45) The determination unit 45 determines the degree to which each of the clothes included in the regions suits the user based on the multiple regions in the distributed representation and the information on the user. For example, in the example of Fig. 4, the determination unit 45 identifies an evaluator whose distributed representation is similar to the distributed representation of the user U1. The determination unit 45 then determines that the clothes indicated by the region whose region vector is more similar to the distributed representation of the identified evaluator are more suited to the user U1, and determines that the clothes indicated by the region whose region vector is less similar to the distributed representation of the evaluator are less suited to the user U1.

[0122] The determination unit 45 may also determine the degree to which the subject evaluates each of the clothes included in the regions as being suitable for the user based on multiple regions in the distributed representation and information about the subject. For example, in the example of FIG. 4, the determination unit 45 identifies an evaluator similar to the distributed representation of the subject. Then, the determination unit 45 determines that the subject evaluates the clothes as being suitable for the user U1 to a higher degree as indicated by a region of the image P1 whose region vector is more similar to the distributed representation of the identified evaluator, and determines that the subject evaluates the clothes as being suitable for the user U1 to a lower degree as indicated by a region whose region vector is less similar to the distributed representation of the evaluator.

[0123] (Regarding output unit 46) The output unit 46 outputs the image in a form based on the distributed representation generated for each region of the image by the convolutional neural network generated by the generation unit 42. For example, in the example of Fig. 4, the output unit 46 estimates the regions that each evaluator judged to look good using the region vectors generated by the CNN, and outputs a heat map showing the estimated regions.

[0124] The output unit 46 may output an image in a manner based on a plurality of regions in the distributed representation and evaluator information indicating the evaluator. For example, in the example of Fig. 4, the output unit 46 estimates the regions that each evaluator judges to be suitable using region vectors generated by a CNN that has been trained to output a distributed representation similar to the distributed representation of the evaluator who evaluated that the image looks good and a distributed representation not similar to the distributed representation of the evaluator information of the evaluator who evaluated that the image does not look good, and outputs a heat map indicating the estimated regions.

[0125] Furthermore, the output unit 46 may output an image in which the degree of emphasis is increased for regions in which the distributed representation is more similar to the distributed representation of the evaluator information and the degree of emphasis is decreased for regions in which the distributed representation is less similar to the distributed representation of the evaluator information. For example, in the example of Fig. 4, the output unit 46 outputs a heat map in which the degree of red is increased for regions in which the distributed representation is more similar to the distributed representation of the evaluator information and the degree of blue is increased for regions in which the distributed representation is less similar to the distributed representation of the evaluator information.

[0126] The output unit 46 may output to the user an image that displays the region in a manner according to the degree of suitability for the user determined by the determination unit 45. For example, in the example of FIG. 4, when the region vector of the region showing the bottoms of the image P1 is similar to the distributed representation of the evaluator specified in step Sb3 and the region vector of the region showing the tops of the image P1 is not similar, the information processing device 10 outputs an image P1-1 in which the region showing the bottoms of the image P1 is displayed in dark red and the region showing the tops is displayed in dark blue. When the region vector of the region showing the tops of the image P1 is similar to the distributed representation of the evaluator specified in step Sb3 and the region vector of the region showing the bottoms of the image P1 is not similar, the information processing device 10 outputs an image P1-2 in which the region showing the tops of the image P1 is displayed in dark red and the region showing the bottoms is displayed in dark blue.

[0127] The output unit 46 may output to the user an image displaying an area in a manner according to the degree to which the subject, as determined by the determination unit 45, evaluates that the subject evaluates that the user U1 is suitable for the clothing indicated by an area in which the area vector is more similar to the distributed representation of an evaluator similar to the distributed representation of the subject, and determines that the subject evaluates that the subject evaluates that the user U1 is suitable for the clothing indicated by an area in which the distributed representation of the evaluator is less similar to the distributed representation of the evaluator, and outputs the image P1 in a manner according to the degree of the evaluation.

[0128] (Regarding Acquisition Unit 47) The acquisition unit 47 acquires feedback from the user or other users having a predetermined relationship with the user with respect to the combination information provided by the provision unit 44. For example, in the example of FIG. 2, the acquisition unit 47 acquires information (feedback) indicating whether the user U1 has purchased clothes included in the coordination #1, #3, and #4 or applied for rental in the electronic commerce service, and stores the information in the user information database 32. The acquisition unit 47 also acquires information (feedback) indicating whether the user U1 has registered the coordination #1, #3, and #4 as favorites in the coordination service. The acquisition unit 47 also acquires information (feedback) indicating whether the target person was positive about the user U1 wearing the coordination indicated by the provided information from the user terminal 100, a terminal device used by the target person, or the like. The acquisition unit 47 also acquires information (feedback) indicating whether the target person was positive about the user U1 who wore the coordination indicated by the provided information when the user U1 gave the target person a gift of clothes included in the coordination indicated by the provided information from the user terminal 100, a terminal device used by the target person, or the like.

[0129] (About Update Part 48) The update unit 48 updates the distributed representation space based on the feedback acquired by the acquisition unit 47. If the user U1 is positive about the coordinate #1, the update unit 48 updates the attribute information B2 and the image P1 in the distributed representation space F4 so as to bring them closer to each other by a predetermined distance. On the other hand, if the user U1 is negative about the coordinate #3, the update unit 48 updates the attribute information B2 and the image P3 in the distributed representation space F4 so as to move them apart by a predetermined distance.

[0130] Furthermore, if the target person has a positive opinion of user U1 wearing the outfit indicated by the provided information, the update unit 48 updates the information corresponding to the target person (e.g., attribute information, responses to a questionnaire, etc.) and the image showing the outfit in the distributed representation space so as to bring them closer together by a predetermined distance. On the other hand, if the target person has a negative opinion of user U1 wearing the outfit indicated by the provided information, the update unit 48 updates the information corresponding to the target person and the image showing the outfit in the distributed representation space so as to move them apart by a predetermined distance.

[0131] Furthermore, when user U1 presents to a target person clothes included in the coordination indicated by the information provided to user U1, if the target person is positive, the update unit 48 updates the information corresponding to the target person and the image showing the coordination to move them closer to each other by a predetermined distance in the distributed representation space. On the other hand, when user U1 presents to a target person clothes included in the coordination indicated by the information provided to user U1, if the target person is negative, the update unit 48 updates the information corresponding to the target person and the image showing the coordination to move them apart by a predetermined distance in the distributed representation space.

[0132] [5. Information processing flow] The information processing procedure (1) of the information processing device 10 according to the embodiment will be described with reference to Fig. 10. Fig. 10 is a flowchart (1) showing an example of the information processing procedure according to the embodiment.

[0133] 10, the information processing device 10 determines whether or not an evaluation for an image of a wearer wearing a combination of multiple clothes has been received (step S101). If the evaluation has not been received (step S101; No), the information processing device 10 waits until the evaluation is received.

[0134] On the other hand, if an evaluation has been received (step S101; Yes), the information processing device 10 generates a distributed representation space into which the image and evaluator information indicating the evaluator are projected based on the evaluation (step S102), and ends the process.

[0135] Next, the information processing procedure (2) of the information processing device 10 according to the embodiment will be described with reference to Fig. 11. Fig. 11 is a flowchart (2) showing an example of the information processing procedure according to the embodiment.

[0136] 11, the information processing device 10 determines whether or not an evaluation for an image of a wearer wearing a combination of multiple clothes has been received (step S201). If the evaluation has not been received (step S201; No), the information processing device 10 waits until the evaluation is received.

[0137] On the other hand, if an evaluation has been received (step S201; Yes), the information processing device 10 generates a convolutional neural network based on the evaluation (step S202). Next, the information processing device 10 outputs the image in a form based on the distributed representation generated by the convolutional neural network for each region of the image (step S203), and ends the process.

[0138] Next, the information processing procedure (3) of the information processing device 10 according to the embodiment will be described with reference to Fig. 12. Fig. 12 is a flowchart (3) showing an example of the information processing procedure according to the embodiment.

[0139] 12, the information processing device 10 determines whether or not a request for providing a clothing combination has been received (step S301). If the request for providing has not been received (step S301; No), the information processing device 10 waits until the request for providing is received.

[0140] On the other hand, if a request for provision is received (step S301; Yes), the information processing device 10 estimates a combination of multiple clothes that suits a specific target based on a distributed representation space in which an image of a wearer wearing a combination of multiple clothes, generated based on the evaluations from the evaluators, and evaluator information indicating the evaluators are projected (step 3202). Next, the information processing device 10 provides information on the estimated combination to the user (step S303), and ends the process.

[0141] 6. Modifications The above-described embodiment is merely an example, and various modifications and applications are possible.

[0142] [6-1. Providing information to evaluators] In the above-described embodiment, the information processing device 10 may provide information on coordination based on the evaluation of the image by the evaluator. For example, the information processing device 10 provides information on coordination corresponding to the image evaluated as looking good by the evaluator in an electronic commerce service or a coordination service.

[0143] [6-2. About the target persons] In the above embodiment, the target person is a person whom the user meets or a person to whom a gift is given, but the target person is not limited to such examples. For example, the target person may be a person served by the user who is a shop clerk or a coordinator. In such a case, the information processing device 10 estimates an outfit that the target person evaluates as suiting the target person (in other words, an outfit that the target person likes) based on the distributed representation space and information about the target person served by the user, and outputs information about the estimated outfit to the user. In addition, the information processing device 10 outputs an image showing the outfit (for example, an image showing which clothes out of the outfit the target person likes) in a mode based on the CNN and information about the target person served by the user.

[0144] The information processing device 10 may match users who are shop clerks or coordinators with target persons who are customers. For example, the information processing device 10 estimates the positions of each user and each target person when projected into a distributed representation space based on information about each user and information about each target person, and provides information about the users and targets who are located within a predetermined range from the projected positions to each of the users and targets.

[0145] [6-3. Providing information on clothing designs] In the above embodiment, the information processing device 10 may receive the evaluator's evaluation of the data of the clothes designed by the manufacturer, and provide the manufacturer with information based on the evaluation. For example, the information processing device 10 generates a CNN using learning data including data (e.g., CAD data) showing a wearer wearing the clothes designed by the manufacturer, the evaluator's evaluation of the data, and evaluator information on the evaluator. As a specific example, the information processing device 10 generates a CNN that has been trained to generate region vectors (the clothes shown in the data are divided into a plurality of regions, and vectors showing the characteristics of each divided region) that are similar to the distributed representations of the evaluators who are likely to have evaluated the region as looking good and dissimilar to the distributed representations of the evaluators who are likely to have evaluated the region as not looking good, for a certain region of the clothes shown in the data. Then, the information processing device 10 uses the region vectors generated by such trained CNN to estimate the regions that each evaluator judged to look good (in other words, parts of the clothes such as the collar and the hem) and outputs a heat map showing the estimated regions.

[0146] To give a more specific example, based on the CNN and information about a specific user (e.g., a clothing purchasing demographic), the information processing device 10 determines the degree to which the user evaluates that each part of the clothing indicated by each area of ​​the data looks good, and outputs data showing the determined degrees in the form of a heat map to the manufacturer.

[0147] This enables the information processing device 10 to grasp which parts of the clothing being designed are evaluated as looking good on the user and which parts are evaluated as not looking good on the user, thereby improving convenience in designing clothing.

[0148] [6-4. Processing mode] Among the processes described in the above embodiments, all or part of the processes described as being performed automatically can be performed manually, and conversely, all or part of the processes described as being performed manually can be performed automatically by a known method. In addition, the information including the processing procedures, specific names, various data and parameters shown in the above text and drawings can be changed arbitrarily unless otherwise specified. For example, the various information shown in each drawing is not limited to the illustrated information.

[0149] In addition, each component of each device shown in the figure is a functional concept, and does not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure, and all or part of them can be functionally or physically distributed and integrated in any unit according to various loads, usage conditions, etc.

[0150] Furthermore, the above-described embodiments can be appropriately combined as long as the processing contents are not contradictory.

[0151] [6-5. Evaluation subject] In the above embodiment, the information processing device 10 receives evaluations from a plurality of evaluators on an image of a wearer wearing a combination of clothes, indicating whether the combination suits the wearer, and generates a distributed representation space and a CNN based on the received evaluations. However, the subject of evaluation is not limited to a combination of clothes, and may be any. For example, the information processing device 10 may receive evaluations from a plurality of evaluators on an image showing a combination of a plurality of pieces of furniture, indicating whether the combination suits the wearer, and generate a distributed representation space and a CNN based on the received evaluations. As a specific example, the information processing device 10 generates a distributed representation space in which an image showing a combination of furniture, information about the furniture shown in the image (for example, furniture type, brand, shop name, manufacturing location, usage scene, color, category (casual, etc.), selling point, material, size, price range, etc.), information about the evaluator, etc. are projected based on the evaluations received from the evaluators, and provides the user with information about the combination of furniture that the user evaluates as looking good. In addition, the information processing device 10 generates a CNN based on the evaluations received from the evaluators, and outputs an image showing the furniture combination in a manner based on the distributed representation generated by the CNN for each region of the image showing the furniture combination.

[0152] Furthermore, for example, the information processing device 10 may receive evaluations from a plurality of evaluators on an image showing a combination of a plurality of dishes (e.g., a menu), indicating whether the combination is appropriate or not (e.g., whether the combination of appearance and taste is appropriate or not), and generate a distributed representation space and a CNN based on the received evaluations. As a specific example, the information processing device 10 generates a distributed representation space in which an image showing the combination of dishes, information on the dishes shown in the image (e.g., the name of the dish, ingredients, genre of the dish, season, taste characteristics, price range, etc.), information on the evaluator, etc. are projected based on the evaluations received from the evaluators, and provides the user with information on the combination of dishes that the user evaluates as appropriate. Furthermore, the information processing device 10 generates a CNN based on the evaluations received from the evaluators, and outputs an image showing the combination of dishes in a manner based on the distributed representations generated by the CNN for each region of the image showing the combination of dishes.

[0153] Furthermore, for example, the information processing device 10 may receive evaluations from a plurality of evaluators on an image showing a combination of a plurality of flowers (e.g., a bouquet of flowers), indicating whether the combination is appropriate or not, and generate a distributed representation space and a CNN based on the received evaluations. As a specific example, the information processing device 10 generates a distributed representation space in which an image showing a combination of flowers, information on the flowers shown in the image (e.g., a usage scene, the name of each flower, color, season, purpose, category (e.g., casual, etc.), selling point, place of origin, price range, etc.), and information on the evaluator are projected based on the evaluations received from the evaluators, and provides the user with information on the combination of flowers that the user evaluates as appropriate. Furthermore, the information processing device 10 generates a CNN based on the evaluations received from the evaluators, and outputs an image showing the combination of flowers in a manner based on the distributed representations generated by the CNN for each region of the image showing the combination of flowers.

[0154] [6-6. About Images] In the above embodiment, the information processing device 10 has been described as processing an image of a wearer wearing a combination of multiple pieces of clothing, but the embodiment is not limited to this, and similar processing may be performed on an image of a wearer wearing a single piece of clothing.

[0155] [6-7. Estimation of clothing based on the user's self-described physical characteristics] In the above embodiment, the information processing device 10 may estimate clothes that suit a user based on physical information (user self-proclaimed physical information) indicating the user's self-proclaimed physical characteristics. The information processing device 10 may also estimate clothes that suit a user based on physical information (user physical information) indicating the user's self-proclaimed physical characteristics and the user self-proclaimed physical information. For example, the information processing device 10 has an estimation unit 43 that estimates clothes that suit a specific target based on a distributed representation space in which an image of a wearer wearing clothes is projected and wearer physical information indicating the wearer's physical characteristics is projected, and a provision unit 44 that provides the user with information about the clothes estimated by the estimation unit 43, and the estimation unit 43 may estimate clothes that suit a specific target based on the distributed representation space, target physical information indicating the physical characteristics of the specific target, and target self-proclaimed physical information indicating the physical characteristics of the specific target. Further, for example, the information processing device 10 may have an estimation unit 43 that estimates clothes that suit a specific subject based on a distributed representation space onto which an image of a wearer wearing clothes is projected, the distributed representation space being generated based on evaluations from a plurality of evaluators indicating whether the clothes suit the wearer, and the image, wearer physical information indicating the physical characteristics of the wearer, and evaluator information indicating the evaluators, and a provision unit 44 that provides a user with information about the clothes estimated by the estimation unit 43, and the estimation unit 43 may estimate the clothes that suit a specific subject based on the distributed representation space, subject physical information indicating the physical characteristics of the specific subject, and subject self-declared physical information indicating the physical characteristics self-declared by the specific subject.Also, for example, the information processing device 10 has an estimation unit 43 that estimates clothes that suit a specific subject based on a distributed representation space into which an image of clothing is evaluated and which is generated based on evaluations from a plurality of evaluators indicating whether the clothing suits the specific subject, and into which an image and evaluator physical information indicating the physical characteristics of the evaluators are projected, and a provision unit 44 that provides a user with information about the clothing estimated by the estimation unit 43, and the estimation unit 43 may estimate the clothes that suit a specific subject based on the distributed representation space, subject physical information indicating the physical characteristics of the specific subject, and subject self-proclaimed physical information indicating the physical characteristics self-proclaimed by the specific subject. For example, the information processing device 10 includes an estimation unit 43 that estimates clothes that suit a specific subject based on a distributed representation space generated based on evaluations from multiple evaluators indicating whether the clothes suit the subject, the distributed representation space projecting the image, evaluator physical information indicating the physical characteristics of the evaluators, and evaluator information indicating the evaluators (excluding the evaluator physical information), and a provision unit 44 that provides a user with information about the clothes estimated by the estimation unit 43, and the estimation unit 43 may estimate clothes that suit a specific subject based on the distributed representation space, subject physical information indicating the physical characteristics of the specific subject, and subject self-proclaimed physical information indicating the physical characteristics that the specific subject claims. The estimation unit 43 may also estimate clothes that suit a specific subject based on the positional relationship between the image, the subject physical information, and the subject self-proclaimed physical information. The estimation unit 43 may also estimate clothes that suit a specific subject based on the area of ​​the space connecting the image, the subject information, and the subject self-proclaimed physical information. The estimation unit 43 may estimate clothes that suit a specific subject based on the positional relationship between the image, the subject's body information, the subject's self-proclaimed body information, and the evaluator information. The estimation unit 43 may estimate clothes that suit a specific subject based on the area of ​​a space connecting the image, the subject's information, the subject's self-proclaimed body information, and the evaluator information. This allows the information processing device 10 to provide information on clothes that suit a user in an electronic commerce service or a coordination service, taking into account the user's self-recognized body type.For example, even if the user's actual body type is "thin", the user may self-identify as "slightly overweight" rather than "thin". In such a case, it is not necessarily the case that clothes that suit the user's body type should be suggested regardless of the user's self-identified body type. In other words, even if the user's actual body type is "thin", if the user self-identifies as "slightly overweight", it is not necessarily the case that clothes that suit the "thin" body type should be suggested. By taking into account the user's self-identified body type, it is possible to suggest clothes that suit the user better. In this case, the information processing device 10 may estimate clothes that suit the user by changing the weight of the user's self-proclaimed physical information and the user's physical information (for example, by increasing the weight of one of them). Note that the weight (or the weight ratio, etc.) of the user's self-proclaimed physical information and the user's physical information may be set in any way, and may be changed appropriately according to feedback from the user, etc.

[0156] Second Embodiment Next, a second embodiment will be described. Note that the same description as in the first embodiment will be omitted as appropriate. In the following embodiments, the physical information is not limited to body type information, and may include any information that indicates physical characteristics. For example, the physical information may include information on the shape of the face, head, eyes, ears, nose, and mouth, skin color (e.g., warm skin tone, cool skin tone, etc.), hair color (e.g., black hair, brown hair, etc.), hairstyle, face type, skeletal system, hand length, neck thickness, etc. In addition, for example, the physical information may include information on attributes such as age, sex, height, weight, race, and nationality. In addition, for example, the physical information may include additional information on the presence or absence of glasses, the shape of glasses, the presence or absence of makeup, and makeup tendencies. In addition, the evaluator's evaluation may be an evaluation of an item alone (e.g., clothing alone) or an evaluation of a combination of items (e.g., coordination). In addition, the evaluator's evaluation may be an evaluation of the wearing of an item alone or a combination. For example, a variation of the wearing may be included in a data set when learning coordination and learning it. For example, if there is data on variations in the way a single outfit is worn, the data can be trained. This makes it possible to suggest which outfits in which outfits are closest to the query.

[0157] 7. Configuration of Information Processing System First, an information processing system 1A according to an embodiment will be described. FIG. 13 is a diagram showing a configuration example of the information processing system 1A according to an embodiment. As shown in FIG. 13, the information processing system 1A includes an information processing device 10A, a user terminal 100, and an evaluator terminal 200. The information processing device 10A, the user terminal 100, and the evaluator terminal 200 are connected to each other via a predetermined communication network (network N) so as to be able to communicate with each other by wire or wirelessly. Note that the information processing system 1A shown in FIG. 13 may include a plurality of information processing devices 10A, a plurality of user terminals 100, and a plurality of evaluator terminals 200.

[0158] The information processing device 10A is an information processing device that receives evaluations from a plurality of evaluators on a plurality of images (images that include clothing in at least a part of the image) indicating whether the clothing looks good on the subject, generates a distributed representation space in which the image and physical information indicating the physical characteristics of the subject (the wearer of the clothing or the evaluator himself / herself) are projected based on the received evaluations, and realizes information processing using the generated distributed representation space, and is realized by, for example, a server device, a cloud system, or the like. Note that, for example, an evaluator gives an annotation (for example, an annotation indicating an evaluation) to an image to evaluate the image. For example, when an evaluator gives an annotation indicating "looks good", the clothing in the image is evaluated as looking good on the subject, and when an evaluator gives an annotation indicating "does not look good", the clothing in the image is evaluated as not looking good on the subject. For this reason, the evaluator may be considered to have given the annotation. In addition, for example, the image includes a poster's tag (for example, information entered by the poster when posting), and the wearer's physical information can be obtained from the poster's tag. In addition, more detailed body measurements may be obtained using a body measurement device as body information of the wearer or evaluator. For example, body measurements may be obtained based on a captured image of the wearer wearing body measurement clothing or body measurement glasses.

[0159] For example, the information processing device 10A provides an electronic commerce service that provides clothing. The information processing device 10A also provides a service (which may be a coordination service) that accepts posts of content showing clothing (which may be content showing coordination) from users and provides the content to other users.

[0160] The information processing device 10A may have a function as a web server that provides a website related to the service. The information processing device 10A may also be a device that distributes information to be displayed on applications related to various services installed in the user terminal 100 to the user terminal 100. The information processing device 10A may also be a device that distributes the application data itself.

[0161] Furthermore, the information processing device 10A may function as a distribution device that distributes control information to the user terminal 100. Here, the control information is described in, for example, a script language such as JavaScript or a style sheet language such as CSS. Note that the application itself distributed from the information processing device 10A may be regarded as the control information.

[0162] The user terminal 100 is an information processing device used by a user. The user terminal 100 displays information distributed by the information processing device 10A or a server device that provides a predetermined service, using a web browser or an application. In the example shown in FIG. 14, the user terminal 100 is a smartphone.

[0163] The evaluator terminal 200 is an information processing device used by an evaluator who evaluates an image of a wearer wearing clothes (or a combination of clothes) (evaluation indicating whether the clothes look good on the wearer). The evaluator terminal 200 is also an information processing device used by an evaluator who evaluates whether the clothes (or a combination of clothes) look good on the evaluator (e.g., evaluation indicating whether the clothes look good on the evaluator when imagining the evaluator wearing the clothes). The evaluator terminal 200 displays information distributed by the information processing device 10A or a server device that provides a predetermined service, using a web browser or an application. Note that the example shown in FIG. 14 shows a case where the evaluator terminal 200 is a smartphone. The evaluator performs evaluation by adding annotations to the image. When the evaluator performs evaluation, the evaluator terminal 200 transmits evaluation information. Then, for example, the information is stored in an evaluator information database 31A described later.

[0164] [8. An example of information processing] Next, an example of information processing implemented by the information processing device according to the present embodiment will be described with reference to FIG.

[0165] Hereinafter, the third information processing realized by the information processing device etc. according to the present embodiment will be described with reference to Fig. 14. Fig. 14 is a diagram showing an example of the information processing according to the embodiment.

[0166] First, the information processing device 10A receives an evaluator's evaluation of an image of clothing (or a coordinated outfit) from the evaluator terminal 200 (step Sc1). For example, the information processing device 10A receives an evaluation indicating whether the clothing in the image suits the wearer of the clothing. As a specific example, the information processing device 10A presents a pair of "suits" and "doesn't suit" to the evaluator and receives an evaluation indicating whether the clothing suits the wearer. The evaluation may be an evaluation regarding the style of the clothing, such as "it would be better to roll up the length," "if the person is tall, it would be better to roll up the length," or "if the person is short, it would be better not to roll up the length." The evaluation may also be an evaluation regarding the style of the clothing coordinated with other items.

[0167] Furthermore, the information processing device 10A is not limited to an evaluation indicating whether the clothes suit the wearer, but also accepts an evaluation indicating whether the clothes suit the evaluator. For example, the information processing device 10A determines whether the image includes a person wearing the clothes, and if the image does not include a wearer, accepts an evaluation indicating whether the clothes (or a coordinated outfit) suit the evaluator. In this case, the information processing device 10A presents a pair of "suits" and "doesn't suit" to the evaluator, and accepts an evaluation indicating whether the clothes suit the evaluator. For example, the evaluation is accepted by adding an annotation indicating the evaluation to the image. The evaluation may be an evaluation regarding the style of the clothes, such as "if it were me, I would prefer to roll up the length" or "if it were me, I would prefer not to roll up the length". The evaluation may also be an evaluation regarding the style of the clothes coordinated with other items.

[0168] In this manner, the information processing device 10A may receive an evaluation indicating whether the clothing suits the evaluator himself / herself. For the sake of convenience, the following description will be given taking as an example a case in which an evaluation indicating whether the clothing suits a wearer is received.

[0169] The information processing device 10A accepts from the evaluator A1 an evaluation that the clothes (or coordinated outfits) #1 shown in the image P1 look good on the wearer #1 shown in the image P1, and an evaluation that the clothes #2 shown in the image P2 do not look good on the wearer #2 shown in the image P2 (i.e., an evaluation that "looks good" for the image P1 and an evaluation that "does not look good" for the image P2). If the clothes #1 shown in the image P1 look good on the wearer #1 shown in the image P1 and the clothes #2 shown in the image P2 look good on the wearer #2 shown in the image P2 are accepted when the clothes #1 look good on the wearer #1 shown in the image P1 and the clothes #2 shown in the image P2 look good on the wearer #2 shown in the image P2 are accepted when the clothes #1 look bad on the wearer #1 shown in the image P1 and the clothes #2 shown in the image P2 do not ... are accepted when the clothes #2 look bad on the wearer #2 shown in the image P2 are accepted. Furthermore, information processing device 10A receives from evaluator A2 an evaluation that clothing #1 shown in image P1 does not suit wearer #1 shown in image P1, and an evaluation that clothing #2 shown in image P2 suits wearer #2 shown in image P2 (i.e., an evaluation that "does not suit" image P1 and an evaluation that "suits" image P2). Similarly, information processing device 10A receives evaluations regarding images P1 and P2 from other evaluators. Furthermore, information processing device 10A may receive evaluations regarding other images (such as images P3 and P4) without being limited to images P1 and P2.

[0170] Next, the information processing device 10A generates a distributed representation space in which the image and the physical information indicating the physical characteristics of the subject (when the evaluation is made on the wearer, the wearer is the subject, and when the evaluation is made on the evaluator himself, the evaluator is the subject) are projected based on the evaluator's evaluation of the image (step Sc2). The distributed representation space includes, for example, a pair of the image and the physical information of the wearer in which the evaluation is made on the wearer, and a pair of the image and the physical information of the evaluator in which the evaluation is made on the evaluator himself. For example, the information processing device 10A generates the distributed representation space using the VSE technology. As a specific example, the information processing device 10A generates a distributed representation space F6 in which the image P1 is projected closer to the physical information E1 of the wearer #1 evaluated as "suitable" by the evaluator A1 than the physical information E2 of the wearer #2 evaluated as "not suitable" by the evaluator A1. Furthermore, the information processing device 10A generates a distributed representation space F6 in which the image P2 is projected closer to the physical information E2 of the wearer #2 evaluated as “suitable” by the evaluator A2 than to the physical information E1 of the wearer #1 evaluated as “unsuitable” by the evaluator A2. Note that the information processing device 10A may generate a distributed representation space in which various information related to the image and physical information indicating various information related to the wearer's body are projected.

[0171] Next, the information processing device 10A receives a request for providing predetermined information from the user terminal 100 via an electronic commerce service or a coordinate service (step Sc3). Specifically, the information processing device 10A receives a request for providing information on physical features that go well with predetermined clothes (or coordinated outfits). For example, when the user U1 specifies clothes on a service such as an electronic commerce service or a coordinated outfit, the information processing device 10A obtains the specified clothing information from the user terminal 100 and receives a request for providing information on physical features that go well with the clothes.

[0172] Then, in response to the request for provision, the information processing device 10A estimates physical features that go well with the specified clothes based on the generated distributed representation space and the information about the clothes specified by the user U1 (step Sc4). Note that the information processing device 10A may estimate a position when the information about the clothes specified by the user U1 is projected into the distributed representation space based on the information projected into the distributed representation space and the information about the clothes specified by the user U1, and estimate physical features that go well with the specified clothes based on the estimated position.

[0173] 14, it is assumed that the information processing device 10A has determined that the estimated position is close to the positions of clothing #1 shown in image P1, clothing #3 shown in image P3, and clothing #4 shown in image P4. In this case, the information processing device 10A provides the user terminal 100 with information on physical features that go well with the specified clothing, based on the information on clothing #1, #3, and #4 (step Sc5).

[0174] (Information provided to users) In step Sc5, the information processing device 10A may provide the user U1 with information including, for example, information on skin color, hair color, hairstyle, face shape, makeup, and the like, as well as information on body type. The information processing device 10A may provide information by identifying the closest items in the distributed representation space (for example, one from each item), or may provide information by identifying all items included in a predetermined range in the distributed representation space (note that information may be provided in order of proximity in the distributed representation space, or information may be provided on all items included in a predetermined range in the distributed space as equal). The information processing device 10A may provide further services by providing information according to information on estimated physical features.

[0175] Here, indices indicating physical features such as personal color, skeletal type, and face type are hereinafter referred to as "physical feature axes" as appropriate. Specific values ​​corresponding to the physical feature axes are hereinafter referred to as "physical feature values" as appropriate. For example, the physical feature value corresponding to the physical feature axis of the "skeletal type" of user U1 is "straight". A snap image or an item-by-item image of a plurality of items constituting a coordination (or a single item-by-item image) is hereinafter referred to as "coordinate image" as appropriate. The information processing device 10A may provide a service that outputs physical features that suit a coordination image when user U1 specifies the coordination image on a service such as an electronic commerce service or a coordination service. In this case, the physical features may be expressed as a set of a physical feature axis and a physical feature value. The physical feature axis may be expressed as a set of expected physical feature values.

[0176] Based on the type diagnosis in the image consulting theory, the personal color is classified as "spring, summer, autumn, winter", the bone structure type is classified as "straight, wavy, natural", and the face type is classified as "cool, cool casual, fresh, soft elegant, elegant, cute, active cute, feminine" to set the physical feature value. The design of the "physical feature axis" and the "physical feature value" according to the image consulting theory and the like described above is merely an example, and may be added or changed as necessary. The physical feature value is also set by classifying the hair color and makeup style based on predetermined information. The physical feature value is not limited to being based on a classification, and may be a continuous value. The physical feature value corresponding to the physical feature axis may be output as a distribution. For example, information such as "straight: 70%, wavy: 20%, natural: 10%" may be output for the physical feature axis of "bone structure type". For example, the information processing device 10A may provide such information according to the closeness between the image in the distributed representation space and the physical feature (each physical feature value). In this way, the user can learn what combination of fashion items suits the user's physical characteristics, in addition to finding products that match their physical characteristics through the use of the service. Therefore, the user can deepen their understanding of fashion that matches their physical characteristics, which may help to eliminate their aversion to fashion and promote their desire to purchase. In addition, when viewing outfits on a specific service or when products or outfits are recommended through email distribution, information such as the physical characteristics that go well with the items or outfits being viewed or recommended may be output.

[0177] (About what to do when a user specifies a coordination that has never been seen before) In step Sc4, for example, when the user U1 specifies an unprecedented coordination (not in the learning data), the information processing device 10A may estimate physical features that suit the coordination. For example, for a coordination not in the learning data, the information processing device 10A may learn a mapping (model) for projecting a coordination image into a VSE space in which physical feature values ​​are projected, project an unknown coordination into the VSE space using the mapping, and output physical feature values ​​that exist in the vicinity of an expression corresponding to the coordination image. At this time, a neighborhood search may be performed independently for each physical feature axis. Also, the VSE space may be configured separately for each physical feature axis. In that case, the information processing device 10A may learn a mapping for projecting a coordination image individually. A method for learning the mapping will be described below.

[0178] It is assumed that a set of coordinated images that an annotator has evaluated as "suitable" and physical feature values ​​(tags) is stored as a data set. Here, the physical feature values ​​may be those of the subject captured in the coordinated image. They may also be more detailed body measurements taken using a body measurement device. The information processing device 10A may use both the physical feature values ​​and the body measurements to construct an embedding vector related to the body of the poster of the coordinated image. The annotator may be the poster of the coordinated image or a third party other than the poster.

[0179] Furthermore, when the coordinated image is a snap image, the information processing device 10A may learn using a method similar to the method of embedding areas corresponding to fashion items in the snap image into the VSE space. Specifically, the information processing device 10A may cut out image areas corresponding to each item in the snap image, and project a vector combining image features of each image area into the VSE space.

[0180] Furthermore, when the coordinate image is made up of a plurality of individual item images, the information processing device 10A may embed the images into the VSE space by constructing a single embedding vector from the embedding vectors of the individual item images using a set transformer or the like. The information processing device 10A may also project a vector combining the embeddings of the individual item images into the VSE space. The information processing device 10A may also construct a single embedding vector by averaging the feature vectors of the plurality of individual item images.

[0181] (Processing when the presented physical characteristics do not match the user's physical characteristics) In step Sc5, if the physical characteristics presented to the user U1 do not match the physical characteristics of the user U1, the information processing device 10A may suggest a change to the item designated by the user U1. At this time, the information processing device 10A may also suggest which item should be changed. The information processing device 10A may perform different processing depending on whether the user U1 judges the presented information to be true (correct) or false (incorrect). The information processing device 10A may automatically determine whether the information is true or false. If the user U1 judges the presented information to be true, the information processing device 10A may, for example, encourage the user U1 to change his or her body within a realistic range. For example, the information processing device 10A may provide feedback to the user U1 saying, "Let's exercise" if "if you lose weight, the system's evaluation value will improve (you will have physical characteristics that suit the designated item)." In such a case, the information processing device 10 may present the presented physical features (physical features that suit the specified items) in a manner that can be set as target physical information indicating the physical features that the user aims for in a predetermined body management service provided by the information processing device 10 (a service that periodically acquires information indicating the user's physical features from a captured image of the user and supports the user's body shape management, etc.). In addition, the information processing device 10A may randomly change, for example, only "tops" out of the coordination images currently input by the user U1 in proposing a change in items, and may propose a combination of items that will have a higher evaluation value than the current combination (that will suit the user's physical features). In this case, the number of attempts to randomly change the items may be set by the system designer, or may be repeated until an item with a higher evaluation value than the current combination is found. In addition, the information processing device 10A may perform similar processing on items other than "tops". Note that the information processing device 10A may estimate items that the user is likely not to want to change (for example, an item specified first, an item specified multiple times, etc.), and perform similar processing only on the other items.Moreover, the information processing device 10A may search for a destination of the item by performing a neighborhood search in the VSE space using the physical information of the user U1, instead of changing it randomly. For example, the information processing device 1 may determine a destination of the item based on an item corresponding to an image located near the physical information of the user U1 in the VSE space. Furthermore, when the user U1 judges that the presented information is false, the information processing device 10A may, for example, collect feedback from the user U1 and update the system if there is rationality. For example, when the confidence level of the output of the system is lower than a predetermined threshold, the information processing device 10A may accept feedback from the user U1 and update the VSE space. At this time, the information processing device 10A may interactively obtain feedback from the user U1 using a method such as active learning. Note that any method of calculating the confidence level may be used. Furthermore, the information processing device 10A may calculate the confidence level to be low when the number of learning data similar to the physical information of the user U1 is less than a certain amount.

[0182] 9. Configuration of Information Processing Device Next, the configuration of the information processing device 10A will be described with reference to Fig. 15. Fig. 15 is a diagram showing an example of the configuration of the information processing device 10A according to the embodiment. As shown in Fig. 15, the information processing device 10A has a communication unit 20, a storage unit 30A, and a control unit 40A. Note that the communication unit 20 is the same as in the first embodiment, and therefore a description thereof will be omitted.

[0183] (Regarding the memory unit 30A) The storage unit 30A is realized by, for example, a semiconductor memory element such as a RAM or a flash memory, or a storage device such as a hard disk or an optical disk. As shown in Fig. 15, the storage unit 30A has an evaluator information database 31A and a user information database 32A.

[0184] (Regarding Evaluator Information Database 31A) The evaluator information database 31A stores various types of information related to evaluators. An example of information stored in the evaluator information database 31A will now be described with reference to FIG. 16. FIG. 16 is a diagram showing an example of the evaluator information database 31A according to an embodiment. In the example of FIG. 16, the evaluator information database 31A has items such as "evaluator ID," "evaluator information," "questionnaire information," and "evaluation information."

[0185] The "evaluator ID", "questionnaire information", and "evaluation information" are the same as those in the first embodiment, and therefore the explanation will be omitted. The "evaluator information" indicates information related to the evaluator, and for example, information such as attribute information and physical information of the evaluator is stored.

[0186] That is, Figure 16 shows an example in which the evaluator information of the evaluator identified by the evaluator ID "AID#1" is "evaluator information #1", the questionnaire information is "questionnaire information #1", the image information of the subject of evaluation identified by the subject ID "DID#1" is "image information #1", the clothing information is "clothing information #1", the wearer information is "wearer information #1", and the evaluation is "evaluation #1".

[0187] (Regarding user information database 32A) The user information database 32A stores various types of information related to users. An example of information stored in the user information database 32A will now be described with reference to FIG. 17. FIG. 17 is a diagram showing an example of the user information database 32A according to an embodiment. In the example of FIG. 17, the user information database 32A has items such as "user ID," "user information," "survey information," "user image," and "feedback information."

[0188] The "user ID", "questionnaire information", "user image", and "feedback information" are the same as those in the first embodiment, and therefore the description will be omitted. The "user information" indicates information about the user, and for example, information such as the user's attribute information and physical information is stored.

[0189] That is, Figure 17 shows an example in which the user information of a user identified by user ID "UID#1" is "user information #11", the survey information is "survey information #11", the user image is "user image #1", and the feedback information is "feedback information #1".

[0190] (Regarding the control unit 40A) The control unit 40A is a controller, and is realized, for example, by a CPU, an MPU, or the like, executing various programs stored in a storage device inside the information processing device 10A using a RAM as a working area. The control unit 40A is also realized, for example, by an integrated circuit such as an ASIC or an FPGA. As shown in FIG. 15, the control unit 40A according to the embodiment has a receiving unit 41A, a generating unit 42A, an estimating unit 43A, a providing unit 44A, a determining unit 45, an output unit 46, an acquiring unit 47A, and an updating unit 48A, and realizes or executes the functions and actions of the information processing described below. Note that the determining unit 45 and the output unit 46 are the same as those in the first embodiment, and therefore description thereof will be omitted.

[0191] (Regarding reception section 41A) The receiving unit 41A receives information similar to that of the receiving unit 41. The receiving unit 41A also receives evaluations from a plurality of evaluators on an image of a wearer wearing clothes, which indicates whether the clothes suit the wearer. For example, in the example of FIG. 14, the receiving unit 41A presents an image P1, etc. to an evaluator, receives an evaluation for each image indicating whether the clothes in the image suit the wearer, and stores the evaluation in the evaluator information database 31A. The receiving unit 41A also receives evaluations from a plurality of evaluators on an image of clothes, which indicates whether the clothes suit the evaluator himself / herself. For example, in the example of FIG. 14, the receiving unit 41A presents an image P1, etc. to an evaluator, receives an evaluation for each image indicating whether the clothes in the image suit the evaluator himself / herself, and stores the evaluation in the evaluator information database 31A. In addition, the receiving unit 41A may determine whether or not there is a wearer for each image, and if there is a wearer, receive an evaluation from the evaluator indicating whether or not the clothing suits the wearer, and if there is no wearer, receive an evaluation from the evaluator indicating whether or not the clothing suits the evaluator himself / herself.

[0192] (Regarding the generation unit 42A) The generating unit 42A generates information similar to that of the generating unit 42. The generating unit 42A also generates a distributed representation space onto which an image and physical information indicating the physical characteristics of the subject (when the evaluator has evaluated the wearer, the wearer is the subject, and when the evaluator has evaluated himself / herself, the evaluator is the subject) are projected based on the evaluation received by the receiving unit 41A. For example, in the example of Fig. 14, the generating unit 42A refers to the evaluator information database 31A and generates a distributed representation space onto which an image and physical information indicating the physical characteristics of the subject are projected using the VSE technology.

[0193] Furthermore, the generation unit 42A may project the image and the body information of the wearer (or the evaluator) closer in the distributed representation space as the evaluator evaluates the clothing as being suitable for the wearer (or suitable for the evaluator himself / herself), and may project the image and the body information of the wearer (or the evaluator) farther in the distributed representation space as the evaluator evaluates the clothing as being unsuitable for the wearer (or suitable for the evaluator himself / herself). For example, in the example of Fig. 14, the generation unit 42A generates a distributed representation space F6 in which the image P1 is projected closer to the body information E1 of the wearer #1 of the image P1 evaluated as "suitable" by the evaluator A1 than the body information E2 of the wearer #2 of the image P2 evaluated as "unsuitable" by the evaluator A1. In addition, the generation unit 42A generates a distributed representation space F6 in which the image P2 is projected closer to the physical information E2 of the wearer #2 of the image P2, which the evaluator A2 evaluated as “suitable”, than to the physical information E1 of the wearer #1 of the image P1, which the evaluator A2 evaluated as “unsuitable”.

[0194] (Regarding the estimation unit 43A) The estimation unit 43A estimates physical features that suit the clothes specified by the user based on the distributed representation space and information about the clothes specified by the user. For example, in the example of FIG. 14, the estimation unit 43A estimates physical features that suit the specified clothes based on the distributed representation space generated in response to a request for providing predetermined information from the user terminal 100 via an electronic commerce service or a coordinate service and information about the clothes specified by the user U1. The estimation unit 43A also estimates physical features that suit the specified clothes based on a distributed representation space in which an image of a wearer wearing the clothes is projected and the physical information of the wearer is generated based on evaluations from multiple evaluators indicating whether the clothes suit the wearer. The estimation unit 43A also estimates physical features that suit the specified clothes based on a distributed representation space in which an image of the clothes is projected and the physical information of the evaluator is generated based on evaluations from multiple evaluators indicating whether the clothes suit the evaluator himself or herself. In addition, the estimation unit 43A estimates physical features that suit a specified garment based on a distributed representation space into which the image and the subject's physical information are projected, the space being generated based on evaluations from multiple evaluators of an image of a wearer wearing the garment, indicating whether the garment suits the wearer, and evaluations from the multiple evaluators of an image of the garment, indicating whether the garment suits the evaluator themselves.

[0195] (About the provider 44A) The providing unit 44A provides the user with information on the physical features estimated by the estimating unit 43A. For example, in the example of Fig. 14, the providing unit 44A provides information on the physical features that go well with the specified clothes in an electronic commerce service. The providing unit 44A also provides information on the physical features that go well with the specified clothes in a coordination service.

[0196] (Regarding Acquisition Unit 47A) The acquisition unit 47A acquires feedback from the user or other users having a predetermined relationship with the user with respect to the information on the physical features provided by the provision unit 44A. For example, in the example of FIG. 14, the acquisition unit 47A acquires information (feedback) indicating whether the user U1 has purchased clothes #1, #3, and #4 or applied for rental in the electronic commerce service, and stores the information in the user information database 32A. The acquisition unit 47A also acquires information (feedback) indicating whether the user U1 has registered clothes #1, #3, and #4 as favorites in the coordination service. For example, when the physical features that suit the clothes designated by the user U1 match the physical features of the user, the acquisition unit 47A may acquire information (feedback) indicating whether the user U1 has purchased the designated clothes or applied for rental in the electronic commerce service, and store the information in the user information database 32A. The acquisition unit 47A may also acquire information (feedback) indicating whether the user U1 has registered the designated clothes in the coordination service as favorites. The acquisition unit 47A also acquires information (feedback) indicating whether the target person was positive about the user U1 wearing the clothes indicated by the provided information from the user terminal 100, a terminal device used by the target person, or the like. The acquisition unit 47A may also acquire information (feedback) indicating whether the target person was positive about the user U1 wearing the clothes specified by the user U1 (whether the target person evaluated the clothes as suiting the user U1 (physical features)) from the user terminal 100, a terminal device used by the target person, or the like. The acquisition unit 47A also acquires information (feedback) indicating whether the target person was positive about the clothes indicated by the provided information when the user U1 presented the clothes to the target person as a present from the user terminal 100, a terminal device used by the target person, or the like. The acquisition unit 47A may also acquire information (feedback) indicating whether the target person was positive about the clothes specified by the user U1 (whether the target person evaluated the clothes as suiting the target person (physical features)) or the like when the physical features that suit the clothes specified by the user U1 match the physical features of the target person to whom the present is to be presented.

[0197] (Regarding update section 48A) The update unit 48A updates the distributed representation space based on the feedback acquired by the acquisition unit 47A. When the user U1 is positive about the clothes #1, the update unit 48A updates the physical information E1 and the image P1 in the distributed representation space F6 so as to bring them closer to each other by a predetermined distance. On the other hand, when the user U1 is negative about the clothes #3, the update unit 48A updates the physical information E3 and the image P3 in the distributed representation space F6 so as to move them apart by a predetermined distance. For example, the update unit 48A acquires information indicating whether the user U1 has purchased the clothes specified by the user U1 or has applied for rental, and updates the physical information of the user U1 and the information indicating the clothes specified by the user U1 so as to bring them closer to each other by a predetermined distance based on such feedback. Also, for example, the update unit 48A acquires information indicating whether the target person was positive about the user U1 wearing the clothes specified by the user U1, and updates the physical information of the user U1 and the information indicating the clothes specified by the user U1 so as to bring them closer to each other by a predetermined distance based on such feedback. Also, for example, the update unit 48A obtains information indicating whether the target person for the gift of the clothing specified by the user U1 was positive or not, and based on such feedback, updates the target person's physical information and the information indicating the clothing specified by the user U1 to bring them closer by a predetermined distance.

[0198] Furthermore, when the subject has a positive opinion of user U1 wearing the clothes indicated by the provided information, the update unit 48A updates the information corresponding to the subject (e.g., physical information, answers to a questionnaire, etc.) and the image showing the clothes to move them closer to each other by a predetermined distance in the distributed representation space. On the other hand, when the subject has a negative opinion of user U1 wearing the clothes indicated by the provided information, the update unit 48A updates the information corresponding to the subject and the image showing the clothes to move them apart by a predetermined distance in the distributed representation space.

[0199] Furthermore, when user U1 presents the clothes indicated in the information provided to user U1 to a target person, if the target person is positive, the update unit 48A updates the information corresponding to the target person and the image showing the clothes to move closer to each other by a predetermined distance in the distributed representation space. On the other hand, when user U1 presents the clothes indicated in the information provided to user U1 to a target person, if the target person is negative, the update unit 48A updates the information corresponding to the target person and the image showing the clothes to move apart by a predetermined distance in the distributed representation space.

[0200] [10. Information processing flow] The information processing procedure (4) of the information processing device 10A according to the embodiment will be described with reference to Fig. 18. Fig. 18 is a flowchart (4) showing an example of the information processing procedure according to the embodiment.

[0201] 18, the information processing device 10A determines whether or not an evaluation for a clothing image has been received (step S401). If the evaluation has not been received (step S401; No), the information processing device 10A waits until the evaluation is received.

[0202] On the other hand, if an evaluation is received (step S401; Yes), the information processing device 10A generates a distributed representation space into which the image and physical information indicating the physical characteristics of the subject are projected based on the evaluation (step S402), and terminates the processing.

[0203] Next, the information processing procedure (5) of the information processing device 10A according to the embodiment will be described with reference to Fig. 19. Fig. 19 is a flowchart (5) showing an example of the information processing procedure according to the embodiment.

[0204] 19, the information processing device 10A determines whether or not a request for providing predetermined information has been received (step S501). If the request for providing information has not been received (step S501; No), the information processing device 10A waits until the request for providing information is received.

[0205] On the other hand, if a request for provision is received (step S501; Yes), the information processing device 10A estimates physical features that suit a specific outfit based on a distributed representation space in which an image of the outfit, generated based on the evaluation by the evaluator, and physical information indicating the physical features of the subject are projected (step 502). Next, the information processing device 10A provides the user with information on the estimated physical features (step S503), and ends the process.

[0206] 11. Effects As described above, the information processing device 10A according to the embodiment includes an estimation unit 43A and a provision unit 44A. The estimation unit 43A estimates physical features that suit a specific piece of clothing based on a distributed representation space in which an image of the clothing and physical information indicating the physical features of a subject are projected, the distributed representation space being generated based on evaluations from a plurality of evaluators indicating whether the clothing suits the subject or not, as an evaluation of an image of the clothing. The provision unit 44A provides the user with information on the physical features estimated by the estimation unit 43A.

[0207] As a result, the information processing device 10A according to the embodiment can estimate physical features that go well with a given piece of clothing by constructing a space that can measure the closeness between an image of the clothing and the subject's physical information. In addition, the information processing device 10A according to the embodiment can provide information on physical features that are estimated to go well with a given piece of clothing by using the generated distributed representation space, thereby achieving an effect that the user can grasp physical features that go well with a given piece of clothing.

[0208] Furthermore, the estimation unit 43A performs estimation based on a distributed representation space that is generated based on evaluations from evaluators indicating whether the clothes suit the wearer.

[0209] As a result, the information processing device 10A of the embodiment enables a user to grasp the physical features that suit a specific piece of clothing based on a distributed representation space generated based on the subjective opinion of each evaluator indicating whether the clothing suits the wearer or not.

[0210] Furthermore, the estimation unit 43A performs estimation based on a distributed representation space onto which physical information of the wearer, who is the subject, is projected.

[0211] As a result, the information processing device 10A according to the embodiment allows the user to grasp physical features that go well with a given piece of clothing, based on the distributed representation space onto which the wearer's physical information is projected.

[0212] Moreover, the estimation unit 43A performs estimation based on a distributed representation space that is generated based on evaluations from evaluators indicating whether or not the outfit suits the evaluator.

[0213] As a result, the information processing device 10A of the embodiment enables a user to grasp the physical features that suit a specific piece of clothing based on a distributed representation space that is generated based on each evaluator's subjective opinion indicating whether the clothing suits the evaluator or not.

[0214] Moreover, the estimation unit 43A performs estimation based on a distributed representation space onto which the physical information of the evaluator, who is the subject, is projected.

[0215] As a result, the information processing device 10A according to the embodiment allows the user to grasp physical features that go well with a given piece of clothing, based on the distributed representation space onto which the physical information of the evaluator is projected.

[0216] In addition, when the image includes a person wearing the clothes, the estimation unit 43A makes an estimate based on a distributed representation space generated based on evaluations from evaluators indicating whether the clothes look good on the wearer, and when the image does not include a person wearing the clothes, the estimation unit 43A makes an estimate based on a distributed representation space generated based on evaluations from evaluators indicating whether the clothes look good on the evaluator.

[0217] As a result, the information processing device 10A of the embodiment allows a user to understand the physical features that suit a specific piece of clothing based on a distributed representation space into which the subject's physical information is projected depending on whether or not the image includes a person wearing the clothing.

[0218] Furthermore, the estimation unit 43A estimates physical features that go well with the clothes designated by the user, based on the distributed representation space.

[0219] In this way, the information processing device 10A according to the embodiment allows the user to grasp physical features that go well with the clothes designated by the user.

[0220] The estimation unit 43A estimates physical features that suit a specific clothing combination based on a distributed representation space in which the image and physical information indicating the physical features of the subject are projected, the distributed representation space being generated based on evaluations of the image of the clothing combination from a plurality of evaluators indicating whether the clothing combination suits the subject or not. The provision unit 44A provides the user with information on the physical features estimated by the estimation unit 43A.

[0221] As a result, the information processing device 10A according to the embodiment can estimate physical features that go well with a given clothing combination by constructing a space that can measure the closeness between a coordinated image and physical information of a subject. Also, the information processing device 10A according to the embodiment can provide information on physical features that are estimated to go well with a given clothing combination by using the generated distributed representation space, thereby achieving an effect that a user can grasp physical features that go well with a given clothing combination.

[0222] In addition, the estimation unit 43A estimates physical features that suit a specified clothing style based on a distributed representation space that is generated based on evaluations from evaluators indicating whether the clothing style suits the image, which are evaluations of images showing different clothing styles.

[0223] As a result, the information processing device 10A according to the embodiment allows the user to grasp physical features that go well with a given outfit, based on the distributed representation space that is generated by adding the outfit to the evaluations of each evaluator.

[0224] [12. Hardware Configuration] Moreover, the information processing device 10A according to each embodiment described above is realized, for example, by a computer 1000 configured as shown in Fig. 20. The information processing device 10A will be described below as an example. Fig. 20 is a hardware configuration diagram showing an example of a computer that realizes the functions of the information processing device 10A. The computer 1000 has a CPU 1100, a ROM 1200, a RAM 1300, a HDD 1400, a communication interface (I / F) 1500, an input / output interface (I / F) 1600, and a media interface (I / F) 1700.

[0225] The CPU 1100 operates and controls each unit based on a program stored in the ROM 1200 or the HDD 1400. The ROM 1200 stores a boot program executed by the CPU 1100 when the computer 1000 is started up, programs that depend on the hardware of the computer 1000, and the like.

[0226] The HDD 1400 stores programs executed by the CPU 1100, data used by such programs, etc. The communication interface 1500 receives data from other devices via a communication network 500 (corresponding to the network N in the embodiment) and sends the data to the CPU 1100, and also transmits data generated by the CPU 1100 to other devices via the communication network 500.

[0227] The CPU 1100 controls output devices such as a display and a printer, and input devices such as a keyboard and a mouse, via the input / output interface 1600. The CPU 1100 acquires data from the input devices via the input / output interface 1600. The CPU 1100 also outputs data generated via the input / output interface 1600 to the output devices.

[0228] Media interface 1700 reads a program or data stored in recording medium 1800 and provides it to CPU 1100 via RAM 1300. CPU 1100 loads the program from recording medium 1800 onto RAM 1300 via media interface 1700 and executes the loaded program. Recording medium 1800 is, for example, an optical recording medium such as a DVD (Digital Versatile Disc) or a PD (Phase change rewritable Disc), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory.

[0229] For example, when the computer 1000 functions as the information processing device 10A, the CPU 1100 of the computer 1000 executes a program loaded on the RAM 1300 to realize the function of the control unit 40A. In addition, the HDD 1400 stores each piece of data in the storage device of the information processing device 10A. The CPU 1100 of the computer 1000 reads and executes these programs from the recording medium 1800, but as another example, these programs may be obtained from another device via a predetermined communication network.

[0230] [13. Others] Although some of the embodiments of the present application have been described in detail above with reference to the drawings, these are merely examples, and the present invention can be embodied in other forms that incorporate various modifications and improvements based on the knowledge of those skilled in the art, including the forms described in the Disclosure of the Invention section.

[0231] Further, the information processing device 10A described above can flexibly change its configuration, for example, by calling an external platform or the like using an API (Application Programming Interface) or network computing to realize some functions.

[0232] Furthermore, the term "unit" in the claims may be read as "means" or "circuit," etc. For example, a reception unit may be read as a reception means or a reception circuit. [Explanation of symbols]

[0233] 10, 10A Information processing device 20 Communications Department 30, 30A storage section 31, 31A Evaluator Information Database 32, 32A User Information Database 40, 40A Control section 41, 41A Reception 42, 42A generation section 43, 43A Estimation part 44, 44A supply department 45 Judgment section 46 Output section 47, 47A Acquisition Department 48, 48A update section 100 User terminals 200 Evaluator terminal

Claims

1. an estimation unit that estimates physical features that go well with a specific piece of clothing based on a distributed representation space onto which the image of the clothing and physical information indicating the physical features of the subject are projected, the distributed representation space being generated based on evaluations of the clothing image from a plurality of evaluators indicating whether the clothing looks good on the subject; a providing unit that provides a user with information about the physical characteristics estimated by the estimating unit; 13. An information processing device comprising:

2. The estimation unit is The estimation is based on the distributed representation space generated based on the evaluations from the evaluators indicating whether the clothing suits the wearer.

2. The information processing apparatus according to claim 1,

3. The estimation unit is The body information of the wearer, who is the subject, is estimated based on the projected distributed representation space.

3. The information processing apparatus according to claim 2.

4. The estimation unit is The estimation is based on the distributed representation space generated based on the evaluation from the evaluator indicating whether the suit suits the evaluator.

2. The information processing apparatus according to claim 1,

5. The estimation unit is The physical information of the evaluator, who is the subject, is estimated based on the distributed representation space onto which the physical information is projected.

3. The information processing apparatus according to claim 2.

6. The estimation unit is When the image includes a person wearing the clothing, the estimation is made based on the distributed representation space generated based on an evaluation from the evaluator indicating whether the clothing suits the wearer of the clothing, and when the image does not include a person wearing the clothing, the estimation is made based on the distributed representation space generated based on an evaluation from the evaluator indicating whether the clothing suits the evaluator.

2. The information processing apparatus according to claim 1,

7. The estimation unit is Based on the distributed representation space, physical features that go well with the clothes designated by the user are estimated.

2. The information processing apparatus according to claim 1,

8. an estimation unit that estimates physical features that suit a specific clothing combination based on a distributed representation space onto which the image and physical information indicating the physical features of a subject are projected, the distributed representation space being generated based on evaluations of an image of a clothing combination from a plurality of evaluators indicating whether the clothing combination suits the subject; a providing unit that provides a user with information about the physical characteristics estimated by the estimating unit; 13. An information processing device comprising:

9. The estimation unit is The distributed representation space is generated based on evaluations of the images in which the clothes are worn in different styles, the evaluations being from the evaluators indicating whether the clothes look good on the person, and physical features that go well with a specific clothing style are estimated based on the distributed representation space.

9. The information processing apparatus according to claim 8,

10. 1. A computer-implemented information processing method, comprising: an estimation step of estimating physical features that go well with a specific piece of clothing based on a distributed representation space onto which the image of the clothing and physical information indicating the physical features of the subject are projected, the distributed representation space being generated based on evaluations of the clothing image from a plurality of evaluators indicating whether the clothing looks good on the subject or not; a providing step of providing a user with information regarding the physical characteristics estimated by the estimating step; 13. An information processing method comprising:

11. an estimation step of estimating physical features that suit a specific piece of clothing based on a distributed representation space onto which the image of the clothing and physical information indicating the physical features of a subject are projected, the distributed representation space being generated based on evaluations of the clothing image from a plurality of evaluators indicating whether the clothing suits the subject or not; a provision step of providing information about the physical characteristics estimated by the estimation step to a user; An information processing program characterized by causing a computer to execute the above.

Images