Information processing apparatus, method and program

Abstract

To learn a model that receives input of evaluation and outputs a new evaluation object with a small number of times of learning.SOLUTION: An information processing apparatus according to an embodiment comprises: an input unit that receives input of a new evaluation on an evaluation object to a model which receives input of an evaluation based on a user's subjective view on an evaluation object and outputs a new evaluation object for the user; an identification unit that identifies a part of the model that is associated with a viewpoint of evaluation on the evaluation object as an object of learning; a learning unit that learns a parameter for the identified part of the model in accordance with a result of input of the evaluation; and a selection unit that selects a result which is output from the learned model using the result of the evaluation as input, as a new evaluation object presented to the user.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] FIELD Embodiments of the present invention relate to an information processing device, method, and program. [Background technology]

[0002] Personalization is widely used in fields where it is easy to create systems for collecting personal data. For example, products, content, services, and advertisements are personalized based on users' browsing, playback, click operations, purchase history, and evaluation logs.

[0003] On the other hand, personalization is not common in fields where it is difficult to create a system for collecting personal data. For example, personalization based on how a user feels about something, such as estimating emotions or likeability based on a user's facial expression, gestures, or voice, or evaluating the aroma, taste, or deliciousness of food or drink using the five senses, is not common.

[0004] If such personalization based on user subjective data becomes possible, it could be used to, for example, convert facial expressions to match user A's emotion recognition characteristics to accurately convey emotions, or to prepare a dish that user A finds most delicious. That is, what is needed is a method for easily constructing a model related to subjective data, that is, a subjective model that inputs a user's subjective evaluation of an evaluation target and outputs a new evaluation target for the user.

[0005] As a method for constructing a model by collecting a small amount of data, for example, Bayesian optimization disclosed in Non-Patent Document 1 can be mentioned. This method uses a rating for a parameter as input and repeatedly learns a model related to the presentation of parameters to be evaluated.

[0006] For example, when a user assigns a score to a dish in a certain recipe, the next recipe to be rated is output, and the user assigns a score to this recipe. This process is repeated until the recipe with the highest user rating is finally output. Furthermore, as disclosed in Non-Patent Document 2, for example, there is a method for optimizing a cookie recipe using Bayesian optimization. [Prior art documents] [Non-patent literature]

[0007] [Non-Patent Document 1] B. Shahriari, K. Swersky, Z. Wang, R. Adams, and N. de Freitas: Taking the human out of the loop: A Review of Bayesian Optimization. Proc. of the IEEE,(1), 12 / 2015(2016) [Non-patent document 2] Google. “The makings of a smart cookie.” https: / / www.blog.google / technology / research / makings-smart-cookie / , 2017. Summary of the Invention [Problem to be solved by the invention]

[0008] However, when the above-mentioned method is applied, there is a problem that the number of evaluations required to learn a complex model is relatively large.

[0009] This invention has been made in light of the above-mentioned circumstances, and its purpose is to provide an information processing device, method, and program that can input a user's subjective evaluation of an evaluation target and learn a model that outputs a new evaluation target in a small number of iterations. [Means for solving the problem]

[0010] An information processing device according to one aspect of the present invention includes an input unit that accepts input of a new evaluation of an evaluation object for a model that inputs a user's subjective evaluation of the evaluation object and outputs a new evaluation object for the user, an identification unit that identifies a portion of the model that is related to the evaluation perspective for the evaluation object as an object to be learned, a learning unit that learns parameters of the identified portion of the model in accordance with the result of the evaluation input, and a selection unit that uses the result of the evaluation as input and selects the result output from the learned model as a new evaluation object to be presented to the user.

[0011] An information processing method according to one aspect of the present invention is a method performed by an information processing device, and includes: an input unit of the information processing device accepts input of a new evaluation of an evaluation target into a model that inputs a user's subjective evaluation of the evaluation target and outputs a new evaluation target for the user; an identification unit of the information processing device identifies a portion of the model related to the evaluation perspective of the evaluation target as an object to be learned; a learning unit of the information processing device learns parameters of the identified portion of the model in accordance with the result of the evaluation input; and a selection unit of the information processing device selects, using the result of the evaluation as input, a result output from the learned model as a new evaluation target to be presented to the user. [Effects of the Invention]

[0012] According to the present invention, a model that receives a user's subjective evaluation of an evaluation target and outputs a new evaluation target can be learned in a small number of iterations. [Brief explanation of the drawings]

[0013] [Figure 1] FIG. 1 is a diagram illustrating an example of a facial expression recognition model. [Figure 2] FIG. 2 is a diagram showing an example of the relationship between the stages of facial expressions that indicate a sense of enjoyment and the degree to which a person appears to be having fun. [Figure 3] FIG. 3 is a diagram showing an application example of an information processing device according to an embodiment of the present invention. [Figure 4] FIG. 4 is a diagram showing an example of the evaluation results by the user in a table format. [Figure 5] FIG. 5 is a diagram showing an example of presented candidate points. [Figure 6] FIG. 6 is a diagram showing an example of presented candidate points. [Figure 7] FIG. 7 is a diagram illustrating an example of a selection result by the model selection unit. [Figure 8] FIG. 8 is a diagram illustrating an example of a model learning result obtained by the model learning unit. [Figure 9] FIG. 9 is a flowchart illustrating an example of a procedure of a processing operation by the information processing device. [Figure 10] FIG. 10 is a block diagram showing an example of the hardware configuration of an information processing device according to one embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0014] Hereinafter, an embodiment of the present invention will be described. FIG. 1 is a diagram illustrating an example of a facial expression recognition model. The subjective model does not need to be able to output the type and intensity for every parameter, but it is sufficient if it can output the intensity between the parameter with the highest intensity for each viewpoint and the reference point.

[0015] For example, in the facial expression recognition model shown in Figure 1, a facial expression is created using parameters such as the strength of various facial muscles and is input, and an intensity indicating how strong the emotion is perceived by the person viewing this input, such as "It looks like it's about 90 out of 100," is output. Figure 1 shows a normal facial expression (symbol a in Figure 1), the facial expression that the user perceives as the happiest (symbol b in Figure 1), the facial expression that the user perceives as the most surprised (symbol c in Figure 1), the facial expression that the user perceives as the most angry (symbol d in Figure 1), and the facial expression that the user perceives as the saddest (symbol e in Figure 1).

[0016] However, it is not necessary to output the intensity of emotion for every combination of facial muscle strength. If the intensity of emotion can be output for each perspective, in this case for each type of emotion such as joy, anger, sadness, or happiness, it will be possible to convert facial expressions to match the emotion recognition characteristics of each perspective and convey emotions correctly.

[0017] To give a specific example, regarding a user's preferences for food and drink, for example, cookies, a recipe including the amounts of various ingredients such as sugar or chocolate is used as parameters, and the cookies are made and input, and the level of deliciousness felt by the person who ate this input cookie is output, such as "It's about 90 out of 100."

[0018] However, it is not necessary to output the intensity of the emotion for every combination of the amounts of ingredients used in cookies. Rather, if the intensity of deliciousness can be output for each type of cookie from a perspective, such as butter, chocolate, or ginger in this case, cookies can be made with a recipe that suits the preferences for each perspective.

[0019] Fig. 2 is a diagram showing an example of the relationship between the stages of facial expressions that indicate enjoyment and the degree to which the user feels that they are having fun. The facial expression indicated by symbol a in Fig. 2 is the normal facial expression indicated by symbol a in Fig. 1, and the facial expression indicated by symbol b in Fig. 2 is the facial expression that the user feels is the most enjoyable, indicated by symbol b in Fig. 1. In this embodiment, first, the strongest parameter for each viewpoint, for example, the most enjoyable facial expression (symbol b in FIG. 1) in FIG. 1, is obtained by Bayesian optimization.

[0020] Next, the strength between the acquired parameters and the reference point is divided into several stages, and based on the user's input of a score, the relationship between the stage of facial expression perceived as fun (x-axis in Figure 2) and the degree of fun perceived (y-axis in Figure 2) is interpolated, as shown in Figure 2.

[0021] In this embodiment, by performing the above-described processing, it is possible to efficiently learn only the necessary parts of the subjective model with a relatively small number of interactions. The necessary part is determined from a given point of view as the part between the optimum value from this point of view and the origin.

[0022] FIG. 3 is a diagram showing an application example of an information processing device according to an embodiment of the present invention. As shown in FIG. 3, an information processing device 100 according to one embodiment of the present invention has a user evaluation input unit 10, an evaluation result storage unit 20, a model learning unit 30, a model storage unit 40, a candidate point selection unit 50, a candidate point presentation unit 60, and a model restriction unit 70.

[0023] The user evaluation input unit 10 accepts input of a model generation instruction, which is a model learning instruction, and input of an evaluation result for an item to be evaluated presented to a user. That is, the user evaluation input unit 10 accepts input of a new evaluation for an item to be evaluated for a model that inputs a subjective evaluation of the item to be evaluated and outputs a new item to be evaluated for the user. The evaluation target item may be, for example, an image, video, audio, or text provided to the user, or may be a food, drink, or other product that has actually been made and is in the user's possession. There are no particular limitations on the item as long as it can be the subject of a user's subjective evaluation. The evaluation result includes the viewpoint of the evaluation target and an evaluation value for the item. The viewpoint of the evaluation target can also be referred to as a feature of the evaluation result. The above-mentioned viewpoints include, for example, the strength of facial muscles or cooking recipes.

[0024] The evaluation result storage unit 20 receives and stores the evaluation results from the users. The model restriction unit 70 identifies and outputs a model of a portion restricted in accordance with the viewpoint given by input to the user evaluation input unit 10 and the viewpoint among the models from which the candidate points are selected. In other words, the model restriction unit 70 identifies a portion of the model related to the viewpoint of evaluation of the evaluation target as the object of learning. The model learning unit 30 learns the evaluation results and the restricted model, stores the learned model in the model storage unit 40, and outputs it to the candidate point selection unit 50. In other words, the model learning unit 30 learns the parameters of the part of the model identified by the model restriction unit 70 according to the input result of the evaluation. The model storage unit 40 stores a model that outputs the above-mentioned evaluation target, and also receives and stores a trained model.

[0025] The candidate point selection unit 50 selects and outputs candidate points that are appropriate items to be evaluated by the user based on the evaluation results and the trained model. That is, the candidate point selection unit 50 receives the evaluation results as input and selects the results output from the trained model as new evaluation targets to be presented to the user.

[0026] The candidate point presenting unit 60 receives the candidate points to be evaluated by the user, selected by the candidate point selecting unit 50, and presents them to the user.

[0027] FIG. 4 is a diagram showing an example of the evaluation results by the user in a table format. In the example shown in FIG. 4, the number of evaluation trials, parameters related to the candidate points to be evaluated, and user evaluation results for the candidate points, here evaluation values, are shown.

[0028] In the example shown in Figure 4, the parameters related to the candidate points for evaluation include the amounts of various ingredients, such as sugar and butter, used in cooking the cookies. Figure 4 shows an evaluation value of "7" out of 10 for the cookies cooked the first time and tasted by the user, which were made with 200 grams of sugar and 100 grams of butter, and an evaluation value of "3" out of 10 for the cookies cooked the second time and tasted by the user, which were made with 50 grams of sugar and 50 grams of butter.

[0029] Next, the details of the processing by the candidate point selection unit 50 and the candidate point presentation unit 60 will be described. The candidate point selection unit 50 inputs the user evaluation results expressed as a vector of features and evaluation values ​​as shown in (1) and (2) below, and a set of past evaluation results expressed as shown in (3) below from the evaluation result storage unit 20.

[0030]

number

[0031] The candidate point selection unit 50 uses Bayesian optimization based on the above input results and the trained model to calculate acquisition functions expressed as in the following (4) to (9), and based on the calculation results, selects candidate points that are appropriate items to be evaluated by the user and outputs them to the candidate point presentation unit 60. α in the above (4) is a weight.

[0032]

number

[0033] The candidate point presentation unit 60 inputs the candidate points selected by the candidate point selection unit 50, and presents a message to the user requesting input of an evaluation for the candidate point via, for example, a display device of a user terminal (not shown). In addition, as necessary, the candidate point presentation unit 60 presents to the user an image or the like that helps the user understand the evaluation target together with the message.

[0034] 5 and 6 are diagrams showing examples of presented candidate points. 5 and 6 show examples in which a message requesting input of an evaluation for a selected candidate point is displayed together with an image relating to the candidate point.

[0035] If the presented candidate point is a smiling face image as shown in Fig. 5, a message is displayed on the display device saying, "Please answer the degree to which you feel happy when you look at the face image below on a scale of 0 to 100." Following this message, the user can input an evaluation value between 0 and 100 for the degree to which they feel happy about the face image.

[0036] Furthermore, if the presented candidate point is a cookie to be sampled, as shown in Figure 6, a message saying "Please taste the following cookie and rate the degree to which you find it delicious on a scale of 0 to 100" is displayed on the display device together with an image showing the type of cookie to be sampled. Following this message, the user can actually taste a cookie of the same type as the type shown in the image, and then input an evaluation value between 0 and 100 for the degree to which they find this cookie delicious. Although the image shown in Figure 6 shows two cookies stacked on top of each other, as long as the user can correctly evaluate the degree to which cookies of the same type are perceived as delicious, the user does not necessarily have to sample more than the number of cookies shown in the image, and can instead sample, for example, just one bite of the cookie and then input the degree to which they are perceived as delicious.

[0037] Next, the processing of the model restriction unit 70 and the model learning unit 30 will be described in detail. 7 is a diagram showing an example of a selection result by the model selection unit, in which symbols a to e in FIG. For example, in a facial expression recognition model, when the viewpoint of the feature value of the input evaluation result is the viewpoint of sadness, the model restriction unit 70 obtains an optimal value from the facial expression recognition model of all viewpoints (symbol f in FIG. 7) by Bayesian optimization, and outputs a model narrowed down to the part between this obtained optimal value and the origin (symbol g in FIG. 7).

[0038] Here, the optimum value is the facial expression that the user feels is the saddest (symbol e in FIG. 7), and the origin is the neutral expression (symbol a in FIG. 7).

[0039] FIG. 8 is a diagram illustrating an example of a model learning result obtained by the model learning unit. For example, in the case of a facial expression recognition model, the model learning unit 30 receives a plot based on the features and evaluation values ​​that are the user evaluation results stored in the evaluation result storage unit 20, and a restricted model from the model restriction unit 70, and learns this model. Through this learning, the model learning unit 30 generates a characteristic curve that follows the plot, stores the learned model in the model storage unit 40, and outputs it to the candidate point selection unit 50. The model learning method may be, for example, logistic regression or linear regression, and is not particularly limited.

[0040] FIG. 9 is a flowchart illustrating an example of a procedure of a processing operation by the information processing device. First, when the user evaluation input unit 10 receives input of a model generation instruction (Yes in S10), the candidate point presentation unit 60 presents at least one evaluation candidate item to the user by transmitting it to a display device or the like (S11).

[0041] When the user evaluation input unit 10 receives an input of a user's evaluation result for one of the presented items, the evaluation result storage unit 20 acquires and stores the evaluation result (S12).

[0042] Next, the model restriction unit 70 narrows down the entire facial expression recognition model stored in the model storage unit 40, i.e., the facial expression recognition models relating to all emotional perspectives, to the part between the optimum point corresponding to the perspective in the stored evaluation results and the origin as the model part to be learned (S13).

[0043] Based on the evaluation results stored in the evaluation result storage unit 20, the model learning unit 30 updates, i.e., learns, the parameters of the model portion narrowed down by the model restriction unit 70 from among the facial expression recognition models stored in the model storage unit 40 (S14).

[0044] The candidate point selection unit 50 selects at least one candidate item to be newly evaluated by the user based on the evaluation results stored in the evaluation result storage unit 20 and the facial expression recognition model after being learned by the model learning unit 30, in accordance with the viewpoint in the evaluation results (S15).

[0045] If the input of evaluation results for some of the selected candidates has not been completed (No in S16), the process returns to S11, the selected candidate items are presented to the user by being sent to a display device or the like, and subsequent processing is carried out. When the input of the evaluation results for all of the selected candidates has been completed (Yes in S16), the process ends.

[0046] FIG. 10 is a block diagram showing an example of the hardware configuration of an information processing apparatus according to an embodiment of the present invention. 10, the information processing device 100 according to the embodiment is configured, for example, by a server computer or a personal computer, and has a hardware processor 111A such as a CPU. A program memory 111B, a data memory 112, an input / output interface 113, and a communication interface 114 are connected to the hardware processor 111A via a bus 115.

[0047] The communication interface 114 includes, for example, one or more wireless communication interface units, and enables transmission and reception of information to and from a communication network. As the wireless interface, for example, an interface that adopts a low-power wireless data communication standard such as a wireless LAN (Local Area Network) is used.

[0048] The input / output interface 113 is connected to an input device 200 and an output device 300 that are attached to the information processing device 100 and used by a user or the like. The input / output interface 113 can take in operation data input by a user or the like through an input device 200 such as a keyboard, a touch panel, a touchpad, or a mouse, and can also perform processing to output and display output data to an output device 300 including a display device using a liquid crystal or organic EL (Electro Luminescence) or the like. Note that the input device 200 and the output device 300 may be devices built into the information processing device 100, or may be input devices and output devices of other information terminals that can communicate with the information processing device 100 via a network.

[0049] The program memory 111B is a non-transitory tangible storage medium that is a combination of a non-volatile memory that can be written to and read from at any time, such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive), and a non-volatile memory such as a ROM (Read Only Memory), and can store programs required to execute various control processes, etc., according to one embodiment.

[0050] The data memory 112 is a tangible storage medium that is, for example, a combination of the above-mentioned nonvolatile memory and a volatile memory such as RAM (Random Access Memory), and can be used to store various data or information acquired and created during various processing steps.

[0051] An information processing device 100 according to one embodiment of the present invention can be configured as a data processing device having the units shown in FIG. 3 as software-based processing function units.

[0052] The information storage unit used as a work memory or the like by each unit of the information processing device 100 can be configured by using the data memory 112 shown in Fig. 10. However, these configured storage areas are not essential components within the information processing device 100, and may be areas provided in an external storage medium such as a USB (Universal Serial Bus) memory, or a storage device such as a database server located in the cloud.

[0053] The processing function units in each of the above units can be realized by reading and executing a program stored in the program memory 111B by the hardware processor 111A. Note that some or all of these processing function units may be realized in various other forms, including integrated circuits such as an application specific integrated circuit (ASIC) or a field-programmable gate array (FPGA).

[0054] The present invention is not limited to the above-described embodiments, and various modifications can be made in the implementation stage without departing from the spirit of the invention. Furthermore, the embodiments may be implemented in appropriate combinations, in which case the combined effects can be obtained. Furthermore, the above-described embodiments include various inventions, and various inventions can be extracted by combining selected elements from the disclosed elements. For example, if the problem can be solved and the desired effect can be obtained even if some elements are deleted from all elements shown in the embodiments, the configuration from which these elements are deleted can be extracted as an invention.

[0055] The methods described in each embodiment can be stored as a program (software means) that can be executed by a computer on a recording medium such as a magnetic disk (e.g., a floppy disk, a hard disk, etc.), an optical disk (e.g., a CD-ROM, a DVD, an MO, etc.), or a semiconductor memory (e.g., a ROM, a RAM, a flash memory, etc.), or can be distributed by transmitting it via a communication medium. The program stored on the medium also includes a configuration program that configures the software means (including not only an execution program but also tables and data structures) that the computer executes. The computer that realizes this device reads the program stored on the recording medium and, in some cases, configures the software means using the configuration program, and executes the above-mentioned processing by controlling the operation of this software means. The term "recording medium" as used herein is not limited to a storage medium for distribution, but also includes a storage medium such as a magnetic disk or semiconductor memory installed inside the computer or in a device connected via a network. [Explanation of symbols]

[0056] 100...Information processing device 10...User evaluation input section 20...Evaluation result storage section 30...Model Learning Section 40...Model storage area 50...Candidate point selection section 60...Candidate point presentation section 70...Model Restriction Section

Claims

1. an input unit that receives an input of a new evaluation of an evaluation target for a model that receives a subjective evaluation of the evaluation target from a user and outputs a new evaluation target for the user; an identification unit that identifies a part of the model related to a viewpoint of evaluation of the evaluation target as a learning target; a learning unit that learns parameters of the identified part of the model in accordance with the result of the input of the evaluation; a selection unit that uses the evaluation result as an input and selects a result output from the trained model as a new evaluation target to be presented to the user; An information processing device comprising:

2. The identification unit obtaining an optimal value in the model according to the viewpoint of the evaluation through Bayesian optimization, and identifying the learning target based on the optimal value; The information processing device according to claim 1 .

3. A method performed by an information processing device, An input unit of the information processing device receives an input of a new evaluation of the evaluation target for a model that receives a subjective evaluation of the evaluation target from a user and outputs a new evaluation target for the user; Identifying, by an identification unit of the information processing device, a part of the model related to a viewpoint of evaluation of the evaluation target as a learning target; learning parameters of the identified part of the model according to the result of the input of the evaluation by a learning unit of the information processing device; a selection unit of the information processing device selecting, using the evaluation result as an input, a result output from the trained model as a new evaluation target to be presented to the user; An information processing method comprising:

4. An information processing program that causes a processor to function as each unit of the information processing device according to claim 1 or 2.

Images