Information processing device, information processing method, and program

Abstract

To provide an information processing apparatus for solving the problem that the influence of a cold start problem is not sufficiently reduced because the viewpoint of a user who is a target of a recommendation is not taken into account.SOLUTION: Processing executed by an information processing apparatus identifies a target user, acquires user characteristics of a plurality of users including the target user, and calculates similarities between the target user and other users. The processing also acquires content characteristics for a plurality of contents and calculates similarities between the plurality of contents. The processing then determines an advertisement suitable for the target user based on each of the calculated similarities and provides the determined advertisement to the target user.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present invention relates to an information processing device, an information processing method, and a program, and in particular to a reinforcement learning technique that can be applied to a recommendation system that recommends various items. [Background technology]

[0002] Recommendation systems that recommend various items are known as an application field of machine learning. Conventionally, in such systems, in order to enhance the effectiveness of recommendations, a method using collaborative filtering is used in which similarity between users is determined based on transaction history of items, and an item purchased by one user is identified as an item suitable for other users similar to the user. However, this method has a problem in that it faces a cold start problem, which makes optimization difficult when historical information such as transaction history is scarce.

[0003] In order to reduce the impact of the cold start problem, a method of transferring knowledge between models is disclosed, for example, in Non-Patent Document 1. Models for recommendations have generally been specialized for specific domains, and each model has been highly independent, but this document discloses an algorithm for transferring knowledge based on the similarity between domains. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Liu, B., Wei, Y., Zhang, Y., Yan, Z., Yang, Q.: Transferable contextual bandit for cross-domain recommendation. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 32 (2018). URL: https: / / ojs.aaai.org / index.php / AAAI / article / view / 11699 Summary of the Invention [Problem to be solved by the invention]

[0005] The technique described in the above document has a problem in that it does not take into account the viewpoint of the user who is the target of the recommendation, and the influence of the cold start problem is not sufficiently reduced.

[0006] The present invention has been made in consideration of the above-mentioned problems, and aims to provide a technique for reducing the effects of the cold start problem in machine learning. [Means for solving the problem]

[0007] In order to solve the above problem, one aspect of an information processing device according to the present invention has an acquisition means for acquiring inter-content similarity, which is the similarity between a target content and one or more other contents, and inter-user similarity, which is the similarity between a target user and one or more other users, an estimation means for estimating a prior distribution of expected reward obtained by the target user executing a process on the target content based on the inter-content similarity and the inter-user similarity, and a derivation means for deriving a posterior distribution of the expected reward using the prior distribution.

[0008] In the information processing device, the acquisition means can acquire the inter-content similarity using features of the target content and the one or more other contents, and can acquire the inter-user similarity using features of the target user and the one or more other users.

[0009] In the information processing device, the estimation means can estimate the prior distribution using a first reward obtained by an execution process for the other content by the target user.

[0010] The first reward may be configured to discount rewards over time such that more recent performances by the target user on the other content result in higher rewards than past performances by the target user on the other content.

[0011] In the information processing device, the estimation means can estimate the prior distribution using a second reward obtained through an execution process by the other user on the target content.

[0012] The second reward may be configured to discount rewards over time such that more recent performances of the target content by the other users result in higher rewards than past performances of the target content by the other users.

[0013] The information processing device may further include a determination means for determining whether to provide the target content to the target user based on the posterior distribution of the expected reward derived by the derivation means.

[0014] The content may be an advertisement for a tangible or intangible product or service, the execution process may be a process of displaying the advertisement, and the reward may indicate whether or not the advertisement has been clicked.

[0015] In order to solve the above problem, another aspect of an information processing device according to the present invention has an acquisition means for acquiring similarities between a plurality of contents and similarities between a plurality of users, and a determination means for determining, using the similarities between the plurality of contents and the similarities between the plurality of users, a content among the plurality of contents that has the highest expected reward as a content suitable for one or more users among the plurality of users.

[0016] In order to solve the above problem, one aspect of an information processing method according to the present invention includes an acquisition step of acquiring inter-content similarity, which is the similarity between a target content and one or more other contents, and inter-user similarity, which is the similarity between a target user and one or more other users, an estimation step of estimating a prior distribution of expected reward obtained by the target user executing a process on the target content based on the inter-content similarity and the inter-user similarity, and a derivation step of deriving a posterior distribution of the expected reward using the prior distribution.

[0017] In order to solve the above problem, one aspect of the program according to the present invention is an information processing program for causing a computer to execute information processing, the program causing the computer to execute processes including an acquisition process for acquiring inter-content similarity, which is the similarity between a target content and one or more other contents, and inter-user similarity, which is the similarity between a target user and one or more other users, an estimation process for estimating a prior distribution of expected reward obtained by an execution process of the target content by the target user based on the inter-content similarity and the inter-user similarity, and a derivation process for deriving a posterior distribution of the expected reward using the prior distribution.

[0018] In order to solve the above problem, another aspect of the information processing method according to the present invention includes an acquisition step of acquiring similarities between a plurality of contents and similarities between a plurality of users, and a determination step of using the similarities between the plurality of contents and the similarities between the plurality of users to determine, among the plurality of contents, a content having the highest expected reward as a content suitable for one or more users among the plurality of users.

[0019] In order to solve the above problem, another aspect of the program of the present invention is an information processing program for causing a computer to execute information processing, the program causing the computer to execute processing including an acquisition process for acquiring similarities between multiple contents and similarities between multiple users, and a determination process for using the similarities between the multiple contents and the similarities between the multiple users to determine, among the multiple contents, a content with the highest expected reward as content suitable for one or more users among the multiple users.

[0020] In order to solve the above problem, one aspect of the learning model according to the present invention is configured to estimate a prior distribution of expected reward obtained by a target user executing a process on a target content based on inter-content similarity, which is the similarity between the target content and one or more other contents, and inter-user similarity, which is the similarity between the target user and one or more other users, and to derive a posterior distribution of the expected reward using the prior distribution. Effect of the Invention

[0021] According to the present invention, it is possible to reduce the influence of the cold start problem in machine learning. [Brief description of the drawings]

[0022] [Figure 1] FIG. 1 shows an example of the configuration of an information processing system. [Diagram 2] FIG. 2 shows an example of the functional configuration of the information processing device 10. [Diagram 3]FIG. 3 illustrates an algorithm of a learning model according to an embodiment. [Figure 4] FIG. 4 shows an example of the hardware configuration of the information processing device 10 and the user device 11. As shown in FIG. [Diagram 5] FIG. 5 shows a flowchart of the process executed by the information processing device 10. [Figure 6] FIG. 6 shows an application example to which the learning model according to the embodiment is applied. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0023] Hereinafter, with reference to the attached drawings, an embodiment for carrying out the present invention will be described in detail. Among the components disclosed below, those having the same functions are given the same reference numerals, and their description will be omitted. Note that the embodiment disclosed below is an example of a means for realizing the present invention, and should be appropriately modified or changed depending on the configuration of the device to which the present invention is applied and various conditions, and the present invention is not limited to the following embodiment. Furthermore, not all of the combinations of features described in the present embodiment are necessarily essential to the solution of the present invention.

[0024] [Information processing system configuration] Fig. 1 shows an example of the configuration of an information processing system according to this embodiment. As an example, as shown in Fig. 1, this information processing system includes an information processing device 10 and multiple user devices 11-1 to 11-M (M>1) used by multiple arbitrary users 1 to M. In the following description, unless otherwise specified, user devices 11-1 to 11-M may be collectively referred to as user device 11. In the following description, the terms user device and user may be used synonymously.

[0025] The user device 11 is, for example, a device such as a smartphone or a tablet, and is configured to be able to communicate with the information processing device 10 via a public network such as LTE (Long Term Evolution) or a wireless communication network such as a wireless LAN (Local Area Network). The user device 11 has a display unit (display surface) such as a liquid crystal display, and users 1 to N can perform various operations using a GUI (Graphic User Interface) provided on the liquid crystal display. The operations include various operations on content such as images displayed on the screen, such as tapping, sliding, and scrolling using a finger or a stylus. User device 11 is not limited to the device shown in Fig. 1, and may be a device such as a desktop PC (Personal Computer) or a notebook PC. In this case, operations by users 1 to M may be performed using input devices such as a mouse and a keyboard. User device 11 may also be provided with a separate display screen.

[0026] The information processing device 10 provides the user device 11 with content for recommending items such as tangible or intangible products and services (e.g., travel products), and the user device 11 is configured to be able to display the content on the display unit of the user device 11. In this embodiment, the information processing device 10 provides the user device 11 with images of advertisements (advertising images, hereinafter also simply referred to as advertisements) related to various items as content, and the user device 11 is configured to be able to display the advertisements on the display unit of the user device 11. The information processing device 10 provides various websites to provide the advertisements. Note that the various websites may be operated by the information processing device 10 or by a server device (not shown). The various websites may include, for example, an e-commerce site, a restaurant reservation site, a hotel reservation site, and the like.

[0027] [Functional configuration of information processing device 10] The information processing device 10 can acquire attributes (information representing attributes) of each of the users 1 to M of the user devices 11-1 to 11M as user features. The information processing device 10 can also acquire a plurality of features related to advertisements to be provided as advertisement features. The information processing device 10 executes an algorithm of a learning model, which will be described later, using the acquired user features and advertisement features, determines advertisements suitable for any one or more of the user devices 11-1 to 11-M, and provides the advertisements to the any one or more of the user devices. The learning model and processing using the learning model will be described later.

[0028] An example of the functional configuration of the information processing device 10 according to this embodiment is shown in Fig. 2. The information processing device 10 according to this embodiment includes, as an example of its functional configuration, a user feature acquisition unit 101, a content feature acquisition unit 102, a parameter setting unit 103, an estimation unit 104, and a provision unit 105.

[0029] The user feature acquiring unit 101 acquires the attributes of each of the users 1 to M of the user devices 11-1 to 11-M as user features. The user features may include at least a part of demographic attributes such as gender, age, annual income, educational background, and place of residence, psychological attributes such as hobbies and interests, behavioral attributes such as past Internet search history, browsing history, and purchasing history, and registration information for a specific application.

[0030] The content feature acquisition unit 102 acquires attributes of the content (in this embodiment, an advertisement) to be provided to the user as content features. In this embodiment, the content is an advertisement, and the content features (advertisement features) may include attributes of the item (tangible or intangible product or service (e.g., travel product)) that is the subject of the advertisement, features of the image that constitutes the advertisement, etc. The content feature acquisition unit 102 is configured to be able to acquire content features of not only content that has been provided to the user in the past, but also content that is scheduled to be provided in the future.

[0031] The parameter setting unit 103 sets predetermined parameters required for the algorithm of the learning model executed by the estimation unit 104. The parameters will be described later. The parameters may be set in advance in the information processing device 10, or may be input by the operator of the information processing device 10.

[0032] The estimation unit 104 executes an algorithm of a learning model according to this embodiment, which will be described later, estimates an expected reward obtained from an execution process for the content, and estimates content suitable for a given user. In this embodiment, the estimation unit 104 executes the algorithm of the learning model, estimates an expected reward obtained from an advertisement display process, and determines an advertisement suitable for display on any one or more of the user devices 11-1 to 11-M. Furthermore, the estimation unit 104 can determine, for any given content, whether the content is suitable for a given user.

[0033] The providing unit 105 provides the advertisement determined by the estimating unit 104 to the user device 11. This enables the user device 11 to display the provided advertisement on the display unit.

[0034] [Learning model algorithm] Next, the algorithm of the learning model according to this embodiment will be described. The learning model according to this embodiment is a model for a bandit algorithm. The bandit algorithm is known as a reinforcement learning algorithm, and aims to maximize the cumulative reward. Specifically, the bandit algorithm aims to draw an arm so as to maximize the expected reward by adjusting the balance between exploitation and exploration for the arm (the ratio of exploitation and exploration). In the field of reinforcement learning, an arm is generally called an action, and the term action will be used in the following description.

[0035] The learning model algorithm according to this embodiment is characterized by transferring knowledge (reward) across multiple domains and multiple users. In this algorithm, advertisement display is used as an action, and the objective is to maximize the cumulative reward for advertisement display. In this algorithm, each domain is a website that handles items such as tangible or intangible goods and services (e.g., travel products). For example, an e-commerce site, a restaurant reservation site, and a hotel reservation site each correspond to different domains.

[0036] Unlike movies and products, advertisements are content that is created when a new sales campaign is launched and deleted when the campaign ends. Therefore, the ratio of newly created advertisements is higher than the ratio of newly created movies and products, so the cold start problem may be prominent in the case of advertisements. In this embodiment, a new learning model based on the well-known Thompson sampling strategy, which is one of the strategies for bandit algorithms, is described as a learning model for reducing the influence of the cold start problem. The algorithm of the learning model according to this embodiment is described below using mathematical expressions.

[0037] First, assume that there are N available sources. Each source corresponds to a widget that displays advertisements. The widget is application software that has a function of displaying small advertisements (e.g., banner advertisements) on the screen of the display unit of a terminal device (corresponding to any user device 11 in FIG. 1 in this embodiment). In this embodiment, it is assumed that each source displays a set of advertisements consisting of multiple advertisements. The multiple advertisements may be configured to be switched and displayed at regular intervals, or may be configured to be displayed using a carousel function. The carousel function is a function that allows the user to switch the display of multiple advertisements in one advertisement display frame by actively operating the user.

[0038] Here, the set of advertisements in any source s (the set of advertisements that can be displayed by source s) is denoted by A sIn addition, each of the M (M>0) users has d u Type(d u >0) and represents the features of M users (user feature set). Therefore, X is M×d u Also, for source s, K s pieces(K s >0) ads are d a Type(d a >0), and K s A set of features representing advertisements (ad feature set) is called Y s Therefore, Y s is K s ×d a It is represented as a matrix of size

[0039] Furthermore, let user i in source s at time step t be user i t s User features at source s at time step t: TIFF0007656743000001.tif7159 and the ad features for source s at time step t: Observe TIFF0007656743000002.tif7159. Then, the user receives advertisements for source s at time step t: View TIFF0007656743000003.tif7159 and get the reward Observe TIFF0007656743000004.tif7159. The reward is given to user i t s Advertisement A t s This represents an implicit reward, indicating whether or not a user clicked on an ad. So the overall observation is, Shown as TIFF0007656743000005.tif7159. The reward may be structured to correspond to an index showing whether the advertisement was clicked and a conversion (a final result such as a product purchase or a request for information) occurred.

[0040] The learning model according to this embodiment is a cumulative reward Advertisement displayed when maximizing TIFF0007656743000006.tif14159 t s The purpose is to determine the maximum possible value of time step t. The maximization of the cumulative reward expectation can be expressed as the minimization of the total regret of user i, as shown in Equation (1).

number

[0041] The learning model according to the present embodiment is to estimate the number of observations of all sources s: This model learns a policy for extracting advertisements from TIFF0007656743000008.tif7159. The learning model according to this embodiment also transfers knowledge between sources (i.e., between advertisements) and between users by utilizing connections between sources. This allows the policy to recognize more generalized user behavior. The learning model according to this embodiment transfers rewards as knowledge between sources and users. The degree of transfer of rewards is based on the similarity between the features of one object and the features of the other object as a target.

[0042] In this embodiment, the degree of transfer of reward between users (that is, the similarity between users) is expressed as shown in Equation (2A) using cosine similarity.

number

[0043] Similarly, the degree of reward transfer between advertisements (i.e., the similarity between advertisements) is expressed as in equation (2B).

number

[0044] Note that advertisement i and advertisement j may be selected from the same domain or may be obtained from different domains. As mentioned above, if the domain is a website dealing in items such as tangible or intangible goods and services (e.g., travel products), advertisement i may be an advertisement in an e-commerce site and advertisement j may be an advertisement in a restaurant reservation site.

[0045] In real-world datasets, especially when the number of users is large, it is difficult to compute the similarity between every pair of users, so implementations may utilize well-known locality sensitive hashing to efficiently obtain the similarity measure.

[0046] As described above, the learning model according to this embodiment is based on the Thompson sampling strategy. The Thompson sampling strategy is a method of selecting the arm with the highest score among multiple sample scores expected from the posterior distribution of the arms in each round, which is derived based on the prior distribution, in order to draw the arm that maximizes the expected reward. In the case of Bernoulli bandits, the likelihood function is formulated by the Bernoulli distribution, and the prior distribution is expressed by the beta distribution as a natural conjugate prior. The beta distribution function can be expressed as shown in equation (3).

number

[0047] In the original Thompson sampling, α k = 1 and β k In the previous example, a uniform distribution of σ = 1 was assumed, but in the learning model according to the present embodiment, the prior distribution is estimated using historical data. Specifically, the aforementioned similarity function between the user and the advertisement is used. This makes it possible to provide a better estimation of the prior distribution. First, we formulate the parameters representing the positive reward (α) and negative reward (β) for prior estimation for a target user i and a target advertisement k at time step t as shown in Equation (4) below.

number

number

[0048] Similarly, f jk (t) is f jkτ It is defined as the cumulative negative reward with the discount rate γ as shown in (5B). Here, f jkτ represents the number of failed recommendations and is equal to 1 if another user j views the target ad k at time τ but does not click on it.

number

[0049] In this way, the parameters representing the positive reward (α) and negative reward (β) for the ex-ante estimation of the target advertisement k for the target user i are the similarity between users (the similarity between the target user and one or more other users (ΣS user )) and the similarity between advertisements (the similarity between the target advertisement and one or more other advertisements (ΣS ad ) can be estimated by Therefore, the reward is transferred based on the similarity between users and between advertisements. In addition, since user preferences may change over time, the cumulative positive reward s ilBy introducing a discount rate γ into (t), a higher value can be given to the reward from the user's recent behavior. In this embodiment, as shown in the above formulas (4), (5A), and (5B), the reward obtained by the execution process (display of advertisement) of another advertisement l by the target user i and the reward by the execution process of the target advertisement k by another user j are discounted (or gradually decreased) over time. Then, the prior distribution is estimated using these discounted rewards. Note that the setting of the discount rate may be optional.

[0050] After estimating the prior distribution, we derive the posterior distribution using the prior distribution. As with the original Thompson sampling strategy, we formulate the posterior distribution using a beta distribution. In the original Thompson sampling strategy, a uniform prior distribution is used, and the parameters of the posterior beta distribution are α k =s k +1, β k =f k It was +1. In this embodiment, the parameters of the posterior beta distribution are formulated as in equation (6) using the prior knowledge of equation (4).

number

[0051] In this way, the learning model algorithm according to this embodiment is based on the original Thompson sampling method and dynamically uses user similarity and content similarity to perform inference, so it can be called dynamic collaborative filtering Thompson sampling.

[0052] An algorithm (policy) of the learning model according to this embodiment is shown as algorithm 1 in Fig. 3. This algorithm is executed by the estimation unit 104 of the information processing device 10. The process of the algorithm in Fig. 3 for an arbitrary target user i will be explained in order. Here, each process 1 to 10 in Fig. 3 is represented as S1 to S10.

[0053] First, we consider the hyperparameters λ and g, the discount rate γ, and the similarity between users (S user ), similarity between advertisements (S ad ) is input. Also, past observations O are input. Then, the processes of S1 to S10 are carried out. S1: Time step t is between 0 and T. S2: A set of user features (user context features) and ads (actions) for user i in source s at time step t s and a set of ad features (ad context features) Y s Observe S3:A s For all advertisements k (k∈A s ), S4: Calculate parameters representing the positive reward (α) and negative reward (β) for the prior estimation for user i and ad k at time step t according to equation (4). S5: Calculate the parameters representing the positive reward (α) and negative reward (β) of the posterior beta distribution according to equation (6). S6: From the beta distribution using the posterior beta distribution parameters α and β, θ k Sampling S7: End of the process loop from S4 to S6 S8:θ kDisplay the advertisement k that gives the maximum value of and observe the reward r. S9: Add observation O S10: End of the process loop from S2 to S9

[0054] In this way, it is possible to continuously display advertisements suitable for any user i by transferring rewards between multiple domains (multiple websites in this embodiment) based on the similarity between users and advertisements and repeating the observation O. Furthermore, this transfer method makes it possible to evaluate rewards for new advertisements with little historical information, thereby reducing the impact of the cold start problem. It should be noted that the functions used for the above-mentioned prior distribution and posterior distribution are not limited to the beta distribution function. For example, a Gaussian distribution function may also be used.

[0055] [Hardware configuration of information processing device 10] FIG. 4 is a block diagram showing an example of a hardware configuration of the information processing device 10 according to the present embodiment. The information processing apparatus 10 according to the present embodiment can be implemented on any single or multiple computers, mobile devices, or any other processing platform. 4, the information processing device 10 is illustrated as being implemented in a single computer, but the information processing device 10 according to the present embodiment may be implemented in a computer system including multiple computers. The multiple computers may be connected to each other via a wired or wireless network so as to be able to communicate with each other.

[0056] 4, the information processing device 10 may include a CPU 41, a ROM 42, a RAM 43, a HDD 44, an input unit 45, a display unit 46, a communication I / F 47, and a system bus 48. The information processing device 10 may also include an external memory. A CPU (Central Processing Unit) 41 generally controls the operation of the information processing device 10, and controls each of the components (42 to 47) via a system bus 48, which is a data transmission path.

[0057] The ROM (Read Only Memory) 42 is a non-volatile memory that stores control programs and the like necessary for the CPU 41 to execute processing. Note that the programs may be stored in a non-volatile memory such as an HDD (Hard Disk Drive) 44 or an SSD (Solid State Drive) or an external memory such as a removable storage medium (not shown). The RAM (Random Access Memory) 43 is a volatile memory and functions as a main memory, a work area, etc. of the CPU 51. That is, the CPU 41 loads necessary programs, etc. from the ROM 42 to the RAM 43 when executing processing, and realizes various functional operations by executing the programs, etc.

[0058] The HDD 44 stores, for example, various data and various information required when the CPU 41 performs processing using a program. The HDD 44 also stores, for example, various data and various information obtained when the CPU 41 performs processing using a program. The input unit 45 is composed of a keyboard and a pointing device such as a mouse. The display unit 46 is configured with a monitor such as a liquid crystal display (LCD), etc. The display unit 56 may be configured in combination with the input unit 55 to function as a GUI (Graphical User Interface).

[0059] The communication I / F 47 is an interface that controls communication between the information processing device 10 and an external device. The communication I / F 47 provides an interface with a network and executes communication with an external device via the network. Various data, various parameters, etc. are transmitted and received between the external device and the communication I / F 47. In this embodiment, the communication I / F 47 may execute communication via a wired LAN (Local Area Network) or a dedicated line conforming to a communication standard such as Ethernet (registered trademark). However, the network available in this embodiment is not limited to this, and may be configured as a wireless network. This wireless network includes wireless PANs (Personal Area Networks) such as Bluetooth (registered trademark), ZigBee (registered trademark), and UWB (Ultra Wide Band). It also includes wireless LANs (Local Area Networks) such as Wi-Fi (Wireless Fidelity) (registered trademark), and wireless MANs (Metropolitan Area Networks) such as WiMAX (registered trademark). It also includes wireless WANs (Wide Area Networks) such as LTE / 3G, 4G, and 5G. Note that the network only needs to connect each device so that they can communicate with each other, and the communication standard, scale, and configuration are not limited to those described above.

[0060] At least some of the functions of each element of the information processing device 10 shown in Fig. 4 can be realized by the CPU 41 executing a program. However, at least some of the functions of each element of the information processing device 10 shown in Fig. 4 may be operated as dedicated hardware. In this case, the dedicated hardware operates under the control of the CPU 41.

[0061] [Hardware configuration of user device 11] The hardware configuration of the user device 11 shown in Fig. 1 may be the same as that of Fig. 4. That is, the user device 11 may include a CPU 41, a ROM 42, a RAM 43, a HDD 44, an input unit 45, a display unit 46, a communication I / F 47, and a system bus 48. The user device 11 can display various information provided by the information processing device 10 on the display unit 46, and perform processing corresponding to an input operation received from the user 1 via a GUI (configured by the input unit 45 and the display unit 46).

[0062] [Processing flow] Fig. 5 shows a flowchart of the process executed by the information processing device 10 according to this embodiment. The process shown in Fig. 5 can be realized by the CPU 41 of the information processing device 10 loading a program stored in the ROM 42 or the like into the RAM 43 and executing it. The process shown in Fig. 5 will be described along with an application example shown in Fig. 6. Fig. 6 shows an application example in which the learning model according to this embodiment is applied with reference to the information processing system shown in Fig. 1.

[0063] In S51, the user characteristic acquisition unit 101 identifies a user (target user) to whom an advertisement is to be provided. The user characteristic acquisition unit 101 can identify, as the target user, not only a user to whom an advertisement has already been provided, but also a new user to whom an advertisement has not been provided. In the example of FIG. 6, the user characteristic acquiring unit 101 identifies, among users 1 to M, user i to whom an advertisement has already been provided, as a target user.

[0064] In S52, the user feature acquisition unit 101 acquires user features of multiple users (users 1 to M) including the target user. The user features include attributes that may change over time, such as past Internet search history, so the user feature acquisition unit 101 may acquire the user features periodically or at any timing.

[0065] In S53, the user feature acquisition unit 101 calculates a similarity (S userThe process of S53 corresponds to the process related to the above-mentioned formula (2A).

[0066] In S54, the content feature acquisition unit 102 acquires the features of the advertisement as content. In S54, the content feature acquisition unit 102 acquires the features of a plurality of advertisements in a plurality of websites. In addition, the content feature acquisition unit 102 can acquire the features of a newly created advertisement that has not yet been provided to any user. In the example of FIG. 6, the content feature acquisition unit 102 acquires the features of advertisements in domain A (e.g., an e-commerce site) and the features of advertisements in domain B (e.g., a restaurant reservation site). Furthermore, the content feature acquisition unit 102 acquires the features of advertisements in domain C (e.g., a hotel reservation site) that have been newly created and have not been provided to users. The advertisements in domain C are new advertisements and have not been provided to users. Therefore, no history information regarding advertisements in domain C has been accumulated.

[0067] In S55, the content feature acquisition unit 102 calculates the similarity (S ad The process of S55 corresponds to the process related to the above-mentioned formula (2B). In the example of FIG. 6, the similarity between any two combinations of advertisement features in domain A, domain B, and domain C is calculated. The order of the processes from S51 to S55 is not limited to that shown in FIG.

[0068] In S56, the estimation unit 104 determines the content suitable for the target user by estimation. First, in order to execute the algorithm 1, the estimation unit 104 acquires the hyperparameters λ and g and the discount rate γ from the parameter setting unit 103, and acquires the similarity between users (S user ), similarity between advertisements (S ad) is obtained. Next, the estimation unit 104 sets the target user as user i, and executes algorithm 1 for user i. In this way, the estimation unit 104 determines an advertisement suitable for the target user. The estimation unit 104 may determine whether or not to provide each advertisement that has been the target of the process in S56 to the target user based on the expected remuneration obtained.

[0069] In the example of FIG. 6, user i was interested in clicking on point-related advertisements (advertisements 61A, 62A) on an e-commerce site (domain A) to earn points. Recently, user i has become interested in eating delicious food, and has started clicking on food-related advertisements (advertisement 63A) on domain A. In this way, user i's preferences may change over time. Such dynamic behavior by user i is represented by s in Algorithm 1. il (t). That is, the parameter of the positive reward (α) in the prior distribution is determined so that more value is placed on recent behavior than on past behavior. Therefore, since the food-related advertisement (advertisement 63A) is clicked on in domain A, the estimation unit 104 assigns a higher reward expectation value to an advertisement whose characteristics are more similar to advertisement 63A than advertisements 61A and 62A (equation (1)). In the example of FIG. 6, a high reward expectation value is assigned to a food-related advertisement (advertisement 61B) in domain B (a restaurant reservation site) and a travel-related advertisement (advertisement 61C) in domain C (a hotel reservation site). Since advertisements in domain C have not been provided to users, no history information about the domain has been accumulated, but the estimated reward can be used by using the similarity between advertisements and the similarity between users. Finally, the estimation unit 104 can determine that advertisements 61B and 61C are suitable advertisements for user i.

[0070] In S57, the provision unit 105 provides the advertisements determined in S56 to the target users. In the example of Fig. 6, the provision unit 105 provides advertisements 61B and 61C to the user i. That is, the provision unit 105 controls the display unit of the user device 11-i to display advertisements 61B and 61C.

[0071] In the above description, an example in which user i is specified as a target user in S51 has been described with reference to FIG. 6. Next, an example in which user j is specified will be described. In this example, it is assumed that user j has user characteristics similar to those of user i. In this case, the estimation unit 104 can execute the above algorithm to determine an advertisement similar to an advertisement suitable for user i as an advertisement suitable for user j. In the example of FIG. 6, the estimation unit 104 can determine advertisement 64A in domain A as an advertisement suitable for user j, and the provision unit 105 can provide it to user j.

[0072] In this way, even if there is little or no prior historical information on the content (advertisement) or user, it is possible to estimate and determine content suitable for any user by using the similarity between contents and the similarity between users.

[0073] <Other embodiments> In the above embodiment, the content is described as an advertisement, but the present embodiment can be applied to any content, for example, movies, books, or various products.

[0074] Furthermore, in the above embodiment, the description has been given on the premise of a content recommendation system, but the learning model according to this embodiment, which transfers rewards between multiple domains and between service provision targets (e.g., users), can be applied to all fields. For example, in the field of finance, the learning model according to this embodiment can be applied to select an optimal portfolio for a customer from multiple portfolios. In addition, in the field of healthcare, the learning model according to this embodiment can be applied to provide treatment methods and medicines to patients. In addition, in the field of dialogue systems, the learning model according to this embodiment can be applied to build a system using a conversation agent, that is, to unify multiple conversation systems (each corresponding to a domain) to build a single system.

[0075] Although specific embodiments have been described above, the embodiments are merely illustrative and are not intended to limit the scope of the present invention. The apparatus and method described in this specification can be embodied in forms other than those described above. Furthermore, the above-described embodiments can be appropriately omitted, substituted, and modified without departing from the scope of the present invention. Such omitted, substituted, and modified forms are included in the scope of the claims and their equivalents, and belong to the technical scope of the present invention. [Explanation of symbols]

[0076] 1 to M: user, 10: information processing device, 11-1 to 11-M: user device, 101: user characteristic acquisition unit, 102: content characteristic acquisition unit, 103: parameter setting unit, 104: estimation unit, 105: provision unit

Claims

1. An acquisition means for acquiring a similarity between contents, which is a similarity between a target content and one or more other contents; An estimation means for estimating a prior distribution of an expected reward to be obtained by an execution process of the target content by the target user by executing an algorithm of a learning model using the similarity between the contents and information on the target user; 13. An information processing device comprising:

2. The information processing apparatus according to claim 1 , wherein the acquiring means acquires the inter-content similarity by using features of the target content and the one or more other contents.

3. 3 . The information processing apparatus according to claim 1 , wherein the estimation means estimates the prior distribution by using a first reward obtained by an execution process for the other content by the target user. 4 .

4. The information processing device according to claim 3, characterized in that the first reward is configured to be discounted over time so that a higher reward is obtained by a recent execution process by the target user for the other content than by a past execution process for the other content.

5. The information processing device according to claim 1 , wherein the estimation means estimates the prior distribution by using a second reward obtained by an execution process for the target content by another user.

6. The information processing device according to claim 5, characterized in that the second reward is configured to be discounted over time so that the reward obtained by a recent execution process by the other user for the target content is higher than the reward obtained by a past execution process by the other user for the target content.

7. An information processing device according to any one of claims 1 to 6, characterized in that the content is an advertisement for a tangible or intangible product or service, the execution process is a display process of the advertisement, and the reward related to the expected reward indicates whether or not the advertisement is clicked.

8. A first acquisition means for acquiring a similarity between a plurality of contents; A second acquisition means for acquiring an expected reward for each of the plurality of contents by executing an algorithm of a learning model using the similarity between the plurality of contents to transfer rewards between the plurality of contents and between a plurality of users; 13. An information processing device comprising:

9. 9. The information processing apparatus according to claim 8, wherein each of the plurality of contents is an advertisement for a tangible or intangible product or service.

10. An information processing method executed by an information processing device, An acquisition step of acquiring a similarity between the target content and one or more other contents; an estimation step of estimating a prior distribution of an expected reward to be obtained by an execution process of the target content by the target user by executing an algorithm of a learning model using the similarity between the contents and information on the target user; 13. An information processing method comprising:

11. An information processing method executed by an information processing device, A first acquisition step of acquiring a similarity between a plurality of contents; A second acquisition step of acquiring an expected reward for each of the plurality of contents by executing a learning model algorithm using the similarity between the plurality of contents to transfer rewards between the plurality of contents and between the plurality of users; 13. An information processing method comprising:

12. An information processing program for causing a computer to execute information processing, the program comprising: An acquisition process for acquiring a similarity between the target content and one or more other contents; and an estimation process for estimating a prior distribution of an expected reward to be obtained by an execution process of the target content by the target user by executing an algorithm of a learning model using the similarity between the contents and information on the target user. Information processing program.

13. An information processing program for causing a computer to execute information processing, the program comprising: A first acquisition process for acquiring a similarity between a plurality of pieces of content; and a second acquisition process for acquiring an expected reward for each of the plurality of contents by executing an algorithm of a learning model using the similarity between the plurality of contents to transfer rewards between the plurality of contents and between the plurality of users. Information processing program.

Images