Information processing device, information processing method, and program

Abstract

In order to enable selection of useful vector series a1, a2, …, aT in a bandit linear optimization problem, for which fixed strategies are not effective, an information processing device (1) is provided with a vector selection unit (11) that selects a vector at in each round t∈[T] (T is a natural number) from a subset A of a d-dimensional vector space Rd (d is a natural number). Defining l1, l2, …, lT∈Rd as a loss vector, the vector selection unit (11) selects a vector at in each round t such that the asymptotic behavior of the expected value of the tracking regret R(u) = Σt∈[T] lt Tat-Σt∈[T]lt Tut with respect to a comparison vector series u1, u2, …, uT∈A, or the asymptotic behavior ignoring logarithmic factors is constrained from above by a predefined function A(d, T, P). Here, P is a natural number greater than or equal to 1 given by P = |{t∈[T-1]|ut ≠ ut+1}|.

Description

[Technical field]

[0001] The present invention relates to an information processing device that solves a bandit linear optimization problem. [Background technology]

[0002] The use of bandit optimization algorithms has been considered for determining advertisements to be presented to users in web advertising, or for determining products to be sold at a discount in web sales. A bandit optimization algorithm refers to an algorithm that selects a vector representing an action in each round under bandit feedback conditions with the objective of minimizing cumulative loss. Among bandit optimization algorithms, an algorithm in which the loss in each round is given as a linear function of the selected vector is called a bandit linear optimization algorithm. For example, Non-Patent Document 1 is an example of a document that discloses a bandit linear optimization algorithm. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Daniely, A., Gonen, A., and Shalev-Shwartz, S., "Strongly adaptive online learning", In International Conference on Machine Learning, pp. 1405-1411, 2015. Summary of the Invention [Problem to be solved by the invention]

[0004] In standard bandit linear optimization algorithms, regret R T =Σ t∈[T] l t T a t -min a*∈A Σt∈[T] l t T a * The asymptotic behavior of the expectation value of T 1 / 2 The vector sequence a1, a2, …, a T Therefore, for a bandit linear optimization problem in which a fixed strategy of selecting the same vector in every round is effective, a useful sequence of vectors a1, a2, …, a T For bandit linear optimization problems where this is not the case, we can select a useful sequence of vectors a1, a2, …, a T There was a problem where you couldn't select it.

[0005] One aspect of the present invention has been made in consideration of the above problems, and an example of the purpose of the present invention is to provide a useful vector sequence a1, a2, ..., a T The present invention aims to realize an information processing device capable of selecting the above. [Means for solving the problem]

[0006] An information processing device according to one aspect of the present invention is a d-dimensional vector space R d At each round t ∈ [T] (T is any natural number), a vector a is drawn from a subset A of (d is any natural number). t The vector selection means selects l1, l2, ..., l T ∈R d is the loss vector, and any comparison vector sequence u1,u2,…,u T Pursuit regret for ∈A R(u) = Σ t∈[T] l t T a t -Σ t∈[T] l t T u t In each round t, the vector a is determined so that the asymptotic behavior of the expected value of a or the asymptotic behavior ignoring the logarithmic factor is bounded from above by a predetermined function A(d,T,P). t Here, P is P = |{t ∈ [T-1]|ut ≠u t +1}| is a natural number greater than or equal to 1. In addition, an information processing method according to one aspect of the present invention includes the steps of: d At each round t ∈ [T] (T is any natural number), a vector a is drawn from a subset A of (d is any natural number). t selecting the vector a t In the selection of l 1 ,l 2 ,…,l T ∈R d is the loss vector, and an arbitrary comparison vector sequence u 1 ,u 2 ,…,u T Pursuit regret for ∈A R(u)=Σ t∈[T] l t T a t -Σ t∈[T] l t T u t In each round t, the vector a is determined so that the asymptotic behavior of the expected value of a or the asymptotic behavior ignoring the logarithmic factor is bounded from above by a predetermined function A(d,T,P). t Here, P is P = |{t ∈ [T-1]|u t ≠u t +1}| is a natural number greater than or equal to 1. A program according to one aspect of the present invention is a program for causing a computer to operate as an information processing device, the program causing the computer to operate in a d-dimensional vector space R d At each round t ∈ [T] (T is any natural number), a vector a is drawn from a subset A of (d is any natural number). t The vector selection means functions as a vector selection means for selecting l 1 ,l 2 ,…,l T ∈R d is the loss vector, and an arbitrary comparison vector sequence u 1 ,u 2 ,…,u T Pursuit regret for ∈A R(u) = Σ t∈[T] l t T a t -Σ t∈[T] l t T u t In each round t, the vector a is determined so that the asymptotic behavior of the expected value of a or the asymptotic behavior ignoring the logarithmic factor is bounded from above by a predetermined function A(d,T,P). t Here, P is P = |{t ∈ [T-1]|u t ≠ut +1}| is a natural number greater than or equal to 1. Effect of the Invention

[0007] According to one aspect of the present invention, even in a bandit linear optimization problem in which a fixed strategy is not effective, a useful vector sequence a1, a2, ..., a T It is possible to realize an information processing device capable of selecting [Brief description of the drawings]

[0008] [Figure 1] 1 is a block diagram showing a configuration of an information processing device according to a first exemplary embodiment. [Diagram 2] 1 is a flow chart showing a flow of an information processing method according to a first exemplary embodiment. [Diagram 3] 3 is a flow diagram showing a first specific example of the information processing method shown in FIG. 2. [Figure 4] 3 is a flow diagram showing a second specific example of the information processing method shown in FIG. 2. [Diagram 5] 1 is a block diagram showing a configuration of a computer that functions as an information processing device according to a first exemplary embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] An exemplary embodiment of the present invention will now be described in detail with reference to the drawings.

[0010] [Bandit Linear Optimization Problem] d-dimensional vector space R d Let A be a subset of T, and for each round t ∈ [T], let l be the loss vector t ∈R d Here, d and T represent any natural numbers. Also, [T] represents the set of natural numbers from 1 to T.

[0011] Vector sequence a1,a2,…,a T Among the problems of selecting ∈A, the cumulative loss Σ t∈[T] l tT a t A problem with the goal of minimizing is called an “online linear optimization problem.” In this exemplary embodiment, we consider the online linear optimization problem under the following bandit feedback condition:

[0012] Bandit feedback condition: At round t, vector a t After selecting, (1) the selected vector a t Loss on l t T a t and (2) the value of the selected vector a t Vector a other than t’ Loss on l t T a t’ It is not possible to refer to

[0013] The online optimization problem under the above bandit feedback conditions is called the "bandit linear optimization problem", and the algorithm for solving the bandit linear optimization problem is called the "bandit linear optimization algorithm".

[0014] In the following, we use an arbitrary sequence of comparison vectors u1, u2, …, u T We use the pursuit regret R(u) defined for ∈A. The pursuit regret R(u) is an evaluation index devised by the inventors of the present application, and is expressed as a vector sequence a1, a2, ..., a1 selected by the bandit linear optimization algorithm. T Accumulative loss Σ t∈[T] l t T a t and the cumulative loss of any comparison vector sequence Σ t∈[T] l t T u t By using this pursuit regret R(u) as an evaluation index, the cumulative loss Σ t∈[T] l tT a t A sequence of vectors a1, a2, …, a that makes T It becomes possible to find out.

[0015] [Configuration of information processing device] The configuration of an information processing device 1 according to this exemplary embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the configuration of the information processing device 1.

[0016] The information processing device 1 is a d-dimensional vector space R d The present invention is an apparatus for solving a bandit linear optimization problem for a subset A of a vector, and as shown in FIG.

[0017] The vector selection unit 11 selects a vector a t The vector selection unit 11 is a means for selecting an arbitrary comparison vector sequence u1, u2, ..., u T Pursuit regret for ∈A R(u) = Σ t∈[T] l t T a t -Σ t∈[T] l t T u t In each round t, the vector a is determined so that the asymptotic behavior of the expected value of a or the asymptotic behavior ignoring the logarithmic factor is bounded from above by a predetermined function A(d,T,P). t Here, P is P=|{t∈[T-1]|u t ≠u t +1}| is a natural number equal to or greater than 1. The vector selection unit 11 selects a vector a t If you select vector a t Loss according to l t T a t is fed back to the vector selection unit 11.

[0018] The vector selection unit 11 is an example of the "vector selection means" in the claims. tmay be provided to a user via a display or the like, or may be provided to another device via a communication network or the like. t may be used in various processes executed inside the information processing device 1.

[0019] Hereinafter, the asymptotic behavior of the pursuit regret R(u) that is constrained from above by the function A(d,T,P) will also be written as R(u) = O(A(d,T,P)). Here, O is Landau's O. In addition, the asymptotic behavior of the pursuit regret R(u) ignoring the logarithmic factor that is constrained from above by the function A(d,T,P) will also be written as R(u) = ~O(A(d,T,P). Here, ~O (~, which is written above O in the formula, is written to the left of O in the text) is Landau's O ignoring the logarithmic factor.

[0020] [Information processing method flow] The flow of the information processing method S1 according to this exemplary embodiment will be described with reference to Fig. 2. Fig. 2 is a flow diagram showing the flow of the information processing method S1.

[0021] The information processing method S1 is a d-dimensional vector space R d This method is for solving a bandit linear optimization problem for a subset A of a vector, and includes a vector selection process S11 as shown in FIG.

[0022] The vector selection process S11 selects a vector a t In the vector selection process S11, an arbitrary comparison vector sequence u1, u2, ..., u T Pursuit regret for ∈A R(u) = Σ t∈[T] l t T a t -Σ t∈[T] l t T u t In each round t, the vector a is determined so that the asymptotic behavior of the expected value of a or the asymptotic behavior ignoring the logarithmic factor is bounded from above by a predetermined function A(d,T,P).t The vector selection process S11 is executed by, for example, the vector selection unit 11 of the information processing device 1.

[0023] [Effects of the information processing device and the information processing method] In standard bandit linear optimization algorithms, regret R T =Σ t∈[T] l t T a t -min a*∈A Σ t∈[T] l t T a * The asymptotic behavior of the expectation value of T 1 / 2 The vector sequence a1, a2, …, a T Therefore, for a bandit linear optimization problem in which a fixed strategy of selecting the same vector in every round is effective, a useful sequence of vectors a1, a2, …, a T For bandit linear optimization problems where this is not the case, we can select a useful sequence of vectors a1, a2, …, a T cannot be selected.

[0024] In contrast, in the information processing device 1 and the information processing method S1 according to this exemplary embodiment, the tracking regret R(u)=Σ t∈[T] l t T a t -Σ t∈[T] l t T u t A sequence of vectors a1, a2, ..., a is determined so that the asymptotic behavior of the expected value of or the asymptotic behavior ignoring the logarithmic factor is constrained from above by a predetermined function A(d,T,P). T In this case, the comparison vector sequence u1, u2, ..., u T does not need to be constant. Therefore, even for bandit linear optimization problems in which a fixed strategy is not effective, there is a useful sequence of vectors a1, a2, …, a T can be selected.

[0025] [Specific example 1 of information processing method] The inventors of the present application have succeeded in proving the following Theorem A regarding bandit linear optimization problems.

[0026] Theorem A: Vector sequence a1, a2, …, a T is a vector sequence selected by the algorithm described in Table 1 below, then any comparison vector sequence u1, u2, ..., u T For ∈A, the following equation (a0) holds: Here, E[·] represents the expected value for the internal randomness of the algorithm.

number

[0027] As a result, the asymptotic behavior of the expected value of regret R(u) ignoring the logarithmic factor is bounded from above by A(d,T,P) given by equation (a1), where β is a constant greater than or equal to 1.

number

[0028] For a particular P, define β as β=Θ((1+P) 1 / 3 ), the asymptotic behavior of the expectation of regret R(u) ignoring the logarithmic factor is bounded from above by A(d,T,P) given by equation (a2).

number

[0029] [Table 1]

[0030] A specific example of an information processing method S1 obtained by realizing this theorem will be described below with reference to Fig. 3. Note that this theorem merely provides one example of this exemplary embodiment, and this exemplary embodiment should not be interpreted as being limited to this theorem.

[0031] FIG. 3 is a flow diagram showing the flow of an information processing method S1 according to this specific example.

[0032] In the information processing method S1 according to this specific example, an initial setting process S10 is executed prior to a vector selection process S11. In the initial setting process S10, a search rate γ∈(0,1), a search base π, a round interval sequence {[s j ,e j ]} j∈N , learning rate sequence {η j} j∈N , perturbation factor sequence {ρ j} j∈N is set.

[0033] Here, the search rate γ is a real number greater than 0 and less than 1. For example, the search rate γ is set to a value specified by a user. The search basis π is a probability distribution on the subset A. The search basis π is, for example, S(π)=Σ a∈A π(a)aa T Using g(π)=max b∈A bS(π) -1 The g(π) defined by b is set so that g(π)≦Cd (C is a constant equal to or greater than 1). j ,e j ] is [s j ,e j ]={s j ,s j+1 ,…,e j-1 ,e j}. The round interval sequence {[s j ,e j ]} j∈N is set, for example, according to the following formula (a3). j is a real number. The learning rate η j For example, the round interval sequence {[s j ,e j ]} j∈N The perturbation factor ρ is set according to the following equation (a4). j is a real number. The perturbation factor ρ j For example, the round interval sequence {[s j ,ej ]} j∈N is set according to the following formula (a5).

number

number

number

[0034] The vector selection process S11 includes an initialization step S11a, a candidate vector setting step S11b, a probability group setting step S11c, a selection index identification step S11d, a first vector selection step S11e, a feedback acquisition step S11f, a first loss vector estimation step S11g, a first weight group update step S11h, a second vector selection step S11i, a second loss vector estimation step S11j, and a second weight group update step S11i. S11k and,

[0035] The initialization step S11a is to assign a weight w1 for each j ∈ Active(t). (j) w1 (j) = η j and set the matrix M to M=S(π) -1 / 2 This is the step of setting.

[0036] The candidate vector setting step S11b is performed by using the loss vectors ^l1, ^l2, ..., ^l t-1 Candidate vector set {a t (j)} j∈Active(t) In this example, the d-dimensional standard normal distribution r t (j) For each j∈Active(t), we use the candidate vector a t (j) is set according to the following equation (a6).

number

[0037] The probability set setting step S11c is a weight set w updated in the previous round t-1. t ={w t (j)} j∈Active(t) Probability set q according to t ={q t (j)} j∈Active(t) In this example, for each j∈Active(t), the probability q t (j) is set according to the following equation (a7).

number

[0038] The index selection step S11d is performed for a probability set q t According to the index j t In this example, for any j∈Active(t), Prob[j t =j]=q t (j) The index j that satisfies t Select .

[0039] The vector selection unit 11 executes either exploratory vector selection or non-exploratory vector selection. The probability that the vector selection unit 11 executes exploratory vector selection is γ, and the probability that the vector selection unit 11 executes non-exploratory vector selection is 1-γ.

[0040] The exploratory vector selection includes a first vector selection step S11e, a feedback acquisition step S11f, a first loss vector estimation step S11g, and a first weight set update step S11f.

[0041] The first vector selection step S11e selects a candidate vector group {a t (j)} j∈Active(t) from vector at This is the step of randomly selecting.

[0042] The feedback acquisition step S11f is a vector a t Feedback according to t T a t This is the step of obtaining the

[0043] The first loss vector estimation step S11g is a step of estimating the feedback vector l t T a t Depending on the loss vector ^l t (^ written above l in the formula is written before l in the text) is the step of estimating the loss vector ^l t ^l t =(l t T a t / γ)(S(π)) -1 a t It is estimated that.

[0044] The first weight set update step S11f is a step of updating the loss vector ^l t Depending on the weight set w t In this example, the weight set w t is updated according to the following equation (a8).

number

[0045] In this specific example, r t is calculated according to the following formula (a9).

number

[0046] The non-exploratory vector selection includes a second vector selection step S11i, a second loss vector estimation step S11j, and a second weight set update step S11k.

[0047] The second vector selection step S11i is performed by selecting a group of candidate vectors {a t (j)} j∈Active(t) from vector a t (jt) This is the step of selecting the index. jt is the probability distribution from Active(t) to the probability group q t Since the indices are randomly selected according to t (jt) is the set of candidate vectors {a t (j)} j∈Active(t) from the probability group q t can be considered to be vectors randomly selected according to

[0048] The second loss vector estimation step S11j estimates the loss vector ^l t ^l t This is the step where we estimate that =0.

[0049] The second weight set update step S11k updates the weight set wt by w t+1 =w t This is the step of updating according to the

[0050] [Specific example 2 of information processing method] The inventors of the present application have succeeded in proving theorem B below regarding bandit optimization problems.

[0051] Theorem B: Vector sequence a1, a2, …, a T is a vector sequence selected by the algorithm described in Table 2 below, then any comparison vector sequence u1, u2, ..., u T For ∈A, the following equation (b0) holds, where E[·] represents the expected value for the internal randomness of the algorithm.

number

[0052] As a result, the asymptotic behavior of the expectation of regret R(u) is bounded from above by A(d,T,P) given by equation (b1), where β is a constant greater than or equal to 1.

number

[0053] For a particular P, define β as β=Θ((1+P) 1 / 2 ), the asymptotic behavior of the expectation of regret R(u) is bounded from above by A(d,T,P) given by equation (b2).

number

[0054] [Table 2]

[0055] A specific example of an information processing method S1 obtained by realizing this theorem will be described below with reference to Fig. 4. Note that this theorem merely provides one example of this exemplary embodiment, and this exemplary embodiment should not be interpreted as being limited to this theorem.

[0056] FIG. 4 is a flow diagram showing the flow of an information processing method S1 according to this specific example.

[0057] In the information processing method S1 according to this specific example, an initial setting process S10 is executed prior to a vector selection process S11. In the initial setting process S10, a search rate γ∈(0,1), a share rate α∈(0,1), a search base π, and a learning rate η>0 are set.

[0058] Here, the search rate γ is a real number greater than 0 and less than 1. The search rate γ is set to, for example, a value specified by the user. The share rate α is a real number greater than 0 and less than 1. The share rate α is set to, for example, α=Θ(1 / T). The search basis π is a probability distribution on the subset A. The search basis π is, for example, S(π)=Σ a∈A π(a)aa T Using g(π)=max b∈A bS(π) -1 The learning rate η is set to η = γ / (2Cd), where γ is the function of Θ(dβ(ClogT / T) 1 / 2 ).

[0059] The vector selection process S11 includes an initialization step S11m, a probability distribution setting step S11n, a vector selection step S11o, a feedback acquisition step S11p, a loss vector estimation step S11q, and a weighting function update step S11r.

[0060] Initialization step S11 m sets the weighting function w1(t):A→R to the identity function w1(x)=1, and sets the weight W1 according to the following equation (b3).

number

[0061] Probability distribution setting step S11 n is the weight function w updated in the previous round t-1. t : Probability distribution p according to A → R t : This is the step of setting A → [0,1]. In this specific example, the probability distribution p t is set according to the following equation (b4).

number

[0062] The vector selection step S11o is carried out by selecting a probability distribution p t From subset A, vector a t This is the step of randomly selecting.

[0063] The feedback acquisition step S11p is a vector a t Feedback according to t T a t This is the step of obtaining the

[0064] The loss vector estimation step S11q estimates the loss vector ̂l in response to the feedback. t In this example, the loss vector ^l t ^l t =l t T a t (S(p t )) -1 a t It is estimated that.

[0065] The weight function update step S11r updates the loss vector ^l t Depending on the weight function w t In this specific example, the weighting function w is updated according to the following equations (b5), (b6), and (b7). t Update.

number

number

number

[0066] [Software implementation example] Some or all of the functions of the information processing device 1 may be realized by hardware such as an integrated circuit (IC chip) or by software. In the latter case, the functions of each part of the information processing device 1 are realized by, for example, a computer that executes instructions of a program, which is software.

[0067] An example of such a computer (hereinafter, referred to as computer C) is shown in Fig. 5. As shown in Fig. 5, computer C includes at least one processor C1 and at least one memory C2. A program P for operating computer C as information processing device 1 is recorded in memory C2. In computer C, processor C1 realizes the functions of each part of information processing device 1 by reading and executing program P from memory C2.

[0068] The processor C1 may be, for example, a central processing unit (CPU), a graphic processing unit (GPU), a digital signal processor (DSP), a micro processing unit (MPU), a floating point number processing unit (FPU), a physics processing unit (PPU), a microcontroller, or a combination of these. The memory C2 may be, for example, a flash memory, a hard disk drive (HDD), a solid state drive (SSD), or a combination of these.

[0069] The computer C may further include a RAM (Random Access Memory) for expanding the program P during execution and for temporarily storing various data. The computer C may further include a communication interface for transmitting and receiving data to and from other devices. The computer C may further include an input / output interface for connecting input devices such as a keyboard and a mouse, and / or output devices such as a display and a printer.

[0070] Furthermore, the program P can be recorded on a non-transitory tangible recording medium M that can be read by the computer C. Such a recording medium M can be, for example, a tape, a disk, a card, a semiconductor memory, or a programmable logic circuit. The computer C can obtain the program P via such a recording medium M. Furthermore, the program P can be transmitted via a transmission medium. Such a transmission medium can be, for example, a communication network or broadcast waves. The computer C can also obtain the program P via such a transmission medium.

[0071] [Application example] The above-described information processing device 1 can be applied to various problems, some examples of which are given below.

[0072] (Discount coupons provided) Consider the problem of deciding which discount coupons to offer to customers at an e-commerce site. In this case, the action of deciding which discount coupons to offer to multiple customers is expressed as a vector a t For example, the action of offering a discount coupon for product 1 to customer A, a discount coupon for product 2 to customer B, and a discount coupon for product 3 to customer C is expressed as vector a t = (1, 2, 3, . . . ), and the loss l t T a tHere, the loss l t T a t The value may be based on whether or not a discount coupon was used, gaze time, whether or not a discount coupon was clicked, the purchase amount of the product, the purchase probability, the purchase amount, and the like. In this case, by applying the above-mentioned information processing method S1, it is possible to determine a discount coupon that reduces losses. In particular, even in cases where customer preferences and utility are prone to change, such as in online marketing, it is possible to provide each customer with an optimal discount coupon.

[0073] (Delivery and pick-up) Consider the problem of determining a delivery route or a pick-up route (hereafter referred to as a "route") for agents such as delivery trucks that deliver packages or pick up customers, or dispatched taxis. In this case, the behavior of determining a route is expressed as a vector a t For example, the action of deciding the route to take the first path, not the second path, and the third path is expressed by the vector a t = (1,0,1,…), and the loss l t T a t (e.g. shipping costs) shall be obtained.

[0074] In this case, by applying the above-mentioned information processing method S1, it is possible to determine a route that reduces losses. In particular, it is possible to optimize a delivery plan that is easily influenced by environmental factors such as weather and congestion.

[0075] (Retail) Consider the problem of deciding the premium / discount rate of each company's beer at a certain store. In this case, the action of deciding the premium / discount rate of each company's beer is expressed as a vector a t For example, the action of setting the price of beer from company A at the regular price, increasing the price of beer from company B by 20%, and offering a 10% discount on beer from company C is expressed by the vector a t= (0, +2, -1, …), and the loss l t T a t In this case, by applying the above-mentioned information processing method S1, it is possible to determine the premium rate / discount rate that reduces the loss.

[0076] (Investment Portfolio) Consider the problem of determining an investor's investment behavior. In this case, the investment (purchase, capital increase), sale, and holding behavior for multiple financial products (stocks, etc.) that an investor holds or intends to hold is represented by a vector a t For example, the actions of making an additional investment in the stock of company A, holding (neither buying nor selling) the bonds of company B, and selling the stock of company C are expressed by the vector a t = (1, 0, 2, . . . ). And as feedback, the loss l t T a t In this case, by applying the above-mentioned information processing method S1, it is possible to determine an investment action that reduces losses.

[0077] (Clinical Trials) Consider the problem of deciding on the medication behavior for a clinical trial of a certain drug in a pharmaceutical company. In this case, the behavior of deciding the amount of medication to be administered to multiple subjects and whether or not to administer the medication is expressed as a vector a t For example, the action of administering a dose of 1 to subject A, not administering a dose to subject B, and administering a dose of 2 to subject C is expressed by the vector a t = (1, 0, 2, . . . ), and the loss l t T a t (For example, the incidence rate of side effects) is assumed to be obtained. In this case, by applying the above-mentioned information processing method S1, it is possible to determine a medication action that reduces losses.

[0078] [Additional Note 1] The present invention is not limited to the above-described embodiment, and various modifications are possible within the scope of the claims. For example, embodiments obtained by appropriately combining the technical means disclosed in the above-described embodiment are also included in the technical scope of the present invention.

[0079] [Additional Note 2] A part or all of the above-described embodiment can be described as follows. However, the present invention is not limited to the aspects described below as an appendix.

[0080] (Appendix 1) d-dimensional vector space R d At each round t ∈ [T] (T is any natural number), a vector a is drawn from a subset A of (d is any natural number). t vector selection means for selecting The vector selection means selects l1, l2, ..., l T ∈R d is the loss vector, and any comparison vector sequence u1,u2,…,u T Pursuit regret for ∈A R(u) = Σ t∈[T] l t T a t -Σ t∈[T] l t T u t In each round t, the vector a is determined so that the asymptotic behavior of the expected value of a or the asymptotic behavior ignoring the logarithmic factor is bounded from above by a predetermined function A(d,T,P). t 4. An information processing device comprising:

[0081] Here, P is P=|{t∈[T-1]|u t ≠u t +1}| is a natural number greater than or equal to 1.

[0082] (Appendix 2) The vector selection means selects a sequence of vectors a1, a2, ..., a1 such that the asymptotic behavior of the expected value of the tracking regret R(u) ignoring the logarithmic factor is restricted from above by a function A(d, T, P). TSelect ∈A, The function A(d, T, P) is given by the following formula (a1) for an unspecified P, or by the following formula (a2) for a specific P: 2. The information processing device according to claim 1 .

number

number

[0083] (Appendix 3) The vector selection means, in each round t, Loss vectors ^l1, ^l2, …, ^l estimated up to the previous round t-1 t-1 Candidate vector set {a t (j)} j∈Active(t) A candidate vector setting step of setting The weight set w updated in the previous round t-1 t ={w t (j)} j∈Active(t) Probability set q according to t ={q t (j)} j∈Active(t) A probability set setting step for setting (1) A set of candidate vectors {a t (j)} j∈Active(t) from vector a t The first vector selection step randomly selects the loss vector ^l depending on the feedback. t The first loss vector estimation step estimates the loss vector ^l t Depending on the weight set w t or (2) a probability set q t According to the candidate vector set {a t (j)} j∈Active(t) from vector a tThe second vector selection step randomly selects the loss vector ^l t ^l t = 0, and the weight set wt is estimated as w t+1 =w t and performing a second weight set update step of updating the weight set according to 3. The information processing device according to claim 2.

[0084] (Appendix 4) The vector selection means selects a sequence of vectors a1, a2, ..., a1 such that the asymptotic behavior of the expectation value of the tracking regret R(u) is constrained from above by a function A(d, T, P). T Select ∈A, The function A(d, T, P) is given by the following formula (b1) for an unspecified P, or by the following formula (b2) for a specific P: 2. The information processing device according to claim 1 .

number

number

[0085] (Appendix 5) The vector selection means, in each round t, The weight function w updated in the previous round t-1 t : Probability distribution p according to A → R t : A probability distribution setting step for setting A → [0,1]; Probability distribution p t From subset A, vector a t A vector selection step in which Depending on the feedback, the loss vector ^l t A loss vector estimation step for estimating Loss vector ^l t Depending on the weight function w tand updating a weighting function by updating the weighting function.

[0086] (Appendix 6) d-dimensional vector space R d At each round t ∈ [T] (T is any natural number), a vector a is drawn from a subset A of (d is any natural number). t vector selection means for selecting The vector selection means, in each round t, Loss vectors ^l1, ^l2, …, ^l estimated up to the previous round t-1 t-1 Candidate vector set {a t (j)} j∈Active(t) A candidate vector setting step of setting The weight set w updated in the previous round t-1 t ={w t (j)} j∈Active(t) Probability set q according to t ={q t (j)} j∈Active(t) A probability set setting step for setting (1) A set of candidate vectors {a t (j)} j∈Active(t) from vector a t The first vector selection step randomly selects the loss vector ^l depending on the feedback. t The first loss vector estimation step estimates the loss vector ^l t Depending on the weight set w t or (2) a probability set q t According to the candidate vector set {a t (j)} j∈Active(t) from vector a t The second vector selection step randomly selects the loss vector ^l t ^l t = 0, and the weight set wt is estimated as w t+1 =w tand performing a second weight set update step of updating the weight set according to 23. An information processing apparatus comprising:

[0087] (Appendix 7) d-dimensional vector space R d At each round t ∈ [T] (T is any natural number), a vector a is drawn from a subset A of (d is any natural number). t vector selection means for selecting The vector selection means, in each round t, Weight function w t : Probability distribution p according to A → R t : A probability distribution setting step for setting A → [0,1]; Probability distribution p t From subset A, vector a t A vector selection step in which Depending on the feedback, the loss vector ^l t A loss vector estimation step for estimating Loss vector ^l t Depending on the weight function w t and updating a weighting function by updating the weighting function.

[0088] (Appendix 8) d-dimensional vector space R d At each round t ∈ [T] (T is any natural number), a vector a is drawn from a subset A of (d is any natural number). t Selecting, The vector a t In the selection of l1, l2, …, l T ∈R d is the loss vector, and any comparison vector sequence u1,u2,…,u T Pursuit regret for ∈A R(u) = Σ t∈[T] l t T a t -Σ t∈[T] l t T u tIn each round t, the vector a is determined so that the asymptotic behavior of the expected value of a or the asymptotic behavior ignoring the logarithmic factor is bounded from above by a predetermined function A(d,T,P). t Select 23. An information processing method comprising:

[0089] Here, P is P=|{t∈[T-1]|u t ≠u t +1}| is a natural number greater than or equal to 1.

[0090] (Appendix 9) A program for causing a computer to operate as an information processing device, The computer is configured to calculate a d-dimensional vector space R d At each round t ∈ [T] (T is any natural number), a vector a is drawn from a subset A of (d is any natural number). t , which serves as a vector selection means for selecting The vector selection means selects l1, l2, ..., l T ∈R d is the loss vector, and any comparison vector sequence u1,u2,…,u T Pursuit regret for ∈A R(u) = Σ t∈[T] l t T a t -Σ t∈[T] l t T u t In each round t, the vector a is determined so that the asymptotic behavior of the expected value of a or the asymptotic behavior ignoring the logarithmic factor is bounded from above by a predetermined function A(d,T,P). t Select A program characterized by:

[0091] Here, P is P=|{t∈[T-1]|u t ≠u t +1}| is a natural number greater than or equal to 1.

[0092] (Appendix 10) A computer-readable recording medium having the program described in appendix 9 recorded thereon.

[0093] (Appendix 11) At least one processor, the processor comprising: d-dimensional vector space R d At each round t ∈ [T] (T is any natural number), a vector a is drawn from a subset A of (d is any natural number). t A vector selection process is performed to select In the vector selection process, l1, l2, ..., l T ∈R d is the loss vector, and any comparison vector sequence u1,u2,…,u T Pursuit regret for ∈A R(u) = Σ t∈[T] l t T a t -Σ t∈[T] l t T u t In each round t, the vector a is determined so that the asymptotic behavior of the expected value of a or the asymptotic behavior ignoring the logarithmic factor is bounded from above by a predetermined function A(d,T,P). t Select 23. An information processing apparatus comprising:

[0094] Here, P is P=|{t∈[T-1]|u t ≠u t +1}| is a natural number greater than or equal to 1.

[0095] (Appendix 12) In addition, these information processing devices may further include a memory, and the memory may store a program for causing the processor to execute the vector selection process. In addition, the program may be recorded in a computer-readable, non-transitory, tangible recording medium. [Explanation of symbols]

[0096] 1. Information processing device 11 Vector selection unit (vector selection means) S1 Information processing method S11 Vector selection processing

Claims

1. d-dimensional vector space R ｄ In each round t ∈ [T] (T is any natural number), a vector a is selected from a subset A of (d is any natural number). ｔ vector selection means for selecting The vector selection means is １ , l ２ , …, l Ｔ ∈R ｄ is the loss vector, and an arbitrary comparison vector sequence u １ , u ２ , …, u Ｔ ∈A, the pursuit regret R(u)=Σ ｔ∈［Ｔ］ l ｔ Ｔ a ｔ -Σ ｔ∈［Ｔ］ l ｔ Ｔ u ｔ In each round t, the vector a is calculated so that the asymptotic behavior of the expected value of a (d, T, P) or the asymptotic behavior ignoring the logarithmic factor is bounded from above by a predetermined function A(d, T, P). ｔ Select 23. An information processing apparatus comprising: Here, P is P = |{t ∈ [T-1]|u ｔ ≠ut+1}| is a natural number equal to or greater than 1.

2. The vector selection means selects a vector sequence a(d, T, P) such that the asymptotic behavior of the expected value of the tracking regret R(u) ignoring the logarithmic factor is suppressed from above by a function A(d, T, P). １ , a ２ , ..., a Ｔ Select ∈A, The function A(d, T, P) is given by the following formula (a1) for an unspecified P, or by the following formula (a2) for a specific P:

2. The information processing apparatus according to claim 1, [0010] Here, β is a constant greater than or equal to 1. [0025]

3. The vector selection means, in each round t, The loss vector ̂l estimated up to the previous round t-1 １ , ^l ２ , …, ^l ｔ－１ Candidate vector group {a ｔ （ｊ） } ｊ∈Ａｃｔｉｖｅ（ｔ） A candidate vector setting step of setting The weight set w updated in the previous round t-1 ｔ = {w ｔ （ｊ） } ｊ∈Ａｃｔｉｖｅ（ｔ） Probability set q according to ｔ = {q ｔ （ｊ） } ｊ∈Ａｃｔｉｖｅ（ｔ） A probability set setting step for setting (1) A set of candidate vectors {a ｔ （ｊ） } ｊ∈Ａｃｔｉｖｅ（ｔ） From vector a ｔ The first vector selection step randomly selects the loss vector ̂l according to the feedback. ｔ A first loss vector estimation step for estimating the loss vector ̂l ｔ Depending on the weight set w ｔ or (2) a first weight set update step for updating the probability set q ｔ According to the candidate vector group {a ｔ （ｊ） } ｊ∈Ａｃｔｉｖｅ（ｔ） From vector a ｔ The second vector selection step randomly selects the loss vector ̂l ｔ ^l ｔ A second loss vector estimation step in which the weight set wt is estimated as w ｔ＋１ =w ｔ and performing a second weight set update step of updating the weight set according to 3. The information processing apparatus according to claim 2.

4. The vector selection means selects a vector sequence a(d, T, P) such that the asymptotic behavior of the expected value of the tracking regret R(u) is bounded from above by a function A(d, T, P). １ , a ２ , ..., a Ｔ Select ∈A, The function A(d, T, P) is given by the following formula (b1) for an unspecified P, or by the following formula (b2) for a specific P:

2. The information processing apparatus according to claim 1, [0030] Here, β is a constant greater than or equal to 1. [0045]

5. The vector selection means, in each round t, The weight function w updated in the previous round t-1 ｔ : Probability distribution p according to A → R ｔ A probability distribution setting step of setting A→[0,1]; Probability distribution p ｔ Vector a from subset A according to ｔ A vector selection step in which Depending on the feedback, the loss vector ̂l ｔ A loss vector estimation step of estimating Loss vector ^l ｔ Depending on the weight function w ｔ 5. The information processing apparatus according to claim 4, further comprising a weighting function update step of updating the weighting function.

6. d-dimensional vector space R ｄ In each round t ∈ [T] (T is any natural number), a vector a is selected from a subset A of (d is any natural number). ｔ vector selection means for selecting The vector selection means, in each round t, The loss vector ̂l estimated up to the previous round t-1 １ , ^l ２ , …, ^l ｔ－１ Candidate vector group {a ｔ （ｊ） } ｊ∈Ａｃｔｉｖｅ（ｔ） A candidate vector setting step of setting The weight set w updated in the previous round t-1 ｔ = {w ｔ （ｊ） } ｊ∈Ａｃｔｉｖｅ（ｔ） Probability set q according to ｔ = {q ｔ （ｊ） } ｊ∈Ａｃｔｉｖｅ（ｔ） A probability set setting step for setting (1) A set of candidate vectors {a ｔ （ｊ） } ｊ∈Ａｃｔｉｖｅ（ｔ） From vector a ｔ The first vector selection step randomly selects the loss vector ̂l according to the feedback. ｔ A first loss vector estimation step for estimating the loss vector ̂l ｔ Depending on the weight set w ｔ or (2) a first weight set update step for updating the probability set q ｔ According to the candidate vector group {a ｔ （ｊ） } ｊ∈Ａｃｔｉｖｅ（ｔ） From vector a ｔ The second vector selection step randomly selects the loss vector ̂l ｔ ^l ｔ A second loss vector estimation step in which the weight set wt is estimated as w ｔ＋１ =w ｔ and performing a second weight set update step of updating the weight set according to 23. An information processing apparatus comprising:

7. d-dimensional vector space R ｄ In each round t ∈ [T] (T is any natural number), a vector a is selected from a subset A of (d is any natural number). ｔ vector selection means for selecting The vector selection means, in each round t, Weight function w ｔ : Probability distribution p according to A → R ｔ A probability distribution setting step of setting A→[0,1]; Probability distribution p ｔ Vector a from subset A according to ｔ A vector selection step in which Depending on the feedback, the loss vector ̂l ｔ A loss vector estimation step of estimating Loss vector ^l ｔ Depending on the weight function w ｔ and a weighting function updating step of updating the weighting functions according to the following equations (b5), (b6), and (b7): [0050] [006] [0070] where η>0 is the learning rate and α∈(0,1) is the share rate.

8. d-dimensional vector space R ｄ In each round t ∈ [T] (T is any natural number), a vector a is selected from a subset A of (d is any natural number). ｔ Selecting, The vector a ｔ In the selection of １ , l ２ , …, l Ｔ ∈R ｄ is the loss vector, and an arbitrary comparison vector sequence u １ , u ２ , …, u Ｔ ∈A, the pursuit regret R(u)=Σ ｔ∈［Ｔ］ l ｔ Ｔ a ｔ -Σ ｔ∈［Ｔ］ l ｔ Ｔ u ｔ In each round t, the vector a is calculated so that the asymptotic behavior of the expected value of a (d, T, P) or the asymptotic behavior ignoring the logarithmic factor is bounded from above by a predetermined function A(d, T, P). ｔ Select 23. An information processing method comprising: Here, P is P = |{t ∈ [T-1]|u ｔ ≠ut+1}| is a natural number equal to or greater than 1.

9. A program for causing a computer to operate as an information processing device, The computer is configured to define a d-dimensional vector space R ｄ In each round t ∈ [T] (T is any natural number), a vector a is selected from a subset A of (d is any natural number). ｔ , which serves as a vector selection means for selecting The vector selection means is １ , l ２ , …, l Ｔ ∈R ｄ is the loss vector, and an arbitrary comparison vector sequence u １ , u ２ , …, u Ｔ ∈A, the pursuit regret R(u)=Σ ｔ∈［Ｔ］ l ｔ Ｔ a ｔ -Σ ｔ∈［Ｔ］ l ｔ Ｔ u ｔ In each round t, the vector a is calculated so that the asymptotic behavior of the expected value of a (d, T, P) or the asymptotic behavior ignoring the logarithmic factor is bounded from above by a predetermined function A(d, T, P). ｔ Select A program characterized by: Here, P is P = |{t ∈ [T-1]|u ｔ ≠ut+1}| is a natural number equal to or greater than 1.

Images