Negotiation method, negotiation apparatus, negotiation system, and negotiation program

Abstract

A negotiation method comprising: obtaining one or more outcomes for one or more negotiation policies; projecting, by a projector, the obtained outcomes into a multidimensional vector, each component of which represents a contribution to utility; and generating, by a generator, one or more outcomes with reference to the multidimensional vector.

Description

NEGOTIATION METHOD, NEGOTIATION APPARATUS, NEGOTIATION SYSTEM, AND NEGOTIATION PROGRAM

[0001] The present application relates to a negotiation method, a negotiation apparatus, a negotiation system, and a negotiation program.

[0002] Automated negotiation is becoming more important in various applications, for example, in business operations. Various techniques for automated negotiation have been developed. For example, a technique disclosed in Non-patent Literature 1 (NPL1) uses a form of relative utility projection, while a technique disclosed in Non-patent Literature 2 (NPL2) uses raw outcomes as inputs and outputs.

[0003] NPL 1: Bakker J, Hammond A, Bloembergen D, Baarslag T. RLBOA: A Modular Reinforcement Learning Framework for Autonomous Negotiating Agents. InAAMAS 2019 May 8 (pp. 260-268).

[0004] NPL 2: Chen S, Zhao J, Zhao K, Weiss G, Zhang F, Su R, Dong Y, Li D, Lei K. ANOTO: Improving Automated Negotiation via Offline-to-Online Reinforcement Learning. In Proceedings of the 23rd International Conference on Autonomous Agents and Multi agent Systems 2024 May 6 (pp. 2195-2197).

[0005] In automated negotiation, it is desirable that negotiation agents adopt to various scenarios while utilizing information available to the agents. In other words, in a technique for automated negotiation, it is desirable to satisfy both of generality and information utilization.

[0006] However, existing methods are either too specific or unable to utilize all the information available to the agent. For example, the technique disclosed in NPL1 uses a form of relative utility projection which makes it unable to utilize most of the information about issues in the opponent offer. For another example, the technique disclosed in NPL2 uses raw outcomes as inputs and outputs, which leads to specificity to a single negotiation domain or a single negotiation scenario.

[0007] An example aspect of the present invention is attained in view of the problem, and an example objective is to provide a technique for automated negotiation which satisfies both of generality and information utilization.

[0008] Solution to Problem In order to attain the objective described above, a negotiation method comprising: obtaining one or more outcomes for one or more negotiation policies; projecting, by a projector, the obtained outcomes into a multidimensional vector, each component of which represents a contribution to utility; and generating, by a generator, one or more outcomes with reference to the multidimensional vector.

[0009] In order to attain the objective described above, a negotiation method comprising: obtaining one or more outcomes for one or more negotiation policies; and generating, by a generator, one or more outcomes with reference to the obtained outcomes, wherein generating one or more outcomes includes: generating one or more projected outcomes from the obtained outcomes based on one or more negotiation policies; and generating the one or more outcomes by inverse-projecting the one or more projected outcomes.

[0010] In order to attain the objective described above, a negotiation apparatus comprising: obtaining means to obtain one or more outcomes for one or more negotiation policies; projection means to project the outcomes obtained by the obtaining means (obtaining section) into a multidimensional vector, each component of which represents a contribution to utility; and generation means to generate one or more outcomes with reference to the multidimensional vector.

[0011] In order to attain the objective described above, a negotiation apparatus comprising: obtaining means to obtain one or more outcomes for one or more negotiation policies; and generation means to generate one or more outcomes with reference to the obtained outcomes, wherein the generation means carries out: generating one or more projected outcomes from the obtained outcomes based on one or more negotiation policies; and generating the one or more outcomes by inverse-projecting the one or more projected outcomes.

[0012] In order to attain the objective described above, a negotiation system comprising: obtaining means to obtain one or more outcomes for one or more negotiation policies; projecting means to project the outcomes obtained by the obtaining means into a multidimensional vector, each component of which represents a contribution to utility; and generating means to generate one or more outcomes with reference to the multidimensional vector.

[0013] In order to attain the objective described above, a negotiation system comprising: obtaining means to obtain one or more outcomes for one or more negotiation policies; and generation means to generate one or more outcomes with reference to the obtained outcomes, wherein the generation means carries out: generating one or more projected outcomes from the obtained outcomes based on one or more negotiation policies; and generating the one or more outcomes by inverse-projecting the one or more projected outcomes.

[0014] In order to attain the objective described above, a negotiation program causing a computer to function as: obtaining means to obtain one or more outcomes for one or more negotiation policies; projecting means to project the outcomes obtained by the obtaining means into a multidimensional vector, each component of which represents a contribution to utility; and generating means to generate one or more outcomes with reference to the multidimensional vector.

[0015] In order to attain the objective described above, a negotiation program causing a computer to function as: obtaining means to obtain one or more outcomes for one or more negotiation policies; and generation means to generate one or more outcomes with reference to the obtained outcomes, wherein the generation means carries out: generating one or more projected outcomes from the obtained outcomes based on one or more negotiation policies; and generating the one or more outcomes by inverse-projecting the one or more projected outcomes.

[0016] According to an example aspect of the present invention, it is possible to provide a technique for automated negotiation which satisfies both of generality and information utilization.

[0017] Fig. 1 is a flowchart showing a flow of a negotiation method according to an example embodiment.Fig. 2 is a block diagram illustrating a configuration of a negotiation apparatus according to an example embodiment.Fig. 3 is a block diagram illustrating a configuration of a negotiation system according to an example embodiment.Fig. 4 is a block diagram illustrating a configuration of a negotiation system according to an example embodiment.Fig. 5 is a schematic illustration of linear aggregated utility function (upper) and generalized linear aggregated utility function (lower), both of which are according to an example embodiment.Fig. 6 is a schematic illustration of processes in training phase according to the second example embodiment.Fig. 7 is a schematic illustration of data flow in training phase, testing phase, and negotiation phase according to an example embodiment.Fig. 8 is a schematic illustration of data flow in training phase according to an example embodiment.Fig. 9 is a schematic illustration of data flow in negotiation phase according to an example embodiment.Fig. 10 is a flowchart showing a flow of a negotiation method according to an example embodiment.Fig. 11 is a flowchart showing a flow of a MUP and IMUP according to an example embodiment.Fig. 12 is a flowchart showing a flow of a UPM according to an example embodiment.Fig. 13 is a flowchart showing a flow of a DPM according to an example embodiment.Fig. 14 is a flowchart showing a flow of a DPM according to an example embodiment.Fig. 15 is a schematic illustration showing effects of an example embodiment.Fig. 16 is a schematic illustration showing display information according to an example embodiment.Fig. 17 is a flowchart showing a flow of a negotiation method according to an example embodiment.Fig. 18 is a block diagram illustrating a configuration of a negotiation apparatus according to an example embodiment.Fig. 19 is a block diagram illustrating a configuration of a negotiation system according to an example embodiment.Fig. 20 is a block diagram illustrating a hardware configuration according to an example embodiments.

[0018] The following description will discuss example embodiments of the present invention. However, the present invention is not limited to the example embodiments described below, but can be altered by a skilled person in the art within the scope of the claims. For example, any embodiment derived by appropriately combining technical means adopted in differing example embodiments described below can be within the scope of the present invention. Further, any embodiment derived by appropriately omitting one or more of the technical means adopted in differing example embodiments described below can be within the scope of the present invention. Furthermore, the advantage mentioned in each of the example embodiments described below is an example advantage expected in that example embodiment, and does not define the extension of the present invention. That is, any embodiment which does not provide the example advantages mentioned in the example embodiments described below can also be within the scope of the present invention.

[0019] <First example embodiment> The following description will discuss a first example embodiment, which is an example embodiment of the present invention, in detail with reference to the drawings. The present example embodiment is basic to each of the example embodiments which will be described later. It should be noted that the applicability of each of the technical means adopted in the present example embodiment is not limited to the present example embodiment. That is, each technical means adopted in the present example embodiment can be adopted in another example embodiment included in the present disclosure, to the extent of constituting no specific technical obstacle. Further, each technical means illustrated in the drawings referred to for the description of the present example embodiment can be adopted in another example embodiment included in the present disclosure, to the extent of constituting no specific technical obstacle.

[0020] (Brief overview of negotiation techniques according to the first example embodiment) Negotiation techniques (method, apparatus, system) according to the first example embodiment deal with negotiation processes between a negotiation agent and one or more opponent (or partner) agents. In particular, the negotiation techniques according to the first example embodiment may include a step (means) of obtaining one or more offers provided by an opponent (or partner) agent, and a step (means) of generating one or more responses to the opponent (or partner) agent with reference to the offers provided by the opponent (or partner). A response can be either an acceptance / of the offer (or one or more of the offers) received leading to agreement and ending the negotiation, or ending the negotiation immediately with no agreement or one or more counter offers that are sent to the opponent (partner). Negotiation techniques must take into account the possibility of an external time-out condition that ends the negotiation without agreement at any time which include but is not limited to a real-time timeout (e.g. ending the negotiation after 2 minutes) or a number of offers exchanged, or a number of rounds of offer exchanges, or any other appropriate timeout condition. The number of opponent (or partner) agents which may be dealt with the negotiation techniques according to the first example embodiment may not be limited to one, but it may be plural.

[0021] As used herein, the term "outcome(s)" may include one or more offers provided by an opponent (or partner) agent, and one or more counter-offers generated by the negotiation agent. The term "negotiation agent" may also be referred to as "negotiation entity". The "negotiation agent" and "opponent (or partner) agent" may be an apparatus, may be a human, or may be an organization.

[0022] The term "negotiation apparatus" refers to, for example, an apparatus that functions as a negotiation entity. Examples of the negotiation apparatus may include a computer, a robot, a drone, an automated driving vehicle, and the like. The term "negotiation" refers to, for example, proceedings in which negotiation entities alternately provide draft agreement candidates (offers and counter-offers) until the negotiation entities reach an agreement, a time limit expires, or a limited number of sessions is reached. The negotiation may be negotiation between entities in conflict of interest, which may correspond to a negotiation between the negotiation agent and an opponent agent. The negotiation may be negotiation (adjustment) between entities not in conflict of interest, which may correspond to a negotiation between the negotiation agent and a partner agent. The negotiation techniques according to the first example embodiment may be applied to various fields including manufacturing and transportation.

[0023] (Negotiation Method S1) The following description will discuss a negotiation method S1 according to the first example embodiment with reference to Fig. 1. Fig. 1 is a flowchart showing a flow of the negotiation method S1. The negotiation method S1 can be carried out, for example, by a negotiation apparatus 10 or a negotiation system 1 explained below. However, this does not limit the first example embodiment.

[0024] (Step S11) In step S11, one or more outcomes for one or more negotiation policies are obtained. For example, in step S11, a obtaining section 11 of the negotiation apparatus 10 may obtain the one or more outcomes for one or more negotiation policies. Here, the outcomes may include a set of all possible agreements in the negotiation process. For example, the outcomes may include one or more offers provided by an opponent (partner) agent. The one or more negotiation policies are one or more policies with which the negotiation entity generates responses including acceptance decision, ending decision, offers and counter offers. A policy may specialize in one or more of these aspects of the negotiation or a single policy (called end-to-end policy) may generate all required offers and responses within a negotiation. The above-mentioned one or more negotiation policies may include a negotiation policy used in the negotiation agent and / or a negotiation policy used in an opponent (partner) agent.

[0025] (Step S12) In step S12, the outcomes obtained in step S11 is projected into a multidimensional vector. For example, in step S12, a projection section 12 of the negotiation apparatus 10 may project the outcomes obtained in step S11 into a multidimensional vector. Here, each component of the multidimensional vector may represent a contribution to utility. For example, an aggregation or a combination of the components of the multidimensional vector may form a utility function.

[0026] The process carried out in step S12 may also be expressed as representing the outcomes for negotiation policies in the form of a utility dependent multidimensional vector based on value functions. Here, each of the value functions may corresponds to each of the components in the above-mentioned multidimensional vector.

[0027] (Step S13) In step S13, one or more outcomes are generated with reference to the multidimensional vector configured in step S12. For example, in step S13, a generation section 13 of the negotiation apparatus 10 may generate one or more outcomes with reference to the multidimensional vector. Here, the one or more outcomes generated in step S13 may form one or more offers or responses (including counter-offers) to the opponent (partner) agent.

[0028] (Advantageous effect) According to the first example embodiment, the negotiation method S1 includes: obtaining one or more outcomes for one or more negotiation policies (step S11); projecting the obtained outcomes into a multidimensional vector, each component of which represents a contribution to utility (step S12), and generating one or more outcomes with reference to the multidimensional vector (step S13).

[0029] As mentioned above, in the negotiation method S1, the outcomes (offers) obtained from the opponent (or partner) agent is projected into a multidimensional vector whose components represent contributions to utility, and then one or more responses including outcomes to the opponent (or partner) agent is generated with reference to the multidimensional vector.

[0030] Therefore, the negotiation method S1 can provide generality, while utilizing the information available to the negotiation agent. In other words, the negotiation method S1 can satisfy both of generality and information utilization.

[0031] (Supplemental Remarks) Note that the above mentioned negotiation policy may generate the following outputs in general: 1) Ending decision: if “end” the negotiation ends immediately without agreement otherwise we continue. 2) Acceptance decision: if “accept” the negotiation ends with agreement otherwise we continue. 3) An outcome (or set of outcomes) which represents a rejection of the offer from the partner and provide a new offer from the agent to be considered by the partner. This can be represented as: response = outcome OR “end” OR “accept”. In other words a response may include "outcome", "end", or "accept".

[0032] Note also that the example embodiments are not limited to the case of the alternating offers protocol which is described by the sequence of offers / counter-offers. The proposed method according to the example embodiments is not limited to this protocol. It can be used with any negotiation protocol at all. For example, another protocol is called the Single Text Protocol which is a mediated protocol in which a mediator agent generates a single offer (outcome), it then sends it to all the negotiators. In some variants, each negotiator either accepts or rejects this outcome. Based on that the mediator decides how to modify the offer. This process is repeated until an agreement is reached, or a time-out condition is met or one of the negotiators leaves the negotiation. There are other variations in which the negotiators respond by some numeric value indicating how much they like the offer. There are many other protocols as well. The main point is that the proposed method can be used in any negotiation using any of these protocols not only the alternating offers protocol.

[0033] Note also that the proposed method can be used in one side of the policy alone. For example, the proposed method can use the multi-dimensional projection of the input offer but then generate single-dimensional utility projections or raw outcomes in the output. In another example, one can use single-dimensional utility projections or raw outcomes in the input and use the proposed method in the output. For example, if a company has a utility function that never changes but the partner utility function can change between negotiations, then using input-only projection may be as good as both-sides projection or even maybe better. If on the other hand, the utility function of a company changes everytime but it knows that the utility function of its partner never changes, then output-only projection may be as good as both-sides projection or even better.

[0034] Note also that because the policy can be decomposed into an acceptance strategy (that makes the accept / reject decision), an ending strategy (that makes the end / no-end decision) and the offering / bidding policy (that generates the outcome in the counter-offer / response), the proposed method can be applied to any one of these three components independently, Moreover, one can train different policies for different components with different number of dimensions. All of these examples are covered the example embodiments in this specification.

[0035] (Negotiation Apparatus 10) The following description will discuss a negotiation apparatus 10 according to the first example embodiment with reference to Fig. 2. Fig. 2 is a block diagram illustrating a configuration of the negotiation apparatus 10. As illustrated in Fig. 2, the negotiation apparatus 10 includes an obtaining section 11, a projection section 12, and a generation section 13.

[0036] (Obtaining section 11) The obtaining section 11 obtains one or more outcomes for one or more negotiation policies. Here, as mentioned above, the outcomes may include a set of all possible agreements in the negotiation process. This set is called an outcome-space. For example, the outcomes may include one or more offers provided by an opponent (partner) agent. The one or more negotiation policies are one or more policies with which the negotiation entity generates offers or responses including counter-offers. The above-mentioned one or more negotiation policies may include a negotiation policy used in the negotiation agent and / or a negotiation policy used in an opponent (partner) agent.

[0037] (Projection section 12) The projection section 12 project the outcomes obtained by the obtaining section 11 into a multidimensional vector. Here, each component of the multidimensional vector represents a contribution to utility. For example, as mentioned above, an aggregation or a combination of the components of the multidimensional vector may form a utility function. The projection section 12 may also be referred to as a projector.

[0038] (Generation section 13) The generation section 13 generates one or more outcomes with reference to the multidimensional vector. Here, the one or more outcomes generated by the generation section 13 may form one or more offers or responses (including counter-offers) to the opponent (partner) agent. The generation section 13 may also be referred to as a generator.

[0039] (Advantageous effect) According to the first example embodiment, the negotiation apparatus 10 includes: an obtaining section 11 to obtain one or more outcomes for one or more negotiation policies; a projection section 12 to project the obtained outcomes into a multidimensional vector, each component of which represents a contribution to utility, and a generation section 13 to generate one or more outcomes with reference to the multidimensional vector.

[0040] As mentioned above, according to the negotiation apparatus 10, the outcomes (offers) obtained from the opponent (or partner) agent is projected into a multidimensional vector whose components represent contributions to utility, and then one or more responses including outcomes to the opponent (or partner) agent is generated with reference to the multidimensional vector.

[0041] Therefore, the negotiation apparatus 10 can provide generality, while utilizing the information available to the negotiation agent. In other words, the negotiation apparatus 10 can satisfy both of generality and information utilization. It is important to note that the specific type of projection (i.e. into dimensions of the utility function) is not arbitrary and is better than any other type of multi-dimensional projection because utility value is what matters to the negotiation not the exact outcome value. This means that this is not just an arbitrary projection and it keeps the most important information needed for the negotiation. That is why it leads to better information utilization compared with any other conceivable multidimensional projection. Moreover, the specific features of the projection used makes the use of this information much easier by the policy. These include: 1) all values are normalized between zero and one which is a good feature for training neural networks as policies. 2) All weights are positive which makes all the contributions of all features positive without loss of generality without loss of generality because the value functions do include the sign. 3) The sum of all weights is one which leads to a more stable representation during training.

[0042] (Negotiation System 100) The following description will discuss a negotiation system 100 according to the first example embodiment with reference to Fig. 3. Fig. 3 is a block diagram illustrating a configuration of the negotiation system 100. As illustrated in Fig. 3, the negotiation system 100 includes an obtaining section 11, a projection section 12, and a generation section 13. The negotiation system 100 may also include one or more opponent (partner) agents.

[0043] As illustrated in Fig. 3, the obtaining section 11, the projection section 12, and the generation section 13 are connected each other via a network N. A specific configuration of the network N does not limit the present example embodiment, and the network N is, for example, a wireless local area network (LAN), a wired LAN, a wide area network (WAN), a public network, a mobile data communication network, or a combination of these networks.

[0044] The specific descriptions pertaining to the obtaining section 11, the projection section 12, and the generation section 13 are described above, and are therefore omitted here. The negotiation system 100 brings about the same advantages effect as the negotiation apparatus 10.

[0045] (Negotiation Method S2) The following description will discuss a negotiation method S2 according to the first example embodiment with reference to Fig. 17. Fig. 17 is a flowchart showing a flow of the negotiation method S2. The negotiation method S2 can be carried out, for example, by a negotiation apparatus 20 or a negotiation system 2 explained below. However, this does not limit the first example embodiment.

[0046] (Step S21) In step S21, one or more outcomes for one or more negotiation policies are obtained. For example, in step S21, an obtaining section 21 of the negotiation apparatus 20 may obtain the one or more outcomes for one or more negotiation policies.

[0047] (Step S22) In step S22, one or more outcomes are generated with reference to the obtained outcomes. For example, in step S22, a generation section 22 of the negotiation apparatus 20 may generate the one or more outcomes with reference to the outcomes obtained in step S21. In an example, the processes carried out in step S22 may include: generating one or more projected outcomes from the obtained outcomes based on one or more negotiation policies; and generating the one or more outcomes by inverse-projecting the one or more projected outcomes.

[0048] Here, the term 'projected' is not intended to limit this example embodiment. The 'projected outcomes' may simply be expressed as 'outcomes' or 'outcomes derived by the one or more negotiation policies'. Note also that each of the projected outcomes may have a plurality of components (elements). The process of the inverse-projection may be expressed as combining or aggregating these elements of the projected outcomes to obtain the (generated) one or more outcomes.

[0049] (Advantageous effect) According to the first example embodiment, the negotiation method S2 includes: obtaining one or more outcomes for one or more negotiation policies (step S21); and generating, by a generator, one or more outcomes with reference to the obtained outcomes (step S22), wherein generating one or more outcomes includes: generating one or more projected outcomes from the obtained outcomes based on one or more negotiation policies; and generating the one or more outcomes by inverse-projecting the one or more projected outcomes.

[0050] As mentioned above, in the negotiation method S2, the outcomes (offers) are obtained from the opponent (or partner) agent, and are referred in generating the one or more outcomes. Here, generating one or more outcomes includes: generating one or more projected outcomes from the obtained outcomes based on one or more negotiation policies; and generating the one or more outcomes by inverse-projecting the one or more projected outcomes.

[0051] Therefore, the negotiation method S2 can provide generality, while utilizing the information available to the negotiation agent. In other words, the negotiation method S2 can satisfy both of generality and information utilization.

[0052] (Negotiation Apparatus 20) The following description will discuss a negotiation apparatus 20 according to the first example embodiment with reference to Fig. 18. Fig. 18 is a block diagram illustrating a configuration of the negotiation apparatus 20. As illustrated in Fig. 18, the negotiation apparatus 20 includes an obtaining section 21, and a generation section 22.

[0053] (Obtaining section 21) The obtaining section 21 obtains one or more outcomes for one or more negotiation policies.

[0054] (Generation section 22) The generation section 22 generates one or more outcomes with reference to the obtained outcomes. In an example, the processes carried out by the generation section 22 may include: generating one or more projected outcomes from the obtained outcomes based on one or more negotiation policies; and generating the one or more outcomes by inverse-projecting the one or more projected outcomes.

[0055] (Negotiation System 200) The following description will discuss a negotiation system 200 according to the first example embodiment with reference to Fig. 19. Fig. 19 is a block diagram illustrating a configuration of the negotiation system 200. As illustrated in Fig. 19, the negotiation apparatus 20 includes an obtaining section 21, and a generation section 22.

[0056] (Advantageous effect) According to the first example embodiment, the negotiation apparatus 2 or the negotiation system 200 includes: an obtaining section 21 to obtain one or more outcomes for one or more negotiation policies; and a generation section 22 to generate one or more outcomes with reference to the obtained outcomes, wherein the generation section 22 carries out: generating one or more projected outcomes from the obtained outcomes based on one or more negotiation policies; and generating the one or more outcomes by inverse-projecting the one or more projected outcomes.

[0057] As mentioned above, in the negotiation apparatus 20 or the negotiation system 200, the outcomes (offers) are obtained from the opponent (or partner) agent, and are referred in generating the one or more outcomes. Here, generating one or more outcomes includes: generating one or more projected outcomes from the obtained outcomes based on one or more negotiation policies; and generating the one or more outcomes by inverse-projecting the one or more projected outcomes.

[0058] Therefore, the negotiation apparatus 20 and the negotiation system 200 can provide generality, while utilizing the information available to the negotiation agent. In other words, the negotiation method S2 can satisfy both of generality and information utilization.

[0059] <Second example embodiment> The following description will discuss details of a second example embodiment of the invention with reference to the drawings. Note that the same reference numerals are given to elements having the same functions as those described in the first example embodiment, and descriptions of such elements are omitted as appropriate.

[0060] (Configuration of negotiation system 100A) The following description will discuss a configuration of a negotiation system 100A according to the second example embodiment with reference to Fig. 4. Fig. 4 is a block diagram illustrating a configuration of the negotiation system 100A. As illustrated in Fig. 4, the negotiation system 100A includes a negotiation apparatus 1A, and a plurality of negotiation agents 50-0, 50-1, 50-2, etc.. Here, the negotiation apparatus 1A and negotiation agents 50-0, 50-1, 50-2, etc. are connected each other via a network N. A specific configuration of the network N does not limit the present example embodiment, and the network N is, for example, a wireless local area network (LAN), a wired LAN, a wide area network (WAN), a public network, a mobile data communication network, or a combination of these networks. Each of the negotiation agents 50-0, 50-1, 50-2 are, for example, an opponent agent or a partner agent.

[0061] (Negotiation apparatus 1A) As illustrated in Fig. 4, the negotiation apparatus 1A includes a control section 10A, a storage section 20A, a communication section 30, and an input-output section 40.

[0062] (Communication section 30) The communication section 30 communicates with an apparatus external to the negotiation apparatus 1A. In an example, the communication section 30 communicates with at least any one selected from the group consisting of the negotiation agents 50-0, 50-1, 50-2. Further, the communication section 30 transmits, to the external apparatus, data supplied from the control section 10A, and / or supplies, to the control section 10A, data received from the external apparatus.

[0063] For example, the communication section 30 receives one or more offers provided by one of the negotiation agents 50-0, 50-1, 50-2, and supplies the received offers to the control section 10A. For another example, the communication section 30 obtains one or more offers or responses (including counter-offers) generated by the control section 10A, and transmits the responses to one of the negotiation agents 50-0, 50-1, 50-2.

[0064] (Input-output section 40) The input-output section 40 includes, for example, an input-output device which is at least any one selected from the group consisting of a keyboard, a mouse, a display, a printer, a touch panel, a camera, a speaker, and the like. The input-output section 40 may include, instead of any of these input-output devices, an interface such as universal serial bus (USB), for example. Further, the interface may be connected to at least any one of the input-output devices.

[0065] The input-output section 40 accepts, from at least any one of the input-output devices, various information for the negotiation apparatus 1A. Further, under control of the control section 10A, the input-output section 40 outputs various information to at least any one of the input-output devices.

[0066] For example, the input-output section 40 may accept, from a user, one or more parameters which specify processes carried out in the control section 10A. For another example, the section 40 may output visualized information relating to the negotiation processes dealt with the negotiation apparatus 1A.

[0067] (Storage section 20A) The storage section 20A stores therein various information referred to by the control section 10A and various information derived by the control section 10A. In an example, as illustrated in Fig. 4, the storage section 20A stores therein the followings: - one or more outcomes OUT - a multidimensional vector VEC - a policy POL - a trained policy TPOL - a projection adjustment model PAM.

[0068] (Outcomes OUT) Here, the outcomes OUT include a set of all possible agreements in the negotiation process. For example, the outcomes OUT may include one or more offers provided by an opponent (partner) agent 50-0, 50-1, or 50-2. In Fig. 4, these offers provided by an opponent (partner) agent 50-0, 50-1, or 50-2 are expressed as received offers ROFF. For another example, the outcomes OUT may include one or more offers or responses (including counter-offers) generated by the generation section 13. In Fig. 4, these offers or responses (including counter-offers) generated by the generation section 13 are expressed as responses COFF.

[0069] The outcomes OUT may also be expressed as elements consisting an outcome space Ω (capital omega). Here, the outcome space Ω is the set of all possible agreements (which may also be referred to as draft agreement candidates). These possible agreements may be expressed as outcomes ω (small omega). The outcome space Ω is a Cartesian product of a set of n issues Ii. In other words, the outcome space Ω may be expressed as follows.

[0070] Here, each issue Iihas a set of values Vithat can be a discrete or continuous set of values. Note that is the value of issue Iiin the outcome ω, where Note also that is a tuple of values, where G (group of K issues) is a tuple of Issues, For example if we have then, we have

[0071] (Multidimensional vector VEC) The multidimensional vector VEC is a vector generated in the projection section 12. Each component of the multidimensional vector VEC represents a contribution to utility. For example, an aggregation or a combination of the components of the multidimensional vector VEC may form a utility function. A specific example of how the multidimensional vector VEC is configured or generated will be described later.

[0072] As used herein, the utility function is a function which receives one or more outcomes and generates a real value between zero and one. If we express the utility function as u, then we have, We call the utility function as Linear Aggregation (LA) utility if it has the form where and viis a function that maps a value for issue Iito [0,1]. The function vimay also be referred to as a value function. We may also impose the following normalization constraint, A schematic example of how the LA utility is configured is shown at the top of Fig. 5.

[0073] On the other hand, we call the utility function as Generalized Linear Aggregation (GLA) utility if it has the form where and viis a function that maps a value for the group of issues Gjto [0,1]. The function vimay also be referred to as a value function. We may also impose the following normalization constraint, A schematic example of how the GLA utility is configured is shown at the bottom of Fig. 5.

[0074] Note that any utility function can be normalized to be in the range [0,1] without any change in the optimal behavior of negotiators (mathematical theorem). Any utility function can be represented as a GLA but not always as an LA. Any GLA can be represented by a neural network.

[0075] (Policy POL and trained policy TPOL) The policy POL or the trained policy TPOL is a policy with which the generation section generates the offers or responses (including counter-offers) COFF. In an example, the policy POL or trained policy TPOL is configured such that: the outcomes OUT, the elements of the multidimensional vector VEC, the elements of up-projected multidimensional vector VEC, or the elements of down-projected multidimensional vector VEC are input; and one or more elements of offers or responses (including counter-offers) are output.

[0076] For example, if the elements of the multidimensional vector VEC are expressed as (x1, x2), the policy may be expressed as a function of (x1, x2), which is a tuple of policy elements (elements of response COFF). Note that each of the policy elements is also a function of the elements of the multidimensional vector VEC, (x1, x2). Note also that in the above explanation, the term "up-projection" indicates a projection which increase the dimensionality or the number of elements, while the term "down-projection" indicates a projection which decrease the dimensionality or the number of elements.

[0077] Note also that one or more parameters included in the policy function (for example, the above-mentioned f(x1, x2)) may be targets of training (machine learning). For example, the above-mentioned trained policy TPOL may be a trained policy function whose parameters have been trained (updated) by the training section 14. A specific example of such training will be described later.

[0078] (Projection adjustment model PAM) The projection adjustment model (PAM) is a model which adjust the inputs to the trained policy TPOL. The projection adjustment model (PAM) may be expressed as a model which receives inputs, and generates the outputs which enhance the availability of value functions (components) included in the utility function used in the negotiation apparatus 1A and / or partner agent 50-0, 50-1, or 50-2. The PAM may be realized as a trained neural network model. In an example, the projection adjustment model (PAM) is a model which adjust the dimensionality of the multidimensional vector VEC to be consistent with the one or more trained negotiation policies TPOL The projection adjustment model (PAM) is also a target of training (machine learning). For example the projection adjustment model (PAM) may be a model which has been trained (updated) by the training section 14. A specific example of such training will be described later.

[0079] (Control section 10A) As illustrated in Fig. 4, the control section 10A includes an obtaining section 11, projection section 12, generation section 13, and a training section 14.

[0080] (Obtaining section 11) The obtaining section 11 obtains one or more outcomes OUT for one or more negotiation policies. Here, as mentioned above, the outcomes OUT may include a set of all possible agreements in the negotiation process. For example, the outcomes OUT may include one or more offers (received offers ROFF) provided by an opponent (partner) agent 50-0, 50-1, or 50-2. In an example, the one or more negotiation policies may include the above-mentioned policy POL or the trained policy TPOL. In another example, the one or more negotiation policies may include negotiation policy used in an opponent (partner) agent 50-0, 50-1, or 50-2.

[0081] (Projection section 12) The projection section 12 project the outcomes OUT obtained by the obtaining section 11 into a multidimensional vector VEC. Here, each component of the multidimensional vector represents a contribution to utility. For example, as mentioned above, an aggregation or a combination of the components of the multidimensional vector VEC may form a utility function. A specific example of how the multidimensional vector VEC is configured or generated will be described later.

[0082] (Generation section 13) The generation section 13 generates one or more outcomes with reference to the multidimensional vector. Here, the one or more outcomes generated by the generation section 13 may form one or more responses COFF to the opponent (partner) agent 50-0, 50-1, or 50-2. As illustrated in Fig. 4, the generation section 13 includes an up-projection section 131, a down-projection section 132, a projection adjustment section 133, and an inverse projection section 134.

[0083] The up-projection section 131 increases the dimensionality of the multidimensional vector VEC to be consistent with the dimensionality of the one or more negotiation policies (POL or TPOL). In an example, the up-projection section 131 may increase the dimensionality of the multidimensional vector VEC to be the same as the input dimensionality of the policy POL, the input dimensionality of the trained policy TPOL, or the input dimensionality of the projection adjustment model PAM.

[0084] The down projection section 132 decreases the dimensionality of the multidimensional vector VEC to be consistent with the dimensionality of the one or more negotiation policies (POL or TPOL). In an example, the down-projection section 132 may decrease the dimensionality of the multidimensional vector VEC to be the same as the input dimensionality of the policy POL, the input dimensionality of the trained policy TPOL, or the input dimensionality of the projection adjustment model PAM.

[0085] The projection adjustment section 133 adjust the inputs to the trained policy TPOL. In an example, the projection adjustment section 133 uses the projection adjustment model (PAM) to adjust the dimensionality of the multidimensional vector VEC, the output of the up-projection section 131, or the output of the down-projection section 132 to be consistent with the one or more trained negotiation policies TPOL.

[0086] (Training section 14) The training section 14 trains at least any one selected from the group consisting of the projection adjustment model (PAM) and the policy POL. As illustrated in Fig. 4, the training section 14 includes a projection training section 141 and a policy training section 142. The projection training section 141 trains the projection adjustment model (PAM), while the policy training section 142 trains the policy POL and generate the trained policy TPOL. In an example, the training of the projection adjustment model (PAM) and the policy POL may be carried out simultaneously. A specific example of training will be described later.

[0087] (Brief overview of processes in negotiation system 100A) The following description will discuss a brief overview of processes carried out in negotiation system 100A with reference to Fig. 6 and Fig. 7. Fig. 6 is a schematic illustration of processes in training phase according to the second example embodiment. Fig. 7 is a schematic illustration of data flow in training phase, testing phase, and negotiation phase according to the second example embodiment.

[0088] (Training Phase) In the training phase, the training section 14 carried out a negotiation session generation process. In the negotiation session generation process, the training section 14 generates and controls the negotiation environment which may be expressed as NegoEnv() as indicated in Fig. 6. The negotiation environment NegoEnv() includes various elements such as one or more negotiation scenarios, a plurality of negotiation agents (opponent or partner), and negotiation mechanism.

[0089] As indicated in Fig. 6, in processes for generating data for training, the training section 14 may play a role of scenario generator to generate a plurality of outcomes (offers) and a plurality of utility functions u1to un. The training section 14 may also play a role of partner (opponent) generator to generate a plurality of negotiation agents (opponent or partner) b1to bn-M. The training section 14 may also play a role of assigner to assign which agent is a learning agent (indicated as ai) and which agent is a background agent (indicated as bi). The training section 14 may also play a role of mechanism generator to generate a negotiation mechanism. In the example indicated in Fig. 6, the training section 14 generates (determines) SAOP (Stacked Alternating Offers Protocol) as the negotiation mechanism.

[0090] The left of Fig. 7 indicates data flows in the training phase. Here, the reference numeral pnindicates the learning agent (agent which is target for training), while the reference numerals p1to pn-1indicate the background agents (agents which are not target for training). As shown in left of Fig. 7, the background agents p1to pn-1are controlled in the negotiation environment by the training section 14 so as to generate their offers with reference to their own utility functions u1to un-1, respectively.

[0091] In the training phase, the state of the learning agent pnis observed by the observation manager. Here, the state of the learning agent pnincludes offer(s) which have been provided by any of the background agents p1to pn-1and then received by the learning agent pn. Note that the states observed by the observation manager may include an extra state other than the state of the learning agent pn. Note also that the training section 14 may play a role of the observation manager. The state (offer) of the learning agent pnis also provided, under a control of the training section 14, to a reward function. Note that the utility un, which may be referred by the learning agent pn, is also provided to the reward function. The reward function calculates a reward corresponding to the state (offer) of the learning agent pnand the utility un.

[0092] The observed state (offer) observed by the observation manager and the reward obtained from the state (offer) are provided to a training algorithm. The training algorithm receives the state (offer) and the reward, then generates observation information with reference to the state (offer) and the reward. The observation information is input to a model. The model receives the observation information and generates an action (offers or responses (including counter-offers)) with reference to the observation information and the policy POL. Note here that the observation information corresponds to an observation observed by the observation manager in the testing phase and the negotiation phase (after deployment). Note also that the action is also input to the training algorithm and taken into account in generating the observation information in the next iterative step.

[0093] The action generated by the model is then provided to the action manager. Here, the training section 14 may play a role of the action manager. The action manager provides the action (offers or responses (including counter-offers)) to the learning agent pn. The learning agent pnprovides the responses to the background agents p1to pn-1and then receives next offers provided by any of the background agents p1to pn-1.

[0094] In the training phase, the training section 14 iterates above processes and trains the model (in other words, the policy POL) until the reward satisfies a predetermined criterion. The policy trained by the above training processes corresponds to the trained policy TPOL as explained.

[0095] (Testing Phase) As shown in the middle of Fig. 7, in the testing phase, the training section 14 does not refer to the training algorithm. The state (offer) observed by the observation manager is provided to the model as the observation information. The training section 14 calculates the reward and check that the reward satisfies the desired condition.

[0096] (Negotiation Phase) As shown in the right of Fig. 7, in the negotiation phase (after deployment), the obtaining section 11 plays a role of the observation manager and obtains the state (received offer). The state (received offer) are referred by the projection section 12 and the generation section 13, and the offers or responses (including counter-offers) is generated by the generation section 13 with reference to the trained policy POL. Note also that in the negotiation phase, the generation section 13 may also play a role of the action manager.

[0097] (Training processes in negotiation system 100A) The following description will discuss a more detailed aspect of the training processes carried out in the negotiation system 100A with reference to Fig. 8. Fig. 8 is a schematic illustration of data flow in the training phase according to the second example embodiment.

[0098] In the Fig. 8, in an example, the partner 50 may correspond to any of the background agents p1to pn-1in Fig. 7. The partner 50 may also correspond to any of the negotiation agent 50-0, 50-1, 50-2 in Fig. 4. The projection trainer 141 in Fig. 8 corresponds to the above-mentioned projection training section 141 in Fig. 4, while the policy trainer 142 in Fig. 8 corresponds to the above-mentioned policy training section 142 in Fig. 4.

[0099] As shown in Fig. 8, in the training phase, the partner provides reference information to the projection trainer 141. The reference information may include offer(s) received by the partner 50 and the utility used in the partner 50. The projection trainer 141 may also obtain negotiation records which includes the outcomes OUT as explained above. More, specifically, the projection trainer 141 may obtain a multidimensional vector VEC. Here, in an example, the multidimensional vector VEC may be the multidimensional vector VEC generated by the projection section 12 with reference to the outcomes OUT.

[0100] The projection trainer 141 trains the projection adjustment model (PAM) with reference to the reference information provided by the partner 50 and the outcomes OUT (or more particularly, the multidimensional vector VEC). Several machine learning methods can be used for this purpose. Here, as mentioned above, the projection adjustment model (PAM) may be regarded as a model which adjust the dimensionality of the multidimensional vector VEC to be consistent with the policy POL or the trained policy TPOL.

[0101] The projection trainer 141 provides the output of the projection adjustment model (PAM), whose dimensionality is consistent with the policy being trained (TPOL), to the policy trainer 142. The policy trainer 142 trains the policy being trained (TPOL) with reference to the output of the projection adjustment model (PAM). Then the output (response before PAM) generated by the policy being trained (TPOL) is provided to the inverse of the projection adjustment model (PAM). Then the inverse of the projection adjustment model (PAM) generates the offers or responses (including counter-offers) with reference to the output (response before PAM) generated by the policy being trained (TPOL). The response generated by the inverse of the projection adjustment model (PAM) is provided to the partner. The partner generates the next offer with reference to the response provided by the inverse of the projection adjustment model (PAM).

[0102] The training section 14 the training section 14 iterates above processes and trains both of the projection adjustment model (PAM) and policy being trained (TPOL) until a reward function or a cost function satisfied the predetermined condition. In this manner, the training of the projection adjustment model (PAM) and the training of policy POL are carried out simultaneously by the training section 14.

[0103] (Negotiation processes in negotiation system 100A) The following description will discuss a more detailed aspect of the negotiation processes carried out in the negotiation system 100A with reference to Fig. 9. Fig. 9 is a schematic illustration of data flow in the negotiation phase according to the second example embodiment.

[0104] In Fig. 9, the partner 50-1 corresponds to the negotiation agent 50-1 in Fig. 4. The multidimensional utility function (Ufun) projector (MUP) 12 correspond to the projection section 12 in Fig. 4. The up-projection module (UPM) 131 corresponds to the up-projection section 131 in Fig. 4, while the down-projection module (DPM) 132 corresponds to the down-projection section 132 in Fig. 4. The projection adjustment module (PAM) 133 corresponds to the projection adjustment section 133 in Fig. 4, while the multidimensional inverse utility function (Ufun) projector (IMUP) 134 corresponds to the inverse projection section 134 in Fig. 4. Here, the term "PAM" is also used to indicate the projection adjustment module 133 without any confusion.

[0105] As shown in Fig. 9, in the negotiation phase, the partner 50-1 provides an offer to the negotiation apparatus 1A, and the negotiation apparatus 1A receives the offer. More specifically, the obtaining section 11 obtains the offer provided by the partner 50-1 via the communication section 30. The obtaining section 11 provides the offer to the MUP 12. The MUP 12 project the offer (an example of outcomes OUT) obtained by the obtaining section 11 into a multidimensional vector VEC. Here, each component of the multidimensional vector represents a contribution to utility. Here, the multidimensional vector VEC generated by the MUP 12 is also referred to as a projected offer. The projected offer is provided to UPM 131. In an example, the UPM 131 increases the dimensionality of the multidimensional vector VEC to be consistent with the input dimensionality of the PAM 133, in a case where the PAM 133 is enabled. In another example, the UPM 131 increases the dimensionality of the multidimensional vector VEC to be consistent with the input dimensionality of negotiation policies (POL or TPOL), in a case where the PAM 133 is disabled. In yet another example, the UPM 131 simply outputs the multidimensional vector VEC in a case where the up-projection is not necessary.

[0106] The output of the UPM 131 is provided to the DPM 132. In an example, the DPM 132 decreases the dimensionality of the multidimensional vector VEC or the output of the UPM 131 to be consistent with the input dimensionality of the PAM 133, in a case where the PAM 133 is enabled. In another example, the DPM 131 decreases the dimensionality of the multidimensional vector VEC or the output of the UPM 131 to be consistent with the input dimensionality of negotiation policies (POL or TPOL), in a case where the PAM 133 is disabled. In yet another example, the DPM 132 simply outputs the multidimensional vector VEC or the output of the DPM 132 in a case where the down-projection is not necessary.

[0107] The output of the DPM 132 is provided to PAM 133. The PAM 133 adjusts the provided input to the trained policy TPOL by using the projection adjustment model (also referred to as PAM). The PAM 133 may be expressed as a module which receives inputs, and generates the outputs which enhance the availability of value functions (components) included in the utility function used in the negotiation apparatus 1A and / or partner agent 50-0, 50-1, or 50-2. In an example, the PAM 133 uses the projection adjustment model (also referred to as PAM) to adjust the dimensionality of the multidimensional vector VEC, the output of the UPM 131, or the output of the DPM 132 to be consistent with the trained negotiation policy TPOL.

[0108] In the generation section 13, the output of PAM 133 is input to the trained negotiation policy TPOL, and the generation section 13 generates the offers or responses (including counter-offers)with reference to the output of PAM 133 and the trained negotiation policy TPOL. Here, the response generated with reference to the trained negotiation policy TPOL may also be referred to as a projected response. Therefore, it may be expressed that the generation section 13 generates one or more projected outcomes (projected response) from the multidimensional vector VEC or from the output of PAM 133, based on one or more negotiation policies. The projected response is provided to the IMUP 134. The IMUP 134 generates a response by operating one or more inverse projection operations to the projected response. In other words, the generation section 13 generates the one or more responses including outcomes by inverse-projecting the one or more projected outcomes (projected responses). The response generated by the IMUP is provided to the partner 50-1, and then the partner 50-1 generates the next offer with reference to the response generated by the IMUP.

[0109] (More detailed processes in negotiation system 100A) The following description will discuss a more detailed processed carried out in the negotiation system 100A with reference to Figs. 10 to 14. Fig. 10 is a flowchart showing a flow of processes in a negotiation method according to an example embodiment.

[0110] (Step S111) In the step S111, the obtaining section 11 obtains the outcomes OUT and read issues included in the target outcome.

[0111] (Step S112) In the step S112, the obtaining section 11 determines whether a GLA utility function is applied or not in the negotiation processes. In a case where the GLA function is applied (YES in the step S112), the process proceeds to the step S113, while in a case where the GLA function is not applied (NO in the step S112), the process proceeds to step S116.

[0112] Note here that the use of GLA utility function contributes to enhancing the information utilization in the negotiation processes. Note also that because of the step S112, in a case where the GLA is not applied, the PAM is applied in the proceeding step S116. This also contributes to enhancing the information utilization in the negotiation processes.

[0113] Note also that the components of the above mentioned GLA utility function or LA utility function corresponds to the components of the multidimensional vector VEC which may be generated by the MUP (projection section) 12 in the proceeding step S121. Note that the value of GLA utility function or LA utility function is a scalar (one dimensional, or one resulting component), while the multidimensional vector VEC a vector which has a plurality of components. Because of this, the use of the multidimensional vector VEC contributes to enhancing the information utilization in the negotiation processes.

[0114] (Step S113, S114, S115) In a case where the GLA is applied, the obtaining section compares the value of n and the value of m in the step S113. Here, n indicates the number of issues included in the target outcome OUT, while m indicates the input dimensionality of the trained policy TPOL. The m may also be expressed as a projection cardinality. Note that the value of m is adjustable in the sense that the value of m may be determined in accordance with the user input obtained by the input-output section 40.

[0115] In a case where n is smaller than m (n < m), the UPM is enabled in the step S114, and then proceeds to the step S117. On the other hand, in a case where n is larger than m (n >m), the DPM is enabled in the step S115, and then proceeds to the step S117. In a case where n is equal to m (n=m), the process proceeds to the step S117.

[0116] (Step S117, S118, S119) In the step S117, the negotiation starts, and the obtaining section 11 obtains the offer (partner offer) provided by the partner agent 50-0, 50-1, or 50-2. If the negotiation is ended (YES in the step S113), the process proceeds to the step S139, otherwise (NO in the step S113) the process proceeds to the step S121.

[0117] (Step S121) In the step S121, the MUP 12 (projection section 12) carries out the multidimensional utility-function projection (also referred to as MUP). The processes carried out by the MUP 12 are explained above, and are therefore omitted here.

[0118] (Step S131) Then, in the step S131, the UPM 131 (up-projection section 131) runs the up-projection process. The processes carried out by the UPM 131 are explained above, and are therefore omitted here.

[0119] (Step S132) Then, in the step S132, the DPM 132 (down-projection section 132) runs the down-projection process. The processes carried out by the DPM 132 are explained above, and are therefore omitted here.

[0120] (Step S133) Then, in the step S133, the PAM 133 (projection adjustment section 133) runs the projection adjustment process. The processes carried out by the PAM 133 are explained above, and are therefore omitted here.

[0121] (Step S134) Then, in the step S134, the negotiation section 13 runs the (trained) negotiation policy (POL or TPOL) and generates the response (or projected response in the sense of Fig. 9). It may be expressed that in the step S134, the generation section 13 generates one or more projected outcomes (projected responses) from the multidimensional vector VEC, from the output of PAM, from the output of DPM, or from the output of UPM, based on one or more negotiation policies (POL or TPOL).

[0122] (Step S 135) Then, in the step S135, the generation section 13 determines whether to leave the negotiation or not. In a case where the generation section 13 determines to leave the negotiation (YES in the step S135), the process proceed to the step S139, otherwise (NO in the step S135) the process proceeds to the step S136.

[0123] (Step S 136) In the step S136, the generation section 13 determines whether to accept the offer provided by the partner offer provided by the partner agent 50-0, 50-1, or 50-2. In a case where the generation section 13 determines to accept the offer (YES in the step S136), the process proceeds to the step S139, otherwise (NO in the step S136) the process proceeds to the step S137.

[0124] (Step S137) In the step S137, the inverse projection section 134 runs the inverse projection processes. The inverse projection processes carried by the inverse projection section 134 (IMUP 134) includes: an inverse of the projection adjustment process, if PAM is enabled; an inverse of the down-projection process, if DPM is enabled; and an inverse of the up-projection process, if UPM is enabled.

[0125] The inverse projection section 134 generates the response including counter-offer with reference to the projected response generated by running the negotiation policy (POL or TPOL). Therefore, it may be expressed that in the step S137, the inverse projection section 134 generates the one or more outcomes (responses) by inverse-projecting the one or more projected outcomes (projected responses).

[0126] (Step S138) In the step S138, the response generated by the inverse projection section 134 is sent to the partner agent 50-0, 50-1, or 50-2, and then proceeds to the step S118.

[0127] (Step S139) In the step S139, the generation section 13 calculates the utility based on the utility function (GLA or LA) used in the negotiation apparatus 1A and then ends the negotiation processes.

[0128] (MUP) The top of Fig. 11 is a flowchart showing a flow of processes of MUP (multidimensional utility-function projector) according to an example embodiment. Fig. 11 corresponds to a detailed explanation of the step S121 in Fig. 10. It may be expressed that the MUP projects an outcome ω into a set of n numbers representing the value functions. It may also be expressed that the MUP takes an outcome OUT and generates n-dimensional representation (n-dimensional vector VEC) by projecting on the utility space with n value functions, v1, ..., vn.

[0129] (Step S1211) In the step S1211, the projection section 12 reads (obtains) outcome ω (small omega) of dimensionality n. In other words, the outcome ω has n components, ω1, ... , ωn. In an example, the outcome ω corresponds to the partner offer.

[0130] (Step S1212) In the step S1212, the projection section 12 sets a parameter i to be 0 (i=0), and a vector x to be a zero vector, or to have no component (x=< >). Note that the vector x corresponds to the multidimensional vector VEC which is now in a process of generation.

[0131] (Step S1213) Then, in the step S1213, the projection section 12 compares i and n. In a case where i is smaller than n (i<n), the process proceeds to step S1214, otherwise the process ends.

[0132] (Step S1214, S1215) In the step S1214, in an example, the projection section 12 may append vi(ωi) to the vector x. Here the vi(ωi) represents a value function or components in the vector x. More particularly, in the step S1214, the projection section 12 may append a component vi(ωi) to the vector x x = (v1(ω1), ..., vi-1(ωi-1)) and obtains an updated vector x x = (v1(ω1), ..., vi-1(ωi-1), vi(ωi)). The above example corresponds to a case of using GLA utility function. Alternatively, the projection section 12 may append ωito the vector x, which corresponds to a case of using LA utility function. More particularly, in the step S1214, the projection section 12 may append a component ωito the vector x x = (ω1, ..., ωi-1) and obtains an updated vector x x = (ω1, ..., ωi-1, ωi). Then, in the step S1215, the projection section 12 returns the updated vector x and proceeds to the Step S1213.

[0133] By carrying out the above steps, the projection section 12 generates the multidimensional vector VEC as a final form of the vector x. Here, the multidimensional vector VEC (x) can be represented as x = (v1(ω1), ..., vi(ωi), ..., vn(ωn)), in a case of GLA, or x = (ω1, ..., ωi, ..., ωn), in a case of LA.

[0134] In other words, by carrying out the above steps, the projection section 12 projects the outcome ω into a multidimensional vector VEC. Note that each component of the multidimensional vector VEC, vi(ωi) or ωirepresents a contribution to utility. In other words, an aggregation or a (linear or non-linear) combination of the components of the multidimensional vector VEC may form a utility function u(ω) as shown in Fig. 5.

[0135] (IMUP) The bottom of Fig. 11 is a flowchart showing a flow of processes of IMUP (inverse multidimensional utility-function projector) according to an example embodiment. Fig. 11 corresponds to a detailed explanation of the step S137 in Fig. 10. It may be expressed that the IMUP finds an outcome ω that corresponds to an input vector of dimensionality m. It may also be expressed that the IMUP takes an m-dimensional vector (representing adjusted value functions) and returns an outcome with a utility approximately equal to

[0136] (Step S1371, S1372) In the step S1371, the inverse projection section 134 read input x of dimensionality m. Here, the input x corresponds to the output of the negotiation policy (POL or TPOL), generated in the step S134 in Fig, 10. Note that the value of m is adjustable in the sense that the value of m may be determined in accordance with the user input obtained by the input-output section 40. In the step S1372, the inverse projection section 134 identifies whether the PAM has been enabled or not. In a case where the PAM has been enabled (YES in the step S1372), the process proceeds to the step S1373, otherwise (NO in the step S1373), the process proceeds to the step S1374.

[0137] (Step S1373) In a case where the PAM has been enabled, the inverse projection section 134 applies an inverse operation of PAM on the input x. In other words, if the PAM is expressed as a function FPAM, the inverse projection section 134 applies an inverse function FPAM-1on the input x. It may also be expressed that the inverse projection section 134 solves an inverse problem specified by FPAM-1and the input x.

[0138] (Step S1374) In the step S1374, the inverse projection section 134 identifies whether the DPM has been enabled or not. In a case where the DPM has been enabled (YES in the step S1374), the process proceeds to the step S1375, otherwise (NO in the step S1374), the process proceeds to the step S1376.

[0139] (Step S1375) In a case where the DPM has been enabled, the inverse projection section 134 applies an inverse operation of DPM on the input x, or on the output of the inverse PAM. In other words, if the DPM is expressed as a function FDPM, the inverse projection section 134 applies an inverse function FDPM-1on the input x, or on the output of the inverse PAM. It may also be expressed that the inverse projection section 134 solves an inverse problem specified by FDPM-1and the input x, or by FDPM-1and the output of the inverse PAM (Step S1376) In the step S1376, the inverse projection section 134 identifies whether the UPM has been enabled or not. In a case where the UPM has been enabled (YES in the step S1376), the process proceeds to the step S1377, otherwise (NO in the step S1376), the process proceeds to the step S1378.

[0140] (Step S1377) In a case where the UPM has been enabled, the inverse projection section 134 applies an inverse operation of UPM on the input x, on the output of the inverse PAM, or on the output of the inverse DPM. In other words, if the UPM is expressed as a function FUPM, the inverse projection section 134 applies an inverse function FUPM-1on the input x, on the output of the inverse PAM, or on the output of the inverse DPM. It may also be expressed that the inverse projection section 134 solves an inverse problem specified by FUPM-1and the input x, by FUPM-1and the output of the inverse PAM, or by FUPM-1and the output of the inverse DPM.

[0141] (Step S1378) In the step S1378, the inverse projection section 134 finds and return ω*, according to Here, ω*corresponds to a response generated by the inverse projection section 134, and u(ω) in the argument represents the utility function (LA or GLA). The y in the argument represents one of the following: the input x, in a case where PAM, DPM, and UPM all disabled; the output of the inverse PAM, in a case where PAM is enabled, and DPM and UPM disabled; the output of the inverse DPM, in a case where DPM is enabled, and UPM disabled; and the output of the inverse UPM, in a case where UPM is enabled. The step S1378 may be expressed as a step to find an outcome ω*which corresponds to y. The step S1378 may also be expressed as a step to find an outcome ω*which corresponds to the output of the negotiation policy (POL or TPOL).

[0142] (Remarks on IMUP) The goal of every “Invert” blocks (S1373, S1375, S1377) in the bottom of Fig. 11 is to reverse the operation of the corresponding block (respectively, S133, S132, S131 in Fig. 10). Inverting PAM can be achieved as follows: During training, PAM is trained using an auto encoder. The encoder part is used as the PAM model and the decoder part as the inverter (inverse PAM).

[0143] Other possible implementations are described as follows.

[0144] Back Projection Method: A common method for inverting all other modules is to train a decoder that takes the output of that module and return the input with appropriate regularization. This will work for both DPM and UPM.

[0145] Sampling Method: Another approach is to find all possible outcomes that have a total utility within ε(epsilon) from the utility corresponding to the input. We then select one of them according to a predefined rule: In practice, a rule that works well is to sample an outcome from the top ftpercent with precedence to ones not offered before where ftis a monotonically decreasing function of negotiation time falling back to the top outcome in the set if none satisfies this condition.

[0146] Specific projection methods may have corresponding inversion method. For example, if DPM is done using PCA, we can use the transpose of the projection matrix to do the inversion.

[0147] (UPM) Fig. 12 illustrates flowcharts showing flows of processes of UPM (up-projection module) according to an example embodiment. It may be expressed that the UPM increases the number of issues from n to m. It may also be expressed that the UPM changes the dimensionality of a vector from n to m when m>n. If m<n or m=n, the UPM outputs the input unchanged. In an example, the UPM recursively add new issues based on the most important issue (i.e. one with highest weight) until the UPM obtains the correct (appropriate) number of issues (m). The top of Fig. 12 shows a flow of UPM before a negotiation starts. The top of Fig. 12 corresponds to the step S111 and the step S114 in Fig. 10. On the other hand, the bottom of Fig. 12 shows a flow of UPM after a negotiation starts. The bottom of Fig. 12 corresponds to the step S131 in Fig. 10.

[0148] (Step S111) Before the negotiation starts, the obtaining section 11 obtains the outcomes OUT and read issues (a set of issues) included in the target outcome OUT.

[0149] (Step S1141) Then, in the step S1141, the generation section 13 identifies whether the UPM is enabled or not. In a case where the UPM is enabled, the process proceeds to the step S1142, otherwise the process ends.

[0150] (Step S1142) In the step S1142, the up-projection section 131 compares n and m. Here, in an example, n indicates the number of issues included in the target outcome OUT, while m indicates the input dimensionality of the negotiation policy (POL or TPOL), or the input dimensionality of PAM. In a case where n is smaller than m (n<m), the process proceeds to the step S1143, otherwise the process ends.

[0151] (Step S1143, S1144) In the step S1143, the up-projection section 131 finds issue with largest weight (find Imaxwith weight αmax(alpha_max)). Then, in the step S1144, in an example, the up-projection section 131 may update the αmaxaccording to Here, the calligraphic N indicates sampling a value in accordance with the (normal) distribution specified above, while min and max indicate taking a minimum or a maximum of the arguments. The factor max() in Eq. 18-1 corresponds to adding a noise to αnewor αmaxin accordance with the above mentioned (normal) distribution. The specific values of 3 and 8 do not limit the present example embodiment. Actually, the specific value of m, 3 and 8 are adjustable in the sense that these values may be determined in accordance with the user input obtained by the input-output section 40.

[0152] In another example, in the step S1144, the up-projection section 131 may update the αmaxaccording to The above example corresponds to updating the αmaxwithout any contribution of noise factor.

[0153] (Step S1145) Then, in the step S1145, the up-projection section 131 adds, to the set of issues, a new issue Inewwith the same value as Imaxand with a weight αnew(while updating Imax' weight). Then, the process proceeds to the step S1142.

[0154] (Step S1311, S1312) After the negotiation starts, in the step S1311, the up-projection section 131 read (obtains) input of dimensionality n. Here, the input corresponds to the offer provided by the partner (opponent) agents 50-0, 50-1, or 50-2. Then, in the step S1312, the up-projection section 131 carries out up-projection of the multidimensional vector VEC using new m issues generated in the above steps.

[0155] As explained above, the up-projection section 131 increases the dimensionality of the multidimensional vector VEC to be consistent with the dimensionality of the one or more negotiation policies or the input dimensionality of PAM. More specifically, the up-projection section 131 increases the dimensionality of the multidimensional vector VEC by decomposing the issue with highest weight, the issue being included in the obtained outcomes OUT.

[0156] (DPM (Iterative implementation)) Fig. 13 illustrates flowcharts showing flows of processes of DPM (down-projection module) according to an example embodiment. It may be expressed that the DPM decreases the number of issues from n to m. It may also be expressed that the DPM changes the dimensionality of a vector from n to m when m<n. If m>n or m=n, the DPM outputs the input unchanged. In this example, the DPM recursively removes the least important issue (i.e. one with lowest weight) by combining it with another issue until the DPM obtains the correct (appropriate) number of issues (m). The top of Fig. 13 shows a flow of an iterative implementation of DPM before a negotiation starts. The top of Fig. 13 corresponds to the step S111 and the step S115 in Fig. 10. On the other hand, the bottom of Fig. 13 shows a flow of an iterative implementation of DPM after a negotiation starts. The bottom of Fig. 13 corresponds to the step S132 in Fig. 10.

[0157] (Step S111) Before the negotiation starts, the obtaining section 11 obtains the outcomes OUT and read issues (a set of issues) included in the target outcome OUT.

[0158] (Step S1151) Then, in the step S1151, the generation section 13 identifies whether the DPM is enabled or not. In a case where the DPM is enabled, the process proceeds to the step S1152, otherwise the process ends.

[0159] (Step S1152) In the step S1152, the down-projection section 132 compares n and m. Here, in an example, n indicates the number of issues included in the target outcome OUT, while m indicates the input dimensionality of the negotiation policy (POL or TPOL), or the input dimensionality of PAM. In a case where n is larger than m (n>m), the process proceeds to the step S1153, otherwise the process ends.

[0160] (Step S1153, S1154) In the step S1153, the down-projection section 132 finds issues with lowest (or lower) weight(s) (find Ix, Iywith αx(alpha_x), αy(alpha_y) and values Vx, Vy). Then, in the step S1154, in an example, the down-projection section 131 may generate a new issue Inewwith a value Vnewas a product of Vxand Vy, and with a new weight αnew= αx+ αy.

[0161] (Step S 1155) Then, in the step S1155, the down-projection section 132 adds, to the set of issues, the new issue Inewwith the weight αnewand removes the two issues Ixand Iy. Then, the process proceeds to the step S1152.

[0162] (Step S1321, S1322) After the negotiation starts, in the step S 1321, the down-projection section 132 read (obtains) input of dimensionality n. Here, the input corresponds to the offer provided by the partner (opponent) agents 50-0, 50-1, or 50-2. Then, in the step S1322, the down-projection section 132 carries out down-projection of the multidimensional vector VEC using new m issues generated in the above steps.

[0163] As explained above, the down-projection section 132 decreases the dimensionality of the multidimensional vector VEC to be consistent with the dimensionality of the one or more negotiation policies or the input dimensionality of PAM. More specifically, the down-projection section 132 decreases the dimensionality of the multidimensional vector VEC by combining issues with lower weights, the issues being included in the obtained outcomes OUT.

[0164] (DPM (PCA implementation)) Fig. 14 illustrates flowcharts showing flows of processes of DPM (down-projection module) according to an example embodiment. It may be expressed that the DPM decreases the number of issues from n to m. In this example, the DPM applies PCA (Principal component analysis) to obtain a reduced number of issues directly. Each resulting issue may be a linear combination of the original set of issues. The top of Fig. 14 shows a flow of a PCA implementation of DPM before a negotiation starts. The top of Fig. 14 corresponds to the step S111 and the step S115 in Fig. 10. On the other hand, the bottom of Fig. 14 shows a flow of an iterative implementation of DPM after a negotiation starts. The bottom of Fig. 14 corresponds to the step S132 in Fig. 10.

[0165] (Step S111) Before the negotiation starts, the obtaining section 11 obtains the outcomes OUT and read issues (a set of issues) included in the target outcome OUT.

[0166] (Step S1151) Then, in the step S1151, the generation section 13 identifies whether the DPM is enabled or not. In a case where the DPM is enabled, the process proceeds to the step S1152, otherwise the process ends.

[0167] (Step S1152) In the step S1152, the down-projection section 132 compares n and m. Here, in an example, n indicates the number of issues included in the target outcome OUT, while m indicates the input dimensionality of the negotiation policy (POL or TPOL), or the input dimensionality of PAM. In a case where n is larger than m (n>m), the process proceeds to the step S1156, otherwise the process ends.

[0168] (Step S1156) In the step S1156, the down-projection section 132 generates a matrix M of size(|Ω|by n ) with rows representing the outcomes OUT and columns representing value functions multiplied by weights. More particularly, the components of the matrix M can be expresses as (Step S1157) Then, in the step S1157, the down-projection section 132 applies PCA (e.g. using SVD (singular value decomposition)) to the matrix M and keep the top m principal vectors V1, ..., Vmwith their principal values β1, ..., βm.

[0169] (Step S1158) Then, in the step S1158, the down-projection section 132 creates m new issues Inew1, ..., Inewm, each of which has values Ω and weight αicomputed by Then, the process proceeds to the step S1152.

[0170] (Step S1321, S1322) After the negotiation starts, in the step S 1321, the down-projection section 132 read (obtains) input of dimensionality n. Here, the input corresponds to the offer provided by the partner (opponent) agents 50-0, 50-1, or 50-2. Then, in the step S1322, the down-projection section 132 carries out down-projection of the multidimensional vector VEC using new m issues generated in the above steps.

[0171] As explained above, the down-projection section 132 decreases the dimensionality of the multidimensional vector VEC to be consistent with the dimensionality of the one or more negotiation policies or the input dimensionality of PAM. More specifically, the down-projection section 132 decreases the dimensionality of the multidimensional vector VEC by projecting the issues (more particularly, by applying PCA on the issues), the issues being included in the obtained outcomes OUT.

[0172] Note here that the implementation of DPM is not limited to the above examples. The DPM may be implemented by nonlinear methods (e.g. IsoMAP), or by trained models (e.g. auto-encoders).

[0173] (Advantageous effects of the negotiation system 100, 100A) The following description will discuss advantageous effects of the negotiation system 100A with reference to Fig. 15. Fig. 15 is a schematic illustration showing effects of the example embodiments.

[0174] As mentioned above, automated negotiation is becoming more important, for example, in business operations. Autonomous agents carrying out automated negotiations need to adapt to different scenarios. However, the existing methods are either too specific or unable to utilize all the information available to the agent during learning.

[0175] According to the configurations explained in the above example embodiments, the negotiation apparatus 10, 1A (negotiating agent) can adapt to a wide variety of applications which utilizing as much information as possible to respond optimally to their partner(50-0, 50-1, or 50-2)’s behavior (offer). This leads to better agreements (i.e. higher utilities) for the negotiating agent. The above explained methods can be applied to any application that requires automated negotiation, including the areas of manufacturing, transportation, etc..

[0176] Fig. 15 shows a schematic space spanned by a degree of generality and a degree of information utilization. All existing negotiation strategies use one of two representations when training the agent, Projection on Utility Space (indicated as A in Fig. 15), or Raw outcomes (indicated as B in Fig. 15). Here, the method of Projection on Utility Space has sufficient generality, but it has a limited information utilization. In other words, the method of Projection on Utility Space cannot use the structure of the utility function (i.e. loses important information). On the other hand, the method of Raw outcomes has sufficient information utilization, but it has a limited generality. In other words, the method of Raw outcomes is specific to a single domain (i.e. specific set of issues), and / or is specific to a single (or small set) of utility functions for self and the partner.

[0177] According to the example embodiments, as explained above, the negotiation apparatus 10, 1A projects the issues to a multidimensional space (multidimensional vector) based on value functions of individual or subsets of issues. This provides advantageous effects such as: as general as Projection on Utility Space, and almost uses all information used by Raw Outcomes.

[0178] More specifically, the negotiation apparatus 10, 1A with the MUP (and IMUP) according to the example embodiments has the following aspects.

[0179] Compared with Projection on Utility Space, the negotiation apparatus 10, 1A can keeps more information by having multiple values to represent each outcome OUT, although it is slightly less general because of the fixed number of value functions are used.

[0180] Compared with Raw Outcomes, the negotiation apparatus 10, 1A provides more generality because it does not depend on exact values of outcomes, although it utilizes less information because multiple outcomes may have the same utility for self but not for the partner.

[0181] On the other hand, the methods according to the example embodiments may introduces the following subsidiary problems SP1 to SP4 in some cases: (SP1) How to handle a case of too few value functions? (SP2) How to care a case of too many value functions? (SP3) How to care a case with unavailability of value functions for self ufun (utility function of the negotiation apparatus 10, 1A)? (SP4) How to care a case with unavailability of value functions for partner ufun(utility function of the partner agent 50-0, 50-1, or 50-2)? According to the example embodiments, as explained above, the solutions to the above subsidiary problems are also appropriately provided. More specifically, in an example, the negotiation apparatus 10, 1A with the UPM and the DPM can increase generality by allowing for variable number of value functions, and therefore solves the above mentioned subsidiary problems SP1 and SP2.

[0182] In another example, the negotiation apparatus 10, 1A with the PAM and the PAM's trainer (the projection trainer 141) can increase information utilization by better modeling of own utility function (no opponent model needed), and therefore solves the above mentioned subsidiary problem SP3. The negotiation apparatus 10, 1A with the PAM and the PAM's trainer (the projection trainer 141) can also increase information utilization further by better modeling of partner utility function (in a case where the opponent model is available), and therefore solves the above mentioned subsidiary problem SP4.

[0183] In the space spanned by a degree of generality and a degree of information utilization, the MUP and IMUP contribute to the improvement indicated by 1 and 2 in Fig. 15. The UPM and the DPM contribute to the improvement indicated by 3 and 4 in Fig. 15, while the PAM and the PAM's trainer contribute to the improvement indicated by 5 and 6 in Fig. 15.

[0184] (Application example) The following description will discuss an application example of the negotiation system 100, 100A.

[0185] (Problem settings and comparison with the existing methods) This application example deals with a case where Company A negotiates with suppliers B, C, D, E, F about buying some inputs. More particularly, this application example deals with the following problem setting: In negotiations with B, the only issue is Quantity (1,2,3).

[0186] In negotiations with C, D the issues are Quantity (1,2,3,4) and Size (10 to 11, fractions allowed).

[0187] In negotiations with E, F the issues are Quantity (1,2,3), Size (10, 11), Color (Red or Blue for E and Green or Brown for F).

[0188] Depending on market conditions, the utility function of all companies can change dramatically. For simplicity we assume that all utility functions are Linear Aggregation (LA). The negotiation apparatus 10, 1A is supposed to function as a negotiating agent for company A.

[0189] Prior art methods (existing methods) can do one of two things for company A: Raw Outcomes method: Train three sets of agents (one for B, one for C,D and one for E,F). Each set will have a very large number of different agents each must be trained independently (one for each possible combination of A’s utility function and its partner’s utility function). The reason is that any large change in utility functions will change the meaning of any given outcome. For example if red color is better than blue, outcome (10, 10, red) will be better than (10, 10, blue) and vice versa.

[0190] Projected Utilities Method: Train one agent that can theoretically negotiate with all these companies. This agent is NOT likely to be good. For example, it cannot notice that company C cares about quantity more than price because it only sees utility values and never sees the outcome itself.

[0191] Therefore, no prior art can solve this problem. The negotiation apparatus 10, 1A according to the example embodiments can solve this problem by training one agent that is capable of negotiating with ALL companies and getting good agreements with all of them (compared with Projected Utilities Method).

[0192] (Detailed explanation of application example) In this application example, it is assumed that we have the following negotiations with the following utility functions:

[0193] The following explanation shows the components MUP, IMUP, DPM, UPM showing how each contribute to the solution of the problem. We assume for simplicity that PAM is disabled. We will use m=2 for all modules (MUP, IMUP, DPM, UPM).

[0194] We will consider the case where quantity is 2, size is 10 and color is blue. We have three issues.

[0195] Calculating all utilities for all negotiation (Sanity check): We assume that we trained a policy with two inputs and two outputs (i.e. m=2). Just for simplicity let’s assume that the policy is given by a function f:

[0196] We rewrite the ufun (utility-function) as where δ (delta) represents a delta function which take a value 1 is the argument is true, otherwise takes a value 0. We have, for example, Here, the reason we needed to do this rewriting is to make u5Aa valid GLA utility function satisfying all conditions.

[0197] Then, applying MUP, we have which leads to 4-dimensional representation of the outcome that depends on value functions. Because m= 2 and n=4 issues, we cannot directly use the policy f. That is why we need the optional parts (UPM, DPM, PAM).

[0198] Then, applying UPM which is disabled because n > m, we have Then, applying DPM which is enabled because n > m, we have: Before negotiation: We combine groups 3, 4 then the resulting group with 1 leading to the following form: During Negotiation: we apply the newly created 2-Issue system leading to:

[0199] Then, applying PAM which is disabled because we do not use it in this example. we have Then, applying the policy f we have,

[0200] Then, applying IMPU, we have: PAM is disabled, therefore, skip Inverse PAM UPM is enabled, therefore, applying Inverse UPM. This simply means just using the new values of the weights and value functions instead of the original values in the inversion procedure DPM is disabled, therefore, skip Inverse DPM.

[0201] Solve the following optimization problem: Calculate the target utility y and outcome ω*: Thus, in an example, we have the counter-offer: quantity is 3, size is 11 and color is red.

[0202] Note that the above process is repeated until agreement is reached or the policy ends the negotiation, timeout is reached or the partner leaves the negotiation. In the example above, we can find all outcomes around the given utility then select the one that minimizes the difference to each of the value functions after the DPM or in the original configuration. Note that, in this example, we ignored acceptance and leaving outcomes of the policy because there is no difference between our proposal and prior art regarding this part of the negotiation strategy.

[0203] (Example of display information) Fig. 15 shows an example of display information generated by the generation section 13. The display information may be output via the input-output section 40. As shown in Fig. 16, the display information may include a received offer, an output dimensionality of MUP, an input dimensionality of the policy, and a response generated by the generation section 13.

[0204] <Example of configuration achieved by software> One or some of or all of the functions of the negotiation apparatus 10, 1A can be realized by hardware such as an integrated circuit (IC chip) or can be alternatively realized by software.

[0205] In the latter case, each of the negotiation apparatus 10, 1A is realized by, for example, a computer that executes instructions of a program that is software realizing the foregoing functions. Fig. 17 illustrates an example of such a computer (hereinafter, referred to as "computer C"). The computer C includes at least one processor C1 and at least one memory C2. The memory C2 stores a program P for causing the computer C to function as any of the negotiation apparatus 10, 1A. In the computer C, the processor C1 reads the program P from the memory C2 and executes the program P, so that the functions of any of the negotiation apparatus 10, 1A are realized.

[0206] As the processor C1, for example, it is possible to use a central processing unit (CPU), a graphic processing unit (GPU), a digital signal processor (DSP), a micro processing unit (MPU), a floating point number processing unit (FPU), a physics processing unit (PPU), a microcontroller, or a combination of these. The memory C2 can be, for example, a flash memory, a hard disk drive (HDD), a solid state drive (SSD), or a combination of these.

[0207] Note that the computer C can further include a random access memory (RAM) in which the program P is loaded when the program P is executed and in which various kinds of data are temporarily stored. The computer C can further include a communication interface for carrying out transmission and reception of data with other devices. The computer C can further include an input-output interface for connecting input-output devices such as a keyboard, a mouse, a display, and a printer.

[0208] The program P can be stored in a non-transitory tangible storage medium M which is readable by the computer C. The storage medium M can be, for example, a tape, a disk, a card, a semiconductor memory, a programmable logic circuit, or the like. The computer C can obtain the program P via the storage medium M. The program P can be transmitted via a transmission medium. The transmission medium can be, for example, a communications network, a broadcast wave, or the like. The computer C can obtain the program P also via such a transmission medium.

[0209] <Additional Remark 1> The present invention is not limited to the foregoing example embodiments, but may be altered in various ways by a skilled person within the scope of the claims. For example, the present invention also encompasses, in its technical scope, any example embodiment derived by properly combining technical means disclosed in the foregoing example embodiments.

[0210] <Additional Remark 2> The whole or part of the example embodiments disclosed above can be described as follows. Note, however, that the present invention is not limited to the following example aspects.

[0211] (Aspect A1) A negotiation method comprising obtaining one or more outcomes for one or more negotiation policies, projecting, by a projector, the obtained outcomes into a multidimensional vector, each component of which represents a contribution to utility, and generating, by a generator, one or more outcomes with reference to the multidimensional vector.

[0212] (Aspect A2) A negotiation method according to Aspect A1, wherein generating one or more outcomes includes generating one or more projected outcomes from the multidimensional vector based on one or more negotiation policies, and generating the one or more outcomes by inverse-projecting the one or more projected outcomes.

[0213] (Aspect A3) A negotiation method according to Aspect A2 further comprising increasing the dimensionality of the multidimensional vector to be consistent with the dimensionality of the one or more negotiation policies.

[0214] (Aspect A4) A negotiation method according to Aspect A3, wherein increasing the dimensionality of the multidimensional vector includes decomposing the issue with highest weight, the issue being included in the obtained outcomes.

[0215] (Aspect A5) A negotiation method according to Aspect A2 further comprising decreasing the dimensionality of the multidimensional vector to be consistent with the dimensionality of the one or more negotiation policies.

[0216] (Aspect A6) A negotiation method according to Aspect A5, wherein decreasing the dimensionality of the multidimensional vector includes combining issues with lower weights, the issues being included in the obtained outcomes.

[0217] (Aspect A7) A negotiation method according to Aspect A5, wherein decreasing the dimensionality of the multidimensional vector includes projecting the issues included in the obtained outcomes.

[0218] (Aspect A8) A negotiation method according to any one of Aspect A2 to A7, wherein the one or more negotiation policies are represented as one or more trained negotiation policies, and generating one or more outcomes includes adjusting the dimensionality of the multidimensional vector to be consistent with the one or more trained negotiation policies.

[0219] (Aspect A9) A negotiation method according to Aspect A8 further comprising training the projector and the one or more negotiation policies simultaneously.

[0220] (Aspect A10) A negotiation apparatus comprising obtaining means to obtain one or more outcomes for one or more negotiation policies, projection means to project the outcomes obtained by the obtaining means into a multidimensional vector, each component of which represents a contribution to utility, and generation means to generate one or more outcomes with reference to the multidimensional vector.

[0221] (Aspect A11) A negotiation system comprising obtaining means to obtain one or more outcomes for one or more negotiation policies, projecting means to project the outcomes obtained by the obtaining means into a multidimensional vector, each component of which represents a contribution to utility, and generating means to generate one or more outcomes with reference to the multidimensional vector.

[0222] (Aspect A12) A negotiation program causing a computer to function as: obtaining means to obtain one or more outcomes for one or more negotiation policies; projecting means to project the outcomes obtained by the obtaining means into a multidimensional vector, each component of which represents a contribution to utility; and generating means to generate one or more outcomes with reference to the multidimensional vector.

[0223] <Additional Remark 3> The whole or part of the example embodiments disclosed above can be described as follows. Note, however, that the present invention is not limited to the following example aspects.

[0224] (Aspect B1) A method for representing outcomes for negotiation policies in the form of a utility dependent multidimensional vector based on value functions or trained approximations of them as GLA utility functions.

[0225] (Aspect B2) A method for representing utility functions with a given number of issues using a GLA utility function using a higher number of issues (UPM) in an invertable manner. In one embodiment this is achieved using iterative decomposition of the issue with highest weight.

[0226] (Aspect B3) A method for representing utility functions with a given number of issues using a GLA utility function using a lower number of issues (DPM) in an invertable manner. In one embodiment this is achieved using iterative combination of the two issues with lowest weight. In another embodiment this is achieved using linear projection (e.g. PCA).

[0227] (Aspect B4) A revirsable method for representing a utility function with a given number of issues an approximately equivalent utility function with a different number of issues (PAM). In one embodiment this is achieved using auto-encoders.

[0228] (Aspect B5) A method for training negotiation strategies using the representation in Aspect B1 that allows for simultaneous training of the negotiation policy and the projection method used to create the representation in Aspect B1 (when using PAM).

[0229] (Aspect B6) A system for negotiating with different partners in different negotiation context using the representation in Aspect B1 by applying MUP, UPM, DPM, PAM followed by the trained policy followed by the IMUP.

[0230] S1 Negotiation Method 10, 1A Negotiation Apparatus 100, 100A Negotiation System 11 Obtaining Section 12 Projection Section 13 Generation Section 131 Up-projection Section 132 Down-projection Section 133 Projection Adjustment Section 134 Inverse Projection Section 14 Training Section 141 Projection Training Section 142 Policy Training Section

Claims

1. A negotiation method comprising obtaining one or more outcomes for one or more negotiation policies, projecting, by a projector, the obtained outcomes into a multidimensional vector, each component of which represents a contribution to utility, and generating, by a generator, one or more outcomes with reference to the multidimensional vector.

2. The negotiation method according to Claim 1, wherein generating one or more outcomes includes generating one or more projected outcomes from the multidimensional vector based on one or more negotiation policies, and generating the one or more outcomes by inverse-projecting the one or more projected outcomes.

3. The negotiation method according to Claim 2 further comprising increasing the dimensionality of the multidimensional vector to be consistent with the dimensionality of the one or more negotiation policies.

4. The negotiation method according to Claim 3, wherein increasing the dimensionality of the multidimensional vector includes decomposing the issue with highest weight, the issue being included in the obtained outcomes.

5. The negotiation method according to Claim 2 further comprising decreasing the dimensionality of the multidimensional vector to be consistent with the dimensionality of the one or more negotiation policies.

6. The negotiation method according to Claim 5, wherein decreasing the dimensionality of the multidimensional vector includes combining issues with lower weights, the issues being included in the obtained outcomes.

7. The negotiation method according to Claim 5, wherein decreasing the dimensionality of the multidimensional vector includes projecting the issues included in the obtained outcomes.

8. The negotiation method according to any one of Claims 2 to 7, wherein the one or more negotiation policies are represented as one or more trained negotiation policies, and generating one or more outcomes includes adjusting the dimensionality of the multidimensional vector to be consistent with the one or more trained negotiation policies.

9. The negotiation method according to Claim 8 further comprising training the projector and the one or more negotiation policies simultaneously.

10. A negotiation method comprising: obtaining one or more outcomes for one or more negotiation policies; and generating, by a generator, one or more outcomes with reference to the obtained outcomes, wherein generating one or more outcomes includes: generating one or more projected outcomes from the obtained outcomes based on one or more negotiation policies; and generating the one or more outcomes by inverse-projecting the one or more projected outcomes.

11. A negotiation apparatus comprising obtaining means to obtain one or more outcomes for one or more negotiation policies, projection means to project the outcomes obtained by the obtaining means into a multidimensional vector, each component of which represents a contribution to utility, and generation means to generate one or more outcomes with reference to the multidimensional vector.

12. A negotiation apparatus comprising: obtaining means to obtain one or more outcomes for one or more negotiation policies; and generation means to generate one or more outcomes with reference to the obtained outcomes, wherein the generation means carries out: generating one or more projected outcomes from the obtained outcomes based on one or more negotiation policies; and generating the one or more outcomes by inverse-projecting the one or more projected outcomes.

13. A negotiation system comprising obtaining means to obtain one or more outcomes for one or more negotiation policies, projecting means to project the outcomes obtained by the obtaining means into a multidimensional vector, each component of which represents a contribution to utility, and generating means to generate one or more outcomes with reference to the multidimensional vector.

14. A negotiation system comprising: obtaining means to obtain one or more outcomes for one or more negotiation policies; and generation means to generate one or more outcomes with reference to the obtained outcomes, wherein the generation means carries out: generating one or more projected outcomes from the obtained outcomes based on one or more negotiation policies; and generating the one or more outcomes by inverse-projecting the one or more projected outcomes.

15. A negotiation program causing a computer to function as: obtaining means to obtain one or more outcomes for one or more negotiation policies; projecting means to project the outcomes obtained by the obtaining means into a multidimensional vector, each component of which represents a contribution to utility; and generating means to generate one or more outcomes with reference to the multidimensional vector.

16. A negotiation program causing a computer to function as: obtaining means to obtain one or more outcomes for one or more negotiation policies; and generation means to generate one or more outcomes with reference to the obtained outcomes, wherein the generation means carries out: generating one or more projected outcomes from the obtained outcomes based on one or more negotiation policies; and generating the one or more outcomes by inverse-projecting the one or more projected outcomes.

Images