Performing multi-objective tasks through the original network and the dual networks trained

Abstract

The example system includes a processor for receiving data for a multi-objective task. The processor also performs the multi-objective task on the received data via a trained original network. The original network and the dual network are trained on the multi-objective task using a Lagrange loss function representing the multiple objectives. The original network is trained to minimize the Lagrange loss function, and the dual network is trained to maximize the Lagrange loss function.

Description

Background Technology

[0001] This technique relates to artificial neural networks. More specifically, this technique relates to training and using neural networks to perform multi-objective tasks. Summary of the Invention

[0002] According to the embodiments described herein, a system includes a processor for receiving data for a multi-objective task. The processor can further perform the multi-objective task on the received data via a trained original network, wherein the original network and a dual network are trained for the multi-objective task using a Lagrange loss function representing the multiple objectives. The original network is trained to minimize the Lagrange loss function, and the dual network is trained to maximize the Lagrange loss function.

[0003] According to another embodiment described herein, a method includes training an original network and a dual network for a multi-objective task using a Lagrange loss function representing multiple objectives. Training the original network and the dual network includes training the original network to minimize the Lagrange loss function and training the dual network to maximize the Lagrange loss function. The method may further include receiving data for the multi-objective task. The method may also include performing the multi-objective task on the received data via the trained original network.

[0004] According to one embodiment, a computer program product is provided, the computer program product including program code means adapted to perform the method described in paragraph 3 or paragraph 7 when the program is run on a computer.

[0005] According to another embodiment described herein, a computer program product for training a neural network to perform a multi-objective task includes a computer-readable storage medium having program code embodied therein. The computer-readable storage medium itself is not a transient signal. The program code is executable by a processor such that the processor trains an original network and a dual network for the multi-objective task using a Lagrange loss function representing the multi-objective task. The program code also causes the processor to train the original network to minimize the Lagrange loss function and to train the dual network to maximize the Lagrange loss function. The program code can also cause the processor to receive data for the multi-objective task. The program code can also cause the processor to perform the multi-objective task on the received data via the trained original network.

[0006] According to one embodiment, a system is provided, including a processor for: receiving a prefix and text input of a session; and generating a completed response based on the prefix and text input of the session via a trained primary network, wherein the primary network is trained to minimize a Lagrange loss function representing a multi-objective, and a dual network is trained to maximize the Lagrange loss function.

[0007] According to one embodiment, a computer-implemented method is provided, comprising: receiving a prefix and text input of a session; and generating a completed response based on the prefix and text input of the session via a trained original network, wherein the trained original network is trained to minimize a Lagrange loss function representing multiple objectives, and a dual network is trained to maximize the Lagrange loss function.

[0008] According to one embodiment, a computer program product for completing response generation is provided, the computer program product including a computer-readable storage medium having program code contained therein, wherein the computer-readable storage medium itself is not a transient signal, the program code being executable by a processor to cause the processor to: train an original network to minimize a Lagrange loss function representing a multi-objective, and train a dual network to maximize the Lagrange loss function; receive a prefix and text input of a session; and generate a completed response based on the prefix and text input of the session via the trained original network. Attached Figure Description

[0009] Preferred embodiments of the invention will now be described by way of example only and with reference to the following figures:

[0010] Figure 1 This is a block diagram of an example min-max neural network that can be trained to perform multi-objective tasks;

[0011] Figure 2 This is a flowchart of an example method for performing multi-objective tasks using a trained raw network;

[0012] Figure 3 This is a flowchart illustrating an example method for automatically generating responses using a trained raw network.

[0013] Figure 4 This is a flowchart of an example method for training a raw network to perform a multi-objective task;

[0014] Figure 5 This is a flowchart of an example method for training the original network to perform automatic response generation;

[0015] Figure 6This is an example of a primitive neural network that includes three Long Short-Term Memory (LSTM) units;

[0016] Figure 7A It is a diagram showing an example chat display that includes a set of completed responses generated;

[0017] Figure 7B It is a diagram showing an example chat display that includes the generated, updated set of completed responses;

[0018] Figure 8 This is a block diagram of an example computing device that can perform multi-objective tasks using a raw network trained with a min-max neural network architecture;

[0019] Figure 9 It is a block diagram of an example computing device that can perform automatic response generation using a raw network trained with a min-max neural network architecture;

[0020] Figure 10 This is a diagram of an example cloud computing environment based on the embodiments described herein;

[0021] Figure 11 This is a diagram of an example abstract model layer based on the embodiments described herein;

[0022] Figure 12 It is a block diagram of an example tangible, non-transient computer-readable medium that can perform multi-objective tasks using a trained min-max neural network; and

[0023] Figure 13 It is a block diagram of an example tangible, non-transitory, computer-readable medium that can train a raw network to perform automatic response generation. Detailed Implementation

[0024] Neural networks are trained on training datasets using objective or loss functions to perform specific tasks related to the specific objective represented by that objective function during training. In some real-world applications, it is desirable to perform several objectives simultaneously. One way to incorporate several objective functions into a learning architecture is by assigning relative weights to the objectives, which transforms a multi-objective problem into a single-objective problem. However, optimizing such weights can be difficult because it may be unclear how to compare different objectives with different units. Furthermore, single-objective approaches may be unsuitable when the learner needs to meet thresholds for some objectives. Additionally, adjusting the weights used to incorporate several objectives into a single objective and adjusting the weights themselves can be computationally difficult.

[0025] According to the techniques described herein, a system may include a processor for receiving data involving a multi-objective task. The processor may perform the multi-objective task on the received data via a trained primary network. The primary network and the dual network are trained alternately using a Lagrange loss function representing the multi-objective task. In one example, the processor may receive a prefix and text input of a session. The processor may then generate a complete response based on the prefix and the input text via the trained primary network. Thus, the techniques described herein enable training neural networks with multi-objective tasks without setting predefined relative weights. Furthermore, the techniques provide improved automatic and semi-automatic responses to queries in a session using a neural network trained on multi-objective tasks. Moreover, the techniques can be used in a variety of applications, including automated text summarization, autonomous machine learning for interacting with its environment, and more.

[0026] In some scenarios, the techniques described in this article can be implemented in a cloud computing environment. See at least the following: Figure 8-13 In more detail, computing devices configured to perform multi-objective tasks can be implemented in a cloud computing environment. It is understood prior to this that while this disclosure may include descriptions relating to cloud computing, implementations of the teachings cited herein are not limited to cloud computing environments. Rather, embodiments of the invention can be implemented in conjunction with any other type of computing environment now known or developed hereafter.

[0027] Cloud computing is a service delivery model for convenient, on-demand network access to a shared pool of configurable computing resources. Configurable computing resources are resources that can be quickly deployed and released with minimal management costs or minimal interaction with the service provider; these can include networks, network bandwidth, servers, processing power, memory, storage, applications, virtual machines, and services. This cloud model can include at least five features, at least three service models, and at least four deployment models.

[0028] Features include:

[0029] On-demand self-service: Cloud consumers can unilaterally and automatically deploy computing power such as server time and network storage on demand without human interaction with service providers.

[0030] Wide network access: Computing power can be accessed over the network through standard mechanisms that facilitate the use of the cloud through various thin client or thick client platforms (such as mobile phones, laptops, and personal digital assistants, PDAs).

[0031] Resource pool: The provider's computing resources are grouped into a resource pool and served to multiple consumers in a multi-tenant model, where different physical and virtual resources are dynamically allocated and reallocated on demand. Generally, consumers cannot control or even know the exact location of the provided resources, but can specify the location at a higher level of abstraction (e.g., country, state, or data center), thus exhibiting location independence.

[0032] Rapid and elastic: The ability to quickly and elastically (sometimes automatically) deploy computing power for rapid scaling and to quickly scale down. From the consumer's perspective, the available computing power for deployment often appears unlimited, and any amount of computing power can be obtained at any time.

[0033] Measurable services: Cloud systems automatically control and optimize resource utilization by leveraging metering capabilities at a certain level of abstraction appropriate to service types (such as storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported, providing transparency for both service providers and consumers.

[0034] The service model is as follows:

[0035] Software as a Service (SaaS): This provides consumers with the ability to use applications running on a provider's cloud infrastructure. These applications can be accessed from various client devices via thin client interfaces such as web browsers (e.g., web-based email). Aside from limited user-specific application configuration settings, consumers neither manage nor control the underlying cloud infrastructure, including the network, servers, operating system, storage, and even individual application capabilities.

[0036] Platform as a Service (PaaS): This provides consumers with the capability to deploy consumer-created or acquired applications on cloud infrastructure, using programming languages ​​and tools supported by the provider. Consumers neither manage nor control the underlying cloud infrastructure, including networks, servers, operating systems, or storage, but they have control over the applications they deploy and may also have control over the configuration of the application hosting environment.

[0037] Infrastructure as a Service (IaaS): This provides consumers with the capability to deploy and run any software, including operating systems and applications, on the underlying cloud infrastructure, including processing, storage, networking, and other basic computing resources. Consumers neither manage nor control the underlying cloud infrastructure, but they have control over the operating system, storage, and the applications deployed thereon, and may have limited control over chosen network components (such as host firewalls).

[0038] The deployment model is as follows:

[0039] Private cloud: The cloud infrastructure runs exclusively for a single organization. The cloud infrastructure can be managed by that organization or a third party and can exist inside or outside the organization.

[0040] Community cloud: A cloud infrastructure shared by several organizations that supports a specific community with common interests (such as mission, security requirements, policy, and compliance considerations). A community cloud can be managed by multiple organizations within the community or by third parties and can exist inside or outside the community.

[0041] Public cloud: Cloud infrastructure provided to the public or large industrial groups and owned by organizations that sell cloud services.

[0042] Hybrid cloud: A cloud infrastructure consisting of two or more cloud deployment models (private cloud, community cloud, or public cloud) that remain distinct entities but are bound together by standardized or proprietary technologies that enable data and applications to be ported together (such as cloud burst traffic balancing for load balancing between clouds).

[0043] The computing environment is service-oriented, characterized by statelessness, loose coupling, modularity, and semantic interoperability. The core of cloud computing is an infrastructure comprising a network of interconnected nodes.

[0044] See now Figure 1 The block diagram illustrates an example min-max neural network that can be trained to perform a multi-objective task. Example min-max neural network 100. Figure 1 The system includes a primary network 102 and a dual network 104. The primary network 102 and the dual network 104 are communicatively coupled to a Lagrange function 106. The dual network 104 is shown as outputting a pair of Lagrange multipliers 108. The primary network 102 includes an LSTM unit 110. The dual network 104 includes an LSTM unit 112. The primary network is shown as receiving values ​​114A, 114B, and 114C and outputting values ​​116A-116D. The dual network is shown as receiving values ​​114A, 114B, and 114C.

[0045] exist Figure 1 In the example, a Lagrange function 106 is used to train the original network 102 and the dual network 104 alternately. For example, the original network 102 can be trained to minimize the Lagrange function 106 while the dual network 104 is kept constant. Similarly, the dual network 104 can be trained to maximize the Lagrange function while the original network 102 is kept constant. Thus, alternating iterations of gradient descent with respect to the policy and gradient ascent with respect to the Lagrange multiplier can be performed.

[0046] exist Figure 1In the example, the original network 102 is generated. Thus, given an input utterance comprising words represented by values ​​114A, 114B, and 114C, the original network 102 generates an output response consisting of words represented by values ​​116A-116D. Given the same input utterance represented by values ​​114A, 114B, and 114C, the dual network 104 outputs a Lagrange multiplier 108 as part of a Lagrange operator 106.

[0047] Specifically, the decision problem can be a Markov Decision Process (MDP) with a finite space of states and actions. Typically, a finite MDP can be represented as a tuple (X, A, R, D, ...). ), where X = {1, ..., n, x} Ter} and A = {1, ..., m} are the state space and action space, respectively, and x Ter It is a recursive terminal state. For state x and action a, R(x,a) can be a bounded reward function, and D1(x,a),…,D n (x,a) is the constrained cost function. The transition probability distribution can be used, and P0(·) can be the initial state distribution. The fixed policy μ(·|x) of MDP is the probability distribution over actions conditioned on the current state. In the policy gradient method, this policy can be parameterized by a k-dimensional vector θ. Using this notation, we can write the policy space as μ(·|x;θ), x∈X, In this setup, the policy μ is uniquely defined by its parameter vector θ. The policy correlation function can be written as a function of μ or θ, and we use μ(·|x;θ) to represent the policy and θ to represent the correlation with the policy (parameters). For a multi-objective MDP, the optimization can be expressed using the following formula:

[0048] maximize θ v θ (x0) Formula 1

[0049] subject to

[0050] Where γ1…γ n This is a user-defined threshold. To address the above problem, Equation 1 can be transformed using a Lagrange relaxation process. The result is an unconstrained problem in the form of a Lagrange function, which is also referred to as the Lagrange function in this paper:

[0051]

[0052] Where λ iIt is the Lagrange multiplier. To achieve multi-objective goals, the original network can be trained to converge to a minima policy. In particular, alternating policy gradient updates, such as gradient descent with respect to the policy and gradient ascent with respect to the λ multiplier, can be used to converge to the optimal policy. Furthermore, the Lagrange variables may be constrained by the state space. In other words, the bivariate model can be viewed as two different data-related models parameterized using parameter ζ, where... As shown in Formula 4 below:

[0053]

[0054]

[0055] The equations in Formulas 3 and 4 begin with an expanded search space and include the case where the λ variable is constant.

[0056] Using the above expression in Formula 4, two different networks (original network 102 and dual network 104) are used to model the two policies simultaneously, such as... Figure 1 As can be seen in the text. Specifically, Figure 1 The original network 102 and the dual network 104 can be modeled as two LSTM models with different parameters for use with chatbots or any other generative task. The objective function of the network is a Lagrangian network, where the original network 102 is configured to find a policy that minimizes the Lagrangian function, while the dual network is configured to find a policy that maximizes the Lagrangian function.

[0057] Therefore, any number of multi-objectives can be represented by Lagrange multipliers and incorporated into the training of the original network. In the specific generation task of generating responses, some reward functions that can be incorporated may include redundancy invisibility objectives, semantic dissimilarity objectives, and semantic consistency objectives. Redundancy invisibility objectives can improve the ease of responses in a dialogue. For example, one of the expected properties of a well-generated dialogue could be that in each round, a semi-automatic or automated agent will generate responses that the other party will find easy to respond to. Training a standard Seq2Seq model can lead to giving redundant answers with a high probability, such as "I don't know what you're talking about," "I don't know," etc. While these answers may be appropriate in terms of the complexity of the language model and suitable for a wide range of questions, it is difficult to continue the dialogue after receiving such responses. Therefore, a lookahead function can be used to measure the ease of turning the conversation around produced by the agent's responses. In some examples, this can be done by leveraging the negative log-likelihood of responses to utterances with redundant responses. To construct such a function, several redundant responses can be obtained a priori, and thus a set S of such responses can be manually constructed that occurs very frequently in a SEQ2SEQ model of a conversation. While finding all such responses manually would be extremely difficult (if not impossible), it can be assumed that similar responses will be tightly embedded in S. Therefore, a system unlikely to generate utterances in the list is also unlikely to generate other dull responses. The redundancy nonlikelihood objective can be calculated using the following formula:

[0058]

[0059] in) S (representing the cardinality of S and) s Indicates the number of tokens in response s and It represents the probability of the language model.

[0060] In some examples, semantic dissimilarity objectives can be used to improve information flow. For instance, another desired property of a conversational agent could be that new information is added to the dialogue in each round. To generate long dialogues, each agent must provide new information in each round. In other words, the dialogue should flow and should avoid repetitive sequences. Therefore, in some instances, semantic similarity between consecutive rounds from the same agent can be penalized. Formally, given e pi ,e pi+1 As an embedded representation of two consecutive rounds pi and pi+1 obtained from the encoder, the reward can be calculated using the negative logarithm of their cosine similarity, as shown in the formula:

[0061]

[0062] Where (·,·) is the Euclidean inner product and ||·| is the Euclidean norm.

[0063] In some cases, semantic coherence objectives can be included in training to improve semantic coherence. Besides previous rewards, another objective might be to ensure that the generated responses are indeed coherent and relevant to the topic of the conversation, or more precisely, to the topics of previous rounds of the conversation. This type of need can be measured using mutual information between action 'a' and previous historical rounds to ensure that the generated responses are consistent, and the following equation is appropriate:

[0064]

[0065] Among them, due to Indicates a given previous dialogue utterance [pi,q] i The probability of generating a response and This represents the backward probability of generating the previous dialogue utterance qi based on the response a. To train this model, the same seq2seq (LSTM model with attention) can be trained with the source and target swapped. The reward can be divided by the length of the utterance to measure the loss.

[0066] To combine all these rewards, one of them can be chosen as the primary loss for the other objectives, which are constrained by a threshold. Therefore, the problem can be transformed into a multi-objective problem using the formula:

[0067]

[0068] subject to

[0069] It should be understood that Figure 1 The block diagram is not intended to indicate that the minimum maximum neural network 100 will include Figure 1 All components shown. Conversely, the minimum maximum neural network 100 may include fewer or... Figure 1 Additional components not shown (e.g., additional inputs, outputs, models, neural networks, units, Lagrange multipliers, etc.).

[0070] Figure 2 This is a flowchart illustrating an example method for performing multi-objective tasks using a trained raw network. Method 200 can be used with any suitable computing device (such as...). Figure 8 This can be achieved using a computing device 800. For example, method 200 can use... Figure 8 The processor 802 of the computing device 800 or using Figure 12 The processor 1202 and the computer-readable medium 1200 are used to implement this.

[0071] In box 202, the original network and dual network of a min-max neural network are trained for a multi-objective task using a Lagrange loss function representing multi-objectives. The original network is trained to minimize the Lagrange loss function, and the dual network is trained to maximize the Lagrange loss function. In some examples, the multi-objective task is a Markov decision process involving a finite state space and a finite action space. In different examples, the original network is pre-trained using a general policy learned from another setting during training or a randomly initialized original network. In some examples, the dual network is randomly initialized during training. In some examples, the gradients of the original and dual networks are estimated based on the likelihood ratio. In various examples, the policy gradients of the original and dual networks are alternately updated based on different stride sizes of the original and dual networks. In some examples, the original and dual networks are alternatively trained using a pre-existing dataset, a simulator, feedback from the environment, or any combination thereof. For example, a dataset can be used... Figure 4 Method 400 is used to train a min-max neural network.

[0072] In box 204, data for a multi-objective task is received. For example, in the case of automated response generation, the data may include words from the input text and conversation prefixes.

[0073] In box 206, a multi-objective task is performed on the received data via a trained original network. For example, the multi-objective task could be automatic response generation, selection, classification, or any other multi-objective task that can be performed using a neural network.

[0074] As indicated by arrow 208, in some examples, additional data can be received for multi-objective tasks and additional multi-objective tasks performed based on additional data. For example, additional data could be additional input text and additional prefixes for the session.

[0075] Figure 2 The process flowchart is not intended to indicate that the operations of method 200 should be performed in any particular order, or that all operations of method 200 should be included in every case. Furthermore, method 200 may include any appropriate number of additional operations.

[0076] Figure 3 This is a flowchart illustrating an example method for automatically generating responses using a trained raw network. Method 300 can be used with any suitable computing device (such as...). Figure 9 This can be achieved using a computing device 900. For example, method 300 can use... Figure 9 The computing device 900 uses a processor 802 or a processor 802. Figure 13 It is implemented using a processor 1302 and a computer-readable medium 1300.

[0077] In box 302, a conversation prefix and text input are received. For example, the conversation prefix may include one or more rounds of dialogue between a first user and a second user. The text input includes one or more words entered by the first user in response to a query from the second user.

[0078] In box 304, a completed response is generated from the session prefix and the input text via a trained primal network. The trained primal network is trained to minimize the Lagrange loss function representing the multi-objective. A dual network is trained to maximize the Lagrange loss function. For example, the primal and dual networks can be trained alternately to minimize and maximize the Lagrange loss function. In different examples, the primal network is trained using a first constraint of multi-turn sessions and gradually increasing that constraint to a second constraint of multi-turn sessions. In some examples, the primal network is trained using sequences that are less likely to generate redundant responses among all sequences in the training dataset. In some examples, multiple completed responses can be generated. In some examples, completed responses can be built iteratively word-by-word from the text input. In some examples, beam search is used to generate multiple completed responses.

[0079] In box 306, a completed response, including a completion response, is presented to the first user for selection. For example, the completed response may be displayed as follows: Figure 7A and 7B The list shown.

[0080] In box 308, the selected response is received from the completed responses. For example, the user can select a response by clicking on it, scrolling down and selecting it from the list of responses, or by adding the completed responses word by word.

[0081] In box 310, the selected response is sent to the second user. For example, the selected response can be sent to the second user as if the first user had typed and sent the response. Therefore, the selected response can be sent via a communication tool or application. In some examples, additional queries can be received from the second user, and the method can restart at box 302.

[0082] Figure 3 The process flowchart is not intended to indicate that the operations of method 300 should be performed in any particular order, or that all operations of method 300 should be included in every case. Additionally, method 300 may include any appropriate number of additional operations. For example, method 300 may be repeated for additional received session prefixes and text input. In some examples, method 300 may include sending a completed response as a response to a query in response to detecting that the confidence score of a completed response exceeds a threshold score.

[0083] Figure 4This is a flowchart illustrating an example method for training a raw network to perform a multi-objective task. Method 400 can be used with any suitable computing device (such as...). Figure 8 This can be achieved using a computing device 800. For example, method 400 can use... Figure 8 The processor 802 of the computing device 800 or using Figure 12 The processor 1202 and the computer-readable medium 1200 are used to implement this.

[0084] In box 402, the training dataset and multi-objective task are received. The training dataset may include data depending on the specific multi-objective task to be performed. For example, a dataset for a generative text task may include data such as... Figure 5 The sessions under discussion. These objectives may include relevance, risk reduction, reduced redundancy, reduced semantic similarity and semantic consistency, and other possible objectives. In some examples, objectives may be received in the form of a loss function.

[0085] In box 404, initialize the original network and dual network of the minimal neural network to be trained. For example, the original network can be pre-trained using a general response policy learned from a fully supervised setup, or it can be randomly initialized. The dual network can be randomly initialized.

[0086] In box 406, the original network and the dual networks are trained alternately using a Lagrange loss function representing multi-objective tasks. In some examples, the multi-objective task can be a Markov decision process involving a finite state space and a finite action space. The original network can be trained to minimize the Lagrange loss function, and the dual networks can be trained alternately to maximize the Lagrange loss function.

[0087] In box 408, the policy gradients of the original and dual networks are updated based on different stride sizes of the original and dual networks. In some examples, the gradients of the original and dual networks are estimated based on a likelihood ratio estimator.

[0088] In decision diamond 410, a determination is made regarding whether the training has been exhausted. For example, preset conditions for validation splits can be set before training, or some measures can be performed for manual checks.

[0089] Training ends at box 412. The original network can then be used to perform multi-objective tasks (such as...) on the received data. Figure 2 (As shown).

[0090] Figure 4The process flowchart is not intended to indicate that the operations of method 400 should be performed in any particular order, or that all operations of method 400 should be included in every case. Furthermore, method 400 may include any appropriate number of additional operations. For example, additional decision diamonds or conditions may be included in method 400, or even manual checks on different metrics measured during training.

[0091] Figure 5 This is a flowchart illustrating the process of training the original network to perform automatic response generation. Method 500 can be used with any suitable computing device (such as...). Figure 9 This can be achieved using a computing device 900. For example, method 500 can use... Figure 9 The processor 802 of the computing device 900 or using Figure 13 The processor 1302 and the computer-readable medium 1300 are used to implement this.

[0092] In box 502, we receive the training dataset and the multi-objective. For example, the training dataset could be the OpenSubtitles dataset of movie conversations. The OpenSubtitles dataset contains sentences uttered by characters in a movie. For example, in this dataset, each utterance can be considered a response to a previous utterance and as context for the next response. Therefore, the training and validation splits could include 62 million sentences (923 million tokens) as training examples, and the test set could have 26 million sentences (395 million tokens). Splitting is performed in such a way that each sentence in a pair either appears together in the training or test set, but not simultaneously in either. Given a wide range of movies, this is an open-domain conversation dataset. Each turn in the dataset can be considered the objective, and the concatenation of two previous sentences can be considered the source input.

[0093] In box 504, the original network is pre-trained based on a pre-selected model, and the dual networks are randomly initialized. For example, a sequence-to-sequence (Seq2Seq) language model can be used to initialize the original network. In some examples, a general response generation policy learned from a fully supervised setting is used to initialize the reinforcement learning (RL) system. The model chosen for pre-training can be a simple model and can be replaced by any other model. The generated sentences can be viewed as actions taken according to a policy defined by the Seq2Seq language model. The policy can be given by a state... The probability distribution is defined as the probability distribution over actions. In some instances, a Seq2Seq LSTM model is used to model this probability distribution. In other instances, the LSTM model can be replaced by any other suitable language generation model.

[0094] Therefore, the action can be the generated utterance. Since the training of Method 500 includes dialogue, the state space can include information about past turns of the conversation. For example, the state can include the previous two dialogue turns [pi, qi]. Thus, the vector representation of the state is encoded by the concatenation of the previously generated response pi and the response from the second agent qi.

[0095] In box 506, the original network and dual network of the minimum neural network are trained alternately on the training dataset using a Lagrange loss function to represent multi-objectives. In some examples, the minimum neural network is trained with an initial constraint of two rounds of a session, and this constraint is gradually increased to five rounds of a session. In some examples, a predetermined number of sequences are used to train the minimum-maximum neural network, said predetermined number of sequences including sequences with a lower probability of generating redundant responses than other sequences in the training dataset. In some examples, as another helpful step in dialogue simulation, a subset of 1 million messages from the OpenSubtit1e dataset can be retrieved, and 8 million sequences are extracted with the lowest probability of generating redundant responses from the set to ensure that the initial input is readily responsive.

[0096] At box 508, the policy gradients of the original network and the dual network are updated alternately based on different stride sizes. For example, different timescale methods can be used to alternately update the policy gradients. Since min-max network training involves two different (alternating) gradient policy updates, each of the original and dual networks will have different stride sizes. For example, during training, the stride size of the original network can be an order of magnitude different from the stride size of the dual network. Therefore, the original network will receive a higher convergence rate, while the dual network will receive a smaller stride size and will therefore converge more slowly. In some examples, a likelihood ratio estimator is used to estimate the gradient of the policy. The likelihood ratio estimator can estimate the gradient based on statistical theory. For example, the likelihood ratio estimator can use the REINFORCE trick published in 1992.

[0097] At decision diamond 510, a determination is made regarding whether the training has been exhausted. For example, a preset number of training iterations can be set before training begins.

[0098] Training ends at box 512. The original network can then be used to generate automatic responses to received queries, as shown above. Figure 3 As described in [the text].

[0099] Figure 5The process flowchart is not intended to indicate that the operations of Method 500 should be performed in any particular order, or that all operations of Method 500 should be included in every case. Furthermore, Method 500 may include any suitable number of additional operations. For example, Method 500 may include additional decision diamonds or conditions, or even manual examination of different metrics measured during training.

[0100] Figure 6 This is an example primitive neural network comprising three Long Short-Term Memory (LSTM) units. Example neural network 600 can be trained using method 500 and can be used in conjunction with method 300 and... Figure 9 The completed response is generated in the computing device 900. For example, the neural network 600 may be a sequence-to-sequence deep learning architecture with or without an attention mechanism. Figure 6 It includes three LSTM units 602, 604, and 606. The first LSTM unit 602 includes words 608A, 608B, 608C, and 608D corresponding to the client's query. The second LSTM unit 604 includes text input 610A and 610B corresponding to text input from a human agent. The third LSTM unit 606 includes words 612A, 612B, 612C, and 612D corresponding to the completion portion of the completed response.

[0101] like Figure 6 As shown, the first LSTM 602 receives the complete prefix of the client's query or session word by word and encodes the words into a fixed-length hidden state vector hA. The second LSTM 604 receives text input from a human agent and transforms vector hA into a hidden state vector hB by encoding the text input word by word. The third LSTM 606 transforms (decodes) vector hB into a sequence of output words 612A-612D, which represents the completion of the response. A completed response can be generated by concatenating the text inputs 610A, 610B with the completions 612A-612D. In some examples, the neural network 600 may extract several completed responses instead of generating a single completed answer. For example, beam search can be used to extract several completed responses.

[0102] During the training phase, all historical sessions are transformed into training triples, which include the client's query, the start of the corresponding human agent's response, and the end of the agent's response. All combinations of start and end points are segmented from the human agent's response. Responses can be segmented at each word to generate different training samples. The neural network 600 can be trained using a multi-objective approach via a Lagrangian loss function that incorporates all objectives. For example, objectives used during training may include word-by-word probabilities, the complexity and relevance of the agent's response ending, and other objectives described herein.

[0103] Figure 7AThis is a diagram showing an example chat display including a set of generated completed responses. It can be used... Figure 3 and 5 Methods 300 and 500 use Figure 9 The computing device 900 was used to generate the example chat display 700A.

[0104] exist Figure 7A The image shows a chat display 700A of a human customer service agent on a communication channel. Figure 7A The first message 702 is automatically generated by the company. The second message 704 is received from the customer. At the bottom of the screen, above the horizontal line, the human agent is typing his response. In plain text, we see the agent's text input 706A, "I'll Be Happy". Below the text input 706A, three suggestions 708A, 708B, and 708C are displayed regarding how the agent should complete the response. For example, suggestions 708A, 708B, and 708C can be generated using the techniques described herein. The selected suggestion 708A also appears above the line, consecutive to the text input in the selected text. As can be seen in the example chat display 700A, the three automated suggestions 708A, 708B, and 708C can be based on both the session context (including the first message 702 and the second message 704) and the agent's text input 706A.

[0105] Figure 7B This is a diagram showing an example chat display including the generated, updated set of completed responses. It can be used... Figure 3 and 5 Methods 300 and 500 use Figure 9 The computing device 900 was used to generate the example chat display 700A.

[0106] exist Figure 7B As the agent continues typing, new groups of suggested responses 708D, 708E, and 708F are displayed below the updated text input 706B in the updated chat display 700B. This allows new suggested responses to be generated in real-time as agent types. For example, each time the agent enters an appendix into text input 706B, a new set of suggested responses can be generated. If one of the suggested responses (such as the selected response 708E) is correct, the agent can select that response, and response 708E will be sent to the user.

[0107] Thus, given a prefix to a session containing zero or more text messages between a human agent and a customer, and given initial text input by the agent corresponding to a partial response, the application can suggest one or more completed responses. Completion can correspond to a specific text input and the ongoing session. In some examples, other types of data and metadata besides text can be included as part of the session prefix, text input, or even the completed response. For example, metadata can include images, videos, web links, etc. In some examples, completed responses can be learned automatically from historical sessions. For example, historical sessions can be used as training data to train a neural network for generating completed responses. Similarly, historical sessions can be used to train neural networks to create end-to-end solutions, such as chatbots. The application can continue to improve with each new session, as the neural network can be trained on additional sessions. In some examples, in those parts of the session where responses are identified with high confidence even before the agent types a single word, the completed response can be fully automated without human intervention. This saves time by using suggested responses or automatically sending high-confidence responses instead of typing them out completely. Furthermore, the quality of responses can be improved by eliminating the chance of typos and other errors.

[0108] Figure 8 This is a block diagram of an example computing device that can perform multi-objective tasks using a raw network trained with a min-max neural network architecture. The computing device 800 can be, for example, a server, desktop computer, laptop computer, tablet computer, or smartphone. In some examples, the computing device 800 can be a cloud computing node. The computing device 800 can be described in the general context of computer system executable instructions (such as program modules) executed by a computer system. Generally, program modules can include routines, programs, objects, components, logic, data structures, etc., that perform a specific task or implement a specific abstract data type. The computing device 800 can be practiced in a distributed cloud computing environment, where tasks are performed by remote processing devices linked via a communication network. In a distributed cloud computing environment, program modules can reside in both local and remote computer system storage media, including memory storage devices.

[0109] The computing device 800 may include a processor 802 for executing stored instructions and a memory device 804 for providing temporary memory space for the operation of said instructions during operation. The processor may be a single-core processor, a multi-core processor, a computing cluster, or any other configuration. The memory 804 may include random access memory (RAM), read-only memory, flash memory, or any other suitable memory system.

[0110] Processor 802 can be connected to system interconnect 806 (e.g., The input / output (I / O) device interface 808 is connected to a computing device 800 adapted to connect the computing device 800 to one or more I / O devices 810. The I / O device 810 may include, for example, a keyboard and a pointing device, wherein the pointing device may include a touchpad or a touchscreen, etc. The I / O device 810 may be a built-in component of the computing device 800 or an external device connected to the computing device 800.

[0111] The processor 802 can also be linked via a system interconnect 806 to a display interface 812 adapted to connect the computing device 800 to a display device 814. The display device 814 may include a display screen as a built-in component of the computing device 800. The display device 814 may also include a computer monitor, television, or projector, etc., externally connected to the computing device 800. Furthermore, a network interface controller (NIC) 816 may be adapted to connect the computing device 800 to a network 818 via the system interconnect 806. In some embodiments, the NIC 816 may use any suitable interface or protocol (such as an Internet Small Computer System Interface) to transmit data. The network 818 may be a cellular network, a radio network, a wide area network (WAN), a local area network (LAN), or the Internet, etc. An external computing device 820 may be connected to the computing device 800 via the network 818. In some examples, the external computing device 820 may be an external web server 820. In some examples, the external computing device 820 may be a cloud computing node.

[0112] Processor 802 may also be linked to storage device 822 via system interconnect 806, which may include hard disk drives, optical disk drives, USB flash drives, drive arrays, or any combination thereof. In some instances, the storage device may include receiver 824, original network 826, and training network 828. Receiver 824 may receive data for a multi-objective task. For example, it may also receive multiple objectives in the form of a loss function. The objective task may be a selection task, a classification task, or a generation task, as well as other possible tasks. For example, multi-objective tasks may include selection, classification, regression, recommendation, generation, or any other type of prediction task. Original network 826 may be trained to perform the multi-objective task on the received data via training network 828. For example, training network 828 may be a minimization-maximization neural network. For example, training network 828 may include an original network and a dual network. Training network 828 may use a Lagrange loss function representing the number of objectives to train the original network and the dual network for the multi-objective task. Training network 828 trains the original network to minimize the Lagrange loss function and trains the dual network to maximize the Lagrange loss function. In some examples, the multi-objective task is a Markov decision process involving a finite state space and a finite action space. In some examples, training network 828 pre-trains the original network using a general policy learned from another setting or random initialization. Training network 828 may randomly initialize the dual network during training. In some examples, the original network has a step size smaller than that of the dual network during training. For example, the step size of the original network may be an order of magnitude or more smaller than that of the dual network. In some instances, training network 828 may estimate gradients based on likelihood ratio estimation. For example, training network 828 may use... Figure 4 Method 400 is used to train the original network.

[0113] It should be understood that Figure 8 The block diagram is not intended to indicate that the computing device 800 will include Figure 8 All the components shown. Instead, the computing device 800 may include... Figure 8 Fewer or additional components not specified herein (e.g., additional memory components, embedded controllers, modules, additional network interfaces, etc.). Furthermore, any functionality of receiver 824, raw network 826, and training network 828 may be implemented partially or wholly in hardware and / or processor 802. For example, this functionality may be implemented using application-specific integrated circuits, logic implemented in an embedded controller, or logic implemented in processor 802, etc. In some embodiments, the functionality of receiver 824, raw network 826, and training network 828 may be implemented using logic, wherein logic as referred to herein may include any suitable hardware (e.g., processor, etc.), software (e.g., application programs, etc.), firmware, or any suitable combination of hardware, software, and firmware.

[0114] Figure 9 This is a block diagram of an example computing device that can perform automatic response generation using a raw network trained with a min-max neural network architecture. Figure 9 The computing device 900 includes Figure 8 The components are numbered similarly. Additionally, the computing device 900 includes a response display 902 and a response transmitter 904.

[0115] In example computing device 900, receiver 824 can receive a conversation prefix and text input. For example, the conversation prefix may include a dialogue between a first user and a second user, and the text input may include a portion of a completed response. Based on the conversation prefix and text input, a completed response is generated via training network 828 and a primary network 826 trained using a pre-existing dataset. For example, training network 828 may be a minimization-maximization neural network. For example, training network 828 may include a primary network and a dual network trained using a Lagrange loss function representing multiple objectives. Multiple objectives may include complex objectives, relevance objectives, redundancy nonlikelihood objectives, semantic dissimilarity objectives, semantic consistency objectives, and other objectives, or any combination thereof. In some examples, the primary network and the dual network are Long Short-Term Memory (LSTM) models with different parameters. In some examples, the trained primary network 826 may be trained to minimize the Lagrange loss function by generating a completed response. The dual network is trained to maximize the Lagrange loss function. Response display 902 is capable of displaying the completed response generated by the primary network 826. For example, the responsive display 902 can present the user with multiple completed responses, including a completed response, for selection. For instance, a completed response may be displayed as such as... Figure 7A and 7B The list of applications in the application. Response sender 904 can receive the selected response from the completed response and send the selected response to the second user.

[0116] See now Figure 10The diagram illustrates an illustrative cloud computing environment 1000. As shown, the cloud computing environment 1000 includes one or more cloud computing nodes 1002, and local computing devices used by cloud consumers (such as personal digital assistants (PDAs) or cellular phones 1004A, desktop computers 1004B, laptop computers 1004C, and / or automotive computer systems 1004N) can communicate with the cloud computing nodes 1002. The nodes 1002 can communicate with each other. They can be physically or virtually grouped (not shown) in one or more networks, such as private clouds, community clouds, public clouds, or hybrid clouds, or combinations thereof, as described above. This allows the cloud computing environment 1000 to provide infrastructure, platform, and / or software as services, without requiring cloud consumers to maintain resources on their local computing devices. It should be understood that... Figure 10 The types of computing devices 1004A-N shown are intended to be illustrative only, and computing node 1002 and cloud computing environment 1000 can communicate with any type of computerized device via any type of network and / or network-addressable connection (e.g., using a web browser).

[0117] See now Figure 11 This demonstrates the 1000 (cloud computing environment) Figure 10 This provides a set of functional abstractions. It should be understood beforehand that... Figure 11 The components, layers, and functions shown are intended to be illustrative only, and embodiments of the invention are not limited thereto. As depicted, the following layers and corresponding functions are provided.

[0118] The hardware and software layer 1100 includes hardware and software components. Examples of hardware components include mainframes, which in one example are... System; a server based on a RISC (Reduced Instruction Set Computer) architecture, in one example being an IBM... System; IBM System; IBM Systems; storage devices; networking and interconnection components. Examples of software components include network application server software; in one example, IBM... Application server software; and database software, in one example, IBM Database software. (IBM, zSeries, pSeries, xSeries, BladeCenter, WebSphere, and DB2 are trademarks of IBM registered in many jurisdictions worldwide.)

[0119] The virtualization layer 1102 provides an abstraction layer from which the following examples of virtual entities can be provided: virtual servers; virtual storage; virtual networks, including virtual private networks; virtual applications and operating systems; and virtual clients. In one example, the management layer 1104 can provide the functionality described below. Resource provisioning provides the dynamic acquisition of computing resources and other resources used to perform tasks within the cloud computing environment. Metering and pricing provides cost tracking as resources are utilized within the cloud computing environment and bills or invoices for the consumption of these resources. In one example, these resources may include application software licenses. Security provides authentication for cloud consumers and tasks, as well as protection for data and other resources. The user portal provides consumers and system administrators with access to the cloud computing environment. Service level management provides the allocation and management of cloud computing resources to ensure that the required service level is met. Service level agreements (SLAs) plan and fulfill the pre-arrangement and procurement of cloud computing resources, anticipating future requirements for those resources according to the SLA.

[0120] Workload 1106 provides examples of functionalities that can be leveraged in a cloud computing environment. Examples of workloads and functionalities that can be provided from this layer include: mapping and navigation; software development and lifecycle management; virtual classroom education delivery; data analytics and processing; transaction processing; and multi-objective task processing.

[0121] This technology can be a system, method, or computer program product. The computer program product may comprise a computer-readable storage medium (or medium) having computer-readable program instructions thereon for causing a processor to perform aspects of the invention.

[0122] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0123] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0124] The computer program instructions used to perform the operations of this technology may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and traditional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing state information from the computer-readable program instructions. This electronic circuitry can execute the computer-readable program instructions to implement various aspects of this technology.

[0125] Various aspects of the present technology are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present technology. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0126] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0127] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0128] See now Figure 12 A block diagram depicts an example tangible, non-transitory computer-readable medium 1200 that can be used to train an original network to perform a multi-objective task. The tangible, non-transitory computer-readable medium 1200 is accessible by a processor 1202 via a computer interconnect 1204. Furthermore, the tangible, non-transitory computer-readable medium 1200 may contain elements for directing the processor 1202 to perform operations. Figure 2 and 4 The code for operations 200 and 400.

[0129] The various software components discussed herein can be stored on a tangible, non-transitory computer-readable medium 1200, such as... Figure 12As indicated in the document. For example, network training module 1206 includes code for training an original network and a dual network for a multi-objective task using a Lagrange loss function representing the multi-objective task. Network training module 1206 also includes code for training the original network to minimize the Lagrange loss function and training the dual network to maximize the Lagrange loss function. Network training module 1206 may also include code for processing the multi-objective task as a Markov decision process including a finite state space and a finite action space. In various examples, network training module 1206 includes code for alternately training the original network and the dual network using a pre-existing dataset, a simulator, feedback from the environment, or any combination thereof. In some examples, network training module 1206 includes code for pre-training the original network using a general policy learned from another setting or by randomly initializing the original network during training. In various examples, network training module 1206 includes code for randomly initializing the dual network during training. In some examples, network training module 1206 includes code for estimating the gradients of the original network and the dual network based on the likelihood ratio. In some examples, network training module 1206 includes code for updating the policy gradients of the original network and the dual network based on different step sizes of the original network and the dual network. Receiver module 1208 includes code for receiving data for a multi-objective task involving multiple objectives. Original neural network module 1210 includes code for performing the multi-objective task on the received data via a trained original network. It should be understood that, depending on the specific application, the tangible, non-transitory computer-readable medium 1200 may contain... Figure 12 Any number of additional software components not shown in the diagram.

[0130] See now Figure 13 A block diagram depicts an example tangible, non-transitory computer-readable medium 1300 that can be trained to perform automatic response generation. The tangible, non-transitory computer-readable medium 1300 is accessible by a processor 1302 via a computer interconnect 1304. Furthermore, the tangible, non-transitory computer-readable medium 1300 may contain elements for instructing the processor 1302 to perform the above... Figure 3 and 5 The code for operations 300 and 500.

[0131] The various software components discussed herein can be stored on a tangible, non-transitory computer-readable medium 1300, such as... Figure 13As indicated in the document. For example, network training module 1306 includes code for training a primary network to minimize a Lagrange loss function representing a multi-objective network and training a dual network to maximize the Lagrange loss function. Network training module 1306 may also include code for training a minimal neural network using a first constraint of rounds of sessions and gradually increasing that constraint to a second constraint of rounds of sessions. As an example, the first constraint could be two rounds of sessions and the second constraint could be five rounds of sessions. Receiver module 1308 includes code for receiving a prefix and text input of a session. Primary neural network module 1310 includes code for generating completed responses based on the prefix and text input of a session. For example, primary neural network module 1310 may include code for iteratively constructing sentences beginning with the text input word by word. Primary neural network module 1310 also includes code for generating multiple completed responses. For example, primary neural network module 1310 may include code for generating multiple completed responses including completed responses using beam search. Response display module 1312 includes code for presenting a multiple completed response, including completed responses, to a user for selection. The response sending module 1314 includes code for receiving a selected response from completed responses and sending the selected response to a second user. The response sending module 1314 may also include code for automatically sending a completed response as a response to a query in response to detecting that the confidence score of the completed response exceeds a threshold score. It should be understood that, depending on the specific application, the tangible, non-transitory computer-readable medium 1300 may contain... Figure 13 Any number of additional software components not shown in the diagram.

[0132] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to different embodiments of the present technology. Each block in a flowchart or block diagram may represent a module, segment, or part of an instruction, which includes one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than indicated in the figures. For example, depending on the functions involved, two consecutively shown blocks may actually be executed substantially simultaneously, or these blocks may sometimes be executed in reverse order. It will also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs the specified function or action or executes a combination of dedicated hardware and computer instructions. It should be understood that, depending on the specific application, Figure 12 and 13 Any number of additional software components not shown may be contained within tangible non-transitory computer-readable media 1200 and 1300.

[0133] The description of different embodiments of this technology has been presented for illustrative purposes, but is not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein has been chosen to best explain the principles of the embodiments, their practical application, or technical improvements found in the market, or to enable those skilled in the art to understand the embodiments disclosed herein.

Claims

1. A system comprising a processor, the processor being configured to: Receive data containing text input corresponding to a session containing a partial response from an agent, the partial response being to be completed through a multi-objective text generation task; and Multiple distinct completed responses are generated via a trained original network, each comprising a partial response to the input and different generated text for completing that partial response. An agent is used to select one response from the multiple distinct completed responses. The original network and the dual network are trained on the multi-objective text generation task using a Lagrange loss function representing multiple objectives. The original network is trained to minimize the Lagrange loss function, and the dual network is trained to maximize the Lagrange loss function, wherein the step size of the original network is smaller than the step size of the dual network during training.

2. The system as claimed in claim 1, wherein, The multi-objective text generation task includes a Markov decision process, which includes a finite state space and a finite action space.

3. The system as described in claim 1, wherein, The original network is pre-trained using a general strategy learned from another setting or random initialization.

4. The system as claimed in claim 1, wherein, The dual network is randomly initialized during training.

5. The system as claimed in claim 1, wherein, The processor is operable to estimate gradients based on likelihood ratio estimation.

6. The system as claimed in claim 1, wherein, The multi-objective text generation task includes selection, classification, regression, recommendation, generation, or prediction tasks.

7. The system as claimed in claim 1, wherein, The processor is operable to receive data that is a session prefix and text input, and wherein the processor is operable to generate a complete response based on the session prefix and the text input via the trained raw network.

8. The system of claim 7, wherein, The original network and the dual network include Long Short-Term Memory (LSTM) models with different parameters and possible additional network elements.

9. The system of claim 7, wherein, These multiple objectives include complex or related objectives.

10. The system of claim 7, wherein, The multi-objective includes a redundant likelihood-independent objective or a semantically dissimilar objective.

11. The system of claim 10, wherein, The multiple objectives include the semantic consistency objective.

12. A computer-implemented method, comprising: The original network and the dual network for a multi-objective text generation task are trained using a Lagrange loss function representing multiple objectives, wherein training the original network and the dual network includes training the original network to minimize the Lagrange loss function and training the dual network to maximize the Lagrange loss function, wherein during training, the stride size of the original network is smaller than the stride size of the dual network. Receive data containing text input, which corresponds to a session containing a partial response from an agent, the partial response being completed through a multi-objective text generation task; as well as The original network trained by the training generates multiple different completed responses, each of which includes a partial response to the input and different generated text for completing the partial response to the input, wherein the agent is used to select one response from the multiple different completed responses.

13. The computer-implemented method of claim 12, comprising the multi-objective text generation task as a Markov decision process, the Markov decision process comprising a finite state space and a finite action space.

14. The computer-implemented method of claim 12, comprising: The original network is pretrained using a general strategy learned from another setting or by randomly initializing the original network during training.

15. The computer-implemented method of claim 12, comprising randomly initializing the dual network during training.

16. The computer-implemented method of claim 12, wherein training the original network and the dual network comprises estimating the gradients of the original network and the dual network based on the likelihood ratio.

17. The computer-implemented method of claim 12, comprising updating the policy gradients of the original network and the dual network based on different step sizes of the original network and the dual network.

18. The computer-implemented method of claim 12, wherein training the original network and the dual network comprises training the original network and the dual network alternately.

19. The computer-implemented method as described in claim 12, wherein, The received data is a session prefix and text input, and the method includes: The original network trained by the training generates a complete response based on the prefix of the session and the text input.

20. The computer-implemented method of claim 19, comprising: In response to detecting that the confidence score of the completed response exceeds a threshold score, the completed response is sent as a response to the query.

21. The computer-implemented method as described in claim 19, wherein, Generating the completed response involves iteratively constructing the completed response, starting with the text input, word by word.

22. The computer-implemented method of claim 19, wherein generating the completed response includes beam searching to generate a plurality of completed responses.

23. The computer-implemented method of claim 19, comprising: The original network is trained using a first constraint of the session's rounds, and the first constraint is gradually increased to a second constraint of the session's rounds.

24. The computer-implemented method of claim 19, comprising: The original network is trained using sequences that have a lower probability of generating redundant responses among all sequences in the training dataset.

25. A computer program product for training a neural network to perform a multi-objective task, the computer program product comprising a computer-readable storage medium having program code embodied therein, wherein, The computer-readable storage medium itself is not a transient signal, and the program code can be executed by a processor to cause the processor to: The original network and the dual network for multi-objective text generation tasks are trained using a Lagrange loss function representing multiple objectives. The original network is trained to minimize the Lagrange loss function and the dual network is trained to maximize the Lagrange loss function, wherein during training, the stride size of the original network is smaller than the stride size of the dual network; Receive data containing text input, which corresponds to a session containing a partial response from an agent, the partial response being completed through a multi-objective text generation task; as well as The original network trained by the training generates multiple different completed responses, each of which includes a partial response to the input and different generated text for completing the partial response to the input, wherein the agent is used to select one response from the multiple different completed responses.

26. The computer program product of claim 25, further comprising program code executable by the processor to train the original network and the dual networks using a pre-existing dataset, a simulator, feedback from the environment, or any combination thereof.

27. The computer program product of claim 25, further comprising program code executable by the processor for pre-training the original network using a general strategy learned from another setup or by randomly initializing the original network during training.

28. The computer program product of claim 25, further comprising program code executable by the processor to estimate the gradients of the original network and the dual network based on the likelihood ratio.

29. The computer program product of claim 25, further comprising program code executable by the processor to update the policy gradients of the original network and the dual network based on different step sizes of the original network and the dual network.

30. The computer program product of claim 25, further comprising program code executable by the processor to randomly initialize the dual networks during training.

31. The computer program product of claim 25, wherein the received data is a session prefix and text input, and wherein the program code is executable by the processor to cause the processor to: The original network trained by the training generates a complete response based on the prefix of the session and the text input.

32. The computer program product of claim 31, further comprising program code executable by the processor to perform the following operations: In response to detecting that the confidence score of the completed response exceeds a threshold score, the completed response is sent as a response to the query.

33. The computer program product of claim 31, further comprising program code executable by the processor to iteratively construct a sentence beginning with the text input word by word.

34. The computer program product of claim 31, further comprising program code executable by the processor to generate a plurality of completed responses including the completed response using beam search.

35. The computer program product of claim 31, further comprising program code executable by the processor to perform: training the original network using a first constraint of a session wheel, and gradually increasing the first constraint to a second constraint of a session wheel.

36. A computer program including program code means, wherein when the program is run on a computer, the program code means is adapted to perform the method as claimed in any one of claims 12 to 24.

Images