How to automatically generate diverse text

Abstract

Embodiments relate to an artificial intelligence (AI) computer platform that incorporates synthetic data and ground truth data to promote diversity and accuracy in generating synthetic data. Synthetic questions are generated by a question generator in response to semantically related ground truth passages and answer data. Each generated question is presented to an answer generator along with the semantically related ground truth passage. Each synthetic question is evaluated for its diversity from previous synthetic questions and answer data generated for the same ground truth passage. Each synthetic question is also evaluated for the accuracy of the answers generated by the answer generator. A reward function that captures both the accuracy and diversity of each synthetic question is utilized to selectively modify the question generator, with the selective modification directed toward increasing the diversity of the text and maintaining the accuracy of the generated synthetic questions.

Description

[Background technology]

[0001] The present embodiments relate to an artificial intelligence platform and optimization method for automatically generating diverse texts corresponding to received input passages and input answers. More specifically, the present embodiments are directed to evaluating the generated texts with respect to quality, specifically diversity and accuracy, and selectively tuning one or more parameters of a reinforcement learning algorithm that supports the diversity and quality characteristics of the generated texts. Summary of the Invention

[0002] The present embodiments include computer systems, computer program products, and methods for reinforcement learning.

[0003] In one aspect, a computer system comprises a processor and memory for use with an artificial intelligence (Al) platform and corresponding AI tools. The processor is operatively coupled to the memory and in communication with the AI ​​platform. The AI ​​platform tools include a question manager, an answer manager, an evaluation manager, and a director. The question manager is operative to generate a first synthetic question corresponding to the ground truth passage and the ground truth answer utilizing a question generator. The answer manager is operative to present the synthetic question to a trained neural network and obtain a first answer from the ground truth passage. The first answer is semantically related to the first synthetic question and the ground truth passage. The evaluation manager is operative to automatically evaluate the first synthetic question with respect to the first answer, the evaluation incorporating a reward function that captures the diversity and accuracy of the first synthetic question. The director functions to employ reinforcement learning to selectively modify the question generator responsive to the evaluations, where the modified question generator produces one or more synthetic questions with increased text diversity than before the modification.

[0004] In another aspect, a computer program device is provided for supporting reinforcement learning. The computer program product includes a computer-readable storage medium having program code embodied therewith. The program code is executable by a processor to generate synthetic questions and produce more textually diverse synthetic questions and accurate answers. The program code utilizes a question generator to generate a first synthetic question corresponding to a ground truth passage and the ground truth answer. The program code submits the synthetic question to a trained neural network to obtain a first answer from the ground truth passage, with the first answer being semantically related to the first synthetic question and the ground truth passage. Program code is provided for automatically evaluating the first synthetic question with respect to the first answer, the evaluation incorporating a reward function that captures the diversity and accuracy of the first synthetic question. The program code employs reinforcement learning to selectively modify the question generator responsive to the evaluation, where the modified question generator produces one or more synthetic questions with increased textual diversity than before the modification.

[0005] In yet another aspect, a method is provided for supporting reinforcement learning. A first synthetic question corresponding to a ground truth passage and a ground truth answer is generated from a question generator. The synthetic question is presented to a trained neural network, and a first answer from the ground truth passage is obtained from the neural network. The first answer is semantically related to the first synthetic question and the ground truth passage. The first synthetic question is automatically evaluated with respect to the first answer. The evaluation incorporates a reward function that captures the diversity and accuracy of the first synthetic question. Reinforcement learning is utilized to selectively modify the question generator based on the evaluation, and the modified question generator produces one or more synthetic questions with increased text diversity than before the modification.

[0006] These and other features and advantages will become apparent from the following detailed description of illustrative embodiments that are to be taken in conjunction with the accompanying drawings.

[0007] The drawings referenced herein form part of this specification and are meant to illustrate only some embodiments and not all embodiments, unless expressly indicated otherwise. [Brief description of the drawings]

[0008] [Figure 1] This is a system diagram showing a computer system. [Diagram 2] A block diagram showing tools from a computing system and their associated application program interfaces. [Diagram 3] 1 is a flowchart illustrating an application of reinforcement learning to support and enable the generation of diverse and accurate questions. [Figure 4] FIG. 4 is a block diagram illustrating an example of a computer system / server of a cloud-based support system implementing the systems and processes described above with reference to FIGS. 1-3. [Diagram 5] FIG. 1 is a block diagram illustrating a cloud computing environment. [Figure 6] FIG. 2 is a block diagram illustrating a set of functional abstraction model layers provided by a cloud computing environment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] It will be readily understood that the components of the present embodiments, as generally described and illustrated in the figures herein, could be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of apparatus, system, method and computer embodiments is not intended to limit the scope of the embodiments as claimed, but rather is merely representative of selected embodiments.

[0010] Throughout this specification, the use of "selected embodiments," "one embodiment," or "an embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment is included in at least one embodiment. Thus, the appearances of the phrases "selected embodiments," "in one embodiment," or "in an embodiment" in various places throughout this specification are not necessarily referring to the same embodiment. Various embodiments may be combined with each other.

[0011] The illustrated embodiments are best understood by referring to the drawings, in which: The following description is intended by way of example only and merely illustrates certain selected embodiments of devices, systems and processes consistent with the embodiments.

[0012] Artificial intelligence (AI) refers to the field of computer science directed at computers and computer behavior in relation to humans. AI refers to intelligence when a machine is able to make decisions based on information that maximize the chances of success in a given topic. More specifically, AI can learn from a data set to solve problems and provide reasonable recommendations. For example, in the field of artificial intelligence computer systems, natural language systems (e.g., IBM Watson® artificial intelligence computer system or other natural language question answering systems) process natural language based on system acquired knowledge. To process natural language, the system may be trained with data derived from a database or corpus of knowledge, but the resulting results may be incorrect or inaccurate for various reasons.

[0013] Machine learning (ML), a subset of artificial intelligence (AI), utilizes algorithms to learn from data and create predictions based on this data. More specifically, ML is the application of AI through the creation of neural networks that can exhibit learning behavior by performing tasks that are not explicitly programmed. ML requires data to be analyzed, formatted, and tuned to build machine learning models and train machine learning algorithms. Those skilled in the art will appreciate that ML algorithms are computerized procedures that, when trained on data, produce ML models. Choosing an ML algorithm is essential to successfully applying ML. Examples of ML include, but are not limited to, regression algorithms, decision trees, instance-based algorithms, and clustering algorithms. Once the data is prepared and the algorithms are trained, the ML model can make decisions or predictions about the data. The greater the amount of data provided, the more the model learns and improves the accuracy of its predictions.

[0014] ML models are categorized into supervised machine learning, unsupervised machine learning, reinforcement machine learning, and deep learning. Supervised learning algorithms learn a mapping function for a data set with an existing classification, and above the existing classification, unsupervised learning algorithms can classify an unlabeled data set based on some hidden features in the data. Reinforcement learning can learn a policy for making decisions in an uncertain environment through iterative exploration of its environment. Deep learning incorporates neural networks in successive layers to learn from data in an iterative manner. A neural network is a model of how the nervous system works. The basic units are called neurons and are typically organized into layers. Neural networks work by simulating a large number of interconnected processing units that resemble an abstract version of a neuron. A neural network typically has three parts, including an input layer with units that represent the input fields, one or more hidden layers, and an output layer with one or more units that represent the target fields. These units are connected with various connection strengths or weights. Input data is fed into the first layer, and values ​​are propagated from each neuron to each neuron in the next layer. Finally, the results are sent out of the output layer. Deep learning complex neural networks are designed to emulate how the human brain works, so computers can be trained to support poorly defined abstractions and problems. Neural networks and deep learning are often used in image recognition, speech, and computer vision applications.

[0015] A language model is a mechanism that takes a sequence of words, such as a phrase or subphrase, as input and produces an output in the form of a probability distribution for the next word. For each word in the vocabulary, the language model predicts the probability that the next word in the sequence is that particular word. A variable Y is used to represent the vocabulary that the words y0, y1, y2, ..., y n-1 are the first n-1 words in the sequence, and y nThe language model produces a probability of evaluating each word y in the vocabulary, and the probabilities are expressed as follows:

[0016]

number

[0017] Similar to language models, sequence-to-sequence models map fixed-length inputs with fixed-length outputs, where the input and output may differ in length. A sequence-to-sequence model maps a sequence x0, ..., x n Take the output sequence y0, ..., y m For example, in machine translation, the sequence x0, ..., x n represents a sentence in the first language with n words, y0, ..., y m represents a sentence in a second language that is different from the first language, possibly with a different amount of words. Sequence-to-sequence models can be used for a variety of tasks, including generating the next utterance to be produced by a chatbot, or generating an answer to a question given the question and the corresponding passage as input.

[0018] The model typically generates text continuously, e.g., one word at a time. Typically, the probability distribution for the next word to be generated depends on the input and words already generated. The "next" word to be generated affects the sentence to be generated, and in one embodiment, changes the sentence to be generated. As shown and described herein, the model is modified to create diversity in the output. The sequence-to-sequence model takes passages and responses as inputs and outputs corresponding questions. In one embodiment, this model with question output is called a question generator (QG). As shown and described herein, one embodiment is directed to teaching the QG to generate diverse questions and generate questions with desired semantics. The diversity is directed to reduce or avoid repeatedly generating the same questions, while reducing or avoiding generating text that deviates from the corresponding questions and answers. The desired semantics is directed to generate questions that a reasonable mechanism would answer as expected.

[0019] The systems, computer program products, and methods shown and described herein are directed to teaching a question generator (QG) to generate synthetic questions that address diversity and semantic characteristics. More specifically, embodiments are directed to teaching a QG to automatically generate diverse questions, constraining the generation of reasonable text, assessing the diversity of the generated questions, and assessing the quality, i.e., accuracy, diversity, fluency, or other relevant attributes, of the generated questions via reinforcement learning.

[0020] Referring to Figure 1, a schematic diagram of a computing system (100) is shown. As shown, a server (110) is provided for communication over links (102) and (104) with a plurality of computing devices (180), (182), (184), (186), (188), and (190) over a computer network (105). The server (110) is configured with a processor (112) in communication with a memory (116) over a bus (114). The server (110) is shown to include an artificial intelligence (Al) platform (150) for reinforcement learning from one or more of the computing devices (180), (182), (184), (186), (188), and (190) over the computer network (105). More specifically, computing devices (180), (182), (184), (186), (188), and (190) communicate with each other and with other devices or components via one or more wired and / or wireless data communications links, each of which may include one or more wires, routers, switches, transmitters, receivers, etc. Server (110) may be used with components, systems, subsystems, or devices other than those illustrated herein, or combinations thereof.

[0021] The AI ​​platform (150) is configured with tools to support machine learning, and more specifically, with tools to support reinforcement learning on synthetic data, e.g., machine-generated data, associated with corresponding ground truth (GT) data, e.g., validated or labeled data. It is understood in the art that synthetic data is artificially produced, and in one embodiment, produced as output from a corresponding neural model. The tools include, but are not limited to, a question manager (152), an answer manager (154), an assessment manager (156), and a director (158). The AI ​​platform (150) utilizes, and in one embodiment, modifies, a trained neural network. The AI ​​platform (150) can receive input from a computer network (105) and selectively process the data utilizing a data source (170), also referred to herein as a corpus or knowledge base. This processing includes, but is not limited to, generating synthetic data and utilizing the synthetic data to optimize output from a selected or identified neural model.

[0022] As shown, the data source (170) includes the model A (174 A ), Model B (174 B ), ..., Model N (174 N ) with a number of neural models. Although only three models are shown, this amount should not be considered limiting. Similarly, in one embodiment, the data source (170) may be considered as one or more additional libraries, each having one or more neural models. Each neural model may combine the functionality of a question generator, QG, and an answer generator, AG, into a single neural model. In one embodiment, the neural model may embody the QG and AG as components of the model, e.g., modelsA (174 A ) First Question Generator QG A,0 (176 A,0 ), the second question generator QG A,1 (176 A,1 ), and A.G. A (178 A ) and model B (174 B ) to QG B (176 B ) and A.G. B (178 B ) and model N (172 N ) to QG N (176 N ) and A.G. N (178 N ) can be embodied. How the model is utilized and, in one embodiment, modified is shown and explained in detail below.

[0023] Supervised learning is understood to leverage data reflected in one of the models in a data source. As presented herein, the data source is referred to as a knowledge base (170) and is comprised of models that are logically grouped together. The question manager (152) functions to identify appropriate libraries and models that are semantically related to the corresponding GT data. Once identified, the question manager applies a first passage of the GT (160 P ) and the corresponding ground first answer (160 A,0 ) to the identified model, e.g. model A (174 A ), which for purposes of illustration is referred to herein as the first QG component, QG A,0 (176 A,0 ) and the AG component AG A (178 A ) and the first QG component, e.g. QG A,0 (176 A,0 ) output from the first passage of GT (160 P) and GT's first answer (160 A,0 ) and the first question form that is synthesized to have a semantic correspondence with

[0024]

number

[0025] Thus, in response to a submission of GT data, the question manager (152) generates a first GQ component QG A,0 (176 A,0 ) the first semantically corresponding synthesis question

[0026]

number

[0027] Receive or obtain.

[0028] The answer manager (154) shown herein is operatively coupled to the question manager (152) and is operable to generate composite questions.

[0029]

number

[0030] More specifically, the answer manager (154) manages the processing of the identified or target neural model, e.g., model A (174 A ) depending on the AG component, e.g. AG A (178 A ) compound question

[0031]

number

[0032] Present or enter AG component AG A (178 A) is the first answer (160) presented to the Answer Manager (154). A,1 ) to generate AG, i.e. AG A (178 A ) generated first answer (160 A,1 ) is a compound question

[0033]

number

[0034] and GT Passage (160 P ) are semantically related to each other. Therefore, QG is a compound question.

[0035]

number

[0036] , and AG generates the answer (160 A,1 ) and both generated components are P ) and GT responses (160 A,0 ) is semantically related to

[0037] The assessment manager (156) shown herein is operatively coupled to both the question manager (152) and the answer manager (154). The assessment manager (156) is operable to generate a first answer (160). A,1 ) the generated synthetic question

[0038]

number

[0039] The evaluation manager (156) functions to automatically evaluate the composite questions. The goal is to generate composite questions that satisfy the requirements of the corresponding semantic relationships while enhancing the output from the AG and mitigating the repetition of the same questions.

[0040]

number

[0041] The reward function that captures both the diversity and the quality of the images is adopted. Thus, the evaluation manager (156) evaluates the diversity and the quality and reflects the evaluation in the corresponding reward function.

[0042] The director (158) shown herein is operatively coupled to the assessment manager (156) and is adapted to generate a first composite query.

[0043]

number

[0044] Based on the evaluation of the first question generator QG A,0 (176 A,0 ) in order to increase the textual diversity of the generated synthetic questions. As shown, the second question generator QG A,1 (176 A,1 ) is generated by the Question Manager (152) and is a model A (174 A ) operatively coupled to a second question generator QG A,1 (176 A,1 ) is the first question generator QG A,0 (176 A,0 ) and the corresponding GT passage (160 P ) and GT Answer (160 A,0 ) and the first synthetic question

[0045]

number

[0046] A second composite question that is different from

[0047]

number

[0048] For example, in one embodiment, the evaluation uses the question manager (152) to generate a modified QG A,1 (176 A,1 ) to the second compound question

[0049]

number

[0050] and the answer manager (154) generates a corresponding second compound question:

[0051]

number

[0052] Based on the second answer (160 A,2 In one embodiment, the optimization can proceed by instructing a first question generator QG to generate a policy gradient algorithm for its parameters based on a reward function. A,0 (176 A,0 ) and modifying the first and second composite questions.

[0053]

number

[0054] and

[0055]

number

[0056] In generating the synthetic questions, the director (158) employs sampling with text generation to promote diversity of the synthetic questions. The sampling may be kernel sampling, or in another embodiment, top-k sampling. The optimization of the question generator is performed by generating the first synthetic question,

[0057]

number

[0058] is expressed as a diversity score, and the first question generator QG A,0 (176 A,0 ) based on the diversity score, thereby generating a second question generator QG A,1 (176 A,1 The optimization employed by the director (158) is based on the first or previous composite query.

[0059]

number

[0060] In a similar context, a model that is a model A (174 A ) as output from the second or subsequent compound question

[0061]

number

[0062] In one embodiment, the evaluation reduces the generation of a first synthetic question.

[0063]

number

[0064] and the corresponding answers (160 A,0It should be appreciated that the director (158) may indicate that a given question satisfies a threshold or parameter of a reward function. Accordingly, the director (158) interfaces with the evaluation manager (156) to selectively generate one or more additional synthesis questions.

[0065] As briefly described above, the composite questions are evaluated for diversity. In addition, the evaluation manager (156) ensures that the first passage (160) of the GT P ) to support synthetic questions, evaluate or consider synthetic questions for accuracy. Synthetic questions are generated by the answer generator AG A (178 A ) and the generated answers associated with each synthetic question are A,0 ) Evaluation entails an evaluation manager (156), which manages the AG, i.e., AG A (178 A ) to the first synthetic question

[0066]

number

[0067] 1st reply in response to (160 A,0 ) and the second compound question

[0068]

number

[0069] 2nd Answer in Response to (160 A,1 ) in the form of GT answers (160 A,0) The evaluation manager (156) further quantifies the accuracy of the evaluation by an accuracy score. Based on the value assigned to the accuracy score, the question manager (152) is to selectively modify one or more parameters, including the generation of synthetic questions. The accuracy score and the diversity score described herein may be combined by the evaluation manager (156) to calculate a reward and communicate with the question manager (152) based on the calculated reward to selectively modify one or more parameters, including the generation of synthetic questions, to reduce or eliminate production by the question manager (152) of synthetic questions that are semantically similar to one or more previously generated synthetic questions. In one embodiment, the reward is a convex combination of the accuracy and diversity scores. Similarly, in one embodiment, the evaluation manager (156) may selectively prioritize one or more components of the reward function by selectively adjusting corresponding components or parameters of the convex combination. Response output (132) in the form of answer data identified within the passage by encouraging diversity in the generated synthetic questions may be presented on an operably coupled visual display (130) or communicated to one or more computing devices (180), (182), (184), (186), (188), and (190).

[0070] As shown, the computer network (105) can include local network connections and remote connections in various embodiments, such that the AI ​​platform (150) can operate in environments of any size, including local and global, e.g., the Internet. Additionally, the AI ​​platform (150) acts as a front-end system that makes available various knowledge extracted from or represented in network-accessible sources and / or structured data sources. In this manner, several processes populate the AI ​​platform (150), which also includes an input interface for receiving requests and responding accordingly.

[0071] The knowledge base (160) is comprised of a library and one or more neural networks populated or logically grouped for use by the AI ​​platform (150). Various computing devices (180), (182), (184), (186), (188), and (190) in communication with the computer network (105) may include access points for the logically grouped models. Some of the computing devices may include devices for a database to store a corpus of data as a body of information used by the AI ​​platform (150) to generate a response output (132) and communicate the response output (132) across the computer network (105) to a server (110) or a corresponding network device, such as a visual display (130), operably coupled to one or more of the computing devices (180), (182), (184), (186), (188), and (190).

[0072] In some exemplary embodiments, the server (110) may be an IBM Watson® system available from International Business Machines Corporation, Armonk, NY, enhanced by features of the exemplary embodiments described below. The managers (152), (154), (156), and (158) are hereinafter collectively referred to as AI tools and are shown embodied or integrated within the AI ​​platform (150) of the server (110). The AI ​​tools may be implemented in separate computing systems (e.g., 190) or, in one embodiment, they may be implemented in one or more systems connected to the server (110) across a computer network (105). Wherever embodied, the AI ​​tools function to dynamically optimize the generation of diverse and accurate synthetic questions to optimize the generated answers.

[0073] The types of devices and corresponding systems that can utilize the artificial intelligence platform (150) range from small handheld devices, such as handheld computers / cell phones (180), to large mainframe systems, such as mainframe computers (182). Examples of handheld computers (180) include personal entertainment devices, such as personal digital assistants (PDAs), MP4 players, portable televisions, and compact disc players. Other examples of information processing systems include pen or tablet computers (184), laptop or notebook computers (186), personal computer systems (188), and servers (190). As shown, the various devices and systems can be networked together using a computer network (105). Types of computer networks (105) that can be used to interconnect various devices and systems include a local area network (LAN), a wireless local area network (WLAN), the Internet, a public switched telephone network (PSTN), other wireless networks, and any other network topology that can be used to interconnect devices and systems. Many devices and systems include non-volatile data storage, such as a hard drive or non-volatile memory, or both. Some of the devices and systems may use separate non-volatile data storage (e.g., a server (190) may store non-volatile data storage (190 A ) and the mainframe computer (182) uses non-volatile data storage (182 A )) Non-volatile data storage (182 A ) may be a component external to the various devices and systems, or may be internal to one of the devices and systems.

[0074] The devices and systems employed to support the artificial intelligence platform (150) can take many forms, some of which are illustrated in FIG. 1. For example, the information processing system may take the form of a desktop, server, portable, laptop, notebook, or other form factor computer or data processing system. Additionally, the devices and systems may take other form factors, such as a personal digital assistant (PDA), gaming device, ATM machine, mobile phone device, communications device, or other device that includes a processor and memory.

[0075] An application program interface (API) is understood in the art as a software intermediary between two or more applications. With respect to the AI ​​platform (150) shown and described in Figure 1, one or more APIs may be utilized to support one or more of the tools (152), (154), (156), and (158) and their associated functionality. With reference to Figure 2, a block diagram (200) is provided illustrating at least some of the tools (252), (254), (256), and (258) and their associated APIs. As shown, multiple tools are embedded within the AI ​​platform (205), including a question manager (152), shown herein as (252) associated with API0 (212), an answer manager (154), shown herein as (254) associated with API1 (222), an assessment manager (156), shown herein as (256) associated with API2 (232), and a director (158), shown herein as (258) associated with API3 (242). Each API may be implemented in one or more languages ​​and interface specifications. API0 (212) provides a query manager for generating one or more synthetic questions. API1 (222) provides functional support for the generation of answers that are semantically related to the corresponding synthetic questions and GT passages. API2 (232) provides functional support for evaluating the generated answers with respect to GT responses, including incorporating a reward function into the evaluation, where the reward function captures both the diversity and accuracy of the first synthetic questions, where accuracy is measured using the first generated answers. API3 (242) provides functional support for selectively modifying the question generator in response to the answer evaluation, where the modifications are directed toward increasing the diversity of the text and maintaining the accuracy of the synthetic questions.As shown, each of the APIs (212), (222), (232), and (242) are operatively coupled to an API orchestrator (260), also known as an orchestration layer, which is understood in the art to act as an abstraction layer that transparently threads the separate APIs. In an embodiment, the functionality of the separate APIs may be combined or combined. As such, the configuration of APIs shown herein should not be considered limiting. Thus, the functionality of the tools may be embodied or supported by their respective APIs, as shown herein.

[0076] Referring to FIG. 3, a flow chart (300) is provided illustrating a process for applying reinforcement learning to a question generator to support and generate diverse and accurate synthetic questions. GT is a term used in machine learning that refers to a training set for machine learning techniques that includes reliable reference truth values. For example, a GT training set includes questions and answers generated by humans that are known to be true. For example, the labels of a data set classified as GT are assumed to be accurate. As shown and described herein and as instructed in training a QG model, a GT is provided in the form of a passage P, a question Q, and a corresponding answer A (302). As shown, the passage P and the answer A are presented as inputs to a trained neural network question generator (QG) (304). In one embodiment, a second neural network is provided to generate answers from the questions. Similarly, in one embodiment, a single neural network is constructed for both the question generator and the answer system. The QG produces an output corresponding to the received input, i.e., generates a synthetic question Q0′ aligned with the presented passage P and answer A (306). The process of generating a compound question Q0' is to select the words q that form the compound question Q0'. j Then, we generate the word q j In one embodiment, and as illustrated herein, word q is subjected to a diversity assessment. jis subjected to sampling, where at each word generation step, the possible next words are sorted. In the case of kernel sampling, a probability mass is established, e.g., a probability bound is established, and the probability of the word q j is limited. In one embodiment, the probability mass is a configurable parameter. Words forming and constituting the synthesis question are generated and evaluated until a word at the end of the synthesis question is reached, after which the synthesis question is generated from the QG model with words selected from the sampling. Thus, the synthesis question Q0' generated from the QG model is created to work with the GT passage P and the GT response A.

[0077] The synthetic questions generated in step (306) are further evaluated and refined for diversity and precision. As shown, an answer generator (AG) model is utilized to generate a second output, referred to herein as answer A0′, corresponding to the synthetic question Q0′ and the passage P (308). A precision reward for the answer is calculated. In one embodiment, the precision reward r acc is as follows:

[0078]

number

[0079] The precision reward evaluates the similarity between the answer A0′ generated by the AG model and the GT answer corresponding to the synthetic question. Furthermore, the diversity reward r div is also calculated 310 for the generated synthetic questions. div is the generated synthetic question for a GT question Q and all the questions {Q'1, ..., Q' i-1 The diversity distance is evaluated to quantify the novelty of the}. The precision reward and the diversity reward are combined to calculate a quality score, e.g., reward, for the QG model. Below is a representation of the calculation of the quality reward R:

[0080]

number

[0081] where w1 and w2 are adjustable weights for the precision and diversity rewards, respectively. Thus, as shown, the reward R is a weighted average of the precision and diversity rewards.

[0082] A loss is obtained from the reward R and provided to the QG model to optimize the quality of one or more future generated synthetic questions (312). The loss is utilized to update the parameters of the QG model (314), and an optimization algorithm is utilized to tune the QG (316) based on the loss obtained in step (312). The optimization in steps (314) and (316) uses reinforcement learning during training. This process, illustrated in FIG. 3, may be repeated across all instances of the training examples present in the training set, and the process may be repeated multiple times across all such instances. Reinforcement learning is utilized to obtain and apply the loss to learn how to generate synthetic questions output from the QG model, where the synthetic questions represent diverse content or context with respect to other generated synthetic questions and precise content or context with respect to the GT passage P and answer A.

[0083] The tool and its corresponding functionality utilizes reinforcement learning and corresponding reinforcement learning algorithms to optimize a question generator based on the synthetic questions and corresponding generated answers. The output from the reinforcement learning algorithm learns values ​​of different states, such as diversity and accuracy, and evaluates corresponding rewards that encompass these states. The reinforcement learning supports selective tuning of components of the reward, e.g., prioritization of one or more components, to enhance the diversity and / or accuracy of the synthetic questions and corresponding answers from the GT passage. Thus, the reinforcement learning shown and described herein can dynamically learn values ​​of different states and apply them to create a corresponding neural model.

[0084] The embodiments shown and described herein may be in the form of a computer system for use with an intelligent computer platform for providing and supporting directed reinforcement learning in generating diverse synthetic questions. Aspects of the tools (152), (154), (156), and (158) and their associated functionality may be embodied in a computer system / server at a single location or, in one embodiment, configured in a cloud-based system sharing computing resources. Referring to FIG. 4, a block diagram (400) illustrates an example of a computer system / server (402), hereafter referred to as a host (402), in communication with a cloud-based support system (410), for implementing the systems, tools, and processes described above with reference to FIGS. 1-3. The host (402) is operable in numerous other general purpose or special purpose computing system environments or configurations. Examples of well-known computing systems, environments, or configurations, or combinations thereof, that may be suitable for use with the host (402) include, but are not limited to, personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, and file systems that include any of the foregoing systems, devices, or equivalents thereof (e.g., distributed storage environments and distributed cloud computing environments).

[0085] The host (402) may be described in the general context of computer system executable instructions, such as program modules, being executed by a computer system. Generally, program modules may include routines, programs, objects, components, logic, data structures, etc. that perform particular tasks or implement particular abstract data types. The host (402) may also be practiced in a distributed cloud computing environment where tasks are performed by remote processing devices linked through a communications network. In a distributed cloud computing environment, program modules may be located in both local and remote computer system storage media, including memory storage devices.

[0086] As shown in FIG. 4, the host (402) is depicted in the form of a general-purpose computing device, such as a computer system and / or server. The components of the host (402) include, but are not limited to, one or more processors or processing units (404), such as a hardware processor, a system memory (406), and a bus (408) that couples various system components, including the system memory (406), to the processor (404). The bus (408) represents one or more of any of a number of types of bus structures, such as a memory bus or memory controller, a peripheral bus, an accelerated graphics port, and a processor or local bus using any of a variety of bus architectures. By way of example, but not limited to, such architectures include an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, an Enhanced ISA (EISA) bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus. The host (402) typically includes a variety of computer system readable media. Such media may be any available media that is accessible by the host (402) and includes both volatile and non-volatile media, removable and non-removable media.

[0087] The memory (406) may include computer system readable media in the form of volatile memory, such as random access memory (RAM) (430) or cache memory (432), or both. By way of example, the storage system (434) may be provided for reading from and writing to a non-removable, non-volatile magnetic medium (not shown, typically referred to as a "hard drive"). Although not shown, a removable, non-volatile magnetic disk (e.g., a "floppy disk") may be provided, as well as an optical disk drive for reading from and writing to a removable, non-volatile optical disk, such as a CD-ROM, DVD-ROM, or other optical media. In such cases, each may be connected to the bus (408) by one or more data media interfaces.

[0088] The programs / utilities (440) may include a set of (at least one) program modules (442) that may be stored in the memory (406) by way of example, but not by way of limitation, in an operating system, one or more application programs, other program modules, and program data. Each of the operating system, one or more application programs, other program modules, and program data, or some combination thereof, may include an implementation of a networking environment. The program modules (442) generally perform the functions and / or methods of the embodiments to dynamically orchestrate activities across one or more domains to minimize risk. For example, the set of program modules (442) may include tools (152), (154), (156), and (158), as described in FIG. 1 .

[0089] The host (402) may also communicate with one or more external devices (414), such as a keyboard or pointing device, a display (424), one or more devices that allow a user to interact with the host (402), or any device that allows the host (402) to communicate with one or more other computing devices (e.g., network cards, modems), or combinations thereof. Such communication may occur through an input / output (I / O) interface (422). Additionally, the host (402) may communicate with one or more networks, such as a local area network (LAN), a general wide area network (WAN), or a public network (e.g., the Internet), or combinations thereof, through a network adapter (420). As shown, the network adapter (420) communicates with other components of the host (402) through a bus (408). In one embodiment, multiple nodes of the distributed file system (not shown) communicate with the host (402) through an I / O interface (422) or through a network adapter (420). It should be understood that other hardware and / or software components, not shown, may be used in conjunction with the host (402). Examples include, but are not limited to, microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archive storage systems.

[0090] In this document, the terms "computer program medium," "computer usable medium," and "computer readable medium" are used generally to refer to media such as main memory (406), including RAM (430), cache (432), and storage systems (434), such as removable storage drives and hard disks installed within hard disk drives.

[0091] Computer programs (also called computer control logic) are stored in the memory (406). The computer programs may also be received via a communication interface, such as a network adapter (420). Such computer programs, when executed, enable the computer system to perform the features of the present embodiment as described herein. In particular, the computer programs, when executed, enable the processing unit (404) to perform the features of the computer system. Thus, such computer programs represent the controller of the computer system.

[0092] A computer readable storage medium may be a tangible device capable of holding and storing instructions for use by an instruction execution device. A computer readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer readable storage media includes portable computer diskettes, hard disks, dynamic or static random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), magnetic storage devices, portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory sticks, floppy disks, mechanically encoded devices such as punch cards or ridge-in-groove structures having instructions recorded thereon, and the like, and any suitable combination of the foregoing. Computer-readable storage media, as used herein, should not be construed as being themselves transitory signals such as electric waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., light pulses through a fiber optic cable), or electrical signals transmitted over wires.

[0093] The computer readable program instructions described herein can be downloaded from a computer readable storage medium to each computing / processing device or to an external computer or storage device via a network, such as the Internet, a local area network, a wide area network, or a wireless network, or a combination thereof. The network can include copper transmission cables, optical transmission fiber, wireless transmission, routers, firewalls, switches, gateway computers, or edge servers, or a combination thereof. A network adapter card or network interface in each computing / processing device receives the computer readable program instructions from the network and sends the computer readable program instructions for storage in a computer readable storage medium in each computing / processing device.

[0094] The computer readable program instructions for carrying out the operations of the present embodiment may be either assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including object oriented programming languages ​​such as Smalltalk, C++, and traditional procedural programming languages ​​such as the "C" programming language or similar programming languages. The computer readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server or cluster of servers. In the latter scenario, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or wide area network (WAN), or the connection may be made to an external computer (e.g., via the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, a field programmable gate array (FPGA), or a programmable logic array (PLA), may execute computer readable program instructions according to state information of the computer readable program instructions to implement aspects of the present embodiments.

[0095] In one embodiment, the host (402) is a node in cloud computing. As known in the art, cloud computing is a model of service delivery for enabling convenient and on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be rapidly provisioned and released with minimal management effort or interaction with a service provider. This cloud model may include at least five characteristics, at least three service models, and at least four deployment models. Examples of such characteristics are: On-Demand Self-Service: Cloud consumers may unilaterally provision computing capacity, such as server time or network storage, automatically as needed without the need for human interaction with the provider of the service. Broad network access: Capabilities are available over the network and accessed through standard mechanisms that facilitate use by heterogeneous thin- or thick-client platforms (e.g., cell phones, laptops, and PDAs). Resource Pooling: Provider computing resources are pooled to serve multiple consumers using a multi-tenant model, where different physical and virtual resources are dynamically allocated and reallocated according to demand. Consumers generally have no control or knowledge of the exact portion of resources provided to them, but there is a sense of portion independence in that they may be able to specify portions at a higher level of abstraction (e.g., country, state, or datacenter). Rapid Elasticity: Features can be provisioned rapidly and elastically, in some cases automatically, and can be quickly scaled out, quickly released, and quickly scaled in. To the consumer, the features available for provisioning often appear unlimited, and any amount can be purchased at any time. Measured Service: Cloud systems automatically control and optimize resource usage by leveraging metering capabilities at a level of abstraction appropriate to the type of service (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported, enabling transparency for both providers and consumers of the services being utilized.

[0096] The service model is as follows: Software as a Service (SaaS): The functionality provided to the consumer is the use of the provider's applications running on a cloud infrastructure. The applications are accessible from a variety of client devices through thin-client interfaces such as web browsers (e.g., web-based email). The consumer does not manage or control the underlying cloud infrastructure, including networks, servers, operating systems, storage, or even individual application functions, with the possible exception of limited user-specific application configuration settings. Platform as a Service (PaaS): The functionality offered to the consumer is the deployment of applications created or acquired by the consumer, written using programming languages ​​and tools supported by the provider, onto a cloud infrastructure. The consumer does not manage or control the underlying cloud infrastructure, including networks, servers, operating systems, or storage, but does have control over the deployed applications and, in some cases, the application hosting environment configuration. Infrastructure as a Service (IaaS): The functionality provided to the consumer is to provision processing, storage, network, and other basic computing resources onto which the consumer can deploy and run any software, which may include operating systems and applications. The consumer does not manage or control the underlying cloud infrastructure, but has control over the operating systems, storage, deployed applications, and possibly limited control over select networking components (e.g., host firewalls).

[0097] The deployment model is as follows: Private Cloud: The cloud infrastructure is operated solely for the organization. The cloud infrastructure can be managed by the organization or a third party and can exist on-premise or off-premise. Community Cloud: The cloud infrastructure is shared by several organizations to support a specific community with shared concerns (e.g., mission, security requirements, policies, and compliance considerations). The cloud infrastructure may be managed by the organization or a third party and may exist on-premise or off-premise. Public Cloud: The cloud infrastructure is made available to the general public or large industry groups and is owned by an organization that sells cloud services. Hybrid Cloud: A composition of two or more clouds (private, community, or public) where the cloud infrastructure remains a unique entity but is bound together by standardized or proprietary technologies that enable data and application portability (e.g., cloud bursting for load balancing between clouds).

[0098] A cloud computing environment is a service that focuses on statelessness, low coupling, modularity, and semantic interoperability. At the heart of cloud computing is an infrastructure that includes a network of interconnected nodes.

[0099] In FIG. 5, a cloud computing network (500) is shown. As shown, the cloud computing network (500) includes a cloud computing environment (550) having one or more cloud computing nodes (510) with which local computing devices used by cloud consumers may communicate. Examples of these local computing devices include, but are not limited to, a personal digital assistant (PDA) or cellular phone (554A), a desktop computer (554B), a laptop computer (554C), or an automobile computer system (554N), or combinations thereof. The individual nodes 300 within the node (510) may communicate with each other. They may be physically or virtually grouped into one or more networks, such as private, community, public, or hybrid clouds, or combinations thereof. This allows the cloud computing environment (500) to provide infrastructure, platform, or software, or combinations thereof, as a service without the cloud consumer having to maintain resources on the local computing device. It should be understood that the types of computing devices (554A-N) shown in FIG. 5 are merely exemplary, and that the cloud computing environment (550) can communicate with any type of computerized device via any type of network and / or network-addressable connections (e.g., using a web browser).

[0100] Referring now to Figure 6, there is shown a set of functional abstraction layers provided by the cloud computing network of Figure 5. It should be understood in advance that the components, layers, and functions shown in Figure 6 are intended to be illustrative only and the embodiments are not limited thereto. As shown, the following layers are provided: a hardware and software layer (610), a virtualization layer (620), a management layer (630), and a workload layer (640), and corresponding functions.

[0101] The hardware and software layer (610) includes hardware and software components. Examples of hardware components include mainframes, such as IBM® zSeries® systems, RISC (reduced instruction set computer) architecture-based servers, such as IBM pSeries® systems, IBM xSeries® systems, and IBM BladeCenter® systems, storage devices, and networks and networking components. Examples of software components include network application server software, such as IBM WebSphere® application server software, and database software, such as IBM DB2® database software (IBM, zSeries, pSeries, xSeries, BladeCenter, WebSphere, and DB2 are trademarks of International Business Machines Corporation, registered in numerous jurisdictions throughout the world).

[0102] The virtualization layer (620) provides an abstraction layer over which the following examples of virtual entities may be provided: virtual servers, virtual storage, virtual networks including virtual private networks, virtual applications and operating systems, and virtual clients.

[0103] As an example, the management layer (630) may provide the following functions: resource provisioning, metering and pricing, user portal, service layer management, and SLA planning and fulfillment. Resource provisioning provides dynamic acquisition of computing and other resources utilized to perform tasks within the cloud computing environment. Metering and pricing provides cost tracking as resources are utilized within the cloud computing environment and charging or billing for the consumption of these resources. As an example, these resources may include application software licenses. Security provides identity verification for cloud consumers and tasks, and protection for data and other resources. User portal provides access to the cloud computing environment for consumers and system administrators. Service layer management provides allocation and management of cloud computing resources such that required service levels are met. Service Level Agreement (SLA) planning and fulfillment provides pre-configuration and acquisition of cloud computing resources in anticipation of future requirements according to SLAs.

[0104] The workload tier (640) provides examples of functionality for which a cloud computing environment may be utilized. Examples of workloads and functions that may be provided from this tier include, but are not limited to, mapping and navigation, software development and lifecycle management, virtual classroom instructional delivery, data analytics processing, transaction processing, and synthetic data generation and reinforcement learning.

[0105] While specific embodiments of the present invention have been shown and described, it will be apparent to those skilled in the art that, based on the teachings herein, variations and modifications may be made without departing from the present invention and its broader aspects. Therefore, the appended claims are intended to encompass within their scope all such variations and modifications as are within the true spirit and scope of the present invention. It will be further understood that the present invention is defined solely by the appended claims. If the incorporation of a specific number of claim elements is intended, such an intention will be expressly recited in the claims, and those skilled in the art will understand that, in the absence of such recitation, there is no such limitation. As a non-limiting example and as an aid to understanding, the following appended claims include the use of the introductory phrases "at least one" and "one or more" to introduce claim elements. However, the use of such phrases should not be construed as implying that introducing a claim feature with the indefinite article "a" or "an" limits any particular claim that includes the introduced claim feature to embodiments containing only one of that feature, even if the same claim also includes the introductory phrase "one or more" or "at least one" and an indefinite article such as "a" or "an." The same applies to the use of definite articles in the claims.

[0106] The present embodiments may be systems, methods, or computer program products, or combinations thereof. Additionally, selected aspects of the present embodiments may take the form of entirely hardware embodiments, entirely software embodiments (including firmware, resident software, microcode, etc.), or embodiments combining software or hardware aspects or both, all of which may be referred to generally herein as "circuits," "modules," or "systems." Additionally, aspects of the present embodiments may take the form of a computer program product embodied in a computer-readable storage medium (or media) having computer-readable program instructions thereon for causing a processor to perform aspects of the present invention. Thus embodied, the disclosed system, method, or computer program product operates to improve the functionality and operation of an artificial intelligence platform to build a federated learning framework.

[0107] Aspects of the present embodiments are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to the present embodiments. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer readable program instructions.

[0108] These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to generate a machine such that the instructions, executed via the processor of the computer or other programmable data processing apparatus, generate means for implementing the functions / operations specified in one or more blocks of the flowcharts and / or block diagrams. These computer readable program instructions may also be stored on a computer readable storage medium that can cause a computer, programmable data processing apparatus, or other device, or combination thereof, to function in a particular manner, and thus a computer readable storage medium having instructions stored therein may include an article of manufacture including instructions that implement aspects of the functions / operations specified in one or more blocks of the flowcharts and / or block diagrams.

[0109] The computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device such that the instructions, which execute on the computer, other programmable apparatus, or other device, implement the functions / operations specified in one or more blocks of the flowcharts and / or block diagrams, causing a series of operational steps to be performed on the computer, other programmable apparatus, or other device to produce a computer-implemented process.

[0110] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block of the flowchart or block diagrams may represent a module, segment, or portion of instructions, including one or more executable instructions for implementing the specified logical function. In some alternative embodiments, the functions described in the blocks may occur out of the order described in the figures. For example, two blocks shown in succession may in fact be executed substantially simultaneously, or the blocks may be executed in reverse order, depending on the functionality involved. It should also be noted that each block of the block diagrams and / or flowchart illustrations, as well as combinations of blocks in the block diagrams and / or flowchart illustrations, may be implemented by a dedicated hardware-based system that performs the specified functions or operations or implements a combination of dedicated hardware and computer instructions.

[0111] Although specific embodiments have been described herein for purposes of illustration, it will be understood that various modifications may be made without departing from the spirit and scope of the embodiments. Accordingly, the scope of protection for the embodiments is limited only by the following claims and their equivalents.

Claims

1. a processor operatively coupled to the memory; An artificial intelligence (AI) platform having tools in communication with the processor to support reinforcement learning. and wherein the tool comprises: a question manager that inputs a first passage of ground truth and a first answer of ground truth to a trained neural network and obtains a first synthetic question generated by a question generator in the trained neural network, the first synthetic question having a semantic correspondence with the first passage of ground truth and the first answer; an answer manager that inputs the first synthetic question to the trained neural network and obtains a first answer generated by the trained neural network from the ground truth passage, the first answer being semantically related to the first synthetic question and the ground truth passage; an evaluation manager that automatically evaluates the generated first synthetic questions with respect to the generated first answers, the evaluation incorporating a reward function that captures the variety and accuracy of the generated first synthetic questions; a director operatively coupled to the assessment manager and employing reinforcement learning to selectively modify the question generator responsive to the assessments, the modified question generator producing one or more synthetic questions with increased text diversity than before the modification; A computer system comprising:

2. The system of claim 1 , wherein the selective modification of the question generator comprises the director employing kernel sampling to generate the one or more synthetic questions that differ from the first synthetic question.

3. 3. The system of claim 2, wherein optimizing the changes to the generated first synthetic question includes the director representing the generated first synthetic question with a diversity score and modifying one or more parameters of the trained neural network based on the diversity score, and the optimizing relaxes the generation of second synthetic questions as output from the trained neural network that have a similar context to the first question.

4. 4. The system of claim 3, further comprising: the evaluation manager evaluating accuracy of the first generated synthetic question by comparing the first generated answer to the ground truth answer and quantifying the accuracy evaluation with an accuracy score; and the question manager selectively modifying one or more question generation parameters responsive to the accuracy score.

5. 5. The system of claim 4, further comprising: the evaluation manager calculating a reward function as a convex combination of the accuracy scores and the diversity scores; the question manager selectively modifying one or more question generation parameters responsive to the calculated reward function; and the modification mitigating production of second synthetic questions that are semantically similar to the generated first synthetic question.

6. 6. The system of claim 5, wherein the evaluation manager further comprises adjusting the convex combination to selectively prioritize one or more components of the reward function.

7. 1. A computer program for supporting reinforcement learning, the computer program comprising: inputting a first passage of ground truth and a first answer of ground truth into a trained neural network, and obtaining a first synthetic question generated by a question generator in the trained neural network, the first synthetic question having a semantic correspondence with the first passage of ground truth and the first answer; inputting the first synthetic question into the trained neural network and obtaining a first answer generated by the trained neural network from the ground truth passage, the first answer being semantically related to the first synthetic question and the ground truth passage; automatically evaluating the generated first synthetic questions with respect to the generated first answers, the evaluation incorporating a reward function that captures the variety and accuracy of the generated first synthetic questions; employing reinforcement learning to selectively modify the question generator responsive to the evaluation, the modified question generator producing one or more synthetic questions with increased text diversity than before the modification. A computer program that enables the execution of the following:

8. 8. The computer program product of claim 7, wherein the selective modification of the question generator comprises employing kernel sampling to generate the one or more synthetic questions that differ from the first synthetic question.

9. 9. The computer program product of claim 8, wherein optimizing the changes to the generated first synthetic question comprises expressing the generated first synthetic question in a diversity score and modifying one or more parameters of the trained neural network based on the diversity score, wherein the optimizing mitigates production of second synthetic questions as output from the trained neural network that have a similar context as the first synthetic question.

10. 10. The computer program product of claim 9, further comprising: evaluating an accuracy of the first generated synthetic question by comparing the first generated answer to the ground truth answer; quantifying the accuracy assessment with an accuracy score; and selectively modifying one or more synthetic question generation parameters responsive to the accuracy score.

11. 11. The computer program product of claim 10, further comprising: computing a reward function as a convex combination of the accuracy scores and the diversity scores; and selectively modifying one or more question generation parameters responsive to the computed reward function, the modification mitigating production of second synthetic questions that are semantically similar to the generated first synthetic questions.

12. 12. The computer program product of claim 11, further comprising adjusting the convex combination to selectively prioritize one or more components of the reward function.

13. A computer-implemented method, the computer comprising: inputting a first passage of ground truth and a first answer of ground truth to a trained neural network; and obtaining a first synthetic question generated by a question generator in the trained neural network, the first synthetic question having a semantic correspondence with the first passage of ground truth and the first answer; inputting the first synthetic question into the trained neural network and obtaining a first answer generated by the trained neural network from the ground truth passage, the first answer being semantically related to the first synthetic question and the ground truth passage; automatically evaluating the generated first synthetic questions with respect to the generated first answers, the evaluation incorporating a reward function that captures the variety and accuracy of the generated first synthetic questions; employing reinforcement learning to selectively modify the question generator responsive to the evaluation, the modified question generator producing one or more synthetic questions with increased text diversity than before the modification.

4. A computer-implemented method comprising:

14. 14. The computer-implemented method of claim 13, wherein the selective modification of the question generator comprises employing kernel sampling to generate the one or more synthetic questions that differ from the first synthetic question.

15. 15. The computer-implemented method of claim 14, wherein optimizing the changes to the generated first synthetic question comprises expressing the generated first synthetic question with a diversity score and modifying one or more parameters of the trained neural network based on the diversity score, wherein the optimizing relaxes generation of second synthetic questions as output from the trained neural network that have a similar context as the first synthetic question.

16. 16. The computer-implemented method of claim 15, further comprising: evaluating accuracy of the first generated synthetic question by comparing the first generated answer to the ground truth answer, quantifying the accuracy assessment with an accuracy score, and selectively modifying one or more synthetic question generation parameters responsive to the accuracy score.

17. 17. The computer-implemented method of claim 16, further comprising: computing the reward function as a convex combination of the accuracy scores and the diversity scores; and selectively modifying one or more question generation parameters responsive to the computed reward function, the modification mitigating production of second synthetic questions that are semantically similar to the generated first synthetic questions.

18. 20. The computer-implemented method of claim 17, further comprising adjusting the convex combination to selectively prioritize one or more components of the reward function.

Images