System and Method for generating multi-language question automatically with a Google Translate API based translation engine

Abstract

A multilingual question automatic generation system capable of constructing an ensemble deep neural network framework for multilingual question automatic generation is disclosed. The multilingual question automatic generation system is characterized by comprising: an input unit for inputting original context information; a text translation unit for translating the input original context information into an English context; a question automatic generation unit for generating a plurality of question information from the English context; a translation engine for translating the question information into a user's set language; and an output unit for outputting the translated question information.

Description

Technology Field

[0001] The present invention relates to a multilingual question generation technology, and more specifically, to a system and method for automatically generating multilingual questions based on the context within a document written in multiple languages. Background Technology

[0002] A chatbot refers to a technology that helps achieve goals through automated conversation. Due to the high demand for chatbots recently, the global market size is growing explosively, and companies specializing in deep neural network-based chatbot services have emerged.

[0003] To develop deep neural network-based chatbots, training the chatbot engine using large-scale question-and-answer datasets is essential. However, in most companies, humans manually build question-and-answer datasets. In this regard, Neural Question Generation (NQG) and Neural Question Answering (NQA), research fields of deep neural network models for automatically generating question-and-answer datasets, have recently emerged.

[0004] However, current research trends in both NQG and NQA are focused on English. Although research is conducted within each country based on its own national language, it remains difficult to match the performance of NQG and NQA models based on English, the global lingua franca. Furthermore, utilizing various recently developed NQG and NQA models enables the generation of diverse question types.

[0005] Therefore, an ensemble deep neural network framework for automatic multilingual question generation is required. Prior art literature

[0006] 1. Korean Registered Patent No. 10-2259390 (Registration Date: May 26, 2021) The problem to be solved

[0007] The present invention is proposed to resolve the problems according to the above background technology, and aims to provide a multilingual question automatic generation system and method capable of constructing an ensemble deep neural network framework for multilingual question automatic generation.

[0008] In addition, the present invention has another objective of providing a multilingual question automatic generation system and method that automatically generates multilingual questions based on the context within a document written in Korean. means of solving the problem

[0009] To achieve the above-mentioned objectives, the present invention provides a multilingual question automatic generation system capable of constructing an ensemble deep neural network framework for multilingual question automatic generation.

[0010] The above multilingual question automatic generation system is,

[0011] Input section for inputting original context information;

[0012] A text translation unit that performs translation of the above-mentioned original context information into an English context;

[0013] A question automatic generation unit that generates multiple question information from the above English context;

[0014] A translation engine that translates the above question information into the user's set language; and

[0015] It is characterized by including an output unit that outputs the above-mentioned translated question information.

[0016] In addition, the text translation unit is characterized by including: a context translation module that translates the original context into the English context using the translation engine; and a question translation module that translates the English question into a national language set by the user using the translation engine.

[0017] In addition, the question automatic generation unit comprises: a question automatic generation module that generates a plurality of question information using the English context received from the context translation module; and a question ensemble module that aggregates the generated plurality of question information into a single list (QL) and then removes duplicate questions among the plurality of question information.

[0018] In addition, the above-mentioned automatic question generation module is characterized by including a plurality of Neural Question Generation (NQG) models designed in parallel to generate a plurality of the above-mentioned question information, respectively.

[0019] In addition, multiple NQG (Neural Question Generation) models are one of a single task model, a multi-task model, and an end-to-end model based on T5 (Text-To-Text Transfer Transformer), and are characterized by having a small or base version depending on the amount of pre-training.

[0020] In addition, the plurality of the above-mentioned question information is characterized in that it is in the form of a dictionary where both questions and answers are generated, or in the form of a list where only the above-mentioned questions are generated.

[0021] In addition, in the case of the above dictionary form, question information corresponding to the key value of each of the multiple NQG models is extracted and added to the above list (QL).

[0022] In addition, in the case of the above list form, it is characterized by the fact that multiple of the above question information are directly added to the above list (QL).

[0023] On the other hand, another embodiment of the present invention provides a multilingual question automatic generation method characterized by comprising: (a) a step of inputting original context information through an input unit; (b) a step of a text translation unit performing translation of the input original context information into an English context; (c) a step of a question automatic generation unit generating a plurality of question information from the English context; (d) a step of a translation engine translating the question information into a user's set language; and (e) a step of an output unit outputting the translated question information.

[0024] Additionally, the above step (b) is characterized by including: a step in which a context translation module translates the original context into the English context using the translation engine; and a step in which a question translation module translates the English question into the national language set by the user using the translation engine.

[0025] Additionally, the above step (c) is characterized by including: a step in which a question automatic generation module generates a plurality of question information using the English context received from the context translation module; and a step in which a question ensemble module combines the generated plurality of question information into a single list (QL) and then removes duplicate questions among the plurality of question information.

[0026] On the other hand, another embodiment of the present invention provides a computer-readable storage medium storing program code for executing the multilingual question automatic generation method described above. Effects of the invention

[0027] According to the present invention, it is possible to automatically generate multilingual questions through multilingual context input that is not limited to a specific national language.

[0028] In addition, another effect of the present invention is that by designing a plurality of automatic question generation models in an ensemble, the amount of questions generated by existing automatic question generation models can be significantly increased.

[0029] In addition, another advantage of the present invention is that, as it is a framework, it is possible to add, modify, or delete NQG models in the future.

[0030] In addition, another advantage of the present invention is that it can be applied to the generation of question-and-answer datasets for training deep neural network-based chatbots, thereby reducing existing labor costs.

[0031] In addition, another advantage of the present invention is that it can be incorporated into a multilingual chatbot engine through the automatic generation of multilingual questions. Brief explanation of the drawing

[0032] FIG. 1 is a block diagram of a multilingual question automatic generation system according to an embodiment of the present invention. Figure 2 is a detailed configuration block diagram of the question automatic generation execution unit illustrated in Figure 1. FIG. 3 is an example of pseudocode for automatic question generation according to an embodiment of the present invention. Figure 4 is a table showing a list of Neural Question Generation (NQG) models illustrated in Figure 2. FIG. 5 is a flowchart showing a process for automatically generating multilingual questions according to an embodiment of the present invention. Figure 6 is an example of a typical in-document context. FIG. 7 is an example of a context translation result according to an embodiment of the present invention. FIG. 8 is an example of an ensemble execution result according to an embodiment of the present invention. FIG. 9 is an example of the execution result of question restoration according to an embodiment of the present invention. Specific details for implementing the invention

[0033] The present invention is capable of various modifications and may have various embodiments, and specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the invention to specific embodiments, and it should be understood that the invention includes all modifications, equivalents, and substitutions that fall within the spirit and scope of the invention.

[0034] Similar reference numerals are used for similar components when describing each drawing. Terms such as "first," "second," etc., may be used to describe various components, but said components should not be limited by said terms. These terms are used solely for the purpose of distinguishing one component from another.

[0035] For example, without departing from the scope of the present invention, the first component may be named the second component, and similarly, the second component may be named the first component. The term "and / or" includes a combination of a plurality of related described items or any of a plurality of related described items.

[0036] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by those skilled in the art to which this invention pertains.

[0037] Terms such as those defined in commonly used dictionaries should be interpreted as having meanings consistent with their meanings in the context of the relevant technology, and should not be interpreted in an ideal or overly formal sense unless explicitly defined in this application.

[0038] A multilingual question automatic generation system and method according to an embodiment of the present invention will be described in detail below with reference to the attached drawings.

[0039] FIG. 1 is a block diagram of a multilingual question automatic generation system (100) according to an embodiment of the present invention. Referring to FIG. 1, the multilingual question automatic generation system (100) is characterized by being configured to include an input unit (110) for inputting original context information, a text translation unit (120) for performing translation into an English context based on the input original context information, a question automatic generation unit (130) for generating question information from the English context, a translation engine (140) for translating question information into a user's set language, and an output unit (150) for outputting translated question information.

[0040] The input unit (110) performs the function of inputting original context information. The input means may be a microphone for inputting voice, a keyboard for inputting text, a touch screen, etc. Of course, a pre-completed file format is also possible. In this case, a storage medium that stores this file format may be used, or it may be possible to download and input it online by connecting to the internet.

[0041] The text translation unit (120) performs the function of performing English context translation on input original context information. It also performs the function of translating question information into a language set by the user. To this end, the text translation unit (120) may be composed of a context translation module (121) that translates the original context into English and a question translation module (122) that translates the English question generated from the question automatic generation unit (130) into the country language desired by the user.

[0042] The context translation module (121) translates the original context, which is in a multilingual form within a document where the user wants to create a question, into an English context. This translation utilizes a translation engine (140). For actual multilingual translation, the translation engine (140) may utilize "Googletrans," an open-source translation library that utilizes the Python-based Google Translate API (Application Programming Interface).

[0043] GoogleTrans is compatible with Python 3.6 and higher, and has high utility as it possesses various features such as automatic language detection and bulk translations. Therefore, it can translate contexts written in various languages, such as Korean, German, French, Japanese, and Chinese, into English.

[0044] The question automatic generation unit (130) performs the function of generating question information from an English context. To this end, the question automatic generation unit (130) is composed of a question automatic generation module (131) that automatically generates questions based on an English context received from a context translation module (110), and a question ensemble module (132) that aggregates and filters the generated questions in an ensemble.

[0045] The automatic question generation module (131) performs on English context written in English input from the context translation module (121). The question ensemble module (220) combines the generated questions into one list and then removes duplicate questions.

[0046] The output unit (150) performs the function of outputting translated question information. Of course, it also performs the function of outputting a menu screen for settings, data input, language selection, etc. To this end, the output unit (150) may be an LCD (Liquid Crystal Display), an LED (Light Emitting Diode) display, a PDP (Plasma Display Panel), an OLED (Organic LED) display, a touch screen, a CRT (Cathode Ray Tube), a flexible display, a micro LED, a mini LED, etc. In the case of a touch screen, it may also be used as an input means.

[0047] FIG. 2 is a detailed block diagram of the automatic question generation unit (130) illustrated in FIG. 1. Referring to FIG. 2, the automatic question generation module (131) designs various forms of Text-To-Text Transfer Transformer (T5)-based Neural Question Generation (NQG) models in parallel to extract all questions within a context. That is, the first to nth NQG models (211-1 to 211-n) are configured in parallel. The NQG models (211-1 to 211-n) are deep neural network models for automatically generating a query dataset. They are pre-trained by setting a public context, such as Wikipedia, as an input value and a question-and-answer dataset based on that context, such as SQuAD, as an output value. Then, utilizing the trained NQG models, when a given context is input, a question corresponding to it is output.

[0048] Multiple NQG models (211-1 to 211-n) can generate all possible questions that can be generated in the corresponding English context (201), thus generating various forms of questions. That is, the first NQG model (211-1) generates the first question, the second NQG model (211-2) generates the second question, the third NQG model (211-3) generates the third question, and the nth NQG model (211-n) generates the nth question.

[0049] However, there may be duplicate questions among the questions generated from each NQG model (211-1 to 211-n). Therefore, the question ensemble module (220) aggregates the questions generated from each NQG model (211-1 to 211-n) into a single list and removes duplicate questions. To this end, the question ensemble module (220) is composed of a question collection module (220) and a filtering module (230). That is, the question collection module (220) aggregates the questions into a single list, and the filtering module (230) removes duplicate questions from the list. In other words, the question ensemble module (220) aggregates and filters the generated questions in an ensemble manner. Ensemble means collecting the questions output from each NQG model into a single list. Filtering means removing duplicate questions from the questions collected within the single list.

[0050] The term "…module" described in FIGS. 1 and 2 refers to a unit that processes at least one function or operation, which may be implemented in software and / or hardware. In a hardware implementation, it may be implemented as an application-specific integrated circuit (ASIC), digital signal processing (DSP), programmable logic device (PLD), field programmable gate array (FPGA), processor, microprocessor, other electronic unit, or a combination thereof, designed to perform the aforementioned function. In a software implementation, it may include software components (elements), object-oriented software components, class components and task components, processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, data, databases, data structures, tables, arrays, and variables. Software, data, etc. may be stored in memory and executed by a processor. Memory or processor may employ various means well known to those skilled in the art.

[0051] FIG. 3 is an example of pseudocode for automatic question generation according to an embodiment of the present invention. Referring to FIG. 3, an English context written in English output from a context translation module (121) is defined as C (line 2). A list of NQG models is defined as ML (line 3). The i-th model to refer to a specific NQG (Neural Question Generation) model is M i Defines as (line 4). Defines the generated question list as QL (line 5).

[0052] Q is the j-th question used to refer to a specific question among the generated questions. jIt is defined as (line 6). First, all NQG models are added to ML (lines 7-9). The various NQG models used in the present invention generate questions in the form of a Python dictionary or a list. Therefore, to distinguish the form of the generated questions, for NQG models that output in dictionary form, the question corresponding to the key value of each model is extracted and added to QL (lines 10-16). Among the output values ​​of NQG models that output in dictionary form, the key value is 'Question', and the value corresponding to that key value is 'Answer'.

[0053] In contrast, for NQG models where the generated questions are in the form of a list, they are added directly to the QL (lines 17-19). To filter out duplicate questions among the generated questions, duplicate questions within the QL are removed (lines 21-22).

[0054] Figure 4 is a table showing a list of Neural Question Generation (NQG) models illustrated in Figure 2. Referring to Figure 4, the list of Neural Question Generation (NQG) models consists of a total of six models utilizing the Text-To-Text Transfer Transformer (T5). The models can be classified into Single Task, Multi Task, and End-to-End types, and each type can be further subdivided based on the amount of pre-training, such as Small and Base.

[0055] The values ​​generated as a result of single-task and multi-task models are question-answer pairs, while end-to-end models generate only questions. The criteria for small and base are classified based on the amount of pre-training the model; for small, the size of the trained model is 230.8 MB, and for base, the size of the trained model is 850.3 MB.

[0056] The values ​​generated as a result of single-task and multi-task NQG models are in the form of dictionaries where both questions and answers are created. On the other hand, the output value of an end-to-end NQG model is in the form of a list because only questions are generated.

[0057] FIG. 5 is a flowchart illustrating a process for automatically generating a multilingual question according to an embodiment of the present invention. Referring to FIG. 5, when an original context is input through an input unit (110), a context translation module (121) of a text translation performing unit (120) converts the original context into an English context (201) using a translation engine (140) (steps S510, S520).

[0058] Afterwards, the question automatic generation module (131) of the question automatic generation unit (130) extracts question information from the English context (201) (step S530).

[0059] Afterwards, the question ensemble module (132) of the automatic question generation unit (130) aggregates the question information in an ensemble manner to generate a list and performs filtering to remove duplicate questions (step S540).

[0060] Afterwards, the question translation module (122) of the text translation execution unit (120) translates the question information into the desired national language and outputs it through the output unit (150) (step S550).

[0061] Figure 6 is an example of a general in-document context. That is, Figure 6 is an example of an original context. Referring to Figure 6, it is the context of the results of searching for "deep learning" in Korean on Wikipedia. In other words, an experiment to automatically generate Korean questions was conducted using the context of the results of searching for "deep learning" in Korean on Wikipedia. This original context is converted into an English context to be input into a context translation module (121 in Figure 1), and Figure 7 is a diagram showing this.

[0062] FIG. 7 is an example of a context translation result according to an embodiment of the present invention. Referring to FIG. 7, the original context shown in FIG. 6 is translated into English and converted into an English context (201 in FIG. 2). That is, the original Korean context is converted into an English context (201) using a translation engine (140).

[0063] FIG. 8 is an example of an ensemble execution result according to an embodiment of the present invention. Referring to FIG. 8, when the English context exemplified in FIG. 7 is input into the question auto-generation module (131), English questions are generated by inputting it into several NQG models. At this time, duplicate questions among the generated questions are deleted. Examples of questions are output as a single list as follows.

[0064] ① ‘What is deep learning called?’,

[0065] ② ‘What is deep learning defined as?’,

[0066] ③ ‘What type of information is represented in many studies?’

[0067] ....

[0068] In Fig. 8, ' [ ] ' represents a single list.

[0069] FIG. 9 is an example of the execution result of question translation according to an embodiment of the present invention. Referring to FIG. 9, English questions generated from the question ensemble module (132) of the automatic question generation unit (130) can be translated into a national language desired by the user. In this example, since the purpose is to generate Korean questions, an example of question generation through Korean translation is shown.

[0070] Additionally, the steps of the method or algorithm described in connection with the embodiments disclosed herein may be implemented in the form of program instructions that can be executed through various computer means, such as a microprocessor, a processor, a CPU (Central Processing Unit), etc., and recorded on a computer-readable medium. The computer-readable medium may include program (instruction) code, data files, data structures, etc., either alone or in combination.

[0071] The program (instruction) code recorded on the above medium may be those specifically designed and configured for the present invention, or those known and available to those skilled in the art of computer software. Examples of computer-readable recording media may include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs, DVDs, and Blu-rays; and semiconductor memory devices specifically configured to store and execute program (instruction) code, such as ROMs, RAMs, and flash memory.

[0072] Here, examples of program (instruction) code include not only machine code, such as that generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter, etc. The aforementioned hardware device may be configured to operate as one or more software modules to perform the operation of the present invention, and vice versa. Explanation of the symbols

[0073] *73100: Multilingual Question Automatic Generation System 110: Input section 120: Text Translation Execution Unit 121: Context Translation Module 122: Question Translation Module 130: Automatic Question Generation Execution Unit 131: Automatic Question Generation Module 132: Question Ensemble Module 140: Translation Engine 150: Output section 201: English Context 211-1 to 211-n: 1st to nth NQG (Neural Question Generation) models 220: Question Collection Module 230: Filtering Module

Claims

Claim 1 The system comprises: an input unit (110) for inputting original context information; a text translation unit (120) for translating the input original context information into an English context (201); a question automatic generation unit (130) for generating multiple question information from the English context (201); a translation engine (140) for translating the question information into a user's set language; and an output unit (150) for outputting the translated question information. The text translation unit (120) includes: a context translation module (121) for translating the original context into the English context (201) using the translation engine (140); and a question translation module (122) for translating the English question into a national language set by the user using the translation engine (140). The question automatic generation unit (130) includes: a question automatic generation module (131) for generating multiple question information using the English context (201) received from the context translation module (121). A question ensemble module (132) that combines the generated multiple question information into a single list (QL) and removes duplicate questions among the multiple question information;The question automatic generation module (131) includes a plurality of Neural Question Generation (NQG) models (211-1 to 211-n) designed in parallel to generate a plurality of question information, and the plurality of question information is in the form of a dictionary where both questions and answers are generated, or in the form of a list where only questions are generated. In the case of the dictionary form, question information corresponding to the key value of each of the plurality of NQG models (211-1 to 211-n) is extracted and added to the list (QL). The input unit (110) is a microphone for inputting voice or a keyboard for inputting text. The translation engine (140) is Googletrans, an open-source translation library utilizing the Python-based Google Translate API (Application Programming Interface). Googletrans performs automatic language detection and bulk translations. The output unit (150) is an LCD (Liquid Crystal Display) or an LED (Light A multilingual question automatic generation system utilizing a Google Translate API-based translation engine characterized by being any one of an Emitting Diode (Emitting Diode) display, a Plasma Display Panel (PDP), an Organic LED (OLED) display, a touch screen, a Cathode Ray Tube (CRT), a flexible display, a Micro LED, and a Mini LED.

Images