Device, method and program

Abstract

To output an answer to a question in a plurality of output forms when outputting the answer through machine reading comprehension.SOLUTION: A question-answer device has: a calculation part which inputs a question and a related document used to answer the question, and calculates information representing a relation between the question and the related document; a plurality of output parts which input the information calculated by the calculation part and representing the relation respectively and output answers to the question in mutually different output forms; and a selection part which selects a predetermined number of answers out of the respective answers that the plurality of output parts output in the respective output forms.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present disclosure relates to a question answering device, a question answering method, and a question answering program. [Background technology]

[0002] Machine reading comprehension is a known question-answering technology that automatically responds to questions entered by a user in natural language. Machine reading comprehension is a technology that inputs a user's question and a related document written in natural language (called a "passage"), and outputs a response to the input question based on information extracted from the passage.

[0003] The output format when outputting a response to a question by the machine reading comprehension is various, for example: - A format that outputs answer sentences generated by sentence generation based on information extracted from the passage, - A format that outputs labels such as YES / NO based on information extracted from the passage, - A format that outputs questions (questions to narrow down answers) generated based on information extracted from the passage, etc. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Kyosuke Nishida, Itsumi Saito, Kosuke Nishida, Kazutoshi Shinoda, Atsushi Otsuka, Hisako Asano, and Junji Tomita, "Multi-style generative reading comprehension", In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL2019), pp. 2273-2284, 2019. [Non-patent document 2] Kosuke Nishida, Kyosuke Nishida, Masaaki Nagata, Itsumi Saito, Atushi Otuka, Hisako Asano and Junji Tomita, "Answering while Summarizing: Multi-task Learning for Multi-hop QA with Evidence Extraction", Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL2019), pp.2273- 2284.2019. [Non-patent document 3] Atsushi Otsuka, Kyosuke Nishida, Itsumi Saito, Hisako Asano, Junji Tomita, and Tetsuji Sato. Proposal of a question-answering method using machine reading comprehension focusing on clarifying the intent of the question. Transactions of the Japanese Society for Artificial Intelligence, vol. 34, no. 5 A-J14, pp. 1-12, 2019. Summary of the Invention [Problem to be solved by the invention]

[0005] To achieve machine comprehension that can output in any of the above output formats, it is possible to combine machine reading comprehension models that correspond to each output format.

[0006] However, if a configuration were to be adopted in which a number of machine reading comprehension models corresponding to the number of output formats are expanded in memory and executed, a large amount of resources would be consumed within the device, and this is unlikely to be feasible in devices with limited resources.

[0007] An object of the present disclosure is to provide a question answering device, a question answering method, and a question answering program that are capable of outputting answers to questions in multiple output formats when using machine reading comprehension. [Means for solving the problem]

[0008] According to one aspect of the present disclosure, a question answering device includes: a calculation unit that receives a question and a related document used in answering the question as input and calculates information indicating the relevance between the question and the related document; a plurality of output units that receive the information indicating the relevance calculated by the calculation unit as an input and output responses to the questions in different output formats; The apparatus further includes a selection unit that selects a predetermined number of responses from among the responses output in each output format by the plurality of output units. [Effects of the Invention]

[0009] According to the present disclosure, it is possible to provide a question answering device, a question answering method, and a question answering program that are capable of outputting answers to questions in multiple output formats when using machine reading comprehension. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram illustrating an example of a hardware configuration of a question answering device. [Figure 2] FIG. 2 is a diagram illustrating a functional configuration of a question answering unit of the comparative example. [Figure 3] FIG. 3 is a diagram illustrating an example of a functional configuration of the question answering unit of the question answering device according to the first embodiment. [Figure 4] FIG. 4 is a first diagram illustrating an example of operation in the learning phase of the question answering device according to the first embodiment. [Figure 5] FIG. 5 is a second diagram illustrating an example of the operation in the learning phase of the question answering device according to the first embodiment. [Figure 6] FIG. 6 is a third diagram illustrating an example of operation in the learning phase of the question answering device according to the first embodiment. [Figure 7] FIG. 7 is a flowchart showing the flow of a question answering process by the question answering device according to the first embodiment. [Figure 8] FIG. 8 is a diagram illustrating an example of a functional configuration of the question answering device according to the second embodiment. [Figure 9] FIG. 9 is a first diagram illustrating an example of operation in the learning phase of the question answering device according to the second embodiment. [Figure 10] FIG. 10 is a second diagram illustrating an example of the operation in the learning phase of the question answering device according to the second embodiment. [Figure 11] FIG. 11 is a third diagram illustrating an example of operation in the learning phase of the question answering device according to the second embodiment. [Figure 12] FIG. 12 is a flowchart showing the flow of a question answering process by the question answering device according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, each embodiment will be described with reference to the accompanying drawings. In this specification and drawings, components having substantially the same functional configurations are designated by the same reference numerals, and redundant description will be omitted.

[0012] [First embodiment] <Hardware configuration of the question answering device> First, a description will be given of the hardware configuration of the question answering device according to the first embodiment. Fig. 1 is a diagram illustrating an example of the hardware configuration of the question answering device.

[0013] 1, the question answering device 100 includes a processor 101, a memory 102, an auxiliary storage device 103, an I / F (Interface) device 104, a communication device 105, and a drive device 106. The hardware components of the question answering device 100 are interconnected via a bus 107.

[0014] The processor 101 has various arithmetic devices such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), etc. The processor 101 reads various programs (not shown) installed in the auxiliary storage device 103 onto the memory 102 and executes them.

[0015] The memory 102 has a main storage device such as a ROM (Read Only Memory) and a RAM (Random Access Memory). The processor 101 and the memory 102 form a so-called computer, and the processor 101 executes various programs read onto the memory 102, thereby enabling the computer to realize various functions.

[0016] For example, in this embodiment, a computer formed by the processor 101 and the memory 102 implements the question answering unit 110 by the processor 101 executing a question answering program read onto the memory 102. As will be described later, the question answering unit 110 is configured to reduce consumption of computer resources within the question answering device 100.

[0017] The auxiliary storage device 103 stores various programs and various data used when the various programs are executed by the processor 101. For example, in this embodiment, the auxiliary storage device 103 has a training dataset storage unit 120 and a passage storage unit 130, and stores various data (training datasets and passages (related documents written in natural language)).

[0018] The I / F device 104 is a connection device that connects the input device 140 and the output device 141 with the question answering device 100. The I / F device 104 receives questions for the question answering device 100 via the input device 140. The I / F device 104 also outputs responses generated by the question answering device 100 in response to the input questions via the output device 141. The input device 140 here includes a device that converts the input questions into voice data, a device that converts the input questions into text data, etc. Similarly, the output device 141 here includes a device that outputs responses using voice data, a device that outputs responses using text data, etc.

[0019] The communication device 105 is a communication device for communicating with other devices via a network.

[0020] The drive device 106 is a device for loading a recording medium 142. The recording medium 142 here includes media that record information optically, electrically, or magnetically, such as a CD-ROM, a flexible disk, a magneto-optical disk, etc. The recording medium 142 may also include semiconductor memory that records information electrically, such as a ROM, a flash memory, etc.

[0021] The various programs to be installed in the auxiliary storage device 103 are installed, for example, by setting the distributed recording medium 142 in the drive device 106 and reading the various programs recorded on the recording medium 142 by the drive device 106. Alternatively, the various programs to be installed in the auxiliary storage device 103 may be installed by being downloaded from a network via the communication device 105.

[0022] Similarly, the various data stored in each storage unit of the auxiliary storage device 103 is stored, for example, by setting the distributed recording medium 142 in the drive device 106 and reading the various data recorded on the recording medium 142 by the drive device 106. Alternatively, the various data stored in each storage unit of the auxiliary storage device 103 may be stored by being downloaded from a network via the communication device 105.

[0023] <Functional configuration of the question answering unit> Next, we will explain the details of the functional configuration of the question answering unit 110. In order to clarify the characteristics of the functional configuration of the question answering unit 110, we will first explain, as a comparative example, the functional configuration of a question answering unit that is constructed by combining a number of machine reading comprehension models corresponding to the number of output formats.

[0024] (1) Functional configuration of the question answering unit in the comparative example 2 is a diagram showing the functional configuration of a question answering unit of a comparative example. When outputting a response to a question by machine reading comprehension, the question answering unit 200 of the comparative example has a configuration that enables output in multiple output formats. Input unit 210, Multiple machine reading comprehension models (in the example of Figure 2, the first machine reading comprehension model 221 to the third machine reading comprehension model 223), a selection unit 230; It has.

[0025] The input unit 210 inputs the input question and a passage (a related document written in a natural language) into each of a plurality of machine reading comprehension models.

[0026] The first machine reading comprehension model 221 outputs, as a first response, an answer sentence generated by sentence generation based on the information extracted from the passage.

[0027] The second machine reading comprehension model 222 outputs a label such as YES / NO as a second response, which is generated based on information extracted from the passage.

[0028] The third machine reading comprehension model 223 outputs a question generated based on information extracted from the passage (a question generated to narrow down the answers (referred to as a revised question)) as a third response.

[0029] Selection unit 230 selects and outputs a predetermined number of responses from among the responses output from first machine reading comprehension model 221 to third machine reading comprehension model 223, respectively.

[0030] As shown in the question answering unit 200 of the comparative example, by combining a number of machine reading comprehension models corresponding to the number of output formats, it is possible to output answers to questions in multiple output formats. However, the question answering unit 200 of the comparative example has the following problems. Since the question answering apparatus 100 is configured to load and execute a number of machine reading comprehension models corresponding to the number of output formats in memory, a large amount of computer resources are consumed within the apparatus. It is not possible to select an appropriate response when selecting a predetermined number of responses from among the responses output by each of the first machine reading comprehension model 221 to the third machine reading comprehension model 223. This is because it is not possible to determine the relative merits of the responses simply by comparing the first to third responses, and it is necessary to calculate some kind of selection index.

[0031] On the other hand, the question answering unit 110 of the question answering apparatus 100 according to the first embodiment has a configuration that solves these problems, as will be explained in detail below.

[0032] (2) Functional configuration of the question answering unit 3 is a diagram showing an example of the functional configuration of a question answering unit of the question answering device according to the first embodiment. The question answering unit 110 is configured to be able to output a response to a question in a plurality of output formats by machine reading comprehension, while suppressing consumption of computer resources, and to be able to select an appropriate response from among the responses to a question in a plurality of output formats. Input unit 310, a comprehension layer 320, which acts as an input layer; A first output layer 321, a second output layer 322, and a third output layer 323 functioning as output layers; an output decision layer 324, which functions as an output layer; a selection unit 330; It has.

[0033] Of these, the input unit 310 inputs the input question and a passage (a related document written in a natural language) to the understanding layer 320.

[0034] The understanding layer 320 is an example of a calculation unit that receives a question and a passage as input, calculates information indicating the relationship between the question and the passage on a deep learning vector, and outputs a state vector or a state tensor. Note that the understanding layer 320 can adopt any structure as long as it can receive a question and a passage as input and calculate information indicating the relationship between the question and the passage.

[0035] For example, the understanding layer 320 can employ BiDAF (see Non-Patent Document 1) using RNN() or Transformer-based BERT (see Non-Patent Document 2). However, the understanding layer 320 must have a structure that outputs data in accordance with the format of the state vector or state tensor that is input to the output layers subsequent to the understanding layer 320 (first output layer 321 to third output layer 323, output decision layer 324).

[0036] The first output layer 321 to the third output layer 323 are examples of output sections, which take as input information (state vector or state tensor) indicating the relationship between the question and the passage output from the comprehension layer 320, and output a response (the output result of machine reading comprehension).

[0037] In the case of Figure 3, the first output layer 321 outputs an answer sentence generated by sentence generation as a first response. The second output layer 322 outputs a label such as YES / NO as a second response. Furthermore, the third output layer 323 outputs a revised question to narrow down the answers to the input question as a third response.

[0038] The first output layer 321 to the third output layer 323 may have any deep learning structure when outputting the first response to the third response, respectively. Furthermore, the output formats output by the first output layer 321 to the third output layer 323 are not limited to answer sentences, labels, and revision questions, and responses in other output formats may also be output.

[0039] 3, first to third output layers are provided, but any number of output layers may be provided, and multiple output layers with the same structure may be provided. For example, two sentence generation decoder structures may be provided, with one output layer trained to generate answer sentences through sentence generation, and the other output layer trained to generate revised questions through sentence generation.

[0040] The output determination layer 324 is an example of an index calculation unit, and calculates the probability distribution of each response output from the first output layer 321 to the third output layer 323.

[0041] Specifically, the output determination layer 324 has a softmax layer with a number of dimensions corresponding to the number N of output formats (N=3 in the example of FIG. 3). The N-dimensional softmax layer receives the state vector or state tensor from the understanding layer 320 and calculates the probability distribution of each response.

[0042] This configuration is effective when the number of output formats N is fixed. However, if an output format needs to be added, the number of dimensions of the softmax layer cannot accommodate this, and so the entire question answering unit 110 must be retrained.

[0043] The selection unit 330 selects the top M responses (M is a predetermined number; in the example of FIG. 3, M=1) in the probability distribution calculated by the output determination layer 324 from the responses output from the first output layer 321 to the third output layer 323, and outputs them to the user via the output device 141.

[0044] In this way, the question answering unit 110 - Establish a shared understanding layer for multiple output formats (share the input layer), The input layer (understanding layer) and the output layer (first output layer 321 to third output layer 323, output judgment layer 324) are separated and each layer is modularized. The selection unit 330 selects the response to be finally output using the probability distribution of each response calculated by the output determination layer 324 installed in the output layer as a selection index. As a result, the question answering unit 110 can output a response to a question using machine reading comprehension in a plurality of output formats, and - Sharing the input layer reduces the consumption of computer resources, and -Selecting the appropriate response by calculating the selection index; This makes it possible.

[0045] <Learning method for the question-answering section> Next, a learning method for the question answering unit 110 will be described. In a learning phase in which learning processing is performed on the question answering unit 110, the question answering unit 110 of the question answering device 100 according to this embodiment first has a comparison / change unit 410 installed in place of the selection unit 330. Next, the question answering unit 110 of the question answering device 100 according to this embodiment performs learning processing on each layer (the understanding layer 320, the first output layer 321 to the third output layer 323, and the output determination layer 324) installed in the question answering unit 110.

[0046] At this time, question answering unit 110 of question answering device 100 uses a learning data set that is used when performing learning processing on the individual machine reading comprehension models (first machine reading comprehension model 221 to third machine reading comprehension model 223).

[0047] Specifically, the question answering unit 110 of the question answering device 100 first sets a flag indicating for which output layer the learning data set is to be used in the learning process.

[0048] Next, the question answering unit 110 of the question answering device 100 stores the correct answer data of the output determination layer 324 in the training dataset. For example, the question answering unit 110 of the question answering device 100 stores vector data in which the value of the dimension corresponding to the set flag is "1" and the values ​​of the other dimensions are "0" in the training dataset as the correct answer data of the output determination layer 324.

[0049] Next, the question answering unit 110 of the question answering device 100 calculates the learning loss between the response output from the output layer corresponding to the set flag and the corresponding correct answer data, and updates the parameters of the output layer and the understanding layer corresponding to the set flag based on the calculated learning loss. At this time, the question answering unit 110 of the question answering device 100 ignores responses output from output layers other than the output layer corresponding to the set flag.

[0050] In addition, the question answering unit 110 of the question answering device 100 calculates the learning loss between the vector data output from the output determination layer and the corresponding correct answer data, and updates the parameters of the output determination layer based on the calculated learning loss.

[0051] 4 is a first diagram showing an example of operation in the learning phase of the question answering device according to the first embodiment. In the case of FIG. 4, it is assumed that a flag indicating that the learning dataset 400 is "a learning dataset used in the learning process for the first output layer 321" is set by the question answering unit 110.

[0052] As shown in FIG. 4, the training dataset 400 includes information items such as "input data," "correct answer data of the first output layer," and "correct answer data of the output determination layer." "Input data" stores questions and passages. "First output layer correct answer data" stores the correct answer data of the answer sentence generated by sentence generation based on the information extracted from the corresponding passage. The "correct answer data of the output judgment layer" includes the "1st dimension" to the "3rd dimension", and stores vector data with the value of the 1st dimension set to "1" and the values ​​of the 2nd and 3rd dimensions set to "0".

[0053] 4, when input unit 310 inputs input data (pairs of questions and passages) from training dataset 400 to comprehension layer 320, first to third responses are output from first to third output layers 321 to 323 as machine reading comprehension output results. Also, output determination layer 324 outputs vector data with N dimensions (N=3 in the example of FIG. 4).

[0054] The comparison / change unit 410 calculates the learning loss between the first response (answer sentence) output from the first output layer 321 and the answer sentence stored in the "correct answer data of the first output layer" of the learning dataset 400. The comparison / change unit 410 also updates the parameters of the first output layer 321 and the understanding layer 320 based on the calculated learning loss.

[0055] Similarly, the comparison / change unit 410 calculates a learning loss between the N-dimensional (N=3 in the example of FIG. 4) vector data output from the output determination layer 324 and the first- to third-dimensional vector data stored in the "correct data of the output determination layer" of the training dataset 400. The comparison / change unit 410 also updates the parameters of the output determination layer 324 based on the calculated learning loss.

[0056] 5 is a second diagram showing an example of operation in the learning phase of the question answering device according to the first embodiment. In the case of FIG. 5, it is assumed that a flag indicating that the learning dataset 500 is "a learning dataset used in learning processing for the second output layer 322" has been set by the question answering unit 110.

[0057] As shown in FIG. 5, the training dataset 500 includes information items such as "input data," "correct answer data of the second output layer," and "correct answer data of the output determination layer." "Input data" stores questions and passages. The "correct answer data for the second output layer" stores correct answer data such as YES / NO labels generated based on information extracted from the corresponding passage. The "correct answer data of the output judgment layer" includes the "1st dimension" to the "3rd dimension", and stores vector data in which the value of the 2nd dimension is "1" and the values ​​of the 1st and 3rd dimensions are "0".

[0058] 5, when input unit 310 inputs input data (pairs of questions and passages) of training dataset 500 to comprehension layer 320, first to third responses are output from first to third output layers 321 to 323 as machine reading comprehension output results. Also, output determination layer 324 outputs vector data of N dimensions (N=3 in the example of FIG. 5).

[0059] The comparison / change unit 410 calculates the learning loss between the second response (label) output from the second output layer 322 and the label stored in the "correct answer data of the second output layer" of the learning dataset 500. The comparison / change unit 410 also updates the parameters of the second output layer 322 and the understanding layer 320 based on the calculated learning loss.

[0060] Similarly, the comparison / change unit 410 calculates a learning loss between the N-dimensional (N=3 in the example of FIG. 5) vector data output from the output determination layer 324 and the first- to third-dimensional vector data stored in the "correct data of the output determination layer" of the training dataset 500. The comparison / change unit 410 also updates the parameters of the output determination layer 324 based on the calculated learning loss.

[0061] 6 is a third diagram showing an example of operation in the learning phase of the question answering device according to the first embodiment. In the case of FIG. 6, it is assumed that a flag indicating that the learning dataset 600 is "a learning dataset used in learning processing for the third output layer 323" is set by the question answering unit 110.

[0062] As shown in FIG. 6, the training dataset 600 includes information items such as "input data," "correct answer data of the third output layer," and "correct answer data of the output determination layer." "Input data" stores questions and passages. The "third output layer correct answer data" stores the correct answer data for the revised question generated based on the information extracted from the corresponding passage. The "correct answer data of the output judgment layer" includes the "1st dimension" to the "3rd dimension", and stores vector data in which the value of the 3rd dimension is "1" and the values ​​of the 1st and 2nd dimensions are "0".

[0063] 6, when the input unit 310 inputs input data (pairs of questions and passages) of the training dataset 600 to the comprehension layer 320, the first output layer 321 to the third output layer 323 output first to third responses as output results of machine reading comprehension. Also, the output determination layer 324 outputs vector data of N dimensions (N=3 in the example of FIG. 6).

[0064] The comparison / modification unit 410 calculates the learning loss between the third response (revised question) output from the third output layer 323 and the revised question stored in the "correct answer data of the third output layer" of the training dataset 600. The comparison / modification unit 410 also updates the parameters of the third output layer 323 and the understanding layer 320 based on the calculated learning loss.

[0065] Similarly, the comparison / change unit 410 calculates a learning loss between the N-dimensional (N=3 in the example of FIG. 6) vector data output from the output determination layer 324 and the first- to third-dimensional vector data stored in the "correct data of the output determination layer" of the training dataset 600. The comparison / change unit 410 also updates the parameters of the output determination layer 324 based on the calculated learning loss.

[0066] In this way, the question answering unit 110 of the question answering device 100 uses the learning datasets 400 to 600 to perform learning processing sequentially on the understanding layer 320, the first output layer 321 to the third output layer 323, and the output determination layer 324.

[0067] <Question-answering process flow> Next, a description will be given of the flow of question answering processing by the question answering device 100. Fig. 7 is a flowchart showing the flow of question answering processing by the question answering device according to the first embodiment. Of these, steps S701 to S703 represent processing in the learning phase, and steps S704 to S707 represent processing in the answering phase.

[0068] In step S701, the question answering unit 110 uses the learning dataset 400 to perform learning processing on the understanding layer 320, the first output layer 321, and the output determination layer 324.

[0069] In step S702, the question answering unit 110 performs a learning process on the understanding layer 320, the second output layer 322, and the output determination layer 324 using the learning dataset 500.

[0070] In step S703, the question answering unit 110 performs a learning process on the understanding layer 320, the third output layer 323, and the output determination layer 324 using the learning dataset 600.

[0071] In step S704, the input unit 310 of the question answering unit 110 accepts the input of a question and a passage, and inputs the input question and passage to the understanding layer 320.

[0072] In step S705, the first output layer 321 to the third output layer 323 receive the state vector output from the understanding layer 320 as an input and output a first response to a third response.

[0073] In step S706, the output determination layer 324 of the question answering unit 110 receives the state vector output from the understanding layer 320 as input, calculates the probability distribution of the first to third responses, and outputs a selection index.

[0074] In step S707, the selection unit 330 of the question answering unit 110 selects a predetermined top M number of responses based on the selection index output from the output determination layer 324, and outputs the selected responses.

[0075] <Summary> As is clear from the above description, the question answering device 100 according to the first embodiment · It has an understanding layer that takes a question and a passage as input and calculates information indicating the relationship between the question and the passage. The system has first to third output layers that use the information indicating the relevance calculated by the understanding layer as their respective inputs and output first to third responses in different output formats. The system has an output determination layer that calculates a probability distribution of the first to third responses based on the information indicating the relevance output by the understanding layer, and further has a selection unit that selects a predetermined number of responses using the probability distribution of the first to third responses calculated by the output determination layer as a selection index.

[0076] As a result, the question answering device 100 according to the first embodiment can output answers to questions in a plurality of output formats, reduce consumption of computer resources, and select an appropriate answer.

[0077] In other words, according to the first embodiment, it is possible to provide a highly feasible question answering device, question answering method, and question answering program that can output answers to questions in multiple output formats using machine reading comprehension.

[0078] [Second embodiment] In the first embodiment described above, it is assumed that the number of output formats is fixed, and when a new output format is added, the question answering unit is configured to perform a re-learning process on the entire question answering unit 110.

[0079] In contrast to this, in the second embodiment, even when a new output format is added, the question answering unit is configured so that there is no need to perform re-learning processing for the entire question answering unit 110. The second embodiment will be described below, focusing on the differences from the first embodiment.

[0080] <Functional configuration of the question answering unit> First, a description will be given of the functional configuration of a question answering unit of the question answering device according to the second embodiment. Fig. 8 is a diagram illustrating an example of the functional configuration of the question answering unit of the question answering device according to the second embodiment.

[0081] The difference from the functional configuration shown in Fig. 3 is that the question answering unit 800 in Fig. 8 is provided with first output determination layer 801 to third output determination layer 803 that calculate individual scores as selection indices for the first output layer 321 to third output layer 323. Also, the question answering unit 800 in Fig. 8 has a different function from the function of the selection unit 330 in Fig. 3.

[0082] The first output determination layer 801 is an example of an index calculation unit, and includes a logit layer that receives the state vector of the first output layer 321 and calculates a scalar value between 0 and 1.0 as the first score.

[0083] Similarly, the second output determination layer 802 is an example of an index calculation unit, and includes a logit layer that receives the state vector of the second output layer 322 and calculates a scalar value between 0 and 1.0 as the second score.

[0084] Similarly, the third output determination layer 803 is an example of an index calculation unit, and has a logit layer that receives the state vector of the third output layer 323 and calculates a scalar value between 0 and 1.0 as the third score.

[0085] The selection unit 810 selects and outputs responses corresponding to the top M predetermined scores based on the first to third scores calculated by the first to third output determination layers 801 to 803.

[0086] As described above, the question answering device 100 according to the second embodiment is provided with the first output determination layer 801 to the third output determination layer 803, which calculate individual scores for the first output layer 321 to the third output layer 323, respectively. As a result, according to the second embodiment, even when a new output format is added, it is sufficient to perform learning processing on the added new output layer and output determination layer and the understanding layer, and there is no need to perform re-learning processing on the output layer and output determination layer that have already been trained.

[0087] <Learning method for the question-answering section> Next, a learning method for the question answering unit 800 will be described. In a learning phase in which learning processing is performed on the question answering unit 110, the question answering unit 800 of the question answering device 100 according to this embodiment first installs a comparison / change unit 910 in place of the selection unit 810. Next, the question answering unit 800 of the question answering device 100 according to this embodiment performs learning processing on each layer (the understanding layer 320, the first output layer 321 to the third output layer 323, and the first output determination layer 801 to the third output determination layer 803) installed within the question answering unit 800.

[0088] In this case, the question answering device 100 uses a learning data set that is used when performing learning processing on the individual machine reading comprehension models (first machine reading comprehension model 221 to third machine reading comprehension model 223), as in the first embodiment.

[0089] Specifically, the question answering unit 800 of the question answering device 100 first sets a flag indicating for which output layer the learning data set is to be used in the learning process.

[0090] Next, the question answering unit 800 of the question answering device 100 stores the correct answer data of the first score to the third score calculated by the first output determination layer 801 to the third output determination layer 803 in the training data set. For example, the question answering unit 800 of the question answering device 100 stores the correct answer data in which the score corresponding to the set flag is set to "1.0" in the training data set.

[0091] Next, the question answering unit 800 of the question answering device 100 calculates the learning loss between the response output from the output layer corresponding to the set flag and the corresponding correct answer data, and updates the parameters of the output layer and the understanding layer corresponding to the set flag based on the calculated learning loss. At this time, the question answering unit 800 of the question answering device 100 ignores responses output from output layers other than the output layer corresponding to the set flag.

[0092] Furthermore, the question answering unit 800 of the question answering device 100 calculates a learning loss between the score output from the output determination layer corresponding to the set flag and the corresponding correct answer data, and updates the parameters of the output determination layer corresponding to the set flag based on the calculated learning loss. At this time, the question answering device 100 ignores scores output from output determination layers other than the output determination layer corresponding to the set flag.

[0093] 9 is a first diagram showing an example of operation in the learning phase of the question answering device according to the second embodiment. In the case of FIG. 9, it is assumed that a flag indicating that the learning dataset 900 is "a learning dataset used in the learning process for the first output layer 321 and the first output determination layer 801" has been set by the question answering unit 800.

[0094] As shown in FIG. 9, the training dataset 900 includes information items such as "input data," "correct answer data of the first output layer," and "correct answer data of the output determination layer." "Input data" stores questions and passages. "First output layer correct answer data" stores the correct answer data of the answer sentence generated by sentence generation based on the information extracted from the corresponding passage. In the "correct data of output determination layer", correct data of the score (first score) output from the first output determination layer 801 is stored.

[0095] 9, when the input unit 310 inputs input data (pairs of questions and passages) of the training dataset 900 to the comprehension layer 320, the first to third responses are output as machine reading comprehension output results from the first to third output layers 321 to 323. In addition, the first to third output determination layers 801 to 803 output first to third scores.

[0096] The comparison / change unit 910 calculates a learning loss between the first response (answer sentence) output from the first output layer 321 and the answer sentence stored in the "correct answer data of the first output layer" of the training dataset 900. The comparison / change unit 910 also updates the parameters of the first output layer 321 and the understanding layer 320 based on the calculated learning loss.

[0097] Similarly, the comparison / change unit 910 calculates a learning loss between the first score output from the first output determination layer 801 and the value stored in the first score of the "correct data of the output determination layer" in the training dataset 900. The comparison / change unit 910 also updates the parameters of the first output determination layer 801 based on the calculated learning loss.

[0098] 10 is a second diagram showing an example of operation in the learning phase of the question answering device according to the second embodiment. In the case of FIG. 10, it is assumed that a flag indicating that the learning dataset 1000 is "a learning dataset used in learning processing for the second output layer 322 and the second output determination layer 802" has been set by the question answering unit 800.

[0099] As shown in FIG. 10, the training dataset 1000 includes information items such as "input data," "correct answer data of the second output layer," and "correct answer data of the output determination layer." "Input data" stores questions and passages. The "correct answer data for the second output layer" stores correct answer data such as YES / NO labels generated based on information extracted from the corresponding passage.

[0100] The "correct data of the output determination layer" stores the correct data of the score (second score) output from the second output determination layer 802.

[0101] 10, when the input unit 310 inputs input data (pairs of questions and passages) of the training dataset 1000 to the comprehension layer 320, the first to third responses are output as machine reading comprehension output results from the first to third output layers 321 to 323. In addition, the first to third output determination layers 801 to 803 output first to third scores.

[0102] The comparison / change unit 910 calculates the learning loss between the second response (label) output from the second output layer 322 and the label stored in the "correct answer data of the second output layer" of the training dataset 1000. The comparison / change unit 910 also updates the parameters of the second output layer 322 and the understanding layer 320 based on the calculated learning loss.

[0103] Similarly, the comparison / change unit 910 calculates a learning loss between the second score output from the second output determination layer 802 and the value stored in the second score of the "correct data of the output determination layer" in the training dataset 1000. The comparison / change unit 910 also updates the parameters of the second output determination layer 802 based on the calculated learning loss.

[0104] 11 is a third diagram showing an example of operation in the learning phase of the question answering device according to the second embodiment. In the case of FIG. 11, it is assumed that a flag indicating that the learning dataset 1100 is "a learning dataset used in learning processing for the third output layer 323 and the third output determination layer 803" has been set by the question answering unit 800.

[0105] As shown in FIG. 11, the training dataset 1100 includes information items such as "input data," "correct answer data of the third output layer," and "correct answer data of the output determination layer." "Input data" contains questions and passages. The "third output layer correct answer data" stores the correct answer data for the revised question generated based on the information extracted from the corresponding passage. The "correct data of the output determination layer" stores the correct data of the score (third score) output from the third output determination layer 803.

[0106] 11, when the input unit 310 inputs input data (pairs of questions and passages) of the training dataset 1100 to the understanding layer 320, the first to third responses are output as machine learning output results from the first to third output layers 321 to 323. In addition, the first to third scores are output from the first to third output determination layers 801 to 803.

[0107] The comparison / modification unit 910 calculates the learning loss between the third response (revised question) output from the third output layer 323 and the revised question stored in the "correct answer data of the third output layer" of the training dataset 1100. The comparison / modification unit 910 also updates the parameters of the third output layer 323 and the understanding layer 320 based on the calculated learning loss.

[0108] Similarly, the comparison / change unit 910 calculates a learning loss between the third score output from the third output determination layer 803 and the value stored in the third score of the "correct data of the output determination layer" in the learning dataset 1100. The comparison / change unit 910 also updates the parameters of the third output determination layer 803 based on the calculated learning loss.

[0109] <Question-answering process flow> Next, a flow of question answering processing by the question answering device 100 according to the second embodiment will be described. Fig. 12 is a flowchart showing the flow of question answering processing by the question answering device according to the second embodiment. The difference from the flowchart described with reference to Fig. 7 in the first embodiment is steps S1201 to S1203.

[0110] In step S1201, the question answering unit 110 performs a learning process on the understanding layer 320, the first output layer 321, and the first output determination layer 801 using the learning dataset 900.

[0111] In step S1202, the question answering unit 110 performs a learning process on the understanding layer 320, the second output layer 322, and the second output determination layer 802 using the learning dataset 1000.

[0112] In step S1203, the question answering unit 110 performs a learning process on the understanding layer 320, the third output layer 323, and the third output determination layer 803 using the learning dataset 1100.

[0113] <Summary> As is clear from the above description, the question answering apparatus 100 according to the second embodiment · It has an understanding layer that takes a question and a passage as input and calculates information indicating the relationship between the question and the passage. The system has first to third output layers that use the information indicating the relevance calculated by the understanding layer as their respective inputs and output first to third responses in different output formats. The system has first to third output determination layers that receive state vectors from the first to third output layers and calculate individual scores (first to third scores) for the first to third output layers. It also has a selection unit that selects a predetermined number of responses using the first to third scores calculated by the first to third output determination layers as selection indices.

[0114] As a result, the question answering device 100 according to the second embodiment can output answers to questions in multiple output formats, reduce consumption of computer resources, and select an appropriate answer, just like the first embodiment. In addition, the question answering device 100 according to the second embodiment does not need to perform re-learning processing on the entire question answering unit even when a new output format is added.

[0115] In other words, according to the second embodiment, it is possible to provide a more feasible question answering device, question answering method, and question answering program that can output answers to questions through machine reading comprehension in multiple output formats.

[0116] [Other embodiments] In the first and second embodiments, different output determination layers are installed, but the decision as to whether to install the output determination layer in the first embodiment or the output determination layer in the second embodiment is arbitrary. For example, the decision may be made taking into consideration the task or purpose to be set, the system configuration of the question answering device, etc.

[0117] In the first and second embodiments, the learning phase and the response phase are described as being executed in the same question answering device 100. However, the learning phase and the response phase may be executed in separate devices. In this case, the device that executes the response phase does not need to have the training dataset storage unit 120, and the comparison / modification unit 410, 910 is not installed.

[0118] The present invention is not limited to the configurations described in the above embodiments, but may be combined with other elements, etc. These aspects can be changed without departing from the spirit of the present invention, and can be appropriately determined depending on the application form. [Explanation of symbols]

[0119] 100: Question answering device 110: Question and Answer Section 120: Learning dataset storage section 130: Passage storage area 310: Input section 320: Understanding layer 321: First output layer 322: Second output layer 323: Third output layer 324: Output decision layer 330: Selection section 400~600: Training dataset 801: First output decision layer 802: Second output decision layer 803: Third output decision layer 810: Selection section 900~1100: Training dataset

Claims

1. a calculation unit that receives a question and a related document used in answering the question as input and calculates information indicating the relevance between the question and the related document; a plurality of output units that receive the information indicating the relevance calculated by the calculation unit as an input and output responses to the questions in different output formats; a selection unit that selects a predetermined number of responses from among the responses output in each output format by the plurality of output units; A question answering device having the above structure.

2. an index calculation unit that receives the information indicating the relevance calculated by the calculation unit and calculates a probability distribution of each response output in each output format by the plurality of output units; The question answering apparatus according to claim 1 , wherein the selection unit selects a predetermined number of responses based on a probability distribution of each of the responses.

3. Further comprising a plurality of index calculation units that receive information calculated when the plurality of output units output each response and calculate a score for each of the plurality of output units, The question answering device according to claim 1 , wherein the selection unit selects a predetermined number of responses based on scores for each of the plurality of output units.

4. a calculation step of calculating information indicating the relevance between a question and a related document used in answering the question, using the question and the related document as input; a plurality of output steps for outputting responses to the questions in different output formats, each using the information indicating the relevance calculated in the calculation step as an input; a selection step of selecting a predetermined number of responses from among the responses output in each output format in the plurality of output steps; A question answering method having the following steps.

5. a calculation step of calculating information indicating the relevance between a question and a related document used in answering the question, using the question and the related document as input; a plurality of output steps for outputting responses to the questions in different output formats, each using the information indicating the relevance calculated in the calculation step as an input; a selection step of selecting a predetermined number of responses from among the responses output in each output format in the plurality of output steps; A question-answering program for a computer to execute the above.

Images