PROGRAM, INFORMATION PROCESSING METHOD, INFORMATION PROCESSING DEVICE, AND METHOD FOR GENERATING LEARNING MODEL

Abstract

To provide a program, etc., which improve work efficiency in creation of various questions.SOLUTION: A program disclosed herein causes a computer to perform processing for receiving exam question creation conditions from a user, generating questions and answers according to the received creation conditions, and outputting the generated questions and answers to the user.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a program, an information processing method, an information processing device, and a method for generating a learning model. [Background technology]

[0002] Creating exam questions for entrance exams, qualification exams, etc. requires knowledge and effort. Therefore, technologies to support the creation of exam questions have been proposed.

[0003] For example, Patent Document 1 discloses an automatic question generation device that can automatically generate complex grammar fill-in-the-blank questions without much effort. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2008-129449 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, the technology described in Patent Document 1 focuses on the automatic creation of fill-in-the-blank questions, and does not consider the generation of questions in formats other than fill-in-the-blank questions. Since test question formats are diverse, a technology that supports the creation of a variety of questions is desired.

[0006] An object of the present disclosure is to provide a program etc. that can improve the efficiency of creation work related to various problems. [Means for solving the problem]

[0007] A program according to one aspect of the present disclosure causes a computer to execute a process of accepting conditions for creating test questions from a user, generating questions and answers based on the accepted conditions, and outputting the generated questions and answers to the user. Effect of the Invention

[0008] According to the present disclosure, the efficiency of creation work related to a variety of problems can be improved. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a schematic diagram of an information processing system. [Diagram 2] FIG. 1 is a block diagram showing an example of the configuration of an information processing device. [Diagram 3] FIG. 2 is a block diagram showing an example of the configuration of a user terminal. [Figure 4] FIG. 2 is an explanatory diagram showing an overview of a first example of a learning model. [Diagram 5] FIG. 11 is an explanatory diagram showing an overview of a second example of a learning model. [Figure 6] FIG. 11 is an explanatory diagram showing an overview of a third example of a learning model. [Figure 7] FIG. 13 is an explanatory diagram showing an overview of a fourth example of a learning model. [Figure 8] FIG. 2 is an explanatory diagram showing an example of a test question. [Figure 9] 13 is a flowchart illustrating an example of a learning model generation processing procedure. [Figure 10] 13 is a flowchart showing an example of a procedure for generating test questions executed by the information processing system. [Figure 11] FIG. 13 is a schematic diagram showing an example of a reception screen. [Figure 12] FIG. 13 is a schematic diagram illustrating an example of a template. [Figure 13] 13 is a flowchart showing an example of a detailed procedure for generating test questions. [Figure 14] 13 is a flowchart showing another example of a detailed procedure for generating test questions. [Figure 15] 13 is a flowchart showing another example of a detailed procedure for generating test questions. [Figure 16] 13 is a flowchart showing another example of a detailed procedure for generating test questions. [Figure 17]FIG. 13 is a schematic diagram showing an example of a confirmation screen. [Figure 18] 13 is a flowchart illustrating an example of a processing procedure for a review request executed by the information processing apparatus according to the second embodiment. [Figure 19] 13 is a flowchart illustrating an example of a procedure for a correction receiving process executed by the information processing apparatus. [Figure 20] FIG. 13 is a schematic diagram showing an example of a screen including a correction history. [Figure 21] 13 is a flowchart illustrating an example of a procedure of a generation process of analysis information executed by the information processing apparatus according to the third embodiment. [Figure 22] FIG. 13 is a schematic diagram showing an example of a screen including analysis information. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] The present disclosure will be specifically described with reference to the drawings showing the embodiments thereof.

[0011] (First embodiment) 1 is a schematic diagram of an information processing system 100. The information processing system 100 includes an information processing device 1 and a user terminal 2. The information processing device 1 and the user terminal 2 are communicatively connected via a network N such as the Internet. The number of user terminals 2 may be two or more.

[0012] The information processing device 1 is a device capable of various information processes and transmitting and receiving information, such as a server computer, a personal computer, a quantum computer, etc. The user terminal 2 is an information terminal device such as a personal computer, a smartphone, a tablet terminal, etc.

[0013] The information processing device 1 generates exam questions in response to a question creation request received from the user terminal 2, and provides the generated exam questions to the user terminal 2. The information processing device 1 realizes a question creation management service to assist users in creating exam questions. A user who enjoys the question creation service can send and receive information between the information processing device 1 and the user terminal 2, and obtain information provided by the information processing device 1.

[0014] In this specification, test questions include questions for various tests such as entrance exams and qualification exams, and study questions for various tests. Test questions include a question statement indicating the content of the question, and an answer indicating a correct answer or a model answer to the question statement. For example, when the answer format is multiple choice, the question statement may include a question body indicating the body of the question, and options. Test questions may further include an explanation regarding the answer.

[0015] 2 is a block diagram showing a configuration example of the information processing device 1. The information processing device 1 includes a control unit 11, a storage unit 12, and a communication unit 13. The information processing device 1 may be configured to perform distributed processing by using a plurality of computers, may be realized by a plurality of virtual machines provided in one server, or may be realized by using a cloud server.

[0016] The control unit 11 includes a processor using one or more CPUs (Central Processing Units), GPUs (Graphics Processing Units), etc. The control unit 11 uses built-in memories such as ROMs (Read Only Memory) or RAMs (Random Access Memory), clocks, counters, etc. to control each component and execute processes.

[0017] The storage unit 12 includes a non-volatile memory such as a hard disk, a flash memory, or an SSD (Solid State Drive). The storage unit 12 may be an external storage device connected to the information processing device 1. The storage unit 12 stores various programs and data referenced by the control unit 11. The storage unit 12 of this embodiment stores a program 1P for causing a computer to execute a process related to the generation of test questions, and a learning model 121 required for executing the program 1P. The learning model 121 is a machine learning model that has learned predetermined training data. The learning model 121 is expected to be used as a program module that constitutes a part of artificial intelligence software. The data stored in the storage unit 12 may include, for example, generated test questions, existing test questions, templates described later, keyword information, advice tables, and the like.

[0018] A program (program product) including the program 1P may be provided by a non-transitory recording medium 1A on which the program is readably recorded. The storage unit 12 stores the program read from the recording medium 1A by a reading device (not shown). The recording medium 1A is, for example, a magnetic disk, an optical disk, a semiconductor memory, or the like. The program may also be downloaded from an external server connected to a communication network and stored in the storage unit 12. The program 1P may be a single computer program or may be composed of multiple computer programs, and may be executed on a single computer or multiple computers interconnected by a communication network.

[0019] The communication unit 13 includes a communication module for communicating with an external device via the network N. The control unit 11 transmits and receives data to and from the user terminal 2 via the communication unit 13.

[0020] The configuration of the information processing device 1 is not limited to the above example, and may include, for example, an operation unit for accepting user operations, a display unit for displaying various information, and the like.

[0021] 3 is a block diagram showing an example of the configuration of the user terminal 2. The user terminal 2 includes a control unit 21, a storage unit 22, a communication unit 23, a display unit 24, and an operation unit 25.

[0022] The control unit 21 includes a processor using one or more CPUs, GPUs, etc. The control unit 21 controls each component unit and executes processing using built-in memory such as ROM or RAM, a clock, a counter, etc. The storage unit 22 includes a non-volatile memory such as a hard disk, flash memory, SSD, etc. The storage unit 22 stores various programs and data referenced by the control unit 21. The communication unit 23 includes a communication module for performing processing related to communication. The control unit 21 transmits and receives data to and from the information processing device 1 via the communication unit 23.

[0023] The display unit 24 includes a display device such as a liquid crystal display or an organic EL (electroluminescence) display. The display unit 24 displays various information according to instructions from the control unit 21. The operation unit 25 is an interface that accepts user operations. The operation unit 25 includes, for example, a keyboard, a touch panel device with a built-in display, a speaker, a microphone, and the like. The operation unit 25 accepts operation input from the user and sends a control signal to the control unit 21 according to the operation content.

[0024] The information processing device 1 receives test question creation conditions from a user via the user terminal 2, and automatically generates test questions according to the creation conditions using the learning model 121. As described above, a test question includes at least a question sentence and an answer.

[0025] The creation conditions include, for example, question creation conditions and answer creation conditions. The question creation conditions include information on the question (question) format, and are classified into a number of conditions, for example, fill-in-the-blanks and prompts. The answer creation conditions include information on the answer format, and are classified into a number of conditions, for example, multiple choice and written. The user can select creation conditions according to the purpose of question creation from the preset question creation conditions and answer creation conditions.

[0026] Fig. 4 is an explanatory diagram showing an overview of a first example of the learning model 121. The learning model 121 shown in Fig. 4 receives question sentences and answers related to existing test questions (hereinafter also simply referred to as existing questions) as input, and outputs question sentences and answers related to new test questions (hereinafter also simply referred to as new questions). The learning model 121 is, for example, a natural language processing model using a neural network. Note that a new test question means a new test question whose content is different from at least the existing questions that are input elements, and does not need to have new content for all existing questions.

[0027] In this embodiment, the learning model 121 generates a question sentence and an answer by using a natural language processing technique using, for example, a Transformer model. The learning model 121 may be a pre-training model using a Transformer, such as, for example, Bidirectional Encoder Representations from Transformers (BERT), Generative Pre-trained Transformer (GPT)-3, GPT-2, or the like.

[0028] The learning model 121 has an input layer that accepts input of existing questions, an intermediate layer (hidden layer) that extracts features of the existing questions, and an output layer that outputs new questions. For example, the existing questions are input to the input layer as tokens generated by analyzing the text of a sentence including a question statement and an answer by morphological analysis or the like. The intermediate layer has a plurality of nodes that extract features of each input value, and passes the features extracted using various parameters to the output layer. The output layer is configured to output new questions based on the features output from the intermediate layer. In the Transformer model, the intermediate layer is provided with an Attention layer, and the position of a word in the entire sentence can be extracted. Note that natural language processing models such as Transformer and BERT are well-known technologies, and therefore further detailed explanations are omitted.

[0029] The learning model 121 stored in the storage unit 12 of the information processing device 1 is a trained model that has been previously subjected to machine learning processing. The learning processing is a process of optimizing various parameters in a neural network using a large amount of previously given training data. The parameters are, for example, weights (coupling coefficients) between neurons.

[0030] The information processing device 1 may fine-tune the pre-trained learning model 121 as a model for generating question sentences and answers. The information processing device 1 generates training data for fine tuning by, for example, collecting a large amount of existing questions and new questions. The training data for fine tuning is, for example, a data set in which a question sentence and an answer related to an existing question are associated with a question sentence and an answer related to a new question. The training data may be a data set in which a question sentence and an answer related to a first existing question are associated with a question sentence and an answer related to a second existing question different from the first existing question.

[0031] The information processing device 1 inputs a sentence linking existing questions and new questions contained in the training data to the input layer of the pre-learning neural network model, and obtains new questions output from the output layer through calculation processing in the intermediate layer. The information processing device 1 compares the new questions output from the output layer with new questions contained in the training data, and optimizes various parameters so that the new questions output from the output layer approach the correct answer. Through the above processing, a learning model 121 can be constructed that has been trained to be able to appropriately output new questions for existing questions.

[0032] The configuration of the learning model 121 is not limited to the above example. The learning model 121 may be capable of generating a new problem different from an existing problem. The learning model 121 may use other learning algorithms such as a recurrent neural network (RNN) or a long short-term memory (LSTM).

[0033] The information processing device 1 stores in advance in the storage unit 12 question group data in which a large amount of collected existing test questions are associated with creation conditions. For example, when the information processing device 1 receives a question format and a description format as creation conditions, the information processing device 1 extracts existing questions corresponding to the question format and the description format from the question group data. The information processing device 1 provides the extracted existing questions as input data to the learning model 121, and obtains new questions to be output. This allows the information processing device 1 to generate new question sentences and answers according to the creation conditions.

[0034] The learning model 121 is not limited to generating new questions based on existing questions. FIG. 5 is an explanatory diagram showing an overview of a second example of the learning model 121. The learning model 121 shown in FIG. 5 is a model that receives a reference sentence and a question for the reference sentence as input, and outputs a question sentence and an answer. The output layer of the learning model 121 is configured to output the question sentence and the answer. The configurations of the other learning models 121 exemplified below are all similar to the learning model 121 shown in FIG. 4, so detailed explanations will be omitted.

[0035] The reference sentence may include, for example, a sentence (text data) extracted from a textbook, a reference book, etc. The question content includes a sentence indicating a content to be asked based on the reference sentence.

[0036] The information processing device 1 uses training data in which reference sentences and question contents are associated with question sentences and answers to train (fine-tune) the learning model 121. This constructs the learning model 121 that has been trained to be able to appropriately output question sentences and answers to the reference sentences and question contents.

[0037] Fig. 6 is an explanatory diagram showing an overview of a third example of the learning model 121. The learning model 121 shown in Fig. 6 is a model that receives a question sentence indicating a question as input and outputs an answer to the question sentence. The output layer of the learning model 121 is configured to output an answer to the question sentence.

[0038] The question sentence to be input to the learning model 121 may be generated based on, for example, the above-mentioned reference sentence, or a longer sentence extracted from a textbook, a reference book, or the like. The information processing device 1 extracts sentences that fall within the scope of the question from the textbook, a reference book, or the like, and generates the question sentence based on the extracted sentence. The question sentence may be generated using, for example, a natural language processing model that generates a question sentence according to a sentence. The question sentence may be generated by an external device or manually.

[0039] The information processing device 1 uses training data in which the generated questions and answers are associated with each other to train the learning model 121. In this way, the learning model 121 that has been trained to be able to appropriately output answers to questions is constructed.

[0040] Fig. 7 is an explanatory diagram showing an overview of a fourth example of the learning model 121. The learning model 121 shown in Fig. 7 is a model that receives a sentence extracted from a textbook, a reference book, etc. as an input and outputs a summary of the sentence. The output layer of the learning model 121 is configured to output a summary.

[0041] The information processing device 1 trains the learning model 121 using training data in which sentences, each of which is a concatenation of sentences and summaries extracted from textbooks, reference books, etc., are associated with the summaries. This allows the learning model 121 to be constructed so as to be able to appropriately output summaries for sentences extracted from textbooks, etc.

[0042] The information processing device 1 can generate a fill-in-the-blank question in which the words in the summary are masked according to a predetermined rule, for example, based on the summary generated using the learning model 121, and an answer corresponding to the fill-in-the-blank portion. Alternatively, a descriptive question asking about a predetermined word and an answer may be generated based on the summary generated using the learning model 121. The word or the predetermined word to be filled in may be determined by, for example, using keyword information stored in advance in the storage unit 12 to identify a word that matches a keyword stored in the keyword information from among the words included in the summary.

[0043] In the above-mentioned learning process, it is preferable that the fine tuning is performed for each user's problem creation purpose using training data corresponding to the user's problem creation purpose. For example, the training data may be data from a textbook or reference book corresponding to the user's problem creation purpose, or existing problems corresponding to the user's problem creation purpose. This makes it possible to appropriately generate problems corresponding to the user's problem creation purpose using the learning model 121 customized for each user.

[0044] Furthermore, when the answer format is multiple choice, the information processing device 1 generates a plurality of options based on the answers generated using the learning model 121. Each option includes a correct answer or an incorrect answer similar to the answer. The method of generating an incorrect answer is not limited, and for example, the information processing device 1 may generate an incorrect answer using a machine learning model that receives an answer as input and outputs an incorrect answer to the answer. The information processing device 1 may generate an incorrect answer by vectorizing a word or phrase and identifying a word or phrase whose similarity to the vector of the answer word or phrase is equal to or greater than a predetermined value.

[0045] The information processing device 1 may also generate an incorrect answer by acquiring data with the second highest priority after the answer identified as the output data with the highest priority (probability) based on the output data of the learning model 121, or data with the second highest priority (probability), etc. Similarly, the information processing device 1 can generate correct and incorrect sentences based on the likelihood of words in a sentence. For example, if the answer to a question that requires the selection of a correct answer is "Transformer uses positional encoding," the information processing device 1 generates an incorrect answer "Transformer uses positional vectors" by replacing "positional encoding" with "positional vectors," which is the next candidate of "positional encoding." The information processing device 1 generates an incorrect answer "Transformer erases positional encoding" by replacing the verb "use" with "erase," which is the next candidate of "use."

[0046] The information processing device 1 may also identify an incorrect answer according to the answer by using a table that classifies and stores similar words. If the answer includes a numerical value, the information processing device 1 may generate an incorrect answer by converting the numerical value of the correct answer according to a predetermined rule. The information processing device 1 may generate an incorrect answer by converting between an affirmative expression and a negative expression. The information processing device 1 generates a predetermined number of options in which the answer is replaced with an incorrect answer.

[0047] FIG. 8 is an explanatory diagram showing an example of a test question. FIG. 8 shows an example of a test question generated when the creation conditions are fill-in-the-blank and multiple choice. The test question includes a question text and an answer generated using the learning model 121. The question text also includes a fixed phrase indicating a question according to the question creation conditions, and options. The automatically generated test question may further include an explanation according to the answer.

[0048] Fig. 9 is a flowchart showing an example of a process procedure for generating the learning model 121. The processes in each of the following flowcharts may be executed by the control unit 11 in accordance with a program 1P stored in the storage unit 12 of the information processing device 1, may be realized by a dedicated hardware circuit (e.g., FPGA or ASIC) provided in the control unit 11, or may be realized by a combination of these. In the following, steps are abbreviated as S. Fig. 9 explains an example of generating the learning model 121 of the third example described above.

[0049] The control unit 11 of the information processing device 1 acquires training data (S11). The control unit 11 acquires a question generated based on a sentence extracted from a textbook or reference book according to the user's purpose for creating questions, and an answer to the question. The control unit 11 acquires training data in which a label indicating the answer, which is a target variable, is assigned to a sentence that connects the question and the answer, which are explanatory variables.

[0050] The control unit 11 generates a learning model 121 that outputs an answer when a question sentence is input based on the training data (S12). Specifically, the control unit 11 inputs a sentence that combines a question sentence and an answer included in the training data as input data to the learning model 121, and obtains an answer output from the learning model 121. The control unit 11 adjusts various parameters using, for example, an error backpropagation method so as to optimize (minimize or maximize) the error between the output answer and the answer included in the training data, i.e., the answer that is the correct answer value. Before the start of learning or fine tuning, the definition information that describes the learning model 121 is assumed to be given an initial setting value or a value that has been previously learned. When the learning is completed by the error and the number of learning times satisfying a predetermined standard, optimized parameters are obtained.

[0051] When the learning is completed, the control unit 11 stores definition information related to the learning model 121 in the storage unit 12 as the learned learning model 121, and ends the series of processes. When fine-tuning of the learning model 121 is performed using training data according to the user's purpose of creating questions, the control unit 11 may store the generated learning model 121 in association with information related to the training data used and identification information of the user to which the learning model 121 should be applied. The control unit 11 may generate multiple types of learning models 121 and store them in the storage unit 12.

[0052] FIG. 10 is a flowchart showing an example of a procedure for generating test questions executed by the information processing system 100.

[0053] The control unit 21 of the user terminal 2 accepts the test question creation conditions input by the user operating the operation unit 25 (S21). In S21, the control unit 21 displays on the display unit 24 a screen for accepting the creation conditions provided by the information processing device 1 in response to a login process using, for example, account information, and accepts the creation conditions specified by the user using the screen for acceptance.

[0054] Fig. 11 is a schematic diagram showing an example of a reception screen. The reception screen includes a reception field 241 that receives the specification of creation conditions. In the example shown in Fig. 11, the reception field 241 is configured to be able to receive question creation conditions, answer creation conditions, and question creation patterns. The information processing device 1 displays each of the question creation conditions and answer creation conditions that are set in advance, and question creation patterns in a selectable manner in the reception field 241. The question creation patterns include, for example, automatic generation and manual generation of question sentences.

[0055] The reception field 241 can also receive detailed conditions such as the number of answers when the question generation condition is fill-in-the-blank, and the number of options when the answer generation condition is multiple choice. The reception field 241 may further be configured to receive the use of existing questions or the generation of new questions when the generation pattern is automatically generated. The user inputs the desired selection of generation conditions using the operation unit 25. The control unit 21 receives the designation of the generation conditions through the operation unit 25.

[0056] The creation conditions may be accepted by using a template. Fig. 12 is a schematic diagram showing an example of a template. For example, when a check box indicating "select from template" is selected on the acceptance screen in Fig. 11, the information processing device 1 displays a screen including the template as shown in Fig. 12 through the user terminal 2.

[0057] A plurality of templates are prepared according to combinations of preset question creation conditions and answer creation conditions. Fig. 12 shows an example of a template corresponding to the creation conditions of a fill-in-the-blank format with three answers and a multiple-choice format with four options. The user specifies the creation conditions by selecting a template according to the creation conditions. As shown in Fig. 12, for example, the layout pattern of correct and incorrect answers in each option can also be preset using the template.

[0058] Returning to Fig. 10, the description will continue. The control unit 21 transmits the accepted creation conditions to the information processing device 1 (S22).

[0059] The control unit 11 of the information processing device 1 receives the creation conditions (S23). The control unit 11 may receive existing questions or reference sentences and questions according to the user's purpose for creating questions, in association with the creation conditions.

[0060] The control unit 11 generates test questions including question sentences and answers using the learning model 121 based on the received creation conditions (S24). In detail, the control unit 11 inputs predetermined input data according to the creation conditions to the learning model 121, and obtains output data output from the learning model 121. The control unit 11 generates question sentences and answers according to the predetermined creation conditions by executing predetermined processing based on the output results from the learning model 121 as necessary.

[0061] 13 is a flowchart showing an example of a detailed procedure for generating test questions. The processing procedure shown in the flowcharts of FIG. 13 to FIG. 16 corresponds to the details of S24 in the flowchart of FIG.

[0062] The control unit 11 refers to the question group data and acquires existing questions corresponding to the received creation conditions (S201). The control unit 11 may receive existing questions corresponding to the user's question creation purpose transmitted from the user terminal 2. The control unit 11 inputs the acquired existing questions into the learning model 121 of the first example (S202), and acquires new questions output from the learning model 121 (S203). The test questions are generated by the above-mentioned process. The control unit 11 returns the process to step S25 in the flowchart of FIG. 10.

[0063] When multiple learning models 121 are stored in the memory unit 12, the control unit 11 may identify a learning model 121 according to at least one of the logged-in user and the creation conditions, and perform question generation using the identified learning model 121.

[0064] Furthermore, the control unit 11 may execute post-processing based on the output result from the learning model 121 according to the creation conditions, and generate question sentences and answers according to the predetermined creation conditions. For example, the control unit 11 complements the question sentence by generating a predetermined number of options by replacing one or more answers with incorrect answers based on the answers output from the learning model 121. In this case, the control unit 11 may generate a combination of multiple patterns of question sentences and options (answers).

[0065] FIG. 14 is a flowchart showing another example of a detailed procedure for generating test questions. The control unit 11 acquires a reference sentence and a question content according to the user's purpose of creating a question (S211) via, for example, the user terminal 2. The control unit 11 inputs the acquired reference sentence and question content to the learning model 121 of the second example (S212), and acquires a question sentence and an answer output from the learning model 121 (S213).

[0066] FIG. 15 is a flowchart showing another example of a detailed procedure for generating test questions. The control unit 11 acquires a sentence corresponding to the subject for which the test question is to be created, for example, through the user terminal 2 (S221). The sentence corresponding to the subject for which the test question is to be created is, for example, a reference sentence corresponding to the user's purpose for creating questions, or a sentence extracted from a textbook, a reference book, etc. The control unit 11 generates a question sentence by converting the acquired sentence into a question format, for example, using a predetermined natural language processing model (S222).

[0067] The control unit 11 inputs the generated question text to the learning model 121 of the third example (S223), and obtains the answer output from the learning model 121 (S224). Through the above-mentioned process, a question text that serves as a problem text and an answer to the question text are generated.

[0068] FIG. 16 is a flowchart showing another example of a detailed procedure for generating test questions. The control unit 11 acquires sentences corresponding to the subject for which the test questions are to be created (S231), for example, through the user terminal 2. The sentences corresponding to the subject for which the test questions are to be created are, for example, a series of sentences extracted from textbooks, reference books, etc., corresponding to the user's purpose for creating questions.

[0069] The control unit 11 inputs the acquired sentence into the learning model 121 of the fourth example (S232), and acquires a summary sentence output from the learning model 121 (S233). The control unit 11 generates a question sentence and an answer based on the acquired summary sentence (S234). The control unit 11 generates, for example, a fill-in-the-blank type question sentence in which some words in the summary sentence are masked according to a predetermined rule, and an answer in which words corresponding to the masked parts are extracted.

[0070] Returning to Fig. 10, the description will continue. The control unit 11 transmits the generated test questions to the user terminal 2 corresponding to the logged-in user (S25).

[0071] The control unit 21 of the user terminal 2 receives the test questions (S26) and displays a confirmation screen including the received test questions on the display unit 24 (S27). The user can confirm the test questions through the confirmation screen and decide which test questions to adopt as test questions to be asked.

[0072] Fig. 17 is a schematic diagram showing an example of a confirmation screen. In the example shown in Fig. 17, the confirmation screen includes test questions including the generated question sentences and answers, a reset button 242 for accepting a request to change the options, and a save button 243 for accepting a request to save the generated test questions.

[0073] The information processing device 1 displays the question sentence and the answer including the options through the confirmation screen. When the information processing device 1 receives a tap operation of the reset button 242 through the user terminal 2, it displays a new option with the content of the option changed on the confirmation screen. The content of the option to be changed is, for example, the content of the incorrect answer included in the option, the combination or arrangement of correct answers and incorrect answers in the option, etc. The information processing device 1 receives the selection of any combination of question sentence and option desired by the user from among a plurality of combinations of question sentences and options using the reset button 242. The information processing device 1 may receive a change to the question sentence itself through the confirmation screen. In this case, the information processing device 1 generates a new combination of question sentence and answer using, for example, different input data, and displays it on the confirmation screen.

[0074] When the control unit 21 receives a tap operation of the save button 243 by the user while the selected option pattern is displayed, the control unit 21 receives a save request for the combination of the question sentence and the answer that is being displayed. By receiving the save request, the selected content of the test question is confirmed.

[0075] Returning to Fig. 10, the description will continue. The control unit 21 accepts a save request, that is, a confirmed selection of a test question (S28), and transmits the accepted selection information to the information processing device 1 (S29).

[0076] The control unit 11 of the information processing device 1 receives the selection information (S30). The control unit 11 stores the selected combination of question sentence and answer as one of the test questions to be asked in the storage unit 12 (S31). The control unit 11 transmits the confirmed test question to the user terminal 2 in a predetermined data format (S32).

[0077] The control unit 21 of the user terminal 2 receives the test questions (S33) and ends the series of processes. The information processing system 100 may repeatedly execute the processes from S21 to S31 until a predetermined number of questions is reached.

[0078] In the above process, the control unit 11 is not limited to generating questions at the timing of receiving a question creation request from the user, but may generate a large number of test questions in advance according to the user's question creation purpose using the learning model 121 and store them in the storage unit 12. When the control unit 11 receives creation conditions from the user, it extracts test questions according to the received creation conditions from among the generated test questions.

[0079] The test questions may be configured so that they can be partially or entirely generated manually by the user. For example, the user uses the template shown in FIG. 12 to generate the question text and specify answer phrases for fill-in-the-blank test questions. The user uses the operation unit 25 to input any sentence that corresponds to the question text in a predetermined field of the template and to specify the fill-in-the-blank portion, i.e., to select the answer phrase. The information processing device 1 acquires the test question input by the user through the user terminal 2.

[0080] The information processing device 1 generates a question text by masking a part of the question text based on the acquired input contents. The information processing device 1 may generate a plurality of incorrect answers based on the words and phrases that are the answers, and generate a plurality of options by combining the answers and the incorrect answers.

[0081] Also, the test questions may be generated by modifying a part of an existing test question. For example, the information processing device 1 may generate new test questions by reading existing question sentences and answers according to the creation conditions from the question group data, and modifying the order of options included in the existing question sentences or the arrangement of correct and incorrect answers in the options according to a predetermined rule. This allows new test questions to be generated efficiently by utilizing existing test questions.

[0082] According to this embodiment, the learning model 121 can be used to automatically generate test questions according to various creation conditions received from the user, which can improve the efficiency of the user's question creation work and enable questions to be created according to various needs. The learning model 121 can accurately generate test questions that are in line with the characteristics of each test by learning using data according to the user's question creation purpose. The user can customize the test questions as desired by specifying the creation conditions, improving convenience.

[0083] Second embodiment In the second embodiment, a configuration for accepting reviews and corrections to the generated test questions will be described. In the following embodiments, differences from the first embodiment will be mainly described, and the same reference numerals will be used to designate the same components as the first embodiment, and detailed descriptions thereof will be omitted.

[0084] FIG. 18 is a flowchart showing an example of a review request processing procedure executed by the information processing device 1 of the second embodiment.

[0085] After generating the test questions, the control unit 11 of the information processing device 1 acquires reviewer identification information for multiple reviewers and a review order for each reviewer, for example, through the user terminal 2 of the user who created the questions (S41). The reviewer identification information is information for identifying a reviewer who is requested to review the test questions, and includes, for example, a reviewer name, a reviewer ID, etc.

[0086] The user terminal 2 may, for example, display a review setting screen provided by the information processing device 1 after creating a question on the display unit 24, and accept the reviewer identification information and review order designation from the user using the review setting screen. The user terminal 2 transmits the accepted reviewer identification information and review order to the information processing device 1.

[0087] The control unit 11 outputs a review request to the user terminal 2 of the reviewer identified by the acquired reviewer identification information in accordance with the acquired review order (S42). The reviewer who received the review request uses the user terminal 2 to check the review screen provided by the information processing device 1, for example, and reviews the generated test questions. The user terminal 2 transmits the review results received from the reviewer to the information processing device 1.

[0088] The control unit 11 acquires the review results transmitted from the user terminal 2 (S43). Based on the acquired review results, the control unit 11 stores in the memory unit 12 a review history including, for example, reviewer identification information of the reviewer who completed the review, the review date and time, etc., in association with the test question to be reviewed (S44).

[0089] The control unit 11 judges whether or not to end the review request (S45). If it is judged that the review request is not to end because the review request has not been output to all of the specified reviewers (S45: NO), the control unit 11 returns the process to S42 and outputs the review request to the next reviewer according to the review order. If it is judged that the review request is to end because the review request has been output to all of the specified reviewers (S45: YES), the control unit 11 ends the series of processes.

[0090] During the above-mentioned review stage or at an appropriate timing independent of the review, the information processing device 1 can accept corrections to the generated test questions from each reviewer or question creator. Fig. 19 is a flowchart showing an example of a correction acceptance processing procedure executed by the information processing device 1.

[0091] The control unit 11 of the information processing device 1 accepts corrections to the test questions through the user terminal 2 (S51). The control unit 11 stores a correction history including the content of the accepted correction, the person who made the correction, the date and time of the correction, etc., in the storage unit 12 in association with the test question to be corrected (S52).

[0092] After accepting the corrections, the control unit 11 accepts a display request for the exam question from, for example, the user terminal 2, and outputs the exam question to the user terminal 2 in association with the correction history (S53).

[0093] Fig. 20 is a schematic diagram showing an example of a screen including a correction history. After generating the test questions, the information processing device 1 can generate the screen shown in Fig. 20 in response to a display request transmitted from the user terminal 2 at any timing, and output the screen to the user terminal 2.

[0094] 20, the screen including the correction history includes, for each test question, a test question display section 244 and a correction history display section 245. The information processing device 1 displays the generated test questions on the test question display section 244. The information processing device 1 also reads out the correction history stored in association with the test question from the storage section 12, and displays it on the correction history display section 245 in chronological order for each correction unit.

[0095] As shown in the upper part of FIG. 20, the information processing device 1 may display detailed review information including the progress of the review, details of the reviewer, review results, etc., on the screen based on the review history stored in the storage unit 12.

[0096] When a user desires to make corrections to a test question, the user inputs the corrections using the operation unit 25. When the information processing device 1 accepts the corrections via the user terminal 2, it causes the corrected test question to be displayed on the test question display unit 244, and also causes the new correction history to be displayed on the correction history display unit 245.

[0097] The user can revert (re-correct) to the state before the selected correction history by selecting any of the correction histories in the correction history display section 245. When the information processing device 1 receives an instruction to re-correct to the state before a specific correction history via the user terminal 2, it executes re-correction to undo all corrections made up to that specific correction history. By re-correcting, the content of the test question can be reverted to a specific version. Based on the information displayed on the screen, the user can clearly understand the correction history and can easily and efficiently execute new corrections.

[0098] The correction may be configured to be executable for each question in a series of questions including a plurality of questions. The correction history is not limited to being displayed for each test question, and the correction history for a series of questions may be displayed collectively.

[0099] According to this embodiment, the generated test questions can be efficiently reviewed among multiple reviewers. By displaying the review and correction history in association with the test questions, information related to the creation of test questions can be easily and reliably grasped. The test question information processing system can favorably support not only question creation but also question management.

[0100] Third embodiment In the third embodiment, a configuration for providing analytical information for generated test questions will be described. Fig. 21 is a flowchart showing an example of an analytical information generation process executed by the information processing device 1 of the third embodiment.

[0101] The control unit 11 of the information processing device 1 acquires the test results for the generated test questions (S61). The control unit 11 generates an analysis screen including analysis information for the test questions based on the acquired test results (S62). The analysis information includes, for example, the correct answer rate for the test questions and advice according to the correct answer rate. The control unit 11 outputs the generated analysis screen to the user terminal 2 (S63), and ends the series of processes.

[0102] 22 is a schematic diagram showing an example of a screen including analysis information. The user terminal 2 displays the analysis screen shown in FIG.

[0103] The analysis screen includes, for example, a correct answer rate display unit 246 that displays the correct answer rate of the test question, and an advice display unit 247 that displays advice. The information processing device 1 causes the correct answer rate of each test question calculated based on the test result to be displayed on the correct answer rate display unit 246 in the form of, for example, a graph. The information processing device 1 also identifies the cause of the deviation of the correct answer rate by comparing the calculated correct answer rate with a target correct answer rate set in advance, and generates advice for bringing the correct answer rate closer to the target correct answer rate. The cause of the deviation may be identified based on, for example, the difficulty level of the question text, the length of the question text, the number of options, etc. The cause of the deviation may be identified based on the tendency of the user's test questions taking into account past test questions related to the same user. The information processing device 1 stores, for example, an advice table in which the cause of the deviation and advice are associated in the storage unit 12 in advance, and generates advice according to the identified cause of the deviation by referring to the advice table, and causes the advice display unit 247 to display it.

[0104] According to this embodiment, by providing analysis information on the generated test questions, it is possible to favorably support the generation of high-quality test questions that better meet the needs of users.

[0105] The embodiments disclosed herein are illustrative in all respects and should not be considered restrictive. The technical features described in each embodiment can be combined with each other, and the scope of the present invention is intended to include all modifications within the scope of the claims and equivalents to the scope of the claims. The sequences shown in each embodiment are not limited, and each processing procedure may be executed in a different order within the scope of no contradiction, and multiple processes may be executed in parallel. The subject of each process is not limited, and the process of each device may be executed by another device within the scope of no contradiction.

[0106] The matters described in each embodiment can be combined with each other. In addition, the independent claims and dependent claims described in the claims can be combined with each other in any and all combinations regardless of the citation format. Furthermore, the claims use a format in which a claim cites two or more other claims (multiple claim format), but this is not limited to this. A multiple claim (multi-multi claim) that cites at least one multiple claim may also be used. [Explanation of symbols]

[0107] 100 Information Processing Systems 1. Information processing device 2. User terminal 11, 21 Control section 12, 22 Storage section 13, 23 Communications Department 24 Display section 25 Control section 1P Program 121 Learning Model 1A Recording medium

Claims

1. Accepts test question creation conditions from the user, A learning model outputs new questions and answers using existing questions and answers, and generates questions and answers by inputting existing questions and answers whose creation conditions correspond to the creation conditions received; The generated question and answer are output to the user. A program that causes a computer to execute a process.

2. As the creation conditions, question creation conditions including fill-in-the-blank or question-asking and answer creation conditions including multiple choice or written answers are accepted. The program according to claim 1.

3. When a multiple-choice question is received as the answer creation condition, the answer and an incorrect answer similar to the answer are generated. The program according to claim 2.

4. Generate a plurality of candidate combinations of the question sentence and answer; A selection of any one of the plurality of generated candidate question sentence and answer combinations is accepted. The program according to claim 1 or 2.

5. Acquire an existing question sentence or a modified question sentence obtained by modifying a part of the existing question sentence; The acquired existing question sentence or the modified question sentence is output together with the generated question sentence. The program according to claim 1 or 2.

6. Obtaining a plurality of reviewers and a review order for the generated question sentence and answer; Send review requests to each reviewer according to the obtained review order The program according to claim 1 or 2.

7. When a correction to the generated question sentence or answer is accepted, a correction history is stored in association with the question sentence or answer. The program according to claim 1 or 2.

8. Accepts test question creation conditions from the user, A learning model outputs new questions and answers using existing questions and answers, and generates questions and answers by inputting existing questions and answers whose creation conditions correspond to the creation conditions received; The generated question and answer are output to the user. An information processing method in which processing is performed by a computer.

9. Accepts test question creation conditions from the user, A learning model outputs new questions and answers using existing questions and answers, and generates questions and answers by inputting existing questions and answers whose creation conditions correspond to the creation conditions received; The generated question and answer are output to the user. Equipped with a control unit that executes processing Information processing device.

10. Acquire conditions for creating test questions and text to be used for creating test questions received from a user, Acquire a question sentence by converting the acquired sentence into a question sentence format according to the creation conditions; generating an answer by inputting the acquired question text into a learning model that uses the question text to output an answer to the question text; Outputting the question and the answer to the user A program that causes a computer to execute a process.

11. Acquire conditions for creating test questions and text to be used for creating test questions received from a user, Acquire a question sentence by converting the acquired sentence into a question sentence format according to the creation conditions; generating an answer by inputting the acquired question text into a learning model that uses the question text to output an answer to the question text; Outputting the question and the answer to the user An information processing method in which processing is performed by a computer.

12. Acquire conditions for creating test questions and text to be used for creating test questions received from a user, Acquire a question sentence by converting the acquired sentence into a question sentence format according to the creation conditions; generating an answer by inputting the acquired question text into a learning model that uses the question text to output an answer to the question text; Outputting the question and the answer to the user Equipped with a control unit that executes processing Information processing device.

13. Acquire conditions for creating test questions and text to be used for creating test questions received from a user, generating a summary by inputting the received sentence into a learning model that uses the sentence to output a summary of the sentence; Using the generated summary, a question sentence and an answer are generated according to the generation conditions in accordance with a predetermined rule; The generated question and answer are output to the user. A program that causes a computer to execute a process.

14. Acquiring test question creation conditions and sentences to be used in creating test questions received from a user, inputting the received sentence into a learning model that uses the sentence to output a summary of the sentence, and outputting the summary; Using the output summary, a question sentence and an answer are generated according to the creation conditions in accordance with a predetermined rule; The generated question and answer are output to the user. An information processing method in which processing is performed by a computer.

15. Acquiring test question creation conditions and sentences to be used in creating test questions received from a user, inputting the received sentence into a learning model that uses the sentence to output a summary of the sentence, and outputting the summary; Using the output summary, a question sentence and an answer are generated according to the creation conditions in accordance with a predetermined rule; The generated question and answer are output to the user. Equipped with a control unit that executes processing Information processing device.

Images