Information processing method, program, and information processing apparatus

Abstract

The present invention provides an information processing method, etc., that can suitably design dialogue tasks for language learners. [Solution] The information processing method involves a computer that acquires learning information representing the content that a learner of a target language should learn, and inputs a first prompt containing the learning information into a language model to generate a specification for a dialogue task to be performed by the learner. Preferably, a second prompt containing the generated specification is input into the language model to generate an instruction document that instructs the learner to perform the dialogue task, and a third prompt that instructs the language model to generate a dialogue sentence corresponding to the dialogue task.

Description

Technical Field

[0001] The present invention relates to an information processing method, a program, and an information processing apparatus.

Background Art

[0002] There is a technology for assisting language learning. For example, in Patent Document 1, a method of generating a reading passage or the like in a language test using a transformer-based language model is disclosed.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] [[ID=�4]] In one aspect, an object is to provide an information processing method or the like that can suitably devise a dialogue task to be given to a language learner.

Means for Solving the Problems

[0005] The information processing method includes a computer executing a process of acquiring learning information representing what a learner who learns a target language should learn, and inputting a first prompt including the learning information to a language model to generate a specification of a dialogue task to be given to the learner.

Effects of the Invention

[0006] In one aspect, a dialogue task to be given to a language learner can be suitably devised. [[ID=5`1]]

Brief Description of the Drawings

[0007] [Figure 1] It is a diagram showing a configuration example of a language learning system. [Figure 2] This is a block diagram showing an example server configuration. [Figure 3] This is a diagram illustrating the outline of the embodiment. [Figure 4] This is an explanatory diagram regarding the process of identifying learning items. [Figure 5] This figure shows an example of a screen for generating dialogue tasks. [Figure 6] This figure shows another example of the dialogue task generation screen. [Figure 7] This figure shows an example of the instruction sheet and third prompt generation screen. [Figure 8] This flowchart shows the procedure for identifying learning items. [Figure 9] This is a flowchart showing the steps involved in generating dialogue tasks. [Figure 10] This is a flowchart showing the steps for the dialogue processing process. [Modes for carrying out the invention]

[0008] The present invention will be described in detail below with reference to the drawings illustrating its embodiments. (Embodiment) Figure 1 shows an example of the configuration of a language learning system. This embodiment describes a language learning system in which learners learn a language by interacting with the system (chatbot) in a target language (e.g., English). The language learning system includes an information processing device 1, a learner terminal 2, and a creator terminal 3. Each device is connected via a network N such as the Internet.

[0009] Information processing device 1 is an information processing device capable of various information processing and information transmission and reception, such as a server computer or a personal computer. In this embodiment, information processing device 1 is assumed to be a server computer, and for simplicity, it will be referred to as server 1 below. Server 1 functions as a chatbot system that conducts dialogue with learners, and as described later, it generates dialogue sentences using a Large Language Model (LLM, hereafter referred to as "LLM" for simplicity), outputs them as voice, and accepts voice input of response sentences from learners.

[0010] Although this embodiment is described as a system for voice-based dialogue (English conversation), this embodiment may also be applied to a system for text-based dialogue (reading and writing).

[0011] In this embodiment, Server 1 semi-automatically generates dialogue tasks to be performed between the learner and the LLM, and causes the learner to perform dialogues in accordance with those dialogue tasks. Specifically, as will be described later, Server 1 generates a specification document for the dialogue task (see Table 1 below) by inputting a predetermined prompt (first prompt) into the LLM, and generates an instruction document (see Table 2) that instructs the learner to perform the dialogue task and a prompt (third prompt, see Table 3) that instructs the generation of dialogue sentences according to the dialogue task by inputting the generated prompts into the LLM, and conducts a dialogue with the learner.

[0012] Learner terminal 2 is a terminal device used by the learner, such as a personal computer, smartphone, or tablet. Server 1 outputs the dialogue sentences generated using LLM to learner terminal 2 for audio playback, and receives voice input of response sentences from the learner via learner terminal 2.

[0013] The author terminal 3 is a terminal device used by a content creator who creates interactive content, such as a personal computer or the like. The author terminal 3 displays a creation screen for interactive content (see FIGS. 5 to 7) described later and accepts setting input of necessary information from the content creator. The server 1 generates a specification document for an interactive task, an instruction document for a learner, a third prompt, etc. based on the setting content of the screen.

[0014] FIG. 2 is a block diagram showing a configuration example of the server 1. The server 1 includes a control unit 11, a main memory unit 12, a communication unit 13, and an auxiliary storage unit 14. The control unit 11 has an arithmetic processing device such as one or more CPUs (Central Processing Units), MPUs (Micro-Processing Units), GPUs (Graphics Processing Units), etc., and reads and executes the program P stored in the auxiliary storage unit 14 to perform various information processing, control processing, etc. The main memory unit 12 is a temporary storage area such as SRAM (Static Random Access Memory), DRAM (Dynamic Random Access Memory), etc., and temporarily stores data necessary for the control unit 11 to execute arithmetic processing. The communication unit 13 is a communication module for performing communication-related processing and transmits and receives information to and from the outside. The auxiliary storage unit 14 is a non-volatile storage area such as a hard disk, a large-capacity memory, etc., and stores the program P (program product) and other data necessary for the control unit 11 to execute processing.

[0015] Note that the auxiliary storage unit 14 may be an external storage device connected to the server 1. Also, the server 1 may be a multi-computer composed of a plurality of computers, or may be a virtual machine virtually constructed by software.

[0016] In addition, in this embodiment, the server 1 is not limited to the above configuration, and may include, for example, an input unit that receives operation inputs, a display unit that displays images, and the like. Further, the server 1 may include a reading unit that reads a portable storage medium 1a such as a CD (Compact Disk)-ROM or a DVD (Digital Versatile Disc)-ROM, and reads and executes the program P from the portable storage medium 1a.

[0017] FIG. 3 is a diagram showing an overview of the embodiment. Based on FIG. 3, the overview of this embodiment will be described.

[0018] By executing the program P, the control unit 11 of the server 1 functions as a language ability measurement unit 31, a needs collection unit 32, a profile generation unit 33, a learning item identification unit 34, a task generation unit 35, a task quality evaluation unit 36, a task recommendation unit 37, a task execution unit 38, an achievement determination unit 39, a recommendation model update unit 40, and an achievement output unit 41. Further, the server 1 stores a learner DB 301, a learning item DB 302, a topic DB 303, a task DB 304, and a recommendation model 305 in the auxiliary storage unit 14.

[0019] Based on the dialogue data that is irregularly conducted with the learner, the language ability measurement unit 31 measures the proficiency (language ability) of the learner in the target language. The proficiency is data that represents the learner's language ability as discrete and / or continuous values. In this embodiment, the CEFR (Common European Framework of Reference for Languages) level, which is evaluated in six levels of A1, A2, B1, B2, C1, and C2, is used. In CEFR, A1 has the lowest level and C2 has the highest level. For example, the language ability measurement unit 3 measuring the learner's proficiency (determination) by analyzing the learner's response (dialogue data) to the utterance from the system side using a machine learning model. The measured proficiency is stored in the learner DB 301.

[0020] Learner DB301 is a database that stores learner information. In addition to the language proficiency measurement results described above, Learner DB301 also stores the following: needs collection results (learner profile, target learning items, etc.), dialogue learning results (recommendation tasks, selection tasks, whether learning items were achieved, etc.), and the achievement status of dialogue tasks (history of achievement or failure of all tasks performed in the past, achievement or failure status of target learning items, etc.).

[0021] The needs collection unit 32 collects the learner's learning needs for the target language by conducting irregular dialogues with the learner. The profile generation unit 33 generates a learner profile from the learning needs (dialogue data with the learner) collected by the needs collection unit 32. The learning item identification unit 34 identifies the learning items (e.g., Can-Do statements) that the learner should aim for, based on the profile generated by the profile generation unit 33 and the learner's proficiency in the target language measured by the language ability measurement unit 31.

[0022] Figure 4 is an explanatory diagram regarding the process of identifying learning items. Based on Figure 4, the processes performed by the needs collection unit 32, the profile generation unit 33, and the learning item identification unit 34 will be explained.

[0023] Server 1 outputs predetermined questions about the learner to collect the learner's learning needs for the target language, and accepts answers from the learner. The table in the upper part of Figure 4 illustrates the question text from the system (Server 1). Server 1 outputs a set of predetermined questions to the learner terminal 2 and accepts answers to each question.

[0024] Next, Server 1 generates a learner profile based on the input responses. The table in the middle of Figure 4 illustrates an example of a learner profile. For example, Server 1 inputs the set of responses from the learner into the LLM and generates the learner profile by summarizing the set of responses.

[0025] Next, in addition to the generated profile, Server 1 identifies learning items that the learner should aim for, based on the learner's proficiency in the target language. Learning items are measures that represent what can be done using the target language, and in this embodiment, Can-Do statements are used as learning items. For example, the learning item DB302 defines Can-Do statements that a person with a certain level of proficiency can perform, corresponding to a level of proficiency (CEFR level). Server 1 refers to the learning item DB302 to identify learning items that the learner should aim for. For example, Server 1 identifies Can-Do statements as target learning items that are close to the learner's proficiency level and whose textual features are similar to the learner's profile. Alternatively, Server 1 may extract keywords from the learner's profile (for example, "hip-hop" in the example in Figure 4), identify topics that the learner might be interested in (for example, "music") from those keywords, and identify learning items related to language acts in relation to those topics. The table in the lower part of Figure 4 illustrates an example of identified learning items. Server 1 identifies one or more learning items from the learner's profile, proficiency level, etc.

[0026] In this embodiment, Can-Do statements are used as learning items, but as will be described later, grammar items, vocabulary items, and other language items may also be used as learning items, and these language items may be integrated with Can-Do statements in the implementation.

[0027] Returning to Figure 3, let's continue the explanation. Processing by the needs collection unit 32, profile generation unit 33, and learning item identification unit 34 is performed periodically, and the learner's learning plan (target learning items) is updated. Needs are collected, for example, when the percentage of learning items achieved among the currently targeted learning items exceeds a threshold (e.g., 90%). Alternatively, as described later, when a dialogue task is recommended by the system, needs are collected if the learner selects "does not apply to my needs" a certain number of times or more. Alternatively, needs are collected when the learner becomes aware that their needs have changed and wishes to change their learning items. Alternatively, a machine learning model may be prepared to estimate the optimal timing for needs collection.

[0028] As described above, Server 1 generates a learner profile from the answers to the questions and identifies the learning items to be targeted. Server 1 stores one or more of the identified learning items (target learning item list), the generated profile, and the original dialogue data (questions and answers) in the learner DB 301.

[0029] For example, Server 1 may receive input from the learner to select the learning item they wish to achieve from among the learning items identified above, and generate an interactive task for the selected learning item.

[0030] The task generation unit 35 generates dialogue tasks for the learner to perform. A dialogue task is a task assigned to the learner in order to achieve learning objectives, and is generated for the purpose of evaluating the learner's language ability, assessing the level of achievement described later, achieving the above-mentioned Can-Do statements, and practicing vocabulary and grammar. Specifically, a dialogue task is, for example, a role-playing exercise in which the learner is given a specific dialogue situation (for example, discussing a hypothetical sports match). The task generation unit 35 creates a first prompt containing learning information that represents the content the learner should learn, and generates a specification for the dialogue task by inputting this first prompt into the LLM. The task generation unit 35 then creates a second prompt containing the generated specification, and generates an instruction document that instructs the learner to perform the dialogue task, and a third prompt that instructs the LLM to generate dialogue sentences corresponding to the dialogue task, by inputting this second prompt into the LLM.

[0031] Figure 5 shows an example of a dialogue task generation screen. Server 1 outputs the screen shown in Figure 5 to the creator terminal 3 for display. This screen is used to create the first prompt and generate the dialogue task specification. The dialogue task generation screen includes an LLM settings field 51, a learner information settings field 52, a learning item settings field 53, and a generate button 54.

[0032] The LLM setting field 51 is a setting field for setting the type of LLM to be used and the temperature parameter (Temperature) for setting the variation in the LLM output. The creator terminal 3 accepts input for the type of LLM to be used and the temperature parameter in the LLM setting field 51.

[0033] The learner information setting field 52 is a setting field for setting learner information about the target learner. The creator terminal 3 accepts input in the learner information setting field 52 for the target age group of the learner, the target level of proficiency in the target language the learner aims to achieve, and the language acts that the learner aims to achieve. Note that the proficiency level may be set by default to the learner's current proficiency level (or a higher proficiency level than the current proficiency level) measured in advance.

[0034] The learning item setting field 53 is a setting field where learners set learning items they should aim for. The creator terminal 3 accepts input in the learning item setting field 53 for one or more learning items (Can-Do statements) that the learner should aim for. Note that the learning items may be set by default to learning items that have been previously identified from the learner's profile, etc. In addition, the learner terminal 3 accepts input for settings such as the power dynamics (hierarchical relationship) and sense of distance between the learner and their conversation partner (LLM).

[0035] In addition, learner terminal 3 can accept various settings as options (for example, settings for dialogue topics).

[0036] Server 1 creates a first prompt that instructs the LLM to generate a specification document for the dialogue task by applying the above learner information, learning items (learning information), etc., to a predetermined prompt template. When input is received from the generate button 54, Server 1 inputs the first prompt to the LLM, thereby generating a specification document for the dialogue task to be performed between the learner and the LLM. Table 1 shows an example of a specification document.

[0037] [Table 1]

[0038] In Table 1, "Title" is the task name. "Non-linguistic outcome" is the result to be achieved by performing the dialogue task. "Learning objectives as in Can-Do statements" are the learning objectives to be achieved in the dialogue task. "The learner's role" and "The tutor's role" are the roles that the learner and the dialogue partner (LLM) should play. "Situational lead-in presented to the user," "Task instruction presented to the user," and "Additional information to the user" are the dialogue situation, task content, and other additional information presented to the learner (User). "Situational lead-in presented to the tutor," "Task instruction presented to the tutor," "Additional information to the tutor," and "Interactional strategies that the tutor should use" are the dialogue situation, task content, other additional information presented to the LLM (Tutor), and the dialogue methods (behaviors) that the LLM should use.

[0039] In this way, Server 1 generates a specification document that includes the task name of the dialogue task, target learning items, situation setting, and instructions for the learner and LLM. As will be described later, Server 1 generates instructions for the learner and a third prompt (instructions for the LLM) to be entered into the LLM by inputting a second prompt containing the specification document into the LLM.

[0040] Figure 6 shows another example of a dialogue task generation screen. In the generation screen of Figure 5, the content creator creates the first prompt by setting the learning items, but the generation screen shown in Figure 6 illustrates a screen that creates the first prompt by setting the dialogue situation, for content creators who do not understand the concept of Can-Do statements.

[0041] This screen includes an LLM settings field 61, a learner information settings field 62, a dialogue status settings field 63, and a generate button 64. The LLM settings field 61 and the learner information settings field 62 are the same as the LLM settings field 51 and the learner information settings field 52 described in Figure 5, so their explanation is omitted.

[0042] The dialogue status setting field 63 is a setting field for setting the dialogue status to be imposed on the learner and LLM. The creator terminal 3 accepts setting inputs in the dialogue status setting field 63, such as the goal to be achieved by the dialogue task, the dialogue topic, the type of dialogue event, and the roles to be played by the learner and LLM. For setting the dialogue topic, the topic DB 303 (see Figure 3), which defines multiple dialogue topics, may be referenced. When input is received to the generate button 64, the server 1 creates a first prompt according to the set dialogue status and inputs the first prompt to the LLM to generate the dialogue task specification document exemplified in Table 1.

[0043] Thus, Server 1 may create the first prompt by accepting the setting of the dialogue state, rather than setting the learning items.

[0044] Server 1 may also create a first prompt based on linguistic information such as grammatical and vocabulary items that the learner should learn. Although the illustration of the screen example is omitted, for example, creator terminal 3 accepts input to select items that the target learner should learn from grammatical items narrowed down based on the learner's proficiency level. Creator terminal 3 also accepts input to select a dialogue topic. Based on the selected grammatical items and topic, Server 1 extracts several example dialogues from a large corpus that frequently use those grammatical items and are of a difficulty level close to that of the target learner. Server 1 analyzes the extracted example dialogues from the perspectives of language acts within the dialogue and the roles of each speaker. Then, based on the analyzed language acts and the roles of each speaker, Server 1 creates a first prompt using the method described in Figure 6.

[0045] In addition, Server 1 may create a first prompt based on learner needs information. For example, Server 1 may create a first prompt based on information stored in Learner DB 301, such as the context in which the learner will use the target language, the topics they want to be able to talk about, and the language acts required.

[0046] Furthermore, for example, Server 1 may, based on the learner's learning history, identify the learner's weaknesses, specifically the target learning items (Can-Do statements) to which they have achieved the least. The server may then input these weak learning items into the first prompt with the aim of acquiring those weaknesses. This generates an interactive task for learning the learner's weaknesses.

[0047] Thus, various modifications are possible to the method of creating the first prompt (the content of the learning information).

[0048] Figure 7 shows an example of a screen for generating instructions and a third prompt. In Figure 7, a second prompt is created based on the specifications of the dialogue task generated above, and an example of a screen for generating instructions for the learner and a third prompt that instructs the LLM to generate dialogue sentences is shown. This screen includes a prompt setting field 71, an LLM setting field 72, a generate button 73, and an input information setting field 74.

[0049] The prompt settings field 71 is a settings field for configuring the settings necessary for creating the second prompt. In the prompt settings field 71, the creator terminal 3 accepts inputs such as the file format of the instructions and the third prompt output from the LLM, the specifications of the dialogue task to be used for the second prompt, and the template to be referenced when creating the second prompt.

[0050] LLM setting field 72 is a setting field that accepts settings for the type of LLM to be used to generate the instruction sheet and the third prompt, as well as the temperature parameters.

[0051] The input information setting field 74 is a setting field for configuring the information necessary to instruct the LLM to generate instructions and a third prompt. The creator terminal 3 accepts settings input in the input information setting field 74, such as the instruction text to the LLM, prerequisites, output format for the instructions and third prompt, and specifications for the dialogue task. This information is displayed in its default state, and the content creator can edit it as needed.

[0052] When input is received from the generate button 73, Server 1 creates a second prompt, including the specifications for the dialogue task, based on the settings configured above. Server 1 then inputs the second prompt to the LLM, generating an instruction document that tells the learner to perform the dialogue task, and a third prompt that instructs the LLM to generate dialogue text corresponding to the dialogue task. Examples of the instruction document and the third prompt are shown in Tables 2 and 3.

[0053] [Table 2]

[0054] In Table 2, "Situation" is a description of the setting for the dialogue task. "Task" is the task content instructed to the learner. "Additional information" is additional information related to the dialogue task.

[0055] [Table 3]

[0056] Table 3 shows the following: "Rules" are the basic rules of the dialogue; "Roles" are the roles that the LLM should play; "Context" is the setting for the dialogue; "Additional constraints" are additional constraints imposed on the LLM; and "Output" is the procedure for the LLM's output.

[0057] As shown in Tables 2 and 3, Server 1 generates an instruction sheet to instruct the learner on a dialogue task and a third prompt to cause the LLM to produce output corresponding to the dialogue task. As will be described later, when Server 1 conducts a dialogue, it outputs the instruction sheet exemplified in Table 2 to the learner terminal 2 to instruct the learner on the dialogue task, and simultaneously inputs the third prompt exemplified in Table 3 to the LLM to generate dialogue text corresponding to the dialogue task. This allows the learner to conduct a dialogue with the LLM.

[0058] Returning to Figure 3, the explanation continues. The task generation unit 35 stores the information of the dialogue tasks generated as described above, namely the dialogue task specifications, instructions, and third prompts, in the task DB 304. The task generation unit 35 repeatedly generates dialogue task specifications, instructions, and third prompts using the procedure described above. That is, the task generation unit 35 generates multiple specifications corresponding to multiple dialogue tasks by repeatedly inputting the first prompt into the LLM (multiple times), and generates instructions and third prompts corresponding to multiple dialogue tasks by inputting a second prompt containing each of the multiple specifications into the LLM. As a result, a large number of dialogue tasks are accumulated in the task DB 304.

[0059] In the above example, the content creator generated the dialogue task by setting the necessary information on each screen. However, this embodiment is not limited to this, and Server 1 may automatically set the necessary information to generate the dialogue task fully automatically.

[0060] The task quality evaluation unit 36 ​​evaluates the quality of the generated dialogue tasks. The quality evaluation of the dialogue tasks is performed, for example, by receiving evaluation input from the content creator (such as a teacher who instructs learners).

[0061] In this embodiment, Server 1 has a simulation function that simulates a dialogue between the system (Tutor) and the learner by inputting the above-mentioned instructions and the third prompt into the LLM. Although the illustration of the example screen is omitted, Server 1 generates the system's dialogue text by inputting the third prompt (Table 3) into the LLM, and also simulates the learner's response text to the system's dialogue text by inputting the instructions to the learner (Table 2) into the LLM. The creator terminal 3 displays the system's dialogue text and the simulated learner's response text in a chat format. The content creator checks whether the dialogue is proceeding smoothly by viewing this dialogue data and evaluates the quality of the dialogue task.

[0062] While the above assumes that the quality evaluation of the dialogue task will be performed manually, it is also possible to automatically perform the quality evaluation using machine learning techniques after setting an evaluation function.

[0063] The task recommendation unit 37, when a learner uses this system to conduct a dialogue, extracts a predetermined number (for example, three) of dialogue tasks from the multiple dialogue tasks registered (stored) in the task DB 304 as recommended tasks and presents them to the learner. The task execution unit 38 receives input from the learner to select one of the multiple dialogue tasks presented, presents the learner with instructions for the selected dialogue task, and conducts the dialogue by inputting a third prompt into the LLM.

[0064] First, let's explain the recommendation process for dialogue tasks. Server 1 determines the recommended task based on the learner profile exemplified in Figure 4 and the dialogue task specification exemplified in Table 1. Specifically, Server 1 first inputs the learner profile and the dialogue task specification into a machine learning model such as BERT (Bidirectional Encoder Representations from Transformers) to extract features from the profile and specification, respectively. For convenience, in the following explanation, the features of the learner profile will be called "learner features," and the features of the dialogue task specification will be called "task features."

[0065] Server 1 determines which dialogue task to recommend to the learner based on learner features and task features. Specifically, Server 1 determines the recommended task using a recommendation model 305 consisting of a task completion expectancy prediction model (first model) and a task selection expectancy prediction model (second model).

[0066] The task achievement expectancy prediction model is a machine learning model (e.g., a neural network) that, given learner features and task features as input, outputs a task achievement expectancy indicating whether or not the learner can complete the dialogue task (learning item). As described later, after the learner has performed a dialogue, the achievement determination unit 39 determines whether or not the learner has completed the learning item using a binary value of "1" (completed) or "0" (not completed). The server 1 labels the learner features of the learner who performed the dialogue and the task features of the dialogue task performed with the determination result as "1" or "0", and uses the data labeled with the determination result as training data to train the task achievement expectancy prediction model.

[0067] The task selection expectancy prediction model is a machine learning model (e.g., a neural network) that, given learner features and task features as input, outputs a task selection expectancy indicating whether the learner will select a dialogue task. As described above, Server 1 presents a recommendation task to the learner and accepts input for selecting the dialogue task to be performed. Server 1 labels the learner features of the learner who performed the dialogue and the task features of the dialogue task presented as a recommendation task with a value of "1" (selected) or "0" (not selected) according to the selection result, and uses the data labeled with the selection result as training data to train the task selection expectancy prediction model.

[0068] In this embodiment, assuming the learner's proficiency level (CEFR level) is L_0 (e.g., B1), the server 1 recommends a total of three dialogue tasks: one at the same level as the learner's proficiency level L_0, one at the next lower level L_(-1) (e.g., A2), and one at the next higher level L_(+1) (e.g., B2). The levels (proficiency levels) of the dialogue tasks are defined in the specifications. By recommending tasks one level below and one level above the learner's proficiency level, the learner can challenge themselves with a higher level task and also review the lower level task. Furthermore, even if the language ability measurement unit 31 misjudges the learner's proficiency level by one, the risk of recommending a task outside the learner's proficiency level can be reduced.

[0069] Server 1 determines the recommended task using the task completion expectation prediction model and task selection expectation prediction model described above. However, in the initial stages of operation of this system, since training data has not yet been accumulated, the recommended task is determined using a rule-based method. Specifically, for the first use, Server 1 determines the task with the highest similarity (e.g., cosine similarity) between the learner features and the task features for each of the L_(-1), L_0, and L_(-1) tasks as the recommended task.

[0070] For subsequent uses, especially in the initial stages of operation when sufficient training data has not yet been accumulated, Server 1 operates using the similarity-based method described above, while also collecting data. For example, if Server 1 selects the task with the highest similarity score, it will recommend the same task as the first time. Therefore, it randomly determines the recommended task based on a roulette selection. Note that if the task selected during the first use has not been completed, the same task may be recommended again.

[0071] It is possible that the similarity of the task with the highest similarity, as described above, is below the threshold, and that none of the tasks registered in Task DB304 are suitable for the learner's needs. In this case, Server 1 sends a signal indicating "No matching tasks" to Learner Terminal 2 or Creator Terminal 3, and the learner or content creator recreates the dialogue task. In this case, Server 1 may also automatically generate a dialogue task. Furthermore, even while a new task is being created using the above procedure, tasks below the threshold may be recommended to the learner, and the learner may learn from these. If automatic generation is possible, the above procedure may be performed sequentially to generate the necessary tasks each time.

[0072] As a result of continuing the above operations, once a sufficient amount of training data has been accumulated, Server 1 constructs a task completion expectation prediction model and a task selection expectation prediction model. That is, Server 1 trains a neural network-based model using the accumulated training data. The recommendation model 305 constructed here is a general-purpose model that recommends tasks to new learners (new users) based on the learner features of the learner and taking into account the trends of the training data. As will be described later, Server 1 optimizes this general-purpose recommendation model 305 for individual users.

[0073] Server 1 outputs the expected task completion rate and the expected task selection rate by inputting the learner features of the target learner and the task features of each dialogue task into the task completion rate prediction model and the task selection rate prediction model, respectively. Server 1 calculates the task recommendation rate by calculating the weighted sum of the expected task completion rate and the expected task selection rate, as shown in equation (1) below.

[0074] Task recommendation score = α × expected task completion rate + β × expected task selection rate … (1)

[0075] The coefficients α and β satisfy α + β = 1. Server 1 calculates a task recommendation score for each dialogue task. Server 1 determines the recommended task by sampling from the task recommendation score distribution of the task group. Tasks with higher recommendation scores are more likely to be selected, but the task with the highest recommendation score is not necessarily selected. By introducing randomness into the recommendation process, the model is prevented from overfitting, and the learner is less likely to be recommended the same task repeatedly, thus improving serendipity.

[0076] As described above, Server 1 will operate using Recommendation Model 305 once sufficient training data has been accumulated. It is preferable that a rule-based (similarity-based) algorithm be selected with a certain probability even during the operation phase of Recommendation Model 305. This reduces the risk of Recommendation Model 305 overfitting and recommending only biased samples. As the amount of learner data increases, the proportion of recommendation tasks determined by Recommendation Model 305 will be gradually increased accordingly.

[0077] The recommendation algorithm described above is merely an example, and this embodiment is not limited to it. Here, dialogue tasks related to the learning items targeted by the learner are called main tasks, and dialogue tasks related to other learning items are called subtasks. For example, learners can be classified into short-term, long-term, and balanced types according to the expected language learning period. For short-term learners, the algorithm could prioritize short-term acquisition and increase the proportion of recommended tasks selected from the main task group, while for long-term learners, it could increase the proportion selected from the subtask group. By increasing the likelihood of recommendations from the main task group for short-term learners, tasks can be provided that align with the learner's needs. On the other hand, by increasing the likelihood of recommendations from the subtask group for long-term learners, a variety of dialogue tasks can be practiced. In addition to the above, algorithms could also be considered in which, if there are consecutive failures in tasks from the main task group, similar tasks from the subtask group are recommended as remedial lessons.

[0078] As described above, Server 1 determines a recommended task from a large number of dialogue tasks. Server 1 outputs the multiple (three) recommended dialogue tasks to the learner terminal 2 and presents them to the learner. Server 1 accepts input to select a dialogue task to perform from these multiple dialogue tasks. The recommendation results and selection results of the dialogue tasks are stored in the learner DB 301.

[0079] Server 1 outputs the instructions for the selected dialogue task (Table 2) to learner terminal 2 and displays them. Server 1 also inputs a third prompt corresponding to the selected dialogue task into the LLM, generating a dialogue sentence appropriate to that dialogue task, outputting it to learner terminal 2, and playing it back as audio. In response, Server 1 receives voice input of a response sentence to the dialogue sentence from the learner. Server 1 repeats this process of generating dialogue sentences and receiving response sentences to conduct the dialogue.

[0080] The achievement determination unit 39 determines whether the learner has achieved the target learning item based on the learner's response sentences (and the system's dialogue sentences). Specifically, the achievement determination unit 39 uses a concept called a dialogue act to extract response sentences related to the target learning item, and determines whether the learning item has been achieved from the response sentences in the extracted sections.

[0081] A dialogue act is a speaker's intention to achieve through language (defined as, for example, "Request," "Proposal," or "Suggestion"), and is a component of a Can-Do statement in the CEFR. For example, if a learner utters "Please do...", the dialogue act of that utterance can be determined (classified) as "Request." When a response is input, Server 1 identifies the dialogue act for each part of the response by inputting the learner's response into a machine learning model (third model) that has been trained to identify the dialogue act indicating the intention of each part of the response (for example, each sentence).

[0082] For example, Server 1 has a master list that associates learning items (Can-Do statements) with the corresponding dialogue acts. Server 1 refers to this list to extract the response sentence sections in the dialogue task where the dialogue act corresponding to the target learning item is identified.

[0083] Next, Server 1 determines the learner's level of achievement (proficiency) based on the eloquence of the extracted response sentence. Specifically, Server 1 outputs the proficiency level by inputting the extracted response sentence into a machine learning model (Model 4) that has been trained to output the learner's proficiency level when a response sentence is input. Server 1 compares the outputted proficiency level with the learner's current proficiency level, which was measured in advance. If the outputted proficiency level is equal to or greater than the current proficiency level, Server 1 determines that the learner has achieved the learning item. Conversely, if the outputted proficiency level is less than the current proficiency level, Server 1 determines that the item has not been achieved.

[0084] In this way, Server 1 determines whether the learner has achieved the learning item from the response statement portion where the dialog act related to the target learning item is identified. The achievement status of the learning item is stored in the learner DB 301.

[0085] The recommendation model update unit 40 updates (retrains) the recommendation model 305 for each learner according to the learning results. Specifically, the recommendation model update unit 40 provides the task achievement expectancy prediction model with data labeled with the achievement or non-achievement result of the learning items (dialogue tasks) determined by the achievement determination unit 39, based on the learner features of the target learner and the task features of the dialogue tasks in which the dialogue was performed, as training data for retraining, and updates the task achievement expectancy prediction model. In addition, the recommendation model update unit 40 provides the task selection expectancy prediction model with data labeled with the learner's selection result of the dialogue task, based on the learner features and task features, as training data for retraining, and updates the task selection expectancy prediction model. As a result, each model is individually optimized according to the learner.

[0086] The achievement output unit 41 outputs the achievement determination result of the achievement determination unit 39, which determines whether the dialogue task has been completed or not, to the learner terminal 2 and presents it to the learner.

[0087] Based on the above, this embodiment collects the learner's needs, identifies the learning items to be targeted, and generates a set of dialogue tasks for the learner to perform. Then, it presents a recommended task from the generated set of dialogue tasks, conducts a dialogue, and determines the degree of achievement. This allows learners to perform appropriate dialogue tasks and effectively supports language learning.

[0088] Figure 8 is a flowchart showing the procedure for identifying learning items. Based on Figure 8, the process for identifying learning items that learners should aim for will be explained below.

[0089] The control unit 11 of server 1 outputs predetermined questions about the learner to the learner terminal 2 (step S11). The control unit 11 receives input of answers to the questions from the learner terminal 2 (step S12). The control unit 11 determines whether or not it has received input of answers for all questions (step S13). If it determines that there are questions for which it has not received input of answers (S13: NO), the control unit 11 returns to step S11 and outputs the next question.

[0090] If the control unit determines that it has received input for all questions (S13: YES), it generates a learner profile by summarizing the answers to each question (step S14). In addition to the generated learner profile, the control unit 11 identifies the learning items (Can-Do statements) that the learner should aim for, based on the learner's proficiency in the target language, which has been measured (determined) in advance (step S15). The control unit 11 stores the needs collection results, including the identified learning items, in the learner DB 301 (step S16), and terminates the series of processes.

[0091] Figure 9 is a flowchart showing the procedure for generating a dialogue task. Based on Figure 9, the process of generating a dialogue task using LLM will be explained.

[0092] The control unit 11 of server 1 creates a first prompt to be input into the LLM in order to generate the specifications for the dialogue task (step S31). For example, the control unit 11 outputs the dialogue task generation screen illustrated in Figure 5 to the creator terminal 3 and accepts input settings for the information necessary to create the first prompt. For example, the control unit 11 accepts input settings such as the target age group of learners, proficiency level, type of dialogue task to be generated, and learning items. The control unit 11 creates the first prompt by applying the input content to a predetermined first prompt template.

[0093] The control unit 11 inputs the created first prompt to the LLM, thereby generating a specification document for the dialogue task to be performed between the learner and the LLM (step S32). Specifically, as shown in Table 1, the control unit 11 generates a specification document that includes the task name, task objectives, learning items, the learner's role, the role of the dialogue partner (LLM), the dialogue status, and task details.

[0094] The control unit 11 creates a second prompt that includes the generated specification (step S33). For example, the control unit 11 outputs the instruction sheet and the third prompt generation screen illustrated in Figure 7 to the creator terminal 3 and accepts input settings for the information necessary to create the second prompt, including the specification. For example, the control unit 11 accepts input settings for the instruction statement to LLM, prerequisites, output format, specification, etc.

[0095] The control unit 11 inputs the created second prompt to the LLM, generating an instruction sheet that instructs the learner to perform a dialogue task, and a third prompt that instructs the LLM to generate dialogue text corresponding to the dialogue task (step S34). Specifically, as shown in Table 2, the control unit 11 generates an instruction sheet that shows the dialogue situation, task content, additional information, etc. Also, as shown in Table 3, the control unit 11 generates a third prompt that shows the basic rules of the dialogue, the role of the LLM, the dialogue situation, the dialogue goal, constraints, output procedure, etc.

[0096] The control unit 11 stores the specifications for the dialogue task generated in step S32, the instructions generated in step S34, and the third prompt in the task DB 304 (step S35), and then terminates the series of processes.

[0097] Figure 10 is a flowchart showing the steps involved in the dialogue process. Based on Figure 10, the processing steps involved in conducting a dialogue according to the dialogue task will be explained.

[0098] The control unit 11 of server 1 determines which dialogue task to recommend to the learner from among multiple dialogue tasks registered (stored) in the task DB 304 (step S51). Specifically, the control unit 11 first extracts the features of the learner's profile (learner features) and the features of the specifications for each dialogue task (task features). In the initial stages of operation of this system, the control unit 11 calculates the similarity between the learner features and the task features and determines the recommended task based on this similarity. After the initial stages of operation, the control unit 11 inputs the learner features and task features into a task achievement expectancy prediction model (first model) to predict (output) the task achievement expectancy, and inputs the learner features and task features into a task selection expectancy prediction model (second model) to predict (output) the task selection expectancy. The control unit 11 then calculates the task recommendation score from the task achievement expectancy and task selection expectancy and determines the recommended task. For example, the control unit 11 extracts a total of three dialogue tasks as recommendation tasks: a dialogue task at the same level as the learner's proficiency level, a dialogue task at one level lower, and a dialogue task at one level higher.

[0099] The control unit 11 outputs and displays the multiple dialogue tasks determined in step S51 to the learner terminal 2 (step S52). The control unit 11 accepts input to select one of the output dialogue tasks (step S53).

[0100] The control unit 11 outputs and displays instructions corresponding to the selected dialogue task on the learner terminal 2 (step S54). Then the control unit 11 performs the dialogue related to the dialogue task (step S55). That is, the control unit 11 generates a dialogue sentence by inputting a third prompt corresponding to the selected dialogue task into the LLM, and outputs it to the learner terminal 2 for audio playback. The control unit 11 also accepts audio input of response sentences to the dialogue sentence. The control unit 11 performs the dialogue by sequentially generating and outputting dialogue sentences using the LLM and sequentially accepting response sentence inputs from the learner.

[0101] The control unit 11 determines whether the learner has achieved the target learning item based on the response sentence input by the learner (step S56). Specifically, when a response sentence is input, the control unit 11 identifies the dialogue act by inputting the input response sentence into a model (third model) that has been trained to identify the dialogue act that indicates the intent of the utterance at each part of the response sentence. The control unit 11 extracts the response sentence at the part where the dialogue act corresponding to the target learning item in the dialogue task has been identified. When a response sentence is input, the control unit 11 outputs the proficiency level by inputting the extracted response sentence into a model (fourth model) that has been trained to output the learner's proficiency level. The control unit 11 determines whether the learner has achieved the learning item by comparing the outputted proficiency level with the learner's proficiency level measured in advance.

[0102] The control unit 11 updates the task selection expectation prediction model and the task completion expectation prediction model based on the selection result of the dialogue task in step S53 and the determination result of whether or not the learning item has been achieved in step S56 (step S57). The control unit 11 outputs the determination result of step S56 to the learner terminal 2 (step S58) and ends the series of processes.

[0103] Based on the above, this embodiment allows for the design of suitable dialogue tasks to be performed by language learners.

[0104] (Variation 1) In the above-described embodiment, a method was explained in which learning items that learners should aim for are identified based on the learner's profile, proficiency level, etc., and dialogue tasks corresponding to those learning items are generated. In this modified example, a method is described in which target learning items are identified while also considering the difficulty level of each learning item.

[0105] In this modified example, Server 1 calculates the difficulty level of the learning item corresponding to the dialogue task based on the achievement or failure determination result of the dialogue task determined by the achievement determination unit 39. For example, Server 1 may calculate the difficulty level as the achievement rate of a simple learning item (number of learners who have achieved the learning item / total number of learners for whom that learning item is set as a target).

[0106] Alternatively, Server 1 may calculate the difficulty level of the learning items using statistical methods such as item response theory (Rush model). Here, item response theory can be expressed by the following equation (2).

[0107]

number

[0108] pnxt1 is the probability that user n receives an achievement rating of 1 (achieved) on task t, where the learning item (Can-Do statement) x is the target of mastery. pnxt0 is the probability that user n receives an achievement rating of 0 (not achieved) on task t, where the learning item x is the target of mastery. θn is the proficiency level of learner n. βx is the difficulty level of learning item x. γt is the difficulty level of task t.

[0109] Server 1 stores the difficulty levels calculated as described above in the learning item DB302, associating them with the learning items. Note that machine learning techniques may be used for these calculations. This allows the difficulty levels of the learning items to be updated based on the actual task completion rates of learners after a certain period of operation.

[0110] Server 1 identifies learning items that learners should aim for, by referring to the difficulty level of the learning items in addition to the learner's profile and proficiency level. For example, Server 1 identifies learning items as target learning items that are similar to the learner's profile and text features, and whose difficulty level is close to the learner's proficiency level. At that time, among the grammatical items closely related to the identified learning items (Can-Do statements), items that the learner has not yet mastered may also be identified and designated as target learning items. As described in the above embodiment, the list of identified learning items is stored in the learner DB 301.

[0111] The number of learning items stored in the learner DB301 may be controlled by a hyperparameter (for example, 10 items).

[0112] Furthermore, Server 1 may use variables such as the target learning period until the learner achieves their desired goal (extractable from the needs collected by Needs Collection Unit 32), the learner's desired learning frequency (extractable from the needs collected by Needs Collection Unit 32), the learner's current proficiency level, the difficulty level of the learning items, and the difference between the learner's proficiency level and the difficulty level of the learning items) to automatically control the number of learning items set as targets for a certain period using a rule-based selection algorithm or a separately trained machine learning model. Through the above processing, it is possible to formulate a learning plan (learning items) that takes into account the learning period remaining for the learner, the learning frequency, and the difference between the learning items and the learner's current ability. The above information may also be formulated as an optimization problem such as an integer linear programming problem using constraints, and the learning plan may be generated by solving it.

[0113] Here, the list of learning items stored in the learner DB301 is presented to the learner, and the learner may be allowed to select priority items they want to learn most immediately. For example, assuming that 10 learning items are identified in the learning plan generated using the method described above, if the learner selects 3 of these as priority items, these 3 learning items will be assigned the highest priority. Other learning items within the learning plan will be assigned a medium priority, and learning items outside the learning plan will be assigned the lowest priority. When generating an interactive task, Server 1 selects learning items in order from highest priority to lowest priority and generates the interactive task. In this process, the learner can refer to and select specific learning opportunities and learning items they wish to learn, allowing them to take the lead in their own learning.

[0114] The priority levels mentioned above may also be continuous values, and the priority of learning items may be determined by a threshold value for these continuous values.

[0115] (Modification 2) In the above-described embodiment, it was explained that, in addition to Can-Do statements, language items such as vocabulary and grammar may be identified as learning items, and a dialogue task may be generated in which the learner learns these language items by inputting them into the first prompt. In this modified example, a method for having a learner learn specific language items such as vocabulary and grammar will be described.

[0116] For example, when Server 1 conducts a dialogue task with a learner, it aggregates the frequency of use of learning items (vocabulary, grammar, Can-Do statements, etc.). Specifically, Server 1 aggregates the frequency of use of learning items for each learner by dividing the number of dialogues in which the target learner used the learning item by the total number of dialogues in which that learner used the learning item, and the frequency of use of learning items for all learners by dividing the number of dialogues in which all learners used the learning item by the total number of dialogues in which all learners used the learning item. Server 1 also calculates the significance level (p-value) and effect size (odds ratio) of these statistics. Server 1 stores these statistics in the learner DB 301.

[0117] By using this information, Server 1 can identify language items that are used extremely infrequently compared to the overall average, based on their statistical significance and effect size, for vocabulary and grammar that other learners frequently use but the target learner is unable to use. Specifically, among the language items with statistically significantly low usage frequency, those with higher effect sizes are defined as weaknesses, and these language items can be entered into the first prompt in the task generation procedure and used for task generation.

[0118] (Variation 3) In the above-described variation 2, we explained a form in which dialogue tasks are generated with language items such as vocabulary and grammar as learning items. In this variation, we will explain a form in which Can-Do statements are associated with learning items (vocabulary, grammar, etc.) that are independent of the Can-Do statements, and dialogue tasks are generated from both.

[0119] In this modified version, the learning item DB302 stores the degree of relevance between Can-Do statements and other language items (vocabulary, grammar, etc.). This degree of relevance is expressed as an arbitrary number (e.g., 0.0 to 1.0) and represents the degree of relevance between the Can-Do statement and the language item.

[0120] For example, when Server 1 identifies a target Can-Do statement based on the learner's profile, proficiency level, etc., it identifies language items highly relevant to that Can-Do statement from the learning item DB302. Then, by inputting these language items in addition to the Can-Do statement into the first prompt, Server 1 can generate a dialogue task for learning specific language items.

[0121] Furthermore, dialogue tasks can also be generated in reverse order. That is, if Server 1 knows from the information stored in Learner DB301 that the learner has weaknesses in certain language items, it identifies Can-Do statements with a high degree of relevance to those language items from Learning Item DB302 and generates a dialogue task by inputting them into the first prompt. This makes it possible to generate dialogue tasks that address the learner's weaknesses.

[0122] There are several ways to implement relevance. One is for Server 1 to use the frequency (or normalized frequency information) of a particular language item appearing when it performs a dialogue task generated based on a specific Can-Do statement as the relevance. Alternatively, Server 1 may use any pre-trained language model (such as BERT) to convert the Can-Do statement and language items into embedding vectors and calculate the similarity (e.g., cosine similarity) between these vectors as the relevance. Another method is for Server 1 to use a machine learning model that takes embedding vectors or the above-mentioned frequencies as input and outputs the relevance.

[0123] The embodiments disclosed herein should be considered in all respects to be illustrative and not restrictive. The scope of the present invention is indicated by the claims, not in the sense described above, and all modifications within the sense and scope equivalent to the claims are intended.

[0124] The matters described in each embodiment can be combined with each other. Furthermore, the independent and dependent claims described in the claims can be combined with each other in any combination, regardless of the form of reference. In addition, the claims use a form in which claims referencing two or more other claims (multi-claim form), but are not limited to this. A form in which multi-claims referencing at least one multi-claim (multi-multi-claim) may also be used. [Explanation of Symbols]

[0125] 1. Server (Information Processing Device) 11 Control Unit 12 Main memory 13 Communications Department 14 Auxiliary storage P Program 2. Learner terminals 3. Creator's terminal

Claims

1. We obtain learning information that represents what learners of the target language should learn, By inputting the first prompt containing the aforementioned learning information into the language model, a specification document for the dialogue task to be performed by the learner is generated. An information processing method in which a computer performs the processing.

2. By inputting the second prompt, which includes the generated specification document, into the language model, an instruction document that instructs the learner to perform the dialogue task and a third prompt that instructs the language model to generate dialogue sentences corresponding to the dialogue task are generated. The information processing method according to claim 1.

3. By inputting the first prompt multiple times into the language model, multiple specifications corresponding to each of the multiple dialogue tasks are generated. By inputting the multiple second prompts, each containing the multiple specifications, into the language model, the instruction document and the third prompt corresponding to each of the multiple dialogue tasks are generated. The aforementioned multiple dialogue tasks are output to the learner, The system accepts input to select one of the aforementioned multiple dialogue tasks. Output the instruction sheet corresponding to the selected dialogue task to the learner. By inputting the third prompt corresponding to the selected dialogue task into the language model, a dialogue sentence corresponding to the dialogue task is generated. The generated dialogue is output to the learner. Accepts input of responses to dialogues. The information processing method according to claim 2.

4. The system outputs predetermined questions to the learners. We accept input of answers to the above questions. Based on the above response, a learner profile will be generated. Based on the aforementioned profile, the learning items that the learner should study in the target language are identified as the learning information. The specification is generated by inputting the first prompt, which includes the identified learning items, into the language model. The information processing method according to claim 3.

5. Extract the feature quantities from the aforementioned profile and specification document, Based on the characteristics of the respective profiles and specifications, the dialogue task to be recommended to the learner is determined from among the multiple dialogue tasks. Output the determined dialogue task to the learner. The information processing method according to claim 4.

6. When the features of the profile and specification are input, the first model, which has been trained to output the expected task completion score that a learner can achieve when those features are input, outputs the expected task completion score by inputting the extracted features of the profile and specification. When the features of the profile and specification are input, the extracted features of the profile and specification are input to a second model that has been trained to output the task selection expectation that the learner will select the dialogue task, and the extracted features of the profile and specification are input to output the task selection expectation. Based on the expected task completion rate and the expected task selection rate, the dialogue task to recommend to the learner is determined. The information processing method according to claim 5.

7. When the aforementioned response sentence is input, the input response sentence is input to a third model that has been trained to identify the dialogue act that indicates the intent of the utterance at each part of the response sentence, thereby identifying the dialogue act at each part of the response sentence. Extract the response sentence from the location where the dialogue act corresponding to the learning item has been identified. When the aforementioned response sentence is input, the extracted response sentence is input to the fourth model, which has been trained to output the learner's proficiency level, and the proficiency level is output. Based on the outputted proficiency level, it is determined whether or not the learner has achieved the learning item. The information processing method according to claim 6.

8. Based on the determination result of whether or not the learning items have been achieved and the selection result of the dialogue task, the first and second models are updated. The information processing method according to claim 7.

9. We obtain learning information that represents what learners of the target language should learn, By inputting the first prompt containing the aforementioned learning information into the language model, a specification document for the dialogue task to be performed by the learner is generated. A program that instructs a computer to perform a process.

10. An information processing device comprising a control unit, The control unit, We obtain learning information that represents what learners of the target language should learn, By inputting the first prompt containing the aforementioned learning information into the language model, a specification document for the dialogue task to be performed by the learner is generated. Information processing device.

Images