Information processing method, information processing device, and program

Abstract

This information processing method causes a computer to perform processing of: acquiring first utterance data of a target language of a user; acquiring the proficiency level of the target language of the user; generating second utterance data obtained by changing the representation of the first utterance data; calculating, on the basis of the proficiency level of the user, a first score indicating learning efficiency of the target language by presenting the second utterance data; and outputting the second utterance data to the user according to the first score.

Description

Information processing method, information processing device, and program

[0001] The present invention relates to an information processing method, an information processing device, and a program.

[0002] There are systems that support language learning. For example, Patent Literature 1 discloses a language learning system that points out errors in grammar, pronunciation, etc. in audio information provided by a learner and presents correct grammar, pronunciation, etc. to the learner.

[0003] Taiwan Patent Application Publication No. 202121357

[0004] In one aspect, an object of the present invention is to provide an information processing method and the like that can present advice suited to the user's level of proficiency in a target language.

[0005] In one aspect, an information processing method includes a computer executing a process to acquire first speech data in a target language by a user, acquire the user's proficiency in the target language, generate second speech data by modifying the expression of the first speech data, calculate a first score indicating the learning efficiency of the target language by presenting the second speech data based on the user's proficiency, and output the second speech data to the user according to the first score.

[0006] In one aspect, advice can be presented that is appropriate to the user's level of proficiency in the target language.

[0007] FIG. 1 is an explanatory diagram showing an example of the configuration of a language learning system. FIG. 2 is a block diagram showing an example of the configuration of a server. FIG. 3 is a diagram showing an overview of a language learning system. FIG. 4 is a diagram for explaining a proficiency conversion process. FIG. 5 is a diagram showing an example of a record layout of a vocabulary dictionary. FIG. 6 is a diagram showing an example of a display of a second uttered sentence. FIG. 7 is a diagram showing an example of a display of a first uttered sentence. FIG. 8 is a flowchart showing an example of a processing procedure executed by a server. FIG. 9 is a flowchart showing a processing procedure of a subroutine for generating a second uttered sentence. FIG. 10 is a flowchart showing a processing procedure of a subroutine for evaluating strengths.

[0008] The present invention will be described in detail below with reference to the drawings showing embodiments thereof. (Embodiment) Fig. 1 is an explanatory diagram showing an example of the configuration of a language learning system. In this embodiment, a language learning system will be described in which a user learns a language by interacting with the system (chatbot) in a target language (e.g., English). The language learning system includes an information processing device 1, a terminal 2, and a generation server 3. Each device is communicatively connected via a network N such as the Internet.

[0009] The information processing device 1 is an information processing device capable of various information processing and information transmission and reception, such as a server computer or a personal computer. In this embodiment, the information processing device 1 is assumed to be a server computer, and for simplicity, will be referred to as server 1 below. The server 1 assists language learning by having a dialogue with a user in a target language via the terminal 2. For example, the server 1 conducts a dialogue in a question-and-answer format, outputs a question in the target language on a predetermined topic, and receives a response (answer) to the question from the user in the target language.

[0010] In this embodiment, the dialogue with the user is carried out in a question and answer format, but the dialogue with the user may be casual chat or everyday conversation, and the format is not limited to the question and answer format.

[0011] In this embodiment, the server 1 determines the user's proficiency in the target language from the user's responses, i.e., the entire utterances (utterance data).The server 1 then extracts utterances that contribute to the user's current proficiency level (utterances that are weak in expressing the next higher proficiency level) as weaknesses, rephrases the utterances to express the next higher proficiency level, generates second utterances (second utterance data), and presents (outputs) the second utterances to the user (see FIGS. 6 and 7).The server 1 also evaluates the degree to which the utterances express the user's strengths in the target language, extracts utterances that show a high degree of proficiency, and presents them as the user's strengths (see FIG. 8).

[0012] The terminal 2 is a terminal device used by the user, such as a personal computer, a smartphone, a tablet terminal, etc. The server 1 acquires utterance data in a target language by the user via the terminal 2, and outputs a second utterance sentence, etc., which is a paraphrase of the user's utterance sentence, to the terminal 2 and presents it to the user.

[0013] The generation server 3 is a server computer that generates a response sentence to an input sentence in response to an external request using an LLM (Large Language Model) 30. The server 1 generates the second utterance sentence using the LLM 30 by requesting the generation server 3 to generate the second utterance sentence through a predetermined API (Application Programmable Interface).

[0014] The LLM 30 is a machine learning model trained with a huge amount of training data, and generates a response sentence when an input sentence is received. The LLM 30 is, for example, a Generative Pretrained Transformer (GPT) or a Bidirectional Encoder Representations from Transformers (BERT), but the model is not particularly limited.

[0015] 2 is a block diagram showing an example configuration of the server 1. The server 1 includes a control unit 11, a main memory unit 12, a communication unit 13, and an auxiliary memory unit 14. The control unit 11 has one or more arithmetic processing devices such as a central processing unit (CPU), a micro-processing unit (MPU), or a graphics processing unit (GPU), and performs various information processing, control processing, and the like by reading and executing a program P stored in the auxiliary memory unit 14. The main memory unit 12 is a temporary storage area such as a static random access memory (SRAM) or a dynamic random access memory (DRAM), and temporarily stores data necessary for the control unit 11 to execute arithmetic processing. The communication unit 13 is a communication module for performing communication-related processing and transmits and receives information to and from the outside.

[0016] The auxiliary storage unit 14 is a non-volatile storage area such as a large-capacity memory or a hard disk, and stores a program P (program product) and other data necessary for the control unit 11 to execute processing. The auxiliary storage unit 14 also stores a proficiency assessment model 40. As will be described later, the proficiency assessment model 40 is a machine learning model consisting of one or more feature extractors and a proficiency assessor, and is a model that, when inputting speech data in the target language by the user, determines (outputs) the user's proficiency in one or more aspects of the target language.

[0017] The auxiliary storage unit 14 may be an external storage device connected to the server 1. The server 1 may be a multi-computer consisting of multiple computers, or may be a virtual machine virtually constructed by software.

[0018] Furthermore, in this embodiment, the server 1 is not limited to the above configuration, and may include, for example, an input unit that accepts operation input, a display unit that displays images, etc. Furthermore, the server 1 may be provided with a reading unit that reads a portable storage medium 1a such as a CD (Compact Disk)-ROM or a DVD (Digital Versatile Disc)-ROM, and may read and execute the program P from the portable storage medium 1a.

[0019] Fig. 3 is a diagram showing an outline of the language learning system, illustrating a sequence of processes executed by the server 1. An outline of this embodiment will be described based on Fig. 3.

[0020] As already mentioned, this system supports the user's learning of the target language by conducting dialogue in a question-and-answer format. Specifically, the server 1 plays multiple audio questions on the terminal 2 according to a predetermined scenario and accepts audio responses (answers) to each audio question. The terminal 2 also captures images of the user during the dialogue using a camera. The server 1 acquires the response audio (speech) and video of the user during the dialogue as data of the user's speech in the target language.

[0021] In this embodiment, the speech data acquired from terminal 2 is described as being voice and video (and characters recognized from voice), but for example, terminal 2 may only acquire voice and not capture video. Also, for example, terminal 2 may acquire text entered via a keyboard or the like as speech data, rather than voice. In this way, "speech data" may be data that includes at least characters or voice. In this embodiment, a character string transcribed from voice among the speech data is called an "utterance sentence," and an utterance sentence is described as consisting of one or more sentences (units separated by periods).

[0022] The control unit 11 of the server 1 executes the program P to function as a language ability diagnosis unit 50 , a first utterance sentence extraction unit 60 , a second utterance sentence generation unit 70 , and a strength evaluation unit 80 .

[0023] The language ability diagnosis unit 50 determines the user's proficiency in the target language from speech data in the target language by the user. Proficiency is data that expresses the user's language ability using discrete and / or continuous values. In this embodiment, the "proficiency" is determined using the CEFR (Common European Framework of Reference for Languages) levels, which are evaluated on a six-level scale: A1, A2, B1, B2, C1, and C2. In the CEFR, A1 is the lowest level and C2 is the highest level.

[0024] The language ability diagnosis unit 50 functions as a preprocessing unit 51, a feature extraction unit 52, a proficiency determination unit 53, and a contribution calculation unit 54. The preprocessing unit 51 processes the speech data to match the input format of one or more feature extractors in the feature extraction unit 52. The preprocessing unit 51 causes the server 1 to perform speech recognition on the speech data (speech), and to perform processes such as dividing video, audio, and speech-recognized character strings into sentence units.

[0025] The feature extraction unit 52 extracts features from the speech data using a feature extractor included in the proficiency assessment model 40. The proficiency assessment unit 53 assesses the user's proficiency in the target language based on the features extracted from the speech data using the proficiency assessor included in the proficiency assessment model 40.

[0026] The proficiency assessment model 40 is a model that assesses (outputs) a user's proficiency in the target language when utterance data in the target language by the user is input. The proficiency assessment model 40 is composed of one or more feature extractors that extract features from the utterance data, and a proficiency assessor that assesses proficiency from one or more perspectives based on the features extracted by the feature extractors.

[0027] Detailed illustrations and descriptions of the proficiency assessment model 40 are omitted. For example, the proficiency assessment model 40 includes multiple (e.g., nine) feature extractors. At least one piece of information from an utterance, speech, or video is input to each feature extractor, and each feature extractor extracts a different feature. For example, one feature extractor is an extractor that extracts text features from an utterance, and is configured, for example, with ALBERT (A Lite BERT). When a sentence-based utterance is input to the extractor in chronological order, each character string in the utterance, divided into predetermined units such as words, is converted into a feature (vector value). In this way, each feature extractor extracts a different feature.

[0028] The proficiency determiner determines (outputs) the user's proficiency based on the features extracted by each feature extractor. The proficiency determiner is configured, for example, by a Transformer, and outputs the proficiency when the features extracted by each feature extractor are input. The proficiency determiner outputs proficiency in one or more language abilities. For example, proficiency levels related to vocabulary range, grammatical accuracy, fluency, phonology, interaction, coherence, and overall proficiency are output. The proficiency determiner outputs a probability value that the user's proficiency belongs to each of the proficiency levels A1 to C2 for each of the seven aspects.

[0029] 4 is a diagram illustrating the proficiency conversion process. As described above, the proficiency assessment model 40 (proficiency assessor) outputs a probability value for each of the proficiencies A1 to C2. In this embodiment, the proficiency assessment unit 53 calculates a weighted sum x of these probability values ​​as a score representing the proficiency, according to the following formula (1):

[0030]

[0031] C is the number of classes (6 classes in CEFR), p ｃ represents the probability value of class c (CEFR level). ｃ satisfies the following formula (2).

[0032]

[0033] The score x calculated based on Equation (1) is conceptually illustrated in the upper part of Fig. 4. As shown in Fig. 4, if the score x is not normalized, the boundaries between A1 and A2, A2 and B1, ..., and C1 and C2 will be unbalanced. Therefore, the proficiency level assessment unit 53 normalizes the weighted sum x according to the following Equation (3) to calculate the value x' as the final score.

[0034]

[0035] a is the boundary value between B1 and B2 before normalization, b is the boundary value between B2 and C1 before normalization, a' is the boundary value between B1 and B2 after normalization, and b' is the boundary value between B2 and C1 after normalization. The user is presented with the normalized score x'.

[0036] Returning to Fig. 3, the explanation continues. The contribution calculation unit 54 calculates the contribution of each element of the utterance data to the proficiency output from the proficiency assessment model 40. The contribution is a measure of the influence each explanatory variable has on the output of the machine learning model, and in this embodiment, the SHAP value (SHapley Additive exPlanations) is used as the contribution. SHAP is a method that applies the Shapley value of cooperative game theory to machine learning.

[0037] Furthermore, "elements of speech data" refer to components obtained by dividing speech data (spoken sentences or spoken voice) into predetermined units, and in this embodiment refer to each word or phrase obtained by dividing a spoken sentence into units of words or phrases. The contribution calculation unit 54 calculates the contribution (SHAP value) of each word or phrase in the spoken sentence to each of the proficiency levels A1 to C2 that can be output from the proficiency assessment model 40. For example, when the user's proficiency level is A2 (when A2 has the highest probability value), the contribution calculation unit 54 can not only calculate the contribution of each word or phrase in the spoken sentence to proficiency level A2, but can also calculate the contribution of each word or phrase to the other proficiency levels A1 and B1 to C2.

[0038] Next, the first utterance sentence extraction unit 60 in FIG. 3 will be described. The first utterance sentence extraction unit 60 extracts first utterance sentences (first utterance data) from the entire utterance data input to the language ability diagnosis unit 50. The first utterance sentences are input to the second utterance sentence generation unit 70 and the strength evaluation unit 80. The first utterance sentences consist of one or more sentences. Extraction conditions can be set for the extracted first utterance sentences depending on the application. For example, if the purpose is to display results on a small screen such as a smartphone, a limit can be set on the length of the extracted first utterance sentences. Furthermore, to improve the efficiency of the processing in the second utterance sentence generation unit 70 and the strength evaluation unit 80, the first utterance sentence extraction unit 60 may detect and remove utterance sentences with low importance, thereby reducing the computational costs of the second utterance sentence generation unit 70 and the strength evaluation unit 80. In other words, the first utterance data may be the entire utterance data or a portion of the entire utterance data. Typically, multiple first utterance sentences (first utterance data) are extracted from the entire utterance data, and the second utterance sentence generation unit 70 and the strength evaluation unit 80 are applied to these multiple first utterance sentences (first utterance data).

[0039] Next, the vocabulary dictionary 92 used when calculating the first score and second score (described later) will be described. The vocabulary dictionary 92 is a database that associates each word (each element of utterance data) that may appear in an utterance sentence with a proficiency level. FIG. 5 is a diagram showing an example of a record layout of the vocabulary dictionary 92. The vocabulary dictionary 92 includes a word string, a proficiency string, and a word meaning string. The word string stores each word. The proficiency string and the word meaning string each store a proficiency level (CEFR level related to Range) and a meaning corresponding to the proficiency level in association with the word. As shown in FIG. 5, the vocabulary dictionary 92 defines the proficiency level of a user who uses each word for each meaning (meaning). An English Vocabulary Profile can be used as such a vocabulary dictionary 92.

[0040] In this embodiment, the server 1 updates the vocabulary dictionary 92 based on the usage history of each word by each user. Specifically, the server 1 first executes a task to resolve the ambiguity of the meaning of each word included in a sentence uttered by the user, thereby identifying the meaning of each word. Word sense disambiguation refers to the task of identifying the meaning in which a word is used in a sentence. While the method for identifying the meaning is not particularly limited, in this embodiment, the word meaning is identified using a machine learning model trained on a predetermined corpus, specifically, a Text-to-Text Transfer Transformer (T5). The second utterance sentence generation unit 70 inputs the second utterance sentence, the word to be disambiguated, and the word meaning list of the word into the model, and obtains, as an output, the word meaning estimated to be appropriate from the word meaning list.

[0041] As described above, the server 1 determines the user's level of proficiency in the target language from the user's speech data. The server 1 stores the words (and their meanings) used by each user and the user's level of proficiency in the used vocabulary DB 91. That is, the server 1 stores the usage history of words and phrases by users of each level of proficiency. Furthermore, the server 1 stores, as the usage history of each word and phrase, information that can identify which word and phrase was used in which sentence of which dialogue, when, and by which user in the used vocabulary DB 91.

[0042] The server 1 refers to the used vocabulary DB 91 and updates the vocabulary dictionary 92. Specifically, the server 1 tallies the number of times each word included in each user's utterance is used, by user proficiency level and by meaning. When the number of times each word is used, by proficiency level and meaning, reaches or exceeds a predetermined number, the server 1 associates the proficiency level at which the number of times of use is greater than or equal to the predetermined number of times with the word and its meaning and stores the association information in the vocabulary dictionary 92. In this way, the server 1 updates the vocabulary dictionary 92.

[0043] As will be described later, the server 1 refers to this vocabulary dictionary 92 to calculate the first score and the second score.

[0044] 3 , the explanation will be continued. As described above, the server 1 determines the user's proficiency level from the utterance data, and calculates the contribution of each word in the utterance sentence that is the basis for the determination. In this embodiment, the server 1 extracts, from the utterances by the user, an utterance sentence that determines the user's proficiency level as the current proficiency level, and generates a second utterance sentence by paraphrasing this utterance sentence to an expression that is one level higher than the user's proficiency level, and presents (outputs) the second utterance sentence to the user.

[0045] The second utterance sentence generation unit 70 generates the second utterance sentence. Specifically, the second utterance sentence generation unit 70 generates a large number of second utterance sentences using the LLM 30, and calculates a first score that represents the learning efficiency of the target language by presenting (outputting) each second utterance sentence (how efficient the learning is when the second utterance sentence is presented to the user). The second utterance sentence generation unit 70 then identifies a predetermined number of second utterance sentences with the highest first scores and presents them to the user (see FIGS. 6 and 7 ).

[0046] In this embodiment, the above-mentioned "first score" is called EoVL (Efficiency of Vocabulary Learning).

[0047] First, the second utterance sentence generation unit 70 receives as input one or more first utterance sentences extracted by the first utterance sentence extraction unit 60, creates a prompt instructing the generation of multiple paraphrases (second utterance sentences) from each first utterance sentence, and transmits the prompt to the generation server 3. The generation server 3 inputs the prompt to the LLM 30, thereby generating multiple second utterance sentences from each first utterance sentence and outputting them to the server 1.

[0048] In this embodiment, the second utterance sentence is generated using the LLM 30, but the present embodiment is not limited to this. For example, the server 1 may refer to a vocabulary dictionary 92 described below to generate a second utterance sentence in a rule-based manner by changing a word included in the first utterance sentence to a word having a proficiency level one step higher than the user's proficiency level. In this way, the means for generating the second utterance sentence is not limited to the LLM 30.

[0049] The second utterance sentence generation unit 70 executes a task of disambiguating the meaning of each word contained in each generated second utterance sentence, and identifies the meaning of each word. The content of the word meaning disambiguation process is the same as that described above, and therefore will not be described here.

[0050] The second utterance sentence generation unit 70 calculates, for each generated second utterance sentence, EoVL, which represents the learning efficiency achieved by presenting the second utterance sentence to the user. Specifically, the second utterance sentence generation unit 70 calculates EoVL according to the following formula (4).

[0051]

[0052] where f ｐ (p=1 to 5) are defined by the following formulas (5) to (9). ｐ represents the weight for the formulas (5) to (9), and is set to 1 in this embodiment.

[0053]

[0054]

[0055]

[0056]

[0057]

[0058] Note that v(w) in equation (9) is defined as in equation (10) below.

[0059]

[0060] φ ｋrepresents the contribution (SHAP value) of the kth word in the utterance. L represents the user's CEFR level (proficiency) for Range. U [i,j] represents the user's response (first utterance sentence) to the system's question (i≦j). In this system, among the sentences included in the user's utterance (for example, sentences ending with ".", "?", or "!"), the number of words is M. ｍｉｎ That's all for M ｍａｘ The following sentences are covered: [i,j],n is generated by LLM30, U [i,j] represents the n-th paraphrase (second utterance sentence) of [i,j],n is S [i,j],n is converted into a colloquial expression (a third spoken sentence, which will be described later). l(w) represents the CEFR level (proficiency level) of the meaning w of the phrase. D represents the used vocabulary DB 91.

[0061] As shown in the formulas (4) to (9), EoVL is calculated by the first to fifth index values ​​f １ ~f ５ The first index value f １ represents how likely the first utterance sentence is to be of the user's proficiency level L, where L is the user's proficiency level. Specifically, the first index value f １ is the contribution φ of each phrase in the first utterance to the proficiency level L k (L) It is calculated based on the cumulative value of f １ By calculating EoVL based on the above, it is possible to extract the first uttered sentence that makes the user's proficiency level the current proficiency level L, and present a second uttered sentence that paraphrases the first uttered sentence.

[0062] Second index value f ２ represents how far the first utterance sentence falls short of the user's proficiency level L+1, which is one level higher than the user's proficiency level L, when the user's proficiency level is L. Specifically, the second index value f ２ is the contribution φ of each phrase of the first utterance to the user's proficiency level L+1, which is one level higher than the user's proficiency level L. k (L+1) It is calculated by taking the negative value of the cumulative value of f ２By calculating the EoVL based on the above, it is possible to extract the first utterance sentence that is the user's weakness, and present the second utterance sentence that is a paraphrase of the first utterance sentence.

[0063] Third index value f ３ represents how related (similar) the second utterance sentence is to the first utterance sentence. Specifically, the third index value f ３ is the first utterance sentence U [i,j] and the second utterance S [i,j],n It is defined as the similarity between

[0064] While the method for calculating the similarity is not particularly limited, in this embodiment, the similarity between two texts (the first utterance sentence and the second utterance sentence) is calculated using an optimal transport planning technique. Specifically, the second utterance sentence generation unit 70 converts each phrase in the first utterance sentence and the second utterance sentence into a vector value using a model such as ALBERT. The second utterance sentence generation unit 70 solves an optimization problem (integer linear programming problem) to minimize the cost of associating (transporting) phrases in one text with phrases in the other text in a vector space, thereby obtaining the correspondence (alignment) of phrases with the lowest transport cost and the optimal transport cost (dissimilarity). For formulation, for example, WRD (Word Rotator's Distance) is used. The second utterance sentence generation unit 70 calculates the similarity by subtracting the obtained dissimilarity from 1.

[0065] Using the above-described method, the second utterance sentence generation unit 70 generates the first utterance sentence U [i,j] and the second utterance S [i,j],n The similarity with the third index value f ３ It is calculated as: f ３ By calculating the EoVL based on the above, it is possible to extract the second utterance sentence related to the original first utterance sentence.

[0066] Fourth index value f ４ represents the colloquialism of the second utterance. Specifically, the fourth index value f ４ is the second utterance sentence S [i,j],n and a third utterance S' obtained by converting the second utterance into a colloquial expression. [i,j],n It is defined as the similarity between

[0067] When the second utterance sentence generation unit 70 generates the second utterance sentence, it converts the second utterance sentence into a third utterance sentence by converting the second utterance sentence into a colloquial expression. The method for generating the third utterance sentence is not particularly limited, but for example, the second utterance sentence generation unit 70 generates the third utterance sentence using a machine learning model that has been trained to convert an input sentence into a colloquial sentence, specifically Styleformer (T5). The second utterance sentence generation unit 70 converts the second utterance sentence into the third utterance sentence by inputting it into the model, and calculates the similarity between the second utterance sentence and the third utterance sentence. The method for calculating the similarity is as follows: ３ As in the above, an optimal transportation planning method is used. Note that the method for calculating colloquialism is not limited to the above method. A machine learning model for scoring colloquialism may be used, or a rule-based method may be used.

[0068] As described above, the second utterance sentence generation unit 70 generates the second utterance sentence S [i,j],n and the third utterance S' [i,j],n The similarity with the fourth index value f ４ It is calculated as: f ４ By calculating the EoVL based on the above, a second utterance sentence that is natural as a dialogue can be extracted.

[0069] Fifth index value f ５ represents how many words with a proficiency level one level higher than the user's proficiency level are included in the second utterance sentence. ５ is calculated as the cumulative total of the value v(w) of each word assigned according to formula (10).

[0070] The second utterance sentence generation unit 70 identifies the level (proficiency level) l(w) of each phrase included in the second utterance sentence by referring to a vocabulary dictionary 92 that associates each phrase (its meaning) with a proficiency level. The second utterance sentence generation unit 70 then determines the value v(w) of each phrase according to Equation (10) based on the level l(w) of each phrase. Specifically, the second utterance sentence generation unit 70 assigns a score of "2" if the phrase has a proficiency level L+1 that is one level higher than the user's proficiency level L, assigns a score of "1" if the phrase is the same as the user's proficiency level L, assigns a score of "-1" if the phrase exceeds the proficiency level L+1 that is one level higher than the user's proficiency level, and assigns a score of "0" if the original first utterance sentence includes a phrase with the same meaning, if the phrase has a proficiency level lower than the user's proficiency level L, or if the user has used a phrase with the same meaning in the past. In this way, by increasing the value v(w) of phrases with a proficiency level L+1, which is one level higher than the user's proficiency level L, the second utterance sentence containing many phrases with a proficiency level L+1, which is one level higher than the user's proficiency level L, is highly evaluated. The above scoring method is an example and is not limited to this. Even for the same L+1 phrase, the importance may differ depending on the user's language learning purpose. For example, by referring to user information DB93 (a database that stores basic information such as the user's gender and age, as well as the user's learning history and learning purpose for the target language), if the user's learning purpose is business English conversation, scoring may be personalized for each user, such as by increasing the weight of phrases that are often used for business purposes.

[0071] The second utterance sentence generation unit 70 generates the first to fifth index values ​​f １ ~f ５The second utterance sentence generation unit 70 then calculates the EoVL of each second utterance sentence based on the calculated EoVL. The second utterance sentence generation unit 70 then outputs the second utterance sentence to the terminal 2 in accordance with the calculated EoVL and presents it to the user. For example, the second utterance sentence generation unit 70 extracts a predetermined number of second utterance sentences with the highest EoVL and displays the second utterance sentences on the terminal 2 in descending order of EoVL. Note that the method of calculating the EoVL is not limited to this. The EoVL may be predicted using a machine learning model trained using as learning data a ranking result obtained by sorting multiple candidate second utterance sentences by a language education expert, taking into consideration the learning efficiency of the learner. Alternatively, the user may evaluate the usefulness of each presented second utterance sentence, and a personalized EoVL may be calculated based on the user's feedback and linked to the user's learning objectives. Furthermore, when statistics are obtained by referring to the user information DB93 and the feedback vocabulary DB94 described below that show that users with certain attributes tend to learn certain words quickly, if a new user fits that attribute, the EoVL may be weighted to encourage the new user to learn the word early in their studies (so that the word is ranked higher).

[0072] 6 and 7 are diagrams showing display examples of second utterance sentences. For example, as shown in Fig. 6 and Fig. 7, terminal 2 displays the second utterance sentences with the highest EoVL in a table format. In the table of Fig. 6, "Rank" indicates the EoVL ranking, and "Dialogue" indicates the dialogue (question and response) between the system and the user. In "Dialogue," sentences in italics indicate questions from the system, sentences in regular text indicate sentences spoken by the user (responses), and sentences in bold within the sentences indicate the target sentences.

[0073] "Paraphrase" represents the second uttered sentence. Phrases (words) underlined in "Paraphrase" represent phrases used with the same meaning as the user's proficiency level L. Furthermore, double-underlined phrases represent phrases used with the meaning of proficiency level L+1, which is one level higher than the user's proficiency level L. When presenting the second uttered sentence, the second uttered sentence generation unit 70 refers to the vocabulary dictionary 92 to identify, from among the phrases included in the second uttered sentence, phrases used with the meaning of the user's proficiency level L and phrases used with the meaning of the next higher proficiency level L+1. Then, the second uttered sentence generation unit 70 displays the second uttered sentence indicating phrases with the same proficiency level as the user's proficiency level L and phrases with the next higher proficiency level L+1.

[0074] The explanation will continue by moving to Figure 7. "Word Sense (B2)" in the table in Figure 7 represents information about each word included in the second utterance sentence that is used with a meaning of proficiency level L+1, which is one level higher than the user's proficiency level L (B1 in Figure 7). Specifically, terminal 2 displays each meaning of the word and highlights the meaning of proficiency level L+1, which is one level higher than the user's proficiency level L, as indicated by double underlining. The second utterance sentence generation unit 70 reads out the meaning of each word included in the second utterance sentence that is used with a meaning of proficiency level L+1, which is one level higher than the user's proficiency level L, from the vocabulary dictionary 92 and displays it on terminal 2.

[0075] "Word Sense (B1)" represents information about each word included in the second utterance sentence that is used with the same meaning as the user's proficiency level L. Specifically, similar to the above-mentioned "Word Sense (B2)," terminal 2 displays each meaning of the word, and highlights the meaning of the word that is the same as the user's proficiency level L, as indicated by underlining. The second utterance sentence generation unit 70 reads out the meaning of each word included in the second utterance sentence that is used with the same meaning as the user's proficiency level L from the vocabulary dictionary 92, and displays it on terminal 2.

[0076] "EoVL" represents the EoVL of the second spoken sentence displayed in "Paraphrase".

[0077] In this way, the second utterance sentence generation unit 70 displays the second utterance sentence obtained by paraphrasing the first utterance sentence in an expression that is one level higher than the user's proficiency, and also displays information (semantics) about each word or phrase included in the second utterance sentence. This allows the second utterance sentence generation unit 70 to extract the utterance sentence that brings the user to their current proficiency level as a weakness, and present the user with an utterance sentence that is paraphrased in an expression that is one level higher than the user's proficiency, thereby supporting learning of the target language.

[0078] The second utterance sentence generation unit 70 stores each word or phrase presented as the user's weakness, i.e., each word or phrase included in the second utterance sentence, as a word or phrase that has been fed back to the user, together with the second utterance sentence in the feedback vocabulary DB 94. The second utterance sentence generation unit 70 also stores, in the feedback vocabulary DB 94, information that can identify which utterance sentence in the dialogue was made by which user and when the first utterance sentence before being paraphrased into the second utterance sentence belongs to.

[0079] Continuing the explanation, returning to Fig. 3, in this embodiment, in addition to presenting the user with paraphrased expressions (second utterances) based on the user's weaknesses, a second score is calculated to indicate the degree to which the strengths of the user's operational ability in the target language are expressed in the first utterance, with the aim of helping the user understand items that do not need to be studied and building confidence in the user, and first utterances with high second scores are presented to the user.

[0080] In this embodiment, the above-mentioned "second score" is called SoVC (Strength of Vocabulary Competence).

[0081] The strength evaluation unit 80 receives one or more first utterance sentences extracted by the first utterance sentence extraction unit 60 as input, calculates the SoVC for each first utterance sentence, and presents (outputs) the first utterance sentence with the highest SoVC to the user. First, the strength evaluation unit 80 performs a task of disambiguating the meaning of each word contained in the first utterance sentence, and identifies the meaning of each word. The content of the word meaning disambiguation process is the same as described above, and therefore will not be described here.

[0082] Next, the strength evaluation unit 80 calculates the SoVC for each first utterance sentence by the user according to the following formula (11).

[0083]

[0084] Here, g ｐ (p=1 to 3) are defined by the following formulas (12) to (14). ｐ represents the weighting of the formulas (12) to (14), which is set to 1 in this embodiment.

[0085]

[0086]

[0087]

[0088] Note that v'(w) in equation (13) is defined as in equation (15) below.

[0089]

[0090] As shown in equations (11) to (14), the SoVC is calculated by the sixth to eighth index values ​​g １ ~g ３ The sixth index value g １ The sixth index value g represents how likely the utterance sentence is to have a proficiency level L+1, which is one level higher than the user's proficiency level L, where L is the user's proficiency level. １ is the contribution φ of each phrase in the first utterance to the user's proficiency level L+1, which is one level higher than L. k (L+1) It is calculated based on the cumulative value of g １ By calculating the SoVC based on the above, it is possible to extract the first utterance sentence that contributes to determining the user's proficiency level as one level higher, L+1, i.e., the first utterance sentence that shows the user's strengths, and present it to the user.

[0091] Seventh index value g ２ The seventh index value g represents the number of words in the first utterance that are used with the meaning of a proficiency level L+1, which is one level higher than the user's proficiency level L, where L is the user's proficiency level. ２ is calculated as the cumulative total of the value v'(w) of each word assigned according to formula (15).

[0092] The strength evaluation unit 80 identifies the level (proficiency) l(w) of each phrase included in the first utterance by referring to a vocabulary dictionary 92 that associates each phrase (its meaning) with a proficiency level. The strength evaluation unit 80 then determines the value v'(w) of each phrase according to Equation (15) based on the level l(w) of each phrase. Specifically, the strength evaluation unit 80 assigns a score of "1" if the phrase has a proficiency level L+1, which is one level higher than the user's proficiency level L, and assigns a score of "0" if the phrase has any other proficiency level. In this way, by increasing the value v'(w) of a phrase with a proficiency level L+1, which is one level higher than the user's proficiency level L, a utterance containing many phrases with a proficiency level L+1, which is one level higher than the user's proficiency level L, is highly evaluated. Note that the above scoring method is merely an example and is not limiting. The weights may be changed depending on the user's learning objectives.

[0093] Eighth index value g ３ represents the expressiveness of the first utterance sentence. In this embodiment, the strength evaluation unit 80 uses the user's utterance sentence (first utterance sentence) extracted by the first utterance sentence extraction unit as input, and uses a machine learning model (proficiency determination model 40) trained using a dataset in which the CEFR level of Range is annotated for the utterance sentence to calculate a probability value of the CEFR level of the first utterance sentence using the machine learning model, and calculates a value x' (see FIG. 4 ) obtained by normalizing the weighted sum x of the probability values ​​calculated based on Equations (1) and (3) as an eighth index value g ３ Used as.

[0094] The strength evaluation unit 80 calculates the sixth to eighth index values ​​g １ ~g ３ The strength evaluation unit 80 then calculates the SoVC of each first utterance sentence based on the calculated SoVC. The strength evaluation unit 80 then outputs the first utterance sentence to the terminal 2 according to the calculated SoVC and presents it to the user. For example, the strength evaluation unit 80 extracts a predetermined number of first utterance sentences with the highest SoVC and displays the first utterance sentences on the terminal 2 in descending order of SoVC. Note that the method of calculating the SoVC is not limited to this. The user may evaluate whether the utterance sentences presented as strengths encouraged the user to study, and a personalized SoVC may be calculated based on this feedback in association with the user's learning objectives.

[0095] Fig. 8 is a diagram showing an example of the display of the first utterance sentence. For example, terminal 2 displays utterance sentences with high SoVC in a table format, as shown in Fig. 8. In the table in Fig. 8, "Rank" indicates the SoVC ranking, and "Dialogue" indicates the dialogue (question and response) between the system and the user. In "Dialogue," italicized text indicates a question from the system, and regular text (bold) indicates a utterance (response) from the user.

[0096] When displaying the first uttered sentence, the terminal 2 highlights, as indicated by double underlining, words used with a meaning of proficiency level L+1, which is one level higher than the user's proficiency level L. The strength evaluation unit 80 refers to the vocabulary dictionary 92 to identify the proficiency level of each word included in the first uttered sentence, and highlights words with a proficiency level L+1, which is one level higher than the user's proficiency level L.

[0097] "Word Sense (B1)" represents information about each word included in the first utterance sentence that is used with a meaning of proficiency level L+1, which is one level higher than the user's proficiency level L (A2 in FIG. 8 ). Specifically, the terminal 2 displays each meaning of the word and highlights the meaning of the word with a proficiency level L+1, which is one level higher than the user's proficiency level L, as indicated by double underlining. The strength evaluation unit 80 reads out the meaning of each word included in the first utterance sentence that is used with a meaning of proficiency level L+1, which is one level higher than the user's proficiency level L, from the vocabulary dictionary 92 and displays it on the terminal 2.

[0098] "SoVC" represents the SoVC of the first spoken sentence displayed in "Dialogue."

[0099] In this way, the strength evaluation unit 80 displays the first uttered sentence that is a strength compared to the user's proficiency, and also displays information (semantics) about each word or phrase included in the first uttered sentence. This makes it possible to present the user's strengths in language learning, help the user understand items that do not need to be studied, and motivate the user to study.

[0100] The strength evaluation unit 80 stores each word or phrase presented as a strength of the user, i.e., each word or phrase included in the first uttered sentence, as a word or phrase that has been fed back to the user, together with the first uttered sentence in the feedback vocabulary DB 94. The strength evaluation unit 80 also stores information that can identify which utterance in which dialogue, by which user, and when the first uttered sentence that has been fed back as a strength belongs to in the feedback vocabulary DB 94.

[0101] As described above, the server 1 calculates the EoVL and SoVC based on the user's proficiency level, etc., and extracts and presents the user's strengths and weaknesses. Here, the server 1 may vary the EoVL and SoVC according to the presentation (output) history of each word stored in the feedback vocabulary DB 94, i.e., the feedback history of each word.

[0102] For example, in the case of SoVC, the strength evaluation unit 80 may refer to the feedback vocabulary DB 94 and increase the SoVC of the first utterance sentence that includes a newly usable phrase (a phrase that the user has not used before) among the phrases previously fed back as weaknesses (phrases included in the second utterance sentence). Since the ability to use a phrase that was previously unusable is a noteworthy growth, the SoVC is increased so that this phrase is ranked higher. In this case, it is preferable to simultaneously present the content previously fed back as paraphrases (the second utterance sentence and the meanings of each phrase included in the second utterance sentence).

[0103] Furthermore, the strength evaluation unit 80 may refer to the feedback vocabulary DB 94 and lower the SoVC of a first utterance sentence that includes a phrase that has previously been fed back as a strength (a phrase included in a first utterance sentence that has already been output). For example, when a phrase that has already been fed back is included in the first utterance sentence, the strength evaluation unit 80 lowers the SoVC by multiplying the SoVC by a coefficient such as (1-1 / number of days) depending on the number of days since the feedback. This makes it possible to avoid repeatedly feeding back the same phrase as a strength within a short period of time.

[0104] Furthermore, when EoVL is taken into consideration, the second utterance sentence generation unit 70 may refer to the used vocabulary DB 91 and the feedback vocabulary DB 94, and if the second utterance sentence contains a word or phrase that has been previously fed back as a weakness and the user has not used that word or phrase, increase the EoVL of the second utterance sentence. This makes it possible to repeatedly present the word or phrase that is a weakness to the user and encourage learning.

[0105] As described above, according to this embodiment, the weaknesses and strengths of the user in learning the target language can be presented, and learning can be supported in an appropriate manner.

[0106] 9 is a flowchart showing an example of a processing procedure executed by the server 1. The processing executed by the server 1 will be described with reference to FIG. 9. The control unit 11 of the server 1 acquires user utterance data in a target language from the terminal 2 (step S11). The utterance data includes, for example, the user's speech and a video image of the user while speaking. The control unit 11 performs speech recognition on the utterance data (speech), and performs processing such as dividing the video, speech, and speech-recognized character strings into sentences (step S12).

[0107] The control unit 11 inputs the speech data (spoken sentence, spoken voice, and / or video) into the proficiency assessment model 40, thereby acquiring the user's proficiency in the target language from the proficiency assessment model 40 (step S13). Specifically, the control unit 11 acquires the probability values ​​that the user's proficiency belongs to each of multiple proficiencies from the proficiency assessment model 40, and calculates a value x' by normalizing the weighted sum x of the probability values.

[0108] The control unit 11 calculates the contribution of each word or phrase of the utterance sentence (each element of the utterance data) to the proficiency level output from the proficiency assessment model 40 (step S14). Specifically, the control unit 11 calculates the contribution (e.g., SHAP value) of each word or phrase of the utterance sentence to each of the proficiency levels A1 to C2 that can be output from the proficiency assessment model 40.

[0109] The control unit 11 extracts a first utterance sentence (first utterance data) from the sentence (utterance data) in the target language uttered by the user, which will be used as input for subsequent generation of a second utterance sentence and strength evaluation (step S15).

[0110] The control unit 11 executes a subroutine to generate second utterance sentences (second utterance data) by changing the expressions of the extracted first utterance sentences (first utterance data) in the target language by the user to expressions that are one level higher than the user's proficiency (step S16). Specifically, the control unit 11 generates a plurality of second utterance sentences by changing the expressions of the first utterance sentences, calculates an EoVL (first score) that represents the learning efficiency of the target language by presenting (outputting) each piece of second utterance data based on the user's proficiency, etc., and determines the second utterance sentences to be presented to the user according to the EoVL.

[0111] The control unit 11 executes a subroutine to evaluate the strength of the user's proficiency in the target language in the first utterance sentence (first utterance data) based on the user's proficiency level, etc. (Step S17). Specifically, the control unit 11 calculates, for each first utterance sentence, a SoVC (second score) indicating the degree to which the strength of the user's proficiency in the target language is expressed, and determines whether to present (output) each first utterance sentence as a strength of the user based on the SoVC.

[0112] The control unit 11 outputs feedback for learning the target language to the terminal 2 and displays it (step S18). Specifically, the control unit 11 displays, on the terminal 2, the second uttered sentence having a higher EoVL and information (meanings) about phrases (elements) that are included in the second uttered sentence and have a proficiency level that is one level higher than the user's proficiency, as shown in Fig. 6 and Fig. 7. In addition, the control unit 11 displays, on the terminal 2, the first uttered sentence having a higher SoVC and information about phrases that are included in the first uttered sentence and have a proficiency level that is one level higher than the user's proficiency, as shown in Fig. 8.

[0113] The control unit 11 stores the words (vocabulary) used by the user in the used vocabulary DB 91, and stores the second uttered sentence output (presented) as the user's weakness, the first uttered sentence output as the user's strength, and the words contained therein in the feedback vocabulary DB 94 (step S19). Furthermore, the control unit 11 also stores information in the feedback vocabulary DB 94 that identifies which user spoke the first uttered sentence before the second uttered sentence output as the user's weakness was paraphrased, and which uttered sentence in which dialogue the first uttered sentence output as the user's strength was spoken (when and by whom). The control unit 11 also refers to the used vocabulary DB 91, and updates the vocabulary dictionary DB 92 that associates words (meanings of words) with proficiency levels based on the results of tallying the number of occurrences of words (elements) spoken by each user by user proficiency level and meaning (step S20). The control unit 11 then terminates the series of processes. For more efficient processing, the subroutines for generating the second utterance sentence and evaluating the strength may be executed in parallel. Alternatively, fragments of utterance data may be sequentially acquired in streaming format while the system is interacting with the user, and the subroutines for generating the second utterance sentence and evaluating the strength may be executed in real time.

[0114] FIG. 10 is a flowchart showing the processing steps of the subroutine for generating second utterance sentences. The processing content of the subroutine in step S16 will be described with reference to FIG. 10 . The control unit 11 of the server 1 generates multiple second utterance sentences (second utterance data) by changing the expressions (phrases) of each of the user's first utterance sentences (step S31). Specifically, the control unit 11 inputs a prompt including the first utterance sentence to the LLM 30, causing the LLM 30 to generate multiple second utterance sentences. The control unit 11 identifies the meaning of each phrase included in each of the generated second utterance sentences (step S32).

[0115] The control unit 11 calculates a first index value that indicates how likely the first uttered sentence is to represent the user's proficiency based on the contribution of each word or phrase of the uttered sentence calculated in step S14 (step S33). For example, when the user's proficiency is L, the control unit 11 calculates the contribution φ of each word or phrase of the first uttered sentence to the proficiency L. k (L) , and the contribution of each phrase φ is calculated based on the formula (5). k(L) The first index value f １ Calculate.

[0116] The control unit 11 calculates a second index value that indicates how far the first uttered sentence does not reach a proficiency level higher than the user's proficiency level, based on the contribution of each word or phrase of the first uttered sentence calculated in step S14 (step S34). For example, when the user's proficiency level is L, the control unit 11 calculates a second index value that indicates how far the first uttered sentence does not reach a proficiency level higher than the user's proficiency level L+1, based on the contribution of each word or phrase of the first uttered sentence calculated in step S14 (step S34). k (L+1) , and the contribution of each phrase φ is calculated based on the formula (6). k (L+1) The second index value f ２ Calculate.

[0117] The control unit 11 calculates the similarity between the first uttered sentence of the user and each of the generated second uttered sentences as a third index value (step S35). For example, the control unit 11 calculates the similarity between the two texts (the first uttered sentence and the second uttered sentence) as a third index value f ３ It is calculated as follows.

[0118] The control unit 11 calculates a fourth index value that indicates how colloquial each of the generated second utterance sentences is (step S36). Specifically, the control unit 11 converts the second utterance sentence into a third utterance sentence that has been converted into a colloquial expression by inputting the second utterance sentence into a machine learning model that has been trained to convert input sentences into colloquial sentences. Then, the control unit 11 calculates the similarity between the second utterance sentence and the third utterance sentence as a fourth index value f ４ It is calculated as follows.

[0119] The control unit 11 refers to the vocabulary dictionary 92 and calculates a fifth index value that indicates how many words (elements) with a proficiency level higher than the user's proficiency level are included in the second uttered sentence (step S37). Specifically, the control unit 11 assigns points to each word in the second uttered sentence so that the score of that word is high when the word has a proficiency level L+1 that is one level higher than the user's proficiency level L, according to formula (10), and calculates the fifth index value f by taking the cumulative total of the scores of each word according to formula (9). ５ Calculate.

[0120] The control unit 11 calculates the EoVL of each second utterance sentence based on the first to fifth index values ​​calculated in steps S33 to S37 (step S38). The control unit 11 extracts a predetermined number of second utterance sentences with the highest calculated EoVLs (step S39). The control unit 11 refers to the vocabulary dictionary 92 and acquires, from the vocabulary dictionary 92, information (meanings) about words and phrases included in the highest ranked second utterance sentences that have a proficiency level one level higher than the user's proficiency level (step S40). The control unit 11 returns the processing result.

[0121] FIG. 11 is a flowchart showing the processing procedure of the strength evaluation subroutine. The processing content of the subroutine in step S17 will be described with reference to FIG. 11. The control unit 11 of the server 1 identifies the meaning of each word in the first utterance sentence by the user (step S51). The control unit 11 calculates a sixth index value indicating how likely the first utterance sentence is to have a higher proficiency level than the user's proficiency level, based on the contribution of each word in the utterance sentence calculated in step S14 (step S52). Specifically, when the user's proficiency level is L, the control unit 11 calculates the contribution φ of each word in the first utterance sentence to a proficiency level L+1 that is one level higher than the user's proficiency level L. k (L+1) Then, the control unit 11 calculates the contribution φ of each word. k (L+1) By taking the cumulative value of １ Calculate.

[0122] The control unit 11 refers to the vocabulary dictionary 92 and calculates a seventh index value that indicates how many words (elements) with a proficiency level higher than the user's proficiency level are included in the first utterance sentence (first utterance data) (step S53). Specifically, the control unit 11 assigns points according to formula (15) so that a word or phrase in the first utterance sentence has a high score when the word or phrase has a proficiency level L+1 that is one level higher than the user's proficiency level L, and calculates the seventh index value g by taking the cumulative total of the scores of each word or phrase according to formula (13). ２ Calculate.

[0123] The control unit 11 calculates an eighth index value representing the proficiency of the user's utterance (step S54). As described above, the control unit 11 acquires a probability value of the proficiency of the user's utterance regarding the proficiency of the user's utterance by inputting each of the user's first utterance sentences extracted by the first utterance sentence extraction unit into a machine learning model (proficiency determination model 40) that has been trained to output a probability value of the user's proficiency regarding the proficiency of the user's utterance when an utterance sentence (utterance data) is input. The control unit 11 normalizes the weighted sum x of the acquired probability values ​​according to formulas (1) and (3) to obtain an eighth index value g ３ It is calculated as follows.

[0124] The control unit 11 calculates the SoVC for each first utterance sentence by the user based on the sixth to eighth index values ​​calculated in steps S52 to S54 (step S55). The control unit 11 extracts a predetermined number of first utterance sentences with the highest calculated SoVCs (step S56). The control unit 11 refers to the vocabulary dictionary 92 and obtains information (meanings) about words and phrases included in the highest ranked first utterance sentences that are one level higher in proficiency than the user's proficiency (step S57). The control unit 11 returns the processing result.

[0125] In the above, the EoVL is calculated for a spoken sentence (text) and fed back as a weakness of the user, but the "speech data" to be processed is not limited to text and may be audio.

[0126] That is, the server 1 may determine the accuracy of pronunciation, etc. as a level of proficiency from the user's speech in the target language, generate second speech data (second speech data) by extracting all or part of one or more first speech sounds (first speech data) from the user's speech, changing them to correct pronunciation, calculate a first score representing the learning efficiency of the target language by presenting (playing back) the second speech sounds, and output the second speech sounds to the user according to the first score.

[0127] A model such as ELSA can be used as a model for evaluating the accuracy of pronunciation. The proficiency assessment model 40 can also extract features of speech to determine the CEFR level of pronunciation (phonology). The server 1 can assess the accuracy (proficiency) of pronunciation for each sentence, word, phoneme, etc., based on the output from these models, and build a database (a pronunciation version of the vocabulary dictionary 92) that associates the pronunciation of each word with the proficiency level.

[0128] The server 1 generates a second speech speech from the user's speech by correcting pronunciation, correcting grammatical errors, and removing hesitations and fillers. For example, if the user utters "I I'd like um I'd like to go London" and the pronunciation of the "'d" in "I'd" is poor, the server 1 corrects the pronunciation as well as the pronunciation to generate a second speech speech "I'd like to go to London."

[0129] The server 1 then refers to the user's proficiency and the database to calculate a first score (conveniently referred to as EoPL (Efficiency of Pronunciation Learning)) that indicates the learning efficiency achieved by presenting the second utterances. The EoPL can be defined by taking into account not only pronunciation but also grammatical accuracy and fluency. The server 1 ranks the second utterances according to the EoPL, and outputs and plays back a predetermined number of the top-ranked second utterances to the terminal 2. Explaining in accordance with the above example, for example, the terminal 2 plays back the second utterance "I'd like to go to London," while displaying text highlighting noteworthy parts (parts with poor pronunciation) and their phonetic symbols. Similarly, text highlighting corresponding parts before and after grammatical error correction may be displayed.

[0130] In this way, the "utterance data" to be processed is not limited to text, but may be audio. Furthermore, the second utterance data may include not only phrase rephrasing but also grammatical error correction, pronunciation correction, and paraphrasing of discourse-level expressions spanning multiple sentences (such as rearranging sentence order to create a logical sentence structure or providing guidance on appropriate use of conjunctions).

[0131] As described above, according to this embodiment, advice suited to the user's level of proficiency in the target language can be presented.

[0132] The embodiments disclosed herein are illustrative in all respects and should not be considered limiting. The scope of the present invention is defined by the claims, not by the above meaning, and is intended to include all modifications within the meaning and scope of the claims.

[0133] The matters described in each embodiment can be combined with each other. Furthermore, the independent claims and dependent claims described in the claims can be combined with each other in any combination, regardless of the reference format. Furthermore, the claims use a format in which a claim references two or more other claims (multiple claim format), but this is not limited to this. A multiple claim (multi-multi claim) that references at least one other multiple claim may also be used.

[0134] DESCRIPTION OF SYMBOLS 1 Server (information processing device) 11 Control unit 12 Main memory unit 13 Communication unit 14 Auxiliary memory unit P Program 40 Proficiency determination model 50 Language ability diagnosis unit 51 Preprocessing unit 52 Feature extraction unit 53 Proficiency determination unit 54 Contribution calculation unit 60 First utterance sentence extraction unit 70 Second utterance sentence generation unit 80 Strength evaluation unit 91 Used vocabulary DB 92 Vocabulary dictionary 93 User information DB 94 Feedback vocabulary DB 2 Terminal 3 Generation server 30 LLM

Claims

1. An information processing method in which a computer executes the following processes: acquire first utterance data in a target language by a user; acquire the user's proficiency in the target language; generate second utterance data by modifying the expression of the first utterance data; calculate a first score indicating the learning efficiency of the target language by presenting the second utterance data based on the user's proficiency; and output the second utterance data to the user according to the first score.

2. An information processing method as described in claim 1, which refers to a database that associates each element of speech data with a level of proficiency, and outputs information regarding elements contained in the second speech data that have a higher level of proficiency than the user's level of proficiency.

3. The information processing method according to claim 2, wherein the database is updated based on a result of tallying the number of occurrences of each element included in the first utterance data of each user by the level of proficiency of the user.

4. The information processing method of claim 1, wherein the acquired first utterance data is input into a model that has been trained to output the user's proficiency when the first utterance data is input, thereby obtaining the user's proficiency from the model; calculating the contribution of each element of the first utterance data to the proficiency output from the model; and calculating the first score based on the calculated contribution of each element.

5. An information processing method as described in claim 4, which calculates the contribution of each element of the first utterance data to a proficiency level that is the same as the user's proficiency level, calculates a first index value that indicates how likely the first utterance data is to the user's proficiency based on the calculated contribution of each element, and calculates the first score based on the first index value.

6. An information processing method as described in claim 4, which calculates the contribution of each element of the first utterance data to a proficiency level higher than the user's proficiency level, calculates a second index value indicating how far the first utterance data does not reach a proficiency level higher than the user's proficiency level based on the calculated contribution of each element, and calculates the first score based on the second index value.

7. An information processing method as described in claim 1, wherein the similarity between the first speech data and the generated second speech data is calculated as a third index value, and the first score is calculated based on the third index value and the user's proficiency.

8. The information processing method of claim 1, further comprising: converting the generated second speech data into third speech data converted into colloquial expression; calculating the similarity between the second speech data and the third speech data as a fourth index value; and calculating the first score based on the fourth index value and the user's proficiency.

9. The information processing method of claim 1, further comprising: referring to a database that associates each element of speech data with a level of proficiency, calculating a fifth index value that indicates the degree to which the second speech data contains elements of a higher level of proficiency than the user's level of proficiency; and calculating the first score based on the fifth index value.

10. An information processing method as described in claim 1, further comprising: calculating a second score indicating the degree to which the first speech data reflects the user's strengths in the user's operational ability in the target language based on the user's proficiency; and outputting the first speech data to the user according to the second score.

11. An information processing method as described in claim 10, which refers to a database that associates each element of speech data with a level of proficiency, and outputs information regarding elements contained in the first speech data that have a higher level of proficiency than the user's level of proficiency.

12. An information processing method as described in claim 10, comprising: inputting the acquired first utterance data into a model that has been trained to output the user's proficiency when the first utterance data is input, thereby obtaining the user's proficiency from the model; calculating the contribution of each element of the first utterance data to the proficiency output from the model; calculating a sixth index value that indicates how likely the first utterance data is to represent a proficiency level higher than the user's proficiency based on the calculated contribution of each element; and calculating the second score based on the sixth index value.

13. An information processing method as described in claim 10, which calculates a seventh index value indicating the extent to which the first utterance data contains elements of a higher proficiency level than the user's proficiency level by referring to a database that associates each element of utterance data with a proficiency level, and calculates the second score based on the seventh index value.

14. An information processing method as described in claim 10, wherein the acquired first utterance data is input into a model that has been trained to output probability values ​​that the user's proficiency level belongs to each of multiple proficiency levels when the first utterance data is input, thereby obtaining the probability values ​​from the model; calculating a normalized value of the weighted sum of the probability values ​​as an eighth index value; and calculating the second score based on the eighth index value.

15. The information processing method according to claim 1, wherein the second utterance data is generated by inputting a prompt including the first utterance data into a language model.

16. An information processing device having a control unit, wherein the control unit: acquires first utterance data in a target language by a user; acquires the user's proficiency in the target language; generates second utterance data by modifying the expression of the first utterance data; calculates a first score indicating the learning efficiency of the target language by presenting the second utterance data based on the user's proficiency; and outputs the second utterance data to the user according to the first score.

17. A program that causes a computer to execute the following processes: acquire first utterance data in a target language by a user; acquire the user's proficiency in the target language; generate second utterance data by modifying the expression of the first utterance data; calculate a first score indicating the learning efficiency of the target language by presenting the second utterance data based on the user's proficiency; and output the second utterance data to the user according to the first score.

Images