Information processing method and program to support conversational learning

Abstract

It provides information processing to support more effective conversational learning. [Solution] Using an information processing device, data of the user's voice in a language conversation with the user using artificial intelligence is acquired, and an evaluation result is output that evaluates the language expressions spoken by the user in the conversation based on the acquired voice data. The language expressions selected by the user from the output evaluation result are stored in the memory as stock information for later learning. After the stock information is stored in the memory, data of the user's voice in a language conversation with the user using artificial intelligence is acquired, and using the acquired voice data and the stored stock information, it is determined whether the expression from the stock information was used in the language conversation with the user using artificial intelligence. If it is determined that the expression from the stock information was used, evaluation information that the expression from the stock information was used is output.

Description

Technical Field

[0001] The present invention relates to an information processing method and a program for assisting the learning of a user's conversation in a certain language.

Background Art

[0002] Conventionally, English conversation support technologies using artificial intelligence have been proposed. These technologies acquire the speech content of the user as text data by speech recognition, and analyze the text meaning and emotions by natural language processing. Then, based on the analysis results, appropriate English expressions and response sentences are generated using a large language model or the like, and are output with natural pronunciation by speech synthesis technology.

[0003] Patent Document 1 discloses a technique for analyzing the pronunciation data of a user and reporting the habits, weaknesses, and alternative expressions of each speaking style.

Prior Art Documents

Patent Documents

[0004] <s

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] The method described in Patent Document 1 only proposes weaknesses and alternative expressions, and does not provide an effective conversation learning method.

[0006] Therefore, an object of the present invention is to provide information processing for assisting more effective conversation learning.

Means for Solving the Problems

[0007] One aspect of the present invention for solving the above problems is an information processing method for supporting a user's learning of conversation in a certain language, wherein an information processing device is used to acquire data of the user's voice in a conversation with the user in the said language using artificial intelligence, and the acquired voice of Based on the data, the system outputs an evaluation result that assesses the linguistic expressions spoken by the user in the conversation. The expressions selected by the user from the output evaluation result are stored in the memory unit as stock information for later learning. After the stock information is stored in the memory unit, the system acquires the user's voice data from the language conversation with the user using the artificial intelligence. Using the acquired voice data and the stored stock information, the system determines whether the expressions from the stock information were used in the language conversation with the user using the artificial intelligence. If the determination determines that the expressions from the stock information were used, the system outputs evaluation information indicating that the expressions from the stock information were used. Furthermore, as the linguistic expression in the evaluation result, it outputs a similar alternative expression to the linguistic expression uttered by the user in the conversation, or the correct expression. It is characterized by the following: [Effects of the Invention]

[0008] According to the present invention, it is possible to provide information processing that supports more effective conversational learning. [Brief explanation of the drawing]

[0009] [Figure 1] This is a diagram showing the configuration of an information processing system. [Figure 2] This diagram shows the configuration of an information processing terminal. [Figure 3] This is a diagram illustrating a flowchart of an information processing method. [Figure 4] This is a diagram showing a GUI screen. [Figure 5] This is a diagram showing a GUI screen. [Figure 6] This diagram shows the screen for saving expressions you want to remember. [Figure 7] This figure shows a screen displaying the evaluation results regarding the use of stock information representations. [Figure 8]This figure shows a screen displaying a list of stock information. [Modes for carrying out the invention]

[0010] <Details of the embodiment> This embodiment describes an information processing system for providing an application (software) to support conversational learning in a predetermined language. As an example of use, it is used to support a user whose native language is a first language in learning conversational second language, which is different from the first language. The predetermined language is not particularly limited and may be any language such as Japanese, English, Chinese, Korean, German, or French.

[0011] Figure 1 shows a diagram of the information processing system in this embodiment. This system comprises an information processing terminal (information processing device) 1, a control server (information processing device) 2, and an AI server (information processing device) 3. These are connected by a network 4. Information processing terminal 1 is a terminal used by a particular user. In addition to the information processing terminal used by user A, multiple information processing terminals used by other users, such as a terminal used by user B, can be connected to network 4.

[0012] In this system, the server controls and manages the data, while the application (client) on the information processing terminal displays that data. On the server side (backend), data is managed in a database, API endpoints are provided, data updates and calculations are performed, data is sent to the client, the display of the application screen is controlled, and security is controlled. On the client side (application), requests are sent to the server's API, the data received from the server is displayed on the screen, a user interface (UI) is provided, and temporary data caching is performed.

[0013] An information processing terminal 1 (hereinafter referred to as terminal 1) is installed with software and application programs for assisting in the conversation learning of a predetermined language. The program is installed by downloading it from a server connected to the terminal 1 via the network 4. The terminal 1 transmits the information input by the user to the terminal 1 and the acquired voice data to the control server 2 via the network 4. For example, the terminal 1 acquires voice data spoken by the user using a microphone or the like and transmits the information to the control server 2 via the network 4. Also, the terminal 1 receives (acquires) the information output by the control server 2 via the network 4 and displays information or an image based on the received information.

[0014] Next, the configuration of the terminal 1 will be described. The terminal 1 has a computer installed in a device such as a personal computer, which is a general-purpose computer, a smartphone, or a tablet. The terminal 1 causes a display device (display screen) to display various screens by means of the installed software and applications (programs).

[0015] FIG. 2 shows a configuration diagram of the terminal 1. The terminal 1 includes a processing unit 11, a memory 12, a storage 13, a communication unit 14, an input unit 15, and a display unit 16. These are electrically connected to each other through a bus 17. The bus 17 is commonly connected to each of the above elements and transmits, for example, address signals, data signals, and various control signals.

[0016] The processing unit 11 is an arithmetic unit that controls the overall operation of the terminal 1, controls the transmission and reception of data between each part, and performs information processing necessary for program execution and authentication processing. The processing unit 11 includes, for example, an arithmetic processing unit such as a CPU, a GPU, or an FPGA, and executes various information processes described later by executing programs and the like stored in the storage 13 and expanded in the memory 12.

[0017] The memory 12 (storage unit) includes a main memory composed of a volatile storage device such as a DRAM, and an auxiliary memory composed of a non-volatile storage device such as a flash memory or an HDD. The memory 12 is used as a work area of the processing unit 11 and stores the BIOS executed at the startup of the terminal 1 and various setting information.

[0018] The storage 13 (storage medium) stores various programs such as applications and programs. A database storing data used for each process is constructed in the storage 13.

[0019] The communication unit 14 connects the terminal 1 to the network 4. The communication unit 14 communicates with external devices directly or via a network access point by means such as a wired LAN, a wireless LAN, Wi-Fi (Wireless Fidelity, registered trademark), infrared communication, Bluetooth (registered trademark), short-distance or non-contact communication. The input unit 15 is an information input device such as a keyboard, a mouse, or a touch panel.

[0020] The display unit (display device) 16 includes a display that displays information obtained by calculation by the processing unit 11 or information received from the outside by the communication unit 14. As the display, an organic EL display, a liquid crystal display, etc. can be used. The display unit 16 provides a graphical user interface (GUI) that displays various information on its screen. The display unit 16 is not limited to being provided integrally with the terminal 1, and may be a display (display device) provided separately from the terminal 1 and connected to the terminal 1.

[0021] Next, the configuration of the control server 2 will be described. The control server 2 includes a computer having a processing unit, a memory (storage unit), a storage (storage medium), and a communication unit. Each unit is electrically connected to each other through a bus. The control server 2 communicates with the terminal 1 connected by the network 4 and controls the display of the display device by executing the installed software program.

[0022] The processing unit of the control server 2 is an arithmetic unit that controls the operation of the entire control server 2, controls the transmission and reception of data between each part, and performs information processing necessary for program execution and authentication processing. The processing unit includes an arithmetic processing unit, such as a CPU, GPU, or FPGA, which is stored in storage and executes programs loaded into memory to perform various information processing and display control as described later. The memory of the control server 2 includes main memory composed of volatile storage devices such as DRAM, and auxiliary storage composed of non-volatile storage devices such as flash memory or HDD. The memory is used as a work area for the processing unit. The memory and storage of the control server 2 store various programs. In addition, a database storing data used for each process may be built in this storage, or it may be configured to be connectable to a separate database server.

[0023] The communication unit of the control server 2 connects the control server 2 to the network 4. The communication unit communicates with external devices directly or via a network access point using various communication methods. This communication unit receives information transmitted from terminal 1 via the network 4. The received information is stored in memory and used for information processing performed by the processing unit, or stored in storage. The control server 2 receives (acquires) information transmitted by terminal 1 via the network 4, performs various information processing based on the received information, and stores the information in memory or storage. Furthermore, the control server 2 controls the communication unit to send the results of the information processing, or the information stored in memory or storage, to terminal 1 in response to a request from terminal 1.

[0024] Control Server 2 is an information processing device (evaluation device) that receives information from both Terminal 1 and AI Server 3 and processes information based on the received information. For example, Control Server 2 receives voice data spoken by the user from Terminal 1, and based on the received information, it performs processing to convert the voice data into text data or to evaluate the content of the user's speech. Control Server 2 transmits information to AI Server 3 via Network 4, receives information output from AI Server 3, and processes information based on the received information.

[0025] AI Server 3 is a server equipped with artificial intelligence (AI). Control Server 2 transmits information about the user's utterance to AI Server 3 via Network 4. Upon receiving this information, AI Server 3 inputs the user's utterance into its pre-trained large-scale language model and other machine learning models, and outputs response sentence data suitable for responding to the user's utterance. The machine learning model also outputs the results of its evaluation of the user's utterance when it receives the user's utterance information. The evaluation results output include an evaluation of the utterance, an evaluation of the grammar of the sentences in the utterance, and suggestions of alternative expressions. In addition to these, AI Server 3's pre-trained large-scale language model can output data that meets the conditions when a prompt (output condition) and the necessary data are input.

[0026] Next, the configuration of AI Server 3 will be described. AI Server 3 has a processing unit, memory (storage unit), storage (storage medium), and communication unit. These are electrically connected to each other via a bus. The processing unit is a computing device that controls the operation of the entire AI Server 3, controls the transmission and reception of data between each unit, and performs information processing necessary for program execution and authentication processing. The processing unit includes a computing device such as a CPU, GPU, or FPGA, and executes programs stored in storage and loaded into memory to perform various information processing described later.

[0027] The storage of AI Server 3 stores various programs such as applications and other programs. A database containing data used for each process is built in this storage. Machine learning models, such as trained large-scale language models, are stored and remembered in the storage or memory. The communication unit connects AI Server 3 to Network 4. This communication unit receives information transmitted from Terminal 1 or Control Server 2 via Network 4. The received information is stored in memory and used for information processing performed by the processing unit, or stored in storage.

[0028] Next, we will explain an information processing method for supporting conversational learning of a given language using the information processing system described above. Figure 3 shows a flowchart of this information processing method. First, terminal 1 launches a conversational learning application (software) for conversational learning in response to input from the user (S1). When the application is launched, terminal 1 sends launch information to control server 2 indicating that the application on terminal 1 has been launched. The processing unit of control server 2 checks the user settings in response to receiving the launch information. Subsequently, it loads the necessary models and language resources (dictionary data, speech synthesis model, UI, etc.) for the application, checks user authentication and previous conversation history, and identifies the learning state and language level. After that, the processing unit generates an initial prompt to determine the content of the conversation sentence to be displayed first, based on the time, user attributes, past conversation trends, etc.

[0029] Information from the generation request is sent to AI Server 3, input into the machine-learned model by the processing unit of AI Server 3, and the machine-learned model outputs information about the response sentences to present to the user. The response sentences presented to the user are sentences in the target language that mean, for example, "Good morning," or "How are you?", but there may be more sentences and a wider variety of sentences. The AI ​​Server 3 sends the information of the outputted response sentences to Control Server 2, and the processing unit of Control Server 2 formats the received response sentences, adjusts the grammar, punctuation, and display layout, and then controls the display to show them in the conversation field on the initial screen of Terminal 1 (S2). At the same time, if the voice output setting is enabled on Terminal 1, Terminal 1 converts the same sentences into speech using speech synthesis processing and plays the audio from Terminal 1's speaker or connected earphones. Terminal 1 displays the initial conversation sentences on the screen, preparing the user to continue speaking or inputting. This completes the series of processes from app launch to the start of a natural conversation.

[0030] Next, the user listens to the audio data output by terminal 1 or looks at the conversation text displayed on the screen of terminal 1 and speaks to terminal 1 accordingly. Terminal 1 uses its microphone to acquire the user's audio data (S3). Terminal 1 then sends the acquired audio data to control server 2.

[0031] The control server 2 receives audio data transmitted from terminal 1 via network 4, and its processing unit performs noise reduction and extraction of audio features, analyzing the audio features and converting them into text data (S4). The control server 2 can also evaluate the accuracy of pronunciation and intonation based on the audio features. Furthermore, the processing unit of the control server 2 uses natural language analysis to understand the syntax, meaning, and intent of the converted text data and identify the context of the conversation. The control server 2 transmits the information obtained from this speech recognition and language analysis to the AI ​​server 3.

[0032] AI Server 3 inputs the information obtained from the received speech recognition and language analysis into a machine learning model, which generates a natural response sentence (S5). AI Server 3 sends the generated response sentence to Control Server 2. Control Server 2 receives the generated response sentence, inputs it into a speech synthesis model, and generates speech data with natural pronunciation and intonation (S6). Control Server 2 sends the generated speech data of the response sentence to Terminal 1, which plays the received speech data and outputs it through Terminal 1's speaker or connected earphones.

[0033] The user speaks to terminal 1 in response to the audio data output by terminal 1 or the displayed conversation text. The control server 2 determines whether to repeat the conversation (S7). If it determines to repeat the conversation, the information processing system repeats the information processing in S3 to S7 in response to the user's utterance, allowing the user to continue the conversation. For example, when a predetermined time has elapsed since conversation learning started on terminal 1 and the conversation response has ended, the control server 2 sends a control instruction to terminal 1 to end the conversation, and conversation learning ends in the application. Alternatively, the conversation may end when a predetermined number of words have been spoken by the user, that is, when the number of words extracted from the text data of what the user has uttered reaches a predetermined number. The conversation may also end when the user selects an end button or the like on the application screen of terminal 1.

[0034] This information processing system includes an evaluation mode for evaluating user conversations. The control server 2 stores and accumulates text data (converted from the user's speech audio to text), pronunciation evaluation results, conversation history, and vocabulary usage history in a memory unit, associating them with each other. The processing unit of the control server 2 can analyze the accumulated data stored in the memory unit and adjust response generation parameters for subsequent interactions based on each user's vocabulary level and pronunciation tendencies.

[0035] After the conversation ends, the control server 2 evaluates the linguistic expressions (sentences, phrases, words, etc.) spoken by the user during the conversation as a review of the conversation, and controls the display to show the evaluation results on the screen of terminal 1 in order to provide feedback to the user. The evaluation results may include, for example, the text of the content output from terminal 1 (information processing system) during the conversation, the text of the user's response to that, the number of words spoken by the user, and the evaluation of the user's conversation content, all of which are displayed on the screen of terminal 1. As part of the evaluation of the user's conversation content, the control server 2 may display alternative expressions or words that are similar in meaning to the sentences, phrases, or words spoken by the user, or it may display correct expressions if there are errors in the sentences, phrases, or words spoken by the user.

[0036] The evaluation of the user's conversation content is performed using AI Server 3. Control Server 2 sends the text data of the user's conversation and output conditions (prompts) such as suggesting similar or alternative expressions as evaluation policies to AI Server 3. AI Server 3 inputs the received text data of the user's conversation and output conditions into a machine learning model. The machine learning model outputs expressions similar to or alternative to the sentences spoken by the user, according to the input output conditions. In addition, by inputting the text data of the user's conversation and output conditions such as error detection and suggestion of correct expressions into the machine learning model, the machine learning model can also output error detection and correct expressions for the sentences spoken by the user, according to the output conditions. AI Server 3 sends the information output by the machine learning model to Control Server 2, and Control Server 2 sends the information output by the machine learning model to Terminal 1 and controls the display to show that information on the screen of Terminal 1. The user can check similar or alternative expressions and correct expressions for what they have spoken on the screen of Terminal 1.

[0037] Figures 4 and 5 show examples of graphical user interfaces (GUIs) displayed on the display unit of terminal 1. In screen M10 shown in Figure 4, the control server 2 controls the display on terminal 1 to show the following information as a review of the conversation. In screen M10, the sentences presented to the user in the AI-assisted conversation are shown in column M1, and the number of words spoken by the user is shown in column M2. In addition, the content of what the user said in response to the conversation sentences in column M1 and their evaluation are shown in column M3. In column M3, the name "Conversation Review" and the evaluation title "Expand your range of expression" are displayed, and the sentences actually spoken by the user are displayed as text under "Your expression". As an indication of alternative expressions, for example, for the text part "〇〇", it is displayed as "The expression 〇〇 is also used as ××." Similar or alternative expressions to the user's utterances are also shown in column M4. In column M4, the memory button B, which will be described later, corresponding to the expression displayed in this column is displayed. Figure 5 shows a screen that displays another evaluation result. In the screen M20 shown in Figure 5, in addition to columns M1 and M2, column M5 displays the content of what the user uttered and its evaluation (errors in expression). Column M5 displays the title "Conversation Review," the evaluation title "Improve to more accurate expression," and the actual sentence uttered by the user under "Your Expression." The correct expression for the user's utterance is displayed in column M6. Column M6 displays the title "Improvement," the correct expression "△△···," and, for the user's utterance "〇〇," the message "〇〇 is used to mean ~, deviating from its original meaning. Generally, △△ is the basic expression." Although text is actually displayed on the screen, in the notation of each figure, symbols are used instead of actual characters for generalization. Column M6 also displays the memory button B, which corresponds to the expression displayed in this column, as described later.

[0038] Control server 2 displays similar or alternative expressions and the correct expression on terminal 1's screen, while also displaying memory buttons B for later learning. Memory buttons, located next to or within the area where each expression is displayed, correspond to each expression. When the user selects memory button B, terminal 1 sends information that memory button B has been selected to control server 2. Upon receiving this information, control server 2 controls the display on terminal 1 to show the text of the similar or alternative expression, or the correct expression, corresponding to the selected memory button.

[0039] Figure 6 shows screen N for saving expressions that the user wants to remember. On screen N, the similar or alternative expression, or the correct expression, selected by the memory button B is shown in column N1. Below that, the expression is separated into phrases or words and shown in column N2. Each phrase or word is displayed in column N2, and in Figure 6, for convenience, symbols are used to display them as "〇〇", "△△", "××", and "AAA". In this way, terminal 1 displays the text of the sentences containing similar or alternative expressions, or the correct expression, on its screen. As a unit of text display, one sentence can be divided into multiple phrases or words and displayed. The user can select one or more sentences, phrases, or words that they want to use in later learning from the sentences, phrases, or words displayed in column N2 on the screen. Terminal 1 sends the information of the sentences, phrases, or words selected by the user to the control server 2. The control server 2 stores the information of the sentences, phrases, or words selected by the user as stock information in its memory unit. The stock information (linguistic information) stored in the memory unit in this way is used to evaluate subsequent conversations. The sentences, phrases, or words stored in the memory unit for later learning are not limited to just one; they may include multiple items from a single sentence spoken by the user, one item from each of multiple sentences spoken by the user, or multiple items from each of multiple sentences.

[0040] After a previous conversation ends, for example after a predetermined amount of time or days have passed, the user can launch the conversation learning application on terminal 1, or select additional learning elements such as buttons or tabs on the screen within the running application, such as starting the next conversation, in order to perform additional conversation learning. During additional conversation learning, the information processing system performs the same processes as S1 to S7 described above. When conversation learning is completed, the control server 2 evaluates the conversation content. In this evaluation, the control server 2 determines whether one or more of the stock information that the user has selected as expressions they want to learn and has stored in the memory unit was used in the user's conversation. The control server 2 refers to the stock information stored in the memory unit and checks whether the text data of the user's conversation contains the stock information stored in the memory unit. If the text data of the user's conversation contains the stock information stored in the memory unit, the control server 2 sends the evaluation result that the expression of the stock information was used to terminal 1, and controls the terminal 1 to display the text of the stock information used by the user in the conversation and the evaluation result that the stock information was used on the terminal 1's screen. As an evaluation result for whether a stock information expression was used, for example, a message such as "Stock expression was successfully used" can be displayed. Outputting such a positive evaluation can increase the user's motivation to learn. Here, a stock expression refers to a linguistic expression that has been memorized as stock information.

[0041] Figure 7 shows screen M30, which represents the evaluation results of whether the stock information expressions were used. In screen M30 shown in Figure 7, in addition to columns M1 and M2, the content spoken by the user is shown in column M7, and the evaluation content regarding the use of the stock information is shown in column M8. In column M7, the name "Conversation Review" and the evaluation title "Let's check the expressions you were able to use" are displayed, and the sentence actually spoken by the user is displayed as text under "Your expressions". In column M8, under the evaluation "You were able to put stock expressions into practice", the expressions used by the user are displayed as the same sentence as "Your expressions" above, or some idioms and words.

[0042] In this way, the control server 2 evaluates whether pre-stored expressions are being used as stock information in the user's conversation, allowing the user to confirm that they were able to use the expressions they wanted to learn in a natural conversation. Conventionally, questions were provided to focus on learning incorrect expressions, but after answering the questions, the user would then focus on learning other expressions, learning expressions one by one in sequence. However, according to this embodiment, the user converses with expressions they have selected and set themselves in mind throughout a series of conversations, rather than being limited to just one set expression, resulting in a learning effect that broadens the range of expressions in a more natural conversation. As stock information is used naturally in conversational sentences, words and phrases become established in context, improving the ability to use expressions effectively.

[0043] Furthermore, even if there are multiple or numerous pre-stored expressions as stock information, the control server 2 can compare whether each expression exists in the text data of the user's conversation, and can also perform evaluation processing to determine if the user was able to use multiple expressions they wanted to learn in their conversation. This allows the user to efficiently evaluate many expressions simultaneously within a series of conversations, and by providing feedback of the evaluation results to the user using terminal 1, the user can receive more evaluations and enhance the learning effect. In addition, since the information stored as stock information is information selected by the user themselves, it can be expected to have the effect of encouraging self-directed learning.

[0044] The control server 2 can set and store one or more expressions from the stock information as conversation goals. Figure 8 shows a screen displaying a list of stock information. In the screen L shown in Figure 8, the language expressions L1 stored in the memory unit are displayed in order based on the user's selection. The control server 2 controls the terminal 1 screen to display the list of stock information, and the user can select one or more of the list of stock information (sentences, phrases, words) displayed on the terminal 1 screen as target expressions. Once a goal is set, the control server 2 controls the terminal 1 screen to display the target expressions before the conversation. Then, in the next conversation learning session, the control server 2 checks whether the target sentences, phrases, or words were used in the user's conversation, and displays an evaluation of whether the goal was achieved on the terminal 1 screen as an evaluation result of the user's conversation. Matching is performed by using terminal 1 to acquire audio data spoken by the user during a conversation provided by terminal 1, and by the control server 2 converting the audio data acquired from terminal 1 into text data and determining whether there is any data in that text data that matches the target sentence, phrase, or word.

[0045] In conventional AI-based conversational learning, the conversation content relies on automatic generation, resulting in few opportunities for learners to consciously use specific expressions. In this embodiment, users can pre-select sentences, phrases, or words they wish to use as target expressions, thus concretizing learning objectives and increasing their awareness of conversational participation and motivation to speak. Furthermore, the control server 2 compares the conversation log with the target expressions and evaluates whether or not they were used in actual conversations, allowing for quantitative visualization of learning outcomes. This enables users to immediately check their level of achievement and easily self-evaluate their learning progress. It is also possible to encourage continuous learning by scoring or badge-ifying the level of achievement.

[0046] Furthermore, control server 2 can reflect the achieved stock information in subsequent conversation generation control. For example, it inputs the stock information, information indicating whether or not the goal was achieved, and conversation generation conditions into the machine learning model of AI server 3 and outputs a conversation. This makes it possible to focus on presenting unachieved goal expressions or to reuse achieved expressions in applied contexts, enabling iterative learning tailored to each user's proficiency level. In addition, by storing the history of goal setting and achievement judgment in the memory unit, control server 2 can analyze long-term achievement trends and extract the user's weak areas. This allows for automatic optimization of the next conversation topic and presented vocabulary, supporting sustainable and planned improvement of conversational skills.

[0047] When initiating a conversation, control server 2 refers to the user's past conversation history and sends a recommended topic to terminal 1 to display on terminal 1's screen. There may be one or more recommended topics. If there are multiple topics, the user selects one from the topics displayed on the screen, and terminal 1 sends information about the selected topic to control server 2. Control server 2 sends the information about the selected topic to AI server 3, which inputs that information and the conditions for creating a response sentence corresponding to the topic into a machine learning model, and sends the information about the response sentence output by the machine learning model according to the topic to control server 2. Control server 2 sends the outputted response sentence to terminal 1 and controls its display to show it on terminal 1's screen.

[0048] When initiating a conversation, the control server 2 may refer to the stock information stored in the memory unit and send a recommended topic to terminal 1 to use at the start of the conversation. This recommended topic is one that the user has selected as an expression they want to learn, one of the stock information stored in the memory unit may be used, and a topic in which a conversation will occur in which the user responds with a sentence they want to express as learning. The AI ​​server 3 inputs the set topic and the conditions for creating a response sentence corresponding to that topic into the machine learning model, and outputs a response sentence corresponding to the topic from the machine learning model.

[0049] In this way, the control server 2 can automatically select conversation themes that are appropriate to the user's interests and learning progress by referring to past conversation history and stock information to suggest recommended topics. This makes it easier for the user to start a conversation naturally and facilitates a smooth introduction to dialogue for learning practical expressions. In addition, since topics that reuse vocabulary and grammatical structures used in the past can be selected, the review effect is enhanced and conversational skills are reinforced. Furthermore, the control server 2 selects topics that include expressions the user wants to learn based on the stock information in the memory unit, and the AI ​​server 3 generates response sentences corresponding to that topic, allowing learners to repeatedly use expressions they have selected in the past in actual conversations. This enables contextual learning rather than rote memorization, and allows for practical retention of expressions. In addition, since the expressions that the learner themselves has selected are reflected in actual conversations, learning satisfaction and motivation to continue are improved.

[0050] While preferred embodiments of this disclosure have been described in detail above with reference to the attached drawings, the technical scope of this disclosure is not limited to these examples. Furthermore, not all components shown in the embodiments are necessarily essential components of this disclosure.

[0051] The information processing method described above can also be executed by one or more information processing devices on which the program is installed, such as cloud services or distributed computers. Alternatively, the functions of AI server 3 may be implemented on control server 2, so that the functions can be realized with a single server and information processing terminal.

[0052] In the above-described embodiment, expressions were stored as stock information by the user's selection, but expressions to be stored as stock information may also be presented by the control server 2. For example, the control server 2 retrieves stock information previously selected or learned by the user from the storage unit and analyzes the features of the user's areas of interest, conversational tendencies, and vocabulary based on this. The processing unit generates a user's hobby and preference profile based on these analysis results. The hobby and preference profile consists of features such as topics frequently used by the user (e.g., travel, movies, sports, business, etc.), frequently occurring vocabulary groups, selected conversation topic history, search history, utterance frequency, and difficulty level. Based on this hobby and preference profile, the control server 2 extracts linguistic expressions (sentences, idioms, words, etc.) that frequently appear in the user's areas of interest and are not included in the current stock information. The extracted expressions are scored based on semantic relevance, frequency of use, learning level, and topic diversity, and the top candidates are determined as "stock candidate expressions".

[0053] The control server 2 sends these stock candidate expressions to terminal 1 and displays them in a list on terminal 1's screen. When displayed, each candidate expression is accompanied by a concise Japanese translation, an example sentence, and a "reason for recommendation" (its relevance to the user's interests and preferences). The user can select the expressions they want to remember from the presented candidates, and when terminal 1 detects the selection, the control server 2 receives the selection information and registers the selected expression as "stock information" in its memory. Furthermore, the control server 2 continuously learns the user's selection tendencies and automatically optimizes the difficulty level, topic field, and writing style level of the stock information candidates presented in subsequent instances. This allows the user to efficiently accumulate expressions that suit them by combining self-learning and automatic suggestions.

[0054] Furthermore, the information processing device may acquire stock information not only by obtaining information entered through an input operation selected by the user on the screen of terminal 1, but also by having the user, after viewing a screen displaying evaluation results of language expressions spoken by the user in a conversation, input the expression as stock information as text using the input means of terminal 1, and store the information processing device in the memory unit. Alternatively, the stock information may be stored by the user freely inputting the expression as stock information at will. For example, the user can input the expression as stock information using an input device such as terminal 1, and input the stock information while confirming the information entered on the display screen of the display device. The entered information is transmitted to the control server 2 and stored in the memory unit as stock information.

Claims

1. In an information processing method for supporting a user's learning of conversation in a certain language, Using an information processing device, The AI ​​acquires the user's voice data in a conversation with the user in the aforementioned language, Based on the acquired audio data, the system outputs an evaluation result that assesses the linguistic expressions spoken by the user during the conversation. The language expressions selected by the user from the output evaluation results are stored in the memory unit as stock information for later learning. After storing the stock information in the storage unit, the system acquires the user's voice data from the language conversation with the user using artificial intelligence, and uses the acquired voice data and the stored stock information to determine whether the expression in the stock information was used in the language conversation with the user using artificial intelligence. If the determination determines that the representation of the stock information was used, the system outputs information evaluating that the representation of the stock information was used. The method is characterized by outputting a similar alternative expression or a correct expression for the linguistic expression uttered by the user in the conversation, as the linguistic expression in the evaluation result.

2. The method according to claim 1, characterized in that the information processing device further acquires information of expressions entered as text by the user as stock information and stores it in the storage unit.

3. The information processing device displays the linguistic expression of the evaluation result on the screen of the display device. The method according to claim 1, characterized in that the language expressions are displayed such that the user can select the expression to be used as stock information from among the language expressions in the evaluation results displayed on the screen.

4. The method according to the previous invention, characterized in that the language expressions in the evaluation results displayed on the screen are displayed on the screen so that the user can select them word by word.

5. The method according to claim 1, characterized in that the information processing device outputs new stock information candidates based on the results of analyzing the stock information previously selected or learned by the user.

6. The memory unit stores multiple different representations as the stock information, The method according to claim 1, characterized in that it determines whether any of the expressions in the stock information were used in the user's conversation by comparing the audio data obtained in the user's conversation with each of the multiple different expressions in the stock information.

7. The method according to claim 1, characterized in that the information processing device sets an expression to be the target of a conversation from among the expressions of the stored stock information, performs a process to evaluate whether the expression set as the target was used in the user's conversation, and outputs the evaluation result that the expression set as the target was used in the user's conversation.

8. The method according to claim 1, characterized in that the information processing device sets a topic in a conversation based on the representation of the stored stock information.

9. A program for causing a computer to perform the method according to any one of claims 1 to 8.

10. In an information processing device for supporting a user's learning of conversation in a certain language, The processing unit, The AI ​​acquires the user's voice data in a conversation with the user in the aforementioned language, Based on the acquired audio data, the system outputs an evaluation result that assesses the linguistic expressions spoken by the user during the conversation. The language expressions selected by the user from the output evaluation results are stored in the memory unit as stock information for later learning. After storing the stock information in the storage unit, the system acquires the user's voice data from the language conversation with the user using artificial intelligence, and uses the acquired voice data and the stored stock information to determine whether the expression in the stock information was used in the language conversation with the user using artificial intelligence. If the determination determines that the representation of the stock information was used, the system outputs information evaluating that the representation of the stock information was used. The device is characterized in that, as a linguistic expression in the evaluation results, it outputs a similar alternative expression to the linguistic expression uttered by the user in conversation, or the correct expression.

Images