System

Abstract

A system is provided.SOLUTION: A system comprising: means for receiving a user's sentence input from a language learning device; means for translating the received user's sentence into a specified target language; means for correcting errors in the user's sentence; means for comparing the user's sentence and the corrected sentence to generate feedback; and means for providing the generated feedback to the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] With today's globalization, acquiring multiple languages ​​has become an important issue for individuals and companies. However, traditional language learning methods are inefficient and have problems maintaining learner motivation. Other problems include the accumulation of uncorrected learning errors and a lack of concrete feedback. To solve these problems, a system that supports language learning efficiently and accurately is needed. [Means for solving the problem]

[0005] The present invention provides a system that includes means for receiving a user's sentence input from a language learning device, translating the received user's sentence into a specified target language, means for correcting errors in the user's sentence, means for generating feedback by comparing the user's sentence with the corrected sentence, and means for providing the generated feedback to the user. This allows the user to efficiently and accurately progress with language learning and specifically identify their own weaknesses during the learning process, thereby overcoming the drawbacks of conventional language learning methods.

[0006] "Language learning device" is a general term for electronic devices and software that users use for language learning.

[0007] A "user's sentence" is a series of characters or sentences that a user inputs or creates via a language learning device.

[0008] A "target language" is a language that a user specifies as the language to learn or translate into.

[0009] A "translation means" is a system or algorithm that provides the functionality to convert a user's text into a target language.

[0010] An "error correction tool" is a system or algorithm that provides the functionality to identify errors in a user's writing and correct them to the correct form.

[0011] The "means for comparing and generating feedback" refers to a system or algorithm that has the function of comparing the user's input text with the corrected text and providing the user with useful information or advice.

[0012] The "means for providing feedback" refers to a system or device that has the function of displaying or notifying the user of the generated feedback information. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0014] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0015] First, the terms used in the following description will be explained.

[0016] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0017] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0018] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0019] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0020] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0021] [First embodiment]

[0022] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0023] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0024] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0025] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0026] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0027] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0028] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0029] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0030] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0031] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0032] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0033] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0034] The present invention is a system for assisting a user in language learning using a language learning device, which receives a user's text, translates the text into a specified target language, and provides feedback after correcting errors.

[0035] Program processing overview

[0036] Translation feature

[0037] The server translates the sentence entered by the user into the target language. When the user enters the sentence to be translated and the target language into the language learning device, it is sent to the server. The server uses a natural language processing model to translate the entered sentence into the specified target language. The translation result is sent from the server to the terminal, and the terminal displays the translation result to the user.

[0038] Specific examples

[0039] If a user wants to translate "Hello, how are you?" into Japanese, they input the sentence and "Japanese" into their terminal. The server receives this and translates it into "Hello, how are you?" using a natural language processing model. The result is sent to the user's terminal, and the terminal displays the translation result to the user.

[0040] Text correction function

[0041] The server corrects errors in sentences entered by the user. When a user enters a sentence to be corrected into a language learning device, it is sent to the server. The server uses a natural language processing model to correct the errors in the sentence. The server sends the correction results to the terminal, which then displays the correction results to the user.

[0042] Specific examples

[0043] If a user types "I went to the store," they enter that sentence into their terminal. The server receives this and uses a natural language processing model to correct it to "I went to the store." The result is sent to the user's terminal, which then displays the corrected result to the user.

[0044] Feedback function

[0045] The server compares the text entered by the user with the corrected text and provides feedback. If the user requests feedback on the corrections to their text, the feedback is sent to the server. The server compares the input text with the corrected text and generates feedback using a natural language processing model. This feedback is sent from the server to the terminal, which displays the feedback to the user.

[0046] Specific examples

[0047] If a user types "I went to the store," and then corrects it to "I went to the store," the user sends a request for feedback. The server receives this and uses a natural language processing model to generate feedback. For example, it generates feedback such as "'goed' is not a correct past tense. The correct form is 'went'." It then sends this feedback to the user's device, which then displays it to the user.

[0048] In this way, the present invention provides an environment in which users can learn languages ​​efficiently and accurately.

[0049] The processing flow will be explained below.

[0050] Translation feature

[0051] Step 1:

[0052] A user inputs a sentence to be translated and a target language through the interface of a language learning device.

[0053] Step 2:

[0054] The terminal receives the user's input and sends the data to the server.

[0055] Step 3:

[0056] The server receives the input data and initiates the translation process, specifically by calling a natural language processing model (e.g., a language model API).

[0057] Step 4:

[0058] The server sends a translation request to a natural language processing model, which translates it into the target language.

[0059] Step 5:

[0060] The server receives the translation result and returns it to the user terminal.

[0061] Step 6:

[0062] The terminal displays the translation result received from the server to the user.

[0063] Text correction function

[0064] Step 1:

[0065] The user inputs the sentence to be corrected through the interface of the language learning device.

[0066] Step 2:

[0067] The terminal receives the user's input and sends the data to the server.

[0068] Step 3:

[0069] The server receives the input data and initiates the sentence correction process, specifically by calling a natural language processing model (e.g., a language correction model API).

[0070] Step 4:

[0071] The server sends a request to generate a corrected sentence to the natural language processing model, and the model corrects the errors in the sentence.

[0072] Step 5:

[0073] The server receives the correction result and returns it to the user terminal.

[0074] Step 6:

[0075] The terminal displays the correction results received from the server to the user.

[0076] Feedback function

[0077] Step 1:

[0078] The user requests feedback on their input sentences and corrected sentences through the interface of the language learning device.

[0079] Step 2:

[0080] The terminal receives the user's request and sends the data to the server.

[0081] Step 3:

[0082] The server receives the request and starts the feedback generation process by calling a natural language processing model (e.g., a feedback generation model API).

[0083] Step 4:

[0084] The server sends a feedback generation request to the natural language processing model, and the model compares the user's input sentence with the corrected sentence and generates the feedback.

[0085] Step 5:

[0086] The server receives the generated feedback and returns it to the user terminal.

[0087] Step 6:

[0088] The terminal displays the feedback received from the server to the user.

[0089] Example 1

[0090] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0091] In conventional language learning systems, the accuracy and quality of the translation and correction results are often insufficient when translating user-input text into the target language and correcting errors. Furthermore, there is a lack of feedback to help users deepen their understanding of the language they are learning. This has resulted in a lack of an environment for users to learn languages ​​efficiently and accurately.

[0092] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0093] In this invention, the server includes means for receiving a sentence input by a user and a target language, means for translating the received user sentence into a specified target language, means for displaying the translation result to the user, means for correcting errors in the user sentence, means for comparing the user sentence with the corrected sentence and generating feedback, and means for providing the generated feedback to the user, thereby enabling the sentence input by the user to be translated with high accuracy, errors to be corrected, and useful feedback to be provided.

[0094] "User" refers to a person who uses a language learning device to input text and receive translation, correction, and feedback.

[0095] "Sentence" refers to text data that a user inputs through a language learning device.

[0096] "Target language" refers to the language into which the user specifies to translate the input text.

[0097] "Translation" refers to the process of converting user-entered text into a specified target language.

[0098] "Correction" refers to the process of correcting errors in text entered by a user.

[0099] "Feedback" refers to the evaluation and advice generated by the server regarding the corrections and translation results for the user's input text.

[0100] "Server" refers to a processor that receives requests from users and translates text, corrects it, and generates feedback.

[0101] "Terminal" refers to a device where a user inputs text and displays translation results, corrections, and feedback from the server.

[0102] "Generative AI model" refers to an artificial intelligence model used to perform natural language processing.

[0103] This invention is a system that supports language learning for users using language learning devices. The system consists of a server, a terminal, and a user. The user inputs a sentence and a target language, and the information is sent to the server. The server uses a natural language processing model to translate and correct the sentence and generate feedback.

[0104] Hardware and Software Configuration

[0105] server

[0106] The server is a computer system with a high-performance processor that works in conjunction with an external natural language processing model (e.g., Google Translate API, DeepL, Grammarly, LanguageTool). The server has software and APIs to receive and process user data.

[0107] Terminal

[0108] A terminal is a device operated by a user, such as a PC, tablet, or smartphone. The terminal includes an interface for user input and a display for displaying results from the server. The terminal has a network connection for communicating with the server.

[0109] User

[0110] The user is a language learner who uses a terminal to input sentences and receive translation, correction, and feedback. The user inputs sentences and their corresponding target language through the terminal interface.

[0111] Data processing and calculation

[0112] The server receives the text and target language sent by the user. The received data is translated into the specified target language using a generative AI model. The translation result is then sent back to the user's device, where it is displayed.

[0113] A natural language processing model is also used to correct errors in the user's writing. The correction results are also sent from the server to the terminal and displayed to the user. The server then compares the user's input sentence with the corrected sentence and generates feedback. This feedback is also provided to the user.

[0114] Specific examples

[0115] As a concrete example, consider the case where a user wants to translate "Hello, how are you?" into Japanese. The user inputs this sentence and "Japanese" into their device. The server receives this and translates it into "Hello, how are you?" using Google Translate API or DeepL. This result is sent to the user's device, which then displays the translation result.

[0116] Also, if a user types "I went to the store," Grammarly or LanguageTool can be used to correct the error in the sentence. The server receives this and corrects it to "I went to the store." This result is then sent back to the user's device, which then displays the correction.

[0117] Prompt Sentence Examples

[0118] 1. Translate the sentence entered by the user into the specified target language. The entered sentence is "Hello, how are you?" and the target language is Japanese.

[0119] 2. Correct the error in the sentence entered by the user. The sentence entered is "I went to the store."

[0120] 3. Compare the user-entered sentence with the corrected sentence and provide feedback. The original sentence is "I went to the store." The corrected sentence is "I went to the store."

[0121] This system allows users to learn languages ​​efficiently and accurately.

[0122] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0123] Step 1:

[0124] The user inputs a sentence and a target language.

[0125] Specifically, the user enters a sentence (e.g., "Hello, how are you?") and a target language (e.g., "Japanese") into a text input field on the terminal.

[0126] Input: User-entered text and target language

[0127] Output: Data entered into the terminal

[0128] Step 2:

[0129] The terminal transmits the input data to the server.

[0130] Specifically, when the user clicks the "send" button, the terminal puts the input text and the target language together into a data packet and sends it to the server.

[0131] Input: User-entered text and target language

[0132] Output: Data packets sent to the server

[0133] Step 3:

[0134] The server analyzes the received data and performs translation.

[0135] Specifically, the server analyzes the received data packets, extracts the input sentence and target language, and then uses a generative AI model (e.g., Google Translate API, DeepL) to translate the sentence into the specified target language.

[0136] Input: Data packet sent to the server (text and target language)

[0137] Output: The translated text

[0138] Step 4:

[0139] The server sends the translation results to the terminal.

[0140] Specifically, the server repackages the translation results into a data packet and sends it to the terminal, which also includes a confirmation of the data transmission.

[0141] Input: translated sentence

[0142] Output: Data packets sent to the terminal

[0143] Step 5:

[0144] The device will display the translation results.

[0145] Specifically, the device receives the data packet from the server and displays the translation result on the user's interface, for example, "Hello, how are you?"

[0146] Input: Data packet sent to the terminal (translation result)

[0147] Output: The translation result that is displayed to the user

[0148] Step 6:

[0149] When the user wishes to correct a sentence, the user inputs the sentence containing the error.

[0150] Specifically, the user inputs the sentence they wish to correct (e.g., "I went to the store.") into the terminal.

[0151] Input: A sentence containing an error

[0152] Output: Data entered into the terminal

[0153] Step 7:

[0154] The terminal sends the sentence to be corrected to the server.

[0155] Specifically, when the user clicks the "Send" button, the terminal assembles the sentence to be corrected into a data packet and sends it to the server.

[0156] Input: A sentence containing an error

[0157] Output: Data packets sent to the server

[0158] Step 8:

[0159] The server analyzes the received data and corrects the text.

[0160] Specifically, the server analyzes the received data packets and corrects errors using a generative AI model (e.g., Grammarly, LanguageTool).

[0161] Input: Data packet sent to the server (text containing an error)

[0162] Output: Corrected sentence

[0163] Step 9:

[0164] The server sends the correction results to the terminal.

[0165] Specifically, the server collects the correction results into a data packet and transmits it to the terminal, which also includes a confirmation of the data transmission.

[0166] Input: Corrected sentence

[0167] Output: Data packets sent to the terminal

[0168] Step 10:

[0169] The terminal will display the correction results.

[0170] Specifically, the terminal receives the data packet from the server and displays the correction result on the user interface, for example, "I went to the store."

[0171] Input: Data packet sent to the terminal (corrected result)

[0172] Output: Corrected results shown to the user

[0173] Step 11:

[0174] A user submits a request for feedback.

[0175] As a specific operation, the user clicks a button to request feedback on the correction results.

[0176] Input: Request for feedback

[0177] Output: Data entered into the terminal

[0178] Step 12:

[0179] The device sends a feedback request to the server.

[0180] Specifically, the terminal assembles the feedback request into a data packet and transmits it to the server.

[0181] Input: Feedback Request

[0182] Output: Data packets sent to the server

[0183] Step 13:

[0184] The server generates the feedback.

[0185] Specifically, the server generates feedback using a generative AI model based on the received feedback request, for example, by comparing the original sentence with the corrected sentence and explaining the cause of the error and the correct form.

[0186] Input: Feedback Request

[0187] Output: Generated feedback

[0188] Step 14:

[0189] The server sends the feedback to the device.

[0190] In particular, the server collects the generated feedback into a data packet and transmits it to the terminal.

[0191] Input: Generated feedback

[0192] Output: Data packets sent to the terminal

[0193] Step 15:

[0194] The device displays feedback.

[0195] Specifically, the device receives data packets from the server and displays feedback on the user's interface, such as "'goed' is not a correct past tense. The correct form is 'went'."

[0196] Input: Data packets sent to the terminal (feedback)

[0197] Output: Feedback that is displayed to the user

[0198] (Application example 1)

[0199] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0200] The present invention relates to a language learning support system that enables factory workers who speak different languages ​​to work efficiently and safely. Specifically, the system aims to reduce worker errors by supporting the understanding of technical terms and safety procedures, correcting incorrect terms and expressions, and providing appropriate feedback. Conventional systems have difficulty supporting a wide range of languages ​​and are inadequate in supporting technical terms and procedures specific to industrial machinery, so improvements are needed.

[0201] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0202] In this invention, the server includes means for receiving user sentences input from a language learning device, means for translating the received user sentences into a specified target language, means for correcting errors in the user sentences, means for generating feedback by comparing the user sentences with the corrected sentences, means for providing the generated feedback to the user, and means for supporting the terminology and procedures used in industrial machines through translation, correction, and feedback. This enables efficient and safe work by correcting errors in terminology and procedures used by factory workers in real time and providing appropriate feedback.

[0203] A "language learning device" is a device that allows a user to input text and processes the text to assist in language learning.

[0204] A "user's sentence" is any text entered by a user using a language learning device.

[0205] The "target language" is the language into which the user wants to translate the text.

[0206] A "means for translating" is a function or process for converting received text into a target language.

[0207] "Correction means" refers to the functions and processes for correcting errors in text entered by a user.

[0208] A "means for comparing and generating feedback" is a function or process for comparing the original and corrected texts and generating feedback that explains the differences.

[0209] A "means for providing feedback" is a function or process for notifying or displaying generated feedback to the user.

[0210] "Terminology and procedures used in commercial machinery" refers to technical terms and operating procedures required in a specific business environment, such as a factory.

[0211] A "natural language processing model" is an artificial intelligence algorithm or system designed to understand and process human language.

[0212] The present invention relates to a system for assisting a user in language learning using a language learning device, and for improving work efficiency and safety, particularly in a factory environment. The system includes the following main functions:

[0213] Translation feature

[0214] The server translates the sentence entered by the user into the target language. When the user enters the sentence to be translated and the target language into the language learning device, it is sent to the server. The server uses a natural language processing model (e.g., GPT-4) to translate the input sentence into the specified target language. The translation result is sent from the server to the device, and the device displays the translation result to the user.

[0215] Specific examples

[0216] If a user wants to translate "Please check the conveyor belt." into Spanish, they input the sentence and "Spanish" into their terminal. The server receives this and uses a natural language processing model to translate it into "Por favor, revise la cinta transportadora." The result is sent to the user's terminal, which then displays the translation to the user.

[0217] Text correction function

[0218] The server corrects errors in sentences entered by the user. When a user enters a sentence to be corrected into a language learning device, it is sent to the server. The server uses a natural language processing model to correct the errors in the sentence. The server sends the correction results to the terminal, which then displays the correction results to the user.

[0219] Specific examples

[0220] If a user types "Check the conveyor bolt," they enter that sentence into their terminal. The server receives it and uses a natural language processing model to correct it to "Check the conveyor bolt." The result is sent to the user's terminal, which then displays the corrected result to the user.

[0221] Feedback function

[0222] The server compares the text entered by the user with the corrected text and provides feedback. If the user requests feedback on the corrections to their text, the feedback is sent to the server. The server compares the input text with the corrected text and generates feedback using a natural language processing model. This feedback is sent from the server to the terminal, which displays the feedback to the user.

[0223] Specific examples

[0224] If a user types "Check the conveyor bolt." and then corrects it to "Check the conveyor bolt.", the user sends a request for feedback. The server receives this and uses a natural language processing model to generate feedback. For example, it generates feedback such as "'conbeyor' is a misspelling. The correct spelling is 'conveyor'." It then sends this feedback to the user's device, which then displays it to the user.

[0225] In this way, the present invention helps users learn language efficiently and accurately, and supports the understanding and application of technical terms and procedures, particularly in factory environments, thereby improving safety and work efficiency.

[0226] Example prompt sentence:

[0227] "Translate the following technical instruction from English to Spanish: 'Please check the conveyor belt.'"

[0228] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0229] Step 1:

[0230] A user inputs a sentence and a target language into a language learning device.

[0231] Input: The sentence you want to translate (e.g., "Please check the conveyor belt.") and the target language (e.g., Spanish)

[0232] Specific actions: The user enters a sentence and the target language into the input field on the device and presses the send button.

[0233] Step 2:

[0234] The terminal receives the user's input and sends it to the server.

[0235] Input: User-entered text and target language

[0236] Output: Input data sent to the server

[0237] Specific operation: The device packages the input data and sends a request to the API endpoint.

[0238] Step 3:

[0239] The server translates the received text into the target language using a natural language processing model.

[0240] Input: Received text and target language

[0241] Output: Translation result (e.g. "Por favor, revise la cinta transportadora.")

[0242] Specific operation: The server inputs the sentence and target language as prompts into the generative AI model (e.g., GPT-4) and obtains the translation result.

[0243] Step 4:

[0244] The server sends the translation results to the terminal.

[0245] Input: Translation result

[0246] Output: Translation results sent to your device

[0247] Specific operation: The server repackages the translation results and sends them to the device.

[0248] Step 5:

[0249] The terminal displays the translation results to the user.

[0250] Input: Translation result

[0251] Output: User-viewable translation results

[0252] Specific operation: The translation results received by the device are displayed on the screen.

[0253] Step 6:

[0254] The user inputs the sentence to be corrected.

[0255] Input: The sentence you want to correct (e.g., "Check the conveyor bolt.")

[0256] Specific operation: The user again enters text into the device's input field and presses the send button.

[0257] Step 7:

[0258] The terminal receives an input sentence for which an error correction is desired and transmits it to the server.

[0259] Input: The text entered by the user that you would like to have corrected

[0260] Output: Input data sent to the server

[0261] Specific operation: The device packages the input data and sends a request to the API endpoint.

[0262] Step 8:

[0263] The server corrects errors in the received text using a natural language processing model.

[0264] Input: Received text

[0265] Output: Correction result (e.g., "Check the conveyor bolt.")

[0266] Specific operation: The server inputs a sentence as a prompt into the generative AI model (e.g., GPT-4) and obtains the corrected result.

[0267] Step 9:

[0268] The server sends the correction results to the terminal.

[0269] Input: Corrected result

[0270] Output: Corrections sent to the terminal

[0271] Specific operation: The server repackages the correction results and sends them to the terminal.

[0272] Step 10:

[0273] The terminal displays the correction results to the user.

[0274] Input: Corrected result

[0275] Output: User-visible display of the corrections

[0276] Specific operation: The correction results received by the device are displayed on the screen.

[0277] Step 11:

[0278] A user submits a request for feedback.

[0279] Input: Request for feedback

[0280] Specific behavior: The user presses the feedback request button on the device.

[0281] Step 12:

[0282] The terminal receives the feedback request and sends it to the server.

[0283] Input: Feedback Request

[0284] Output: Request data sent to the server

[0285] Specific operation: The device packages the request data and sends the request to the API endpoint.

[0286] Step 13:

[0287] The server compares the original and corrected sentences and generates feedback using a natural language processing model.

[0288] Input: Original and corrected sentences

[0289] Output: Feedback (e.g. "'conbeyor' is an incorrect spelling. The correct spelling is 'conveyor'.")

[0290] Specific operation: The server inputs the original sentence and the corrected sentence as prompts to the generative AI model (e.g., GPT-4) and obtains feedback.

[0291] Step 14:

[0292] The server sends the feedback to the device.

[0293] Input: Feedback

[0294] Output: Feedback sent to the device

[0295] What happens: The server repackages the feedback and sends it to the device.

[0296] Step 15:

[0297] The device displays the feedback to the user.

[0298] Input: Feedback

[0299] Output: User-visible feedback display

[0300] Specific behavior: The device displays the feedback it receives on the screen.

[0301] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0302] This invention is a system that supports language learning using a language learning device, and also combines it with an emotion engine that recognizes and analyzes the user's emotions. The purpose of this system is to promote language learning efficiently while taking into account the user's emotional state. The system receives the user's text, translates it into the specified target language, corrects errors, and generates feedback. Furthermore, when the system receives the user's text, the emotion engine analyzes the user's emotions and reflects the results in the translation, correction, and feedback processes.

[0303] Program processing overview

[0304] Translation feature

[0305] The server translates sentences entered by the user into the target language. First, the user enters the sentence they want to translate and the target language into the language learning device. The terminal receives this and sends it to the server. The server uses an emotion engine to analyze the user's emotions and performs the translation by calling a natural language processing model while taking into account their emotional state. The results are returned to the terminal, which displays the translation results to the user.

[0306] Specific examples

[0307] Suppose a user wants to translate the sentence "Hello, how are you?" into Japanese. The user inputs this sentence and "Japanese" into the device. The device sends this to the server, which uses an emotion engine to analyze the user's emotion. If the result is determined to be "positive," a natural language processing model is used for translation, and the translation result, "Hello, how are you?", is generated. The translation result is sent to the device, which displays it to the user.

[0308] Text correction function

[0309] The server corrects errors in sentences entered by the user. The user enters the sentence they wish to correct into the language learning device, which receives it and sends it to the server. The server uses an emotion engine to analyze the user's emotions, takes that state into account, invokes a natural language processing model, and corrects the sentence. The correction results are returned to the device, which displays them to the user.

[0310] Specific examples

[0311] If a user types "I went to the store," they enter this into their device and send it to the server. The server uses an emotion engine to analyze the user's emotion. If the result is determined to be "negative," a natural language processing model is used to correct it, and the corrected result, "I went to the store," is generated. The corrected result is sent to the device, which then displays it to the user.

[0312] Feedback function

[0313] The server compares the sentence entered by the user with the corrected sentence and generates feedback. When the user requests feedback on the corrections to their sentence, the device sends the request to the server. The server uses an emotion engine to analyze the user's emotions, takes that state into account, invokes a natural language processing model, and generates feedback. The generated feedback is sent to the device, which displays it to the user.

[0314] Specific examples

[0315] If a user types "I went to the store," and then corrects it to "I went to the store," the user sends a request for feedback. The server uses an emotion engine to analyze the user's emotion. If the result is determined to be "frustration," a natural language processing model is used to generate feedback, such as "'goed' is not the correct past tense. 'went' is the correct form." This feedback is sent to the device, which then displays it to the user.

[0316] The present invention allows for more personalized and effective language learning assistance by taking into account the user's emotional state.

[0317] The processing flow will be explained below.

[0318] Translation feature

[0319] Step 1:

[0320] A user inputs a sentence to be translated and a target language through the interface of a language learning device.

[0321] Step 2:

[0322] The terminal receives the user's input and sends the data to the server.

[0323] Step 3:

[0324] The server receives the input data and analyzes the user's emotions using an emotion engine.

[0325] Step 4:

[0326] Based on the sentiment analysis results, the server calls a natural language processing model to start the translation process.

[0327] Step 5:

[0328] The server sends a translation request to a natural language processing model, which translates it into the target language, taking into account the user's emotional state.

[0329] Step 6:

[0330] The server receives the translation result and sends it to the terminal.

[0331] Step 7:

[0332] The terminal displays the translation result received from the server to the user.

[0333] Text correction function

[0334] Step 1:

[0335] The user inputs the sentence to be corrected through the interface of the language learning device.

[0336] Step 2:

[0337] The terminal receives the user's input and sends the data to the server.

[0338] Step 3:

[0339] The server receives the input data and analyzes the user's emotions using an emotion engine.

[0340] Step 4:

[0341] Based on the sentiment analysis results, the server calls a natural language processing model to initiate the sentence correction process.

[0342] Step 5:

[0343] The server sends a request to generate a correction sentence to the natural language processing model, and the model corrects the sentence, taking into account the user's emotional state.

[0344] Step 6:

[0345] The server receives the correction result and transmits it to the terminal.

[0346] Step 7:

[0347] The terminal displays the correction results received from the server to the user.

[0348] Feedback function

[0349] Step 1:

[0350] The user sends a request through the interface of the language learning device for feedback on their input sentence and the corrected sentence.

[0351] Step 2:

[0352] The terminal receives the user's request and sends the data to the server.

[0353] Step 3:

[0354] The server receives the request and analyzes the user's emotions using an emotion engine.

[0355] Step 4:

[0356] Based on the sentiment analysis results, the server invokes a natural language processing model to initiate the feedback generation process.

[0357] Step 5:

[0358] The server sends a feedback generation request to the natural language processing model, which then compares the user's input sentence with the corrected sentence and generates feedback, taking into account the user's emotional state.

[0359] Step 6:

[0360] The server receives the feedback result and sends it to the terminal.

[0361] Step 7:

[0362] The terminal displays the feedback result received from the server to the user.

[0363] Example 2

[0364] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0365] Conventional language learning systems perform translation, correction, and feedback without considering the user's emotional state, which can affect the user's learning efficiency and motivation. Furthermore, due to the lack of a means to provide appropriate feedback according to emotions, it is not possible to provide adequate learning support tailored to each individual user.

[0366] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0367] In this invention, the server includes means for receiving a user's text input from a language learning device, means for translating the received user's text into a specified target language, means for correcting errors in the user's text, means for generating feedback by comparing the user's text with the corrected text, means for providing the generated feedback to the user, means for analyzing the user's emotions in processing the received text and the generated translation, correction results, or feedback, and means for reflecting the results of the user's emotion analysis in the processing results, thereby enabling appropriate translation, correction, and feedback that take the user's emotional state into consideration.

[0368] "Language learning device" refers to a device that allows a user to learn a language, and includes electronic devices such as smartphones, tablets, and personal computers.

[0369] "User" refers to an individual who uses this system to learn a language.

[0370] "Text" refers to documents or sentences entered by a user.

[0371] A "server" refers to a computer system that provides data processing and storage over a network.

[0372] "Means for receiving" refers to a function for transmitting the user's text from the language learning device to the server and for the server to receive the data.

[0373] "Means for translating" refers to functionality for converting received text into a specified target language, using natural language processing techniques such as generative AI models.

[0374] "Means for correcting errors" refers to functionality for correcting errors in received text, using natural language processing techniques such as generative AI models.

[0375] "Means for generating feedback" refers to the functionality that compares the user's text with the corrected text and generates learning feedback.

[0376] "Means for providing" refers to the functionality for displaying the generated feedback to the user.

[0377] "Means for analyzing emotions" refers to the function for reading and analyzing emotions from the user's input text and reactions, and uses technologies such as emotion engines.

[0378] "Means for reflecting emotion analysis results in processing results" refers to the function of adjusting the results of translation, correction, and feedback based on the analyzed emotion information.

[0379] MODE FOR CARRYING OUT THE INVENTION

[0380] This invention is a system that supports language learning for users using a language learning device, and also combines it with an emotion engine that recognizes and analyzes the user's emotions. The purpose of this system is to promote language learning efficiently while taking into account the user's emotional state.

[0381] Hardware and Software Configuration

[0382] Language learning device (terminal): An electronic device used by a user, such as a smartphone, tablet, or computer.

[0383] Server: A computer system that receives, analyzes, and processes data.

[0384] Emotion engine: Software for analyzing user emotions (e.g., emotion analysis API, emotion engine technology).

[0385] Natural Language Processing models: Generative AI models for translation, correction, and feedback generation (e.g., GPT-3, BERT, Google Cloud Translation API).

[0386] System Operation

[0387] 1. User Input: The user inputs the sentence they want translated or corrected into the language learning device. Examples of input data include "Hello, how are you?" or "I went to the store."

[0388] 2. The device receives and sends data: The device receives the entered data and sends it to the server over the internet. This process uses data transmission protocols and API calls.

[0389] 3. The server analyzes emotions: The server analyzes the received data and extracts sentences from it. At the same time, it uses an emotion engine to analyze the user's emotions. For example, it uses the Azure Emotion API to determine emotional states such as "positive," "negative," and "frustrated."

[0390] 4. The server performs translation and correction processing: The server takes into account the sentiment analysis results and calls the generative AI model for processing. Google Cloud Translation API is used for translation, and GPT-3 is used for correction.

[0391] 5. The server returns the results: The generated translation results and corrections are sent back to the terminal.

[0392] 6. Device displays: The device displays the received results to the user. The displayed results may include the translated sentence "Hello, how are you?" or the corrected sentence "I went to the store."

[0393] Specific operation example

[0394] Translation feature

[0395] Prompt statement:

[0396] "Please translate the following English sentence into Japanese. The user's emotional state is positive. 'Hello, how are you?'"

[0397] Text correction function

[0398] Prompt statement:

[0399] "Please correct the following sentence to use correct English grammar. The user's emotional state is negative. 'I went to the store.'"

[0400] Feedback function

[0401] Prompt statement:

[0402] "Please provide feedback on the corrected sentence below. The user's emotional state is frustration. Original: 'I went to the store.' Corrected: 'I went to the store.'"

[0403] This system can provide appropriate, personalized translation, correction, and feedback that takes into account the user's emotional state, thereby enabling efficient language learning support.

[0404] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0405] Step 1:

[0406] User input

[0407] The user inputs the sentence they wish to translate or correct and the target language into the language learning device. This input data includes text such as "Hello, how are you?" or "I went to the store." This text becomes the input data for the system.

[0408] Step 2:

[0409] Device receives and sends data

[0410] The terminal receives text data entered by the user. The received data is sent to the server via the Internet. This process uses an HTTP request to send the data. The data sent includes the entered text and the target language.

[0411] Step 3:

[0412] The server analyzes emotions

[0413] The server analyzes the received data and extracts text data. It then uses an emotion engine to analyze the user's emotions from the received text data. This emotion analysis is performed using, for example, the Azure Emotion API. The server outputs the emotional state, such as "positive," "negative," or "frustrated," as a result of the emotion analysis.

[0414] Step 4:

[0415] The server performs translation and correction processing

[0416] The server takes into account the sentiment analysis results and calls a natural language processing model (e.g., Google Cloud Translation API or GPT-3) to translate or correct the input text. In the case of translation, it converts the received text into the target language and generates a translated text. In the case of correction, it corrects errors and generates text that follows correct English grammar. The server outputs the generated translation or correction result.

[0417] Step 5:

[0418] The server returns the results

[0419] The server then returns the translation results and corrections to the device. The returned data includes the translated or corrected text, the user's sentiment analysis results, etc. This process uses HTTP responses to send data.

[0420] Step 6:

[0421] The device is displayed

[0422] The device then displays the results received from the server to the user. The displayed results include translated and corrected sentences. For example, the translation result for "Hello, how are you?" is "Hello, how are you?", and the correction result for "I went to the store." is "I went to the store."

[0423] This allows users to receive appropriate translations, corrections, and feedback that take emotions into account through the system.

[0424] (Application example 2)

[0425] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0426] Conventional language learning systems have difficulty providing a personalized learning experience that takes into account the user's emotional state. Furthermore, when learning using video content, the lack of real-time subtitle translation and feedback functions makes it difficult for users to learn efficiently.

[0427] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving a user's sentence input from a language learning device, means for translating the received user's sentence into a specified target language, means for correcting errors in the user's sentence, means for generating feedback by comparing the user's sentence with the corrected sentence, means for providing the generated feedback to the user, means for translating subtitles of videos watched by the user in real time, means for analyzing the user's emotions, and means for adjusting the translation results and feedback based on the analyzed emotions. This enables personalized, efficient, and interactive language learning that takes the user's emotional state into consideration.

[0428] A "language learning device" is an electronic device that a user uses when learning a language and that has input and display functions.

[0429] A "text" is a character string or sentence entered by a user.

[0430] "Target language" refers to the language that the user wishes to learn or translate into.

[0431] "Translation" is the act or process of converting from one language to another.

[0432] "Error correction" is the process of correcting grammatical or semantic errors in input text.

[0433] "Feedback" refers to information such as corrections, supplementary explanations, and evaluations provided to the user.

[0434] "Real-time subtitle translation" is the process of instantly translating subtitles displayed while watching a video into another language.

[0435] "Emotion analysis" is a technology that analyzes a user's emotional state at any given time based on their facial expressions and biometric information.

[0436] A "natural language processing model" is an artificial intelligence model for understanding and generating natural language, and uses machine learning and deep learning technologies.

[0437] A "server" is a computer system that provides computing resources and data to client devices via a network.

[0438] This invention is a system that supports language learning for users using a language learning device, and also combines it with an emotion engine that recognizes and analyzes the user's emotions. The purpose of this system is to efficiently advance language learning while taking into account the user's emotional state.

[0439] The server receives sentences entered by the user into the language learning device and translates them into the specified target language. First, the user enters the sentence they want to translate and the target language into the language learning device. The terminal receives this and sends it to the server. The server uses an emotion engine to analyze the user's emotions and performs translation by calling a natural language processing model while taking into account their emotional state. The results are returned to the terminal, which displays the translation results to the user.

[0440] Specifically, the server uses the Python-based natural language processing library googletrans for translation, and the DeepFace library for emotion analysis, analyzing the user's facial expression data captured by the camera. The hardware used is the laptop's built-in camera or an external webcam.

[0441] Suppose a user wants to translate the sentence "Hello, how are you?" into Japanese. The user inputs this sentence and "Japanese" into the device. The device sends this to the server, which uses an emotion engine to analyze the user's emotion. If the result is determined to be "positive," a natural language processing model is used for translation, and the translation result, "Hello, how are you?", is generated. The translation result is sent to the device, which displays it to the user.

[0442] The service also offers a real-time subtitle translation feature. The subtitles of the video the user is watching are translated in real time, and the tone and depth of the translation are adjusted according to the results of sentiment analysis. This optimizes the learning experience based on the user's emotional state. For example, if a user sets "English to Japanese translation" while watching a movie and the sentiment analysis determines "happy," the captions will be displayed in a positive tone, such as "I went to the store 😊."

[0443] An example prompt sentence, input to a generative AI model, is:

[0444] "When the user's emotional state is 'happy,' translate 'I went to the store.' into Japanese and add a positive expression."

[0445] In this way, more personalized and effective language learning assistance is possible by taking into account the user's emotional state.

[0446] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0447] Step 1:

[0448] A user inputs a sentence and a target language into a language learning device.

[0449] Specifically, the user inputs the sentence "Hello, how are you?" and "Japanese" into the terminal.

[0450] Step 2:

[0451] The terminal receives the input and sends it to the server.

[0452] The input data is "Hello, how are you?" and "Japanese."

[0453] The output is the transmitted input data.

[0454] Step 3:

[0455] The server invokes the emotion engine to analyze the received text.

[0456] The input data is the user's sentence "Hello, how are you?" and the emotion engine.

[0457] A deep learning model is used to analyze the user's emotions and determine them as "positive."

[0458] The output is the sentiment analysis result "positive."

[0459] Step 4:

[0460] The server calls a natural language processing model based on the sentiment analysis results and translates the text into the target language.

[0461] The input data is the user's sentence "Hello, how are you?", the target language "Japanese", and the sentiment analysis result "positive".

[0462] The translation result is "Hello, how are you?"

[0463] The output is the translation result "Hello, how are you?"

[0464] Step 5:

[0465] The server sends the translation results to the terminal.

[0466] The input data is the translation result "Hello, how are you?"

[0467] The output is sent from the server to the terminal.

[0468] Step 6:

[0469] The terminal receives the translation result and displays it to the user.

[0470] The input data is the translation result "Hello, how are you?" sent from the server.

[0471] The output is the translation displayed to the user.

[0472] Step 7:

[0473] A process is run to translate subtitles of videos watched by users in real time.

[0474] The input data is the subtitles of the video the user is watching, the target language "Japanese," and the user's emotional state.

[0475] Based on the results of sentiment analysis, appropriate expressions are added to the translation results and displayed.

[0476] The output is a real-time display of translated subtitles.

[0477] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0478] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0479] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0480] [Second embodiment]

[0481] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0482] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0483] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0484] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0485] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0486] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0487] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0488] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0489] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0490] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0491] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0492] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0493] The present invention is a system for assisting a user in language learning using a language learning device, which receives a user's text, translates the text into a specified target language, and provides feedback after correcting errors.

[0494] Program processing overview

[0495] Translation feature

[0496] The server translates the sentence entered by the user into the target language. When the user enters the sentence to be translated and the target language into the language learning device, it is sent to the server. The server uses a natural language processing model to translate the entered sentence into the specified target language. The translation result is sent from the server to the terminal, and the terminal displays the translation result to the user.

[0497] Specific examples

[0498] If a user wants to translate "Hello, how are you?" into Japanese, they input the sentence and "Japanese" into their terminal. The server receives this and translates it into "Hello, how are you?" using a natural language processing model. The result is sent to the user's terminal, and the terminal displays the translation result to the user.

[0499] Text correction function

[0500] The server corrects errors in sentences entered by the user. When a user enters a sentence to be corrected into a language learning device, it is sent to the server. The server uses a natural language processing model to correct the errors in the sentence. The server sends the correction results to the terminal, which then displays the correction results to the user.

[0501] Specific examples

[0502] If a user types "I went to the store," they enter that sentence into their terminal. The server receives this and uses a natural language processing model to correct it to "I went to the store." The result is sent to the user's terminal, which then displays the corrected result to the user.

[0503] Feedback function

[0504] The server compares the text entered by the user with the corrected text and provides feedback. If the user requests feedback on the corrections to their text, the feedback is sent to the server. The server compares the input text with the corrected text and generates feedback using a natural language processing model. This feedback is sent from the server to the terminal, which displays the feedback to the user.

[0505] Specific examples

[0506] If a user types "I went to the store," and then corrects it to "I went to the store," the user sends a request for feedback. The server receives this and uses a natural language processing model to generate feedback. For example, it generates feedback such as "'goed' is not a correct past tense. The correct form is 'went'." It then sends this feedback to the user's device, which then displays it to the user.

[0507] In this way, the present invention provides an environment in which users can learn languages ​​efficiently and accurately.

[0508] The processing flow will be explained below.

[0509] Translation feature

[0510] Step 1:

[0511] A user inputs a sentence to be translated and a target language through the interface of a language learning device.

[0512] Step 2:

[0513] The terminal receives the user's input and sends the data to the server.

[0514] Step 3:

[0515] The server receives the input data and initiates the translation process, specifically by calling a natural language processing model (e.g., a language model API).

[0516] Step 4:

[0517] The server sends a translation request to a natural language processing model, which translates it into the target language.

[0518] Step 5:

[0519] The server receives the translation result and returns it to the user terminal.

[0520] Step 6:

[0521] The terminal displays the translation result received from the server to the user.

[0522] Text correction function

[0523] Step 1:

[0524] The user inputs the sentence to be corrected through the interface of the language learning device.

[0525] Step 2:

[0526] The terminal receives the user's input and sends the data to the server.

[0527] Step 3:

[0528] The server receives the input data and initiates the sentence correction process, specifically by calling a natural language processing model (e.g., a language correction model API).

[0529] Step 4:

[0530] The server sends a request to generate a corrected sentence to the natural language processing model, and the model corrects the errors in the sentence.

[0531] Step 5:

[0532] The server receives the correction result and returns it to the user terminal.

[0533] Step 6:

[0534] The terminal displays the correction results received from the server to the user.

[0535] Feedback function

[0536] Step 1:

[0537] The user requests feedback on their input sentences and corrected sentences through the interface of the language learning device.

[0538] Step 2:

[0539] The terminal receives the user's request and sends the data to the server.

[0540] Step 3:

[0541] The server receives the request and starts the feedback generation process by calling a natural language processing model (e.g., a feedback generation model API).

[0542] Step 4:

[0543] The server sends a feedback generation request to the natural language processing model, and the model compares the user's input sentence with the corrected sentence and generates the feedback.

[0544] Step 5:

[0545] The server receives the generated feedback and returns it to the user terminal.

[0546] Step 6:

[0547] The terminal displays the feedback received from the server to the user.

[0548] Example 1

[0549] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0550] In conventional language learning systems, the accuracy and quality of the translation and correction results are often insufficient when translating user-input text into the target language and correcting errors. Furthermore, there is a lack of feedback to help users deepen their understanding of the language they are learning. This has resulted in a lack of an environment for users to learn languages ​​efficiently and accurately.

[0551] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0552] In this invention, the server includes means for receiving a sentence input by a user and a target language, means for translating the received user sentence into a specified target language, means for displaying the translation result to the user, means for correcting errors in the user sentence, means for comparing the user sentence with the corrected sentence and generating feedback, and means for providing the generated feedback to the user, thereby enabling the sentence input by the user to be translated with high accuracy, errors to be corrected, and useful feedback to be provided.

[0553] "User" refers to a person who uses a language learning device to input text and receive translation, correction, and feedback.

[0554] "Sentence" refers to text data that a user inputs through a language learning device.

[0555] "Target language" refers to the language into which the user specifies to translate the input text.

[0556] "Translation" refers to the process of converting user-entered text into a specified target language.

[0557] "Correction" refers to the process of correcting errors in text entered by a user.

[0558] "Feedback" refers to the evaluation and advice generated by the server regarding the corrections and translation results for the user's input text.

[0559] "Server" refers to a processor that receives requests from users and translates text, corrects it, and generates feedback.

[0560] "Terminal" refers to a device where a user inputs text and displays translation results, corrections, and feedback from the server.

[0561] "Generative AI model" refers to an artificial intelligence model used to perform natural language processing.

[0562] This invention is a system that supports language learning for users using language learning devices. The system consists of a server, a terminal, and a user. The user inputs a sentence and a target language, and the information is sent to the server. The server uses a natural language processing model to translate and correct the sentence and generate feedback.

[0563] Hardware and Software Configuration

[0564] server

[0565] The server is a computer system with a high-performance processor that works in conjunction with an external natural language processing model (e.g., Google Translate API, DeepL, Grammarly, LanguageTool). The server has software and APIs to receive and process user data.

[0566] Terminal

[0567] A terminal is a device operated by a user, such as a PC, tablet, or smartphone. The terminal includes an interface for user input and a display for displaying results from the server. The terminal has a network connection for communicating with the server.

[0568] User

[0569] The user is a language learner who uses a terminal to input sentences and receive translation, correction, and feedback. The user inputs sentences and their corresponding target language through the terminal interface.

[0570] Data processing and calculation

[0571] The server receives the text and target language sent by the user. The received data is translated into the specified target language using a generative AI model. The translation result is then sent back to the user's device, where it is displayed.

[0572] A natural language processing model is also used to correct errors in the user's writing. The correction results are also sent from the server to the terminal and displayed to the user. The server then compares the user's input sentence with the corrected sentence and generates feedback. This feedback is also provided to the user.

[0573] Specific examples

[0574] As a concrete example, consider the case where a user wants to translate "Hello, how are you?" into Japanese. The user inputs this sentence and "Japanese" into their device. The server receives this and translates it into "Hello, how are you?" using Google Translate API or DeepL. This result is sent to the user's device, which then displays the translation result.

[0575] Also, if a user types "I went to the store," Grammarly or LanguageTool can be used to correct the error in the sentence. The server receives this and corrects it to "I went to the store." This result is then sent back to the user's device, which then displays the correction.

[0576] Prompt Sentence Examples

[0577] 1. Translate the sentence entered by the user into the specified target language. The entered sentence is "Hello, how are you?" and the target language is Japanese.

[0578] 2. Correct the error in the sentence entered by the user. The sentence entered is "I went to the store."

[0579] 3. Compare the user-entered sentence with the corrected sentence and provide feedback. The original sentence is "I went to the store." The corrected sentence is "I went to the store."

[0580] This system allows users to learn languages ​​efficiently and accurately.

[0581] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0582] Step 1:

[0583] The user inputs a sentence and a target language.

[0584] Specifically, the user enters a sentence (e.g., "Hello, how are you?") and a target language (e.g., "Japanese") into a text input field on the terminal.

[0585] Input: User-entered text and target language

[0586] Output: Data entered into the terminal

[0587] Step 2:

[0588] The terminal transmits the input data to the server.

[0589] Specifically, when the user clicks the "send" button, the terminal puts the input text and the target language together into a data packet and sends it to the server.

[0590] Input: User-entered text and target language

[0591] Output: Data packets sent to the server

[0592] Step 3:

[0593] The server analyzes the received data and performs translation.

[0594] Specifically, the server analyzes the received data packets, extracts the input sentence and target language, and then uses a generative AI model (e.g., Google Translate API, DeepL) to translate the sentence into the specified target language.

[0595] Input: Data packet sent to the server (text and target language)

[0596] Output: The translated text

[0597] Step 4:

[0598] The server sends the translation results to the terminal.

[0599] Specifically, the server repackages the translation results into a data packet and sends it to the terminal, which also includes a confirmation of the data transmission.

[0600] Input: translated sentence

[0601] Output: Data packets sent to the terminal

[0602] Step 5:

[0603] The device will display the translation results.

[0604] Specifically, the device receives the data packet from the server and displays the translation result on the user's interface, for example, "Hello, how are you?"

[0605] Input: Data packet sent to the terminal (translation result)

[0606] Output: The translation result that is displayed to the user

[0607] Step 6:

[0608] When the user wishes to correct a sentence, the user inputs the sentence containing the error.

[0609] Specifically, the user inputs the sentence they wish to correct (e.g., "I went to the store.") into the terminal.

[0610] Input: A sentence containing an error

[0611] Output: Data entered into the terminal

[0612] Step 7:

[0613] The terminal sends the sentence to be corrected to the server.

[0614] Specifically, when the user clicks the "Send" button, the terminal assembles the sentence to be corrected into a data packet and sends it to the server.

[0615] Input: A sentence containing an error

[0616] Output: Data packets sent to the server

[0617] Step 8:

[0618] The server analyzes the received data and corrects the text.

[0619] Specifically, the server analyzes the received data packets and corrects errors using a generative AI model (e.g., Grammarly, LanguageTool).

[0620] Input: Data packet sent to the server (text containing an error)

[0621] Output: Corrected sentence

[0622] Step 9:

[0623] The server sends the correction results to the terminal.

[0624] Specifically, the server collects the correction results into a data packet and transmits it to the terminal, which also includes a confirmation of the data transmission.

[0625] Input: Corrected sentence

[0626] Output: Data packets sent to the terminal

[0627] Step 10:

[0628] The terminal will display the correction results.

[0629] Specifically, the terminal receives the data packet from the server and displays the correction result on the user interface, for example, "I went to the store."

[0630] Input: Data packet sent to the terminal (corrected result)

[0631] Output: Corrected results shown to the user

[0632] Step 11:

[0633] A user submits a request for feedback.

[0634] As a specific operation, the user clicks a button to request feedback on the correction results.

[0635] Input: Request for feedback

[0636] Output: Data entered into the terminal

[0637] Step 12:

[0638] The device sends a feedback request to the server.

[0639] Specifically, the terminal assembles the feedback request into a data packet and transmits it to the server.

[0640] Input: Feedback Request

[0641] Output: Data packets sent to the server

[0642] Step 13:

[0643] The server generates the feedback.

[0644] Specifically, the server generates feedback using a generative AI model based on the received feedback request, for example, by comparing the original sentence with the corrected sentence and explaining the cause of the error and the correct form.

[0645] Input: Feedback Request

[0646] Output: Generated feedback

[0647] Step 14:

[0648] The server sends the feedback to the device.

[0649] In particular, the server collects the generated feedback into a data packet and transmits it to the terminal.

[0650] Input: Generated feedback

[0651] Output: Data packets sent to the terminal

[0652] Step 15:

[0653] The device displays feedback.

[0654] Specifically, the device receives data packets from the server and displays feedback on the user's interface, such as "'goed' is not a correct past tense. The correct form is 'went'."

[0655] Input: Data packets sent to the terminal (feedback)

[0656] Output: Feedback that is displayed to the user

[0657] (Application example 1)

[0658] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0659] The present invention relates to a language learning support system that enables factory workers who speak different languages ​​to work efficiently and safely. Specifically, the system aims to reduce worker errors by supporting the understanding of technical terms and safety procedures, correcting incorrect terms and expressions, and providing appropriate feedback. Conventional systems have difficulty supporting a wide range of languages ​​and are inadequate in supporting technical terms and procedures specific to industrial machinery, so improvements are needed.

[0660] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0661] In this invention, the server includes means for receiving user sentences input from a language learning device, means for translating the received user sentences into a specified target language, means for correcting errors in the user sentences, means for generating feedback by comparing the user sentences with the corrected sentences, means for providing the generated feedback to the user, and means for supporting the terminology and procedures used in industrial machines through translation, correction, and feedback. This enables efficient and safe work by correcting errors in terminology and procedures used by factory workers in real time and providing appropriate feedback.

[0662] A "language learning device" is a device that allows a user to input text and processes the text to assist in language learning.

[0663] A "user's sentence" is any text entered by a user using a language learning device.

[0664] The "target language" is the language into which the user wants to translate the text.

[0665] A "means for translating" is a function or process for converting received text into a target language.

[0666] "Correction means" refers to the functions and processes for correcting errors in text entered by a user.

[0667] A "means for comparing and generating feedback" is a function or process for comparing the original and corrected texts and generating feedback that explains the differences.

[0668] A "means for providing feedback" is a function or process for notifying or displaying generated feedback to the user.

[0669] "Terminology and procedures used in commercial machinery" refers to technical terms and operating procedures required in a specific business environment, such as a factory.

[0670] A "natural language processing model" is an artificial intelligence algorithm or system designed to understand and process human language.

[0671] The present invention relates to a system for assisting a user in language learning using a language learning device, and for improving work efficiency and safety, particularly in a factory environment. The system includes the following main functions:

[0672] Translation feature

[0673] The server translates the sentence entered by the user into the target language. When the user enters the sentence to be translated and the target language into the language learning device, it is sent to the server. The server uses a natural language processing model (e.g., GPT-4) to translate the input sentence into the specified target language. The translation result is sent from the server to the device, and the device displays the translation result to the user.

[0674] Specific examples

[0675] If a user wants to translate "Please check the conveyor belt." into Spanish, they input the sentence and "Spanish" into their terminal. The server receives this and uses a natural language processing model to translate it into "Por favor, revise la cinta transportadora." The result is sent to the user's terminal, which then displays the translation to the user.

[0676] Text correction function

[0677] The server corrects errors in sentences entered by the user. When a user enters a sentence to be corrected into a language learning device, it is sent to the server. The server uses a natural language processing model to correct the errors in the sentence. The server sends the correction results to the terminal, which then displays the correction results to the user.

[0678] Specific examples

[0679] If a user types "Check the conveyor bolt," they enter that sentence into their terminal. The server receives it and uses a natural language processing model to correct it to "Check the conveyor bolt." The result is sent to the user's terminal, which then displays the corrected result to the user.

[0680] Feedback function

[0681] The server compares the text entered by the user with the corrected text and provides feedback. If the user requests feedback on the corrections to their text, the feedback is sent to the server. The server compares the input text with the corrected text and generates feedback using a natural language processing model. This feedback is sent from the server to the terminal, which displays the feedback to the user.

[0682] Specific examples

[0683] If a user types "Check the conveyor bolt." and then corrects it to "Check the conveyor bolt.", the user sends a request for feedback. The server receives this and uses a natural language processing model to generate feedback. For example, it generates feedback such as "'conbeyor' is a misspelling. The correct spelling is 'conveyor'." It then sends this feedback to the user's device, which then displays it to the user.

[0684] In this way, the present invention helps users learn language efficiently and accurately, and supports the understanding and application of technical terms and procedures, particularly in factory environments, thereby improving safety and work efficiency.

[0685] Example prompt sentence:

[0686] "Translate the following technical instruction from English to Spanish: 'Please check the conveyor belt.'"

[0687] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0688] Step 1:

[0689] A user inputs a sentence and a target language into a language learning device.

[0690] Input: The sentence you want to translate (e.g., "Please check the conveyor belt.") and the target language (e.g., Spanish)

[0691] Specific actions: The user enters a sentence and the target language into the input field on the device and presses the send button.

[0692] Step 2:

[0693] The terminal receives the user's input and sends it to the server.

[0694] Input: User-entered text and target language

[0695] Output: Input data sent to the server

[0696] Specific operation: The device packages the input data and sends a request to the API endpoint.

[0697] Step 3:

[0698] The server translates the received text into the target language using a natural language processing model.

[0699] Input: Received text and target language

[0700] Output: Translation result (e.g. "Por favor, revise la cinta transportadora.")

[0701] Specific operation: The server inputs the sentence and target language as prompts into the generative AI model (e.g., GPT-4) and obtains the translation result.

[0702] Step 4:

[0703] The server sends the translation results to the terminal.

[0704] Input: Translation result

[0705] Output: Translation results sent to your device

[0706] Specific operation: The server repackages the translation results and sends them to the device.

[0707] Step 5:

[0708] The terminal displays the translation results to the user.

[0709] Input: Translation result

[0710] Output: User-viewable translation results

[0711] Specific operation: The translation results received by the device are displayed on the screen.

[0712] Step 6:

[0713] The user inputs the sentence to be corrected.

[0714] Input: The sentence you want to correct (e.g., "Check the conveyor bolt.")

[0715] Specific operation: The user again enters text into the device's input field and presses the send button.

[0716] Step 7:

[0717] The terminal receives an input sentence for which an error correction is desired and transmits it to the server.

[0718] Input: The text entered by the user that you would like to have corrected

[0719] Output: Input data sent to the server

[0720] Specific operation: The device packages the input data and sends a request to the API endpoint.

[0721] Step 8:

[0722] The server corrects errors in the received text using a natural language processing model.

[0723] Input: Received text

[0724] Output: Correction result (e.g., "Check the conveyor bolt.")

[0725] Specific operation: The server inputs a sentence as a prompt into the generative AI model (e.g., GPT-4) and obtains the corrected result.

[0726] Step 9:

[0727] The server sends the correction results to the terminal.

[0728] Input: Corrected result

[0729] Output: Corrections sent to the terminal

[0730] Specific operation: The server repackages the correction results and sends them to the terminal.

[0731] Step 10:

[0732] The terminal displays the correction results to the user.

[0733] Input: Corrected result

[0734] Output: User-visible display of the corrections

[0735] Specific operation: The correction results received by the device are displayed on the screen.

[0736] Step 11:

[0737] A user submits a request for feedback.

[0738] Input: Request for feedback

[0739] Specific behavior: The user presses the feedback request button on the device.

[0740] Step 12:

[0741] The terminal receives the feedback request and sends it to the server.

[0742] Input: Feedback Request

[0743] Output: Request data sent to the server

[0744] Specific operation: The device packages the request data and sends the request to the API endpoint.

[0745] Step 13:

[0746] The server compares the original and corrected sentences and generates feedback using a natural language processing model.

[0747] Input: Original and corrected sentences

[0748] Output: Feedback (e.g. "'conbeyor' is an incorrect spelling. The correct spelling is 'conveyor'.")

[0749] Specific operation: The server inputs the original sentence and the corrected sentence as prompts to the generative AI model (e.g., GPT-4) and obtains feedback.

[0750] Step 14:

[0751] The server sends the feedback to the device.

[0752] Input: Feedback

[0753] Output: Feedback sent to the device

[0754] What happens: The server repackages the feedback and sends it to the device.

[0755] Step 15:

[0756] The device displays the feedback to the user.

[0757] Input: Feedback

[0758] Output: User-visible feedback display

[0759] Specific behavior: The device displays the feedback it receives on the screen.

[0760] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0761] This invention is a system that supports language learning using a language learning device, and also combines it with an emotion engine that recognizes and analyzes the user's emotions. The purpose of this system is to promote language learning efficiently while taking into account the user's emotional state. The system receives the user's text, translates it into the specified target language, corrects errors, and generates feedback. Furthermore, when the system receives the user's text, the emotion engine analyzes the user's emotions and reflects the results in the translation, correction, and feedback processes.

[0762] Program processing overview

[0763] Translation feature

[0764] The server translates sentences entered by the user into the target language. First, the user enters the sentence they want to translate and the target language into the language learning device. The terminal receives this and sends it to the server. The server uses an emotion engine to analyze the user's emotions and performs the translation by calling a natural language processing model while taking into account their emotional state. The results are returned to the terminal, which displays the translation results to the user.

[0765] Specific examples

[0766] Suppose a user wants to translate the sentence "Hello, how are you?" into Japanese. The user inputs this sentence and "Japanese" into the device. The device sends this to the server, which uses an emotion engine to analyze the user's emotion. If the result is determined to be "positive," a natural language processing model is used for translation, and the translation result, "Hello, how are you?", is generated. The translation result is sent to the device, which displays it to the user.

[0767] Text correction function

[0768] The server corrects errors in sentences entered by the user. The user enters the sentence they wish to correct into the language learning device, which receives it and sends it to the server. The server uses an emotion engine to analyze the user's emotions, takes that state into account, invokes a natural language processing model, and corrects the sentence. The correction results are returned to the device, which displays them to the user.

[0769] Specific examples

[0770] If a user types "I went to the store," they enter this into their device and send it to the server. The server uses an emotion engine to analyze the user's emotion. If the result is determined to be "negative," a natural language processing model is used to correct it, and the corrected result, "I went to the store," is generated. The corrected result is sent to the device, which then displays it to the user.

[0771] Feedback function

[0772] The server compares the sentence entered by the user with the corrected sentence and generates feedback. When the user requests feedback on the corrections to their sentence, the device sends the request to the server. The server uses an emotion engine to analyze the user's emotions, takes that state into account, invokes a natural language processing model, and generates feedback. The generated feedback is sent to the device, which displays it to the user.

[0773] Specific examples

[0774] If a user types "I went to the store," and then corrects it to "I went to the store," the user sends a request for feedback. The server uses an emotion engine to analyze the user's emotion. If the result is determined to be "frustration," a natural language processing model is used to generate feedback, such as "'goed' is not the correct past tense. 'went' is the correct form." This feedback is sent to the device, which then displays it to the user.

[0775] The present invention allows for more personalized and effective language learning assistance by taking into account the user's emotional state.

[0776] The processing flow will be explained below.

[0777] Translation feature

[0778] Step 1:

[0779] A user inputs a sentence to be translated and a target language through the interface of a language learning device.

[0780] Step 2:

[0781] The terminal receives the user's input and sends the data to the server.

[0782] Step 3:

[0783] The server receives the input data and analyzes the user's emotions using an emotion engine.

[0784] Step 4:

[0785] Based on the sentiment analysis results, the server calls a natural language processing model to start the translation process.

[0786] Step 5:

[0787] The server sends a translation request to a natural language processing model, which translates it into the target language, taking into account the user's emotional state.

[0788] Step 6:

[0789] The server receives the translation result and sends it to the terminal.

[0790] Step 7:

[0791] The terminal displays the translation result received from the server to the user.

[0792] Text correction function

[0793] Step 1:

[0794] The user inputs the sentence to be corrected through the interface of the language learning device.

[0795] Step 2:

[0796] The terminal receives the user's input and sends the data to the server.

[0797] Step 3:

[0798] The server receives the input data and analyzes the user's emotions using an emotion engine.

[0799] Step 4:

[0800] Based on the sentiment analysis results, the server calls a natural language processing model to initiate the sentence correction process.

[0801] Step 5:

[0802] The server sends a request to generate a correction sentence to the natural language processing model, and the model corrects the sentence, taking into account the user's emotional state.

[0803] Step 6:

[0804] The server receives the correction result and transmits it to the terminal.

[0805] Step 7:

[0806] The terminal displays the correction results received from the server to the user.

[0807] Feedback function

[0808] Step 1:

[0809] The user sends a request through the interface of the language learning device for feedback on their input sentence and the corrected sentence.

[0810] Step 2:

[0811] The terminal receives the user's request and sends the data to the server.

[0812] Step 3:

[0813] The server receives the request and analyzes the user's emotions using an emotion engine.

[0814] Step 4:

[0815] Based on the sentiment analysis results, the server invokes a natural language processing model to initiate the feedback generation process.

[0816] Step 5:

[0817] The server sends a feedback generation request to the natural language processing model, which then compares the user's input sentence with the corrected sentence and generates feedback, taking into account the user's emotional state.

[0818] Step 6:

[0819] The server receives the feedback result and sends it to the terminal.

[0820] Step 7:

[0821] The terminal displays the feedback result received from the server to the user.

[0822] Example 2

[0823] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0824] Conventional language learning systems perform translation, correction, and feedback without considering the user's emotional state, which can affect the user's learning efficiency and motivation. Furthermore, due to the lack of a means to provide appropriate feedback according to emotions, it is not possible to provide adequate learning support tailored to each individual user.

[0825] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0826] In this invention, the server includes means for receiving a user's text input from a language learning device, means for translating the received user's text into a specified target language, means for correcting errors in the user's text, means for generating feedback by comparing the user's text with the corrected text, means for providing the generated feedback to the user, means for analyzing the user's emotions in processing the received text and the generated translation, correction results, or feedback, and means for reflecting the results of the user's emotion analysis in the processing results, thereby enabling appropriate translation, correction, and feedback that take the user's emotional state into consideration.

[0827] "Language learning device" refers to a device that allows a user to learn a language, and includes electronic devices such as smartphones, tablets, and personal computers.

[0828] "User" refers to an individual who uses this system to learn a language.

[0829] "Text" refers to documents or sentences entered by a user.

[0830] A "server" refers to a computer system that provides data processing and storage over a network.

[0831] "Means for receiving" refers to a function for transmitting the user's text from the language learning device to the server and for the server to receive the data.

[0832] "Means for translating" refers to functionality for converting received text into a specified target language, using natural language processing techniques such as generative AI models.

[0833] "Means for correcting errors" refers to functionality for correcting errors in received text, using natural language processing techniques such as generative AI models.

[0834] "Means for generating feedback" refers to the functionality that compares the user's text with the corrected text and generates learning feedback.

[0835] "Means for providing" refers to the functionality for displaying the generated feedback to the user.

[0836] "Means for analyzing emotions" refers to the function for reading and analyzing emotions from the user's input text and reactions, and uses technologies such as emotion engines.

[0837] "Means for reflecting emotion analysis results in processing results" refers to the function of adjusting the results of translation, correction, and feedback based on the analyzed emotion information.

[0838] MODE FOR CARRYING OUT THE INVENTION

[0839] This invention is a system that supports language learning for users using a language learning device, and also combines it with an emotion engine that recognizes and analyzes the user's emotions. The purpose of this system is to promote language learning efficiently while taking into account the user's emotional state.

[0840] Hardware and Software Configuration

[0841] Language learning device (terminal): An electronic device used by a user, such as a smartphone, tablet, or computer.

[0842] Server: A computer system that receives, analyzes, and processes data.

[0843] Emotion engine: Software for analyzing user emotions (e.g., emotion analysis API, emotion engine technology).

[0844] Natural Language Processing models: Generative AI models for translation, correction, and feedback generation (e.g., GPT-3, BERT, Google Cloud Translation API).

[0845] System Operation

[0846] 1. User Input: The user inputs the sentence they want translated or corrected into the language learning device. Examples of input data include "Hello, how are you?" or "I went to the store."

[0847] 2. The device receives and sends data: The device receives the entered data and sends it to the server over the internet. This process uses data transmission protocols and API calls.

[0848] 3. The server analyzes emotions: The server analyzes the received data and extracts sentences from it. At the same time, it uses an emotion engine to analyze the user's emotions. For example, it uses the Azure Emotion API to determine emotional states such as "positive," "negative," and "frustrated."

[0849] 4. The server performs translation and correction processing: The server takes into account the sentiment analysis results and calls the generative AI model for processing. Google Cloud Translation API is used for translation, and GPT-3 is used for correction.

[0850] 5. The server returns the results: The generated translation results and corrections are sent back to the terminal.

[0851] 6. Device displays: The device displays the received results to the user. The displayed results may include the translated sentence "Hello, how are you?" or the corrected sentence "I went to the store."

[0852] Specific operation example

[0853] Translation feature

[0854] Prompt statement:

[0855] "Please translate the following English sentence into Japanese. The user's emotional state is positive. 'Hello, how are you?'"

[0856] Text correction function

[0857] Prompt statement:

[0858] "Please correct the following sentence to use correct English grammar. The user's emotional state is negative. 'I went to the store.'"

[0859] Feedback function

[0860] Prompt statement:

[0861] "Please provide feedback on the corrected sentence below. The user's emotional state is frustration. Original: 'I went to the store.' Corrected: 'I went to the store.'"

[0862] This system can provide appropriate, personalized translation, correction, and feedback that takes into account the user's emotional state, thereby enabling efficient language learning support.

[0863] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0864] Step 1:

[0865] User input

[0866] The user inputs the sentence they wish to translate or correct and the target language into the language learning device. This input data includes text such as "Hello, how are you?" or "I went to the store." This text becomes the input data for the system.

[0867] Step 2:

[0868] Device receives and sends data

[0869] The terminal receives text data entered by the user. The received data is sent to the server via the Internet. This process uses an HTTP request to send the data. The data sent includes the entered text and the target language.

[0870] Step 3:

[0871] The server analyzes emotions

[0872] The server analyzes the received data and extracts text data. It then uses an emotion engine to analyze the user's emotions from the received text data. This emotion analysis is performed using, for example, the Azure Emotion API. The server outputs the emotional state, such as "positive," "negative," or "frustrated," as a result of the emotion analysis.

[0873] Step 4:

[0874] The server performs translation and correction processing

[0875] The server takes into account the sentiment analysis results and calls a natural language processing model (e.g., Google Cloud Translation API or GPT-3) to translate or correct the input text. In the case of translation, it converts the received text into the target language and generates a translated text. In the case of correction, it corrects errors and generates text that follows correct English grammar. The server outputs the generated translation or correction result.

[0876] Step 5:

[0877] The server returns the results

[0878] The server then returns the translation results and corrections to the device. The returned data includes the translated or corrected text, the user's sentiment analysis results, etc. This process uses HTTP responses to send data.

[0879] Step 6:

[0880] The device is displayed

[0881] The device then displays the results received from the server to the user. The displayed results include translated and corrected sentences. For example, the translation result for "Hello, how are you?" is "Hello, how are you?", and the correction result for "I went to the store." is "I went to the store."

[0882] This allows users to receive appropriate translations, corrections, and feedback that take emotions into account through the system.

[0883] (Application example 2)

[0884] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0885] Conventional language learning systems have difficulty providing a personalized learning experience that takes into account the user's emotional state. Furthermore, when learning using video content, the lack of real-time subtitle translation and feedback functions makes it difficult for users to learn efficiently.

[0886] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving a user's sentence input from a language learning device, means for translating the received user's sentence into a specified target language, means for correcting errors in the user's sentence, means for generating feedback by comparing the user's sentence with the corrected sentence, means for providing the generated feedback to the user, means for translating subtitles of videos watched by the user in real time, means for analyzing the user's emotions, and means for adjusting the translation results and feedback based on the analyzed emotions. This enables personalized, efficient, and interactive language learning that takes the user's emotional state into consideration.

[0887] A "language learning device" is an electronic device that a user uses when learning a language and that has input and display functions.

[0888] A "text" is a character string or sentence entered by a user.

[0889] "Target language" refers to the language that the user wishes to learn or translate into.

[0890] "Translation" is the act or process of converting from one language to another.

[0891] "Error correction" is the process of correcting grammatical or semantic errors in input text.

[0892] "Feedback" refers to information such as corrections, supplementary explanations, and evaluations provided to the user.

[0893] "Real-time subtitle translation" is the process of instantly translating subtitles displayed while watching a video into another language.

[0894] "Emotion analysis" is a technology that analyzes a user's emotional state at any given time based on their facial expressions and biometric information.

[0895] A "natural language processing model" is an artificial intelligence model for understanding and generating natural language, and uses machine learning and deep learning technologies.

[0896] A "server" is a computer system that provides computing resources and data to client devices via a network.

[0897] This invention is a system that supports language learning for users using a language learning device, and also combines it with an emotion engine that recognizes and analyzes the user's emotions. The purpose of this system is to efficiently advance language learning while taking into account the user's emotional state.

[0898] The server receives sentences entered by the user into the language learning device and translates them into the specified target language. First, the user enters the sentence they want to translate and the target language into the language learning device. The terminal receives this and sends it to the server. The server uses an emotion engine to analyze the user's emotions and performs translation by calling a natural language processing model while taking into account their emotional state. The results are returned to the terminal, which displays the translation results to the user.

[0899] Specifically, the server uses the Python-based natural language processing library googletrans for translation, and the DeepFace library for emotion analysis, analyzing the user's facial expression data captured by the camera. The hardware used is the laptop's built-in camera or an external webcam.

[0900] Suppose a user wants to translate the sentence "Hello, how are you?" into Japanese. The user inputs this sentence and "Japanese" into the device. The device sends this to the server, which uses an emotion engine to analyze the user's emotion. If the result is determined to be "positive," a natural language processing model is used for translation, and the translation result, "Hello, how are you?", is generated. The translation result is sent to the device, which displays it to the user.

[0901] The service also offers a real-time subtitle translation feature. The subtitles of the video the user is watching are translated in real time, and the tone and depth of the translation are adjusted according to the results of sentiment analysis. This optimizes the learning experience based on the user's emotional state. For example, if a user sets "English to Japanese translation" while watching a movie and the sentiment analysis determines "happy," the captions will be displayed in a positive tone, such as "I went to the store 😊."

[0902] An example prompt sentence, input to a generative AI model, is:

[0903] "When the user's emotional state is 'happy,' translate 'I went to the store.' into Japanese and add a positive expression."

[0904] In this way, more personalized and effective language learning assistance is possible by taking into account the user's emotional state.

[0905] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0906] Step 1:

[0907] A user inputs a sentence and a target language into a language learning device.

[0908] Specifically, the user inputs the sentence "Hello, how are you?" and "Japanese" into the terminal.

[0909] Step 2:

[0910] The terminal receives the input and sends it to the server.

[0911] The input data is "Hello, how are you?" and "Japanese."

[0912] The output is the transmitted input data.

[0913] Step 3:

[0914] The server invokes the emotion engine to analyze the received text.

[0915] The input data is the user's sentence "Hello, how are you?" and the emotion engine.

[0916] A deep learning model is used to analyze the user's emotions and determine them as "positive."

[0917] The output is the sentiment analysis result "positive."

[0918] Step 4:

[0919] The server calls a natural language processing model based on the sentiment analysis results and translates the text into the target language.

[0920] The input data is the user's sentence "Hello, how are you?", the target language "Japanese", and the sentiment analysis result "positive".

[0921] The translation result is "Hello, how are you?"

[0922] The output is the translation result "Hello, how are you?"

[0923] Step 5:

[0924] The server sends the translation results to the terminal.

[0925] The input data is the translation result "Hello, how are you?"

[0926] The output is sent from the server to the terminal.

[0927] Step 6:

[0928] The terminal receives the translation result and displays it to the user.

[0929] The input data is the translation result "Hello, how are you?" sent from the server.

[0930] The output is the translation displayed to the user.

[0931] Step 7:

[0932] A process is run to translate subtitles of videos watched by users in real time.

[0933] The input data is the subtitles of the video the user is watching, the target language "Japanese," and the user's emotional state.

[0934] Based on the results of sentiment analysis, appropriate expressions are added to the translation results and displayed.

[0935] The output is a real-time display of translated subtitles.

[0936] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0937] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0938] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0939] [Third embodiment]

[0940] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0941] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0942] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0943] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0944] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0945] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0946] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0947] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0948] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0949] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0950] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0951] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0952] The present invention is a system for assisting a user in language learning using a language learning device, which receives a user's text, translates the text into a specified target language, and provides feedback after correcting errors.

[0953] Program processing overview

[0954] Translation feature

[0955] The server translates the sentence entered by the user into the target language. When the user enters the sentence to be translated and the target language into the language learning device, it is sent to the server. The server uses a natural language processing model to translate the entered sentence into the specified target language. The translation result is sent from the server to the terminal, and the terminal displays the translation result to the user.

[0956] Specific examples

[0957] If a user wants to translate "Hello, how are you?" into Japanese, they input the sentence and "Japanese" into their terminal. The server receives this and translates it into "Hello, how are you?" using a natural language processing model. The result is sent to the user's terminal, and the terminal displays the translation result to the user.

[0958] Text correction function

[0959] The server corrects errors in sentences entered by the user. When a user enters a sentence to be corrected into a language learning device, it is sent to the server. The server uses a natural language processing model to correct the errors in the sentence. The server sends the correction results to the terminal, which then displays the correction results to the user.

[0960] Specific examples

[0961] If a user types "I went to the store," they enter that sentence into their terminal. The server receives this and uses a natural language processing model to correct it to "I went to the store." The result is sent to the user's terminal, which then displays the corrected result to the user.

[0962] Feedback function

[0963] The server compares the text entered by the user with the corrected text and provides feedback. If the user requests feedback on the corrections to their text, the feedback is sent to the server. The server compares the input text with the corrected text and generates feedback using a natural language processing model. This feedback is sent from the server to the terminal, which displays the feedback to the user.

[0964] Specific examples

[0965] If a user types "I went to the store," and then corrects it to "I went to the store," the user sends a request for feedback. The server receives this and uses a natural language processing model to generate feedback. For example, it generates feedback such as "'goed' is not a correct past tense. The correct form is 'went'." It then sends this feedback to the user's device, which then displays it to the user.

[0966] In this way, the present invention provides an environment in which users can learn languages ​​efficiently and accurately.

[0967] The processing flow will be explained below.

[0968] Translation feature

[0969] Step 1:

[0970] A user inputs a sentence to be translated and a target language through the interface of a language learning device.

[0971] Step 2:

[0972] The terminal receives the user's input and sends the data to the server.

[0973] Step 3:

[0974] The server receives the input data and initiates the translation process, specifically by calling a natural language processing model (e.g., a language model API).

[0975] Step 4:

[0976] The server sends a translation request to a natural language processing model, which translates it into the target language.

[0977] Step 5:

[0978] The server receives the translation result and returns it to the user terminal.

[0979] Step 6:

[0980] The terminal displays the translation result received from the server to the user.

[0981] Text correction function

[0982] Step 1:

[0983] The user inputs the sentence to be corrected through the interface of the language learning device.

[0984] Step 2:

[0985] The terminal receives the user's input and sends the data to the server.

[0986] Step 3:

[0987] The server receives the input data and initiates the sentence correction process, specifically by calling a natural language processing model (e.g., a language correction model API).

[0988] Step 4:

[0989] The server sends a request to generate a corrected sentence to the natural language processing model, and the model corrects the errors in the sentence.

[0990] Step 5:

[0991] The server receives the correction result and returns it to the user terminal.

[0992] Step 6:

[0993] The terminal displays the correction results received from the server to the user.

[0994] Feedback function

[0995] Step 1:

[0996] The user requests feedback on their input sentences and corrected sentences through the interface of the language learning device.

[0997] Step 2:

[0998] The terminal receives the user's request and sends the data to the server.

[0999] Step 3:

[1000] The server receives the request and starts the feedback generation process by calling a natural language processing model (e.g., a feedback generation model API).

[1001] Step 4:

[1002] The server sends a feedback generation request to the natural language processing model, and the model compares the user's input sentence with the corrected sentence and generates the feedback.

[1003] Step 5:

[1004] The server receives the generated feedback and returns it to the user terminal.

[1005] Step 6:

[1006] The terminal displays the feedback received from the server to the user.

[1007] Example 1

[1008] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1009] In conventional language learning systems, the accuracy and quality of the translation and correction results are often insufficient when translating user-input text into the target language and correcting errors. Furthermore, there is a lack of feedback to help users deepen their understanding of the language they are learning. This has resulted in a lack of an environment for users to learn languages ​​efficiently and accurately.

[1010] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1011] In this invention, the server includes means for receiving a sentence input by a user and a target language, means for translating the received user sentence into a specified target language, means for displaying the translation result to the user, means for correcting errors in the user sentence, means for comparing the user sentence with the corrected sentence and generating feedback, and means for providing the generated feedback to the user, thereby enabling the sentence input by the user to be translated with high accuracy, errors to be corrected, and useful feedback to be provided.

[1012] "User" refers to a person who uses a language learning device to input text and receive translation, correction, and feedback.

[1013] "Sentence" refers to text data that a user inputs through a language learning device.

[1014] "Target language" refers to the language into which the user specifies to translate the input text.

[1015] "Translation" refers to the process of converting user-entered text into a specified target language.

[1016] "Correction" refers to the process of correcting errors in text entered by a user.

[1017] "Feedback" refers to the evaluation and advice generated by the server regarding the corrections and translation results for the user's input text.

[1018] "Server" refers to a processor that receives requests from users and translates text, corrects it, and generates feedback.

[1019] "Terminal" refers to a device where a user inputs text and displays translation results, corrections, and feedback from the server.

[1020] "Generative AI model" refers to an artificial intelligence model used to perform natural language processing.

[1021] This invention is a system that supports language learning for users using language learning devices. The system consists of a server, a terminal, and a user. The user inputs a sentence and a target language, and the information is sent to the server. The server uses a natural language processing model to translate and correct the sentence and generate feedback.

[1022] Hardware and Software Configuration

[1023] server

[1024] The server is a computer system with a high-performance processor that works in conjunction with an external natural language processing model (e.g., Google Translate API, DeepL, Grammarly, LanguageTool). The server has software and APIs to receive and process user data.

[1025] Terminal

[1026] A terminal is a device operated by a user, such as a PC, tablet, or smartphone. The terminal includes an interface for user input and a display for displaying results from the server. The terminal has a network connection for communicating with the server.

[1027] User

[1028] The user is a language learner who uses a terminal to input sentences and receive translation, correction, and feedback. The user inputs sentences and their corresponding target language through the terminal interface.

[1029] Data processing and calculation

[1030] The server receives the text and target language sent by the user. The received data is translated into the specified target language using a generative AI model. The translation result is then sent back to the user's device, where it is displayed.

[1031] A natural language processing model is also used to correct errors in the user's writing. The correction results are also sent from the server to the terminal and displayed to the user. The server then compares the user's input sentence with the corrected sentence and generates feedback. This feedback is also provided to the user.

[1032] Specific examples

[1033] As a concrete example, consider the case where a user wants to translate "Hello, how are you?" into Japanese. The user inputs this sentence and "Japanese" into their device. The server receives this and translates it into "Hello, how are you?" using Google Translate API or DeepL. This result is sent to the user's device, which then displays the translation result.

[1034] Also, if a user types "I went to the store," Grammarly or LanguageTool can be used to correct the error in the sentence. The server receives this and corrects it to "I went to the store." This result is then sent back to the user's device, which then displays the correction.

[1035] Prompt Sentence Examples

[1036] 1. Translate the sentence entered by the user into the specified target language. The entered sentence is "Hello, how are you?" and the target language is Japanese.

[1037] 2. Correct the error in the sentence entered by the user. The sentence entered is "I went to the store."

[1038] 3. Compare the user-entered sentence with the corrected sentence and provide feedback. The original sentence is "I went to the store." The corrected sentence is "I went to the store."

[1039] This system allows users to learn languages ​​efficiently and accurately.

[1040] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1041] Step 1:

[1042] The user inputs a sentence and a target language.

[1043] Specifically, the user enters a sentence (e.g., "Hello, how are you?") and a target language (e.g., "Japanese") into a text input field on the terminal.

[1044] Input: User-entered text and target language

[1045] Output: Data entered into the terminal

[1046] Step 2:

[1047] The terminal transmits the input data to the server.

[1048] Specifically, when the user clicks the "send" button, the terminal puts the input text and the target language together into a data packet and sends it to the server.

[1049] Input: User-entered text and target language

[1050] Output: Data packets sent to the server

[1051] Step 3:

[1052] The server analyzes the received data and performs translation.

[1053] Specifically, the server analyzes the received data packets, extracts the input sentence and target language, and then uses a generative AI model (e.g., Google Translate API, DeepL) to translate the sentence into the specified target language.

[1054] Input: Data packet sent to the server (text and target language)

[1055] Output: The translated text

[1056] Step 4:

[1057] The server sends the translation results to the terminal.

[1058] Specifically, the server repackages the translation results into a data packet and sends it to the terminal, which also includes a confirmation of the data transmission.

[1059] Input: translated sentence

[1060] Output: Data packets sent to the terminal

[1061] Step 5:

[1062] The device will display the translation results.

[1063] Specifically, the device receives the data packet from the server and displays the translation result on the user's interface, for example, "Hello, how are you?"

[1064] Input: Data packet sent to the terminal (translation result)

[1065] Output: The translation result that is displayed to the user

[1066] Step 6:

[1067] When the user wishes to correct a sentence, the user inputs the sentence containing the error.

[1068] Specifically, the user inputs the sentence they wish to correct (e.g., "I went to the store.") into the terminal.

[1069] Input: A sentence containing an error

[1070] Output: Data entered into the terminal

[1071] Step 7:

[1072] The terminal sends the sentence to be corrected to the server.

[1073] Specifically, when the user clicks the "Send" button, the terminal assembles the sentence to be corrected into a data packet and sends it to the server.

[1074] Input: A sentence containing an error

[1075] Output: Data packets sent to the server

[1076] Step 8:

[1077] The server analyzes the received data and corrects the text.

[1078] Specifically, the server analyzes the received data packets and corrects errors using a generative AI model (e.g., Grammarly, LanguageTool).

[1079] Input: Data packet sent to the server (text containing an error)

[1080] Output: Corrected sentence

[1081] Step 9:

[1082] The server sends the correction results to the terminal.

[1083] Specifically, the server collects the correction results into a data packet and transmits it to the terminal, which also includes a confirmation of the data transmission.

[1084] Input: Corrected sentence

[1085] Output: Data packets sent to the terminal

[1086] Step 10:

[1087] The terminal will display the correction results.

[1088] Specifically, the terminal receives the data packet from the server and displays the correction result on the user interface, for example, "I went to the store."

[1089] Input: Data packet sent to the terminal (corrected result)

[1090] Output: Corrected results shown to the user

[1091] Step 11:

[1092] A user submits a request for feedback.

[1093] As a specific operation, the user clicks a button to request feedback on the correction results.

[1094] Input: Request for feedback

[1095] Output: Data entered into the terminal

[1096] Step 12:

[1097] The device sends a feedback request to the server.

[1098] Specifically, the terminal assembles the feedback request into a data packet and transmits it to the server.

[1099] Input: Feedback Request

[1100] Output: Data packets sent to the server

[1101] Step 13:

[1102] The server generates the feedback.

[1103] Specifically, the server generates feedback using a generative AI model based on the received feedback request, for example, by comparing the original sentence with the corrected sentence and explaining the cause of the error and the correct form.

[1104] Input: Feedback Request

[1105] Output: Generated feedback

[1106] Step 14:

[1107] The server sends the feedback to the device.

[1108] In particular, the server collects the generated feedback into a data packet and transmits it to the terminal.

[1109] Input: Generated feedback

[1110] Output: Data packets sent to the terminal

[1111] Step 15:

[1112] The device displays feedback.

[1113] Specifically, the device receives data packets from the server and displays feedback on the user's interface, such as "'goed' is not a correct past tense. The correct form is 'went'."

[1114] Input: Data packets sent to the terminal (feedback)

[1115] Output: Feedback that is displayed to the user

[1116] (Application example 1)

[1117] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1118] The present invention relates to a language learning support system that enables factory workers who speak different languages ​​to work efficiently and safely. Specifically, the system aims to reduce worker errors by supporting the understanding of technical terms and safety procedures, correcting incorrect terms and expressions, and providing appropriate feedback. Conventional systems have difficulty supporting a wide range of languages ​​and are inadequate in supporting technical terms and procedures specific to industrial machinery, so improvements are needed.

[1119] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1120] In this invention, the server includes means for receiving user sentences input from a language learning device, means for translating the received user sentences into a specified target language, means for correcting errors in the user sentences, means for generating feedback by comparing the user sentences with the corrected sentences, means for providing the generated feedback to the user, and means for supporting the terminology and procedures used in industrial machines through translation, correction, and feedback. This enables efficient and safe work by correcting errors in terminology and procedures used by factory workers in real time and providing appropriate feedback.

[1121] A "language learning device" is a device that allows a user to input text and processes the text to assist in language learning.

[1122] A "user's sentence" is any text entered by a user using a language learning device.

[1123] The "target language" is the language into which the user wants to translate the text.

[1124] A "means for translating" is a function or process for converting received text into a target language.

[1125] "Correction means" refers to the functions and processes for correcting errors in text entered by a user.

[1126] A "means for comparing and generating feedback" is a function or process for comparing the original and corrected texts and generating feedback that explains the differences.

[1127] A "means for providing feedback" is a function or process for notifying or displaying generated feedback to the user.

[1128] "Terminology and procedures used in commercial machinery" refers to technical terms and operating procedures required in a specific business environment, such as a factory.

[1129] A "natural language processing model" is an artificial intelligence algorithm or system designed to understand and process human language.

[1130] The present invention relates to a system for assisting a user in language learning using a language learning device, and for improving work efficiency and safety, particularly in a factory environment. The system includes the following main functions:

[1131] Translation feature

[1132] The server translates the sentence entered by the user into the target language. When the user enters the sentence to be translated and the target language into the language learning device, it is sent to the server. The server uses a natural language processing model (e.g., GPT-4) to translate the input sentence into the specified target language. The translation result is sent from the server to the device, and the device displays the translation result to the user.

[1133] Specific examples

[1134] If a user wants to translate "Please check the conveyor belt." into Spanish, they input the sentence and "Spanish" into their terminal. The server receives this and uses a natural language processing model to translate it into "Por favor, revise la cinta transportadora." The result is sent to the user's terminal, which then displays the translation to the user.

[1135] Text correction function

[1136] The server corrects errors in sentences entered by the user. When a user enters a sentence to be corrected into a language learning device, it is sent to the server. The server uses a natural language processing model to correct the errors in the sentence. The server sends the correction results to the terminal, which then displays the correction results to the user.

[1137] Specific examples

[1138] If a user types "Check the conveyor bolt," they enter that sentence into their terminal. The server receives it and uses a natural language processing model to correct it to "Check the conveyor bolt." The result is sent to the user's terminal, which then displays the corrected result to the user.

[1139] Feedback function

[1140] The server compares the text entered by the user with the corrected text and provides feedback. If the user requests feedback on the corrections to their text, the feedback is sent to the server. The server compares the input text with the corrected text and generates feedback using a natural language processing model. This feedback is sent from the server to the terminal, which displays the feedback to the user.

[1141] Specific examples

[1142] If a user types "Check the conveyor bolt." and then corrects it to "Check the conveyor bolt.", the user sends a request for feedback. The server receives this and uses a natural language processing model to generate feedback. For example, it generates feedback such as "'conbeyor' is a misspelling. The correct spelling is 'conveyor'." It then sends this feedback to the user's device, which then displays it to the user.

[1143] In this way, the present invention helps users learn language efficiently and accurately, and supports the understanding and application of technical terms and procedures, particularly in factory environments, thereby improving safety and work efficiency.

[1144] Example prompt sentence:

[1145] "Translate the following technical instruction from English to Spanish: 'Please check the conveyor belt.'"

[1146] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1147] Step 1:

[1148] A user inputs a sentence and a target language into a language learning device.

[1149] Input: The sentence you want to translate (e.g., "Please check the conveyor belt.") and the target language (e.g., Spanish)

[1150] Specific actions: The user enters a sentence and the target language into the input field on the device and presses the send button.

[1151] Step 2:

[1152] The terminal receives the user's input and sends it to the server.

[1153] Input: User-entered text and target language

[1154] Output: Input data sent to the server

[1155] Specific operation: The device packages the input data and sends a request to the API endpoint.

[1156] Step 3:

[1157] The server translates the received text into the target language using a natural language processing model.

[1158] Input: Received text and target language

[1159] Output: Translation result (e.g. "Por favor, revise la cinta transportadora.")

[1160] Specific operation: The server inputs the sentence and target language as prompts into the generative AI model (e.g., GPT-4) and obtains the translation result.

[1161] Step 4:

[1162] The server sends the translation results to the terminal.

[1163] Input: Translation result

[1164] Output: Translation results sent to your device

[1165] Specific operation: The server repackages the translation results and sends them to the device.

[1166] Step 5:

[1167] The terminal displays the translation results to the user.

[1168] Input: Translation result

[1169] Output: User-viewable translation results

[1170] Specific operation: The translation results received by the device are displayed on the screen.

[1171] Step 6:

[1172] The user inputs the sentence to be corrected.

[1173] Input: The sentence you want to correct (e.g., "Check the conveyor bolt.")

[1174] Specific operation: The user again enters text into the device's input field and presses the send button.

[1175] Step 7:

[1176] The terminal receives an input sentence for which an error correction is desired and transmits it to the server.

[1177] Input: The text entered by the user that you would like to have corrected

[1178] Output: Input data sent to the server

[1179] Specific operation: The device packages the input data and sends a request to the API endpoint.

[1180] Step 8:

[1181] The server corrects errors in the received text using a natural language processing model.

[1182] Input: Received text

[1183] Output: Correction result (e.g., "Check the conveyor bolt.")

[1184] Specific operation: The server inputs a sentence as a prompt into the generative AI model (e.g., GPT-4) and obtains the corrected result.

[1185] Step 9:

[1186] The server sends the correction results to the terminal.

[1187] Input: Corrected result

[1188] Output: Corrections sent to the terminal

[1189] Specific operation: The server repackages the correction results and sends them to the terminal.

[1190] Step 10:

[1191] The terminal displays the correction results to the user.

[1192] Input: Corrected result

[1193] Output: User-visible display of the corrections

[1194] Specific operation: The correction results received by the device are displayed on the screen.

[1195] Step 11:

[1196] A user submits a request for feedback.

[1197] Input: Request for feedback

[1198] Specific behavior: The user presses the feedback request button on the device.

[1199] Step 12:

[1200] The terminal receives the feedback request and sends it to the server.

[1201] Input: Feedback Request

[1202] Output: Request data sent to the server

[1203] Specific operation: The device packages the request data and sends the request to the API endpoint.

[1204] Step 13:

[1205] The server compares the original and corrected sentences and generates feedback using a natural language processing model.

[1206] Input: Original and corrected sentences

[1207] Output: Feedback (e.g. "'conbeyor' is an incorrect spelling. The correct spelling is 'conveyor'.")

[1208] Specific operation: The server inputs the original sentence and the corrected sentence as prompts to the generative AI model (e.g., GPT-4) and obtains feedback.

[1209] Step 14:

[1210] The server sends the feedback to the device.

[1211] Input: Feedback

[1212] Output: Feedback sent to the device

[1213] What happens: The server repackages the feedback and sends it to the device.

[1214] Step 15:

[1215] The device displays the feedback to the user.

[1216] Input: Feedback

[1217] Output: User-visible feedback display

[1218] Specific behavior: The device displays the feedback it receives on the screen.

[1219] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1220] This invention is a system that supports language learning using a language learning device, and also combines it with an emotion engine that recognizes and analyzes the user's emotions. The purpose of this system is to promote language learning efficiently while taking into account the user's emotional state. The system receives the user's text, translates it into the specified target language, corrects errors, and generates feedback. Furthermore, when the system receives the user's text, the emotion engine analyzes the user's emotions and reflects the results in the translation, correction, and feedback processes.

[1221] Program processing overview

[1222] Translation feature

[1223] The server translates sentences entered by the user into the target language. First, the user enters the sentence they want to translate and the target language into the language learning device. The terminal receives this and sends it to the server. The server uses an emotion engine to analyze the user's emotions and performs the translation by calling a natural language processing model while taking into account their emotional state. The results are returned to the terminal, which displays the translation results to the user.

[1224] Specific examples

[1225] Suppose a user wants to translate the sentence "Hello, how are you?" into Japanese. The user inputs this sentence and "Japanese" into the device. The device sends this to the server, which uses an emotion engine to analyze the user's emotion. If the result is determined to be "positive," a natural language processing model is used for translation, and the translation result, "Hello, how are you?", is generated. The translation result is sent to the device, which displays it to the user.

[1226] Text correction function

[1227] The server corrects errors in sentences entered by the user. The user enters the sentence they wish to correct into the language learning device, which receives it and sends it to the server. The server uses an emotion engine to analyze the user's emotions, takes that state into account, invokes a natural language processing model, and corrects the sentence. The correction results are returned to the device, which displays them to the user.

[1228] Specific examples

[1229] If a user types "I went to the store," they enter this into their device and send it to the server. The server uses an emotion engine to analyze the user's emotion. If the result is determined to be "negative," a natural language processing model is used to correct it, and the corrected result, "I went to the store," is generated. The corrected result is sent to the device, which then displays it to the user.

[1230] Feedback function

[1231] The server compares the sentence entered by the user with the corrected sentence and generates feedback. When the user requests feedback on the corrections to their sentence, the device sends the request to the server. The server uses an emotion engine to analyze the user's emotions, takes that state into account, invokes a natural language processing model, and generates feedback. The generated feedback is sent to the device, which displays it to the user.

[1232] Specific examples

[1233] If a user types "I went to the store," and then corrects it to "I went to the store," the user sends a request for feedback. The server uses an emotion engine to analyze the user's emotion. If the result is determined to be "frustration," a natural language processing model is used to generate feedback, such as "'goed' is not the correct past tense. 'went' is the correct form." This feedback is sent to the device, which then displays it to the user.

[1234] The present invention allows for more personalized and effective language learning assistance by taking into account the user's emotional state.

[1235] The processing flow will be explained below.

[1236] Translation feature

[1237] Step 1:

[1238] A user inputs a sentence to be translated and a target language through the interface of a language learning device.

[1239] Step 2:

[1240] The terminal receives the user's input and sends the data to the server.

[1241] Step 3:

[1242] The server receives the input data and analyzes the user's emotions using an emotion engine.

[1243] Step 4:

[1244] Based on the sentiment analysis results, the server calls a natural language processing model to start the translation process.

[1245] Step 5:

[1246] The server sends a translation request to a natural language processing model, which translates it into the target language, taking into account the user's emotional state.

[1247] Step 6:

[1248] The server receives the translation result and sends it to the terminal.

[1249] Step 7:

[1250] The terminal displays the translation result received from the server to the user.

[1251] Text correction function

[1252] Step 1:

[1253] The user inputs the sentence to be corrected through the interface of the language learning device.

[1254] Step 2:

[1255] The terminal receives the user's input and sends the data to the server.

[1256] Step 3:

[1257] The server receives the input data and analyzes the user's emotions using an emotion engine.

[1258] Step 4:

[1259] Based on the sentiment analysis results, the server calls a natural language processing model to initiate the sentence correction process.

[1260] Step 5:

[1261] The server sends a request to generate a correction sentence to the natural language processing model, and the model corrects the sentence, taking into account the user's emotional state.

[1262] Step 6:

[1263] The server receives the correction result and transmits it to the terminal.

[1264] Step 7:

[1265] The terminal displays the correction results received from the server to the user.

[1266] Feedback function

[1267] Step 1:

[1268] The user sends a request through the interface of the language learning device for feedback on their input sentence and the corrected sentence.

[1269] Step 2:

[1270] The terminal receives the user's request and sends the data to the server.

[1271] Step 3:

[1272] The server receives the request and analyzes the user's emotions using an emotion engine.

[1273] Step 4:

[1274] Based on the sentiment analysis results, the server invokes a natural language processing model to initiate the feedback generation process.

[1275] Step 5:

[1276] The server sends a feedback generation request to the natural language processing model, which then compares the user's input sentence with the corrected sentence and generates feedback, taking into account the user's emotional state.

[1277] Step 6:

[1278] The server receives the feedback result and sends it to the terminal.

[1279] Step 7:

[1280] The terminal displays the feedback result received from the server to the user.

[1281] Example 2

[1282] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1283] Conventional language learning systems perform translation, correction, and feedback without considering the user's emotional state, which can affect the user's learning efficiency and motivation. Furthermore, due to the lack of a means to provide appropriate feedback according to emotions, it is not possible to provide adequate learning support tailored to each individual user.

[1284] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1285] In this invention, the server includes means for receiving a user's text input from a language learning device, means for translating the received user's text into a specified target language, means for correcting errors in the user's text, means for generating feedback by comparing the user's text with the corrected text, means for providing the generated feedback to the user, means for analyzing the user's emotions in processing the received text and the generated translation, correction results, or feedback, and means for reflecting the results of the user's emotion analysis in the processing results, thereby enabling appropriate translation, correction, and feedback that take the user's emotional state into consideration.

[1286] "Language learning device" refers to a device that allows a user to learn a language, and includes electronic devices such as smartphones, tablets, and personal computers.

[1287] "User" refers to an individual who uses this system to learn a language.

[1288] "Text" refers to documents or sentences entered by a user.

[1289] A "server" refers to a computer system that provides data processing and storage over a network.

[1290] "Means for receiving" refers to a function for transmitting the user's text from the language learning device to the server and for the server to receive the data.

[1291] "Means for translating" refers to functionality for converting received text into a specified target language, using natural language processing techniques such as generative AI models.

[1292] "Means for correcting errors" refers to functionality for correcting errors in received text, using natural language processing techniques such as generative AI models.

[1293] "Means for generating feedback" refers to the functionality that compares the user's text with the corrected text and generates learning feedback.

[1294] "Means for providing" refers to the functionality for displaying the generated feedback to the user.

[1295] "Means for analyzing emotions" refers to the function for reading and analyzing emotions from the user's input text and reactions, and uses technologies such as emotion engines.

[1296] "Means for reflecting emotion analysis results in processing results" refers to the function of adjusting the results of translation, correction, and feedback based on the analyzed emotion information.

[1297] MODE FOR CARRYING OUT THE INVENTION

[1298] This invention is a system that supports language learning for users using a language learning device, and also combines it with an emotion engine that recognizes and analyzes the user's emotions. The purpose of this system is to promote language learning efficiently while taking into account the user's emotional state.

[1299] Hardware and Software Configuration

[1300] Language learning device (terminal): An electronic device used by a user, such as a smartphone, tablet, or computer.

[1301] Server: A computer system that receives, analyzes, and processes data.

[1302] Emotion engine: Software for analyzing user emotions (e.g., emotion analysis API, emotion engine technology).

[1303] Natural Language Processing models: Generative AI models for translation, correction, and feedback generation (e.g., GPT-3, BERT, Google Cloud Translation API).

[1304] System Operation

[1305] 1. User Input: The user inputs the sentence they want translated or corrected into the language learning device. Examples of input data include "Hello, how are you?" or "I went to the store."

[1306] 2. The device receives and sends data: The device receives the entered data and sends it to the server over the internet. This process uses data transmission protocols and API calls.

[1307] 3. The server analyzes emotions: The server analyzes the received data and extracts sentences from it. At the same time, it uses an emotion engine to analyze the user's emotions. For example, it uses the Azure Emotion API to determine emotional states such as "positive," "negative," and "frustrated."

[1308] 4. The server performs translation and correction processing: The server takes into account the sentiment analysis results and calls the generative AI model for processing. Google Cloud Translation API is used for translation, and GPT-3 is used for correction.

[1309] 5. The server returns the results: The generated translation results and corrections are sent back to the terminal.

[1310] 6. Device displays: The device displays the received results to the user. The displayed results may include the translated sentence "Hello, how are you?" or the corrected sentence "I went to the store."

[1311] Specific operation example

[1312] Translation feature

[1313] Prompt statement:

[1314] "Please translate the following English sentence into Japanese. The user's emotional state is positive. 'Hello, how are you?'"

[1315] Text correction function

[1316] Prompt statement:

[1317] "Please correct the following sentence to use correct English grammar. The user's emotional state is negative. 'I went to the store.'"

[1318] Feedback function

[1319] Prompt statement:

[1320] "Please provide feedback on the corrected sentence below. The user's emotional state is frustration. Original: 'I went to the store.' Corrected: 'I went to the store.'"

[1321] This system can provide appropriate, personalized translation, correction, and feedback that takes into account the user's emotional state, thereby enabling efficient language learning support.

[1322] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1323] Step 1:

[1324] User input

[1325] The user inputs the sentence they wish to translate or correct and the target language into the language learning device. This input data includes text such as "Hello, how are you?" or "I went to the store." This text becomes the input data for the system.

[1326] Step 2:

[1327] Device receives and sends data

[1328] The terminal receives text data entered by the user. The received data is sent to the server via the Internet. This process uses an HTTP request to send the data. The data sent includes the entered text and the target language.

[1329] Step 3:

[1330] The server analyzes emotions

[1331] The server analyzes the received data and extracts text data. It then uses an emotion engine to analyze the user's emotions from the received text data. This emotion analysis is performed using, for example, the Azure Emotion API. The server outputs the emotional state, such as "positive," "negative," or "frustrated," as a result of the emotion analysis.

[1332] Step 4:

[1333] The server performs translation and correction processing

[1334] The server takes into account the sentiment analysis results and calls a natural language processing model (e.g., Google Cloud Translation API or GPT-3) to translate or correct the input text. In the case of translation, it converts the received text into the target language and generates a translated text. In the case of correction, it corrects errors and generates text that follows correct English grammar. The server outputs the generated translation or correction result.

[1335] Step 5:

[1336] The server returns the results

[1337] The server then returns the translation results and corrections to the device. The returned data includes the translated or corrected text, the user's sentiment analysis results, etc. This process uses HTTP responses to send data.

[1338] Step 6:

[1339] The device is displayed

[1340] The device then displays the results received from the server to the user. The displayed results include translated and corrected sentences. For example, the translation result for "Hello, how are you?" is "Hello, how are you?", and the correction result for "I went to the store." is "I went to the store."

[1341] This allows users to receive appropriate translations, corrections, and feedback that take emotions into account through the system.

[1342] (Application example 2)

[1343] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1344] Conventional language learning systems have difficulty providing a personalized learning experience that takes into account the user's emotional state. Furthermore, when learning using video content, the lack of real-time subtitle translation and feedback functions makes it difficult for users to learn efficiently.

[1345] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving a user's sentence input from a language learning device, means for translating the received user's sentence into a specified target language, means for correcting errors in the user's sentence, means for generating feedback by comparing the user's sentence with the corrected sentence, means for providing the generated feedback to the user, means for translating subtitles of videos watched by the user in real time, means for analyzing the user's emotions, and means for adjusting the translation results and feedback based on the analyzed emotions. This enables personalized, efficient, and interactive language learning that takes the user's emotional state into consideration.

[1346] A "language learning device" is an electronic device that a user uses when learning a language and that has input and display functions.

[1347] A "text" is a character string or sentence entered by a user.

[1348] "Target language" refers to the language that the user wishes to learn or translate into.

[1349] "Translation" is the act or process of converting from one language to another.

[1350] "Error correction" is the process of correcting grammatical or semantic errors in input text.

[1351] "Feedback" refers to information such as corrections, supplementary explanations, and evaluations provided to the user.

[1352] "Real-time subtitle translation" is the process of instantly translating subtitles displayed while watching a video into another language.

[1353] "Emotion analysis" is a technology that analyzes a user's emotional state at any given time based on their facial expressions and biometric information.

[1354] A "natural language processing model" is an artificial intelligence model for understanding and generating natural language, and uses machine learning and deep learning technologies.

[1355] A "server" is a computer system that provides computing resources and data to client devices via a network.

[1356] This invention is a system that supports language learning for users using a language learning device, and also combines it with an emotion engine that recognizes and analyzes the user's emotions. The purpose of this system is to efficiently advance language learning while taking into account the user's emotional state.

[1357] The server receives sentences entered by the user into the language learning device and translates them into the specified target language. First, the user enters the sentence they want to translate and the target language into the language learning device. The terminal receives this and sends it to the server. The server uses an emotion engine to analyze the user's emotions and performs translation by calling a natural language processing model while taking into account their emotional state. The results are returned to the terminal, which displays the translation results to the user.

[1358] Specifically, the server uses the Python-based natural language processing library googletrans for translation, and the DeepFace library for emotion analysis, analyzing the user's facial expression data captured by the camera. The hardware used is the laptop's built-in camera or an external webcam.

[1359] Suppose a user wants to translate the sentence "Hello, how are you?" into Japanese. The user inputs this sentence and "Japanese" into the device. The device sends this to the server, which uses an emotion engine to analyze the user's emotion. If the result is determined to be "positive," a natural language processing model is used for translation, and the translation result, "Hello, how are you?", is generated. The translation result is sent to the device, which displays it to the user.

[1360] The service also offers a real-time subtitle translation feature. The subtitles of the video the user is watching are translated in real time, and the tone and depth of the translation are adjusted according to the results of sentiment analysis. This optimizes the learning experience based on the user's emotional state. For example, if a user sets "English to Japanese translation" while watching a movie and the sentiment analysis determines "happy," the captions will be displayed in a positive tone, such as "I went to the store 😊."

[1361] An example prompt sentence, input to a generative AI model, is:

[1362] "When the user's emotional state is 'happy,' translate 'I went to the store.' into Japanese and add a positive expression."

[1363] In this way, more personalized and effective language learning assistance is possible by taking into account the user's emotional state.

[1364] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1365] Step 1:

[1366] A user inputs a sentence and a target language into a language learning device.

[1367] Specifically, the user inputs the sentence "Hello, how are you?" and "Japanese" into the terminal.

[1368] Step 2:

[1369] The terminal receives the input and sends it to the server.

[1370] The input data is "Hello, how are you?" and "Japanese."

[1371] The output is the transmitted input data.

[1372] Step 3:

[1373] The server invokes the emotion engine to analyze the received text.

[1374] The input data is the user's sentence "Hello, how are you?" and the emotion engine.

[1375] A deep learning model is used to analyze the user's emotions and determine them as "positive."

[1376] The output is the sentiment analysis result "positive."

[1377] Step 4:

[1378] The server calls a natural language processing model based on the sentiment analysis results and translates the text into the target language.

[1379] The input data is the user's sentence "Hello, how are you?", the target language "Japanese", and the sentiment analysis result "positive".

[1380] The translation result is "Hello, how are you?"

[1381] The output is the translation result "Hello, how are you?"

[1382] Step 5:

[1383] The server sends the translation results to the terminal.

[1384] The input data is the translation result "Hello, how are you?"

[1385] The output is sent from the server to the terminal.

[1386] Step 6:

[1387] The terminal receives the translation result and displays it to the user.

[1388] The input data is the translation result "Hello, how are you?" sent from the server.

[1389] The output is the translation displayed to the user.

[1390] Step 7:

[1391] A process is run to translate subtitles of videos watched by users in real time.

[1392] The input data is the subtitles of the video the user is watching, the target language "Japanese," and the user's emotional state.

[1393] Based on the results of sentiment analysis, appropriate expressions are added to the translation results and displayed.

[1394] The output is a real-time display of translated subtitles.

[1395] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1396] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1397] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1398] [Fourth embodiment]

[1399] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1400] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1401] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1402] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1403] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1404] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1405] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1406] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1407] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1408] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1409] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1410] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1411] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1412] The present invention is a system for assisting a user in language learning using a language learning device, which receives a user's text, translates the text into a specified target language, and provides feedback after correcting errors.

[1413] Program processing overview

[1414] Translation feature

[1415] The server translates the sentence entered by the user into the target language. When the user enters the sentence to be translated and the target language into the language learning device, it is sent to the server. The server uses a natural language processing model to translate the entered sentence into the specified target language. The translation result is sent from the server to the terminal, and the terminal displays the translation result to the user.

[1416] Specific examples

[1417] If a user wants to translate "Hello, how are you?" into Japanese, they input the sentence and "Japanese" into their terminal. The server receives this and translates it into "Hello, how are you?" using a natural language processing model. The result is sent to the user's terminal, and the terminal displays the translation result to the user.

[1418] Text correction function

[1419] The server corrects errors in sentences entered by the user. When a user enters a sentence to be corrected into a language learning device, it is sent to the server. The server uses a natural language processing model to correct the errors in the sentence. The server sends the correction results to the terminal, which then displays the correction results to the user.

[1420] Specific examples

[1421] If a user types "I went to the store," they enter that sentence into their terminal. The server receives this and uses a natural language processing model to correct it to "I went to the store." The result is sent to the user's terminal, which then displays the corrected result to the user.

[1422] Feedback function

[1423] The server compares the text entered by the user with the corrected text and provides feedback. If the user requests feedback on the corrections to their text, the feedback is sent to the server. The server compares the input text with the corrected text and generates feedback using a natural language processing model. This feedback is sent from the server to the terminal, which displays the feedback to the user.

[1424] Specific examples

[1425] If a user types "I went to the store," and then corrects it to "I went to the store," the user sends a request for feedback. The server receives this and uses a natural language processing model to generate feedback. For example, it generates feedback such as "'goed' is not a correct past tense. The correct form is 'went'." It then sends this feedback to the user's device, which then displays it to the user.

[1426] In this way, the present invention provides an environment in which users can learn languages ​​efficiently and accurately.

[1427] The processing flow will be explained below.

[1428] Translation feature

[1429] Step 1:

[1430] A user inputs a sentence to be translated and a target language through the interface of a language learning device.

[1431] Step 2:

[1432] The terminal receives the user's input and sends the data to the server.

[1433] Step 3:

[1434] The server receives the input data and initiates the translation process, specifically by calling a natural language processing model (e.g., a language model API).

[1435] Step 4:

[1436] The server sends a translation request to a natural language processing model, which translates it into the target language.

[1437] Step 5:

[1438] The server receives the translation result and returns it to the user terminal.

[1439] Step 6:

[1440] The terminal displays the translation result received from the server to the user.

[1441] Text correction function

[1442] Step 1:

[1443] The user inputs the sentence to be corrected through the interface of the language learning device.

[1444] Step 2:

[1445] The terminal receives the user's input and sends the data to the server.

[1446] Step 3:

[1447] The server receives the input data and initiates the sentence correction process, specifically by calling a natural language processing model (e.g., a language correction model API).

[1448] Step 4:

[1449] The server sends a request to generate a corrected sentence to the natural language processing model, and the model corrects the errors in the sentence.

[1450] Step 5:

[1451] The server receives the correction result and returns it to the user terminal.

[1452] Step 6:

[1453] The terminal displays the correction results received from the server to the user.

[1454] Feedback function

[1455] Step 1:

[1456] The user requests feedback on their input sentences and corrected sentences through the interface of the language learning device.

[1457] Step 2:

[1458] The terminal receives the user's request and sends the data to the server.

[1459] Step 3:

[1460] The server receives the request and starts the feedback generation process by calling a natural language processing model (e.g., a feedback generation model API).

[1461] Step 4:

[1462] The server sends a feedback generation request to the natural language processing model, and the model compares the user's input sentence with the corrected sentence and generates the feedback.

[1463] Step 5:

[1464] The server receives the generated feedback and returns it to the user terminal.

[1465] Step 6:

[1466] The terminal displays the feedback received from the server to the user.

[1467] Example 1

[1468] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1469] In conventional language learning systems, the accuracy and quality of the translation and correction results are often insufficient when translating user-input text into the target language and correcting errors. Furthermore, there is a lack of feedback to help users deepen their understanding of the language they are learning. This has resulted in a lack of an environment for users to learn languages ​​efficiently and accurately.

[1470] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1471] In this invention, the server includes means for receiving a sentence input by a user and a target language, means for translating the received user sentence into a specified target language, means for displaying the translation result to the user, means for correcting errors in the user sentence, means for comparing the user sentence with the corrected sentence and generating feedback, and means for providing the generated feedback to the user, thereby enabling the sentence input by the user to be translated with high accuracy, errors to be corrected, and useful feedback to be provided.

[1472] "User" refers to a person who uses a language learning device to input text and receive translation, correction, and feedback.

[1473] "Sentence" refers to text data that a user inputs through a language learning device.

[1474] "Target language" refers to the language into which the user specifies to translate the input text.

[1475] "Translation" refers to the process of converting user-entered text into a specified target language.

[1476] "Correction" refers to the process of correcting errors in text entered by a user.

[1477] "Feedback" refers to the evaluation and advice generated by the server regarding the corrections and translation results for the user's input text.

[1478] "Server" refers to a processor that receives requests from users and translates text, corrects it, and generates feedback.

[1479] "Terminal" refers to a device where a user inputs text and displays translation results, corrections, and feedback from the server.

[1480] "Generative AI model" refers to an artificial intelligence model used to perform natural language processing.

[1481] This invention is a system that supports language learning for users using language learning devices. The system consists of a server, a terminal, and a user. The user inputs a sentence and a target language, and the information is sent to the server. The server uses a natural language processing model to translate and correct the sentence and generate feedback.

[1482] Hardware and Software Configuration

[1483] server

[1484] The server is a computer system with a high-performance processor that works in conjunction with an external natural language processing model (e.g., Google Translate API, DeepL, Grammarly, LanguageTool). The server has software and APIs to receive and process user data.

[1485] Terminal

[1486] A terminal is a device operated by a user, such as a PC, tablet, or smartphone. The terminal includes an interface for user input and a display for displaying results from the server. The terminal has a network connection for communicating with the server.

[1487] User

[1488] The user is a language learner who uses a terminal to input sentences and receive translation, correction, and feedback. The user inputs sentences and their corresponding target language through the terminal interface.

[1489] Data processing and calculation

[1490] The server receives the text and target language sent by the user. The received data is translated into the specified target language using a generative AI model. The translation result is then sent back to the user's device, where it is displayed.

[1491] A natural language processing model is also used to correct errors in the user's writing. The correction results are also sent from the server to the terminal and displayed to the user. The server then compares the user's input sentence with the corrected sentence and generates feedback. This feedback is also provided to the user.

[1492] Specific examples

[1493] As a concrete example, consider the case where a user wants to translate "Hello, how are you?" into Japanese. The user inputs this sentence and "Japanese" into their device. The server receives this and translates it into "Hello, how are you?" using Google Translate API or DeepL. This result is sent to the user's device, which then displays the translation result.

[1494] Also, if a user types "I went to the store," Grammarly or LanguageTool can be used to correct the error in the sentence. The server receives this and corrects it to "I went to the store." This result is then sent back to the user's device, which then displays the correction.

[1495] Prompt Sentence Examples

[1496] 1. Translate the sentence entered by the user into the specified target language. The entered sentence is "Hello, how are you?" and the target language is Japanese.

[1497] 2. Correct the error in the sentence entered by the user. The sentence entered is "I went to the store."

[1498] 3. Compare the user-entered sentence with the corrected sentence and provide feedback. The original sentence is "I went to the store." The corrected sentence is "I went to the store."

[1499] This system allows users to learn languages ​​efficiently and accurately.

[1500] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1501] Step 1:

[1502] The user inputs a sentence and a target language.

[1503] Specifically, the user enters a sentence (e.g., "Hello, how are you?") and a target language (e.g., "Japanese") into a text input field on the terminal.

[1504] Input: User-entered text and target language

[1505] Output: Data entered into the terminal

[1506] Step 2:

[1507] The terminal transmits the input data to the server.

[1508] Specifically, when the user clicks the "send" button, the terminal puts the input text and the target language together into a data packet and sends it to the server.

[1509] Input: User-entered text and target language

[1510] Output: Data packets sent to the server

[1511] Step 3:

[1512] The server analyzes the received data and performs translation.

[1513] Specifically, the server analyzes the received data packets, extracts the input sentence and target language, and then uses a generative AI model (e.g., Google Translate API, DeepL) to translate the sentence into the specified target language.

[1514] Input: Data packet sent to the server (text and target language)

[1515] Output: The translated text

[1516] Step 4:

[1517] The server sends the translation results to the terminal.

[1518] Specifically, the server repackages the translation results into a data packet and sends it to the terminal, which also includes a confirmation of the data transmission.

[1519] Input: translated sentence

[1520] Output: Data packets sent to the terminal

[1521] Step 5:

[1522] The device will display the translation results.

[1523] Specifically, the device receives the data packet from the server and displays the translation result on the user's interface, for example, "Hello, how are you?"

[1524] Input: Data packet sent to the terminal (translation result)

[1525] Output: The translation result that is displayed to the user

[1526] Step 6:

[1527] When the user wishes to correct a sentence, the user inputs the sentence containing the error.

[1528] Specifically, the user inputs the sentence they wish to correct (e.g., "I went to the store.") into the terminal.

[1529] Input: A sentence containing an error

[1530] Output: Data entered into the terminal

[1531] Step 7:

[1532] The terminal sends the sentence to be corrected to the server.

[1533] Specifically, when the user clicks the "Send" button, the terminal assembles the sentence to be corrected into a data packet and sends it to the server.

[1534] Input: A sentence containing an error

[1535] Output: Data packets sent to the server

[1536] Step 8:

[1537] The server analyzes the received data and corrects the text.

[1538] Specifically, the server analyzes the received data packets and corrects errors using a generative AI model (e.g., Grammarly, LanguageTool).

[1539] Input: Data packet sent to the server (text containing an error)

[1540] Output: Corrected sentence

[1541] Step 9:

[1542] The server sends the correction results to the terminal.

[1543] Specifically, the server collects the correction results into a data packet and transmits it to the terminal, which also includes a confirmation of the data transmission.

[1544] Input: Corrected sentence

[1545] Output: Data packets sent to the terminal

[1546] Step 10:

[1547] The terminal will display the correction results.

[1548] Specifically, the terminal receives the data packet from the server and displays the correction result on the user interface, for example, "I went to the store."

[1549] Input: Data packet sent to the terminal (corrected result)

[1550] Output: Corrected results shown to the user

[1551] Step 11:

[1552] A user submits a request for feedback.

[1553] As a specific operation, the user clicks a button to request feedback on the correction results.

[1554] Input: Request for feedback

[1555] Output: Data entered into the terminal

[1556] Step 12:

[1557] The device sends a feedback request to the server.

[1558] Specifically, the terminal assembles the feedback request into a data packet and transmits it to the server.

[1559] Input: Feedback Request

[1560] Output: Data packets sent to the server

[1561] Step 13:

[1562] The server generates the feedback.

[1563] Specifically, the server generates feedback using a generative AI model based on the received feedback request, for example, by comparing the original sentence with the corrected sentence and explaining the cause of the error and the correct form.

[1564] Input: Feedback Request

[1565] Output: Generated feedback

[1566] Step 14:

[1567] The server sends the feedback to the device.

[1568] In particular, the server collects the generated feedback into a data packet and transmits it to the terminal.

[1569] Input: Generated feedback

[1570] Output: Data packets sent to the terminal

[1571] Step 15:

[1572] The device displays feedback.

[1573] Specifically, the device receives data packets from the server and displays feedback on the user's interface, such as "'goed' is not a correct past tense. The correct form is 'went'."

[1574] Input: Data packets sent to the terminal (feedback)

[1575] Output: Feedback that is displayed to the user

[1576] (Application example 1)

[1577] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1578] The present invention relates to a language learning support system that enables factory workers who speak different languages ​​to work efficiently and safely. Specifically, the system aims to reduce worker errors by supporting the understanding of technical terms and safety procedures, correcting incorrect terms and expressions, and providing appropriate feedback. Conventional systems have difficulty supporting a wide range of languages ​​and are inadequate in supporting technical terms and procedures specific to industrial machinery, so improvements are needed.

[1579] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1580] In this invention, the server includes means for receiving user sentences input from a language learning device, means for translating the received user sentences into a specified target language, means for correcting errors in the user sentences, means for generating feedback by comparing the user sentences with the corrected sentences, means for providing the generated feedback to the user, and means for supporting the terminology and procedures used in industrial machines through translation, correction, and feedback. This enables efficient and safe work by correcting errors in terminology and procedures used by factory workers in real time and providing appropriate feedback.

[1581] A "language learning device" is a device that allows a user to input text and processes the text to assist in language learning.

[1582] A "user's sentence" is any text entered by a user using a language learning device.

[1583] The "target language" is the language into which the user wants to translate the text.

[1584] A "means for translating" is a function or process for converting received text into a target language.

[1585] "Correction means" refers to the functions and processes for correcting errors in text entered by a user.

[1586] A "means for comparing and generating feedback" is a function or process for comparing the original and corrected texts and generating feedback that explains the differences.

[1587] A "means for providing feedback" is a function or process for notifying or displaying generated feedback to the user.

[1588] "Terminology and procedures used in commercial machinery" refers to technical terms and operating procedures required in a specific business environment, such as a factory.

[1589] A "natural language processing model" is an artificial intelligence algorithm or system designed to understand and process human language.

[1590] The present invention relates to a system for assisting a user in language learning using a language learning device, and for improving work efficiency and safety, particularly in a factory environment. The system includes the following main functions:

[1591] Translation feature

[1592] The server translates the sentence entered by the user into the target language. When the user enters the sentence to be translated and the target language into the language learning device, it is sent to the server. The server uses a natural language processing model (e.g., GPT-4) to translate the input sentence into the specified target language. The translation result is sent from the server to the device, and the device displays the translation result to the user.

[1593] Specific examples

[1594] If a user wants to translate "Please check the conveyor belt." into Spanish, they input the sentence and "Spanish" into their terminal. The server receives this and uses a natural language processing model to translate it into "Por favor, revise la cinta transportadora." The result is sent to the user's terminal, which then displays the translation to the user.

[1595] Text correction function

[1596] The server corrects errors in sentences entered by the user. When a user enters a sentence to be corrected into a language learning device, it is sent to the server. The server uses a natural language processing model to correct the errors in the sentence. The server sends the correction results to the terminal, which then displays the correction results to the user.

[1597] Specific examples

[1598] If a user types "Check the conveyor bolt," they enter that sentence into their terminal. The server receives it and uses a natural language processing model to correct it to "Check the conveyor bolt." The result is sent to the user's terminal, which then displays the corrected result to the user.

[1599] Feedback function

[1600] The server compares the text entered by the user with the corrected text and provides feedback. If the user requests feedback on the corrections to their text, the feedback is sent to the server. The server compares the input text with the corrected text and generates feedback using a natural language processing model. This feedback is sent from the server to the terminal, which displays the feedback to the user.

[1601] Specific examples

[1602] If a user types "Check the conveyor bolt." and then corrects it to "Check the conveyor bolt.", the user sends a request for feedback. The server receives this and uses a natural language processing model to generate feedback. For example, it generates feedback such as "'conbeyor' is a misspelling. The correct spelling is 'conveyor'." It then sends this feedback to the user's device, which then displays it to the user.

[1603] In this way, the present invention helps users learn language efficiently and accurately, and supports the understanding and application of technical terms and procedures, particularly in factory environments, thereby improving safety and work efficiency.

[1604] Example prompt sentence:

[1605] "Translate the following technical instruction from English to Spanish: 'Please check the conveyor belt.'"

[1606] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1607] Step 1:

[1608] A user inputs a sentence and a target language into a language learning device.

[1609] Input: The sentence you want to translate (e.g., "Please check the conveyor belt.") and the target language (e.g., Spanish)

[1610] Specific actions: The user enters a sentence and the target language into the input field on the device and presses the send button.

[1611] Step 2:

[1612] The terminal receives the user's input and sends it to the server.

[1613] Input: User-entered text and target language

[1614] Output: Input data sent to the server

[1615] Specific operation: The device packages the input data and sends a request to the API endpoint.

[1616] Step 3:

[1617] The server translates the received text into the target language using a natural language processing model.

[1618] Input: Received text and target language

[1619] Output: Translation result (e.g. "Por favor, revise la cinta transportadora.")

[1620] Specific operation: The server inputs the sentence and target language as prompts into the generative AI model (e.g., GPT-4) and obtains the translation result.

[1621] Step 4:

[1622] The server sends the translation results to the terminal.

[1623] Input: Translation result

[1624] Output: Translation results sent to your device

[1625] Specific operation: The server repackages the translation results and sends them to the device.

[1626] Step 5:

[1627] The terminal displays the translation results to the user.

[1628] Input: Translation result

[1629] Output: User-viewable translation results

[1630] Specific operation: The translation results received by the device are displayed on the screen.

[1631] Step 6:

[1632] The user inputs the sentence to be corrected.

[1633] Input: The sentence you want to correct (e.g., "Check the conveyor bolt.")

[1634] Specific operation: The user again enters text into the device's input field and presses the send button.

[1635] Step 7:

[1636] The terminal receives an input sentence for which an error correction is desired and transmits it to the server.

[1637] Input: The text entered by the user that you would like to have corrected

[1638] Output: Input data sent to the server

[1639] Specific operation: The device packages the input data and sends a request to the API endpoint.

[1640] Step 8:

[1641] The server corrects errors in the received text using a natural language processing model.

[1642] Input: Received text

[1643] Output: Correction result (e.g., "Check the conveyor bolt.")

[1644] Specific operation: The server inputs a sentence as a prompt into the generative AI model (e.g., GPT-4) and obtains the corrected result.

[1645] Step 9:

[1646] The server sends the correction results to the terminal.

[1647] Input: Corrected result

[1648] Output: Corrections sent to the terminal

[1649] Specific operation: The server repackages the correction results and sends them to the terminal.

[1650] Step 10:

[1651] The terminal displays the correction results to the user.

[1652] Input: Corrected result

[1653] Output: User-visible display of the corrections

[1654] Specific operation: The correction results received by the device are displayed on the screen.

[1655] Step 11:

[1656] A user submits a request for feedback.

[1657] Input: Request for feedback

[1658] Specific behavior: The user presses the feedback request button on the device.

[1659] Step 12:

[1660] The terminal receives the feedback request and sends it to the server.

[1661] Input: Feedback Request

[1662] Output: Request data sent to the server

[1663] Specific operation: The device packages the request data and sends the request to the API endpoint.

[1664] Step 13:

[1665] The server compares the original and corrected sentences and generates feedback using a natural language processing model.

[1666] Input: Original and corrected sentences

[1667] Output: Feedback (e.g. "'conbeyor' is an incorrect spelling. The correct spelling is 'conveyor'.")

[1668] Specific operation: The server inputs the original sentence and the corrected sentence as prompts to the generative AI model (e.g., GPT-4) and obtains feedback.

[1669] Step 14:

[1670] The server sends the feedback to the device.

[1671] Input: Feedback

[1672] Output: Feedback sent to the device

[1673] What happens: The server repackages the feedback and sends it to the device.

[1674] Step 15:

[1675] The device displays the feedback to the user.

[1676] Input: Feedback

[1677] Output: User-visible feedback display

[1678] Specific behavior: The device displays the feedback it receives on the screen.

[1679] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1680] This invention is a system that supports language learning using a language learning device, and also combines it with an emotion engine that recognizes and analyzes the user's emotions. The purpose of this system is to promote language learning efficiently while taking into account the user's emotional state. The system receives the user's text, translates it into the specified target language, corrects errors, and generates feedback. Furthermore, when the system receives the user's text, the emotion engine analyzes the user's emotions and reflects the results in the translation, correction, and feedback processes.

[1681] Program processing overview

[1682] Translation feature

[1683] The server translates sentences entered by the user into the target language. First, the user enters the sentence they want to translate and the target language into the language learning device. The terminal receives this and sends it to the server. The server uses an emotion engine to analyze the user's emotions and performs the translation by calling a natural language processing model while taking into account their emotional state. The results are returned to the terminal, which displays the translation results to the user.

[1684] Specific examples

[1685] Suppose a user wants to translate the sentence "Hello, how are you?" into Japanese. The user inputs this sentence and "Japanese" into the device. The device sends this to the server, which uses an emotion engine to analyze the user's emotion. If the result is determined to be "positive," a natural language processing model is used for translation, and the translation result, "Hello, how are you?", is generated. The translation result is sent to the device, which displays it to the user.

[1686] Text correction function

[1687] The server corrects errors in sentences entered by the user. The user enters the sentence they wish to correct into the language learning device, which receives it and sends it to the server. The server uses an emotion engine to analyze the user's emotions, takes that state into account, invokes a natural language processing model, and corrects the sentence. The correction results are returned to the device, which displays them to the user.

[1688] Specific examples

[1689] If a user types "I went to the store," they enter this into their device and send it to the server. The server uses an emotion engine to analyze the user's emotion. If the result is determined to be "negative," a natural language processing model is used to correct it, and the corrected result, "I went to the store," is generated. The corrected result is sent to the device, which then displays it to the user.

[1690] Feedback function

[1691] The server compares the sentence entered by the user with the corrected sentence and generates feedback. When the user requests feedback on the corrections to their sentence, the device sends the request to the server. The server uses an emotion engine to analyze the user's emotions, takes that state into account, invokes a natural language processing model, and generates feedback. The generated feedback is sent to the device, which displays it to the user.

[1692] Specific examples

[1693] If a user types "I went to the store," and then corrects it to "I went to the store," the user sends a request for feedback. The server uses an emotion engine to analyze the user's emotion. If the result is determined to be "frustration," a natural language processing model is used to generate feedback, such as "'goed' is not the correct past tense. 'went' is the correct form." This feedback is sent to the device, which then displays it to the user.

[1694] The present invention allows for more personalized and effective language learning assistance by taking into account the user's emotional state.

[1695] The processing flow will be explained below.

[1696] Translation feature

[1697] Step 1:

[1698] A user inputs a sentence to be translated and a target language through the interface of a language learning device.

[1699] Step 2:

[1700] The terminal receives the user's input and sends the data to the server.

[1701] Step 3:

[1702] The server receives the input data and analyzes the user's emotions using an emotion engine.

[1703] Step 4:

[1704] Based on the sentiment analysis results, the server calls a natural language processing model to start the translation process.

[1705] Step 5:

[1706] The server sends a translation request to a natural language processing model, which translates it into the target language, taking into account the user's emotional state.

[1707] Step 6:

[1708] The server receives the translation result and sends it to the terminal.

[1709] Step 7:

[1710] The terminal displays the translation result received from the server to the user.

[1711] Text correction function

[1712] Step 1:

[1713] The user inputs the sentence to be corrected through the interface of the language learning device.

[1714] Step 2:

[1715] The terminal receives the user's input and sends the data to the server.

[1716] Step 3:

[1717] The server receives the input data and analyzes the user's emotions using an emotion engine.

[1718] Step 4:

[1719] Based on the sentiment analysis results, the server calls a natural language processing model to initiate the sentence correction process.

[1720] Step 5:

[1721] The server sends a request to generate a correction sentence to the natural language processing model, and the model corrects the sentence, taking into account the user's emotional state.

[1722] Step 6:

[1723] The server receives the correction result and transmits it to the terminal.

[1724] Step 7:

[1725] The terminal displays the correction results received from the server to the user.

[1726] Feedback function

[1727] Step 1:

[1728] The user sends a request through the interface of the language learning device for feedback on their input sentence and the corrected sentence.

[1729] Step 2:

[1730] The terminal receives the user's request and sends the data to the server.

[1731] Step 3:

[1732] The server receives the request and analyzes the user's emotions using an emotion engine.

[1733] Step 4:

[1734] Based on the sentiment analysis results, the server invokes a natural language processing model to initiate the feedback generation process.

[1735] Step 5:

[1736] The server sends a feedback generation request to the natural language processing model, which then compares the user's input sentence with the corrected sentence and generates feedback, taking into account the user's emotional state.

[1737] Step 6:

[1738] The server receives the feedback result and sends it to the terminal.

[1739] Step 7:

[1740] The terminal displays the feedback result received from the server to the user.

[1741] Example 2

[1742] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1743] Conventional language learning systems perform translation, correction, and feedback without considering the user's emotional state, which can affect the user's learning efficiency and motivation. Furthermore, due to the lack of a means to provide appropriate feedback according to emotions, it is not possible to provide adequate learning support tailored to each individual user.

[1744] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1745] In this invention, the server includes means for receiving a user's text input from a language learning device, means for translating the received user's text into a specified target language, means for correcting errors in the user's text, means for generating feedback by comparing the user's text with the corrected text, means for providing the generated feedback to the user, means for analyzing the user's emotions in processing the received text and the generated translation, correction results, or feedback, and means for reflecting the results of the user's emotion analysis in the processing results, thereby enabling appropriate translation, correction, and feedback that take the user's emotional state into consideration.

[1746] "Language learning device" refers to a device that allows a user to learn a language, and includes electronic devices such as smartphones, tablets, and personal computers.

[1747] "User" refers to an individual who uses this system to learn a language.

[1748] "Text" refers to documents or sentences entered by a user.

[1749] A "server" refers to a computer system that provides data processing and storage over a network.

[1750] "Means for receiving" refers to a function for transmitting the user's text from the language learning device to the server and for the server to receive the data.

[1751] "Means for translating" refers to functionality for converting received text into a specified target language, using natural language processing techniques such as generative AI models.

[1752] "Means for correcting errors" refers to functionality for correcting errors in received text, using natural language processing techniques such as generative AI models.

[1753] "Means for generating feedback" refers to the functionality that compares the user's text with the corrected text and generates learning feedback.

[1754] "Means for providing" refers to the functionality for displaying the generated feedback to the user.

[1755] "Means for analyzing emotions" refers to the function for reading and analyzing emotions from the user's input text and reactions, and uses technologies such as emotion engines.

[1756] "Means for reflecting emotion analysis results in processing results" refers to the function of adjusting the results of translation, correction, and feedback based on the analyzed emotion information.

[1757] MODE FOR CARRYING OUT THE INVENTION

[1758] This invention is a system that supports language learning for users using a language learning device, and also combines it with an emotion engine that recognizes and analyzes the user's emotions. The purpose of this system is to promote language learning efficiently while taking into account the user's emotional state.

[1759] Hardware and Software Configuration

[1760] Language learning device (terminal): An electronic device used by a user, such as a smartphone, tablet, or computer.

[1761] Server: A computer system that receives, analyzes, and processes data.

[1762] Emotion engine: Software for analyzing user emotions (e.g., emotion analysis API, emotion engine technology).

[1763] Natural Language Processing models: Generative AI models for translation, correction, and feedback generation (e.g., GPT-3, BERT, Google Cloud Translation API).

[1764] System Operation

[1765] 1. User Input: The user inputs the sentence they want translated or corrected into the language learning device. Examples of input data include "Hello, how are you?" or "I went to the store."

[1766] 2. The device receives and sends data: The device receives the entered data and sends it to the server over the internet. This process uses data transmission protocols and API calls.

[1767] 3. The server analyzes emotions: The server analyzes the received data and extracts sentences from it. At the same time, it uses an emotion engine to analyze the user's emotions. For example, it uses the Azure Emotion API to determine emotional states such as "positive," "negative," and "frustrated."

[1768] 4. The server performs translation and correction processing: The server takes into account the sentiment analysis results and calls the generative AI model for processing. Google Cloud Translation API is used for translation, and GPT-3 is used for correction.

[1769] 5. The server returns the results: The generated translation results and corrections are sent back to the terminal.

[1770] 6. Device displays: The device displays the received results to the user. The displayed results may include the translated sentence "Hello, how are you?" or the corrected sentence "I went to the store."

[1771] Specific operation example

[1772] Translation feature

[1773] Prompt statement:

[1774] "Please translate the following English sentence into Japanese. The user's emotional state is positive. 'Hello, how are you?'"

[1775] Text correction function

[1776] Prompt statement:

[1777] "Please correct the following sentence to use correct English grammar. The user's emotional state is negative. 'I went to the store.'"

[1778] Feedback function

[1779] Prompt statement:

[1780] "Please provide feedback on the corrected sentence below. The user's emotional state is frustration. Original: 'I went to the store.' Corrected: 'I went to the store.'"

[1781] This system can provide appropriate, personalized translation, correction, and feedback that takes into account the user's emotional state, thereby enabling efficient language learning support.

[1782] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1783] Step 1:

[1784] User input

[1785] The user inputs the sentence they wish to translate or correct and the target language into the language learning device. This input data includes text such as "Hello, how are you?" or "I went to the store." This text becomes the input data for the system.

[1786] Step 2:

[1787] Device receives and sends data

[1788] The terminal receives text data entered by the user. The received data is sent to the server via the Internet. This process uses an HTTP request to send the data. The data sent includes the entered text and the target language.

[1789] Step 3:

[1790] The server analyzes emotions

[1791] The server analyzes the received data and extracts text data. It then uses an emotion engine to analyze the user's emotions from the received text data. This emotion analysis is performed using, for example, the Azure Emotion API. The server outputs the emotional state, such as "positive," "negative," or "frustrated," as a result of the emotion analysis.

[1792] Step 4:

[1793] The server performs translation and correction processing

[1794] The server takes into account the sentiment analysis results and calls a natural language processing model (e.g., Google Cloud Translation API or GPT-3) to translate or correct the input text. In the case of translation, it converts the received text into the target language and generates a translated text. In the case of correction, it corrects errors and generates text that follows correct English grammar. The server outputs the generated translation or correction result.

[1795] Step 5:

[1796] The server returns the results

[1797] The server then returns the translation results and corrections to the device. The returned data includes the translated or corrected text, the user's sentiment analysis results, etc. This process uses HTTP responses to send data.

[1798] Step 6:

[1799] The device is displayed

[1800] The device then displays the results received from the server to the user. The displayed results include translated and corrected sentences. For example, the translation result for "Hello, how are you?" is "Hello, how are you?", and the correction result for "I went to the store." is "I went to the store."

[1801] This allows users to receive appropriate translations, corrections, and feedback that take emotions into account through the system.

[1802] (Application example 2)

[1803] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1804] Conventional language learning systems have difficulty providing a personalized learning experience that takes into account the user's emotional state. Furthermore, when learning using video content, the lack of real-time subtitle translation and feedback functions makes it difficult for users to learn efficiently.

[1805] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving a user's sentence input from a language learning device, means for translating the received user's sentence into a specified target language, means for correcting errors in the user's sentence, means for generating feedback by comparing the user's sentence with the corrected sentence, means for providing the generated feedback to the user, means for translating subtitles of videos watched by the user in real time, means for analyzing the user's emotions, and means for adjusting the translation results and feedback based on the analyzed emotions. This enables personalized, efficient, and interactive language learning that takes the user's emotional state into consideration.

[1806] A "language learning device" is an electronic device that a user uses when learning a language and that has input and display functions.

[1807] A "text" is a character string or sentence entered by a user.

[1808] "Target language" refers to the language that the user wishes to learn or translate into.

[1809] "Translation" is the act or process of converting from one language to another.

[1810] "Error correction" is the process of correcting grammatical or semantic errors in input text.

[1811] "Feedback" refers to information such as corrections, supplementary explanations, and evaluations provided to the user.

[1812] "Real-time subtitle translation" is the process of instantly translating subtitles displayed while watching a video into another language.

[1813] "Emotion analysis" is a technology that analyzes a user's emotional state at any given time based on their facial expressions and biometric information.

[1814] A "natural language processing model" is an artificial intelligence model for understanding and generating natural language, and uses machine learning and deep learning technologies.

[1815] A "server" is a computer system that provides computing resources and data to client devices via a network.

[1816] This invention is a system that supports language learning for users using a language learning device, and also combines it with an emotion engine that recognizes and analyzes the user's emotions. The purpose of this system is to efficiently advance language learning while taking into account the user's emotional state.

[1817] The server receives sentences entered by the user into the language learning device and translates them into the specified target language. First, the user enters the sentence they want to translate and the target language into the language learning device. The terminal receives this and sends it to the server. The server uses an emotion engine to analyze the user's emotions and performs translation by calling a natural language processing model while taking into account their emotional state. The results are returned to the terminal, which displays the translation results to the user.

[1818] Specifically, the server uses the Python-based natural language processing library googletrans for translation, and the DeepFace library for emotion analysis, analyzing the user's facial expression data captured by the camera. The hardware used is the laptop's built-in camera or an external webcam.

[1819] Suppose a user wants to translate the sentence "Hello, how are you?" into Japanese. The user inputs this sentence and "Japanese" into the device. The device sends this to the server, which uses an emotion engine to analyze the user's emotion. If the result is determined to be "positive," a natural language processing model is used for translation, and the translation result, "Hello, how are you?", is generated. The translation result is sent to the device, which displays it to the user.

[1820] The service also offers a real-time subtitle translation feature. The subtitles of the video the user is watching are translated in real time, and the tone and depth of the translation are adjusted according to the results of sentiment analysis. This optimizes the learning experience based on the user's emotional state. For example, if a user sets "English to Japanese translation" while watching a movie and the sentiment analysis determines "happy," the captions will be displayed in a positive tone, such as "I went to the store 😊."

[1821] An example prompt sentence, input to a generative AI model, is:

[1822] "When the user's emotional state is 'happy,' translate 'I went to the store.' into Japanese and add a positive expression."

[1823] In this way, more personalized and effective language learning assistance is possible by taking into account the user's emotional state.

[1824] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1825] Step 1:

[1826] A user inputs a sentence and a target language into a language learning device.

[1827] Specifically, the user inputs the sentence "Hello, how are you?" and "Japanese" into the terminal.

[1828] Step 2:

[1829] The terminal receives the input and sends it to the server.

[1830] The input data is "Hello, how are you?" and "Japanese."

[1831] The output is the transmitted input data.

[1832] Step 3:

[1833] The server invokes the emotion engine to analyze the received text.

[1834] The input data is the user's sentence "Hello, how are you?" and the emotion engine.

[1835] A deep learning model is used to analyze the user's emotions and determine them as "positive."

[1836] The output is the sentiment analysis result "positive."

[1837] Step 4:

[1838] The server calls a natural language processing model based on the sentiment analysis results and translates the text into the target language.

[1839] The input data is the user's sentence "Hello, how are you?", the target language "Japanese", and the sentiment analysis result "positive".

[1840] The translation result is "Hello, how are you?"

[1841] The output is the translation result "Hello, how are you?"

[1842] Step 5:

[1843] The server sends the translation results to the terminal.

[1844] The input data is the translation result "Hello, how are you?"

[1845] The output is sent from the server to the terminal.

[1846] Step 6:

[1847] The terminal receives the translation result and displays it to the user.

[1848] The input data is the translation result "Hello, how are you?" sent from the server.

[1849] The output is the translation displayed to the user.

[1850] Step 7:

[1851] A process is run to translate subtitles of videos watched by users in real time.

[1852] The input data is the subtitles of the video the user is watching, the target language "Japanese," and the user's emotional state.

[1853] Based on the results of sentiment analysis, appropriate expressions are added to the translation results and displayed.

[1854] The output is a real-time display of translated subtitles.

[1855] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1856] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1857] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1858] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1859] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1860] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1861] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1862] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1863] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1864] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1865] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1866] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1867] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1868] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1869] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1870] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1871] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1872] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1873] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1874] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1875] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1876] The following is further disclosed regarding the above embodiment.

[1877] (Claim 1)

[1878] means for receiving a user sentence input from a language learning device;

[1879] means for translating the received user text into a specified target language;

[1880] means for correcting errors in the user's writing;

[1881] means for comparing the user's writing with the corrected writing to generate feedback;

[1882] means for providing the generated feedback to a user;

[1883] A system including:

[1884] (Claim 2)

[1885] 10. The system of claim 1, wherein the means for translating the received user text into the specified target language performs the translation using a natural language processing model.

[1886] (Claim 3)

[1887] 2. The system according to claim 1, wherein the means for correcting errors in the user's writing uses a natural language processing model to correct the writing.

[1888] "Example 1"

[1889] (Claim 1)

[1890] means for receiving a sentence input from a user and a target language;

[1891] means for translating the received user text into a specified target language;

[1892] means for displaying the translation results to a user;

[1893] means for correcting errors in the user's writing;

[1894] means for comparing the user's writing with the corrected writing to generate feedback;

[1895] means for providing the generated feedback to a user;

[1896] A system including:

[1897] (Claim 2)

[1898] 10. The system of claim 1, wherein the means for translating the received user text into the specified target language uses a generative AI model to perform the translation.

[1899] (Claim 3)

[1900] 2. The system of claim 1, wherein the means for correcting errors in a user's writing uses a generative AI model to correct the writing.

[1901] "Application Example 1"

[1902] (Claim 1)

[1903] means for receiving a user sentence input from a language learning device;

[1904] means for translating the received user text into a specified target language;

[1905] means for correcting errors in the user's writing;

[1906] means for comparing the user's writing with the corrected writing to generate feedback;

[1907] means for providing the generated feedback to a user;

[1908] A means of supporting the terminology and procedures used in business machines through translation, correction and feedback;

[1909] A system including:

[1910] (Claim 2)

[1911] 10. The system of claim 1, wherein the means for translating the received user text into the specified target language performs the translation using a natural language processing model.

[1912] (Claim 3)

[1913] 2. The system according to claim 1, wherein the means for correcting errors in the user's writing uses a natural language processing model to correct the writing.

[1914] "Example 2: Combining Emotion Engines"

[1915] (Claim 1)

[1916] means for receiving user text input from a language learning device;

[1917] means for translating the received user text into a specified target language;

[1918] a means of correcting errors in the user's text;

[1919] means for comparing the user's text with the corrected text to generate feedback; and

[1920] a means for providing the generated feedback to a user;

[1921] means for analyzing user sentiment in the processing of the received text and the generated translation, correction or feedback;

[1922] A means for reflecting the result of the user's emotion analysis in the processing result;

[1923] A system including:

[1924] (Claim 2)

[1925] 10. The system of claim 1, wherein the means for translating the received user text into the specified target language uses a generative AI model to perform the translation.

[1926] (Claim 3)

[1927] 10. The system of claim 1, wherein the means for correcting errors in the user's text uses a generative AI model to perform the text corrections.

[1928] "Application example 2 when combining emotion engines"

[1929] (Claim 1)

[1930] means for receiving a user sentence input from a language learning device;

[1931] means for translating the received user text into a specified target language;

[1932] means for correcting errors in the user's writing;

[1933] means for comparing the user's writing with the corrected writing to generate feedback;

[1934] means for providing the generated feedback to a user;

[1935] A means for translating subtitles of videos watched by users in real time;

[1936] means for analyzing user emotions;

[1937] A means to adjust translation results and feedback based on the analyzed sentiment; and

[1938] A system including:

[1939] (Claim 2)

[1940] 10. The system of claim 1, wherein the means for translating the received user text into the specified target language performs the translation using a natural language processing model.

[1941] (Claim 3)

[1942] 2. The system according to claim 1, wherein the means for correcting errors in the user's writing uses a natural language processing model to correct the writing. [Explanation of symbols]

[1943] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for receiving a user sentence input from a language learning device; means for translating the received user text into a specified target language; means for correcting errors in the user's writing; means for comparing the user's writing with the corrected writing to generate feedback; means for providing the generated feedback to a user; A system including:

2. 2. The system of claim 1, wherein the means for translating the received user text into the specified target language performs the translation using a natural language processing model.

3. 2. The system of claim 1, wherein the means for correcting errors in the user's writing uses a natural language processing model to correct the writing.

Images